DECISION SYNTHESIS The principles and practice of decision analysis
DECISION SYNTHESIS The principles and practice of decision analysis
STEPHEN R. WATSON University of Cambridge, Department of Engineering
DENNIS M. BUEDE Decision Logistics, Reston, Virginia
CAMBRIDGE UNIVERSITY PRESS
Published by the Press Syndicate of the University of Cambridge The Pitt Building, Trumpington Street, Cambridge CB2 1RP 40 West 20th Street, New York, NY 10011-4211, USA 10 Stamford Road, Oakleigh, Melbourne 3166, Australia © Cambridge University Press 1987 First published 1987 Reprinted 1989, 1994 British Library cataloguing in publication data Watson, Stephen R. Decision synthesis: the principles and practice of decision analysis i. Decision-making. I. Title. II. Buede, Dennis M. 8 ' HD30.23 Library of Congress cataloguing in publication data Watson, Stephen R. Decision synthesis. Bibliography. Includes index. 1. Decision-making. I. Buede, Dennis M. II. Title. HD30.23.w38 1987 6$8.4'c>3 87-10270 ISBN 0 521 31078 4 paperback Transferred to digital printing 2003
CE
CONTENTS
List of List of tables Foreword Preface
figures
I INTRODUCTION AND READER'S GUIDE
page x xiii xv xvii I
PART I THEORY 2 RATIONALDECISION-MAKING
2.1 2.2 2.3 2.4
The difficulties of decision-making The meaning of rationality The rise of operational research Requirements for a satisfactory rationality for decisionmaking
3 DECISIONTHEORY
3.1 Conflicting objectives 3.1.1 Utility and value 3.1.2 Value functions 3.1.2.1 Problem formulation 3.1.2.2 Value functions: definition and alternatives 3.1.2.3 Existence of the value function 3.1.2.4 Additive value functions 3.1.3 Summary Exercises for 3.1 3.2 Uncertainty 3.2.1 Descriptions of uncertainty 3.2.2 Theoretical background to subjective probability 3.2.3 Using probabilities 3.3 Risk preference 3.3.1 Utility theory
9
9 11 13 16 l8
18 19 21 21 22 24 25 27 28 28 29 32 34 35 35
vi
Contents 3.3.2 Comparing risky options 3.3.3 Implications of utility theory 3.3.4 Criticisms of utility theory 3.3.5 The value of information Exercises for 3.3 3.4 Risk preference and conflicting objectives 3.4.1 Introduction 3.4.2 Keeney's approach 3.4.3 The Stanford approach 3.4.4 Comments Exercises for 3.4 3.5 Other normative approaches 3.5.1 Introduction 3.5.2 Cost-benefit analysis 3.5.3 The multi-criterion decision-making movement 3.5.4 Social judgment theory 3.5.5 Outranking 3.5.6 The analytic hierarchy process 3.5.7 Conclusions
4 THEPSYCHOLOGY OF DECISION-MAKING
4.1 Introduction 4.2 Uncertainty 4.2.1 Is man an intuitive statistician? 4.2.2 Heuristics and biases in judgments of probability 4.2.3 Calibration 4.3 Expected utility 4.4 Judgment 4.5 Implications for decision synthesis Exercise for 4 5 DECISION ANALYSIS IN ORGANIZATIONS
5.1 Introduction 5.2 Organizational theory and decision-making 5.3 Normative theories for group decision-making 5.3.1 Collective choice 5.3.2 Game theory 5.4 Using decision analysis in organizations 5.4.1 The individual in the organization 5.4.2 Designing programmed decisions 5.4.3 Management decisions in a participative setting 5.4.4 Public policy analysis 5.4.5 Negotiations 5.4.6 Conclusions
38 45 50 53 59 62 62 63 65 67 67 68 68 69 70 72 73 76 78 8l
81 82 82 84 87 90 95 98 100 IOI
101 102 106 106 109 111 112 112 113 115 117 118
Contents
vii
PART II PRACTICE 6 ANALYTIC STRATEGY
6.1 Introduction 6.2 The MODELING strategy 6.2.1 Background and definition 6.2.2 MODELING application to the Mexican electrical system 6.2.3 MODELING application to Xerox consumables 6.2.4 Requirements for success in the MODELING strategy 6.3 The INTROSPECTING strategy 6.3.1 Background and definition 6.3.2 INTROSPECTING strategy for electrical energy production in Wisconsin 6.3.3 INTROSPECTING strategy for transmission conductor selection 6.3.4 Requirements for success with the INTROSPECTING strategy 6.4 The RATING strategy 6.4.1 Background and definition 6.4.2 RATING application to a research program 6.4.3 Requirements for success in the RATING strategy 6.5 The CONFERENCING strategy 6.5.1 Background and definition 6.5.2 CONFERENCING strategy for tanker safety 6.5.3 CONFERENCING strategy for factory location 6.5.4 Requirements for success with the CONFERENCING strategy 6.6 The DEVELOPING strategy 6.6.1 Background and definition 6.6.2 DEVELOPING strategy for resource allocation 6.6.3 Requirements for success with the DEVELOPING strategy 6.j ALLOCATE and NEGOTIATE decision modes 6.8 Summary 7 ANALYTIC TACTICS
7.1 Introduction 7.2 Problem-and model-structuring 7.2.1 Problem-structuring aids 7.2.1.1 The rational manager - Kepner and Tregoe (1981) 7.2.1.2 Van Gundy's shopping list
123
123 127 127 132 133 134 135 135 137 143 144 145 145 146 148 148 148 150 151 154 155 155 157 159 159 160 163
163 163 164 165 165
viii
Contents
7.2.1.3 Report Card 7.2.2 Model-structuring aids 7.2.2.1 Influence diagrams 7.2.2.2 Cognitive mapping 7.2.3 How to get started 7.2.4 Requisite decision-modeling Exercises for 7.2 7.3 Iterative analysis 7.3.1 Four types of iterative analysis 7.3.2 How much analysis is enough? 7.4 Probability elicitation 7.4.1 Research findings 7.4.2 A recommended elicitation process 7.4.3 Discretizing continuous distributions Exercise for 7.4 7.5 Value and utility elicitation 7.5.1 Research 7.5.2 Definitions 7.5.3 Assessing additive value functions 7.5.4 Assessing additive value weights 7.5.5 Constructing hierarchical additive value weights 7.5.6 Assessing utility functions Exercises for 7.5 7.6 Sensitivity analysis Exercise for 7.6 8 APPLICATIONS
8.1 General introduction 8.2 Problem-structuring 8.2.1 Introduction 8.2.2 Hertz and Thomas (1983) — new product and facilities planning 8.2.3 Dyer and Miles (1976) - selection of trajectories 8.3 Model-structuring 8.3.1 Introduction 8.3.2 North and Merkhofer (1976) - analyzing emission control strategies 8.3.3 Flyback and Keeney (1983)- constructing an index of trauma severity 8.4 Iterative analysis 8.4.1 Introduction 8.4.2 Spetzler and Zamora (1983) - facilities investment and expansion
167 168 168 176 179 180 180 181 181 183 184 184 186 189 189 190 190 191 193 200 203 204 206 209 213 215
215 216 216 216 217 226 226 226 232 236 236 237
Contents 8.4.3 Buede and Choisser (1984) — system design 8.5 Probability assessment 8.5.1 Introduction 8.5.2 Howard, Matheson and North (1972) - hurricaneseeding 8.5.3 North and Stengel (1982) - program choices in magnetic fusion 8.6 Value assessment 8.6.1 Introduction 8.6.2 Dyer and Lund (1982) — merchandizing package 8.6.3 Kuskey, Waslov and Buede (1981) - Marine Corps's POM development 8.7 Sensitivity analysis 8.7.1 Introduction 8.7.2 Macartney, Douglas and Spiegelhalter (1984) cardiac catheterization 8.7.3 Kirkwood (1982) - nuclear power plant site selection 9 DECISION SYNTHESIS APPRAISED
9.1 The validation of decision analysis 9.2 Research directions 9.2.1 Research on how decisions are made 9.2.2 Research on the foundations of decision theory 9.2.3 Research on problems of applying decision theory 9.2.4 Research into methods for communicating analyses 9.2.5 Implications for decision synthesis 9.3 The ethics of decision analysis References Index
ix 243 247 247 247 252 256 256 256 260 263 263 263 266 271
271 274 275 276 276 279 279 279 282 295
FIGURES
I.I 3.1 3.2 3.3 3.4 3.5 3.6 3.7 3.8 3.9 3.10 3.11 3.12 3.13 3.14 4.1 4.2 4.3 5.1 5.2 6.1 6.2 6.3 6.4 6.5 6.6 6.7 6.8 6.9 6.10
Decision synthesis Dominance Decision tree for a simple decision problem The concept of equivalence Equivalence produced by the substitution principle Decision tree for the decision-maker's extended problem Decision-maker's utility function for final asset position Decision tree and utilities for the decision-maker's extended problem Insuree's decision problem Insurer's decision problem A job choice problem Allais's paradox Decision tree to compute the value of perfect information When the value of perfect information is zero Value of imperfect information A set of empirical graphs describing calibration Ellsberg's paradox Choice situations for Tversky and Kahneman's framing experiment The prisoners'dilemma Decision analysis in negotiations Howard's decision analysis cycle Deterministic model Deterministic sensitivity Stochastic analysis Clairvoyance The revised decision analysis cycle A decision analysis model of the Mexican electrical system A decision analysis of Xerox copier consumables Preferential independence Multi-attributed lottery questions
page 4 23 35 38 39 42 43 43 47 48 49 51 54 56 57 88 91 94 no 118 128 128 129 130 131 131 132 134 138 139
Figures 6.11 6.12 6.13 6.14 6.15 6.16 7.1 7.2 7.3 7.4 7.5 7.6 7.7 7.8 7.9 7.10 7.11 7.12 7.13 7.14 7.15 7.16 7.17 7.18 7.19 7.20 7.21 7.22 7.23 7.24 8.1 8.2 8.3 8.4 8.5 8.6 8.7 8.8 8.9 8.10 8.11
The individual utility functions Transmission utility functions Initial value structure Final value structure DSS architecture DSS evolution Problem-solving funnels Definitions used in influence diagrams New information and influence diagrams Uncoalesced decision tree Coalesced decision tree Manipulation forming cycle not permitted Symbols used in an influence diagram Initial beach investment influence diagram Final beach investment influence diagram An exemplary cognitive map Enjoyment attributes Bisection method Decreasing returns to scale Other shapes for value functions Value function for "walking time" Differences method Value function for "time spent" Relative and cumulative weights Risk preference lotteries Risk preference function Stochastic dominance One-way policy region analysis (proportional) One-way policy region analysis (NPV-constrained) Two-way policy region analysis Initial Egg 'N Foam flow chart Egg 'N Foam investment strategies Graph of cumulative probability density functions for each strategy Illustration of total cost comparison Overview of pollution model Total social cost versus pollution cost, existing coal-fired plant (non-urban) Results for existing coal-fired, non-urban plant Results for new coal-fired, non-urban plant Results for old reconverted urban plant Hierarchy of trauma severity concerns Severity adjustment factor due to age
xi 140 143 153 155 156 158 166 169 170 171 171 172 173 175 176 177 192 195 196 197 197 198 200 204 205 205 209 211 211 212 217 218 219 226 227 228 230 231 231 233 235
xii 8.12 8.13(3) 8.i3(b) 8.14 8.15 8.16 8.17 8.18 8.19 8.20 8.21 8.22 8.23 8.24 8.25 8.26 8.27 8.28 8.29 8.30 8.31 8.32
Figures Facilities investment deterministic model Distributions of individual judgments Probability distribution for raw material costs Facilities investment decision tree WWDSA value tree Plot of initial WWDS A evaluation results Plot of intermediate WWDS A results WWDS A iterative evaluation of alternatives Hurricane-seeding decision tree Maximum sustained winds over time Results for nominal hurricane Decision tree structure for magnetic fusion R & D Influence diagram for assessing probability distributions on fusion reactor outcomes The Christmas tree of merchandizing alternatives Structural model of merchandizing alternatives Allocation of sales margins Evaluation of merchandizing strategies (relative to current) Catheterization decision Two-way policy region analysis of catheterization decision Three-way sensitivity analysis of catheterization Decision tree for nuclear plant siting Three-way sensitivity analysis for nuclear plant siting
237 239 239 240 24 3 244 24 5 246 24 8 249 251 253 254 257 258 259 260 264 266 267 268 270
TABLES
6. i 6.2 6.3 6.4 6.5 6.6 7.1 8.1 8.2 8.3 8.4 8.5 8.6 8.7 8.8 8.9 8.10 8.11 8.12 8.13
page Attributes for evaluating energy policy 137 Final utility function 141 Expected utilities 144 Initial alternatives 152 Final alternatives 154 Strategy summaries 162 Beach cottage results 210 The MJS77 science investigations 220 Ordinal rankings and cardinal utility function values 222 Collective choice rankings and values 224 Ordinal rankings for preferred trajectory pairs 225 Expected value of resolving uncertainty 232 Trauma concerns and measures 234 Severity adjustment factors due to other diseases 235 Deterministic results for facilities investment 238 Combined effect of discount rate and risk aversion on the certain equivalent 239 Value of perfect information 241 Initial WWDSA alternatives 24 2 Definition of catheterization variables 265 Attributes for nuclear plant siting 269
Xlll
FOREWORD
Decision Synthesis makes up and lives up to its title. The authors have done an exemplary job of piecing together the diverse strands of the emerging, multidisciplinary field of behaviorally oriented decision theory in a way that serves their stated goal - improving decision-making by synthesizing techniques and perspectives with a sensitivity to the peculiarities of individual decision problems and decision-makers. The text draws upon diverse literatures whose creators generally know about one anothers' existence, but never have quite found the time to work out the interrelationships between their respective approaches. Although there are too many of these pairwise relationships for many to be fleshed out in any great detail, seminal ideas can be found at many junctures. As such, it is a book that gives some of the future of decision-making, as well as well-chosen pieces of its past and present. In creating this synthesis, Watson and Buede have adopted a decidedly disorderly approach to the often tidy world of decision theory. They present rudiments of the important normative axiomatizations of formal prescriptive approaches. However, they also caution the reader against taking it all too literally. Following much current thinking, they thoughtfully discuss the limits to imputing descriptive validity to normative theories — that is, assuming that people make decisions the way they are supposed to. Rooted in common sense and psychological research, this consideration of human foibles could be taken as grounds for a blanket endorsement of decision aids (and aides), helping to make people more like the ideal. Indeed, a variety of approaches to aiding are presented. However, the same tale of foibles is also used to show the limits to these helpers. Most pointedly, the authors challenge the common and, by same accounts, essential assumption that people have articulated values and beliefs. If this were true, the aiding procedures could be conceived as inert servants, eliciting from people the pieces of their decision problems and then showing how to reassemble them in order to identify the best course of action. Abandoning this assumption makes decision-making a more creative process in which the decision aide plays a more active role. Rather than reporting their perceptions to an aide entrusted with synthesizing xv
xvi
Foreword
them, decision-makers are seen as synthesizing those perceptions from pieces of their experience. In this conception, the aide may prompt them regarding where to look, suggest alternative perspectives (drawn from the aide's own experience), and even force them to work harder. This is a stance of unabashed activism which conflicts with images endorsed by many people in the question asking business, whether as decision aides or social scientists. Questions are not neutral stimuli eliciting ready responses. Rather, they are opportunities for reflection, requiring an inferential process (as to "just what do I think?"). Such an activist role imposes a particular responsibility on decision aides. They need somehow to engage their clients without taking over the decision-making process. To this end, Watson and Buede provide a consumers' (and producers') guide to decision-aiding approaches, showing their strengths, weaknesses, and manipulative potential. They articulate the decision-makers' right to ignore advice, because a method does not fit their problems, because it places too much power in the hands of those who know the method (as opposed to those who know the problem), or because its complexities lead them to lose cognitive control of the problem (and, with it, their intuitive feel for what is happening). These issues are discussed both in general terms and in the context of a rich set of applied contexts. Here, as elsewhere, the writing allows readers to learn easily enough to know whether they want to learn more. Ample references are applied to that "more." This book, like most written with a feeling of mission, represents a landmark in its authors' eyes. The story whose time has come to be told here seems to be that of thirty odd years of chaotic development in decisionmaking research and application. In it, enough has happened to allow the authors to identify the flowers that have bloomed among the starts that have been tried. To their credit, the authors resist the temptation to begin a period of codification just as they resist the temptation to fill pages with routine technical expositions. For each approach, they force themselves to ask and answer the hard question of "what good is it?" For the field as a whole, they force us to ask ourselves the hard question of, "What is it all about, this decision-making business?" The text provides many hints on more detailed issues such as how we know what we want, how we know the limits to our own knowledge, how we tell people what we are thinking with enough precision to enlist their aid or win their support, and how much credit do people deserve for managing their own affairs and how much help can aides and aids give them? Cambridge and Reston are quiet, thoughtful, verdant communities - albeit of quite different vintages - many of whose inhabitants are dedicated to providing decision-makers on the outside with defensible, practical tools. Decision Synthesis is a worthy contribution to that clumsy enterprise of producing wisdom. Baruch Fischhoff, Eugene, Oregon
PREFACE
During the four years that have elapsed from the initial concept to the publication of this book, we have received helpful advice from many people. We want to record our particular thanks to a number of our colleagues who have read part or all of the manuscript, and given constructive criticism. These are, in alphabetical order, Val Belton, Terry Bresnick, Rex Brown, Sue Brownlow, Nick Butterworth, Elizabeth Collins, Baruch Fischhoff, Vivien Fleck, Anthony Freeling, Gary Frisvold, Elizabeth Garnsey, Colin Gill, Lisa Griesen, Kenneth Kuskey, Warner North, James Onken, and Jake Ulvila. In recording our thanks, however, we also want to state that any inaccuracies in our text are our responsibility alone, and, if readers find any, we would be grateful to learn of them. Many authors and publishers have kindly granted us permission to quote from their works, and we would also like to record our thanks to all these. In addition we wish to thank Richard Lund, who provided the photograph used to reproduce Figure 8.25. Readers willfinda guide to the book - what it contains, and how to use it - in Chapter 1. We do not mention there the use of the exercises in chapters 3, 4, and 7, however. These are designed to help readers to apply the ideas in the text. We have not given any answers to the examples, believing that the availability of answers can stand in the way of learning. We would be happy to provide any reader with answers, however, and readers seeking these should make application to either author. Finally we would like to thank our families, and particularly our wives, Rosemary and Anne, for their forbearance and support during the writing. Without such support we doubt that this book would ever have been finished!
INTRODUCTION AND READER'S GUIDE
This is a book about decision-making — about how people do make decisions, and about how they may improve their decision-making. To say this makes some assumptions, and, before we go on to show the reader how decision-making will be tackled in this book, as we will do in the rest of this introductory chapter, we will examine these assumptions. First, can decision-making be studied? One of the authors, when discussing his research work with a senior (and traditionally minded) colleague, was asked what subject he studied. On replying that he studied decision-making, the response was "Is that a subject?" Many readers will not need our prompting to answer that question affirmatively, but it is as well to spell out the nature of our study. It is principally concerned with the determinants of decision - that is to say, what it is that makes us take one course of action rather than another. In some cultural and intellectual traditions there is an easy answer to this question. Extreme forms of Calvinism, for example, itself a particularly severe kind of Protestantism, deny that free will exists; nobody may make decisions, therefore, since everything is fore-ordained by God. Then, as Howard (1980) points out, in an excellent article on the philosophy of analyzing decisions, many Eastern philosophies will not see the study of decision-making as a fruitful endeavor, since in these philosophies one accepts the unfolding of life as it presents itself. The idea of a "decision" is a quintessentially Western idea, an act of hubris to a believer in Eastern philosophy and a joke to the enlightened. (Can you imagine Buddha or Lao-tzu making a decision?) Howard 1980, p.5
We imagine, however, that most readers of this book will sit within the humanist tradition, the dominant world-view in the West since the Enlightenment. To such readers it will seem self-evident that we do make choices, and that we do have the freedom to decide to take one or another of several different avenues open to us. We can then properly ask what it is that makes us take one path rather than another. Psychologists who study human cognition have now been investigating this issue for many decades,
2
Introduction and reader's guide
and we describe some of their findings in Chapter 4 of this book. The principal question addressed has been whether people make decisions in accordance with plausible models of what might be called coherent decision-making; the answers have been mainly negative! Nonetheless, these studies have been fruitful in exploring what drives decision-making; they indicate that decision-making is indeed a proper subject of study. The second implicit supposition of our opening sentence is whether decision-making can be improved. At the intuitive level the reader's response is likely to be that it clearly can, at least in some instances. We are ready to point the finger at many bad decisions, such as Napoleon's decision to march on Moscow, or the investor who places all his savings in a company that goes bankrupt. If bad decisions can be made, so too, presumably, can good ones; and thus decision-making can be improved. But, if we investigate more closely what is meant by "good" and "bad" decision-making, wefindthe idea difficult to define. Is a decision good if it turns out to have led to the ideal outcome given the eventual circumstances? Or should we call a decision good only if the process which leads to it is satisfactory? We will discuss these issues in Chapter 2, where we will define a good decision as a rational one, and a rational one as one that conforms to freely chosen rules of behavior. Assuming that the quality of decision-making can be improved, the next question to ask is how this may be done. Over the last half century an increasingly sophisticated theory has developed, idealizing how decisions might be made. This decision theory has its roots in economics, psychology, and mathematical logic. We give an account of it in Chapter 3. A general point that we may make here is that throughout this book our object will be to give readers an understanding of the key ideas, without dwelling on them at length. We do aim, however, to provide sufficient references to the literature for an interested reader to be able to pursue points of interest elsewhere. For example, in Chapter 3 we describe value theory, subjective probability theory, and utility theory in sufficient detail (we hope) for readers to grasp the essentials, but we give references to the original literature and to other more specialized textbooks where these topics are described in greater detail. There are other subjects that a book such as this might have covered but which we have left out in order not to make the text too unwieldy; for example, we considered including a presentation of the mathematics of probability, a working knowledge of which is necessary to get to grips with much of Chapter 3. Chapter 3, then, presents a normative, but idealized, theory of decisionmaking. An idealized theory of what constitutes a good decision is very different, however, from a guide to making decisions better in practice. The art of applying decision theory to practical situations has had a more recent history, but since about i960 considerable experience has been
Introduction and reader's guide
3
developed in this applied art; the art has come to be called decision analysis. Here we should digress to explain the title of this book. So far as we are aware, the term decision synthesis has not been used previously to describe the craft of taking a decision-maker's problem and providing help in reaching a decision in conformity with the principles of decision theory. We have used it rather than the more traditional term (decision analysis) not only to provide a distinctive title for the book, but also because the craft involvesfirstthe identification of the component parts of the decision, and then the construction, or synthesis, of recommendations for action. Despite our title we will talk mostly of decision analysis (rather than synthesis) in this book, since this is the term which is now imbedded in the literature. We still feel, however, that decision synthesis is a better term for describing what the craft involves. The next question the reader will ask is how we should do decision synthesis; how may an idealized normative theory be used to help real people with real problems? We give our account of the application of decision theory to real problems in Part II of this text. First, in Chapter 6 we describe different approaches to decision synthesis that have been successfully practiced. There are now many different people in the Western world with experience of applying decision theory. As one might expect in an applied subject which involves as much art as science, different schools of practice have emerged. We call them analytic strategies, and we hope that, by studying Chapter 6, readers will be able to gain an impression of the different flavors of decision analysis that are currently practiced. Secondly, in Chapter 7, we discuss some of the techniques that are available for carrying out an analysis effectively. These include: how to structure a problem, and to model that problem once structured; how to develop an analysis iteratively; and how to elicit numerical representations of subjective judgments from decision-makers. We refer to these as analytic tactics. Then we go on, in Chapter 8, to give examples of a number of successful applications of decision theory; it is here that readers will be able to judge for themselves whether decision synthesis can improve decision-making. We hope that they will agree with us that, if good decision-making is judged by its conformity to principles of consistency, its clarity of reasoning, and its ability satisfactorily to articulate goals, then decision synthesis, properly carried out, can improve decision-making. A more detailed representation of the structure of this book is given in Figure 1.1. Notice that certain topics, such as the modeling of uncertainty, of risk preference, and of conflicting objectives, are discussed in most of the chapters. One way of using this book could be to concentrate on just one of these issues at a time, looking at first the normative theory (in Chapter 3), then the experimental evidence of whether that theory
Rationality
Psychology
2.1 The difficulties of decision-making
4.1 Introduction
2.2 The meaning of rationality
Strategy
Organizations
7
8
Tactics
Applications
5.1 Introduction
7.1 Introduction
5.2 Organization theories
7.2 Structurings:
k
5.3 Normative group theories
2.3 The rise of operational research
5.4 Decision analysis in organizations
8.1 Introduction
»-6.1 Introduction "Why do DA?."
• 7.3 Iterative analysis "How much analysis?"
8.3 Model structuring
» 8.4 Iterative analysis
(6.2-6.7) 2.4 A satisfactory rationality
7.4 Probability elicitation
6.4 RATING
Figure I.I
8.5 Probability assessment
6.2 MODELING •6.3INTROSPECTIN
3.4 Risk preference and conflicting objectives
6.5 CONFERENCING
3.5 Other normative approaches
6.7 ALLOCATE and NEGOTIATE
Decision synthesis.
6.6 DEVELOPING
9.1 The validation of decision analysis
"• 8.2 Problem structuring
7.5 Value/utility elicitation
» 8.6 Value assessment
7.6 Sensitivity analysis
• 8.7 Sensitivity analysis
9.2 Research directions
9.3 The ethics of decision analysis
Introduction and reader's guide
5
describes behavior (Chapter 4), then the way that issue is handled by different schools of practice (Chapter 6), next the practical approaches available for coping with the concept (Chapter 7), and finally how other analysts have put these ideas together (Chapter 8). We recommend this mode of study as a possibly more creative learning procedure than the usual one of reading through chapter by chapter. We have not yet mentioned Chapters 5 and 9. Chapter 9 is a concluding statement, where we discuss, in the light of the rest of the book, why, and in what context, decision synthesis is a valid and useful procedure to adopt; we also give our view of the research topics that need to be addressed to render the approach of yet more value. Chapter 5, on the other hand, addresses a more substantial issue which affects the validity of many applications of decision analysis. Decision theory is a normative theory for how an individual decision-maker might think through his or her decisions and determine sensible actions; it does not set out to do the same for groups of people, or for corporations, or for public bodies. Yet decision analysis has been widely used in such contexts. What we do in Chapter 5 is first to give a brief review of what is known about decision-making in organizations, and an introduction to the literature here. Next we describe some of the normative approaches that have been developed for group decision-making, and discuss their limitations. Finally, however, we show why, and in what contexts, it is valid to apply decision theory to organizational decision problems. We describe the bounds for organizational decision synthesis. The final topic we want to discuss in this introductory chapter is our motive for writing this book. So much has been written on decision theory and its applications that it behooves the authors of a new text to explain why they wish to add to that literature. Many excellent texts give a satisfactory account of decision theory; amongst these are Luce and Raiffa (1957), Raiffa (1968), Schlaifer (1969), Lindley (1973), Brown et al. (1974), Keeney and Raiffa (1976), Moore and Thomas (1976), Morris (1977), Behn and Vaupel (1982), Howard and Matheson (1984), Bunn (1984), and Bodily (1985). Other texts have put the theory into its psychological context (Lee 1971, Wright 1984, or von Winterfeldt and Edwards 1986) or its organizational setting (Clough 1984). Despite this existing coverage, we have not found a text that quite shares the emphasis that we have sought here; indeed the idea of writing this text came from one of us who had found no suitable single text available to support his teaching of decision analysis to final-year engineering undergraduates at Cambridge. What we feel is needed in a single book is an account of decision analysis that: (i) describes the nature of decision theory in a way that students with a quantitative background could find appealing; (ii) shows interested readers gateways into the literature, allowing them to
6
Introduction and reader's guide
pursue topics more intensively if they so wish; (iii) gives to the readers interested in the practical utility of these ideas a handbook for ways to do decision analysis in practice; and (iv) gives an up-to-date account of the practice of decision synthesis. Readers must decide for themselves whether we have achieved these goals.
I THEORY
After defining what we mean by rationality in decision-making in Chapter 2, we give an account of decision theory in Chapter 3. Chapter 4 consists of a review of psychological research in decision-making, and we discuss organizational decision-making in Chapter 5.
RATIONAL DECISION-MAKING
2.1 THE DIFFICULTIES OF DECISION-MAKING
In some ways we can describe decision-making as the most common human activity. Almost everything we do involves decisions. Whether we are driving a car, preparing a menu for a meal, working out what to say next in a piece of writing, or handling a difficult social situation, to give just a few examples, we have to make choices and decide to adopt one course of action rather than another. Since we are involved in so much decision-making, we might expect that the human species has evolved to be good at it. Viewed in one way this is indeed the case, in that we have developed many innate mechanisms for effective intuitive decisionmaking in our best interests. As babies, our unconscious decision to cry when hungry, or in pain, surely is the wisest way of improving our lot! As mature adults, our instinct to dodge a stone thrown at us is an intuitive decision that maximizes our welfare. These are at the unconscious level. At the conscious level, however, we have also developed ways of making decisions without much thought. Habits of driving, acting according to conventions of social behavior, following norms of hygiene or diet, all lead to good decision-making. Are we to conclude, then, that human decision-making is as good as it can be, and that nothing is to be gained by studying how it is, or should be, done? Unfortunately, this is not so. In the rest of this section we will discuss why it is that some kinds of decision-making are difficult and give rise to the need to think through the process of decision-making. We usually think of decision-making as being good if it is rational. We discuss the key concept of rationality in Section 2.2. One expression of rationality that has emerged in the last forty years is in the methods of operational research or systems analysis. We give a brief history of these ideas in Section 2.3, together with an appreciation of the difficulties tha't some versions of this interpretation of rationality lead to. Then in Section 2.4 we argue for a different emphasis on the meaning of rationality — more subjective, and more related to perceptions. We hope that by reading
io
Theory
through this chapter the reader will perceive the need for an expression of rationality that will provide a guide for decision-making; decision theory provides such an expression, and we turn to study it in Chapter 3. Although, as we have observed above, there are many examples of decision-making behavior which are clearly sensible, and are in the best interests of the decider, we still speak of "bad" decision-making, as well as "good." The decision-making that led to the Bay of Pigs incident has been described as bad; the decision of the British and French governments to support the manufacture of Concorde was widely felt to be a bad decision; the reader can probably add to this list from his own experience. In some ways criticisms of this kind can be unfair. It could be that, in the light of the information available to the decision-maker at the time, he made a wise decision, and only after the event did it become clear that some other action would have been better. We could argue, as indeed we will at a later stage in this book, that one should distinguish between the decisionmaking process and the outcomes that may result from the decision. It is quite possible for the outcomes resulting from a chosen alternative to be good, even though the decision-making process was bad, and vice versa. Clearly decisions can be made which are bad in both process and outcome. What are the circumstances that lead to this phenomenon? Perhaps the most obvious is the complexity of a decision. Man has always faced decisions with far-reaching consequences, but it is only in recent times that the complex web of interactions in our world has reached the state of development that it has today. In deciding whether to launch a new product, the managers of a company are faced with a complex set of issues. What effect will the new product have on the rest of their product line? How will their competitors react? Will the reaction of some foreign competitors harm their market position in some overseas markets? Will it affect trade sanctions by foreign governments? How will it be made using foreign labor, or local? Will there be adequate suppliers of components, or will parts have to be made by the company, and if so where? The list of questions can be extended almost indefinitely. As we shall see in Chapter 4, there appear to be rather important limitations on man's ability to take all such factors into account at once. Bad decision-making may result from the decision-maker's inability to incorporate all the important factors into his thinking, thus leaving himself open to a disaster caused by a factor that he did not consider. Many of the questions above involve uncertainty, and decision-making is often difficult just because of the many uncertainties involved. A manufacturing company's director might say: "If only I knew what my competitor intended, then I would see more clearly what to do; but because I am uncertain I find it difficult to decide." Some authors have
Rational decision-making
11
seen uncertainty as the chief problem with decision-making, and it is surely a major aspect of nearly all important decision problems. It may be, moreoever, that not only are we unsure what will happen, or what the nature of the world is, but we are also unclear about how important the outcomes would be to us, even if we knew for sure what would happen. For example, the manufacturing director, in deciding where a new manufacturing plant should go, will want to trade off the low labor costs in some countries with the possibly poorer quality of goods and the stability of the economy in that country. The articulation of trade-off judgments is an important problem in decision-making, and when we find it difficult to make such trade-off judgments we may make the decision badly. It is particularly difficult to come to any agreed solution on trade-offs if, as is usually the case, there are many different actors involved in a decision, with different views on the relative importance of the objectives. The increased size of organizations in the last fifty years has exacerbated this problem. Decision-making in such contexts is hard because of the differences of opinion that will emerge from different parts of the organization, all of which must be involved in reaching a conclusion. In the face of complexity, uncertainty, conflicting objectives, and multiple decision-makers, the typical response, at least of Western man, is to attempt to be rational. We seek a rational framework to help us think through our decisions. Indeed, our intuition is that some decisions are good and some bad, so that it is worthwhile seeking to improve decisionmaking. The intuitive approach is to define a good decision as a rational one. To improve decision-making then is to make each decision conform to a norm of rationality. Rationality is a concept that we intuitively look for; it is, however, more difficult to define than might appear at first sight. 2 . 2 THE MEANING OF RATIONALITY
There are some concepts which we use in common speech whose meaning is rarely questioned, but which are in fact rather elusive. Rationality is one such concept (probability is another). Many philosophers have explored what it means to be rational, and indeed Simon (1983, p.vii) speaks of the study of human reason as being his "central preoccupation for nearly fifty years." We do not have space here to explore this question in depth; what we will do, however, is to make passing reference to some entry points in the literature on rationality, before going on to a statement of what we take it to mean. Apart from the major contributions of H. A. Simon to this subject (see, for example, Simon (1957b, 1969)), the reader is recommended to consult March (1978) for a discussion of theories of rationality. Both Diesing
12
Theory
(1962) and Steinbruner (1974) define several different forms of rationality; these are summarized in Sage (1981). For a stimulating and very wide-ranging account of the problems of what it means to be rational, we recommend Jon Elster's Ulysses and the Sirens (1979); one reviewer described this as "the most exhilarating thing to hit the philosophy of the social sciences since Popper" - a view with which we concur. We argue that rationality has a lot to do with consistency. If I assert that A implies B, and also that B implies C, then I will presumably assert that A implies C. We say that it is irrational to do otherwise. What we are doing is making an inference according to a principle of logic, namely that implication should be transitive. We are being inconsistent if we assert that A does not imply C in this example. Similarly, we might invoke as a principle that our preferences should display transitivity. If I assert that I prefer X to Y, and Y to Z, but I do not prefer X to Z, I might be accused of irrationality. This accusation is probably based on the assumption that I wish my preferences to display transitivity, and on the observation that the expressed preferences are inconsistent with this. Notice that, in both these examples, the demonstration of irrationality depended on the presence of a rule and showing that statements were inconsistent with that rule. This provides the model for our concept of rationality. We will say that we are rational when, having adopted rules which our statements or actions should conform to, we act in a way that is consistent with them. Note that this definition allows many different kinds of rules to be adopted, and that therefore whether a behavior is rational will depend on what rules have been adopted. For example, is it rational to say that I am indifferent between A and B, and between B and C, but that I prefer A to C? If we require our indifference judgments to be transitive, then this is irrational. If we only require clear preference judgments to be transitive, then this is not irrational. To be rational in decision-making, therefore, we will need to construct a set of rules that we will wish to adopt in determining what to do in complex decision-making situations; then we will satisfy our need for rationality by conforming to them. Chapter 3 is devoted to describing the rules of decision theory, and we will argue that they constitute a sensible set of rules to follow. By our definition of rationality, however, it does not necessarily follow that people who do not abide by the precepts of decision theory are irrational; they may well have perfectly sensible rules of their own which they are following most rationally. Before developing decision theory, however, we shall digress- to discuss the development of operational research and systems analysis, approaches to decision-making which are often characterized as rational. We do so in order to put the rationality of decision analysis in a wider context. The
Rational decision-making
13
reader who wants to get on with the main theme of this book is recommended to skip the next two sections, or to come back to them at a later stage. 2.3 THE RISE OF OPERATIONAL RESEARCH
To be rational is one thing; to be scientific is another. To apply the scientific method, a hypothesis must be constructed about the world, and then data must be collected to attempt to falsify the hypothesis; in the absence of such falsification, the hypothesis may be used to explain phenomena. Engineers take the hypotheses of science, and use them to construct artifacts to perform useful functions. The procedures of science and engineering are sometimes described as "scientific rationalism," and this nomenclature fits in with our definition of rationality, by taking these procedures as the rules to which we wish to conform. The application of scientific rationality to the organization and management of human activity has come to be referred to as operational research, or systems analysis. In this section we shall give a brief account of the history of this subject and the difficulties it now faces, before going on to describe, in Section 2.4, what is needed to meet those difficulties. The meaning of the term operational (or operations) research has always been a matter of contention. It is a great pity that it is one of the few academic subjects whose title does not convey to the layman some idea of its area of study. The term grew up in the Second World War to describe the activities of units researching into how to carry out military operations more effectively. It was recognized that considerable improvements in performance might be obtained if engineering methods were applied to logistics. For example, in determining whether large or small Atlantic convoys stood the best chance of moving material from the US to Europe against the threat of German attack (small convoys stood a better chance of passing undetected, large convoys a better chance of resisting an attack, at least in part, because of the greater number of support ships), a mathematical model was built of this situation, and used to explore the probabilities. On the strength of this analysis the policy on convoy size was changed, leading to an improvement in the Allied position. In many other contexts the operational research groups proved very valuable to the war effort (see Waddington (1973) for an account of the wartime contribution of OR). The novelty of the approach was to take the methods which had been successfully used in engineering contexts to design effective machinery, and apply them to the design of effective logistical operations. That is to say, once the purpose of the activity had been defined, a mathematical model of the system was set up, and used to evaluate different design options. One could ask questions of the model
14
Theory
without having to try out the many different practicable possibilities to see which was best; this was the approach that had been used by engineering designers for a long time. OR proved to be so successful during the war that it was widely recognized to have potential value as an approach to the management of civilian operations, particularly in manufacturing industry, but also wherever managers had to decide which of several options to pursue in organizing their activities. During the fifties and sixties there was considerable enthusiasm for OR, particularly in the US and the UK, but also in the rest of the Western world. Considerable successes were achieved. For example, the recognition that the problem of how to schedule the operations of an oil refinery could be formulated as a linear program led to considerable savings in costs. Now no refinery operates without extensive use of linear programming. Similarly, the design of road systems, manufacturing systems, multi-user computer systems, and even queuing systems in banks and public offices, to avoid congestion, has benefited from the same kind of analysis; by investigating mathematical models of such systems with the mathematical tools of queuing theory, it is possible to construct systems which do not lead to unacceptable congestion. Because of the obvious successes of the OR approach to some problems there was great interest in applying it to others. In the United States, perhaps the most famous institution set up to apply OR methodology was the RAND corporation created by the US Army Air Forces in the late forties. (For a history of this influential organization, see Smith (1966).) The term used by this organization for the work they performed was "systems analysis"; although there were differences of emphasis, it is fair to think of the RAND approach as being essentially that of OR. RAND was not the only organization practicing OR, however. The approach gained much in popularity, and by 1965 it had become so entrenched in the US government that the Department of Defense appointed an Assistant Secretary (a very senior appointment) specifically for systems analysis. Other governments, and private industry, followed the fashion of the times and set up large departments to carry out OR studies. At about the same time, however, criticisms of OR began to emerge, and during the 1970s this led to what some commentators have described as an intellectual crisis. Hoos (1972) gave a most significant criticism of systems analysis; later the OR literature began to contain many critical contributions as well (see, for example, Ackoff (i979a,b), Majone and Quade (1980), Geoffrion (1983), Tinker and Lowe (1984), Tomlinson and Kiss (1984)). The problem was that OR methods were, no longer providing the same noticeable improvements to management problems that had been the case in the early years of the subject. To some extent this was a consequence of OR being oversold, as often happens with inno-
Rational decision-making
15
vations in management methods. A more fundamental problem was that the mind-set of the OR approach, while ideal for problems of logistics, for which the approach was developed, was inadequate for the higher-level management problems that were beginning to be faced. There is now a sizeable literature on this issue. For example, Linstone (1984) in a stimulating recent book gives a personal view of three different perspectives to analyzing decisions: the technological, by which he means the OR approach; the organizational; andthepersonal. We have not sufficient space to repeat Linstone's arguments here, but his last two perspectives stress the importance of organizational forces and personal perceptions in the making of decisions, aspects often ignored in an OR approach. H. Boothroyd, who at the time had worked for ten years in operational research at the National Coal Board in the UK, states the inadequacies well: By 1966 . . . I had begun to be aware of two particular problems: (a) clients were inclined to treat the findings of investigations as complete and final, rather than as a stage in a joint process of exploration and experiment — this was related to their picture of science as providing definitive final answers on both facts and desirable choice of action; (b) in the literature of operational research and systems analysis the implicit picture of decision-making was quite unlike the untidy, episodic nature of the experience of making decisions about the conduct of investigations. Boothroyd 1984, p.35
The analyses failed to involve the problem-owners; and the methods used to decide how to do the analysis were very different from those recommended by the analysis. Checkland has long made similar criticisms of OR, or systems analysis. In Checkland (1981), he gives a careful account of what he calls "hard" systems thinking (in which category he places OR), and points out its inadequacies when applied to certain kinds of problem. One of his chief criticisms is that the "hard" systems thinker is concerned with how to achieve a given end most efficiently, rather than with what that end should be in the first place. The exploration of goals, their creation, and their expression are not important parts of the analysis. But, as Checkland points out, in many practical problems, such investigation of goals is probably the most essential need in coping with the problem. He reiterates his criticisms in a recent paper (Checkland 1983). In summary then, we conclude that inadequacies have appeared in some (though not all) attempts to apply the scientific rationality of OR to management. These include: - failure to involve the decision-maker in the experimental and iterative nature of the analysis; -failure to appreciate the organizational and personal context of decision-making;
16
Theory
- failure to explore ends as well as means. We now go on to see what is required to meet these inadequacies. 2.4 REQUIREMENTS FOR A SATISFACTORY RATIONALITY FOR DECISION-MAKING
If the OR approach is an inadequate methodology for the synthesis of decisions (at least as articulated in textbooks and taught in many university courses), we must ask what may be done to improve matters. Checkland's answer is to use his "soft" systems approach. We have not the space here to describe this approach in detail, and any summary may well fail to capture the essence of what is a very sophisticated approach to thinking about problems. The reader is referred to Checkland (1981) for an account of "soft" systems thinking. We will note, however, that its essence involves a more open-minded, iterative, and flexible way of thinking about a problem than is typical of the traditional kinds of OR and the harder kinds of systems thinking. Although Checkland's approach emphasizes the importance of including problem-owners (a slightly more general term than decision-makers) in the analysis and analyzing the problem as seen by them, it does not provide a specific methodology for eliciting the problem-owners' perceptions of the real world they face. An interesting and novel approach which does just that has been developed by Colin Eden and his colleagues at Bath University in the UK. They stress the need to understand the problem-owners' perceptions of their problems if an effective way is to be found for creating a rationality that will satisfy them and help them in decision-making (see, for example, Eden et al. (1983)). They use the notion of a "cognitive map," a representation of an individual's thinking, to help him to structure his perceptions of the decision problem that he faces. They provide a formal procedure for constructing cognitive maps, and have prepared an interactive computer program to assist with this procedure. In contrast to the "hard" systems approach, to use Checkland's characterization, Eden's approach unashamedly attempts to model, not the real world in some objective sense, but that world as perceived by the problem-owners. We believe that, as a way to structure problems, cognitive mapping has great potential. It stops short, however, of providing a formal calculus for fleshing out the problem structure, once it has been identified by mapping. We make further reference to this idea in Section 7.2.2. These two ways of thinking provide very useful insights, but, in our view, they give only part of the answer. We shall nowfinishthis chapter by drawing together the various threads that we have explored, and argue the need for a calculus of decision-making with particular properties. We first argued that decision-making was difficult because of the
Rational decision-making
17
complexity of the decision being faced, the uncertainty that generally surrounds the problem, the existence of conflicting objectives, and the presence of multiple actors in the decision situation. We then suggested that, if we wished to handle these problems in some rational way, it was necessary for a set of rules of decision-making to be established; rationality would then be achieved by conforming to these rules. We digressed to describe the procedures of OR, often identified with rationality, and highlighted some of the inadequacies that have been seen in applications of the OR method. What then is needed? The properties listed below follow from our analysis above. (i) A set of rules for decision-making must be defined in order that we may know what it is to be rational. (ii) The rules must address the values of the problem-owners. It must be possible to articulate their preferences and perceptions. (iii) The rules must state what it is to be rational in the face of perceptions of uncertainty. (iv) The rules must provide a calculus for steering the thinking of the decision-makers through complex problems. The next chapter gives an account of decision theory, a body of ideas with a long intellectual history and one which meets these requirements.
DECISION THEORY
In the previous chapter we argued that, to assist people in thinking through complex decision problems, a quantitative framework is needed to encourage consistency, the essence of rationality. Decision theory provides such a framework, and in this chapter we give an account of this theory. We start, in Section 3.1, with value function theory, which is concerned with conflicting objectives. In Section 3.2 we discuss uncertainty, and how the calculus of probability may be used to describe it. Section 3.3 contains a description of utility theory, which is concerned with decision-making in the face of uncertainty, and in Section 3.4 we show how uncertainty and conflicting objectives may be handled at the same time. Finally, in Section 3.5, we discuss other approaches to creating a calculus for decision-making. 3.1 CONFLICTING OBJECTIVES
We all make decisions all the time. What clothes shall I wear this morning? What shall I have for lunch? Shall I watch television or read a book? Most of us have no difficulty in deciding matters like these, but on occasion we find ourselves spending a lot of mental effort in making up our minds. Choosing a career and then a particular job, or where to live, often falls into this category. The stakes at risk are more significant on such occasions, and we find that we need to spend more effort than normal insuring that we choose correctly. In our working lives, when we are part of a decision-making group, or when we have responsibility for decisions that affect many other people, we are highly motivated to get the decision right. Whether it is the choice between several different locations for a new plant, or the decision to institute a new payments system in a factory, or the selection of a new computer system, we seek to satisfy ourselves that the choice we make is consistent with our own beliefs and values, and with the values of the organization that we serve. The practical role of decision theory is to provide a framework that assists people in achieving such consistency. In 18
Decision theory
19
this chapter we will set out the theory, and leave until Part II descriptions of how the theory is applied to practical problems. Notice how, in many of the problems mentioned above, the essence of the difficulty is that some of the objectives conflict with each other. Do I take the job with the higher salary, the better prospects or the nicer working conditions? Do I choose a computer system which is compatible with an older system currently in use, or the one which provides new and useful features? Of course, while conflict between objectives may seem the most significant factor in such decisions, we are generally also worried by uncertainty. In fact most decisions that cause us to pause and think involve both uncertainty and conflicting objectives. In Section 3.4 we shall see how decision theory can provide a framework for thinking about these two aspects at once; but we shall devote the current section to the handling of conflicting objectives when we have no uncertainty at all about the outcomes of our decision-making. We do this for three reasons: first, there are a small number of problems where no uncertainty exists (although most practical problems of interest are not in this class); secondly, many real problems are dominated by conflicting objectives, so that a satisfactory framework for decision-aiding can be obtained by ignoring uncertainty; and, thirdly, the theory of how to handle both problems at once is founded on a theory for how to handle each problem separately. 3.1.1 Utility and value Choice is simple if we have only one thing to worry about. If we can buy the same product in several different shops, we usually use cost as a single criterion and choose the shop where the price is lowest. In awarding a university prize, examiners could concentrate on aggregate marks alone and give the prize to the student with the highest score. In both these examples, though, other things could well be taken into account. The closer shop, or the one with the more pleasant salespeople, might be worth going to even if we have to spend a few pennies more. Perhaps the prize should go to the student with a slightly lower aggregate score but a better all-round performance. If we could establish a single numerical measure which took these other factors into account, then our choice procedure would again be straightforward; the difficulty is how to construct such a measure in the first place. The concept of a numerical measure to describe the value of alternative choices has come to be referred to as utility theory, with the 'Utility function being the numerical measure itself. Utility appears to have been first discussed by Daniel Bernoulli (1738). His concern was to explain the St Petersburg paradox, which involves the proper evaluation of a par-
20
Theory
ticular uncertain gamble (see Section 3.3.1 for a discussion of this problem). Bernoulli's explanation was that money was not an adequate measure of value. Instead he suggested that the worth, or utility, of money for each individual was non-linear and had a decreasing slope; marginal utility decreased as wealth increased. At this stage, however, the concept of utility had not been refined beyond this simple idea. In the following century, economists continued to use and develop the concept of utility to explain economic behavior, even if no uncertainty was present; indeed, in such a standard work as Marshall's Principles of Economics (1890), the idea of utility is a central notion in explaining economic phenomena. A thorough review of the development of different concepts of utility in economic enquiry is given by Stigler (1950). The utilitarian philosophers, such as Bentham and Mill, also talked of utility. To them, however, it was not a device for understanding economic behavior; rather, it was a tool for constructing a theory of ethics. Bentham's dictum that "the greatest good for the greatest number of people", should be pursued carries the implication that utility, a measure of the benefit of the options available to a person, is to be maximized in sum. While both theories were attractive, they failed in practice, for it was not possible at that time actually to measure a person's utility function in a satisfactory way. In the 1920s, several economists argued that economic theory could be founded on a less demanding assumption about measuring utility (see, in particular, Pareto (1927)). Instead of making the assumption that a number was available representing the worth of any bundle of goods {cardinal utility), one could simply assume that it was possible to say, for any two bundles of goods, which was preferred (ordinal utility). This latter notion almost entirely replaced the former in economic thinking in the inter-war years. It was not until the 1940s and 1950s that the concept of cardinal utility was resurrected, and this happened in two rather different guises. In the study of human interaction, the theory of games was developed both to describe how people behave when engaging with others with conflicting goals and to prescribe how rational people ought to behave in such situations. In their seminal book on this subject, the great mathematician John von Neumann and the no less eminent economist Oskar Morgenstern found the need to develop a cardinal theory of utility to prescribe how people should evaluate options about which they were uncertain (von Neumann and Morgenstern 1944). While their utility theory is expounded almost as an aside in their book, it has probably been the most influential contribution to the development of modern decision analysis. When the concept wasfirstsuggested, it sparked considerable controversy (see, for example, the excellent paper by Ellsberg (1954), who gives a very
Decision theory
21
clear account of the distinctions between different utility theories). We shall study von Neumann and Morgenstern's utility theory in Section 3.3 below. A separate development of ideas about utility was based on new procedures for measuring the utility of the older kind; most notably the theories of difference measurement and conjoint measurement (see Krantz et al. (1971)). Although ordinal utility is all that is needed to direct choice between multi-attributed alternatives, as the economic theorists noted, it is a difficult task to attempt to elicit such a function by plotting indifference curves. Luce and Tukey's theory of conjoint measurement (Luce and Tukey 1964) and other developments in measurement theory have spawned a revival of interest in cardinal utility, distinct from that of von Neumann and Morgenstern but applicable in the absence of uncertainty. This cardinal utility turns out to be more easily measured, and therefore more useful, than ordinal utility. We will describe these recent developments in the remainder of Section 3.1. Decision analysts have made this distinction between two types of utility, which can be characterized as the distinction between value and risk preferences, ever since the subject first developed. Value preferences are made between competing objectives or attributes when no uncertainty is present. Risk preference (as measured by the utility functions discussed above) addresses the decision-maker's aversion to, indifference to, or desire for risk-taking. We will discuss the former in Section 3.2.1 and the latter in Section 3.3. There have been many reviews of different concepts of utility in the last forty years. As well as Stigler (1950) and Ellsberg (1954) mentioned above, the interested reader is referred to Edwards (1954) and the many writings of P.C. Fishburn, and in particular Fishburn (1976). A useful discussion of the theoretical and practical distinctions between these concepts is given in Barron et al. (1984). 3.1.2 Value functions 3.1.2.1 Problem formulation As with much of this text, our account of value functions must necessarily be succinct, because of the breadth of issues we discuss. We will state the key elements of value function theory, and direct the reader elsewhere for a fuller account. For much of this chapter the reader will find the presentation of Keeney and Raiffa (1976, pp. 66-129) an excellent expansion of ours. This section will be concerned with the properties a value function should have; first, however, we will review why we want to construct one in the first place. We introduced the idea of a value function to assist people faced with decision problems; we can characterize this problem as follows. A set of
22
Theory
options has been well defined. Note that the terms alternatives, choices, decisions, or actions are used by some writers in place of the word options-, the literature sometimes makes careful distinctions between them and sometimes, as here, uses them interchangeably. We assume that we know what the options are and, perhaps more importantly, that we know what the attributes (or factors) are which discriminate amongst them. We suppose that there are n such attributes, named Xx, X 2 ,..., Xn, and that it is possible unequivocally to provide a numerical score for each option with respect to each attribute. The set of scores with respect to the attributes for a particular option will be represented by a vector x = {xx, x2, . . ., xn). The problem is now reduced to deciding which vector of scores is most attractive. For example, consider the problem of deciding which car to buy — a problem often addressed in discussions of multi-attribute decision-making. The options are the different cars we might choose to buy; the attributes are things like purchase price, mileage, reliability, and so on. Although at first sight the idealization above may seem to be a reasonable abstraction, real problems are rarely like this, at least to begin with. First, it is often the case that the option set is not completely known, or at the very least we may not know whether all the possible options have been identified. There may, to follow our example, be cars available which we have overlooked, or models which differ from what we had imagined them to be. We could decide not to bother about this point, arguing that decision theory is only concerned with what to do once the options are specified. It is obviously of practical importance, however, for people wanting guidance with real problems. The generation and delineation of decision options have been active research interests in recent years, and we shall discuss this point again in sections 7.2 and 7.3. Secondly, it may not be clear how to characterize the attributes that distinguish between the options, nor how to measure each option with respect to some of the attributes. We know, for example, that reliability is one attribute that distinguishes among cars; but is reliability to do with the frequency of breakdown, or the severity of breakdown, or both? Even if we can be precise, how should it be measured? We will return to these issues later; for the present we will assume that they can be overcome sufficiently for the analysis of our idealized problem to be useful in tackling real-choice problems. 3.1.2.2 Value functions: definition and alternatives Suppose that we can construct a real-valued function v{ . ), with the property that ^(x^) > v{xB) if and only if we prefer the option whose vector of scores is xA to another with score vector xB; and with the further property that v{xA) = v(xB) if and only if we are indifferent between xA and xB. We
Decision theory
2.3
Figure 3.1 Dominance. In the figure £, F, G are dominated, E by B, F by C, and G by B and C. A, B, C, D are not dominated, and form the efficient frontier. shall refer to such a function as a ftf/«e function. Then the choice of option is simple. We merely choose the option with the score vector x for which v(x) is largest. The difficulty will be in constructing this function v( . ). If the only way to do so were to elicit from the decision-maker his preference between all possible score vectors, then we would not have achieved anything by trying to construct a value function. The choice problem would have been answered by direct intuition. It is the fact that we find it so difficult to compare sets of scores directly that leads us to choose some other route to decision-aiding. Fortunately, as we shall see, there are ways of computing v{ . ) that rely on general properties of our preference structure and just a few simple comparative judgments; the power of the method is that if our preferences obey certain conditions (and they often do) we can synthesize v{ . ) from an analysis of only parts of our set of preferences. Before we discuss this theory, we note that many authors have investigated what may be done without a complete specification of v( . ). The simplest observation is that the notion of dominance might lead to a unique choice. An option A is said to dominate another option B if, for all / in 1,. . . , «, xf ^ xf and, for at least one /, x£ > xf, using an obvious notation. The notion of dominance is illustrated in Figure 3.1 in the case of two attributes. Clearly, if one option dominates all the others, it is
24
Theory
rational to choose it; that is to say, the choice of any other option is inconsistent with our statement of preferences. (Note that we have assumed that scores have been arranged so that an increase in the score on any attribute is a good thing; it is always possible to do this.) Needless to say, dominance rarely occurs with problems that are taxing enough to require analysis, and we need to search further for appropriate methods for most problems we might face. Note, in passing, that there are more sophisticated methods which use the dominance concept (see Keeney and Raiffa, 1976, p. 128). Another device that is often very valuable in seeking the best from a set of options is related to dominance. If we remove from the option set all options dominated by some other option, the remaining set is called the efficient frontier, or the Fareto-optimal set (see Figure 3.1). If our preference structure implies a best choice, it is certainly on the efficient frontier; but to find it we need more information. One idea, which does not involve the complete construction of v( . ) to complete the search, is to conduct a myopic search along the efficient frontier. We will mention some of this work in Section 3.5; here we just note that the input information for procedures of this kind is preferences between neighboring points on the efficient frontier. Within the community of decision analysts, there are some who approach problems with multiple attributes by pricing out all but one attribute (see, for example, Howard et al. (1972), or the more extended discussion in Keeney and Raiffa (1976, Section 3.8)). This approach finds a point on the same indifference surface as the point in question, but where all attributes but one have a standard set of values. Comparison of score vectors is then done by the simple comparison of values on the last attribute, which is usually taken to be money. The main difficulty with this approach is the problem of tracing along indifference surfaces. We will now restrict attention to methods that construct an explicit value function, since these are the most tractable and useful in practice. 3.1.2.3 Existence of the value function The notion of effecting an explicit construction of v{ . ) as a way of solving the idealized decision problem makes intuitive sense. There are many practical situations where this approach has been widely accepted as a sensible device for routine decision-making. Credit-scoring, examination marks, and rules for the UK Rate Support Grant are obvious examples. What are of interest to us here are the procedures that ought to be employed in constructing such evaluation indices. Note that the existence of a value function to represent preferences is equivalent to there being a complete preference order over the options. Clearly if a value function exists it induces a complete ordering on the
Decision theory
25
vectors x. Conversely, it is easy to see that if we do have a complete preference order, then we can construct a value function by merely placing the vectors in increasing preference order and attaching any increasing sequence of numbers to the vectors. If then we feel confident that, given enough introspection, we could produce a complete ordering on the vectors x, there must be some value function v( . ) that can represent our preferences. Some authors have criticized utility and value theory on the grounds that, if we could articulate all such preferences, then we would not need to construct v( . ); we could decide by simple introspection. The fact that we find it difficult to make such comparisons must imply, they argue, that either no value function exists or, if it does, it cannot be found. Such criticism misses the point. So long as we assert that there must be some preference ordering of all vectors x but that we are unable to articulate all of the constituents of this ordering, then we can use a value function to express that ordering. This is a most important point in the interpretation and use of value function theory. Since a partial ordering of our preferences can be inferred from any set of decisions we make, the point in creating a value function is to help us construct preferences between score vectors which we find difficult to compare directly. The problem, then, is to construct our preferences within and across the attributes Xj of x in such a way that we have a complete ordering of all vectors x. This theory is normative, showing us how we should behave if we want to be rational, rather than descriptive of how we do behave. We may use value function theory to construct preferences which we find difficult to articulate directly, using only those preferences that we can express easily and adopting the principle of complete transitivity for all our preferences. Having assumed the existence of v( . ) for a particular problem (and note that there is no reason why v( . ) should remain constant in time, or from problem to problem), we now have to measure it. Clearly any monotonic function of v( . ) will now also be a value function, since it will satisfy the definition at the beginning of Section 3.1.2.2. This means that the critical aspect of the value function is the set of indifference curves it implies, and, as we have already noted, it is cumbersome and difficult to elicit this, particularly if the number of variables is large. Fortunately, however, there are properties that our preference structure might (and often does) have which lead us to special forms of the value function which can be more easily elicited. 3.1.2.4 Additive value functions The most often used form for a'value function is the additive one: n
~
.•(*,•).
(3-i)
z6
Theory
The simplicity of this form and the ability to work on each attribute separately make it an attractive candidate for our consideration, although it is obviously quite possible for a particular preference structure to lead to a non-additive value function. For example, suppose we wish to evaluate different possible designs for a microwave relay station as part of a communications network. The attributes of importance to us might be the ability to receive signals clearly, the ability to transmit them, and the cost. Suppose that these are measured by the variables R, T, and C respectively. Then it would be plausible that someone might have the following preferences: and
(RH, TH, CH) preferred to (RH, TL, CL) (RL, TL, CL) preferred to (RL, TH, CH)
where the suffixes L and H indicate specific high or low values of the variables. The first preference expresses the view that it is worth paying the higher price, CH, for the improved transmission ability {TH instead of TL) if the receiving ability is good. The second preference states that the high price is not worthwhile if receiving ability is poor. In this case no additive value function may be found consistent with both these preferences, as may be readily demonstrated. Despite examples such as these, it turns out that we can get quite a long way with additive value functions. We will now, therefore, explore what the conditions on our preferences have to be to justify additivity. The essential condition is that of preference independence. A pair of attributes, Xi, X2, is said to be preference independent of all the other attributes {X,, / = 3, . . . , « } if preferences between different combinations of levels of Xi, X2, with the level of all other attributes being held at constant values, do not depend on what these constant values are. Thus, in the car-buying example, if we say that cost and mileage are preference independent of reliability, it means that if we prefer a car with cost cu mileage m^ and reliability r (measured on some suitable scale) to one with cost c2, mileage m2, and reliability r, then we also prefer a car with cost cu mileage mi, and reliability r' to one with cost c2, mileage w2, and reliability r', no matter what r' is. Moreover, this holds for all possible cu c2, mi, m2. The microwave relay station example above shows a situation where preference independence does not hold. Now we can quote what Keeney and Raiffa (1976) refer to as a "truly remarkable" theorem. First we define mutual preference independence. If our preference structure is such that any two attributes are preference independent of all the others, we say that the attributes display mutual preference independence. It can now be shown that, if this condition is satisfied by our preference structure, then there must be a value function with the additive form given by Equation (3.1) above. Moreover, the
Decision theory
27
function v( . ) is unique up to a positive linear transformation; that is to say, if there is another function v'( . ) with the same properties, there must be some numbers a and b, with b > o, such that v'(x) = a + bv{x). This type of function obeys the properties of an interval (or cardinal) scale; that is, an interval of some given number of units on the scale represents the same increment in value no matter what the starting point on the scale is. The analyst can arbitrarily assign numbers to any two points on the scale; or, equivalently, he may choose a and b as he wishes. An example of such a scale is temperature; the Fahrenheit and Celsius scales are but two possible ways to measure temperature. Fishburn (1970, chapters 4 and 5) and Krantz et al. (1971, Section 6.2.4) give detailed proofs of this result, and Keeney and Raiffa (1976, Sections 3.5 and 3.6) give a rather more readable discussion of the same body of theory. We will not reproduce these accounts here, and merely refer the interested reader to these other texts. Mutual preference independence is often a plausible condition and, as we shall see, it is possible to test whether our preference structure does satisfy this condition in many practical contexts. While being the most common form of value function, both in theoretical accounts and in practice, the additive form is not the only possibility. Several authors have discussed multiplicative or other forms. An interesting possibility is a part additive and part multiplicative form. Keeney and Raiffa (1976) give a fuller account of this point in their Section 3.6. 3.1.3 Summary In this section we have described the purpose and theory of value functions. We first discussed the problem of decision-making in the absence of uncertainty, and the appropriateness of the standard idealization of the problem. Then after a brief history of ideas in this area, we described the axiomatic approach to value functions, and in particular the conditions which lead to an additive form. We will show how additive functions may be elicited and used in Section 7.5. There is much more that needs to be said, and will be said, about this topic in this book. The problem of structuring a decision prior to applying the theory raises some important philosophical and practical issues, and these will be discussed in Section 7.2. Eliciting values is also a problem area, as will be seen in Section 7.5. The use of this theory, both for individual decision-making and for decision-making within organizations, has implications which go far beyond its mathematical'formalism; these are discussed in Chapter 5 and in Part II. Other formalisms have been suggested for addressing the same idealized problem that we have discussed here. These differ from the theory we
28
Theory
have presented here, sometimes in significant and subtle ways. We review some of them in Section 3.5. EXERCISES FOR 3.1
Sue has a job decision to make. She has been offered four jobs, each having different starting salaries, promotion opportunities, locations, and required levels of creativity. Sue has had a course in decision analysis and has structured her decision problem with the following attributes, values, and weights:
Jobs A B C D
Weights
Starting salary
Promotion opportunity
Location
Required level of creativity 60 40
0
20
100
0
30
65
100
0
0
2-5
75
100
100
0.4
0.2
O.I
0.3
(a) Assuming that all pairs of attributes are preference independent of the other attributes, which job maximizes Sue's value function? (b) If Sue changes her weights for the attributes so that starting salary has a weight of 0.2 and promotion opportunity a weight of 0.4, does the job with the highest value change? (c) Do either of these results change if job C is removed from the analysis? 2.
Select a decision problem of concern to you, such as the choice of a job, and apply the methods of multi-attribute value analysis to help you think through it.
3.
The committee of a students' union is considering whether to introduce a copying machine for students' use and, if so, which machine to purchase. It is recognized that the available machines differ with respect to quality of copying, reliability, ease of maintenance, and cost. Describe in detail how the methods of multi-attribute value theory may be used to assist with this decision problem. Pay particular attention to the assumptions necessary for the method to work, and their likely validity in this case. 3.2 UNCERTAINTY
In Section 3.1 we analyzed the problem of choosing between options when our objectives conflicted. We assumed that we knew with certainty
Decision theory
29
what the results of choosing an option would be, so that the only problem was how to cope with the conflict between our objectives. All these assumptions are unrealistic for most real decision problems, but the problem of uncertainty is of particular importance, since it is very hard to discover any real decision problems where we are not at least slightly uncertain about the outcomes. In the car choice problem, for example, we may be unsure about how reliable the particular car we will buy will be, or how we will rate the car's comfort once we are using it regularly, and so on. In order to use value function theory to aid in the car choice problem, we had to ignore our uncertainty and hope that this would not affect our conclusion. This is a judgment that can only reasonably be made if we have some idea of what kind of analysis we might do if we did address the uncertainty explicitly, and what its outcome might be; such an analysis will be described in this chapter. We will find that it will generally be reasonable to ignore uncertainty in the car choice problem, but there are many problems where the chief difficulty is uncertainty. In most financial decisions, for example, the key to a satisfactory decision is the proper assessment of uncertainty within an analysis. If I am wondering whether to buy stocks or fixed interest securities with some spare cash, I have to balance the surer return from the latter against the possibly higher return from the former. If a company is deciding whether or not to build a new warehouse, it is usually balancing the uncertain cost streams of doing so against not doing so. Some years ago, most texts on decision analysis presented the subject as being almost entirely concerned with uncertainty, and generally where there was only one attribute of interest, usually money (see, for example, Raiffa (1968) or Lindley (1973)). The subject has advanced, however, and, as we argued above, the analysis of conflicting objectives is now emerging as the more useful area in practice. Nonetheless, dealing with uncertainty is vital in many applications. 3.2.1 Descriptions of uncertainty If we are to incorporate uncertainty into our theories of choice, then a calculus for handling uncertainty must be developed. The theory that is most commonly employed to do this is that of probability. Why should we use this theory to describe uncertainty? Might it be that the only reason for using it is that the notion is so well imbedded in our scientific culture that we do not stop to question if it is the only possible, let alone the correct, calculus to employ? Other calculuses have recently been suggested for handling uncertainty, and so we shall need to argue for the use of probability theory rather than some other. Probability theory is itself not very old, being usually dated back to the study of games of chance by seventeenth- and eighteenth-century mathematicians such as Pascal, the
30
Theory
Bernoullis, and Laplace. A theory describing games of chance is not necessarily appropriate for describing uncertainty in other contexts, however. Even if we adopt probability theory as our tool for describing uncertainty, we must decide which of the many versions of that theory to employ. Readers whose only exposure to probability has been in high school mathematics courses, or even in university statistics courses, may be surprised to learn that there is any dispute about probability. The theory is often presented as a branch of pure mathematics, and as such it is as non-controversial as arithmetic or set theory. Contention arises, however, as to the meaning of probability, and in particular its relation to real-world phenomena. Over the last two centuries, theories of probability have been many and varied. Most prominent philosophers have contributed to the debate, as have scholars in other areas who needed the concept of probability in their own work. Most authors presenting new theories of probability also argue against older theories, for obvious reasons. Very few comparative accounts exist which allow a disinterested reader to balance the arguments in favor of one or another interpretation. Exceptions are the text by Fine (1973), which gives a detailed account of the axiomatic development of the competing theories, and the more recent comparative study of Weatherford (1982). Hacking (1975) provides an excellent historical account of the development of probability ideas. While recognizing that other categorizations of probability theories are possible, Weatherford sees four distinct interpretations for probability. We have not the space here to describe these distinct theories in any detail, and the reader interested in pursuing them is recommended to consult Weatherford (1982). It is important to note, however, that the original ideas of the classical theorists, such as Laplace, were found to be inadequate in dealing with the real world, since the theories did not incorporate a satisfactory method for measuring probabilities. The a priori, or logical, theories of Keynes and Carnap, the relative frequency theories of Venn, von Mises, and Reichenbach, and the subjective theories of Ramsey, de Finetti, and Savage all claimed to improve on these ideas, but in different ways. The most important conflict that we need to mention here is between the last two of these theories. The relative frequency theory holds that the probability of an event is the long-run frequency with which the event occurs in an infinite repetition of an experiment; it is thus seen as an objective property of the real world. Subjective theories present probability as the degree of belief which an individual has in a proposition; this is a property of the individual's subjective perception (or state of knowledge) of the real world. The former theory has dominated scientific thinking for many years, because it has the appearance of conforming to the empirical objectivity which
Decision theory
31
science is sometimes claimed to need and possess. It will not do, however, as a theory for helping an individual to model his perceptions of uncertainty when making decisions. The interpretation of probability which must be adopted for the application of decision theory has to be the subjective one, and we will describe this in detail in the next section. The reader should be aware, however, that not all scholars are prepared to accept this theory (see, for example, the discussion in Weatherford (1982)), but that, if the subjective theory of probability is not adopted, the intellectual justifications for the procedures of decision analysis are much weakened. Alternatives to probability theory are beginning to emerge. The following brief account of some of these alternatives may appear frustratingly inadequate to readers. Our purpose, however, is not to give a detailed comparative account of these theories; we have insufficient space to do this. Rather, we mention them so that readers who wish to consider alternatives to probability may know that such alternatives exist and where to go to find more detailed accounts. First, we mention Dempster's theory of upper and lower probabilities (Dempster 1968) and the related but more general theory of belief functions, due to Shafer (1976). These endeavor to construct a richer description of uncertainty than is provided by probability theory, allowing people to refrain from making judgments on relative likelihoods between some pairs of events. These theories have not yet, to our knowledge, been used satisfactorily with practical problems. Secondly, Zadeh's fuzzy set theory has also been suggested as an alternative to probability theory (see, for example, Zadeh (1983)). Zadeh's theory is intended to describe imprecision, rather than uncertainty, and, although these two concepts are obviously related, they are distinct. The use of fuzzy set theory to describe the imprecision that an individual might perceive about his probabilities, or utilities, has been suggested by Watson et al. (1979) and by Freeling (1980). A good set of readings on fuzzy set theory has been given by Mamdani and Gaines (1981).
Thirdly, we should make brief mention of a calculus for uncertainty developed in the course of constructing the expert system for medical diagnosis called MYCIN. Its constructors needed to be able to represent uncertainty and propagate it through the system. Shortliffe (1976, Chapter 4) gives a detailed account of the "certainty factors" employed in MYCIN, including why they were created and how they are to be manipulated. The theory has presented problems in application, however, despite its attractive properties, and the Stanford University research group responsible for MYCIN is now considering the use of Shafer's theory instead (see Buchanan and Shortliffe 1984, Chapter 13).
32
Theory
Finally, we wish to mention L. J. Cohen's theory of inference (Cohen 1977). Cohen is concerned with evidence and how to present its uncertain implications in terms of the number of different kinds of ways the conclusion could turn out to be false. This theory has not yet been developed into a practical tool for representing uncertainty, but it may well be so developed in the future.
3.2.2 Theoretical background to subjective
probability
Probability theory is an obvious construct for describing subjectively perceived uncertainty. Not only has it been deep in scientific culture for the last fifty years, at least; in chance situations, such as card games, it is the proper calculus to describe relative proportions. To apply this calculus in other situations, however, requires more than an act of faith. What is needed is a set of assumptions about an individual's judgments which, if satisfied, leads us to infer that a set of numbers must exist which describe that individual's perceptions of uncertainty; and, moreoever, that these numbers should be combined using the rules of the probability calculus to infer what numbers should be used to describe other uncertainties. This development was successfully undertaken by Ramsey (1931), Savage (1954), and, most extensively, by De Finetti (1974). (See Kyburg and Smokier (1964) for a collection of the most important papers on this topic.) Most accounts of this work are far from transparent, since considerable mathematical development is required. DeGroot (1970, Chapter 6) and Lindley (1973, Chapter 2) both give a clear presentation. The behavioral suppositions are set out as axioms in Exhibit 3.1. We first suppose that, given two uncertain events, a person can say which is more likely, or whether they are equally likely. Next, we suppose that three highly plausible conditions are satisfied by these judgments of relative likelihood (axioms 2, 3, and 4 in Exhibit 3.1). Finally we suppose the existence of a set of events which acts as a standard in probability judgment. Lindley (1973) takes this to be the angular position adopted by a pointer on a perfectly balanced wheel after it has been spun; but many possible sets of events will do. All that is needed is to be able to find one member of the set whose likelihood is judged to be equal to any uncertain event being considered. With these apparently flimsy assumptions it can be proved (see DeGroot 1970, Chapter 6) that a set of numbers exists which corresponds to the judgments of relative likelihood, in the sense that, if A is judged to be more likely than B, then the number for A is bigger than the number for B. What is more, these numbers must, collectively, satisfy the properties of probabilities; that is:
Decision theory (1) (2) (3)
33
O^p(A)^l; p(A or B) = p{A) + p{B) - p{A and B); p(A and B)=p(A)p(B\ A).
Here A and B are any events, and we represent the probability of an event X by p(X); the probability of B occurring conditional on A occurring is written p(B \ A). Axioms for probability 1.
For any two uncertain events, A is more likely than B, or B is more likely than A, or they are equally likely.
2.
If /A1 and A2 are any two mutually exclusive events, and 81 and B2 are any other mutually exclusive events; and if /*-, is not more likely than B-,, and A2 is not more likely than B2\ then (A| and A2) is not more likely than (B-, and B2). Further, if either A<\ is less likely than B-i or A2 is less likely than B2, then (A^ and /42) is less likely than (8 1 and B2).
3.
A possible event cannot be less likely than an impossible event.
4.
Suppose A|, A2, . . . is an infinite decreasing sequence of events; that is, if A occurs, then A,-occurs, for any /'. Suppose further that each A,is not less likely than some other event 8, again for any /. Then the occurrence of all the infinite set of events Ah i = 1, . . ., 00, is not less likely than B.
5.
There is an experiment, with a numerical outcome, such that each possible value of that outcome, in a given range, is equally likely.
Exhibit 3.1 If an individual is able to, or wishes to, express his judgments of likelihood according to these axioms, then numbers must exist which describe his perceptions of the uncertainty of any event which satisfy the rules of the probability calculus. Conformity to these rules is our definition of what it means to be rational in evaluating uncertainty. The power of these results is remarkable, and yet many scholars are not prepared to accept the notion of subjective probability. They argue that the first of our assumptions, that we can compare the relative likelihood of any two events, is patently not true. Since the premise is false, they argue, then the conclusion is not true either. But, as we pointed out when discussing value functions in the last section, one of the points of developing decision theory at all is to establish a framework to guide rational behavior. It is not important whether we observe our judgments conforming to the axioms; rather the question is whether we choose to behave in a way such that our judgments do have this property. What the theory we described in the last paragraph tells us is that, if we would like all our judgments of relative likelihood to exist and to exhibit transitivity, then subjective probabilities should be used.
34
Theory
Recently an alternative development has been proposed which also leads inexorably to probability theory as the proper way to describe subjectively perceived uncertainty (Lindley 1982). This is based on scoring systems rather than judgments of relative likelihood, but still supports the use of subjective probability. 3.2.3 Using probabilities It is one thing to show that, if our judgments of uncertainty are to be represented by numbers, then those numbers must satisfy the rules of probability; it is quite another to elicit such numbers in specific cases. Probability elicitation is not an easy matter; we give an account of ways to do it in Section 7.4. Faithful application of the methods described in that section may still leave us with problems, however. Subjective probabilities have to obey the laws of probability; they must cohere with each other. For example, if we assess the probabilities for the events A, B, (A given B) and (B given A) to be /?(A), p{B)y p(A I B) and p(B | A), respectively, they must satisfy the equation: p(B\A)p(A) = p(A\B)p(B) since both must be equal to the probability that A and B both occur. If this relationship is not satisfied by the elicited numbers, then one or more of the four numbers must be changed. The traditional answer to the question of just how this should be done was that they should be adjusted until the subject was "happy" with them. While this is a reasonable rule of thumb, it would be more satisfactory if a prescriptive approach to the problem of reconciling incoherence were established. This problem has recently been addressed in a series of papers on the resolution of incoherent judgment (see Lindley et al. 1979). One of the reasons for the observed phenomenon of incoherence in a set of probability estimates may be that the subject is suffering from one of a number of cognitive biases. We shall be discussing this issue more fully in Chapter 4; here we just note that there is considerable evidence that, in assessing perceptions of uncertainty, people are often subject to biases in cases where a norm for comparison exists, and that there is therefore reason to expect this behavior in other contexts as well. If people are consistently biased in assessing probabilities, it is natural to ask whether those biases may be corrected for. This issue has been explored in the literature on the calibration of probability appraisers, which we shall also discuss in Chapter 4. The literature is controversial, and no general conclusions can be yet derived about the possibilities of correcting for biases.
Decision
theory
35 Final asset position £1,000
Do not accept gamble
Acceptgamble
-£2,000 Figure 3.2 Decision tree for a simple decision problem. To use probabilities in a decision analysis it will be necessary to know some of the details of probability theory. In particular, the following concepts will be needed: (1) (2) (3) (4) (5)
random variables, both discrete and continuous; cumulative probabilities, probability density functions; moments of distributions, means, and variances; conditional, joint, and marginal probabilities; independence; distributions of functions of random variables.
We have chosen not to present this theory here, for reasons of space. The reader who is unfamiliar with these concepts should consult one of the many standard introductory texts in probability theory, such as Ross (1976). 3.3
RISK PREFERENCE
3.3.1 Utility theory Now that we have seen how to describe our subjectively perceived uncertainties in numerical terms, we can move on to discussing how this formalism can be used to aid decision-makers faced with alternative courses of action, each involving uncertainty. To assist our description of the theory in this section, we will construct a hypothetical exafhple. A decision-maker currently has £1,000 of liquid assets, and has been offered a gamble which on the toss of a coin could win or lose him £3,000. The situation that he faces can be described by the diagram in Figure 3.2. A
36
Theory
diagram of this kind is referred to as a decision tree. Its branching structure gives rise to its name, and it has proved to be a useful way of representing a decision problem. Indeed, merely describing a problem in this way can often lead to an appreciation of what the best option ought to be, if it is not obvious to begin with. However, the power of decision analysis is best appreciated when we have to do more than merely construct the decision tree. It is conventional to represent a point where several alternative developments may occur and over which the decision-maker has no control (a chance node) by a circle (as here) or a dot; and a point where the decision-maker has to make a choice (a decision node) by a square. We now seek some way to evaluate the uncertain options (or lotteries or gambles, as we shall refer to them) facing a decision-maker. We want to construct an evaluation method such that, once a problem has been specified, its decision tree drawn, and the uncertainties measured using probabilities, we will be able to compute a number that represents its value, in a way that is consistent with all our judgments. Perhaps the most obvious candidate, at least in problems where the only important attribute is money, is the expected monetary value for the option. We multiply the probability of a particular future set of events by the final asset position if that future occurs, and sum over all possible futures. This was the measure first explored by Daniel Bernoulli (1738) when he described the St Petersburg paradox. He considered the following situation: somebody is given the option (for a price) of playing a game which involves the successive tossing of a fair coin (that is, a coin for which it is reasonable to take the probability of heads as 0.5) until a head appears for the first time. If the first head appears on the nth toss, the prize is zn. The expected value of this game is 2(0.5) + 4(0.5) 2 + 8(0.5) 3 + . . . which clearly has infinite value. Thus, if expected value is a sensible measure of the attractiveness of an uncertain option, we should all be prepared to offer any sum, no matter how large, for the privilege of playing! The fact that most of us would be unwilling to give more than a few pounds for the privilege of playing this game (if denominated in pounds) is the source of the paradox. The solution lies, of course, in the inadequacy of the assumption that expected value is a suitable measure for an uncertain option. If we return to the exemplary decision problem depicted in Figure 3.2, we see the same phenomenon. The expected value on each branch of the decision tree in Figure 3.2 is the same, namely £1,000; yet not many of us would accept the gamble, unless we felt that our non-liquid assets could easily absorb a loss of £2,000.
Decision theory
37
Bernoulli's solution was to argue that the value (to the individual) of money was a non-linear function; for example, an increase of the prize by £1,000 would be far less significant if it already stood at £100,000 than if it only stood at £2,000. For suitable forms of this non-linear function (Bernoulli suggested a logarithm), the expected value was finite, even though the expected monetary value was infinite. As we discussed in Section 3.1, this insight was one of the origins of the concept of utility. However, early writers in this area made no distinction between attitudes towards value differences and attitudes towards risk. A modern view of Bernoulli's argument would be that he was correct in seeking to evaluate gambles in terms of expected utility, but that his utility should have been defined in terms of risk preference rather than preferences among value differences. We have already discussed the latter concept in Section 3.1, in our account of the definition and use of value functions. Here we will develop a utility function, based on attitudes towards preferences among risky options, which solves the St Petersburg paradox. It was with the publication of von Neumann and Morgenstern's The Theory of Games and Economic Behavior in 1944 that modern utility theory can be said to have begun. Their imaginative definition of utility led, for the first time, to a coherent normative theory for decision-making in the presence of uncertainty. The idea is to define the utility of an outcome as equal to the probability of winning a standard prize in a gamble such that the individual is indifferent between receiving the outcome for sure and accepting the gamble. This rather non-obvious definition needs fleshing out, but we shall see that coupled with some fairly innocuous behavioral postulates, it leads to an operational utility theory, and to expected utility as a sensible measure for gambles. We will start by exploring the utility for money in a defined range, although we will see that the concept can be extended to any amount of money, and also to any non-monetary attribute. Let the money values in which we are interested lie in the range [L, H], so that L is the lowest possible monetary value, and H is the highest. Now consider some intermediate amount X. We shall now compare the gamble {L, H;p}, with the sure receipt of an amount X. The notation we have just used to describe gambles is explained by the display of this choice in Figure 3.3. We have assumed that L < X < H, so that, presumably, if p were equal to 1, then the decision-maker would prefer the gamble to the certain option (of course, both options would be certain in this case). Similarly, if p were zero, then the preference would be reversed. If one accepts our ability to make fine distinctions in probability judgments, then it is reasonable to suppose that there is some intermediate value of p such that the decision-maker is indifferent between the two options in Figure 3.3. The utility of X is defined to be equal to this value of p. Using the symbol
38
Theory
Figure 3.3 The concept of equivalence. For any outcome X which is preferable to L, and not as good as H, there is some probability p such that the decision-maker is indifferent between X for sure and a gamble giving a chance p of getting H and 1 —p of getting L. ~ to denote the indifference of the decision-maker between the options before him, we have that: as our definition of his utility function u( . ). 3.3.2 Comparing risky options How can we now use this construct to evaluate gambles? Consider the gamble {S, T; q}, where S and Tare prizes intermediate between L and H. Since, by definition, the decision-maker is indifferent between S and {L, H; u(S)}> he will not mind, presumably, if we replace the prize S by the right to enter a second gamble {L, H;u(S)}. (This is the substitution principle, a source of considerable controversy.) We can make a similar argument about the prize T. Symbolically, we are asserting that the decision-maker ought to have the preferences: {S,T;q}~{{L,H-MS)},{L,H;u(T>};q}. This is represented in Figure 3.4. Now the two-stage gamble has four possible branches, but only two monetary outcomes, namely L and H. The probability of getting H is q' = qu(T) + (l-q)u(S). The two-stage gamble is clearly equivalent to the one-stage gamble {L, H; qu(T) + (1 — q) u(S)}9 as indicated in Figure 3.4. Observe now that S, T and q were arbitrary, so that we can assert that the decision-maker should be indifferent between any simple gamble with two prizes in the range [L, H] and a simple gamble on the standard prizes (L and H), with a probability of getting H given by the expression above.
Decision theory
39
Figure 3.4 Equivalence produced by the substitution principle. The decisionmaker is always ready to swap the sure thing X for a gamble on H and L with a probability u(X) of getting H.
Presumably the decision-maker will prefer gambles with a bigger chance of winning H. So, faced with a choice between several simple gambles of the form {S/5 T/;^,}, he should choose the gamble for which the expression is largest. This is just the expected utility of the gamble, however; that is, it is the probability-weighted sum of the possible utilities. We have thus concluded that the best course of action is to choose the option with the highest expected utility. This definition of utility has justified Bernoulli's insight that expected utility is a proper measure for the value of a gamble. Several times in this discussion we have made assumptions about the preferences of the decision-maker. The reader may or may not agree with us that they are plausible statements of how a rational decisionmaker might want to think, even if he did not always carry them out in practice. It is possible to take these assumptions, or others equivalent to them, as axioms, and from them deduce quite rigorously what we have just outlined above, namely that the axioms imply that the decisionmaker ought to choose the option which maximizes his expected' utility. One can axiomatize expected utility in several ways, and this has been achieved by several authors; but Savage (1954), in his joint axiomatization of subjective probability and utility, gives what is probably the
40
Theory Axioms for utility
1.
The decision-maker can place all possible lotteries in a preference order. That is, for any two lotteries, A and B, he can state which he prefers, or whether he is indifferent between them; furthermore, if he prefers A to B and B to C, then he prefers A to C.
2.
If the decision-maker prefers P^ to P2 and P2 to P3, where P^, P2, P3 are three prizes, then there is some probability p, between 0 and 1, such that he is indifferent between getting P2 for sure and a lottery giving him P: with probability p and P3 with probability (1 - p).
3.
If the decision-maker is indifferent between two prizes, P^ and P2, then he is also indifferent between two lotteries, the first giving P, with probability p and a prize P3 with probability (1 - p), and the second giving P2 with probability p and P3 with probability (1 - p). This must hold whatever P3 and pare.
4.
If two lotteries each lead only to the two prizes P^ and P2l and the decision-maker prefers P-i to P2, then he prefers the lottery with the higher chance of winning P1#
5.
Suppose lottery A\ gives a prize P^ with probability pit and a prize P2 with probability (1 - pi), for / = 1, 2, 3; and suppose lottery B gives entry to lottery A2 with probability g, and entry to lottery A3 with probability (1 - q); then the decision-maker is indifferent between lottery A^ and lottery B if, and only if; Pi = QP2 + (1 - q)Ps-
Exhibit 3.2 If an individual can subscribe to these properties of his preferences for lotteries, then it can be shown (see Baumol 1972, pp. 548-51) that a utility function must exist and that the rational action is to choose the option with the highest expected utility. most comprehensive account available. Exhibit 3.2 contains a version of these axioms, and the reader is referred to Baumol (1972, pp.548-51) for a proof that they imply that expected utility should be maximized. We shall return later to a discussion of whether the behavioral postulates set out in Exhibit 3.2 are reasonable ones to follow; for the present we point out that this theory merely shows that, if we wish to act in accordance with them, then we should always maximize expected utility. Several observations need to be made about the utility we have defined. First we see that, since L ~~ {L, H; «(L)}, u(L) = o. Similarlyit follows that u(H) = 1. Secondly, any other utility function will differ from u by at most a positive linear transformation; that is, if u' is another utility function, then
Decision theory
41
tt'(X) = a + bu(X) for some positive number b. To see this, note that the only arbitrary things about the definition of u are the reference prizes L and H. So suppose we construct another utility function using some other reference prizes, U and H', and, without loss of generality, we suppose that L' < L
Theory
42.
Final asset position (£) Do nothing
1,000
Lose £3,000
Win £3,000
4,000
Investment fails 0.1
Patent infringed
0.7/ 0 3\
Success
800 -200 2,800
Failure Associate
0.9/
-2,000
pays Associate does not pay
0 1,800
800
Figure 3.5 Decision tree for the decision-maker's extended problem. For a more complex example, let us suppose that the decision-maker is offered another more complicated gamble (or business opportunity) as an alternative to the gamble depicted in Figure 3.2, as illustrated in Figure 3.5. For an investment of £200 of his original £1,000, he can accept a share of the business opportunity. There is a 10% chance that after a month information will become available showing that the investment will yield nothing, and another 10% chance that at this time the investment will turn out to infringe a patent, resulting in the decisionmaker paying afineof £1,000. If neither of these eventualities occurs, the decision-maker may still withdraw at this stage, selling his option to an unreliable associate, who will offer £1,000 but with probability 0.1 may never pay up. Alternatively, he may wait until the investment matures; there is a 70% chance that the decision-maker's share will be £2,000, but a 30% chance that he will lose a further £800. This problem, though more complicated than the original problem, is still considerably simpler than
Decision theory
-2,000 Figure 3.6
-1,000
0
1,000
43
2,000
3,000
4,000
Decision-maker's utility function for final asset position. Final asset position (£) Do nothing
—
Lose £3,000
Win £3,000
0.83I Investment fails Patent infringed
0.7/
Success
n^S
Failure
0.9/
Associate pays
0i\
Associate
does not pay
1,000
Utility 0.85
-2,000
0.00
4,000
1.00
800
0.83
-200
0.67
2,800
0.97
0
0.71
1,800
0.92
800
0.83
Figure 3.7 Decision tree and utilities for the decision-maker's extended problem. The utilities for the ending positions are included, and expected utilities are in boxes.
44
Theory
many decisions that involve complex uncertainties in practice; nonetheless, it is sufficiently complicated for the rational choice not to be obvious. We shall now demonstrate how the analysis of this problem, using decision theory, leads to the synthesis of a recommended decision. Thefirststep is to determine the decision-maker's utility function for all the potential final asset positions. Let us suppose that by applying the utility elicitation techniques described in Section 7.5.6 a curve such as that in Figure 3.6 has been produced. We can read off the decision-maker's utility values from this curve, and we have included the utilities for the final asset positions in Figure 3.7. The figures in square boxes are the expected utilities at the appropriate point in the decision tree. We have already seen that the expected utilities for the first two options that the decision-maker considered were 0.85 and 0.5. To see that the third has an expected utility of 0.88, we start by considering the choice before the decision-maker at the decision node Q] . If he found himself in this position, should he sell his option to his unreliable associate, or should he keep his share of the investment? The expected values of these options are £1,700 and £1,960 respectively, so that, if the decision-maker were to use expected value as his criterion for choice, then he should not sell. This view is inconsistent with his expressed attitude towards risk, however, as reflected in his utility function. The expected utilities of the two options facing him are 0.91 and 0.89, showing a slight preference for selling to his associate. This reversal is caused by the convexity of the utility curve in Figure 3.6. Assuming that he will adopt the action with the largest expected utility, if he ever gets to [T| , it is proper to evaluate the utility of that branch of the decision tree as 0.91. On that basis, the expected utility is 0.1 x 0.83 + 0.1 x 0.67 + 0.8 x 0.91 = 0.88. This has a higher expected utility than doing nothing (0.85). Thus, if the decision-maker wishes to follow the precepts of decision theory, he should take the complicated gamble and, in the happy event of good news after a month, he should sell to his associate, unreliable though he may be. This process of evaluating a decision tree is often referred to as "folding back" for obvious reasons. In passing, we must point out that it is not proper to interpret differences in expected utility as measures of intensity of preference. In order to explore by how much one option is preferred to another, one possibility is to ask what monetary sum must be added to all the outcomes of the less attractive gamble to make the expected utilities of the two gambles equal. This sum will not be equal, in general, to the monetary amount that, in Figure 3.6, gives a utility value equal to the difference between the expected utilities of the two gambles being compared. Thus, in the example above, if £270 is added to the top branch
Decision theory
45
in Figure 3.5, the expected utility rises to 0.88, equal to the expected utility for the bottom branch. This is the proper measure for the extra value of the investment opportunity. The point of this digression is that a common error is to compute the difference in expected utility (which in this case would be 0.88 — 0.85 = 0.03) and find by how much this increment in utility represents an increase in monetary value, starting from the zero point. In this case this would give a figure of £51. We want to stress that such a calculation is meaningless! Note also that in considering the relative value of gambles we might have asked what monetary amount must be subtracted from all the outcomes of the more attractive gamble to make it only as attractive as the less attractive one. This will, in general, be different from the amount calculated by the procedure outlined above. This is because the attractiveness of gambles depends on the initial asset position as well as the amounts at stake. Exercise 5, in the exercises at the end of Section 3.3, explores this phenomenon. 3.3.3 Implications of utility theory In this section we shall describe some of the ideas that have resulted from the concept of utility. The first is "certainty equivalence." One of the key ideas in utility theory is the substitution principle; this is the supposition that, in the analysis of a decision problem, we can always substitute a gamble for a certain consequence, and vice versa. In a particular instance, the sum of money which a decision-maker is prepared to swap for a gamble is his "certainty equivalent" of the gamble. Clearly the utility of the certainty equivalent must be equal to the expected utility of the gamble; so folding back a decision tree could be interpreted as determining the certainty equivalent at each node in the tree. In our decision-maker's original problem (Figure 3.2), his certainty equivalent for the gamble was less than its expected value, which was £1,000. Indeed, since the expected utility of the gamble was 0.5, we can use Figure 3.6 to read off the certainty equivalent as — £854. His certainty equivalent is considerably smaller than the expected value; this often happens in practice, and the phenomenon is called risk aversion. The difference between the two is sometimes referred to as the risk premium; this is £1,000 — ( — £854) = £1,854 in our example. The risk premium is a measure of how much the decision-maker would be prepared for an expected value to be reduced, in order to eliminate uncertainty, for a given lottery. Risk aversion is associated with a convex utility function. A little thought should convince the reader that, if a utility function has a decreasing first derivative, then the owner of that utility function will be
46
Theory
risk-averse for any gamble he faces; that is, the certainty equivalent will always be less than the expected value. Although risk aversion is by far the most common attitude towards risk, there are some situations where the opposite preferences may be displayed. If the certainty equivalent exceeds the expected value, the owner of the utility function is displaying risk-seeking behavior. If a utility function is concave (i.e. its first derivative is increasing), then the person whose risk preferences are modeled by the utility function will be risk-seeking for all gambles he faces. If the utility function is neither convex nor concave over the whole range, his attitude towards risk will depend on which gamble he is considering. People who enjoy betting might be risk-seeking for gambles involving small amounts but risk-averse for uncertain options involving larger stakes. Finally, if a utility function is linear, we say that its owner is risk-neutral; in this case his certainty equivalent is equal to the expected value for any gamble that he faces, and so he need not compute utilities at all. It is sometimes useful to work with special analytic forms for the utility function. Since utility functions are elicited empirically, there is no reason why any special analytic form should emerge. But, in many other areas of the application of mathematics, empirical curves may often be well approximated by an analytic function, and by doing so analysis may be easier; and the same is true in utility theory. Perhaps the most often used form is the exponential u(x) = a — b exp( — ex).
This may not fit all situations, since it is bounded above and determined by only one parameter; but, apart from its advantage in statistical calculations, it has some interesting properties. If the decision-maker has an exponential utility function, then his certainty equivalent for increments in wealth for a gamble will not depend on his initial asset position, contrary to the general case. To see this, consider the gamble {s, t;p} involving increments to his initial assets, z. The expected utility of this gamble is p(a - b exp( - c(z + t))) + (1 - p)(a - b exp( - c(z + s))), which may be written as a — b exp( - cz)(p exp( - ct) + (1 - p) exp( - cs)). If the certainty equivalent for the gamble is z + e (so that the certainty equivalent expressed in increments to assets is e), then e must satisfy exp( — ce) = p exp( — ct) + (1 — p) exp( — cs). Since this equation does not involve z, our assertion is proved. Some decision analysts refer to this as the delta property, a usage derived from
Decision theory
47 Final asset position
£1,000
£0
£1,000 -f
Figure 3.8
Insuree's decision problem.
the practice of referring to increments in a quantity as a delta. It can be shown that the only utility functions to display the delta property are the exponential and linear ones. Another useful concept in utility analysis is that of the coefficient of risk aversion, defined as
the ratio of the second derivative of the utility function to the first. As may be easily seen, for the exponential utility function the coefficient of risk aversion is a constant, namely c. Moreover, if the coefficient of risk aversion is constant, then the utility function has to be exponential. When the coefficient of risk aversion is positive, then the owner of the utility function will be risk averse for gambles whose prizes lie in the region of positivity; in regions of negative coefficient of risk aversion, he is risk-seeking. Another phenomenon which has a rational explanation using utility theory is insurance. It is true that insurance companies may be able to accept a risk because of the statistical properties of bearing a large number of independent risks; that is, the coefficient of variation (standard deviation divided by the expectation) for their asset position will usually go down as the number of independent risks they bear goes up. It also makes sense, however, for individuals with a low coefficient of risk aversion to insure individuals with a high coefficient of risk aversion, even
48
Theory Increments in final assets €f
£f-1,000
£0
Figure 3.9 Insurer's decision problem. if no other risks are involved; both parties can be better off, as measured by their expected utilities, or certainty equivalents. Suppose that a decision-maker, whom we will call the insuree, is facing the gamble in Figure 3.8, namely {£0, £1,000; 0.95}; there is a one in twenty chance that he will suffer a loss of £1,000, wiping out his initial savings. He wishes to insure against the unlikely event that he ends up with nothing. Suppose that he has an exponential utility function, with coefficient of risk aversion equal to 0.001. Now imagine that a less risk-averse individual, the insurer, offers to insure the insuree against the risk he faces, for a premium £f. For what value of f should the insuree accept the insurance? The expected utility of the gamble is a - fo(0.95 exp( - 0.001 x 1,000) + 0.05 exp( - 0.001 x 0)) while the utility of accepting the insurance is a-b exp( - 0.001(1,000 - / ) ) . The insurance should be accepted if the latter expression is bigger than the former, and it is easy to check that this is so provided f < £82.42. A similar decision problem faces the insurer, and it is displayed in Figure 3.9. For simplicity let us suppose that the insurer also has an exponential utility function, so that his initial asset position is irrelevant to the question of whether he should offer insurance. Let us take his coefficient of risk aversion to be 0.0001, that is, he is ten times less averse
Decision theory
49 Location
Figure 3.10
Utilities
Paris
0.85
London
0.00
Chicago
0.40
San Francisco
0.75
Manchester
0.15
Tokyo
0.05
Honolulu
1.00
Vienna
0.25
A job choice problem. Computed expected utilities are in boxes.
to risk than the insuree. A calculation similar to that above shows that the insurer should offer insurance so long as f > £52.45. Thus, so long as the premium is in the range (£52.45, £82.42), both insuree and insurer are better off entering into an insurance contract. The reader should not imagine that we have produced a descriptive theory of insurance here. All we have shown is that if both insuree and insurer wish to follow the axioms of decision theory, and they are able to elicit their utility functions, then there are circumstances where both can improve their position by the insurance contract. So far in this section, the discussion has solely discussed utility for money. The idea of utility is not limited to money, however; we can use the von Neumann-Morgenstern (1944) definition to construct utility for other variables, and even for quantities for which there is no ^natural numeraire. Consider, for example, the decision problem of choosing a job. To simplify the discussion, suppose that the only thing the decision-maker cares about in this choice is job location, and that each job he may take
5
Theory
may offer him what is essentially a gamble over location, as indicated in Figure 3.10. Suppose that, of the possible locations in Figure 3.10, the decisionmaker believes London to be the least desirable and Honolulu the most attractive. To define utilities we ask him to consider a series of hypothetical gambles. For example, consider the options {London, Honolulu; /?} and
Paris.
What must the value of p be for the decision-maker to be indifferent between them? Since he prefers Paris to London, and in turn Honolulu to Paris, there should be some value of p such that indifference is obtained. This value is defined to be the utility of Paris (not in some absolute sense, of course; just for our particular decision-maker in this particular decision problem!) Imagine that by asking a series of questions of this type, the following utilities are obtained: w(London) 0.00 w(Tokyo) 0.05 w(Manchester) 0.15 w(Vienna) 0.25 w(Chicago) 0.40 «(San Francisco) 0.75 w(Paris) 0.85 w(Honolulu) 1.00 The decision tree in Figure 3.10 can now be folded back as before, giving the expected utilities in the boxes on the initial branches. If he wishes to act according to his expressed attitudes towards risk (as given by the indifference judgments which led to his utilities), then he should choose the second job, which gives him a fairly high chance of being posted to San Francisco. Several problems with using this theory will no doubt have occurred to the reader already, and we shall address some difficulties in the next section; we have shown here, however, that a rigorous framework does exist for the numerical representation of attitudes towards risk, even when what is important cannot be simply represented in monetary terms. The second half of this book describes how these ideas have been used with success in practical contexts. 3.3.4 Criticisms of utility theory The ideas of von Neumann and Morgenstern's (1944) utility theory have proved far more useful than the other, more major topics addressed in their book. Nonetheless, utility theory is not universally accepted. In this section we will briefly outline, and discuss the validity of, some of the
Decision theory £1,000,000
£5,000,000 0.1,
£5,000,000
£1,000,000
£1,000,000
Problem 1 Figure 3.11
Problem 2
Allais's paradox.
criticisms that have been leveled against using expected-utility theory as a guide for decision-making under uncertainty; the reader will, however, be able to make his own judgment in the light of the practical examples reported in Chapters 6 and 8. One of the earliest criticisms, and now a notorious one, was due to Allais (1953). We shall now describe what has come to be known as Allais's paradox. Before reading further, readers are recommended to consider the two decision problems in Figure 3.11, and decide whether they would take A or B in problem 1, and C or D in problem 2. Most people choose A over B, and C over D; the argument, presumably, is that in problem 1, £1,000,000 is so attractive that they wish to avoid the very small chance of gaining nothing, while in problem 2, since the probabilities are so close, it makes sense to go for the gamble with the bigger prize. The problem is that these judgments are inconsistent with expected utility theory. To see this, let us define a utility function with #(5) = I ; M ( I ) = u;u(o) = o, in units of £1,000,000. Remember that we can fix any two points on the utility scale arbitrarily. Then the judgment that A is more attractive than £, interpreted as expected utility, implies u > 0.1 x 1 + 0.89 x u + 0.01 x 0
52
Theory
or that u > 10/11. On the other hand, the judgment that C is better than D implies that 0.1 x 1 + 0.9 x 0 > 0.11 x « + 0.89 x 0 or that u < 10/11! Raiffa (1968, pp.80-6) provides an interesting discussion of the implications of this paradox. He gives two arguments why people might wish to reverse their preferences, and an argument that brings in another attribute, the presence of which makes the judgments consistent with expected utility theory. Our view of this paradox is slightly different. There is no moral imperative to adopt decision theory. Decision-makers are free to make their decisions in whatever way takes their fancy. The theory exists as a guide for those who wish to abide by its precepts, because they see them as providing a sensible framework for thinking through complex decision problems involving uncertainty. So if, after the inconsistencies of a set of judgments are pointed out to a decision-maker, he refuses to change any of them, the matter must rest. Rationality is defined in terms of adherence to a set of rules; there is no reason why a decision-maker ought to abide by the rules of decision theory, or indeed any rules for that matter! More recently there have been some more trenchant criticisms of the methods of decision analysis, particularly of the ideas of subjective probability and utility theory. Tocher (1977) is a good example of these. He argues that we do not obey the axioms of subjective probability, so that it has to be a normative, rather than a prescriptive, theory. With this we agree; he then goes on to say that proponents of subjective probability accuse others of irrationality if they are not prepared to attempt to make their judgments conform to the principles of subjective probability. Here he is too sensitive, for, if rationality is defined as conforming to a particular set of behavioral axioms, then a failure to do so is irrational by definition. What Tocher should be criticizing is an arrogant assertion by some theorists that their rules of behavior, defining what it is to be rational, are the only ones. As we argued in Chapter 2, we believe that each person is free to choose these rules as he or she thinks fit. If he or she wishes not to behave in conformity with the axioms of subjective probability, so be it. We must point out, however, that other frameworks for supporting decision-making in a consistent manner are quite scarce! Tocher has a further criticism of utility theory which has more practical import. He points out that to apply the theory a utility function has to be
Decision theory
53
elicited, and to do that the decision-maker has to ask himself hypothetical questions about his choice between gambles which he may never face in actuality. As Tocher says: Such experiments in the mind offer no guarantee as to the subject's reaction when faced with a similar real choice; we all know from bitter experience that men do not act as they say that they will. Tocher 1977 We will return in Chapter 4 to discuss what is known about man's ability to make such judgments. It will be sufficient here to reiterate that one is not bound to follow the procedures of decision analysis. We see them rather as a sensible set of ideas to structure, in a consistent and comprehensible way, a complex and uncertain problem. We use a utility function not because we are convinced that it is the "correct" one, but because it represents our risk preferences at the time we are making the decision as well as we can determine; the adoption of a utility function is the adoption of a basic set of attitudes, in conformity to which we wish to act, and from which we can use the precepts of decision analysis to determine a consistent set of judgments. Still more recently, Dreyfus (1984) has argued that the analytical philosophy of decision analysis, which breaks a decision problem down into small judgmental tasks, such as eliciting separate probabilities and utilities, is at variance with the holistic approach to problems taken by most experienced decision-makers. He argues that decision analysis has not caught on as some were predicting twenty or thirty years ago because real decision-makers are typically not interested in the analytical approach (and here he echoes another point made by Tocher). In an interesting reply, Brown (1984) argues that expected utility theory can be, and has been, of use. Brown has also been an author of two interesting reviews of the use of decision analysis (Brown 1970, Ulvila and Brown 1982) which show that, despite these important criticisms of subjective probability and utility theory, there are many people, in many different contexts, who have used the ideas effectively. 3.3.5 The value of information Some decision analysts argue that one of the most valuable insights provided by expected utility theory is its ability to handle the value that some potential piece of information might have; we will discuss this notion in this section. Faced with a difficult decision involving uncertainty, many a decision-maker will seek to reduce the uncertainty before making the decision. Very often, however, getting the information will be a costly and time-consuming business; so before collecting the information it would be well to assess whether the cost is worth the value gained.
Theory
54
Final asset position (£) 2,800
Keep investment
1,800
800
Buy information about success
2,800 -p Sell investment 0.9/
Associate pays
1,800-p
Associate ~p
does not pay
0-p Keep investment Associate
Sell
\
investment
0.9/
—pays Associate does not pay
1 800
-
-"
800 -p
Figure 3.12 Decision tree to compute the value of perfect information. refers to the node with this label in Figure 3.5.
0
We once more consider the example of Figure 3.5. There are several uncertainties in this problem, but let us concentrate for the moment on whether the investment will succeed, supposing it gets beyond the first stage. Imagine that it is possible to discover whether or not the-investment will succeed, prior to deciding about selling the option, but at a price. How much is it worth paying for this information? The situation is represented by the decision tree of Figure 3.12.
Decision theory
55
Note that Figure 3.12 differs from Figure 3.5. by moving the chance node for investment success from after the final decision to before it. p is the price at which the information is available to the decision-maker and it is conventional to represent the fact that an extra payment is made if information is bought by a gate across the appropriate path in the decision tree, as in Figure 3.1 z. Presumably the probabilities that the information will imply success or failure are the same as those that were assessed before we started talking about information, as indicated in thefigure.It is easy to see, by folding back the tree, that the expected utility from buying information at a price p is 0.7w(2800 - p) + 0.3(0.9^(1800 - p) + 0.1w(800 - p)) and the decision-maker should buy this information so long as this expression exceeds 0.91, the expected utility of selling the investment, the best course of action without further information. We define the value of perfect information here to be equal to the value of p which insures equality. Computing this value otp is quite difficult, since the function u is defined by the empirical curve in Figure 3.6. Even if we find a suitable analytic approximation for this curve the solution may not be straightforward, since the equation may have to be solved numerically. This is merely a computational problem, however; the reader may check that, when p is about £710, the expression above is about 0.91. (We have, in fact, calculated this value by using an analytical form: the utility curve in Figure 3.6 is very close to an exponential, with coefficient of risk aversion equal to 0.00058. If readers wish to follow the details of the calculations in the rest of this chapter, we suggest that they use this analytic form.) We now argue that it is not worth paying more than £710 to learn in advance about how our uncertainties will be resolved. This concept is sometimes referred to as the value of clairvoyance; but we prefer to call it the value of perfect information, to contrast with the value of imperfect information, which we discuss below. To be more formal, we say that the value of perfect information about some uncertain event is equal to the maximum amount that is worth paying for the uncertainty to be resolved prior to making the relevant decision; it is that sum which, when deducted from all possible outcomes on the decision branch "buy information," reduces the expected utility of that branch to be equal to the next highest expected utility on other branches. There are some circumstances where the mere realization that an upper limit exists to the amount worth paying for information may cause a decision-maker to think very carefully before undertaking an expensive information-gathering exercise. The value of perfect information may be zero, of course. Consider, in Figure 3.12, that information may be available about whether the
Theory
Final asset position (£) Buy information about success
Buy information about associate
Investment will fail
Buy information about associate
800 - p - p '
Figure 3.13 When the value of perfect information is zero. It is not worth buying information about the associate, since this information will not change the decisions. [A\ refers to the node with this label in Figure. 3.5.
Decision theory Success
57 Final assets (£) 2,800
0-7
Keep investment
does not pay
800 -p
Figure 3.14 Value of imperfect information. The consultant advises the decisionmaker on whether the investment will succeed. unreliable associate will pay up, but after information about the other uncertainty is resolved. This situation is represented in Figure 3.13. It is easy to see that in this case the value of information about the associate is zero, no matter what the state of knowledge is about the business venture. On folding back the tree of Figure 3.13, we see that, no matter how small the fee paid for this information, it is not optimal to buy it; and this is true on both branches of the tree. The reason for this is that the information will not change our preferred decision; if the venture will be successful then we will not sell it, no matter if our associate turns out to be reliable, and if it will be unsuccessful, then we will sell, even if he does not-pay in the end. Information value is quite a subtle concept; it is not something which can be permanently attached to an item of information, since value
58
Theory
depends very much on context. If, for example, we were to ask w h a t the value of perfect information about the reliability of the associate would be in the absence of information about the outcome of the business venture, then the answer is that it is positive and equal to about £ 8 9 . The reader might like to check this by constructing a decision tree similar to that of Figure 3.12 and folding it back. In practice, perfect information is very rarely available. When, for example, the managers in a company are involved in market research to estimate h o w well a new product will sell, their uncertainties are, typically, reduced rather than removed altogether. This imperfect kind of information can also be evaluated using expected utility theory, although the considerations are a little more involved. To illustrate these ideas, imagine that instead of perfect information being available, as in Figure 3.12, we have the option of buying advice from a consultant. He will give us his opinion on whether the investment will succeed or not. In the light of that advice we then have to decide whether to sell or not. If the consultant were totally reliable, of course, then we would be in receipt of perfect information, and the analysis above would apply. It is more probable, however, that all that the consultant does is to revise our subjective probability of success. The situation is set out in the decision tree which is partially represented in Figure 3.14. Before folding back the tree to compute the m a x i m u m a m o u n t of money worth paying to the consultant, the decision-maker has to determine w h a t probabilities have to be applied to the branches at each chance node. Perhaps the most obvious probabilities to assess first are the ones that have to be applied at the end nodes, namely the conditional probabilities that the investment will succeed, conditional, that is, on what the consultant says. Let us suppose that the consultant is quite reliable, so that we assess the probability of success, given that he predicts success, at 0.8. Suppose also that the probability of failure, given the consultant's prediction of failure, is 0.7. The only probability missing from the tree is n o w that for w h a t the consultant will say! If we are to be coherent, however, the probability that the investment succeeds, which we have previously assessed at 0.7, must also be calculated as equal to 0.7 when it is computed through the appropriate conditional probabilities. Thus to obey the rules of the probability calculus, that is, to be coherent, we must adjust our probabilities so that: Pr(S) = Pr(S I s) Pr(s) + Pr(S \f) Pr(/) where S stands for the success of the investment, and s, f sta-nd for the prediction of success and failure, respectively, by the consultant. It is easy to see that the only value of Pr(s) consistent with those probabilities that have already been assessed (namely Pr(S) = 0.7, Pr(5 | s) = 0.8,
Decision theory
59
Pr(S I f) — 0.3) is 0.8. This is the probability that we have associated with the consultant's prediction of success in the decision tree. Note that this particular sequence for eliciting probabilities is just one among many. In practice it might be necessary to get at the appropriate probabilities in many different ways, and to overspecify them. We discuss the practical elicitation of probabilities in Section 7.4; all we need to stress here is that it is important for all the probabilities which enter a decision tree to be consistent with each other. Now we can find the value of p in Figure 3.14 which reduces the expected utility of the bottom branch to that of the second branch. It is equal to £82, as the reader may check, by folding back the decision tree with this value of p. This is the value of imperfect information in this case. Not surprisingly, the value of the consultant's information is less than the value of perfect information. This will always be the case, although it is quite tricky to prove it in general. Thus the value of perfect information is an upper bound on the value of imperfect information; and if, as is usually the case, the computations required in the case of imperfect information are complex and difficult, a calculation of the value of perfect information may be sufficient to determine whether a proposed information-gathering exercise is worthwhile. EXERCISES FOR
3.3
1. Clare and Adrian, students on a course in decision analysis, take part in some electronic games. The first game involves flipping an electronic coin which has been set to come up heads with probability 0.7 and tails with probability 0.3. Players win £5 on heads, but lose £5 on tails. The second game uses the same electronic coin, but this time, if it lands heads, the player flips a second electronic coin, with a probability of heads equal to 0.2. If this second coin lands heads, the player wins £15; if tails, he or she wins £5. If the first coin lands tails, a second coin is flipped with a tails probability of 0.3. If this coin lands tails, the player loses £15; if heads, the loss is £5. Clare and Adrian may each have one play at one game or the other, but not both. (a) Draw out the decision tree that faces Clare and Adrian. (b) Clare is risk-neutral for the range of prizes offered, but Adrian is risk-averse, and he estimates his utility function for gains or losses to be: u(x) = - 0.0008 x2 + 0.033 x + 0.68. Which game maximizes expected utility for (i) Clare (ii) Adrian?
60
Theory (c) What is the value of perfect information, (i) for Clare (ii) for Adrian, on the outcome of the flip of the coin in the first game? Suppose that information is available, at a price, on the outcomes of the flips of both possible second coins in the second game. What is the value of this information for Clare? (d) Suppose that Clare can perform a test which yields partial information on the outcome of the coin flip in the first game. Analysis of this test shows that the first coin will come up heads with probability 0.9 when the test says "heads will occur," and that the coin will land tails with probability 0.7 when the test says "tails will occur." How much is this test worth to Clare?
2.
A decision-maker's utility function, w, for reward, x, is given by u(x) = a — be~x/5, where a and b are positive constants. The decision-maker is faced with a choice of doing nothing with reward zero, or undertaking one of two gambles, A or B. Depending on which of three events X, Y, or Z occurs, gamble A will yield rewards of — 5, 10, or 15 respectively, and gamble B will yield rewards of — 10, 20 or 30 respectively. One and only one of the events X, Y, and Z is bound to occur and event Y is thought to be twice as likely as event X. Calculate the ranges of values of p, the probability of event X, for which each decision alternative is optimal.
3.
Show that the utility function for money given by ln(x + 35)/ln(35) — 1 represents decreasing risk aversion (i.e. the coefficient of risk aversion is a decreasing function of x). A decisionmaker with this utility function has assets of £50 and realizes that £25 of it is in a risky situation which has one chance in ten of collapsing and losing him his money. What is a reasonable premium for him to pay for insurance against the loss? What would the premium be if his assets were £100?
4.
An insurance company has assets of £40111 (40 million pounds), and its utility is given by w(x) = ln(x + 35)^(35) — 1, where x is measured in millions of pounds. It is asked to insure against an earthquake disaster of £40111 and assesses the chance of the earthquake at 0.05. What is a reasonable premium for it to request? (Remember you are looking at the situation from the company's side, not from that of the earthquake sufferers.) Administrative expenses can be ignored. The company finds a second company with the same assets and utility function and agrees to share the risk. That is, it insures against a loss of £2om with probability 0.05. What is the premium now? Show that even twice this premium is less than the original premium
Decision theory
61
so that the insurance is improved from everybody's point of view by the companies sharing the risk. An individual's current wealth is £x. He is offered the opportunity to invest part of this wealth, an amount £s, in a venture which may yield an immediate benefit of £r where r > s. There is, however, the possibility that the venture will fail, yielding no benefit. Describe the concepts of utility and subjective probability, and outline in this particular case how they might help the individual to decide whether or not to invest. If the individual's utility function for wealth £y is ln{y) and if he considers that the venture has an equal chance of success or failure, show that he should invest if: 6. By choosing appropriate analytic forms for the utility functions, show that the buying and selling prices for afifty-fiftychance of winning £20 are the same for a decision-maker with a constant risk aversion, but differ if he has decreasing risk aversion. Why is this? 7. An entrepreneur feels that there is an even chance that an event X will occur and that a completely independent event Yhas a probability 0.4 of occurring. He owns a business venture which will yield a profit of £100,000 if X occurs but nothing otherwise. He is offered in exchange for ownership of this venture an alternative business opportunity which will yield £80,000 if Y occurs and £30,000 if it does not. He has linear utility for money. Perfect information may be purchased about X and Y. Show that the value of perfect information about Y, once it has become known whether X has occurred, is independent of this information about X, but that the value of perfect information about X, once it becomes known whether Y has occurred, does depend on whether or not Y has occurred. The entrepreneur can buy perfect information about X for £30,000 and about Y for £11,000. He can buy in any order, or not buy at all. What should he do? 8. An investor can speculate on a stock by buying long or by selling short. Alternatively he can make no investment. The investor makes a good investment if he buys long and the stock price rises, or if he sells short and the stock price falls; by doing this he makes a profit/) f £k. If he makes a bad investment he will lose £k. Initially, he thinks the price is equally likely to rise or fall. His utility for a monetary increment of x is u(x).
6z
Theory A consultant offers to predict how the stock price will move. The investor believes that the consultant has a probability of p {p > 0.5) of correctly predicting whether the stock price will rise or fall. Find an expression for f, the value of imperfect information provided by the consultant. Show that if the investor has a constant coefficient of risk aversion c, then the value of imperfect information is: f = - (1/c) ln[min{l, pe~ck + (1 - p) ec*}]. Check that this formula gives the correct value of /", both when the consultant is completely informative and when he is completely uninformative.
3.4 RISK PREFERENCE AND CONFLICTING OBJECTIVES
3.4.1
Introduction
So far we have seen how to construct a calculus for evaluating decision options when the decision-maker has conflicting objectives and no uncertainty; and also a single objective but considerable uncertainty. In reality decision problems usually involve both. For example, the car choice problem in Section 3.1, while perhaps being satisfactorily approximated by ignoring uncertainty, may well involve uncertainty about how the cars will in fact perform once they have been purchased. In choosing a job, as we discussed in the last section, a job-seeker will probably want to include more attributes than simply the location of possible employments. Even problems which may at first sight appear to be single-attributed may turn out to be better formulated as a multi-attribute problem. Consider the managers of a company, contemplating a new project. There may well be considerable uncertainty about the financial outcome of the project. What will the costs be? Will the cash flow avoid excessive bank loans? — and so on. It would appear that all the outcomes could be described in terms of a single variable, namely money; so, by using discounting to reduce a potential future cash flow to an equivalent present value, the single-attribute theory developed in the last section could be applied directly. Often this will be a sensible approach to decision-aiding for commercial organizations, but sometimes it will be useful to represent the problem as a multi-attribute one. For example, market share, company image or employee morale may be better expressed as separate variables, rather than attempting to find monetary equivalents for them at the beginning of the analysis. Case studies in sections 6.5.3, 8.3.3, 8.4.3
Decision theory
63
and 8.6.2 of this book, as well as the case reported in Phillips (1982), illustrate such multi-attribute problems. We clearly need to be able to handle uncertainty and conflicting attributes at the same time; this section is devoted to discussing how this might be done. Our first observation is that the theory is straightforward for extending the single-attribute utility theory, developed in the last section, to the multi-attribute case. Suppose that a set of attributes, Xi, . . ., Xw, has been identified, and that all conceivable outcomes of the variable x, are less desirable than xj and more desirable than xf; i.e. [xf, xj] includes all possible outcome values. We shall write the vector (x\,. . ., x*n) as x*, and similarly (JC?, . . ., x%) as x°. Then the utility of a set of outcome values x will be defined as equal to the probability p that makes the decision-maker indifferent between x and
{x°, x*;p}.
With this definition of utility, the theory may be developed exactly as before, leading to the recommendation that expected utility should be maximized. The chief difficulty with this approach is its practicability, however. While there are non-trivial problems in determining satisfactory singleattribute utility functions, as we shall discuss in Section 7.5.2, these are magnified when dealing with multiple attributes. Part of the power of decision analysis lies in the replacement of complex judgmental tasks by simpler ones; determining the value p in the expression above is by no means simple. Several different approaches have been suggested for coping with this difficulty and we will describe some of them in the next sections. 3.4.2 Keeney's approach The assessment of a multi-attribute utility function would be much simplified if it could be written as some simple function of single-attribute utility functions; then we could assess the single-attribute functions without reference to the other attributes, and so reduce the complexity of the utility assessment task to manageable proportions. An additive form is perhaps the most simple, and the search for conditions that would justify using the additive form was an active research area for some time. This work is well described in the seminal text by Keeney and Raiffa (1976), which has now become a standard reference work; the reader is recommended to consult it for a fuller account of the theory we present here. Keeney's theorem, which we will now state, gives sufficient conditions for a multi-attribute utility function to have a simple form which is slightly more general than the additive one but which gives additivity in a special
64
Theory
case. The conditions Keeney needs are preference independence, which we defined in Section 3.1, and utility independence. An attribute X is said to be utility independent of other attributes Y, Z, . . . if the decision-maker's preferences amongst gambles on X do not depend on the actual levels of the other attributes. We consider first the case of three attributes only. Suppose that: (a) X and Y are preference independent of Z; (b) X and Z are preference independent of Y; (c) X is utility independent of Y and Z. Then Keeney's theorem proves that the utility function for the attributes X, Y, Z, namely u{x, y, z), must be such that: (1 + ku(x, y, z)) = (1 + kkxux(x))(l
+ kkyuy(y))(l + kkzuz(z))
where k, kx, ky and kz are constants. The proof of this result is given in Section 6.3 of Keeney and Raiffa (1976). What is more, each of the utility functions is normalized so that
u(x°,yoyz°) = 0; u(x*,y*,z*) = 1; ux(x°) = 0; ux(x*) = 1; and similar expressions hold for the other single-attribute functions. We can now use this result to assess a three-attribute utility function, in certain cases, namely those where the independence conditions hold. First we have to check that the decision-maker's preference structure does indeed display these independence properties. Keeney and Raiffa (1976, Section 6.6.1) give some suggestions of how this might be done, largely involving asking the decision-maker whether his judgments of equivalence between many hypothetical options are indeed independent of the other attributes as required. Examples of these questions are presented in Section 6.3.2. Determining whether the independence conditions really hold is a difficult and time-consuming business, and this is one of the many areas where Keeney's approach has been criticized (we shall discuss others later in this section). In the spirit of interpreting decision theory which we have adopted throughout this chapter, however, we argue that analysis might create a preference structure, rather than simply reveal it, at least in part; thus, if the decision-maker is satisfied that, as far as he can tell, he does no injustice to his preferences by assuming that the independence conditions hold, he will not be led astray by assuming this in his analvsis. On the assumption that they do hold, the utility function must take the multiplicative form given above. Moreover, by setting all three variables to their upper limits in that equation, and recalling the normalization conditions above, we see that: 1 + k = (1 + kkX)(l + kky)(l + kkz)
(3.2)
Decision theory
65
By setting x to its upper value, and y and z to their lowest, we get: u(x\ y°, z°) = xk with similar expressions for the other two variables. It is not easy to avoid at least some of the difficult judgments from which we are trying to escape in setting up this theory. Keeney and Raiffa (1976, Section 6.6.6) suggest that the scaling constants may be determined by direct assessment of the appropriate utility. That is, to determine kx we ask the decision-maker what the value of p is such that he is indifferent between (x\y°,z°)
and
{(x°, y°, z°), (**, **); y\ p).
By the definition of utility and the expression above we conclude that kx = p. On deriving each of the scaling constants in this way, they may be substituted in Equation (3.2) above to give a cubic equation for k. The condition that the utility function must lie between zero and unity will determine which root of this equation to choose. Note that if kx + ky + kz = 1, then k = o, and the utility function reverts to the additive form. This theorem may be generalized to arbitrarily many dimensions, and then it can be used to assess utility functions in these more general cases. Considerable prominence has been given to this approach for the multiple attribute problem over the last ten years, and there have been many reports of its use on practical problems. (See Kirkwood 1982 or Bell 1977 as just two among many examples.) The number of practitioners of decision analysis making these applications has remained small, however, and some significant criticisms can be made of this approach. First, to carry it out properly, considerable effort and time is required, both from the analyst and from the decision-maker, and in many contexts this is not available. Secondly, there are some practitioners who use the method to construct a multi-attribute value (rather than utility) function for assessing options in the situations where there is no uncertainty. (Observe that any multi-attribute utility function will also be a value function, but that the reverse is not true.) This may seem a surprising procedure to adopt, since it involves determining risk preferences when they need not be revealed for the decision problem to be solved. It seems very strange to elicit a value function to aid decision-making in the absence of uncertainty by asking somebody questions about his attitudes towards gambles. Finally, other methods exist for establishing satisfactory utility functions which are simpler to use. We go on to discuss these now. Further discussion of this approach is given in Section 6.3. 3.4.3 The Stanford approach The application of decision analysis, in common with other practical subjects, such as psychoanalysis or architecture, has a markedly different
66
Theory
flavor in different schools of practice. The problem of constructing multi-attribute utility functions which properly reflect risk preference occurs whenever decision theory is applied to a practical problem. Yet it appears that the Stanford school, in which the key figure has been R.A. Howard, has not perceived the need to follow the procedures of the last section, which can be associated with the Harvard school, for which the central figures have been H. Raiffa and R. L. Keeney. The procedures followed by the Stanford school can be best seen in the series of readings published by Howard and Matheson (1984). In modeling multi-attribute problems, the Stanford school recommends first constructing a multi-attribute value function, as we described in Section 3.1. Since value is a single attribute, it must be that U{XU X 2 , . . ., Xn) = U'(V{XU X 2 , • • ., X n))
where u'( . ) is a single-attribute utility function and v( . ) is the value function constructed. Thus the problem offindinga multi-attribute utility function is reduced to two perhaps more tractable problems; first a value function is constructed, and then we find the utility of that value function by normal single-attribute methods. This is consistent with the recommendation of the Stanford school that any decision analysis should pass through a deterministic phase, where a model of the problem is constructed without reference to uncertainty, before the probabilistic phase, when uncertainty and risk aversion are explicitly considered. This approach certainly avoids the criticism we mentioned above, namely, the inappropriateness of asking gamble questions in the absence of uncertainty. It is in fact not necessary to construct a complete value function for this method to work. Suppose that instead of constructing value functions with a particularly simple form, as we did in Section 3.1, we trace the curves of constant preference (the isoquants) to one of the boundaries. That is, for any point x, we find the value of x\9 namely X\{x), such that:
This procedure is termed pricing-out, since the variable chosen is usually money. Now we may construct a single-attribute utility function for Xi, conditional on X2,. . ., Xn all having their lowest values. There are problems with this approach, however. Sometimes it is difficult to carry out the pricing-out properly. As Keeney and Raiffa (1976, Section 3.8) point out, it is only proper to price out each variable separately if certain conditions on preference independence hold. Moreover, in some cases it may not be a very easy set of judgments to make. Since one of the chief purposes of decision analysis is to replace difficult
Decision theory
6j
judgments by easier ones, we should be careful that our procedures do, in fact, satisfy this condition. 3.4.4 Comments We have seen two distinct approaches to constructing a multi-attribute utility function, but this does not exhaust the possibilities. A more detailed procedure, for example, is given in Keelin (1981). Moreover, it is possible to carry out utility elicitation procedures which simplify the assessment task, even if the preference structure does not satisfy the required independence conditions for either of the two methods above. For example, Bell (1979a, b) explores some ways to use conditional utility functions. We have not seen reports of the use of these methods on practical problems yet, and it may be that they are too complicated to be used in any but the most sophisticated analyses. Summarizing then, we see that there is a theoretically sound definition for the utility of multi-attribute consequences which may be used in the maximization of expected utility. It is difficult to assess this directly, however, and various different approaches exist to assist a decision-maker in indirect assessment. The approach adopted may be more or less sophisticated, and the analyst will need to exercise judgment here, as in so many other parts of decision synthesis, on the proper balance between accuracy and the use of resources. EXERCISES FOR
3.4
1. In making decisions about this company's future, a manager is concerned about three conflicting attributes: cash flow, safety, and company image. The manager's value structure is such that cash flow is utility independent of safety and company image, cash flow and safety are preference independent of company image, and cash flow and company image are preference independent of safety. Company image and safety are measured on a o— ioo scale, and the manager's utility for these attributes is linear with respect to these measures. He is risk-averse to cashflow,however, with a constant coefficient of risk aversion of (lnc2)/ioo (K£)~a. He assesses the following combinations of attributes to be equally desirable: Cash flow (K£)
Safety
Company image
IOO
0
0
0
IOO
0
0
0
IOO
He also judges that a gamble giving a one-third chance of (ioo, 100, 100), and a two-thirds chance of (o, o, o) is just as desirable as
68
Theory (ioo, o, o) with certainty. Derive a von Neumann—Morgenstern utility function for the manager with respect to these attributes. The manager has to choose between two options; thefirstproduces an uncertain cash flow, whose probability distribution is negative exponential with mean 80 K£, a safety score of 80, and a company image of 20; the second again produces a negative exponentially distributed cash flow, with mean 70 K£, but in this case the safety score is 45 and the company image is 60. Which option should the manager take, and why? (K£ = £1,000)
2. A decision-maker wishes to construct her multi-attribute utility function for choosing her job. She identifies three attributes which will adequately describe the options available to her: initial salary, measured in K£ per year; salary after ten years, also measured in K£ per year; and challenge of the job, which she measures on a o—10 subjective scale. She believes that her preferences for job challenge are utility independent of the other two attributes. What is now needed to insure that her utility function is additive? She assumes that these conditions are satisfied, and goes on to make the following assessments of indifference judgments: [(9, 15, 10); (5, 9, o); 0.3] ~ (9, 9, o) [(9, 15, 10); (5, 9, o); 0.5] ~ (5, 15, o) [(9, 15, 10); (5, 9, o); 0.4] ~ (5, 9, 10) where {a,fo,c) means the values for the three attributes above, in that order, for a particular outcome, [(a, b, c); (d, e, /); p] means an uncertain outcome giving a probability p of getting (a, b, c), and a probability 1— p of getting {d> e, /). The symbol ~ indicates indifference. Is the decision-maker justified in assuming that her utility function is additive? 3.5 OTHER NORMATIVE APPROACHES
3.5.1 Introduction We have so far described an approach to handling decision problems which is based on reasonable behavioral postulates; we have constantly stressed, however, that there is no obligation for any decision-maker to follow this approach. It is not the case, in our view, that a man is necessarily committing a folly if he fails to follow the precepts of classical decision theory, since he may well not be willing to accept that he can make the preference judgments that the axiomatic development of decision theory requires. Moreover, arguments that show that he will
Decision theory
69
inevitably lose in the long run if he fails to conform to the axioms (such as the Dutch book or money pump arguments for subjective probability see Lindley (1973)) will rarely be supported by practical instances of their occurrence. Most people will modify their judgments, or, more likely, refuse to make them, before unacceptable loss appears. We do not take an imperialist view of decision theory, then, as some advocates do. Rather, we see the framework as a sensible one to adopt when confronting uncertainty and conflicting objectives, and our experience is that many decision-makers agree with us, as the case studies mentioned in Part II of this text testify. We feel it proper, therefore, to describe several alternative normative frameworks that have appeared in the literature of the last thirty years, particularly for attacking the problem of conflicting objectives. Some of these methods are closely related to the decision theory we have described in this chapter, while others stand more on their own. Although there are numerous versions of these techniques, we will outline justfiveof the most prominent of them in this section, in each case giving references where the interested reader can further study the ideas. Some readers may think that the distinctions we make between these methods are not sharp, since there are close links between some of them and with classical decision theory. The principal difference between them, however, is that each procedure is championed by one or more key academics, or by distinct academic traditions. After our brief description of each method, we will discuss, in Section 3.5.7, what aspects of each method are worth considering for use in a particular situation. 3.5.2 Cost-benefit analysis When looking for a technique to compare the positive with the negative effect of possible courses of action, many people will say that what is needed is a cost—benefit analysis. In a general use of this term, this is indeed true; what is the search for a multi-attribute value function if it is not an attempt to balance costs with benefits? Nonetheless, cost—benefit analysis (CBA), as developed in economic theory, is, from our viewpoint, a rather specialized technique, designed specifically for the economic appraisal of social projects, rather than as a general decision-aiding procedure. In carrying out a cost-benefit analysis, however, an implicit construction of a multi-attribute value function takes place, and so in this section we give a very brief summary of CBA, showing how it differs from more general methods. In applying CBA to determine whether a project should be undertaken or not, all the factors of importance are identified, outcomes are measured for each factor, a price is determined for each factor, and all the resulting
jo
Theory
costs are added up. In so doing, an additive multi-attribute value function is constructed. Prices may be interpreted as weights, and then the net cost is simply a weighted sum. At first sight, therefore, there may appear to be little to distinguish CBA from the techniques we have been describing here. There are, however, several important differences in the way numbers are computed for entry into the value function, and the meaning of the answers is interpreted very differently. (See Watson (1981) for a discussion of this.) The most important of these is the philosophical justification of the techniques. The goal of CBA, as developed in economic theory, is to provide an objective measurement of the economic effects of a project. The question to be answered is whether society as a whole will be better off as a result of undertaking the project. Our goal in this chapter, however, has been the very different one of providing a framework for an individual decision-maker to tie together his judgments in a consistent way. As we shall see in Chapter 5, these goals merge to some extent when decision analysis is applied to public policy issues. Despite this, they still remain distinct. Other important distinctions include the handling of equity and the assumption of additivity. Since CBA is predicated on an assumption that net social benefit should be maximized, if one man gains more (in money terms) than another man loses by a transaction involving only the two of them, then the value function of CBA will approve of the transaction, even though equity considerations might render the transaction ethically unacceptable. Multi-attribute value theory is broader in allowing equity to be explicitly considered, by, for example, a measure of inequity featuring as an attribute in the value function. There is a large literature on CBA, and criticisms such as these have not gone unanswered by CBA theorists. Readers interested in studying CBA further are recommended to consult some of the standard texts, such as Layard (1972), Sassone and Schaffer (1978) or Sugden and Williams (1978). Discussions on the limitation of the ideas in practice are also important; a good critical review is given in Fischhoff et al. (1981). In summary, then, CBA has been developed as a tool for the objective analysis of the social desirability of public projects, measuring the relative importance of different factors by prices in the market-place. It was not designed as an aid for an individual decision-maker trying to weigh up his options against the conflicting objectives that he faces. The other four techniques we shall look at, on the contrary, do have this goal. 3.5.3 The multi-criterion decision-making movement A group of researchers, principally in the United States, has developed what has come to be known as the multi-criterion decision-making
Decision theory
71
(MCDM) movement. MCDM is a set of ideas distinct both from classical decision theory and from the main stream of management science. The text by Zeleny (1982), who is one of the key figures in this movement, provides an excellent account of the ideas of MCDM and a wealth of useful references. The intellectual roots of MCDM lie in the study of linear programming (LP). The invention of the simplex algorithm for discovering the optimum of a linear function when there are linear constraints on the variables was a most significant step in the development of management science. For the first time it was recognized that many logistic problems could be considered in this way, and considerable savings were achieved by applying linear programming in a wide range of contexts. Many academics studied developments of LP, and the technique was one of the principal tools of operational research in the 1960s. It was recognized, however, that determining the appropriate objective function in an application was not at all straightforward. Most decision problems involve conflicting objectives and, at least at first sight, the objectives can appear incommensurate. In the LP world, it became natural to study what could be done with linear programs where more than one objective had to be taken into account. Terms such as multi-objective linear programming, goal programming, and compromise programming were used to denote new ways of looking at the problem, usually associated with new algorithms. As Zeleny (1982) points out, the MCDM movement now embraces wider concerns than just the computations (his book contains interesting chapters on the process of decision-making and the generation of decision options). Nonetheless, its methods are still principally centered on the application of concepts from mathematical programming to the multiple criterion problem. The methods are generally suited to problems where there is a continuum of options (e.g. control variables that may take any values in a given range) rather than a set of discrete alternatives. There are, however, some important contributions that the MCDM movement can contribute to the classical decision theory that we have described in this section. The most important of these comes from interactive multiple criterion programming. As we saw in Section 3.1, in applying multi-attribute value theory to a problem, it is necessary to compute weights before the value function can be used to guide choice. It may be, however, that a decision-maker's judgment about weights can only be effectively elicited when real decision options are before him; to ask him questions about what his judgment would be in many hypothetical situations that he is not actually facing involves a degree of mental agility and role-playing that is not widespread among decision-makers. The interactive versions of multiple criterion programming are not subject to this criticism, since they entail the exploration of the set of good options
72
Theory
interactively with the decision-maker. Typically, once the several conflicting objectives have been represented as (usually linear) objective functions, the interactive computer program will explore the Pareto-optimal boundary, allowing the direction of search to be specified at each stage by the decision-maker who is using the computer. Although this still involves asking the decision-maker hypothetical questions about choices he may know that he does not want to consider, it has the potential for being far more useful as a tool for eliciting values than the more straightforward approach of determining weights at the outset. We believe that decision aids which interactively elicit trade-off judgments at the same time that the option space is explored are likely to prove useful in the multiattribute value analysis methods we described in Section 3.1. 3.5.4 Social judgment theory The study of decision-making attracts researchers from many different disciplines. Engineers, mathematicians, statisticians, and philosophers, as well as management scientists, have made many contributions to our understanding of how to improve decision-making. It is to psychologists that we look, however, for an understanding of how decision-making actually takes place, and, as we will describe in Chapter 4, there is a very large literature on this subject. Many of these psychologists have recognized, however, the importance of going beyond the study of how decisions are made, to how decisionmaking could be improved. K. R. Hammond, of the University of Colorado, one of these innovators, is the leadingfigurein social judgment theory, the goals of which theory are perhaps best explained by quoting Hammond directly: Social judgment theorists are interested in creating cognitive aids for human judgment- particularly for those persons who must exercise their judgment in the effort to formulate social policy and who will ordinarilyfindthemselves embroiled in bitter dispute as they do so. Social judgment theorists intend not only to understand human judgment but to create and develop ways of improving it. Hammond et al. 1975, p.276
Social judgment theory is concerned with far more than a technique for determining the best action to take when confronted with conflicting objectives; but it does contain such a methodology. Once attributes have been determined and options laid out (Hammond refers to these as cues and profiles, respectively), the decision-maker is asked for his holistic judgment on the attractiveness of a representative set of options, measured on some suitable scale. A multi-attribute value function, generally of the additive and linear variety, is then fitted to these data by
Decision theory
73
the method of least squares. Since this is a regression analysis, the number of options needed to make an adequate fit goes up as the number of attributes increases. No attempt is made to check whether the appropriate independence conditions are satisfied, although Hammond does allow for non-linear models to be used in certain circumstances. In fact, even the issue of statistical independence is largely ignored, although there are circumstances where the collinearity of data may seriously affect results. In practice, it seems that the method of SJT uses a linear, additive, multi-attribute value function, where the weights are determined by the application of multivariate linear regression to some holistic judgments. We could summarize by saying that a decision-maker's preference structure is elicited by fitting an appropriate functional form to his expressed judgments in a few cases, and then used to guide what his preference ought to be in other cases. There is more to SJT than this, however. Hammond lays stress on the importance of feedback in improving judgment. To this end an interactive computer program has been constructed that presents pictorial information to the decision-maker, and this assists him in refining his judgments (see Hammond et al.y p.6). This program has also been used to assist social judgment when there are many interested parties. Hammond et al. (1980) report several applications of their method where the preference structures of different people involved with the decision were elicited using the program. These were then analyzed using the methods of cluster analysis to identify factions. This information was useful in clarifying the distribution of views among the decision-makers and, more importantly, to separate fact from value. This last property is seen by many users of multi-attribute decision-aiding techniques (not only the users of SJT) as being one of the most important advantages of using an analysis of this kind. In summary, then, SJT has much in common with multi-attribute value theory. SJT differs from other approaches, though; the technique is rooted in a descriptive psychological theory of judgment, and preference structures are elicited by the use of holistic judgments and regression analysis. 3.5.5 Outranking An approach to the problem of decision-making in the face of conflicting objectives which has become widely known in continental Europe (and particularly in France and Belgium) is the set of methods known as "outranking" {surclassement) developed by*B. Roy at the University of Paris. Succinct descriptions of Roy's work in English are not widely available, but extended accounts were given in some conference proceedings during the 1970s (Roy 1973, 1977). We shall now describe the
74
Theory
method recommended by Roy in 1973; we should stress, however, that in recent years there have been developments of these methods. Readers interested in studying the development of outranking are recommended to study the recent literature. Descriptions (in French) of outranking and its relation to other methods can be found in de Montgolfier and Bertier (1978), Scharlig (1985), and Roy (1985). Roy seems to have been motivated in the construction of his methods by an awareness of the false sense of precision which can so easily be associated with numerical representations of the attractiveness of decision options, particularly if they emerge from a computer. He points out that any method which purports to determine (in some objective manner) the best of a set of decision options might well be claiming too much. While it may be possible to determine unequivocally that, say, option A is better than option B, there may be an option C such that we cannot say whether A is better, worse, or on a level with C. Roy therefore seeks a method which results in a clear ordering of options when that is appropriate, but does not produce an ordering if it is inappropriate. He assumes that a well-defined set of options exists, and that each option can be unambiguously defined in terms of a set of attributes with respect to which each option can be measured. (We note that, in common with many other approaches to decision-aiding, little is said about the hard problem of generating options and choosing attributes in thefirstplace (see Chapter 7).) In mathematical terms, we suppose that there are m attributes and n options, and that a score Xij for the / th option with respect to the z'th attribute, is available. We further suppose that a weight W; can be attached to the ith attribute. The writings on outranking pay little attention to where these weights come from or what they mean; as we will see in Section 7.5.4, there are important and subtle difficulties in weight elicitation. Inputs to the method of outranking clearly involve at least as much elicitation as is needed to construct a multi-attribute value function. Now Roy defines two measures to assess the extent to which one option can be said to be better than another. (We should point out that the definitions we are about to give are not the only ones that are used in outranking methods; they are the originals, however.) Roy first defines the concordance measure Crs for the assertion that option r is better than option s by:
crs= where B is the set of indices for the attributes where xir > xis. If the weights are all normalized, so that they sum to unity, the concordance measure is just the sum of the weights on which r is superior to s.
Decision theory
75
The discordance measure for the assertion that r is better than s is defined by: max(Wi(xis - xir)) Drs= ' e max (\wi(xn - xik)\)
where W is the set of indices for which xir < xisy and the maximum in the denominator is over all /, all /, and all k. The discordance is, thus, the biggest weighted deviation of those attributes which score option s higher than option r, normalized by the maximum possible deviation. If Crs = 1, in which case Drs = o, we can say unequivocally that r is better than s; the reverse conclusion holds if Crs = o. If Crs is close to 1, and Drs is close to zero, we might conclude, so it is argued, that r has the edge over s, with these conclusions becoming less strong the smaller C is and the bigger D is. The ELECTRE suite of computer programs has been constructed at the University of Paris to use these ideas. Given the weights and scores, the concordance and discordance measures are computed for each pair of options. Those pairs of options for which the concordance measure is larger than some value coy and the discordance measure smaller than some value do are identified. For such a pair, thefirstoption is said to outrank the second. By this means, a set of options, N, can be found, with the property that no option within N outranks any other with N, but for each option outside N there is at least one option within N that outranks it. As co and do vary, the set N changes; Roy argues that this procedure gives insight to the decision-maker. So long as co is not too low and do is not too high, the exclusion of an option from N probably means that it is not worth any further consideration. By a continuation of this process, a small set of options can be identified which almost surely contains the best option, but the method does not reach afinalconclusion on which is best. Outranking is reported to have been applied to many practical problems, particularly in France and Belgium. If the success of an approach to multi-criterion decision-aiding is to be judged by user satisfaction, then the method is indeed successful. It appears that users find the discipline provided by the method helpful; and the fact that the method serves to rule out many options, but not to make afinalrecommendation about the best option, seems to be a virtue, in that it is clear that the decisionmaker's prerogative to decide is not being abrogated by a computer. There are several criticisms that one can make of the outranking method. First, the mechanisms involved are somewhat arbitrary. The concordance and discordance measures are clearly sensible indicators, but there are many others that would work just as well, and there is no strong reason for adopting these two. Indeed there are versions of outranking which use other, equally arbitrary, formulae. Moreover, this concordance
j6
Theory
measure does not depend on the extent to which one option is better than another on any particular attribute, and the discordance measure depends only on the extreme value of a difference. Thus information contained in the scores and weights is thrown away by the method. Secondly, while a good argument can be made that no analysis should end with a totally unqualified conclusion, the outranking method achieves this end by deliberately confusing a clear situation. The weights and scores that are the input to the method can lead to a definitive conclusion; it seems more reasonable to explore the adequacy of these numbers as descriptors of the problem than to accept them unchallenged as inputs to a complicated algorithm. In summary, then, we see the main contribution of outranking as being the insight that the output of a decision-aiding procedure should not be totally definitive, but rather identify the options that look good. We believe that this is better achieved by a sensitivity analysis on the input numbers for a multi-attribute value analysis than by the outranking algorithms. 3.5.6 The analytic hierarchy process The fourth distinct approach to multi-criterion decision-aiding which we shall consider here is the analytic hierarchy process (AHP) devised by T. L. Saaty of the University of Pittsburgh. His book, entitled The Analytic Hierarchy Process (Saaty 1980), probably gives the best available description of the approach, and interested readers are referred there for a more extensive account than we are able to give here. In advising the US Government, Saaty became aware that in many problems it is natural to think of the factors which are of importance in determining policy as being arranged in a hierarchy. For example, in evaluating policy for managing radioactive waste, we may think of cost, radiological hazard, and environmental degradation as being the chief areas of concern; within the category of radiological hazard we may think of separating hazard to workers from hazard to the general public; within that latter category we may want to distinguish between hazard to the present generation and that to our descendants, and so on. In this way a hierarchy of factors is built up. Saaty's method centers around the idea of a hierarchy. Once such a hierarchy has been constructed, numbers are elicited and computations performed on those numbers to indicate which of the options being considered is best. Once again little attention is paid to how these options are generated in the first place. While we might interpret the process as one way of carrying out a straightforward multi-attribute value analysis, in that weights and scores are computed and the final measure of attractiveness is a weighted score, Saaty and his
Decision theory
77
associates see the method as being very different from MAVT. Indeed the elicitation methods employed in AHP to compute weights and scores are distinctive and unique. First, it is recommended that the inputs from decision-makers should be given in verbal form, although Saaty does allow those who wish to provide numerical inputs to do so. To elicit the weights of attributes at one level in the hierarchy, for example, a decision-maker is asked questions such as: "Consider a pair of attributes. Are they of equal importance, or is one more important than the other? If one is more important, which one and to what extent? Is it weakly more important, strongly more important, very strongly more important, or absolutely more important?" In this case, and this is typical, the decision-maker is allowed nine different possible responses, and he has to choose one of these nine. Then the verbal response is interpreted numerically by attaching one of the numbers 1,3, 5, 7, 9, or 1/3, 1/5, 1/7, 1/9, to the responses listed above and identifying this number with the ratio of the weights of the two attributes being compared. Thus to say that attribute / is weakly more important than attribute ; is interpreted as implying that wjwj = 3. This procedure is repeated for each pair of attributes at the same level on the branch of the hierarchy being studied. It is most likely that the set of ratios elicited in this way will be inconsistent, in the sense that, for some /, / and k: #= (Wi/Wk).
Saaty recommends that the decision-maker should think seriously about inconsistencies and resolve them; that is, the ratios should be altered so that for all / and /, {WiltUj^WjIWk) = (Wi/Wk).
Although perhaps the most obvious way to create consistency would be a least-squares analysis, Saaty argues for a method involving eigenvalue analysis. Once a set of ratios has been derived, a set of weights follows from the requirement that the sum of the weights should be unity. This procedure is repeated to obtain weights at all levels in the hierarchy, and is also used to obtain the scores of the various options at the bottom level, although slight modifications are necessary to the method in this case. Just as a decision tree is folded back, the hierarchy is evaluated by computing the weighted scores for each option at each level in the hierarchy, starting at the bottom. An axiomatization of AHP, defining what rationality means for this process, has recently been given by Saaty (1986). As is the case with the other methods considered in this section, there are many users who are very satisfied with the AHP, and believe it to be a very useful approach to decision-aiding. The aspects which most appeal to users appear to be the ease with which judgments can be elicited and the
jS
Theory
clarity that the formal structuring of the problem provides. The stress on pairwise comparisons as the type of judgment required is most often reported as a strength of the AHP method. While most of the methods we have described here, including MAVT, provide the clarity of formality, there are difficulties over eliciting numerical representations of judgments in some other methods. Some of the advantages of AHP here may by illusory, however. Although it may be easier to state that one attribute is weakly more important than another than to state that a given unit on one scale is three times as significant as a unit on the other scale, AHP makes the untested assumption that, given thefirst,we may infer the second. The problem of making difficult judgments is taken away from the decisionmaker and replaced by standard assignments. We doubt, however, whether this is a virtue; instead of getting a decision-maker to wrestle with his values, the AHP tells him what they ought to be! There are further problems with AHP. For example, the failure to distinguish clearly between options and attributes reduces rather than increases the clarity with which a problem is perceived, in our view. Another difficulty is that the identity of the recommended option can depend on whether or not the analysis includes an option which is clearly not a good one (see Belton and Gear (19 8 3) or Dyer and Wendell (1985)). One aspect of AHP which does seem to us to provide valuable insight is the need to overspecify a model. In using the methods of decision analysis, such as multi-attribute value theory, it makes sense to validate a judgment by getting at it in several ways,. Thus, in obtaining weights, the judgments that one unit of attribute A is worth two of attribute B and that one unit of attribute B is worth three of attribute C imply that one unit of A is worth six of C. This should be tested by direct elicitation, or some other comparable method. In our experience this is often not done in practical decision analysis, usually to conserve analytic resources (i.e. to save time). A virtue of the AHP is that it formalizes the overspecification process. In summary, then, we believe AHP uses a somewhat arbitrary representation of words by numbers which may be implying an underlying judgment structure which the decision-maker is unaware of and may disagree with. It does require more straightforward judgments from the decision-maker than some other methods, however, and the need to overspecify inputs we see as a virtue. 3.5.7 Conclusions Readers who are looking for a formal approach as an aid to decisionmaking on a multi-criterion problem will by now be asking themselves whether all these approaches to decision-aiding compete with each other, and if they do which should be chosen. Some of the approaches have
Decision theory
79
distinct characteristics which set them aside from the others. CBA was originally designed for the economic evaluation of public policy options and, we believe, is best used in this context. A broader version of CBA has, however, been effectively applied elsewhere (see the discussion in sections 6.1, 6.6, and 6.7). Most of the methods in the MCDM set, concerned as they are principally with problems where the decision variables are continuous, are not appropriate to the discrete choice problems with which the other techniques deal. If the problem faced is well modeled by using continuous variables, however, then we recommend investigating the use of MCDM. Similarly, outranking is an approach that stands alone. It could be applied to any procedure that creates a linear additive model, including MAVT, SJT, and AHP. In fact, it seems to have been applied only to the simplest of techniques for creating a linear additive model, where the decision-maker is simply asked what the weights are directly. The other methods we have considered can all be interpreted essentially as procedures for generating weights in a linear additive value function. The question then arises whether any of the methods is better at doing this than the others.To answer such a question we must define what is meant by "better." If the scores produced by the value function could be compared with some objective norm, then goodness could be defined in terms of how well the methods predicted the correct answer. But, if such a norm did in fact exist, there would be little point in constructing a value function at all. An alternative might be to define a procedure to be good if it produces high correlations with directly assessed judgments. Thus decision-makers would first be asked to order, or even to score, options, without any attempt at constructing a decomposed assessment procedure or a value function; the computed orderings produced by any one of the methods would be compared with the holistic orderings. This is the criterion used by Schoemaker and Waid (1982) in an experimental test of MAVT, SJT, and AHP as weight assessment methods. The reader is recommended to consult this paper for a thorough and detailed analysis of a good experiment; they concluded that there was no marked difference between the methods, as measured by the criterion they adopted. We may well ask, however, if this criterion is sensible. The main purpose in constructing a value function is to improve on simple holistic judgments; the intuition is that, by breaking the problem into its component parts and then reassembling them, we may obtain a better representation of our preferences than may be obtained by direct judgment. If we use direct judgment as our criterion for judging the adequacy of an analytic procedure, our argument may fairly be criticized for being circular. On the other hand, if we can validate a methodology on problems where we are confident in the adequacy of direct judgment,
80
Theory
perhaps it is reasonable to argue that the methodology is good for other problems. The alternative to experimental validation of an approach is to investigate each step in the methodology theoretically, and to judge adequacy by whatever principles one is willing to adopt. The theoretical support for the weight elicitation procedures that we give in Section 7.5.4 is strong. That does not entail their adoption, however. So long as a decision-maker is aware of what a method does and is happy to use it, she should choose the one which seems to her to make most sense. This suggests another criterion: how easy is it to use a method? Schoemaker and Waid (1982) addressed this question as well by asking their subjects to rate each of the methods they used according to its mental difficulty and perceived trustworthiness in capturing true preferences. AHP was thought to be the least difficult and SJT the most; perceptions of trustworthiness were about the same for each method. Perhaps the most satisfactory way of eliciting weights is to use insights from each method. The hierarchies and overspecification of AHP seem to be virtues; representing judgments in words (a semantic scale) is quite easy for a decision-maker, but may entail misrepresentation. If some holistic judgments can be made, then a regression analysis may give useful information about weights. The meaning of a weight must be clear to a decision-maker when judgments are being made, however; this is one of the significant virtues of the method we will describe in Section 7.5.
THE PSYCHOLOGY OF DECISION-MAKING
4.1
INTRODUCTION
As we have already remarked, decision-making is a subject which attracts the attention of scholars in numbers of different disciplines. In practical fields where decision-making is a frequent and crucial activity, such as management, medicine, town planning or engineering, much has been written on the subject. The theoretical literature is even larger, however. Mathematicians have studied the implications of the axioms of decision theory; statisticians have been concerned with how to decide in the face of uncertainty; economists have studied how human decision-making causes, and is caused by, economic activity; philosophers have grappled with what it means to make rational decisions. It is to psychologists that we turn, however, for a consideration of some of the central questions of decision-making. These are: (1) What causes us to make a decision? What are the determinants of decision-making ? and, more usefully: (2) How can we make better decisions? Psychologists have given considerable attention to these questions since the 1950s; the purpose of this chapter will be to describe some of this work and its implication for the practice of decision analysis. Good reviews of the literature are available elsewhere (see Edwards 1961, Slovic etal. 1977, Einhorn and Hogarth 1981, Pitz and Sachs 1984 or Slovic et al. 1986). Hogarth (1980) is a good text on this material. Our purpose here is not to repeat these detailed surveys; rather, we shall summarize some of the more important findings and show readers where they may look for a more detailed description of this work. In the remainder of this section we shall discuss what some of the research goals have been in this area of psychology. In the following three sections we shall describe the extent to which these goals have been achieved in three important areas: the perception of uncertainty, subjective expected utility, and judgment. Finally, in Section 4.5, we draw implications from this 81
82
Theory
work for the construction of procedures to help people to make better decisions. What, then, have been the goals of research by psychologists into decision-making? After the normative principles of decision-making, as described in Chapter 3, had been developed, psychologists began asking whether human decision-making conformed to this theory. Although developed as a normative theory (that is, a theory which tells us what to do), the question was asked whether it could be used as a descriptive theory (that is, describing what we actually do). As we have seen, decision theory requires us to express our uncertainties as probabilities and our attitudes towards risk as utilities, and to compute expected utility to guide our actions. There has been considerable attention to the ability of the human mind to express uncertainty in terms of probabilities, and we discuss this in the next section. Similarly, whether we can describe the way we make decisions as though what we are doing is maximizing expected utility has been explored in depth. This work is the subject of Section 4.3. Then, in Section 4.4, we describe what is known about our ability to represent our value judgments in a value function, as required for applying the multi-attribute value theory of Section 3.1. As we will see, there is considerable evidence that the normative theory is not descriptive. It is possible to argue, then, that we should put it to one side and discover better descriptive theories. A different conclusion might be that the normative theories could be modified to conform to the newer descriptive theories. If we were to follow this route, however, it would amount to asserting that good decision-making is identified with what we actually do, since, as we saw in Chapter 2, one of the most satisfactory definitions of good decision-making is whether our processes of decision conform to a norm. Not only would this conclusion clash with our intuition that decision-making could be better (i.e. what we often do is not good decision-making); there is also a considerable body of psychological literature pointing to the inadequacies of human judgment, going beyond the work on the descriptive inadequacy of the normative model of decision theory. Finally, therefore, in Section 4.5, we shall see how the insights from this psychological research should inform our construction of procedures for sound decision-making. 4 . 2 UNCERTAINTY
4.2.1 Is man an intuitive statistician* If we accept that probability theory is the proper way to describe uncertainty, it is of interest to inquire whether, when we use numerical probabilities, we do in fact handle them in conformity with the theory.
The psychology of decision-making
83
One of the key consequences of the axioms of probability theory is Bayes's theorem, which gives the relationships which should be satisfied by conditional probabilities. We may state it as:
where p{B \ A) is the conditional probability of B given A, p(A) the probability of A, and the other terms are similarly defined. If A refers to some uncertain event and B is some other event that relates to it, then Bayes's theorem describes how the probability judgments involved should be related. For example, suppose that A is "Our competitors have begun production of a new product based on material X," and B is "There is a 50% larger than usual demand for material X this month than we predicted three months ago." Bayes's theorem tells us how our original opinion about the likelihood of our competitor's intentions (p(A)) should be modified in the light of the information about material demand. Our new opinion should be p(A | B). The modification factor involves how likely it would be for a new order of X to be placed if the competitor were producing a new product, p(B | A), and how likely this would be anyway, P(B). In the early 1960s, Ward Edwards and some of his co-workers developed a series of experiments to test whether people revised their probability judgments according to Bayes's theorem (see Phillips and Edwards (1966), Phillips et al. (1966), Peterson and Beach (1967), the title of which we have borrowed from for this section, and Edwards (1968)). The conclusion of these, and other similar, experiments is that people seem to revise their opinions less far than the theorem would indicate (see Slovic and Lichtenstein (1971) or Wright (1984) for a clear account of this work). This property of conservatism in updating perceptions of uncertainty seems to have been widespread in laboratory experiments, and as a result a general hypothesis became current in the community of scientists studying this problem that people are generally poor at changing their perceptions of uncertainty in the light of data. Edwards himself is one of many researchers who have questioned this generalized result (Winkler and Murphy 1973, Edwards 1983). They point out that conditions in the real world are very different from those in the laboratory, and that to generalize from one to the other may not be justifiable here. Moreover, they argue, it may not be our perceptions of uncertainty that are at fault; it could just be our ability to represent those perceptions as numerical probabilities that gives us difficulty. It is this issue that was addressed in some very significant work to which we now move.
84
Theory
4.2.2 Heuristics and biases in judgments of probability In 1971 Amos Tversky and Daniel Kahneman published the first of a sequence of papers on biases that they had observed in the judgment of frequency (Tversky and Kahneman 1971, 1973, Kahneman and Tversky 1972, 1973). Their work received considerable public notice, partly through their review article which appeared in the journal Science the following year (Tversky and Kahneman 1974). More recently they have republished these articles, and other contributions to this research field, in a set of readings (Kahneman et al. 1982). In studying whether people are good at properly perceiving the uncertainty they face and then representing those perceptions as probabilities, there is a tricky philosophical problem at the outset. In the subjective interpretation of probability, which, as we argued in Section 3.2, is necessary in decision analysis, there is no such thing as an objectively correct probability; all that is required of a probability assessor is that the numbers he assesses are coherent, in the sense that they should relate with each other according to the rules of the probability calculus. Our intuition that some coherent sets of probability judgments might be better than others at representing our perceived uncertainty has no place in the theory. We are rescued from this problem by recognizing that what we really require is coherence in the large; that is, our judgments should cohere not only among themselves, but also with all other potential probability judgments we might make, and in particular with judgments of probability in true chance situations, such as games of chance. To study perceptions of uncertainty, therefore, Tversky and Kahneman created experimental situations where it was possible to compute probabilities from assumptions of randomness and the application of the probability calculus; their experimental subjects were asked to assess relative likelihoods without carrying out these computations or indeed, in many cases, knowing how to carry them out. Here are short accounts of three of the many experiments they performed; the reader is referred to the original papers, or to Kahneman et al. (1982), if he is interested in more details. One experiment (Kahneman and Tversky 1973) involved giving a group of subjects the following personality sketch: Tom W. is of high intelligence, although lacking in true creativity. He has a need for order and clarity, and for neat and tidy systems in which every detail finds its appropriate place. His writing is rather dull and mechanical, occasionally enlivened by somewhat corny puns and by flashes of imagination of the sci-fi type. He has a strong drive for competence. He seems to have little feel and little sympathy for other people and does not enjoy interacting with others. Selfcentered, he nevertheless has a deep moral sense.
The psychology of decision-making
85
The subjects were then told that Tom W. was a graduate student, and were asked to rank nine possible fields of specialization according to their likelihoods. More than 95% of the subjects judged that Tom W. was more likely to be studying computer science than the humanities or education, even though it was widely perceived at the time that there were many more graduate students study the latter than the former. It appeared to the experimenters that the subjects were ignoring this information about the chances for a randomly selected student to be studying the different subjects when they made their assessments about Tom W. In another experiment (Tversky and Kahneman 1980), subjects were given the following description: A cab was involved in a hit and run accident at night. Two cab companies, the Green and the Blue, operate in the city. You are given the following data: (a) 85% of the cabs in the city are Green and 15% are Blue. (b) a witness identified the cab as Blue. The court tested the reliability of the witness under the same circumstances that existed on the night of the accident and concluded that the witness correctly identified each one of the two colours 80% of the time and failed 20% of the time. What is the probability that the cab involved in the accident was Blue rather than Green? Most subjects gave the answer as 80%. Now, if we are prepared to accept that without the witness's information the probability is 15% that the cab was Blue (the base-rate for Blue cabs), then after getting the witness's information, the probability should be changed to 4 1 % , on application of the rules of probability. Subjects appeared to ignore the base-rate in making their judgments. In yet another experiment (Tversky and Kahneman 1973), subjects were asked whether the letter r was more likely to be in the first or third position in a randomly chosen English word. A clear majority thought that the first position was more likely, yet the reverse is the more frequent occurrence. In trying to explain the consistent failure of their subjects to estimate probabilities in the way that an analysis of underlying frequency would support, Tversky and Kahneman suggested that people typically make these biased judgments as the result of using certain heuristics, or mental rules of thumb. They suggested that their data were explained by the following three heuristics: a. Representativeness. The likelihood that object A belongs to class B is measured by the extent to which A is perceived to be representative of class B. This heuristic explains the results of the first experiment above. The description of Tom W. is perceived to be highly representative of computer scientists and much less so of humanities students; thus the former category is judged more probable than the latter, despite the known distribution of students.
86
Theory
b. Availability. Alternatively, probabilities may be assessed by the ease with which similar occurrences may be called to mind. The results of the third experiment we mention above are explained by observing that it is easier to think of English words beginning with the letter r than words with r in the third place. The availability heuristic leads to a biased judgment of probability. c. Anchoring and adjustment. Sometimes people will estimate a probability by recalling some similar event for which the probability is known and then adjusting that number; the problem is that adjustment is usually insufficient. An example of this phenomenon can be seen in the conservatism which is usually displayed when people are asked for ranges in which an unknown quantity lies. In an interesting series of experiments (see Alpert and Raiffa 1982) people were asked to estimate a series of numerical quantities about which they had some knowledge but which they were not able to specify exactly, such as the tonnage of steel imported into France in 1984 or the population of Australia. They were further asked to give a range of values such that they were 90% sure that the true value fell in that range. We would presumably expect about 90% of such ranges to contain the true value of the number. In fact the proportion was almost always much smaller. Most people appeared to be unable to express properly the true extent of their uncertainty. This phenomenon has been explained by suggesting that subjects first think of their best guess, somewhere in the middle of the range, and then adjust outwards from that anchor. A more subtle bias was investigated by Fischhoff (1975a, b). When making assessments of decision-making it is clearly important to know whether an assessment of uncertainty in the past was a reasonable one. Fischhoff discovered that people tended to be biased in their recollection of how uncertain they were when reconsidering an uncertain event after it had been resolved; he called it the "hindsight" bias. For example, subjects in one experiment were asked, just before President Nixon's visit to China and the USSR in 1972, to make probabilistic predictions about current events, such as: "Will President Nixon meet Mao Tse Tung at least once?" and "Will the USA and the USSR agree to a joint space program?" Some time after the visits, when these events had (or had not) occurred, subjects were asked what their probabilistic predictions had been. There was a clear bias towards what actually occurred; thus subjects' recollections of what their probabilities had been were larger than they actually had been for events that had occurred and smaller for events that had not occurred. This phenomenon of the hindsight bias has the following implications: (i) We must be careful, when judging the decision-making of others, to recognize that what looks almost inevitable in hindsight may have looked quite unlikely at the time; (ii) our memories of past events (on which we
The psychology of decision-making
87
may base our assessments of future events) may be distorted; (iii) overcoming this bias is not easy; and (iv) our ability to learn from experience might not be as good as we sometimes believe. Another good summary of this work is in Fischhoff (1980a). These hypotheses about the heuristics we might use in assessing probability, and the biases or inaccuracies they might lead to, have proved very influential. Many other studies replicated the results of Tversky and Kahneman (see Kahneman et al. 1982); a widespread view appears to have emerged that this work demonstrated a fundamental failing in human judgmental abilities. In 1981, however, in some important papers presented at an international conference in Budapest, considerable criticism was voiced about the meaning of the extensive research work on heuristics and biases which had appeared in the previous ten years (Edwards 1983, Fischhoff 1983, Phillips 1983). As we remarked above, Edwards pointed out that what may be true for artificial situations set up in the laboratory may not be true in real life; moreover, he argued that it is not reasonable to expect subjects ignorant of probability theory to make judgments according to that theory. In practice, he argued, when we need probability assessments, we will make sure that the tools of that theory are available to us. He felt that generalizations from this work had led to a widespread, and unjustified, view that man has severe cognitive limitations. Fischhoff was more cautious in assessing the research work on heuristics and biases; while acknowledging that one could not accept some of the wider generalizations that had been made about man's intellectual incapacities in this regard, the research did show, he felt, that people do need the help of formality in making sensible probabilistic judgments. We will return to discuss further the implications of this body of research in Section 4.5. 4.2.3 Calibration Before we leave our discussion of the research work that psychologists have undertaken on human abilities to judge uncertainty, we will give a brief account of the literature on the property known as "calibration." We mentioned above that the axiomatic approach to subjective probability leads us to accept any assessments of probability as adequate as long as they are coherent. A further property however can be reasonably sought. Suppose a probability appraiser has the task of making a large series of probability judgments: is it not reasonable to hope that, for all propositions assigned a particular probability, the proportion which are true should equal the probability assigned? Thus we would expect 80% of those propositions assigned the probability 0.8 to be true. If a probability-
Theory
88
0.7
0.8
Subjects' responses
Figure 4.1 A set of empirical graphs describing calibration. Subjects were asked how likely statements were to be true. For all those responses of a subject giving a particular probability, the proportion correct are plotted on the graph. Taken from Lichtenstein et al. 1982, Figure 2.
appraiser displays this property he is said to be well calibrated. Some authors have criticized the notion of calibration on the grounds that the property is not required by subjective probability theory; but our intuition is that it seems a sensible property to require, and, as Dawid (1982) has shown, if assessors are given outcome feedback after each assessment, they can expect to be well calibrated if they are honest and coherent. Experiments to explore whether people are well calibrated have been conducted by many different researchers. A review of this work is given in Lichtenstein et al. (1982). One of the most important practical areas where calibration has been explored is in weather forecasting. In the United States, weather forecasters have been expressing their forecasts of rain in probabilistic terms since 1965, and a record has been kept of their success rate (Murphy and Winkler 1977). These studies report very good
The psychology of decision-making
89
calibration. When predicted probability is plotted against observed frequency for these weather forecasters, an almost straight line is obtained, going through the origin and (1, 1), as would be expected for a perfectly calibrated appraiser (see Murphy and Winkler 1984 for a recent review of this work). The situation is not so satisfactory in laboratory experiments. Numerous studies have used a set of factual questions, the answers to which are known but not to the subject. They have to choose between two possible answers for each question and state the probability that they are correct in their judgments. In these experiments a consistent over-confidence is discovered. Figure 4.1 is taken from Lichtenstein et al. (1982) and shows how, except for judged probabilities of about 0.5, the proportion correct is consistently smaller than the probability estimated. It is not possible to conclude from the results we have mentioned that miscalibration is only discovered in artificial laboratory settings, since other practical studies have also demonstrated poor calibration. It has also been suggested that calibration depends on factors such as the ease of the task and the expertise of the assessor, and there seems to be some substance in these suggestions. The question now arises as to how we can improve our calibration. This problem is discussed in Lichtenstein and Fischhoff (1980). They point out that attempts to correct a bias by adjusting an assessment according to a previously determined calibration curve are unlikely to be satisfactory. First, calibration changes as feedback is provided; and, secondly, even if this is avoided, calibration appears to depend on the difficulty and context of the task, and so a whole set of calibration curves might be needed. They felt that it would be better to devise methods of training people to become better calibrated in the first place. Their paper reports two experiments aimed at training people to be better calibrated probability-appraisers. Most of their subjects inproved their calibration as a result of the training they underwent, which involved feedback following each session. Lichtenstein and Fischhoff were careful not to claim unsubstantiable generalizations of their results, but there can be no doubt that, if calibration is a desirable property, and we believe it to be so, then it is possible to construct training procedures to encourage better calibration. We ought to note, at this stage, that the weather forecasters who exhibit the excellent calibration mentioned above are routinely provided with feedback and, what is more, part of their remuneration depends on how well calibrated they are. The Brier scoring system has been used for many years now to reward well-calibrated assessments of probability (see Lichtenstein et al. 1982 for a discussion of this topic). Put simply, if a forecaster assesses the probability of rain at p, and it rains, then he receives a sum proportional to p2; if it does not rain his prize is proportional to
90
Theory
(i — p)2. It can readily be shown that over the long term the forecaster maximizes his income by being honest and well calibrated. It has been argued that the excellent calibration of weather forecasters can be attributed to the use of the Brier scoring rule for rewarding them. Finally, note that while calibration can be seen to be a desirable characteristic of probability appraisers, the property, even combined with coherence, does not guarantee that a probability-appraiser conforms with our intuitive notions of "goodness." If a weather forecaster always quoted, as his probability of rain, the long-run frequency with which rain occurs, he would be very well calibrated, but of little use to us. Fortunately the Brier scoring system penalizes this kind of behavior! 4 . 3 EXPECTED UTILITY
We discussed in the last section what is known about man's ability to represent his perceptions of uncertainty as probabilities, and to conform to the rules of probability theory when revising his perceptions. We now address the larger question of whether we appear to maximize expected utility when making decisions in the face of uncertainty. Since the evidence suggests that we do not conform to the prescriptions of probability theory, it should be no surprise to the reader to learn that we do not naturally conform to the principle of expected utility either. There has been a considerable amount of interest in this topic; a good review is to be found in Schoemaker (1980), to which the reader should refer for a more detailed account of this research area. We shall limit ourselves here to a brief discussion of some of the experimental work and field studies that are particularly significant. The Allais paradox, which we described in the last chapter, is probably the most famous demonstration of people's failure to choose according to the precepts of expected utility theory. There are many others, however. That due to Ellsberg (1961) is almost as well known. The problem situation presented by Ellsberg to his subjects was as follows, in our words: An urn contains 90 balls, of which 30 are red, and the remaining 60 are black and yellow, but in unknown proportion. A ball is drawn at random and, depending on its identity, you may or may not win a prize of $100. We denote the possible prizes you might win if a red, black or yellow ball is drawn by a triplet (R, B, Y). * Situation A Which of the following gambles do you prefer? AI:($IOO,
or Az: ($0,
$0,
$0)
$100, $0)
The psychology of decision-making Outcome
Utility
100
1
100
1
100
1
100
1
100
1
100
1
Situation A
Situation B
Figure 4.2 Ellsberg's paradox. A preference for Ai over A2 is inconsistent with a preference for Bz over Bi.
92
Theory * Situation B Which of the following gambles do you prefer? Bi: or Bz:
($100, ($o,
$o, $100,
$100) $100)
Ellsberg found that most of his subjects preferred A i to Az, and Bz to B i. These choices can be shown to be inconsistent with expected utility theory, as we shall now demonstrate. We should presumably assess the chance of getting a red ball at 1/3 (although note that there is nothing in subjective probability theory to require us to do so — it is merely that it is reasonable to suppose that a random drawing should be interpreted as one where the chance we give to any particular ball being drawn is the same as that for any other). Since we do not know the proportion of balls that are black, it might be reasonable to give a probability of 1/3 to the ball drawn being black; but we do not need to do this in order to demonstrate a contravention of utility theory. Suppose the proportion of black balls is /?, so that the proportion of yellow balls is 2/3 — p. Then, again taking these proportions as our subjective probabilities, we can represent the situations in decision trees, as in Figure 4.2. Now let us apply expected utility theory. Let us take the utility of $100 to be 1 and that of $0 to be o. Then the expected utility of Ai is 1/3, and of Az is p; a preference of Ai over Az implies that p < 1/3. On the other hand the expected utility of Bi is 1 - /?, and that of Bz is 2/3; hence a preference of Bz over Bi implies that p > 1/3! The usual explanation of this failure to follow the norm or expected utility theory is that subjects prefer to gamble on options for which the probabilities are more clearly established. Slovic and Tversky (1974) report some interesting experiments based both on this problem of Ellsberg and on Allais's paradox. Subjects were presented with the two problems and asked to state their preferences. Then each subject was presented with a counter-argument for his choice on Allais's problem, and then asked to reconsider his decisions on both problems. Not much revision of opinion occurred, and most subjects were happy to maintain a set of choices on both problems that were inconsistent with expected utility theory. Slovic and Tversky end their paper with an intriguing hypothetical dialogue between two decision theorists discussing the implications of these findings. Was people's failure to choose in a manner consistent with expected utility theory simple pig-headedness in the face of rational arguments, or was it that the normative theory was unacceptable for some good reasons? Slovic and Tversky did not give a definitive answer to this question, and indeed none can be given. We reiterate the argument that we made in Chapter 3. There is no moral imperative to follow the prescriptions of expected utility theory. It is, however, an attractive procedure to follow, and, as the mythical Dr S of
The psychology of decision-making
93
Slovic and Tversky's paper points out: "I have observed that, in general, the deeper the understanding of the [theory], the greater the readiness to accept it." We share this observation. The descriptive inadequacy of expected utility theory was demonstrated in another way by Lichtenstein and Slovic (1971), who arranged for subjects to declare a preference between gambles in two different ways. Firstly they were asked to choose between some gambles directly; then they were asked for how much money they would sell the rights to play the gambles. The orderings of gambles produced by these processes were hardly correlated at all. In fact, when simply choosing between gambles, people tended to prefer those with a high chance of winning something; higher prices were asked for those gambles where the prizes for winning were bigger. These inconsistencies of preference were replicated in a study involving gamblers in a Las Vegas casino (Lichtenstein and Slovic 1973). Tversky and Kahneman (1981), in yet another sequence of experiments, supported and generalized these results to show that choices between uncertain options can depend on the way the problem is "framed," to use their term. For example, in one experiment, they gave their subjects the following situations to compare: * Situation A Which of the following options do you prefer? Ai: a sure win of $30. Az: an 80% chance to win $45 (and a 20% chance of winning nothing). * Situation B You have entered a two-stage game. In the first stage there is a 7 5% chance of ending the game without winning anything, and a 25% chance of moving on to the second stage. If you reach the second stage you enter one of two gambles, which must be chosen now. They are: JBi: a sure win of $30. Bz: an 80% chance to win $45 (and a 20% chance of winning nothing). * Situation C Which of the following options do you prefer? Ci: a 25% chance to win $30. Cz: a 20% chance to win $45. These decision options are portayed in Figure 4.3. Now let us analyze these situations using expected utility theory. Since we can arbitrarily define two points on the utility scale, let us set «($45) = 1 and «($o) = o, and write u for u($$o). Then on folding back the tree in situation A, we see that A1 should be preferred to A2 if u > 0.8.
94
Theory
Situation A
0.25
$30
Situation C
Figure 4.3 Choice situations for Tversky and Kahneman's framing experiment. In situation B, the expected utilities of Bi to Bz are, respectively, o.z$u and 0.2, so that we should prefer Bi to Bz if, once again, u > 0.8. Situation C is easily seen to be similar to situation B, in that the expected utilities of Ci and Cz are identical to those of Bi and B2, respectively. Thus consistency with expected utility theory would require that subjects should prefer either the first choice in each situation or the second choice in each situation. The experimenters found that a clear majority of their
The psychology of decision-making
95
subjects preferred Ai to A2 and Bi to B2, but they also preferred Cz to Ci. The contrasts between A and C display a phenomenon similar to that in Allais's paradox. Subjects seemed to ignore the monetary rewards when comparing a certainty with an uncertain option; but when the chances in two options were similar they went for the gamble with the higher prize. A different phenomenon is seen in the inconsistencies between situations B and C. Here it appears that subjects are attracted to the sure outcome in the second part of the two-stage gamble, even though there is, in fact, only a 25% chance of getting it at the outset, just as in situation C. Choice clearly depends on more than just the outcomes and probabilities; it also depends on how the problem is framed. It is not only in laboratory situations that people appear to violate expected utility theory. Schoemaker (1980) describes numbers of field studies which show the failure of expected utility theory to describe how people make choices in practical and uncertain situations. We could now leave the matter here, arguing that while the practice of decision-making does not seem to be described by expected utility theory, this merely shows that the person who is attracted by the normative reasonableness of the theory will need to take care when making decisions if he wants to abide by its precepts. He will need aids to overcome his biases in assessing uncertainty and in evaluating uncertain decision options. It makes sense, however, before taking our discussion further, to mention the possibility of creating alternative descriptive theories, and for two reasons. The more we know of how people actually do make decisions, the easier it will be to know what to do in constructing decision aids to help them conform with the normative theory, if they want to. And, secondly, it may be that adequate descriptive theories will suggest modifications to the standard normative theories, to produce a new normative procedure that is even more compelling, and perhaps nearer to natural decision-making procedures. Chapter 3 of Schoemaker (1980) provides a good review of alternatives, and the volume edited by Wallsten (1980) contains a number of interesting discussions of different descriptive theories of decision-making, particularly in the articles by Pitz, Payne, Wallsten, and Estes. We are not yet in a position, however, to suggest compelling alternatives to the normative theory described in Chapter 3. 4 . 4 JUDGMENT
In the previous two sections we have looked at the evidence that psychologists have produced on the extent to which our perceptions of uncertainty and our decision-making in the face of uncertainty follow the prescriptions of the corresponding normative theory. We now turn to the
96
Theory
issue for which we presented the normative theory first in Chapter 3, namely decision-making when we have no uncertainty about outcomes. Here the problem is solely how to represent our preferences, or our value judgments about conflicting factors. Now we can ask whether the prescriptions of multi-attribute value theory can describe our decisionmaking practice. The first answer to this question is, once again, that it does not. It is, for example, quite easy to find examples of people taking decisions according to the lexicographic model (see Hogarth 1980, p. 57). Here the decisionmaker concentrates on one attribute only, and only brings others into play if options are equally good with respect to that attribute. For example, in choosing an employee he might look first for formal qualifications; if two or more candidates had equal qualifications and these were better than those of all the other candidates, he would eliminate all but those best qualified and then search among those who remained, using a second criterion, such as work experience. It is easy to show that it is not possible to create a value function that is consistent with this choice procedure. Despite this first answer, the second answer to our question above is that in most situations it is quite possible to describe actual decisionmaking as though people were making a decision by maximizing some function of variables describing the attributes of importance. In fact, the linear additive value function (which we discussed in detail in Chapter 3) has demonstrated considerable descriptive power (see Dawes and Corrigan 1974, Dawes 1979, or Fischhoff et al. 1982). At this stage, the interesting question to ask is whether we can learn anything about man's ability to represent his preferences in the form of a value function. Are there, for example, biases in representing judgments of relative importance, just as there appear to be biases in representing perceptions of uncertainty? The literature on these aspects of judgment is large, and it is not possible for us to give a full review of it here. The book by Hogarth (1980) provides an extensive discussion, and the reader is referred there for a fuller account. We shall limit ourselves here to presenting just a few of the ideas and results. One interesting piece of research worth mentioning here is the "bootstrapping" idea (see Dawes and Corrigan 1974). In experiments of this type, experts are observed making judgments in their field of expertise (such as diagnosis in clinical psychiatry [Goldberg 1970] or prediction of success of graduate students [Wiggins and Kohen 1971]), and at the same time the options are measured in terms of a defined set of attributes (personality profiles in psychiatry, and background and aptitude measures for the graduate students). Weights are determined by estimating a linear relationship between the expert judgments (diagnosis as neurotic or psychotic and predicted grade point average, respectively) and
The psychology of decision-making
97
the measured attributes. The computed weights are then used in a linear additive model, giving, of course, different predictions. What is surprising is that these smoothed values are closer to the true values (which could be computed in each case) than are the original judgments. It appears that judgments can be improved by this smoothing process. Dawes and Corrigan (1974) demonstrated an even more remarkable phenomenon; if weights were set equal, once the attribute measurements had been standardized to have the same mean and variance across the options being considered, then the predictive power of the additive model was even greater. Dawes (1979) gives an interesting discussion of the meaning of these results; he sees them as further support of the conclusion of Meehl (1954) that the prediction of numerical descriptor variables is better done by using a linear additive model than by using intuitive judgment. We must be careful not to generalize too freely from these results, to conclude, for example, that in all judgmental situations a simple additive value function will outperform holistic judgments. Nonetheless, there is no evidence here to discourage us from seeking to construct value functions as an aid to decision-making (see Dawes 1977 for a discussion of this point). Another interesting experiment exploring the relationship between holistic judgment and judgment based on a consideration of separate attributes was conducted by Phelps and Shanteau (1978). They ran a series of experiments with livestock judges, whose task is to assess the quality of meat before an animal is slaughtered. Thus any judgment they make can be compared with the actual quality of the meat, which can be objectively determined after slaughtering. One of their experiments was to compare the judgmental ability of their subjects when information about animals was presented as a series of measurements (such as weight, length, etc.) with their performance when information was presented in photographs. They discovered that in the latter case performance was not as good, and could be well modeled by an additive value function using only three or four of the twelve attributes originally identified. On the other hand, they found that, when information was provided in the form of verbal statements, between nine and eleven dimensions were used. We take this as further evidence that, in some circumstances, it is possible to improve on judgments by using the value function approach described in Section 3.1. Shanteau and Phelps (1977) have used these experiments as the basis for an interesting discussion of research work on judgment in general. Another observation from the literature, which is perhaps not surprising, is that the context in which a judgment is made may affect the judgment. We saw this in the last section, in the context of decisionmaking in the face of uncertainty, in Tversky and Kahneman's framing
98
Theory
experiments. The same is true for judging values. For example, Einhorn and Hogarth (1982) showed that the addition of extra choices given to, or supplied by, a subject can change his preferences among the original set of choices. A discussion of some of the other literature on this topic is given in Pitz and Sachs (1984). Fischhoff et al. (1980) have used the word "labile" to describe the difficulty we have in pinning down our values. The Oxford English Dictionary defines labile as "slippery; prone to undergo displacement and change; unstable." In the light of the experimental work described above this is a particularly appropriate adjective. Fischhoff et al. address the question of what might happen when an analyst attempts to measure a subject's labile values. They give a careful account of the ways in which this activity can cause values to move; readers are encouraged to read the paper if they wish to explore this literature further. We just quote some of their conclusions here: Expressed values seem to be highly labile. Subtle changes in elicitation mode can have marked effects on what people express as their preferences.. . . Confronting these effects is unavoidable if we are to elicit values at a l l . . . . If one is interested in what people really feel about a value issue, there may be no substitute for an interactive, dialectical elicitation procedure, one that acknowledges the elicitor's role in helping the respondent to create and enunciate values. Fischhoff et al. 1980, P- 137 4.5 IMPLICATIONS FOR DECISION SYNTHESIS
Our brief review of some of the psychological research on man's ability to fit in with the prescriptions of decision theory may have led the reader to presume that "man is a cognitive cripple," to use a term identified (and also refuted) by Edwards (1983). Before doing so, we do well to assess the validity of this research as a whole. An important critique of the validity of the kind of decision-making research that we have discussed in this chapter has been provided by Ebbesen and Konecni (1980). Their main point echoes that of Edwards which we referred to in Section 4.2.1: to what extent can results obtained in the laboratory be assumed to hold true in real-world settings? By way of example, Ebbesen and Konecni reported a study of the decisions on bail-setting of a number of judges in a criminal court. First, the judges took part in an experiment as subjects in a laboratory setting that simulated their legal role. They were asked to set bail, in dollars, exactly as they would have done on a real case, but for a number of hypothetical cases. Background information was given in the form of the four cues that previous research had shown were the chief determinants of decision. At a later stage, and unknown to them, the same judges were observed making real bail-setting decisions in court. It was
The psychology of decision-making
99
found that, although the same four cues explained their decision-making behavior, a very different pattern of dependence on these cues emerged. While in the laboratory setting the extent of local ties of the suspect was by far the most significant variable, in real decision-making the recommendations of the district attorney were the most significant. Ebbesen and Konecni argue effectively that, although they had expected to learn about the judges' real decision-making process by the laboratory work it was clearly not possible to do so. This obviously leaves open the whole question of whether we can learn anything from laboratory studies to assist us in aiding real decision-making. This point is followed up by Einhorn and Hogarth (1981). They point out that, lacking hard experimental evidence, we can only give a judgment on the prevalence of judgmental biases in daily life. There are two extreme views. The most optimistic asserts that biases are limited to laboratory situations which are unrepresentative of the natural ecology. . . . The pessimistic viewpoint is that people suffer from cognitive conceit (Dawes 1976); i.e. our limited cognitive capacity is such that it prevents us from being aware of its limited nature. Even in its less pessimistic form, this view is highly disturbing and emphasizes the importance of further research. Einhorn and Hogarth 1981 The issue of generalizability has been recently discussed in greater depth by Hammond (1986) and Hogarth (1986). We can argue, of course, that in Ebbesen and Konecni's case the judges were giving a response that was normatively correct in the artificial cases, but were responding in a pragmatic way to the others. The lack of generalizability can thus be attributed to a mismatch between the experiments and the real world, which could, in principle, be remedied. It does not prove that laboratory results are irrelevant. Where, we may ask, does this rather inconclusive discussion leave us if we wish to apply decision theory to real problems? It should first warn us to be careful in practical attempts to capture values and beliefs in models of the real world, recognizing that the biases demonstrated in the laboratory may be present. Thus, it will be prudent to watch out for the biases of representativeness, availability, and anchoring when assessing probabilities; we will want to recognize how our choices depend on the frame in which the problem is set; and we would do well to address our values in several different ways to check that they are not so labile that they have slipped away from being a representation of our preference structure. But, secondly, we might be tempted to argue that, since we do not conform to the norms of decision theory, we should not use it as a guide for behavior. We argue against this view by reiterating our earlier statement that decision analysis is a normative procedure which we can
ioo
Theory
choose to follow or not as we please; if we do choose to accept its recommendations, then the results of this research will merely show us how necessary it is to attend to the procedures of decision analysis carefully. EXERCISE FOR 4
Identify the bias associated with each of the following examples of erroneous judgment: (a) Subjects repeatedly overestimate the relative frequency of causes of death such as murder and tornadoes, but they underestimate causes such as diabetes and heart disease. (b) In a particular experiment, subjects' estimates of the outcome of a quantitative process were highly correlated with random cues given by the experimenter. (c) Subjects often answer A to the following question: There are two programs in a high school. Boys are in a majority (65%) in program A, and in a minority (45%) in program B. There is an equal number of classes in each of the two programs. You enter a class at random, and observe that eleven out of the twenty students in the class, i.e. 55%, are boys. Is the class more likely to belong to program A or program B?
DECISION ANALYSIS IN ORGANIZATIONS
5.1
INTRODUCTION
In following through the discussion of decision theory that we presented in Chapter 3 the reader will have become aware that it is concerned with how an individual person might reasonably make decisions. Most significant decisions, however, are not made by an individual acting alone but by groups of people. Many examples spring to mind: committee decisions determined by a majority vote; the results of the adversarial process in a law court; or the subtle combination of political pressures that determine events in government. It is also the case that most practical applications of decision analysis are carried out in an organizational setting and focus on organizational decision-making rather than on improving the decision processes of a single decision-maker. It is most important, then, to ask how a normative theory developed to guide individuals can have a role to play in organizational decision-making. It clearly does have such a role in practice; we must ask if that role is valid, or simply bogus in that it does not lead to better decisions. To do this we look first at what is known about how decisions are in fact made in organizations. The literature on this is diverse and enormous, and several different intellectual traditions have contributed to our understanding. Organizational theory is concerned with how groups of people operate when working together; public administration scholars have studied how decisions are arrived at in governmental organizations; social psychologists have investigated how individual decision-making is affected by group pressure. We will not attempt to review all of this literature here; rather we give the reader some entry points to it, and mention what we believe to be the most important findings for a consideration of the use of decision analysis on organizational problems. We will do this in the next section. We have already argued, in earlier chapters, that how people actually make decisions does not necessarily tell us how they ought to make decisions. This is just as true for a group of people as it is for individuals. 101
102
Theory
Much theoretical attention has been paid to normative theories for group decision-making, and we review them below in Section 5.3. Our review will discuss the appropriateness of using these theories for designing real decision-making procedures in organizations. Finally, in Section 5.4, we return to the question of how the normative single-person decision theory of Chapter 3 fits in to organizational decision processes. 5.2 ORGANIZATIONAL THEORY AND DECISION-MAKING
The purpose of this section is to explore two questions: "What do we know about how decision-making is carried out in organizations?" and "What do we know about organizations which will help us in determining whether and, if so, in what way decision theory may be of use for organizational decision-making?" As we observed above, the literature on this question is very large, and it is very diverse. As an introduction to the ideas in organization theory, the reader is recommended to consult the texts by Mintzberg, and in particular Mintzberg (1979). Another way into this literature is the set of readings edited by Pugh (1971), together with the accompanying text (Pugh et ai 1964), which gives useful summaries of the main ideas of some of the principal writers in this field. Finally, the report of a conference held in 1981 in Oregon (Ungson and Braunstein 1982) contain some very useful papers. It was attended by some of the principal scholars in behavioral decision theory and in organizational decision-making, and the connections between the two streams of thought are explored in some detail. Now we shall discuss some of the key ideas from this literature as they impinge on the use of decision analysis in organizational settings. The early social theorists, such as Marx, Weber, and Durkheim, were interested in describing how organizations were formed and how they worked. The concept of a bureaucracy had been discussed as early as the mid-eighteenth century, usually with pejorative overtones (see Albrow (1970) for a history of the idea). Weber, who was a German sociologist at the turn of the century, argued, on the contrary, that a properly run bureaucracy was necessary and desirable. He believed that bureaucratic decision-making could free organizations from the inertia of custom and traditional practices, which usually involved favoritism and the whims of the powerful; instead it could become consistent, impersonal, and objective. His ideas have had considerable influence. In Weber's bureaucracy, there are fixed jurisdictional areas, which are ordered by rules; there is a well-defined hierarchy of positions for people to occupy; and the management, or decision-making, is prescribed by well-defined procedures. This is not the only type of organization that Weber discusses, but it is his
Decision analysis in organizations
103
principal interest (Weber 1947). A most significant investigation of whether Weber's ideal type of bureaucracy actually describes real organizations was carried out by Pugh and his group at Aston University in the sixties (Pugh et al. 1963). They concluded that the characteristics used by Weber to define his bureaucracy were indeed highly correlated in a random sample of organizations, and that it was possible to define a unique measure of the formalization of organizations. If this measure was high, the organization could properly be said to conform to Weber's ideal type of bureaucracy. Critics of Weber's ideal type have argued that, due to its rigidity, decision-making can often fail to lead to what everybody involved can see as a sensible conclusion. While this work gave a framework for studying the nature of organizations, it did not concentrate on decision-making within them. This was the focus of H. A. Simon's seminal work, to which we now turn. Simon has been perhaps the most important thinker on organizational decisionmaking in the last forty years (see, for example, Simon i957a,b, i960). His best-known idea, that of bounded rationality, resulted from his observation that, in administrative decision-making, a manager typically does not maximize some utility in making a choice. His actions are better described by assuming that he just looks for a course of action that is satisfactory rather than optimal. Satisficing behavior has been widely observed, and some economic theories have been founded in the last thirty years on the assumption that administrative man behaves in this way. On the supposition that people make decisions in this way, Simon recommends that organizations should be designed so that this behavior does not lead the organization away from its goals. He introduced the notion of a programmed decision; in a Weberian bureaucracy, with a high degree of formalization, it is possible to prescribe how a decision should be made, prior to its appearance. An example of a set of programmed decisions is in the rules that can be established for operating a warehouse in a factory. The time at which some item should be reordered and the quantity to be reordered can be laid down in advance, as a function of projected demand and the costs involved. The stores clerk does not have to go through a complex decision-making process; he merely carries out a programmed decision. Of course, even in the most formalized bureaucracy, only a small proportion of decisions are highly programmed; but Simon argues that, as computers become more powerful and our understanding of human decision-making becomes more satisfactory, there will be great advantages in attempting to program more and more of the decisions that need to be made within an organization. We shall return to consider the implications of this view for decision analysis in Section 5.4. Simon's contribution to our understanding is not the only one we should mention, however. Clough (1984), who gives an excellent and
104
Theory
entertaining account of both the theory and practice of organizational decision-making, provides a summary of some other theories in his Chapter 6, to which we refer the reader. He does not mention, however, the contribution of Lindblom (1959), which has had an important effect on thinking in this area, and to which we now turn. Lindblom argues that, far from making rational decisions (even of Simon's bounded type), organizations proceed by "muddling through." His work appears to have been performed in response to the over-eager enthusiasm for quantitative approaches to decision-making that were common in the late fifties, particularly in US government circles. A naive application of decision theory to public policy analysis suggests that, in formulating a policy, all possible policies should be explicitly set out, they should each be evaluated in terms of all possible attributes of importance, and then the policy should be adopted which maximizes expected utility. As Lindblom points out, this is a task of enormous proportions for most policy issues; not only are the data requirements of such a program generally infeasible, but most organizations do not have the resource of analysts' time in sufficient quantities to carry this program out. Lindblom argues that, instead of this rather grandiose approach, what people actually do in organizations is to "muddle through"; or, to be more formal, decision-making happens incrementally, with policies being compared with slightly different ones but not with all possible policies. As a description of how policy-making happens, there is much evidence to support Lindblom's view; some writers have gone on to justify this approach as a proper way of reaching decisions in organizations (see, for example, Jannsson and Taylor (1978)). Lindblom's more recent thinking on this topic, after twenty years of discussion of his ideas, is in Lindblom (1979). He still believes that, as a sensible guide for organizational decision-making, incrementalism has much to offer. In his extensive review of the literature in behavioral science that has a bearing on the design of decision support systems, Sage (1981) discusses the various descriptions of decision-making in organizations that have been put forward. As well as the theories we have already mentioned, he discusses the garbage-can model, put forward by Cohen et al. (1972). As Sage says: This . . . model views organizational decision-making as resulting from four variables: problems, solutions, choice opportunities, and people. Decisions result from the interaction of solutions looking for problems, problems looking for solutions, decision opportunities, and participants in the problem solving process. The model allows for these variables to be selected more or less at random from a garbage can. Doubtlessly this is a realistic descriptive model. It is especially able to cope with ambiguity of intention, understanding, history, and organization. Sage 1981, p.666
Decision analysis in organizations
105
As we mentioned in the introduction to this section, it is not only the organizational theorists who have contributed to our understanding of how decision-making happens in organizations. A subject of great interest to political scientists is the way in which the political process results in decisions. Hall and Quinn (1983) have edited a collection of original essays from political scientists which provides a good account of the contribution of public administration theorists to this topic. The readings edited by McGrew and Wilson (1982) are also of great value here, although they range more widely than the political science literature alone. The political science model sees organizational decision-making as a result of the interplay of pressure groups with different views and different power. While most political science literature is concerned with national or local government, its model of decision-making is equally applicable to other kinds of organizations, such as companies, universities or hospital authorities. The outcome of a decision is determined not by rational decision-making, nor by incremental development, but by the confrontation of different groups, negotiation between them, and the exploitation of the influence and power that they have at their command. This is clearly an adequate description of how decisions are arrived at in many contexts; when we come to discuss the role of normative theories of individual decision-making in an organization in Section 5.4, we will need to note that in many contexts it is not possible to think of an organization as being a single actor, as this theory requires. Any application of normative decision theory must recognize the political process that leads to decisions. Finally in this section, we give a brief account of the contribution of social psychology to how decisions are reached in organizations. Janis and Mann (1977) give a good review of the research in this area. One well-known phenomenon identified by Janis is that of "groupthink" (Janis 1972). He argues that if a decision-making group is: (1) highly cohesive; (2) insulated from many external influences; (3) lacking in procedures for systematically evaluating alternatives; (4) subject to a directing leader; and (5) in a condition of high stress; then there is a concurrence-seeking tendency which he calls groupthink. Some of the symptoms of groupthink are: (1) an illusion of the invulnerability of the group to error; (2) collective rationalization of an action; (3) a common belief in the morality of the group; (4) direct pressure on dissenters to conform. Groupthink can lead to defective decision-making in incompletely searching for alternative courses of action or acquiring relevant data, and to biases in processing information at hand, amongst other undesirable effects. Although the concept of groupthink was developed as a result of a consideration of the way decisions were made in major political crises, such as the Cuban missile affair (Allison 1971), it undoubtedly is a phenomenon that occurs in many other contexts.
106
Theory
Social psychologists who study decision-making are interested in how social pressures affect the decisions that an individual makes. Two such pressures, identified by Janis and Mann (1977), are: anticipatory regret, our tendency to worry about how disappointed we, and others, might feel after the event, if we take the wrong decision; and threats, or constraints, imposed by others. These are clearly important influences on decisionmaking, and must be borne in mind when prescribing how to use decision theory in group decision-making. We do not pretend, in this section, to have done any more than touch very briefly on some of the social science literature on decision-making in organizations. Two conclusions can be made, however, which have a bearing on the practice of decision synthesis. The first is that an organization does not make decisions in the way that an individual does; applying decision theory to organizational decision-making will require careful argument. The second is that there are many ways in which organizational decision-making is inadequate; prescriptions for how to make better decisions are necessary. In the next section we discuss some of the normative approaches that have been developed for group decisionmaking, and which are distinct from the decision theory for individuals developed in Chapter 3. Then in Section 5.4 we give our recommendations for how all this theory may be used in organizational contexts. 5.3 NORMATIVE THEORIES FOR GROUP DECISION-MAKING
j.3.1 Collective choice It seems to be easy for us to think of an organization in human terms. For example, when we find that a credit card company, in writing to us apologizing for failing to note our changed address, sends the letter to the old address, we are angry. Our anger, however, may be based on a judgment of the company as though it were an individual. We wonder how they could be so stupid, given that we have informed them, without realizing that the person who wrote the letter is not the same as the person who alters the address list! Similarly, the legal status of a company or a corporation as an individual reflects the convenience of treating a group of people as though they were a single person. A further example of this phenomenon is the tendency of the Press to seek for a single view on some issue of public importance from identifiable groups, such as the Church, the environmental lobby, or the arms industry, even when there is no identifiable common view on the issue among members of the group. It is not surprising, then, that attempts are made to apply decision theory to organizations as though they were individuals. Now, as we have seen, the measurement of both probability and utility depend on it being
Decision analysis in organizations
107
possible to articulate a preference order on options. While it is plausible that an individual acting alone may be able to do this, it becomes problematic to determine what an organization's preference order is, particularly if there are marked differences of opinion. The immediate answer that we might be tempted to make is that the preferences of a group of people can be determined by the traditional method of majority voting. Unfortunately, this does not always lead to a clear-cut group preference, as may be seen from the following example, originally due to Condorcet (1785). Consider a group of three people, Amy, Briony, and Celia. They all wish to go out together, and there are three alternatives before them: they may go fishing, play golf, or go hospital visiting. Amy prefers to fish, but if this is not possible she would rather play golf than go hospital visiting. Briony, the active member of the group, would prefer golfing to the other alternatives. She finds fishing very boring, however, and prefers visiting the hospital to doing that. Celia gains great satisfaction from hospital visiting, however, and prefers this to the other two activities. If she can't exercise her desire, however, she would prefer fishing to playing golf. Their preferences can be displayed in the following table, using an obvious notation. Amy Briony Celia
f > g > h g > h > f h > f > g
Briony suggests that they should determine what to do by voting. She (wisely) suggests that they first determine a group preference between hospital visiting and fishing. Celia joins with her in voting for hospital visiting over fishing, and so fishing is ruled out. Now they vote between golf and hospital visiting; again Briony is in a majority, but this time with Amy, and they vote two to one in favor of golf. Amy is not very happy at this result, and so she suggests, just to confirm matters, that they vote between golfing and fishing. When they do, they find that there is a majority in favor of fishing, which was ruled out at first. Thus by using majority voting the group prefers fishing to golf, golf to hospital visiting, but also hospital visiting to fishing. This is a very simple example; but it shows that, if a group attempts to determine its preferences by simple majority voting, a non-transitive set of preferences may well result; that is, a simple ordering of the options, as is required for utility theory to hold, may not be derived. In passing, note that we often expect a committee to display consistent preferences; Condorcet's paradox shows that, even if each member of a committee displays consistent views, it is perfectly possible for the decisions of the committee to be inconsistent.
108
Theory
Since majority voting is not a satisfactory method for determining group preferences, it is natural to ask what method is satisfactory. This is the question that K. J. Arrow asked, and to which he got the startling reply that there is none. Arrow's impossibility theorem (Arrow 1951), as it has come to be known, was one of the most important contributions made by the social welfare economists, in the years after the Second World War. The general question they asked was whether there were general principles for the allocation of resources which were ethically neutral, in some sense, and could therefore become widely accepted. Arrow's result dealt a great blow to that search. What Arrow did was to establish four reasonable conditions that he felt should be fulfilled by a procedure for determining a group's preferences between a set of options, as a function of the preferences of the group members. Arrow's conditions can be stated as follows: (1) Whatever the preferences of the group members, the method must produce a group preference order for the options being compared. (2) If every member of the group prefers option A to option B, then the group must also prefer A to B. (3) The group choice between two options, A and B, depends only on the preferences of the group members between A and B, and not on their preferences between A and B and any other option. (4) There is no dictator; no individual always gets his way. Arrow proved that there is no aggregation procedure which satisfies these four conditions. The reader is referred to Sen (1970) for a proof of this theorem, and also a wider discussion of these issues. Sen refers to Arrow's theorem as a stunning result, which indeed it is. When first published, it created great interest, not only amongst academics. Idealistic hopes that ethical principles could be established for resolving differences of opinion were dashed. A considerable literature was spawned from this result, endeavoring to rescue the hope of creating satisfactory procedures for aggregating group views. Arrow's result is concerned with the combination of preference orders, and does not use the intensity of preferences of individuals, nor, more significantly, interpersonal comparisons of utility. Much has been written on whether it makes sense to consider interpersonal comparisons of intensity of preference in a normative theory of social choice, even though we clearly take such perceptions into account in practice (see Rawls 1971, for example). The problem is one of honesty. In a community of trust, we will often take the minority view if that minority reports an intense opposition to a proposal, while the majority is mildly in favor. If, as is usually the case, we suspect other members of the group of misreporting their feelings in order to get their way, then we are less likely to accommodate them. Harsanyi, amongst other authors, has studied
Decision analysis in organizations
109
these questions in depth, and we recommend readers to consult Harsanyi (1977) if they wish to pursue them. A particular approach to constructing a group utility function, as a function of the utility functions of individuals, has been put forward by Keeney and Raiffa, and is well described in Chapter 10 of Keeney and Raiffa (1976). They imagine a supradecision-maker, whose job it is to construct a utility function for a group of people, maximizing the well-being of its members. They explore the conditions under which the group utility function can be a weighted sum of the individual utility functions. As Brock (1980) points out, however, in an interesting contribution to this debate, there is no consensus on whether this procedure or others that have been suggested make sense even at the theoretical level. What is more, the enormous practical difficulties that would be associated with any attempt to apply these approaches to real group decision problems suggest that this line of approach will be unproductive as the basis for a practical decision aid. We conclude that there is still no satisfactory quantitative method for determining group preferences by some mathematical operation on the expressed preferences of individuals. So attempts to apply decision theory to an organization by the mathematical aggregation of individual beliefs will encounter difficulties. As we shall demonstrate in Section 5.4, however, this does not mean that decision synthesis for an organization is a lost cause! 5.3.2 Game theory An alternative way of prescribing decision-making behavior in organizations is the theory of games. This has more in common with the political science view of decision-making as the interplay of power between different operators than it does with other views of organizational decision-making. Game theory probably originated in the work of Borel (1938), but received far more attention after the publication of von Neumann and Morgenstern's book in 1944. Much of the literature of the forties and fifties is excellently summarized in Luce and Raiffa (1957); after that time, advances in the theory were slow. Game theory is concerned not only with the decisions that an individual might make but also with those of his adversaries in a social situation. To illustrate the concept of a game, consider perhaps the most famous example, that of the prisoners' dilemma. Luce and Raiffa give a good description of this problem, which we reproduce here: Two suspects are taken into custody and separated. The district attorney is certain that they are guilty of a specific crime, but he does not have adequate evidence to convict them at a trial. He points out to each prisoner that each has two alternatives: to confess to the crime the police are sure they have done, or not to
Theory
no
Prisoner B's choices Not confess
Not confess Prisoner A's choices Confess
1 year each 3 months for A 10 years for B
Confess 10 years for-A 3 months for B 8 years each
Figure 5.1 The prisoners' dilemma.
confess. If they both do not confess, then the district attorney states he will book them on some very minor trumped-up charge such as petty larceny and illegal possession of a weapon, and they will both receive minor punishment; if they both confess they will be prosecuted, but he will recommend less than the most severe sentence; but if one confesses and the other does not, then the confessor will receive lenient treatment for turning state's evidence, whereas the latter will get "the book" slapped at him. Luce and Raiffa 1957, P-95 The situation can be described by the pay-off table in Figure 5.1. The problem for either prisoner lies in guessing what the other prisoner might do. If he confesses, he does better than he would if he did not confess, whatever the other decides to do. He could argue that it would be rational, therefore, to confess. But, if they both argue this way and both confess, they both do worse than if they had agreed not to confess! This is truly a dilemma. The theory of games is set up to analyze situations of this kind. It differs from standard decision theory in modeling the decision behavior of other individuals as well as that of the interested party; it is concerned with occasions when the outcome depends on the decisions made by both parties. In reality social situations are often determined by the decisionmaking of more than two people; game theory recognizes this fact, and the study of w-person games, as they are called, has received a great deal of attention. Thirty years ago, when these ideas were first explored, there was considerable enthusiasm for the potential of game theory for analyzing social decision-making and offering normative procedures for guidance in such situations. Unfortunately, these early hopes have not borne fruit except in certain very limited ways. There are several reasons for this. First, the theory has not led to unequivocal recommendations except in very simple cases. The two-person zero-sum game (that is, a game in which, whatever the outcome, what one person gains is entirely at the
Decision analysis in organizations
111
expense of the other - the sum of their gains is zero) is fully analyzed, and the minimax theorem gives strong support for the player adopting a particular strategy (see Luce and Raiffa 1957, Chapter 4, for a discussion of this theory). Very few other cases are amenable to such analysis, however; in the w-person game, the complexities are such that it is difficult to argue for any particular strategy to be followed. Secondly, since in game theory one has to think about one's opponent's processes of analysis, one has to assume some knowledge of his ability to think rationally and of the information available to him. In practice this is very difficult. Normally it is unsafe to suppose that an opponent will act rationally; as we argue elsewhere in this book, there is much room for improvement in the extent to which people apply rational thought to decision-making. There is also the problem of guessing how much bluffing the opponent will engage in. This is the difficulty of the prisoners' dilemma; game theory offers no solution to it. In summary, then, we conclude that the original enthusiasm for the theory of games as a guide for rational decision-making in social contexts has not been justified. Not only is the theory inadequate to cope with the idealizations that can be made of real problems, but also those idealizations are inadequate representations of the real world. We should not dismiss game theory as a useful normative tool for decision-making in social contexts without mentioning some more recent developments which have led to reports of useful applications. Brams (1978, 1979, 1980) has used game theory to produce some imaginative analyses of historical decisions. Metagame theory (Howard 1971) and hypergame theory (Bennett 1977) both offer an analytical tool for complex multi-actor decision problems. We have not space here to describe either of these ideas, or the connections between them, but both have been used to analyze practical problems. For example, metagames have been used on the Arab-Israeli conflict (Howard 1970) and on an international water dispute (Fraser and Hipel 1980), and hypergames on the fall of France (Bennett and Dando 1979) and soccer hooliganism (Bennett et al. 1980). Neither theory claims to provide clear normative advice to an actor in a conflict; both deal with understanding and description. Thus, while these approaches may well become very useful tools for the description of conflict, they do not yet provide us with prescriptions for action. 5.4 USING DECISION ANALYSIS IN ORGANIZATIONS
So far in this chapter we have discussed what we know of how decisionmaking happens in organizations, and the normative methods that have been explored for such organizational decision-making. As we saw in the
ii2
Theory
last section, these normative theories are not very compelling. Decision theory is used, however, and used successfully, to synthesize decisions in organizational settings. In this section we will explain how this may be done satisfactorily. We will describe how a theory developed for individuals can make sense when applied to organizations. j.4.1 The individual in the organization Although, as we saw in Section 5.2, the instances of important organizational decisions being taken by one person acting alone are rare, it is still the case that an individual may have to decide what line to take in the process that leads to a decision. Thus a manufacturing director might have to decide whether or not to support a proposal from the marketing department that a new product should be launched. An example of this may be found in Section 8.4.2. He may properly use decision analysis to help him to answer this question, although he may want to do so privately. This will allow him to include attributes of importance to him that his colleagues might not share (such as personal promotion prospects, the well-being of the manufacturing department, and so on). Very few executives will be able to afford decision analysts to assist on all problems of this type; only problems of sufficient complexity will warrant the employment of a professional decision analyst (we discuss the question of how much analysis to devote to a decision in Chapter 7). Nonetheless, an increasing proportion of managers are being trained in the ideas of decision theory, and microcomputer programs for structuring and evaluating multi-attribute value functions and decision trees are now becoming available. We expect that increasingly decision analysis will be used by managers to help them think through their own actions in the corporate power struggle. 5.4.2 Designing programmed decisions As we saw in Section 5.2, the distinguishing marks of a bureaucracy include programmed decisions, that is, the rules and procedures that are applied to determine many day-to-day decisions. At some stage, of course, somebody has to determine what these rules should be: decisions have to be made about the design of the programs themselves. In a sense these design decisions are like any other organizational decision; so an attempt to use decision analysis will involve the problem of whose values to insert into the analysis. Despite this, some success has been achieved in helping rule-makers to think through the complicated problems of the effect of different rules on the goals of the organization. For example, in health care management, Alemi et al. (1981) report the use of the framework of
Decision analysis in organizations
113
decision theory to set rules for the allocation of nursing care. They constructed a multi-attribute utility function which was used to compute a score for each patient which determined by rule how much nursing care he was allocated. The weightings in the model were determined by a committee and published. The procedure had the following advantages: because decisions were programmed, time was saved; consistency in the allocation of scarce resources improved fairness; the reasons for the decision, as explicated in the model, were clear for all to see. Such an approach to allocating care may alarm some readers: is not the take-over by the machine or the bureaucracy a short step away from this programmed decision process? We too have doubts about such procedures; if the rule goes against common sense or compassion, may it not lead to unintended evil? It is logically impossible for rule-makers to think of all conceivable eventualities. Despite these doubts, we believe that, if used intelligently, this procedure has a lot to be said for it, since its advantages, which we have listed above, are well worth having. There are close similarities between rule-making in a bureaucracy and designing regulations or laws for public actions. As Keeney (1981) points out, we have to regulate activities such as the disposal of toxic waste or the operation of a nuclear power station in order that the decisions of some actors in society (e.g. public utilities) do not have an unacceptable effect on other actors (e.g. residents near a waste disposal facility). The problem for the regulator is how to determine what appropriate regulations should be. Keeney gives a valuable discussion of this problem and shows how decision analysis may help in the design of regulations. The alternative regulations are the decision variables, and a schematic decision tree can be used to scope out the consequences of these different possible regulations. Utility functions for the actors in the process can then be used to explore the desirability of the different regulations from different points of view, namely those of the utility companies, the affected populations, and so on. Keeney does not recommend any formal method for aggregating these different value judgments; in our view this must be a matter for the political process. We know of no attempts actually to determine regulations by an application of the procedures that Keeney discusses; however, it is clear that, as a framework for clarifying the issues in the formation of regulations, decision analysis has something to offer. 5.4.3 Management decisions in a participative setting We discuss the method of decision conferencing in Section 6.5, and we shall not therefore do more here than give a brief description of the method and reasons for its success. (The literature on the decision conference is not yet very large; one of the few papers which describes the
ii4
Theory
approach is Kraemer and King 1983.) The idea is to take a group of people who are responsible for a decision (such as the board of a company) away from their usual place of work for a two- or three-day period, and to devote the time to solving their problem. An analyst, well-versed in the ideas of decision theory, acts as facilitator to the group and, by exploring with them options, goals, and uncertainties, he is able to help them create what Phillips (1984) refers to as a "shared social reality." This is expressed in a model, involving perhaps a multi-attribute value or utility function or a decision tree, which is produced with the aid of appropriate computer software. We recommend the reader to consult Phillips's article for a thoughtful discussion of this idea. We here quote some of his observations about this approach to organizational decision-making. Experience with decision conferences suggests that the adversarial process helps participants to broaden their individual perspectives on the problem, to change their views, to invent new options acceptable to everyone, in short, to create a model that fairly represents all perspectives. But not even the adversarial process can prevent biases entering the model through the influence of an ever-present variable, the climate of the organization. A pervading complex of norms, values, expectations, the acceptable and the unacceptable, influences decisions throughout the organization, often in helpful ways, but sometimes in unhelpful ways. It is here that the decision analyst's role can provide some correction to the unhelpful effects of climate. First, the experienced analyst who has worked with different organizations has both the perspective and detachment to detect and reflect back many of the biases that climate can introduce. Secondly, an analyst who has minimal investment in the consequences of the decision and who works with problem owners away from their company creates a reasonably neutral territory, thus minimizing the effects of the company's climate. This neutral climate also helps the group to maintain a more balanced view of the problem, thus making them more impervious to manipulation by particularly persuasive individuals who do not have a good case, especially when the analyst, detecting attempts to manipulate the group, asks less vocal participants for their views. Phillips 1984, p.44
Thus in a decision conference, there is no attempt to construct some sort of objectively faithful representation of the problem, and then to apply mathematical techniques to determine the optimum solution, as would be done in some versions of systems analysis. Rather, the constructed model is used as a vehicle for discussing issues between the interested parties. No attempt is made, for example, to determine what the trade-off weights in an additive value function "ought" to be; instead the decision implications of different weights are traced through and, usually, a shared view emerges of what to do (rather than what the weights ought to be). Even if a decision conference does not lead to consensus, it provides all the participants with a clearer understanding of the issues involved and why different members of the group prefer different actions. We see this
Decision analysis in organizations
115
application of decision analysis to organizational decision-making as becoming one of the most important in the future. 5.4.4 Public policy analysis Determining public policy on an issue is a more complicated decisionmaking process, by and large, than is decision-making within a company. Not only are there differences of opinion within the decision-making body (the government and its analysts), but in most countries at least some notice must be taken of public opinion on the issues. While a decision conference might help a group of ministers to clarify issues and to reach consensus (although it must be admitted that we do not know of any attempt to run a decision conference at Cabinet level), the need for analyses of public policy to be open provides further opportunities for the ideas of decision theory to be of use. The best-known technique for analyzing public policy is cost-benefit analysis. Indeed, the US government's Executive Order 12291 requires that cost—benefit analysis be applied to projects of a certain kind. We gave a brief discussion of the relation between this technique and decision analysis in Section 3.5; it is sufficient here to note that, in many areas of public policy analysis, enthusiasm for the approach is on the wane. There are many reasons for this, but one of the most important concerns the determination of trade-offs. Integral to a cost-benefit analysis is the expression of all attributes of importance in monetary terms. This amounts to creating a linear additive value function, where the weights are determined by pricing each attribute. Some economists argue that, by a suitable application of economic theory, this can be done in an objective fashion. Others recognize that there are limits to the objectivity of this approach. If we accept that the proper value for a good is the price that it commands in the market-place, it may be plausible to use such prices to determine social trade-offs. Unfortunately, however, in many cases markets do not exist (who buys and sells radiological hazards thousands of years hence?) and even when they do they are often subject to severe imperfections. We have argued (Watson 1981) that on this point, as on others, decision analysis may be more use than the traditional forms of cost—benefit analysis. In a recent study of nuclear waste management in the United Kingdom, multi-attribute value analysis was used to elucidate the best practicable environmental option for the disposal of radioactive waste (Watson 1985). For a particular kind of waste several different disposal options were available. The distinguishing characteristics for these options included cost and radiological exposure. Attributes were defined to measure these factors for each disposal option, and the options were
n6
Theory
scored with respect to each attribute. A multi-attribute value function would now have been very useful, to place the disposal options in some order. To do so, of course, required trade-off judgments. On the one hand, the analysts felt that these judgments should be supplied by the political process, and not by the analysts as sometimes happens. On the other hand, it was felt to be unreasonable to expect busy decision-makers to devote the time to determining the many trade-off weights that were required; nor would the specification of numerical trade-offs appeal to many politicians (understandably, since clarity of stance removes future potential for flexibility). As a compromise, four different sets of weights were suggested by the analysts, chosen to span their perceptions of the variety of public opinion on these trade-offs. These weights were applied to the measures and produced different rank-orderings of the options, corresponding to each of the four sets of weights. The advantage of this approach was to signal to readers of the report that questions of value were involved in this problem, and that it was improper for analysts to insert their own values. It also allowed the decision-makers to see the implications of different views of the relative importance of the attributes. It would, perhaps, further enhance the democratic process if the actual values of different interested parties were incorporated in the analysis, rather than an analysts's guess at them. This approach was adopted by Chen et al. (1979) in an application of decision analysis to the decision by a local authority on whether or not to install a solid-waste shredding plant. They created a decision tree representation of the problem which all the members of the council agreed was an adequate structure; they disagreed, however, on the trade-offs. The analysts helped each member of the authority to elicit values and to roll back the tree. Information about the views of different members was fed back to all the other members of the decision-making body. The result was not to determine what the decision ought to have been; rather, as is often the case with decision conferences, a shared understanding of the problem emerged, which led to the final decision being made more easily. There have been other similar, but more Utopian, suggestions that the structure of decision analysis could be used to determine social decisions. Both Howard (1975) and Edwards (1977) have suggested a formalization of the process of analysis by legislating for social decisions to be made according to the framework of decision analysis. We doubt that their suggestions were put forward as realistic schemes. They were more discussions of what an ideal might look like and, as such, they deserve our notice. Howard suggests that there should be three bodies of people established for solving the problems of society. The first should be responsible for defining the problems and establishing alternatives for dealing with them. The second would flesh out the consequences of each
Decision analysis in organizations
117
alternative, probabilistically if necessary. The third would determine the preferences of society; members of this body might well be elected, as in a representative democracy. The other two bodies would consist, apparently, of technocrats. We see formidable philosophical and practical objections to the introduction of this scheme; it does not surprise us that we know of no attempt to adopt it in practice. Whatever scheme of decision-making is introduced, there will always be scope for misrepresentation and fixing the system. Nevertheless, the vision of creating a system where the views of the citizens are more easily and more obviously taken into account, and where decision-making is more open and more even-handed, has much to commend it. 5.4.5 Negotiations As we observed in Section 5.2, very often decision-making occurs in organizations as a result of the resolution of conflict. Union-management disputes are an obvious example, but there are many others, such as international disagreements or intra-company arguments. The standard approach to these difficulties is to negotiate, and the art of negotiation is sufficiently refined for some people to devote their livelihoods to it. Recently some interesting applications of decision analysis to enhance the process of negotiation have been reported. Raiffa's very readable book (Raiffa 1982) discusses much more than the role of decision analysis in negotiations, but it does cover these ideas. The reader is recommended to consult some of the original papers for detailed accounts (for example, Barclay and Peterson 1976 or Ulvila and Snider 1980). The idea is quite straightforward, but very useful. A multi-attribute value function is constructed for each side in the negotiations, representing their values. If, as is normal, the analysis is carried out to aid just one of the actors in the negotiations, the values of the opposing side must be guessed; good negotiators are skilled in making such guesses, however. Then the scores of each possible agreement are investigated as determined by these value functions. When plotted on a graph, as in Figure 5.2, various conclusions can be drawn. Some of the options dominate others in the sense that, if both sides agree that the value functions properly capture their value structure, they should agree to exclude the dominated option. In Figure 5.2, for example, agreements E and F are dominated. An analysis of this kind can identify dominated options where this fact is not obvious. Moreover, it can lead to the specification of new potential agreements which are more attractive than the positions being advocated by either party. The method does not suggest which solution on the efficient frontier should be adopted; this must be resolved by negotiation tactics. It would be possible, of course, for either party to undertake a
Theory
n8
Value to second party
Figure 5.2 Decision analysis in negotiations. The value of potential agreements A, 6, C, D, £, F to the different parties. Agreement E is dominated by B, and F is dominated by C. decision analysis of the best tactics to be employed, given a previously defined efficient frontier, but, to our knowledge, this has yet to be attempted. The basic methodology was, however, used by US negotiators in the Panama Canal negotiations of 1974—5 (Raiffa 1982, Chapter 12), by the US side in the Philippine military base negotiations of 1978 (see Ulvila and McDonagh 1978), and in international negotiations on tanker standards (Ulvila and Snider 1980). Somewhat less successful applications have been conducted in the field of industrial relations. The value of this approach to negotiations is as difficult to assess as any application of decision analysis. Users spoke positively, however. It at least helped some of the negotiators to think through the complex web of issues before them; moreover, "inducing staff analysts to formalize their assumptions and tradeoffs helped to generate creativity" (Raiffa 1982, p.185). j.4.6
Conclusions
We have seen that, even though decision theory was constructed as a tool to guide individuals in decision-making, there are contexts in organizational decision-making where the ideas can be of great value. First, individuals still have to decide how they will act, even if the context is
Decision analysis in organizations
119
organizational politics; secondly, the framework of decision theory can provide a useful structure for the construction of programmed decisions; thirdly, the decision conference, based on decision theory, can be very useful for guiding participative management decisions; fourthly, in public policy analysis, the framework of multi-attribute value functions can clarify issues and enhance communication; and, fifthly, decision analysis can be used in negotiations to think through complex positions. We have not given a specific discussion of the role of computers in all these contexts, although the reader will have been aware that this role is important in all of these analytical roles, and vital in some, such as the decision conference or in negotiations. There is, of course, a great deal of discussion at present on the role of the computer in management decision-making. A computer system which not only provides a manager with information about his problem in a way that is of use to him, but also allows the manager to analyze that information effectively, has come to be referred to as a decision support system (DSS). The ideas of decision analysis are, arguably, central to the concept of decision support. Indeed, as we discuss in Section 6.6, one way of doing decision analysis is through a DSS. There is now emerging a literature on how to design a DSS, using not only the insights of decision theory, but also the body of knowledge on how decision-making happens in organizations, which we dipped into in Section 5.1 (see, for example, Sage 1981 or Vari and Vecsenyi 1984). It seems to us that decision analysis will continue to play an important and increasingly central role in organizational decision-making in the future; but it is the interaction of the theory with the abilities of the computer which offers the most exciting prospects.
II PRACTICE
Having addressed the abstract and theoretical issues related to rational individual and organizational decision-making, we now turn to the practice of decision analysis. Chapter 6 addresses the strategic (and stylistic) approaches to implementing decision theory for real problems. In Chapter 7 we present the analytic skills (or tactics) needed to practice decision analysis. Then these skills are illustrated with real applications in Chapter 8. Thefinalchapter discusses the validation of analyses, the ethics involved in practicing decision analysis, and research areas in thefieldthat need attention in the future.
ANALYTIC STRATEGY
6.1
INTRODUCTION
The initial discussion between the decision analyst and the decision-maker is pivotal in determining whether the analyst will get a chance to "practice his wares." During this discussion the analyst must ascertain the nature of the decision, the concerns of the decision-maker, and the problems that he is likely to encounter during the course of the analysis. By the end of the discussion the analyst must have described an analytic process that assures the decision-maker that his concerns will be addressed. During the course of the discussion the analyst must tailor his approach to the decisionmaker and his organization, the nature of the decision, and any potential problems. This tailored process must convince the decision-maker that the process will be worth his while. In a nutshell the analyst must convince the decision-maker that the analytic process can be expected to produce new insights into his decision that will be worth the time and money being spent by the decision-maker. The analyst must always remember that it is these insights (as to which alternative best addresses the decision-maker's uncertainties and values) that the analyst has to offer. And it is the analytic process (in which the decision analysis paradigm is imbedded) that will produce these insights. This chapter is devoted to describing five distinct analytic processes (which we will call strategies) that decision analysts have employed in the pursuit of insight. Before defining and describing these five strategies it will be helpful to examine three distinct modes that typically characterize the nature of decisions. We have observed that decision-makers usually want to do one of the following: (1) CHOOSE one alternative from many, (2) ALLOCATE a scarce resource across many alternative projects, or (3) NEGOTIATE a settlement with one or more other parties. In the CHOOSE mode the decision-maker faces the fewest constraints. His alternatives are finite in number, usually between five and fifteen, 123
124
Practice
some of which may be sequential. This is the mode for which the decision tree was devised and to which decision analysis has been applied most often. The ALLOCATE mode is commonly found in the budgeting environment. This mode can be transformed to the CHOOSE mode when the number of projects is limited in number - that is, all possible combinations of projects satisfying the constraint can be enumerated and analyzed. However, the CHOOSE mode can be easily overwhelmed when even a small number of projects are enumerated, especially when there is some uncertainty as to how scarce the scarce resource really is. Just as the ALLOCATE mode can be transformed into the CHOOSE mode, the reverse transformation (CHOOSE to ALLOCATE) is also possible and is sometimes a wise thing to do. When the decision-maker is in the CHOOSE mode but operating under a constraint, the analyst can often provide tremendous insight by turning the constraint into a scarce resource and analyzing a higher-level decision. In both the CHOOSE and ALLOCATE modes the decision-maker is not in a direct give-and-take situation with other decision-makers. This is not to say that the decisionmaker does not have competitors in the market-place making similar decisions or colleagues fighting for the same scarce resource. The NEGOTIATE mode (introduced in Section 5.4.5) occurs when the decisionmaker is interacting with one (or more) other decision-maker(s) for the purpose of mutually accepting a binding agreement. This mode is the most difficult to analyze rigorously and the one for which decision analysis has contributed the least. Throughout the remainder of this section and sections 6.2-6.5 the CHOOSE mode will be used for illustrative purposes. In Section 6.6 an ALLOCATE application is used for exemplary purposes. Section 6.7 provides^ an overview of the uses of decision analysis to support decision-makers who are in the ALLOCATE and NEGOTIATE modes. The reasons for undertaking a decision analysis are the same as the concerns voiced by a decision-maker as he is explaining why he needs help. So decision analysis helps: (1) when the decision-maker flips between options as he emphasizes different factors (criteria, events) and cannot decide which option provides the best balance across all factors; (2) when different actors in an organization are proponents of different options; (3) when the decision-maker cannot combine recommendations across varying areas of expertise (marketing, production); and (4) when the decision must be justified to superiors or the public. There is no direct correspondence between any of these reasons for analysis and the five strategies about to be defined. Yet each reason (or combination), together with the characteristics of the decision-maker and
Analytic strategy
125
his organization, determines the best strategy for a given situation. In addition, the analyst must always factor his own skills and experience (and those of his colleagues) into the choice of a strategy. Five distinctive strategies (or processes) for conducting a decision analysis can be identified in the literature, and were initiated by different decision analysts: (1) MODELING: constructing a model relating options to values; (2) INTROSPECTING: interrogating the decision-maker thoroughly; (3) RATING: using a simple quantitative, multi-dimensional value model to force thought and discussion; (4) CONFERENCING: directing communication between experts and the decision-maker; and (5) DEVELOPING: building a decision support system (DSS) that guides the decision-maker and enhances his productivity. The above terms (MODELING, INTROSPECTING, etc.) are our inventions for these five strategies and should not be attributed to the initiators of each strategy. We choose to use these terms because they, hopefully, more readily communicate the essence of each strategy than do the names of the initiators. These strategies are not meant to be mutually exclusive; in fact, they can be used in combination with each other. Buede (1979) has earlier referred to the MODELING strategy as an "engineering science" approach to decision analysis, and to the CONFERENCING strategy as a "clinical art" approach. Howard (1980) has divided this strategy space into two schools: modeling and direct assessment. MODELING, INTROSPECTING, and RATING were the first three strategies that were applied. Howard (1968) first described the MODELING strategy by using systems terms: decision and state variables. He was the first to define an iterative process for applying decision analysis. Several decision analysisfirmshave adopted this MODELING strategy as their primary business basis. Keeney and Raiffa (1976) provide several examples of INTROSPECTING, eliciting probability and complex utility functions from the decision-maker and calculating the expected utility of the alternatives. This strategy has also been used extensively by decision analysts. RATING, our abbreviation for the simple multi-attribute rating technique (SMART), was justified by Edwards (1971) on the basis of its simplicity and robustness. SMART was conceived as a response to the frustrations of obtaining answers to complex elicitation questions. Typically additive value models are used to evaluate a well-defined set of alternatives. All three of these strategies date back to the late sixties. As part of a basic research project funded by the US Defense Advanced Research Projects Agency from 1972 to 1978, Cameron Peterson and other analysts at Decisions and Designs Inc. of McLean, Virginia, developed the CONFERENCING strategy (Kuskey 1980, Buede 1982,
126
Practice
Kraemer and King 1983, O'Connor 1985). This strategy emphasizes direct and intense communication between the decision-maker and his appointed experts rather than quantitative models or carefully elicited functions. Decision analysis models, of the type used in the RATING approach, are defined to serve as a platform for dialogue and debate between the conference participants. The goal is to get the group thinking and acting as a unit. Quantitative assessments are made so as to provide a conclusion that the group can challenge, test, and discuss. Thefinalstrategy, DEVELOPING a DSS, was developed independently of decision analysis. Nevertheless, it provides a vehicle for the implementation of decision analysis that is very suitable for some decision-makers and problems. Keen and Scott Morton (1978) defined a DSS as an interactive aid tailored to a decision-maker through an iterative development process. This definition is consistent with the principles of decision analysis as long as the aid utilizes the concepts of probability and value or utility in analyzing options. Now that thefivestrategies have been introduced, the reader can easily see that, while distinct, the strategies still share the basic elements of decision analysis: problem structuring, model development, elicitation of subjective judgments, and analysis. The MODELING strategy emphasizes the second element; INTROSPECTING stresses the elicitation element; and RATING and CONFERENCING provide a balance of all four. INTROSPECTING utilizes a rigorous elicitation session with the analyst and decision-maker interacting by themselves. The RATING and CONFERENCING strategies use a far less rigorous format for elicitation. CONFERENCING operates in a group discussion mode, by definition; while RATING and MODELING strategies usually opt for the one-onone interview format. The DEVELOPING strategy is orthogonal to this discussion since it can use any or all possible formats. While all five of these strategies differ significantly in implementation details, there is one common thread that is the key to success in all five: start simple and iteratively develop a more detailed model and results. Any analyst who forgets this basic principle is certain to have less than satisfactory results: at best, limited insights due to too much detail and complication; at worst, an analysis of the wrong decision. Good advice is never to assume the decision-maker (1) knows exactly what his decision involves and where the boundaries of the decision space are and (2) can specify a set of consistent and complete alternatives. This iterative aproach provides: (1) feedback about which variables of the analysis need more attention and which variables have had too much attention, and (2) opportunities to stop when no further analysis is needed. As we stated at the beginning of this chapter, the value of a decision
Analytic strategy
127
analysis is contained in the new insights provided to the decision-maker. These insights can be a realization that one or two factors lie at the heart of the decision and that values for these factors in a given region favor one option while other values favor another option. Alternatively the insights can be as far-ranging as the identification of a new alternative that had not even been imagined or the acceptance of an alternative that had been considered infeasible. There are two other side benefits that each strategy promotes in its own way: increased and improved communication between the decisionmaker and his experts, and an "audit trail" - that is, a statement of reasons, assumptions, and information on which the decision was based. The next five sections explore each of these strategies in detail and illustrate their methods for providing insights, communication, and an audit trail. 6.2 THE MODELING STRATEGY
6.2.1 Background and definition The MODELING strategy focuses upon the linkage from the alternatives through the uncertainties to the outcomes. Thus the knowledge and information of the decision-maker and his experts are used to quantify not only the uncertainties and the values of the decision-maker but also the process underlying the decision problem that relates the way the alternatives impact these uncertainties and values. As described in Howard (1968) the decision analysis cycle is composed of three analytical phases and an information gathering stage (Figure 6.1). In the first phase a deterministic model of the decision is created so that sensitivity analyses can be used to determine which state variables (those not under the control of the decision-maker) are the most critical in causing one alternative to be preferred to another. Figures 6.2 and 6.3 illustrate the deterministic model and sensitivity analysis, respectively. The decision variables are under the control of the decision-maker and are shown as knobs under the model. The state variables (shown to the left of the model) are set by nature (forces outside the control of the decisionmaker). Figure 6.2 also points out that the value variable(s) can vary with time and that the time preference of the decision-maker may require explicit representation. The application discussed in Section 8.4.2 describes this deterministic phase in detail. The critical state variables from the deterministic sensitivity analysis are then the focus of the second phase. Probability distributions are assessed from experts for these critical state variables and the expected value (utility if risk aversion is present) of each alternative is calculated. This
Practice
128 Prior information
Deterministic phase
Probabilistic phase
New
Informational phase
Decision
Act
Gather new information Information gathering
Figure 6.1 Howard's decision analysis cycle. The first description of iterative analysis in support of a decision-maker. Source: Howard 1968, Figure 3.
0 ,0
o
Value
©
00
0
Decision variables
Figure 6.2 Deterministic model. Prior to introducing uncertainty, the complexity of the decision is modeled. Source: Howard 1968, Figure 4.
computation is depicted in Figure 6.4, in which the decision variables have been collapsed into a vector (d) and the time preference function is part of the model. A value lottery for each alternative setting of d is the output of the model, and the risk preference function produces a single number, the expected value
. Howard uses the bracket notation {v \ d, §} to denote a probability distribution of value given the decision variable
Analytic strategy
129
8 c
Model
2? CD
Tim
a
State variable s,
Figure 6.3 Deterministic sensitivity. Does the uncertainty in the value of the state variable, sh change the recommended alternative? Source: Howard 1968, Figure 5-
and the prior state of information, £. These brackets are also used for the probability assignments to the state variables. As a result of this second phase the MODELING strategy produces a preferred alternative. However, more analysis is possible, namely, the value of additional information. The inforr ^tional phase uses the concept of clairvoyance (or perfect information;, as shown in Figure 6.$ and discussed in Section 3.3.5, to determine whether the decision-maker would be better off collecting more information before acting upon his preferred decision. If this analysis suggests that additional information should be acquired, the decisionmaker enters the fourth phase, information-gathering, and then repeats the entire process with the resulting information.
Practice
130 Probability assignments
Value lottery
f\ Utility
z\
{*/!«
0
{*JB
Figure 6.4 Stochastic analysis. Probability distributions on the critical state variables result in a value lottery for each alternative (d), which is then converted to an expected utility by using the risk preference function. Source: Howard 1968, Figure 7.
Of particular relevance to our definition of the MODELING strategy is Howard's use of systems theory to reflect these phases. This systems theoretical approach to decision analysis clearly asserts that the analyst's primary function is the specification of the "model" and the accompanying decision and state variables. Other tasks include assessing probabilities and time and risk preference functions; but without the model everything stops. And, in fact, this model never exists when the decision analyst begins working with the decision-maker but is synthesized as a result of their communication process. It is during the development of this model that the architecture for communication and insight is built. The model is built within the confines of physical and economic theories to the extent that they apply. However, the analyst often synthesizes a great deal of expert judgment in defining the model that is based upon years of experience with the real world rather than theory. This MODELING strategy enjoys continuous successful application and evolution. Howard (1984a) redefines the initial flow diagram of analytic tasks to that shown in Figure 6.6. The basis is defined as the three
Analytic strategy
131
<(/!*,.,*,,,
O :g. cost of clairvoyance on s,
After s, known,
Figure 6.5 Clairvoyance. A clairvoyant is hypothesized as a means of collecting perfect information on each of the state variables. Source: Howard 1968, Figure 8.
BASIS DEVELOPMENT
t
DETERMINISTIC STRUCTURING
PROBABILISTIC EVALUATION
BASIS APPRAISAL
ITERATION
Figure 6.6 The revised decision analysis cycle. This revision places as much emphasis on the synthesis of values and alternatives as was initially placed on the assessment of uncertainty. Source: Howard 1983, Figure 1.
elements: alternatives, information, and preferences. Informationgathering is now included in basis development. The old informational phase has been renamed basis appraisal, but has been expanded to a reappraisal of alternatives and preferences, as well as information. In summary, the MODELING strategy can be said to use the tenets and axioms of decision analysis in conjunction with a mathematical approximation to the real decision environment in order to provide the decisionmaker with feedback concerning his best decision. The model is used by the analyst to focus communication between the decision-maker and his experts.
132
Practice
Figure 6.7 A decision analysis model of the Mexican electrical system. This is one of the most complex and comprehensive decision models reported in the literature. Source: Matheson 1983, Figure 8.
6.2.2 MODELING application to the Mexican electrical system The first MODELING application we will discuss is a decision analysis model of the Mexican electrical system. This decision analysis was conducted in the sixties by the (now splintered) Decision Analysis Group at SRI at Menlo Park, California, for the Mexican government. The decision-analytic effort adhered to Howard's phases, and was implemented in pilot, prototype, and production stages. It was based upon a computer simulation model of the Mexican electrical system as shown in Figure 6.7 and described by Matheson (1983): The pilot phase demonstrated the need for elaborate models that were capable of capturing the complexities of the electrical system problem. Thus, during the prototype phase, a modular system of computer programs was constructed. This modular system facilitated the implementation of changes that would naturally occur in the transition to the production phase and permits the appropriate module to be easily updated as the nature of the electrical system changes in the future.
Analytic strategy
133
This electrical system model is fed by submodels involving finances, energy, technology, and the market place: The financial model characterizes the terms at which capital is available from both domestic and world financial institutions and markets, as a function of the profitability, debt, and equity of the power utility. The energy model describes the price of all potential fuels - such as oil, natural gas, and uranium - as well as the availability of other energy sources - such as water power - over the time period considered in the analysis. Similarly, the technology model characterizes the availability and prices of various types of generation equipment. Finally, the demand model characterizes the electrical demand growth over time, ideally as a function of the price charged for electrical service. At the bottom of thefigureis the policy stating the conditions under which the first nuclear plant should be installed. Thefigureshows that his policy must be embedded in the general nuclear policy, which in turn is embedded in the system's investment, operating, and pricing policy. In addition, Mexico's nuclear policies were taken into account. The value model for the decision analysis consisted of a monetary system profit as well as the "social profit" made up of numerous attributes such as employment and balance of payments. These social profit attributes were priced out (assigned a monetary value) so that they could be added to the system profit for each alternative. In addition, both time and risk preference were important in this power system planning decision for the Mexican government and so these were modeled as well. This computer-based model was constructed by a team of SRI's decision analysts and Mexico's engineers in about one year. The model has been, and is continuing to be, used by Mexican officials to determine both what type of new power plants should be installed by Mexico and where. 6.2.3 MODELING application to Xerox consumables This decision analysis was conducted by decision analysts (Smallwood and Morris 1980) who were previously with the Xerox Corporation. In this case, a task-force composed of representatives from manufacturing, marketing, planning, finance, facilities, and service organizational units was charged with recommending a decision on the construction of new manufacturing facilities for copier/duplicator consumables. The analysts developed a simulation model to help analyze the decision and served as a communication channel for the many representatives on the task-force. The complexities and scope for this analysis are depicted in the model structure of Figure 6.8. This simulation model consisted of several mathematical submodels, as shown in the boxes of the figure. For example, the consumables demand model was a set of differential
Practice
134
Forecast
COMPETITION MODEL
1ncremental
Comp. price
m
MACHINE MARKET MODEL
Xerox price "
Placements
SHORTAGE EFFECTS MODEL
CONSUMABLES DEMAND MODEL
Competitive demand tract. Comp. capacity
Market reduction
Copy volum<
F 1 N
Comp. Xerox demand , demand bnort age CONSUMABLES Revenues MARKET MODEL
A N C 1 A L
Sales DISASTER MODEL
INVENTORY POLICY
PRODUCTION POLICY
Occurrences
PRODUCTION MODEL
Xero X capac ty
UNSCHEDULED MAINTENANCE MODEL
Incremental service costs
M 0 D
Production costs Inventory costs
m
CAPITAL COST MODEL
Capital costs
Present value cash flow
Figure 6.8 A decision analysis of Xerox copier consumables. This model was designed to take advantage of the expertise of task force members. Source: Smallwood and Morris 1980, Figure 1.
equations. The competition model began as a decision-analytic model of Xerox competitors and was later modified to be a satisficing model. The entire analytical process consisted of a base-case simulation which showed one clearly dominant decision, a series of sensitivity analyses and finally several scenario analyses. The sensitivity and scenario analyses indicated that the dominant decision remained so except under the most extreme cases. 6.2.4 Requirements for success in the MODELING strategy These two MODELING applications are characterized by complex mathematical models of the decision process. The decision analyst would
Analytic strategy
135
claim these complex models were necessary to give both the decision analyst and the decision maker enough insight into his decision so that a "good decision" could be made. Since the main goal of a decision analysis is to provide the decisionmaker with insight into and understanding of the problem at hand, an autonomous decision-maker, receptive and willing to participate in an analytic process, would clearly be best served by the MODELING strategy. In addition, it is easier to develop a complex mathematical model of a decision problem when uncertainty must be explicitly captured and there are only a few well-defined decision alternatives and parameters. The complex mathematical models are well accepted (1) when the experts only understand pieces but have difficulty explaining how the pieces might be combined into a system, and (2) when there are accepted theories applicable to the entire system, such as microeconomics or physics. These experts must be at least comfortable with, and hopefully competent at, mathematics and modeling. 6.3 THE INTROSPECTING STRATEGY 6.3.1 Background and definition The second strategy, INTROSPECTING, is associated with the Harvard Business School, and in particular Howard Raiffa and Ralph Keeney. Keeney (1977) states that "it is sometimes important to instigate a deep, soul-searching process on the part of the decision maker." This is the basic rationale for the INTROSPECTING strategy (and the reason for our descriptive name). The typical INTROSPECTING application revolves around the two major components of the expected utility calculation: a probability density function and a utility (including risk preference) function. Since the insight-producing mechanism of the INTROSPECTING strategy is a reflection on the part of the decision-maker, extreme care is taken in the assessment of the probability and utility functions. Keeney (1977) provides a detailed example of the utility assessment process, which he divides into five parts: (1) familiarization with the terminology and motivation for the assessment — this includes the definition of the attributes and the proper perspective for the decision-maker; (2) verification of the independence assumptions concerning preferences — both value (preferential independence) and risk (utility independence) are examined here; (3) assessment of the trade-offs among attributes - here the scaling constants (or relative importances in moving from the bottom to top
136
Practice
of each attribute scale) are determined. First the decision-maker rank-orders the attribute swings, and then each swing is assessed to be a percentage of another, more important, swing. Finally a lottery question must be answered to size one of the scaling constants; (4) assessment of the individual utility functions - this part begins with complex lottery questions aimed at determining the form (additive, multiplicative or other) of the overall utility function. Then the individual utility functions are elicited using many simpler lottery questions. "Curve-fitting" techniques are often used to specify the functions; and (5) checking for consistency and modification — as with any intensive thought process and consistent with the decision-analytic philosophy, several iterations are needed to provide time for reflection and gathering outside opinions. In summary this utility elicitation process is very structured, is typically conducted one-on-one (with an analyst interrogating the decision-maker), permits outside opinions to be sought between sessions, and uses statistical "curve-fitting" to make use of redundant data obtained in assessing the individual utility functions. Probability assessment has benefited from considerable psychological research, as described previously in Section 4.2. Spetzler and Stael von Holstein (1975) n a v e described a commonly accepted, well-structured probability assessment process (see Section 7.4). The five steps in the process (motivating, structuring, conditioning, encoding, and verifying) emphasize the minimization of biases operating on the decision-maker and assume that the probability model (defining probabilistic dependences) has already been established. The above summary of utility elicitation applies to this process as well. The insights produced for the decision-maker by the INTROSPECTING strategy often occur during the elicitation process(es) as the decision-maker synthesizes his answers to the assessment questions. The expected utility calculation provides a quantitative evaluation of the options, and the analyst should provide an interpretation as to why each option is evaluated high or low. This interpretation is developed by tracing back through the expected utility calculation to produce explanations such as: - option A is expected to perform well only on attributes that are unimportant; - option B is expected to do moderately well across all attributes; - option C will do very well on the two most important attributes. These interpretations should provide a final consistency check with the decision-maker's intuition.
Analytic strategy
137
Table 6.1 Attributes for evaluating energy policy Level" Attribute Xi, fatalities X2, permanent land use X 3 , temporary land use X 4 , water evaporated X 5 , pollution X 6 , particulate pollution X 7 , thermal energy needed X 8) radioactive waste X 9 , nuclear safeguards X 10 , chronic effects X n , electricity generated
Measure Deaths Acres i o 3 acres i o 1 2 gallons io 6 tons io 6 tons 12
i o kW hr (thermal) Metric tons Tons of plutonium produced Tons of lead i o 1 2 kW hr (electric)
Worst 700 2,000 200
Best 100 0 10
i-5 80
0.5
10
0.2
6 200
50 2,000 0.5
5 3 0 0 0
3
a
Note: The worst and best levels are the key to ranking the attributes and assigning weights to their importance. Source: Table 1 of Keeney 1977.
6.3.2 INTROSPECTING strategy for electrical energy production in Wisconsin This first application of the INTROSPECTING approach is the basis for Keeney (1977) and provides a detailed illustration of the utility assessment process. For this reason it is longer than the other discussions of applications in this chapter. This application examined several alternatives for producing electrical energy in Wisconsin between the midseventies and the year 2000. The identification of energy objectives and eleven attributes was the first step. The eleven attributes that were identified are shown in Table 6.1. In order to evaluate the energy alternatives on these eleven attributes, the generation of energy via each alternative had to be traced from beginning to end. Included in this trace was obtaining, transporting, using, and disposing of the fuel. Buehring (1975) describes such an evaluation. Our discussion here will focus on the assessment of Dr Buehring's utility function. This assessment took place in two sessions. Thefiveparts of the assessment process were completed in the first session. After reviewing the definitions of the eleven attributes was addressed. Charts such as those shown in Figure 6.9 were used to determine that all attributes were preferentially independent. Next utility independence was examined and verified. This was accomplished by comparing the certainty equivalent on lotteries over one particular attribute, first holding all other attributes fixed at their worst levels and then at their best. Since the two certainty equivalents were equal, utility independence pertained.
Practice
i38
Attributes X 1# X 4 , . . . , X n are fixed at their best levels
2000 f-C
Points B' and D' are indifferent
™ 1000
A' 10
200
100 X 3 ( 1 0 3 acres) Temporary land use
Temporary versus permanent land use: other attributes at best level.
%
2000
Attributes X l f X 4 , . . . , X n are fixed at their worst levels
5*8 1500 g ~ 1000
10
Points B and D are indifferent
100
200
X 3 ( 1 0 3 acres) Temporary land use Temporary versus permanent land use: other attributes at worst level.
Figure 6.9 Preferential independence. Since the preference between temporary and permanent land use did not change as all of the other attributes were varied from worst to best, these two attributes pass the preferential independence test. Source: Keeney 1977, Figures 3 and 4.
Ordering and assessing the scaling constants for the eleven attributes was next. The initial preference ordering of the value swings from worst to best on each attribute was: fatalities chronic effects electricity generated nuclear safeguards SO2 pollution radioactive waste permanent land use temporary land use water evaporated
Analytic strategy
139
Option B
Option A
All attributes at best levels
Fatalities at best level, 100 deaths. All other attributes at worst levels (see Table 1)
All attributes at worst levels
Assessing the indifference probability For p = 0.6 options A and B were indifferent; this sets the scaling constant of fatalities at 0.6.
Option II
Option I
0 x
' io= 2000)
(X, = 700, X 10 = 2000)
700,X 1 0 =0)
( X , measured in deaths and X l o in tons of lead) Preferences for combinations of fatalities and tons of lead Options I and II were found to be indifferent, suggesting that the multi-attribute utility function was additive and not multiplicative.
Figure 6.10 and 19.
Multi-attributed lottery questions. Source: Keeney 1977, Figures 18
particulate pollution thermal energy needed Dr Buehring then answered a lottery question involving all eleven attributes that pegged his scaling constant for fatalities at 0.6 (see Figure 6.10). A second type of lottery was used to determine whether the overall utility function should be additive or multiplicative as shown in Figure 6.10. Dr Buehring's responses justified the use of an additive function. Finally the individual utility functions were assessed (Figure 6.11).
Between the first and second assessment sessions Dr Buehring assessed the utility function for a colleague, Dr Foell. Using what he learned during his interaction with Dr Foell, he made several revisions in the second session. First his ordering of attributes changed as shown below:
Practice
140
700
400
x,
100
2000
1000
x2
0
Revised 1.5
1.05
x4
0
80
50
5
10
6
0.2 X,
X5
U7
6
4.5
3
200
100
X7
0
50
35
0 Xo
X«
" Revised
U,,
2000
1100 X, o
Figure 6.11
0
0.5
1.25
3.0 X,,
The individual utility functions. Source: Keeney 1977, Figure 21.
First order
fatalities chronic effects electricity generated nuclear safeguards SO2 pollution radioactive waste permanent land use temporary land use water evaporated particulate pollution thermal energy needed
Second order chronic effects fatalities nuclear safeguards SO2 pollution radioactive waste electricity generated temporary land use permanent land use water evaporated particulate pollution thermal energy needed
Table 6.2 Final utility function The single-attribute utility functions Range Attribute
u,
Unit of measurement for x,
X|, fatalities X 2 , permanent land use X.*, temporary land use X 4 , water evaporated X 5 , SO 2 pollution X 6 , particulate pollution X 7 , thermal energy needed X x , radioactive waste X 9 , nuclear safeguards Xio, chronic effects X| 1, electricity generated
U\(x\) = (700 — x 1 )/6oo M,(JC,) = (2,000 -xJ/2.,O « ^ ( J C O = 1 . 4 9 6 - 0.466 e<)()()5xlv'> u4{x4) = 1 . 7 8 4 - 0 . 5 2 0 e 0 0 " ^ « 5 (*5) = 1 . 7 8 4 - 0 . 7 4 2 e (U)ll - v * u6)X(,) = ( i o - x 6 ) / 9 . 8 e(U)S92^ M7 (^7) = 4 . ^ 0 - 2 . 4 9 5 UH(XH) = (2OO-X S )/2OO () ()3A Vy W9 (JC 9 ) = 1 . 1 9 8 - 0 . 1 9 8 e Wio(X|oj — 3.017 —2.017 e « M ( X I I ) = l -O39 - 2.003 e"'- n - V v
Deaths Acres TO^ acres TO 12 gallons TO6 tons TO6 tons TO 12 kWhr (thermal) Metric tons Tons of plutonium Tons of lead TO 12 kWhr (electric)
Source: Table 5 of Keeney T977.
Worst 700
Best
2,000
TOO O
200
TO
M
80 TO
6 200
5O
2,000 0.5
O.5
5
0.2
3 0 0 0
3
Table 6.2 (cont.) Solving for the scaling constants0 ki
Relationship to ki 0
k, equals
ki k9 k5
ki = k 10 «io(500) k9 = k5 =
o.768k10
kg
kn
kg = k 5 « 5 (io) = k 10 «io(i,70o)« 5 (io) ku = k5« 5(2o) = k10«io(i,70o)« 5(2.o) k3 = k5« 5(6o) = k10«io(i,70o)« 5(6o)
k4 k7 = k 3 « 3 (i25) = k6 =
o.i77k 10 o.i52k 10
2.953k1
Value of kf 0.339 0.266 0.152 0.060 0.057 0.051 0.021 0.018 0.016 O.OII 0.008 2 = 1.0
Note: ^Solving 2 k, = 2.953k 10 = 1 yields ki 0 = 0.339, from which the other k,s are evaluated. Source: Table 6 of Keeney 1977.
Analytic strategy
$40 M
$110 M
$4M
(a) X, = INITIAL COST
$500 M
$960 M
(c) X.. = COSTS IN FAR FUTURE
U 4 (X 4 )
(b) X . • COSTS IN INTERMEDIATE FUTURE
143
0.5
-*
(d)
X
4 = AUDIBLE NOISE COMPLAINTS
Figure 6.12 Transmission utility functions. These utility functions were assessed using the lottery techniques described earlier in this section. Taken from Crawford et al. 1978, Figure 2.
New scaling constants were assessed as a result of these changes. Checks were made to verify that an additive function was still appropriate. Several of the individual utility functions were revised, as shown in Figure 6.11. The final utility function is defined in Table 6.2. 6.3.3 INTROSPECTING strategy for transmission conductor selection Crawford, Huntzinger, and Kirkwood (1978) u s e d t n e INTROSPECTING strategy for a management unit of the Consumers Power Company. A 765 kV transmission system was being planned in the mid 1970s for the mid 1980s. Previous analyses had identified a set of eleven options that satisfied requirements for reliability, safety, insulation, radio and television interference, and other environmental effects. The primary considerations remaining consisted of costs in three time periods and potential audible noise problems. The options varied the conductor size, the number of conductors per bundle, and the tower type (regular or prebuilt) of the transmission system. The four value attributes were: Xi: initial outlay, in year o dollars; Xz: cost of losses plus tax annuity during the first twenty years of operation, in year o dollars; X3: cost of losses plus tax annuity plus reconductoring cost (if any) during the years 21 through 50, in year o dollars; and X4: number of audible noise complaints officially received during the first three years of line operation.
Practice Table 6.3 Expected utilities
144
Conductor/number Tower* per bundle
4-2,156 6-954 6-954
4—1,780 4-1,590
4-954
4-1,431 4-i,43i 4-i,35i 4-i,35i
4-954
Regular Regular Prebuilt Regular/Prebuilt Regular/Prebuilt Regular Regular Prebuilt Regular Prebuilt Prebuilt
Expected utility
Certainty equivalent for X! with X 2 , X 3 , and X 4 held constant (in millions of year 0 dollars)
0.908 0.908 0.904 0.900 0.900 0.897 0.895 0.894 0.894 0.893 0.892
50.0 50.0 53.2 57-1 57-1 59-7 61.5 62.3 62.3 63.3 64.2
Note: a The regular towers always outperformed the prebuilt towers. Source: Table III of Crawford et al. 1978.
A probability distribution was directly assessed from the company expert on audible noice for the fourth attribute. To obtain the distribution on initial cost, probability distributions on costs of material and labor and on escalation rates were assessed from other experts. Probability distributions on energy costs, line loading, and delay costs were assessed to obtain the distributions on X3 and X4. All of these distributions were discretized so that a decision tree could be used to calculate the expected utilities of the eleven options. The utility assessment process verified preferential and utility independence. The four individual utility functions were nearly linear, as shown in Figure 6.12. The multiplicative form of the utility function was determined to be appropriate. Table 6.3 provides the results of the analysis. Extensive sensitivity analyses yielded the result that prebuilt towers were never preferred to regular towers, and that the two options with the highest expected utility were also the most robust. 6.3.4 Requirements for success with the INTROSPECTING strategy Clearly, a well-defined decision-maker is a prerequisite for the INTROSPECTING strategy. Alternatively, an individual who feels responsible for making an independent recommendation can be effectively helped to synthesize his position with this approach. Diverse expertise can be incorporated via probability distributions on critical factors, but this is done analytically (the second example is a good illustration of this). Any modeling is limited to decision tree calculations.
Analytic strategy
145
This strategy is characterized by the extreme care taken in the elicitation process. First the form of the utility function demands that very complex questions be answered by the decision-maker. Second, this approach is best justified when "deep soul-searching" on the part of the decisionmaker is likely to synthesize the insights needed for the decision. Decisions that are often approached via INTROSPECTING are: - policy decisions that require the responsible official to balance (with justification) several sensitive value attributes; — technical decisions, especially those for which the available options have already been filtered. 6.4 THE RATING STRATEGY 6.4.1 Background and definition SMART was first proposed by Edwards (1971) as an alternative to the more complex approach described as INTROSPECTING, above. A simple weighted average calculation is used to compute the relative value of the options being considered by the decision-maker. The relative simplicity of the RATING questions being posed, combined with the general robustness of additive models, hopefully counterbalances the loss of precision provided by the INSTROSPECTING elicitation. The RATING insights are generated simply by structuring the problem and quantifying the values of alternatives in a systematic way. The fact that both of these strategies have been applied successfully for roughly fifteen years suggests that both have a place in the analyst's toolbag. The following ten steps of the SMART process are taken from Gardiner and Edwards (1975): Step 1: Identify the person or organization whose utilities are to be maximized. (SMART is often applied to problems in which multiple parties have a stake in the decision, and the utilities of each party are elicited.) Step 2: Identify the issue or issues (that is, decisions) to which the utilities needed are relevant. Step 3: Identify the entities to be evaluated. (Entities refer to the decision alternatives.) Step 4: Identify the relevant dimensions of value. (Dimensions are what we call attributes. The initial SMART formulation strongly recommended that the number of dimensions be held to less than fifteen.) Step 5: Rank the dimensions in order of importance. (This initial SMART formulation focused on general importance weights for the attributes, not the more rigorous swing weights. See Section 7-5-4-)
146 Step 6: Step 7: Step 8:
Step 9:
Practice Rate dimensions in importance, preserving ratios. (In the initial formulation, the least important attribute was assigned a value of 10, and the others were scaled upwards.) Sum the importance weights, divide each by the sum, and multiply by 100. (This is the first purely computational step.) Measure the location of each entity being evaluated on each dimension. (Measure is often interpreted as scoring, and the approach is to assign each alternative a score between o and 100 on each dimension or attribute.) Calculate utilities (values) for entities. (Here the linear additive formula,
is used to calculate the values of the ith entity across the j dimensions.) Step 10: Decide. (Actually, at this point we need to review what we have done, and repeat the entire process at least one more time.) The most common application arena of this initial SMART formulation is to social problems, often set in a political context. This RATING application provides an explicit representation of the disagreements between participants. Variants of the RATING strategy include (i) a larger, hierarchical representation of objectives and attributes (between twenty and several thousand), (2) detailed definitions and value curves on every attribute prior to the weighting steps, and (3) careful elicitation of swing weights, often from the bottom level of attributes upwards. Section 7.5 describes this process in detail. Significant numbers of government, and some commercial, applications have been made with these variations. The two applications of the CONFERENCING strategy that are described in section 6.5 used the first and third variations of SMART, so only one RATING application will be discussed here. 6.4.2 RATING application to a research program In the fall of 1972 three decision analysts (Guttentag, Snapper, and Edwards) used the RATING strategy to support John Busa and the Office of Child Development (OCD) of the US Department of Health, Education, and Welfare (now the Department of Health and Human Services) in planning a research program. The research program, which was to be implemented in July of 1973, was action-oriented in promoting child welfare and dealing with specific problems faced by children. This application is documented in Guttentag and Snapper (1974) and in Edwards (1977).
Analytic strategy
147
The major problems faced by the analysts in conducting their analysis were exactly those for which the RATING strategy is best suited: (1) an undefined and multi-faceted value structure for choosing between the alternatives; (2) no systematic or theoretical relationships between the alternatives and the value functions; (3) a decision-making organization with no clear-cut decision-maker, but populated with numerous experts whose contributions are needed and sought for the final decision; and (4) a diffuse, poorly-defined set of alternatives requiring a broad set of value attributes. Nonetheless, analytical progress was difficult and improved OCD awareness of value conflicts amongst its staff members with a general sense of its aggregated values is cited as the most important and useful outcome of the project. Edwards (1977) states that steps 1-3 of the RATING strategy were hastily carried out, and as a result had to be repeated later. The initial emphasis of the analysis was the definition and weighting of value attributes. Two major goals for supporting research projects were identified as "benefits to children and families" and "benefits to OCD as an organization." A group brainstorming session was held to suggest attributes for each of these goals, resulting in thirty-five to forty for each. These lists were reduced by recognizing redundancies and scratching unimportant attributes. A total of fourteen attributes were finally used to represent both goals. Examples of these attributes are: (A) the extent to which a recommended activity is consonant with administration and departmental policies and philosophy, or reflects prevailing public and social thinking; (B) the extent to which a recommended activity is likely to make public leadership more sensitive to the needs of children; (C) the extent to which a recommended activity is likely to promote the individualization of services or programs, or is likely to contribute to this goal; and (D) the extent to which a recommended activity is likely to contribute to making the public and institutions more sensitive to the developmental needs of children. Thirteen OCD individuals were queried via questionnaire to obtain their own weights for these attributes. Examination of the responses showed considerable disagreement. So, rather than averaging the results, the Acting Director of OCD assigned the final importance weights. The first multi-attribute analysis for this application examined fifty-six research recommendations, created specifically for this effort. Three OCD members scored each of the fifty-six alternatives on all fourteen attributes,
148
Practice
using a o to 1,000 scale. The inter-judge reliability was quite good so averages were used to obtain a single score. Ten of the recommendations were then eliminated as having no real value. Finally a benefit-to-cost ratio was calculated for each of the remaining recommendations after the values had been converted to a ratio scale. Unfortunately, a review of the recommendations showed that they were too specific and not complete enough to support the decision-making process. So step 3 was revisited and a comprehensive taxonomy of research areas was developed. This taxonomy was a shorter list of general research foci. Now each general focus was evaluated using only the five most important attributes. The resultant numerical evaluations were used to guide the allocation of funds to each of the general research foci. 6.4.3 Requirements for success in the RATING strategy Since the RATING strategy focuses on the value trade-offs being made by the decision-maker, uncertainty cannot be a major factor in the decision. For many more decisions than one would suspect the decision-maker is most troubled by questions of value. This is especially true in government or social policy settings where profit is not available as an objective function. Limited types of uncertainty can be incorporated in the value hierarchy as conditioning scenarios (e.g. low versus high inflation). The probabilities for these scenarios are incorporated into the weights and cannot be dependent upon the alternatives being evaluated. The RATING strategy works well with groups, as we will illustrate in the next section. There are many decision-makers and experts who are very naive, if not distrustful, of complex mathematics. Both the MODELING and the INTROSPECTING strategies are doomed to failure with these individuals because they will not believe any of the quantitative results, let along understand the value in answering the questions being asked. 6.5 THE CONFERENCING STRATEGY
6. J.I
Background and definition
The CONFERENCING strategy uses a model that is more structural than mathematical for the purpose of displaying the dimensions of value and uncertainty to a decision-making group in a systematic way. The group can then discuss, probe, and place in proper perspective each aspect of the decision. Thus, the CONFERENCING strategy can be summarized as the synthesis and use of a structured process for dialog and debate amongst the decision-maker and his experts during an intense, short-term confer-
Analytic strategy
149
ence. These conferences are best when held away from the usual workplace of the decision-maker since total involvement is critical. The objective of a decision conference is to focus group attention on the problem at hand, to identify and evaluate a set of meaningful and practical options, and to synthesize a group consensus in favor of a single option. The achievement of this objective generates a common understanding of the issues and priorities involved, the underlying rationale for the chosen course of action, and a clear plan for implementation. This common understanding for and agreement to a single course of action is considered the most valuable quality of the conference. Decision-making groups have stated that a greater willingness exists within the group to carry out the decision as a result of the conference. The conference provides everyone with a chance to state his case. Some decisions can be handled adequately in a two-day conference; others require three days or a sequence of two- to three-day conferences. Trial and error have shown that a one-day (eight to ten hours) conference is almost a sure failure, unless a previous analysis has been completed and the day is being devoted to updating this previous analysis with new information. The decision-analytic models must be simplistic or the group will get bogged down in modeling issues rather than substantive issues. The models are used to structure the debate and are critical for focusing the group's attention. It is the model that permits both the lead analyst and the group itself to minimize any negative effects of "groupthink." The models assure the group that all issues have been addressed and permit everyone a chance to influence the importance assigned to each issue. Two to four analysts are required for such a conference. The lead analyst acts as a facilitator, eliciting the group's purpose, problem definition, options, value structure, and other important information. The other analyst(s) operate modeling software, take notes, and assemble summary documents and visual aids. Sometimes a summary document is delivered to the group as they are leaving the conference. One of the more sophisticated conference facilities consists of a large, round table around which the ten to twelve key decision-makers/experts sit. Additional seating is provided for the remaining participants (conferences with as many as forty participants have been conducted althoughfifteento twenty is the preferred upper limit). In addition, there is usually about a hundred square feet of "white-board" space on which the facilitator defines the problem and the decision-analytic structure. This white-board serves as the group's short-term memory and allows the facilitator to keep the group focused on a specific portion of the analysis. A large-screen video projector is connected to the computer so that the analytic results can be presented to the group with only one focal point, minimizing the distraction of paper copies and side conversations. In addition, overhead
150
Practice
and slide projectors are available for presentation of data from previous analyses. In summary, the tenets of decision analysis are maintained in the CONFERENCING strategy and are used to develop a simple structural model of the factors entering into the decision. However, the decision process and the full range of analysis (including time and risk preference and explicit uncertainties) are often downplayed. The result is often a strong group commitment to the chosen course of action via a process that was designed to time with no guarantee of optimality. 6.5.2 CONFERENCING strategy for tanker safety The first application of the CONFERENCING approach that we will discuss is one in which decision analysts (Peterson, Barclay, and Ulvila) assisted the US Coast Guard and the US Department of Transportation in an international negotiation on tanker safety and pollution prevention. This international negotiation was completed in February 1978 as result of demands by the United States that stricter regulations be developed to prevent the pollution and safety hazards associated with oil-tankers. There were about one hundred nations involved in this negotiation as well as many alternatives that included such things as segregated ballast, double bottoms, and so on. Finally, there were numerous criteria that could be used to distinguish good treaties from bad treaties for each country. Due to the large number of countries involved in this negotiation, the analysis was approached from the CHOOSE mode, and made use of a multi-attribute (RATING) model. The first iteration of this decision analysis involved developing a quantitative scale for each of twelve criteria so that any negotiation alternative could be scored on twelve scales. Then two initial alternative treaties, status quo and President Carter's initiative, were scored on each of these twelve criteria. Last, value weights for each country were synthesized so that each of the two alternatives could be evaluated from the point of view of each of the many countries. The second iteration involved getting better value weights for each country, developing new alternatives, and structuring two submodels for world oil and oil in own waters so that better evaluations could be made for the alternatives on these two criteria. (Note that these models are a slight departure from the CONFERENCING approach made possible by a series of conferences.) The improved value weights were generated by having Coast Guard representatives enter into bilateral discussions with representatives of the other nations to find out what issues those nations felt were particularly critical in their preparation for the negotiation. In addition to gathering these value weights, the Coast Guard representa-
Analytic strategy
151
tives also queries these representatives from the other countries about alternatives that these countries were considering as potential treaties. At the end of the second iteration the two new oil models and the new value weights for each country were used to evaluate the new potential treaties. An examination was conducted to see which types of potential treaties might be most attractive across the broad spectrum of countries involved in the negotiation. The third iteration began by having the Coast Guard communicate the results of its analyses to various US interest groups such as environmentalist and industrial representatives. Feedback from these interest groups provided the Coast Guard representatives information that was useful for fine tuning the US value weights as well as the value weights of other countries. In addition, it provided feedback on the relative attractiveness of the alternatives in terms of the twelve criteria that were being used. The result of this decision-analytic effort was a well-prepared negotiation team that was able to concentrate on the tactics involved in the negotiation. This application is described in more detail by Ulvila and Snider (1980). 6.5.3 CONFERENCING strategy for factory location The second CONFERENCING application to be discussed was conducted by Buede and Peterson at Decisions and Designs Inc. of McLean, Virginia, for a commercial client wishing to remain anonymous. This project was completed in a two-day conference during 1980. To safeguard the client's anonymity this description is disguised in terms of potential factory locations as well as the exact product being produced and sold. The first morning of the two-day conference was devoted to specifying some new factory alternatives (Table 6.4). The firm sold products consisting of a book plus either component A or component B assembled in a case. At the time of the analysis the products were assembled in two countries on two separate continents, Europe and Asia. For proprietary reasons these countries will be referred to as EURX and ASIAX. The books, component B, and the cases were bought in ASIAX for those finished products assembled in ASIAX. All of the component As were made in the firm's aged factory in EURX. The cases and component Bs were also products of the EURX factory. The books were bought in both EURX and ASIAX. The other alternatives were based upon: (1) building a large new facility in EURX to replace the aged one, (2) building a large new facility in another European country (EURY) to replace the aged one in EURX and take advantage of cheaper labor rates, (3) shifting some of the EURX efforts to ASIAX and closing most of the EURX factory, and (4) adding another regional capability on the North American continent.
Practice
152
Table 6.4 Initial alternatives Alternatives"
Books
Component A
Component B
Cases
Assembly
Status quo
Make in EURX Make in EURX Make in EURX Make in EURX Make and buy Buy in EURX Buy in ASIAX Buy in ASIAX in EURX and and ASIAX ASIAX
Large new facility in EURX
Buy in EURX
Buy in EURX
Make in EURX Make in EURX Make and buy Buy in ASIAX Buy in ASIAX in EURX and ASIAX
Large new facility in EURY
Buy in EURY
Buy in EURY
Make in EURY Make in EURY Make in EURY
Centralized in ASIAX
Buy in Far East
Buy in EURX
Buy in ASIAX
Make or buy in EURX and ASIAX
Make or buy in EURX and ASIAX
3 regions
Buy from least expensive source
Buy in EURX
Buy in ASIAX
Buy in 3 regions
Buy in 3 regions
Note: a These alternatives were developed in the first two hours of the conference. Source: Table 1 of Buede 1986.
The next step in the two-day conference was the identification of value attributes that discriminated between the previously defined alternatives. This was done in a two-phased process: initially the group of decisionmakers were asked to specify differences between the alternatives in a free-wheeling fashion and then the decision analyst defined and structured these differences hierarchically into attributes and objectives. This structure is shown in Figure 6.13. By the end of the first day the initial analysis was completed. This analysis used a relative value scale (o points of value for the worst alternative and 100 points for the best) for each attribute. (The case study in Section 8.4.3 describes this approach in more detail.) Finally each attribute was weighted in terms of its change in value between the best and worst alternatives. (Note that preferential independence was assumed between all the the attributes so that it was proper to assume an additive value model. This type of simplification is often necessary in the CONFERENCING strategy.) As a result of this preliminary analysis the alternatives were modified and the value structure refined. The initial "status quo" alternative was considered inadequate and two variations to it were synthesized (Table 6.5). Both were improved by changing the source from which books were purchased. In addition, the second "status quo" alternative consolidated two EURX facilities into one existing factory, rather than building a large
Analytic strategy
153
— Start up $
i— Product $ - Costs- Shipping-
— Books — Component A Component B — Cases — Assembly - Inputs Final product
L-Ongoing $ -
• Inventory - Management • Import duties
Total _ value ~
• Taxes - Product -
r~ Quality Continuity
- Stability-
• Materials -Political • Financial • Labor
- Flexibility-
— Technical improvement — Production — Market sensitivity *— Communication
L— Corporate image
Figure 6.13 Initial value structure. This initial set of attributes focused primarily on the costs of the alternatives. Source: Buede 1986, Figure 3.
new facility in EURX. The analysis of the first day convinced the group that a large new facility in EURX was not worth further analysis. The large new facility option in EURY remained, but was renamed "Consolidate in EURY." A second EURY alternative without major consolidation was added. Finally the initial three-region alternative was separated into two distinct options, one for tax advantages and one for marketing and management flexibility. Next the value structure was revised as shown in Figure 6.14. Comparison of figures 6.13 and 6.14 shows that attributes such as transition effort and interruption were added; others, such as the subdivision of product costs into books through assembly, were deleted on the basis of insufficient discrimination between the alternatives; and still others, such as image, taxes, and communication, were relocated. By the end of the second day the analysis of the alternatives in Table 6.5 with the value structure in Figure 6.14 was completed. The decisionmakers decided to begin the implementation of one of the specified alternatives with the intention of ultimately transitioning into one of the three-region alternatives.
Practice
154
Table 6.5 Final alternatives Alternatives'1
Books
Status quo with Buy from least improvements expensive source
Component A
Component B
Cases
Assembly
Make in EURX Make in EURX Make in EURX Make and buy in Buy in ASIAX Buy in ASIAX EURX and ASIAX
Status quo consolidation (1 factory, buy A)
Buy from least expensive source
Buy in EURZ
Make in EURX Make in EURX Make and buy in Buy in ASIAX Buy in ASIAX EURX and ASIAX
Large new facility in EURY
Buy from least expensive source
Buy in EURY
Make in EURY Make in EURY Make in EURY Buy in ASIAX Buy in ASIAX Make and buy in ASIAX
Consolidate in EURY
Buy in EURY
Buy in EURY
Make in EURY Make in EURY Make in EURY
3 regions for tax advantages
Buy from least expensive source
Buy in EURZ
Buy in ASIAX
Buy in 3 regions Make in 3 regions
3 regions for marketing and management
Buy from least expensive source
Buy in EURZ
Buy in ASIAX
Buy in 3 regions Buy in 3 regions
Note: a These alternatives were synthesized on the second morning of the conference using the results of the initial analysis. Source: Table 6.5 of Buede 1986.
6.5.4 Requirements for success with the CONFERENCING strategy The CONFERENCING strategy works when the decision-maker is either a very specific individual or a group (e.g. a task-force). In fact, experience has shown that this strategy works best when the decision-maker is a clearly defined individual who, while being advised by the group, can be asked to break up debates during the conference that may not be meaningful. In order to use the CONFERENCING strategy the single decision-maker or group leader must want to foster open discussion amongst the participants. A couple of conferences have been held for decision-makers who wanted to use this opportunity to communicate their own positions. In each case the conference sputtered and failed since no decisions were formulated and the participants felt misled as they were expecting the opportunity to voice their opinions. Organizations trying to promote participative management have made extensive use of CONFERENCING. This strategy is best suited to decisions requiring inputs from across the whole organization. These broad-ranging problems typically are most difficult to model. This strategy typically devotes a much larger percentage
Analytic strategy
155
• Start up $
—i Transition
-Difficulty
r Effort — • Interruption • Product • Shipping - Storage - Managed (overhead) Import duties
- Ongoing $ -
- Taxes
- Effectiveness -
- Management resources - Communication -R&D - Product quality -Continuity - Market expansion - Image
-Flexibility-
• Technical - Product - Market responsiveness
— Stability-
•Political - Currency -Material -Labor
Total _ value"
Figure 6.14 Final value structure. Non-cost attributes were the dominant feature of the final value structure. Source: Buede 1986, Figure 4.
of the analyst's time to structuring the alternatives than the other strategies do. CONFERENCING provides the organization with an opportunity for structured soul-searching. 6.6
THE
DEVELOPING
STRATEGY
6.6.1 Background and definition The DEVELOPING strategy involves the structuring and implementation of a computer-based decision support system (DSS) for the decisionmaker's use. This strategy is broader than decision theory in that the provision of any analytical support to a decision-maker for what are typically repetitive decisions can be considered a DSS. We will only focus here on DSS that use the concepts of decision theory in aiding decision-makers. Keen and Scott Morton (1978) provided the initial definition and motivation for DSS. A DSS is differentiated from other computer-based systems in that it is: (1) tailored to specific decision-makers and their decision tasks;
iS6
Practice user
Dialog Generation and Management Software
Figure 6.15
system.
DSS architecture. The major components of a decision support
(2) developed with a high degree of user involvement during the design phase; and (3) implemented through an iterative process that sees the quick introduction of a partial system so that user feedback can be incorporated as part of the design process. DEVELOPING a DSS falls within our definition of decision synthesis because a DSS is based upon providing a decision-maker with analytical support, as shown in the DSS architecture in Figure 6.15. The user interface, which is supposed to be very friendly and sophisticated, is labeled the Dialog Generation and Management Software (DGMS). The Data Base Management System (DBMS) and Model Base Management System (MBMS) provide the system control over this analytical support. Descriptions of the DSS development process emphasize the use of an iterative sequence. Hurst et al. (1983) use an agricultural analogy: preparing the ground (convincing the decision-maker that he will be well served by and receive a lot of insight from a DSS); choosing the crops (identifying which decision tasks to support first); cultivating (continual monitoring and modifying the DSS as the decision-maker finds things he likes or dislikes, wants or does not want); transplanting (making use of
Analytic strategy
157
other successful DSS applications to enhance the current one); harvesting (providing successful DSS applications to users with similar or identical tasks); and dealing with crop failure (a tough task for any analyst). Sprague and Carlson (1982) describe a very useful, process-independent method for the design of a DSS, called ROMC. The four elements of the ROMC method are representations, operations, memory aids, and control mechanisms. Representations are ways that the decision-maker conceptualizes his problems through the use of special charts, graphs, or tables. Operations are those analyses that can produce the values that go into a particular representation, modify data in a representation, or transform data from one representation to another. Memory aids correspond to the checklists, memos, and summary information that a decisionmaker uses. Finally, control mechanisms include the DGMS, as well as the techniques by which the decision-maker is trained to use the system. Finally we need to mention the distinction that Sprague and Carlson make between a DSS and a "DSS generator." A DSS generator is a generic system that has the DSS basics for specific tasks imbedded in it, so that a user can tailor his own DSS. Spread-sheet programs for microcomputers are an excellent example of a DSS generator. The DEVELOPING application described below is taken from an ongoing project of Decisions and Designs Inc. The decision-makers in this application have to allocate resources (dollars) amongst as many as 300 competing projects. 6.6.2 DEVELOPING strategy for resource allocation This application involved the developlment of a DSS for the United States Marine Corps (USMC) to use in the yearly preparation of a five-year funding plan, called the Program Objectives Memorandum (POM). This application started in 1977 and has continued each year. The preparation of this yearly decision document is a ten-month effort which includes the request for and receipt of data from field units, the definition and costing of programs, the prioritization of programs, the adjustment of funding profiles in order to meet the five-year budget constraint being imposed from above, the modification of all of the above to achieve approval of the general officers. Decision analysts assist the USMC organization responsible for POM preparation in evaluating the relative benefits of the competing projects. The project sponsors and a specially appointed program evaluation group provide the benefit judgments. Kuskey, Waslov, and Buede (1981) describe the first five years of this DSS application. Thefirstyear's (1977) effort addressed thefive-yearperiod beginning in 1979, called POM 79. This initial analysis addressed only new procure-
Practice
i58 400 - | 360
300-
POM83
• Civilian manpower program • Military construction initiatives
Application system III • Packaging of items for POM • Net-benefit analysis
POM 82
• Current capability procurement • 'Other' procurement • Blue-dollar aviation program
Application system II • Capacity: 300 items • New reports • Add/delete missions
• • • •
Application system I • IBM 5110 computer with diskettes • Fast data editing features • Multiple benefits for items • Variable escalation rates for multiple appropriations • Capacity: 125 items
200POM 81
POM 8 0
100POM 79
POM cycle
Marine corps mission areas Ammunition initiatives Military manpower initiatives Marine corps reserves initiatives
• Research, development, and studies program • Operations and maintenance initiatives • Marine Corps functional areas • Procurement initiatives
POM topics
DARPA-sponsored DBMS for research on system specifications: • IBM 5100 computer • Sykesdisk unit • Interactive menu format for computer-naive users • Collect, display, report information • Cost-benefit analysis DBMS functions
Figure 6.16 DSS evolution. As the POM DSS was improved, more decision functions were supported and the data base functions were enhanced.
ment initiatives within functional areas such as training and logistics. The second year's effort was expanded to include all of the procurement initiatives as well as the operations and maintenance intitiatives. A separate analysis of research, development and studies initiatives was also undertaken in this second year. POM 81, the third year, categorized the initiatives into mission areas (such as close combat) rather than the functional areas. In addition, the list of initiatives included in the analysis was expanded to include ammunition, manpower, and reserves. Further expansions were seen in the fourth year as expenditures on ongoing (current capability) procurements and miscellaneous (or other) procurements were added. Another separate analysis was undertaken for aircraft initiatives which was submitted separately (and called "blue dollar" POM). Finally the fifth year saw the addition of civilian manpower and military construction initiatives. Figure 6.16 depicts this evolutionary expansion of topics. Also shown in Figure 6.16 is the evolution of the DSS functions over the firstfiveyears of this decision analytic effort. The vertical axis of the figure gives a rough indication of the computer programming effort involved.
Analytic strategy
159
Data base management tasks of structuring, entry, display and editing are supported. Modeling tasks for prioritizing and reporting tasks are also handled. This application illustrates the iterative development necessary for a successful DSS. Each of the application systems shown in Figure 6.16 were either totally or partially reprogrammed in order to incorporate new user needs or suggestions. 6,6.3 Requirements for success with the DEVELOPING
strategy
The DEVELOPING strategy works when two stringent criteria are satisfied. First, the decision maker must be amenable to using a computerbased system. By using, we mean sitting in front of a screen and controlling the computer's operations via instructions entered from a keyboard or other input device. In the 70s there were few decision makers with any authority who were so inclined. Now, due to many factors the number of such decision makers is increasing rapidly. As a result this strategy should be applied more and more often in the future. The second criterion is that the decision task should be of a repetitive nature; otherwise the cost of DEVELOPING a DSS could not be justified. Most of the applications of the other strategies are to unique, one-of-akind decisions. However, when a decision-maker faces repetitive evaluation or budgeting decisions, the DEVELOPING strategy can be used quite effectively to enhance the decision-maker's quality of choice, productivity, and final product (e.g. report or briefing). 6.7 ALLOCATE
AND
NEGOTIATE
DECISION MODES
As discussed in the introduction to this chapter, there are three primary decision modes (CHOOSE, ALLOCATE, and NEGOTIATE) for which decision-analytic techniques can provide useful insights. Sections 6.2 through 6.6 have concentrated on the strategies for assisting decisionmakers who are choosing one alternative from a list of several. This section expands upon the ALLOCATION and NEGOTIATION modes of decision. The ALLOCATION of a scarce resource across many alternative projects has received most attention in the traditional operations research/ management science literature. Most of these traditional approaches assume the absence of uncertainty and a well-defined objective function, such as profit. Decision analysts have approached this problem by recognizing that (1) for decisions under certainty the value function of the organization may be unknown and require careful elicitation and (2) for decisions under uncertainty new formulations are needed. Kuskey et al.
160
Practice
(1981), discussed in Section 8.6.3, provide a detailed elicitation procedure based upon conjoint measurement (see Krantz et al. (1971)) for eliciting the organization's values under certainty. Adelman (1984) describes a resource allocation application that used a similar approach. The basis of both of these examples is the concept of benefit-to-cost ratio for prioritizing the options competing for the scarce resource. However, what is important is the recognition that benefits (or values) are subjective in most decision situations, and must be quantified accordingly. For those ALLOCATING decisions that require an explicit characterization of uncertainty, Keefer (1978) and Madey and Dean (1984) provide rigorous approaches. Keefer captures both probability and utility functions and uses a non-linear programming algorithm to find a solution. Madey and Dean describe the use of several multi-objective optimization techniques with a multi-attribute utility formulation for this type of decision problem. Raiffa (1982) presents a through review of NEGOTIATION situations and analysis techniques. Distinctions are made between two-party bargaining with one continuous issue, two-party bargaining with more than one issue, and negotiations with many parties. Decision-analytic approaches are described and illustrated; a good example is Barclay and Peterson (1976). Barclay and Peterson posit a set of negotiation issues, each having a discrete set of positions to which each could agree. The positions on each issue should span the preferred positions of both parties, with potential compromises in between. Once the issues and associated positions have been defined, a quantitative value system is developed for each party using an additive value function. These value systems are used to compute the total value for all potential treaties or agreements, a treaty being a collection of positions, one from each issue. With these value systems, the set of treaties that satisfies the property of Pareto optimality can be found. A Pareto-optimal treaty is one such that no other possible treaty can increase one party's value without decreasing the other party's value. Typically, these Pareto-optimal treaties do not consist of an equal compromise on each issue. Rather, one party will get its preferred position (or close to it) on issues that have greater relative importance to it but less importance to the other party, and vice versa. This is called the principle of asymmetry of interest and is what makes such an analysis valuable to potential bargainers.
6.8
SUMMARY
This chapter has defined and illustrated five distinct strategies that have been and continue to be used in the application of decision analysis. While
Analytic strategy
161
these strategies are distinct, they do not exhaust the space of strategies. There remains a great deal of "grey" area between them, thus providing room for creativity on the part of the analyst. The four principal attributes that should be used by the analyst in choosing a strategy are: (i) the decision problem characteristics, (2) the decision-maker and his organization, (3) the type of insights most needed, and (4) the skills and experience of the analyst. The final subsections of sections 6.2-6.6 describe the problem/decision-maker characteristics that are best suited to each strategy. MODELING is best suited to a highly analytical client whose problem is typified by uncertainty and complexity. An autonomous decision-maker, whose problem requires a balancing of uncertainty with values and risk preference, would be well served by the INTROSPECTING strategy. Problems with no clear-cut value structure, faced by decision-makers with no analytical sophistication or predilection, are best approached via the RATING strategy. When the decisionmaker is most interested in the participation of his subordinates or when the problem requires that a great deal of information be shared amongst the group participants, then the CONFERENCING strategy is appropriate. DEVELOPING is most appropriate for a decision-maker willing to use a computer for repetitive decisions. As for insights, all of the strategies attempt to identify the critical factors of the decision. MODELING focuses more on uncertainty and problem complexity, RATING and CONFERENCING on value trade-offs, and INTROSPECTING has the best balance. However, for communication amongst the decision-maker and his experts, INTROSPECTING and DEVELOPING are most pointed to helping an individual. CONFERENCING explicitly addresses communication, while RATING and MODELING provide convenient structures to enhance communication during the analysis. All of the strategies result in an audit trail that can be used to communicate "why." The audit trail of the MODELING and INTROSPECTING strategies is more defensible to analytic and technical reviewers due to their more explicit representation of complexities. However, many decision-makers have been confused by or distrustful of these quantitative representatives. Others have expressed an unwillingness to sit through lengthy assessment sessions during which they are asked to answer the difficult and hypothetical lottery comparisons of the INTROSPECTING strategy. To these decision-makers the RATING and CONFERENCING approaches are more defensible. These requirements and benefits of the strategies are summarized in Table 6.6. Finally, each of these strategies requires the analyst to have different skills. In surveying the practicing analysts one finds that most of them apply only one to three of the above strategies, and havefinelytuned their analytical skills accordingly. This is probably the result of a predilection
Practice
i6z
Table 6.6 Strategy summaries Decision problem i. Time available 2. Frequency 3. Scope of expertise 4. Integrative theory
Decision-maker and organization 1. Number 2. Management style 3. Value system 4. Thinking style 5. Mathematical perspective 6. Computer background Insights needed 1. Problem complexity 2. Uncertainty 3. Value 4. Time preference 5. Risk preference
The MODELING and DEVELOPING strategies require multiple months to complete, while the other strategies can be completed in weeks. DEVELOPING requires an iterative decision problem for its cost to be justified; the others are typically applied to one-of-a-kind decisions. Since the INTROSPECTING strategy focuses primarily on the decision-maker, the problem scope is limited to the expertise of the decision-maker. The MODELING strategy often uses a scientific theory, such as economics, to integrate the expertise of the organization. The other strategies rely more on global judgment.
The CONFERENCING strategy was designed for groups. INTROSPECTING and DEVELOPING have the individual as the focus. CONFERENCING works only when the decision-maker embraces a participative management style. MODELING relies on a simpler and better-defined value structure than the other strategies. The complexity of MODELING and INTROSPECTING allows the analysis to focus on details. RATING and CONFERENCING focus on chunks. The decision-maker must trust mathematical analyses for the MODELING and INTROSPECTING strategies to be accepted. DEVELOPING requires that the decision-maker be a computer user. MODELING and DEVELOPING MODELING, INTROSPECTING, and DEVELOPING INTROSPECTING, RATING, CONFERENCING, and DEVELOPING MODELING, INTROSPECTING, and DEVELOPING MODELING, INTROSPECTING, and DEVELOPING
towards a particular strategy that is well suited for the analyst's client base, but is also the recognition that to apply a strategy successfully requires finely tuned skills.
7 ANALYTIC TACTICS
7.1
INTRODUCTION
In the previous chapter we discussed the strategic choices that the analyst has in conducting an analysis of a decision. While these strategic approaches vary considerably and on many dimensions, there a number of skills (or tools) that the analyst needs to complete any one strategy, and these skills are amazingly common across all strategies. First in Section 7.2 we will talk about structuring, by which we mean both the definition of the problem and the identification of the elements of the model to be used in analyzing the alternatives. The choice of an analytic strategy is included in problem-structuring. Also included in this section is some practical advice on how to get the analysis started and a philosophical, yet pragmatic, view of modeling. The third section of this chapter addresses the use of iterative analysis, the sequential application of structuring and quantification. Four types of iterative analysis are discussed, followed by a discussion of how to use iterative analysis in deciding how much analysis to do. Probability elicitation occupies Section 7.4. Research findings on good and bad ways to assess probabilities are summarized before an encoding process is described. Finally we comment on the discretization of continuous probability distributions for decision tree analysis. The elicitation of values and utilities is a natural follow-on topic to probability assessment. Again research findings in this area are summarized prior to the discussion of the many techniques for assessing value functions, value weights, and utility functions. In Section 7.6 we describe sensitivity analysis and give examples of stochastic dominance and one-way and two-way sensitivity analyses. 7 . 2 PROBLEM- AND MODEL-STRUCTURING
Problem-structuring includes the identification of the decision-makers within the organization and the relationship of the client (or primary 163
164
Practice
decision-maker) to this decision-making structure, the determination of the decision boundaries that the decision-maker must accept as fixed as well as those that he feels he can challenge or has control over, the determination of the principal objective of the decision-maker and the problem(s) that will be solved if an alternative other than the status quo is chosen, and the willingness and ability of other decision-makers and experts in the organization to provide inputs to the analysis. The analyst should be in a position after learning the above information to suggest and describe to the decision-maker the benefits that an analytic strategy will have. The resolution of the analytic strategy is included in problemstructuring. Quite often decision-makers have already defined their problem before enlisting the services of a decision analyst. However, there are occasions when the analyst must help define the problem, or more often redefine the problem correctly after the analysis has begun. Section 7.2.1 discusses several techniques that have proven useful in the definition of the problem. Model-structuring is the creation of an evaluation tool for comparing the alternatives that were defined during problem-structuring. Here the analyst must take the decision-maker's objective and structure a value system. Value systems can vary from one attribute (maximize profit) to a hierarchical representation of hundreds of attributes, all having different priorities. Also the analyst must ascertain the role that uncertainty should play in the analysis. If the uncertainties are minimal or revolve around major environmental variables that cannot be influenced by the alternatives, then a multi-attribute value analysis is possible. However, if the uncertainties depend upon the alternative chosen or are many in number and require significant conditioning, then a decision tree or simulation model will be required. Section 7.2.2 describes two modeling aids that have been developed. Two extreme, model-structuring approaches, top-down (objectivedriven) and bottom-up (alternative-driven), are described in the next section. Finally requisite decision-modeling is discussed in Section 7.2.4. J.Z.I
Problem-structuring aids
This section summarizes a set of problem-structuring aids that have proven useful to decision analysts and the analytic community at large. The best-known problem-solving approach is that proposed by Kepner and Tregoe (1981) in The New Rational Manager. Second, Van Gundy (1981) summarizes many such aids, and Section 7.2.1.2 reviews three of them. Finally we document a procedure called the Report Card by its founder, Cameron Peterson.
Analytic tactics
165
7.Z.I.I The rational manager - Kepner and Tregoe (1981) Kepner and Tregoe define three phases for the rational manager: (1) problem analysis, (2) decision analysis, and (3) potential problem analysis. Problem analysis is the area for which Kepner and Tregoe are most widely recognized and is the focus of this discussion. The decision-analytic approach that Kepner and Tregoe propose is a simplified version of SMART (Section 6.4) and is not rigorous enough for discussion. Portions of the Kepner and Tregoe post-problem analysis have been included throughout the analytic paradigm presented in this book. The remainder of their potential problem analysis falls within the purview of implementation. Kepner and Tregoe developed the following five-step process for problem analysis: (1) definition of the problem — a brief statement of "what is broken" or, if this is not possible, "why we know that something is broken"; (2) description of the problem - expanded statement of the problem in four dimensions (identity - "what," location - "where," timing "when," and magnitude — "how much, how many"); (3) extraction of key information in all four dimensions - what IS known and what COULD BE wrong, followed by what possibilities are suggested by comparing the IS and COULD BE statements; (4) testing for the most probable cause - throw out the unlikely and examine the feasilibity of the most likely cause; and (5) verification of the true cause. The major reason for the success of the Kepner-Tregoe problemdefinition process is that it structures the thought process to produce a thorough and systematic synthesis of all possible causes. 7.2.1.2 Van Gundy's shopping list Van Gundy (1981) has collected and described a number of problem-solving techniques. He uses an interesting analogy of funnels (Figure 7.1) to illustrate the expansion and contraction of concepts that occur as iterative analyses proceed to conclusion. The initial problem analysis and redefinition phase starts, as described by Kepner and Tregoe, by generating all possible causes and systematically eliminating them. Then solution ideas are generated with little restriction so that no good alternatives are eliminated too early. Finally the evaluation and selection process reduces the number of alternatives until one isfinallychosen. (In Section 7.3 we discuss ways of winnowing the list of alternatives via iterative analysis.) Dimensional analysis is one analytical problem-solving technique that Van Gundy discusses. Like the Kepner and Tregoe approach, dimensional analysis segments the problem into five elements: (1) substantive - examines such issues as are we doing something that we should not be doing, or not doing something that we should be doing?
Practice
i66
Figure y.i
Problem analysis and redefinition
Idea generation
Brainstorming, followed by problem analysis
Problem definition leads to the creation of alternatives
Idea evaluation and selection
Decision analysis results in the recommendation of an alternative
Problem-solving funnels. Source: Van Gundy 1981, Figure 1—1.
(2) spatial - can the location of the problem be pinpointed to a small area, or is it widespread? (3) temporal - is the problem recent or has it been with us for a long time? If recent, has it already begun or is it about to occur? Is it a constant problem, or are there cycles of occurrence? (4) quantitative - how many people and factors are involved? Is it a question of scarcity or over-abundance? (5) qualitative - questions of philosophy and values are addressed. This approach also forces a systematic examination of the problem by having the problem-solvers answer a number of questions. However, no systematic process for producing the answers to the questions is provided. Another technique addressed by Van Gundy, relevance systems, develops a hierarchical representation of information about the problem. Once the structure of problem elements has been developed, a mirror structure is developed by associating partial or total solutions with each bottomlevel problem element. Once the bottom level of solutions is created to match the problem elements, then an inverted solution hierarchy is created. The problem and solution hierarchies can then be examined in detail to provide structure to the problem. This is the only technique that systematically synthesizes the solution space. The final technique from Van Gundy's list that we will discuss is lateral thinking, which was developed by de Bono (1970) as a way of breaking out of logical thought processes to develop radical views of a problem. The three primary activities that are used in lateral thinking to stimulate new ideas are (1) becoming aware of the problem, including its boundaries and assumptions; (2) generating as many alternative views of the problem as possible without using old concepts of the problem; and (3) the use of provocative methods for introducing discontinuities into the
Analytic tactics
167
thought process. This technique is far too complex to cover adequately here, and we refer the interested reader to de Bono (1970). y.2.1.3 Report Card Cameron Peterson of Decision Conferences Inc. has developed a method to identify an organization's most significant problems in terms of its strengths and weaknesses. This approach, called a Report Card, begins with the development of a hierarchical representation of the problem, similar to multi-attribute value analysis. If the decision-maker were worried about his business strategy, a hierarchy of business elements (e.g. marketing/sales, engineering, production, maintenance, and management) might be developed. To account for the business environment, these elements might be evaluated for profit potential under several economic and competition-based scenarios. Next the decision-maker and his managers score the business elements on a grading scale (e.g. A, B + , C - ) in terms of their profit-making potential as currently configured within each of the scenarios. Finally the decisionmaker assigns weights to (1) the scenarios to capture their likelihood and (2) the business elements to capture their importance to the organization's profit and the severity of the problems identified by the grade. These business-element weights can be different for each scenario and measure the importance infixingthe elements' problems within the given scenario. Then these grades can be translated into numbers and aggregated using the weights to calculate a weighted grade. Analyzing the results includes looking at the composite grades for each scenario to determine if there is any scenario with which the organization is particularly ill-equipped to deal. Then the scores can be aggregated across scenarios for each element to determine which elements are in the most trouble. As a result of this group process, a consensus statement of the biggest problem facing the group is synthesized from many differing initial options. Now the analyst can begin model-structuring by assisting the decision-maker in formulating alternative business strategies that address the major problems found via the Report Card. This technique has also been used during the model-structuring phase to conduct a cursory evaluation of alternatives or to begin the development of value attributes. Keys to the successful use of the Report Card method include keeping the hierarchical problem representation small (fifteen tofiftybottom-level factors to grade). This process should not take a long time and it is better to repeat the process with a second, small hierarchy than to get bogged down in one big grading session. A second key is to keep good notes on the definitions of the problems associated with each bottom-level factor as the grades are assigned. The group should be prompted to talk about the problems, not just ask for a grade. This Report Card process was developed for use with groups as a way
168
Practice
of achieving consensus via a systematic examination of the organization's strengths and weaknesses. This process is only needed for relatively ill-structured problems. 7.2.2 Model-structuring aids Once a problem has been structured, a model has to be developed to express that structure. This section describes two network-based aids for the structuring of models. The approach using influence diagrams primarily focuses on decisions and events, while the cognitive mapping approach defines more basic variables and can be used in problemstructuring as well. 7.2.2.1 Influence diagrams The only rigorous modeling aid developed by decision analysts for their use is the influence diagram. This research was begun in the mid-seventies and thefirstpublished discussion appeared in Miller, Merkhofer, and Howard (1976). The early research focused exclusively on the use of influence diagrams during model-structuring with a natural carry-over into probability elicitation. This work has continued and has been generalized. Also recent efforts have shown that in addition to being used to structure the model, influence diagrams can replace decision trees for some problems as the solution mechanism. Our discussion will begin with the early developments of influence diagrams, move to the recent generalizations as a modeling aid, and conclude with the recent efforts that have influence diagrams replacing decision trees as an analysis tool. Early developments of influence diagrams The influence diagram is a representation of the relationship between variables in a decision by a network of nodes connected by directed arcs. This definition fits the early work done by Miller (in Miller et al. 1976), by Owen (1979), and by Howard and Matheson (1981). The works of Bodily (1985) and Shacter (1986), which will be presented later, expand upon this definition. This early work on influence diagrams defined two types of influence: conditioning (or probabilistic) and informational. The top two examples of influence in Figure 7.2 define conditioning influence, and the bottom two examples define informational influence. (A third possible type of influence, value, is ignored in these early efforts but examined by Bodily 1985.) A conditioning influence, then, is merely a statement about the presence or absence of probabilistic independence. It is important to remind the reader at this point that probabilistic independence exists if we can state (using an obvious notation) that:
Analytic tactics
I69
O
THE PROBABILITES ASSOCIATED WITH ALEATORY VARIABLE B OEPENO ON THE OUTCOME OF ALEATORY VARIABLE A
o
c
DEPENDS ON DECISION C
O
F
THE DECISION MAKER KNOWS THE OUTCOME OF ALEATORY VARIABLE E WHEN DECISION F IS MADE
THE DECISION MAKER KNOWS DECISION G WHEN DECISION H IS MADE
Figure 7.2 Definitions used in influence diagrams. The top two influences are conditioning, the bottom two are informational. Source: Howard and Matheson 1981, Figure 8.1.
and does not exist if we have to state that: P(x, y l) = p(x y, %) p(y | = p{y
I x, I) p(x I
or, equivalently, (7.1)
where 2; refers to all prior information. Thus a conditioning influence is identical to our probabilistic beliefs concerning the relationship of two random variables. This conditioning influence can be changed if we introduce a third random variable, z, and state our beliefs concerning independence as: p{x9 y 9 z \ l ) = piy I z ,
p(z
p(x
(7.2)
The top portion of Figure 7.3 shows the two-random-variable case with a conditioning influence, and the bottom portion of Figure 7.3 shows the three-variable case corresponding to Equation (7.2). It is important to realize that a conditioning influence is not a statement of causality. Likewise, the absence of a conditioning influence is not a statement of mutual exclusivity (in fact, mutual exclusivity implies probabilistic dependence, a conditioning influence).
170
Practice
two variables
modification by introduction of third variable
Figure 7.3 New information and influence diagrams. When z is introduced, the influence of x on y becomes indirect. Informational influences, indicated by directional arrows leading into decision nodes, do depict an ordering based upon causality. That is, an arrow leading into a decision node states that the decision or chance outcome at the other end of the arrow will be known before the decision-maker needs to make the decision associated with the first node. Having defined influence diagrams as we have to this point, the first and perhaps most important use of these diagrams is one of communication between a technical decision analyst and substantive decision-maker (and experts). Owen (1979, p. 12) reports that decision analysts who have used influence diagrams as communication aids have in fact observed strong correlations between the decision-maker's intuitive definition of influence and the more rigorous definitions above. Rigorous empirical research is needed to validate this conclusion. As such, influence diagrams are indeed useful in problems composed of many random variables and/or subsequent decisions, because the key dependencies can be identified and quantified without complicating the analysis unnecessarily. While influence diagrams are certainly useful and compact, they essentially assume a symmetric problem structure; that is, the initial decision alternatives are followed by the same random variables. For highly asymmetric problems, influence diagrams are not useful. For slightly asymmetric problems, the asymmetries can be reproduced everywhere to make the problem symmetrical by assuming degenerate probability and/or value functions. A second important use of influence diagrams that was identified in this early work was the coalescence of decision trees. Coalescence is a process of identifying and removing redundant portions of decision trees for calculational speed and for reductions in probabilistic elicitation.
Analytic tactics
Figure 7.4 Uncoalesced decision tree. This decision tree could be used for any set of influence diagrams on the three random variables x, y, and z.
X /
GY" ^ \
\
\
r-fx V z
\
/
/
X
/ \
/ \
*
Figure 7.5 Coalesced decision tree. This decision tree can be used if the influence diagram at the bottom of Figure 7.3 pertains.
Decision tree redundancy occurs when two or more sections of a decision tree contain the same structure, probabilities, and values. (Note that these early efforts have ignored or assumed away the notions of value influences.) In order for two or more sections of a decision tree to have the same structure and probabilities, there must have been some probabilistic independence earlier in the tree. If there is a decision alternative that results in the three discrete random variables exhibiting the probabilistic dependencies shown in Figure 7.3, then the probability tree shown in Figure 7.4 can be drawn to represent this uncertainty. Note that the two y' nodes are identical, as are the two y" nodes. Thus the coalesced decision tree in Figure 7.5 can be drawn to represent the full decision tree in Figure 7.4. Several computer programs that use the notions of influence and
172.
Practice
0=0=0 This diagram depicts maximal conditioning.
Reversing any arc of the above diagram via Bayes' theorem leads to a cyclic diagram.
Figure j.6
Manipulation forming cycle not permitted.
coalescence for solving decision trees have been developed by decision analysis consulting firms. All of the early researchers recognized the relationship between the influence diagram and the decision tree. There are examples of influence diagrams that can be translated directly into decision trees in all of the papers, and examples in which direct translation is not possible. Manipulations of influence diagrams are possible so that a different but equivalent influence diagram can be achieved by manipulating the arcs coming into probability nodes. Since the two statements in Equation (7.1) are equivalent, the arrow between any two probability nodes can be reversed, as long as no cycles are defined. Figure 7.6 displays a manipulation of arrows that is not permitted because of the cycle that develops. The presence of such a cycle suggests an impossible set of states of information for a decision-maker to have. Owen (1979) develops an influence calculus in the form of matrices that permit the random variables included in the influence diagram to be prioritized on the basis of their importance for impacting the decision. Bodily s extension of influence diagrams Bodily (1985) has extended this initial work on influence diagrams into a much broader notation, including certain and uncertain influences between random and certain
Analytic tactics Decisions
Intermediate variables
173 Outcome attributes
Example A Certain influence
Example B Uncertain influence
Example C Random variable; certain influence
Example D Random variable; uncertain influence
Example E Preference influence
Example F Preference influence; certain influence
Figure 7.7 Symbols used in an influence diagram. Bodily extends influence diagrams to cover preference. Source: Bodily 1985, Figure 3.2. intermediate variables and decision variables and outcome variables, as well as preference influences. Bodily defines three types of variables: - decision variables, indicated by a rectangle; - intermediate variable, indicated by a circle; and - attribute, indicated by an ellipse. Intermediate variables can be either certain (no marks) or uncertain (random) and marked with a tilde ( ~ ) . Three types of influences are also defined: - certain influence, denoted by a single straight line with an arrow on one end; - uncertain influence, denoted by an arrow with a squiggle in it; and - preference influence, denoted by a double straight line with an arrow on one end. These variables and influences are illustrated in Figure 7.7. Bodily does not mention informational influence as defined above, thus restricting his
174
Practice
ability to model sequential decisions. However, there is no reason not to include this concept in his development since his sole purpose for using influence diagrams is as an aid in developing and defining the decision model. For this reason he also permits loops in the influence diagrams as long as the loop does not include an uncertain variable or an uncertain influence. A loop of certain influences between certain variables could be solved via a set of simultaneous equations. Finally, we should point out that, in Bodily's convention, any variable that is influenced by a random variable or is the object of any uncertain influence is indeed a random variable itself. Bodily does not signify this with a tilde; rather it is just understood. Evolution of influence diagrams as analysis tools Shacter (1986) defines an influence diagram to be a network consisting of a directed graph and associated node sets and functions. The three types of nodes permitted are decision nodes, chance nodes, and at most one value node. Arcs into chance and value nodes represent probabilistic dependence, while arcs into decision nodes are informational, as we discussed above. Shacter provides the first algorithm for manipulating a proper, wellformed influence diagram so that all nodes can be eliminated in a value-preserving manner and the answer is obtained. Shacter calls an influence diagram proper if it is an unambiguous representation of a single decision-maker's view of the world. In addition, he develops a set of definitions and theorems to determine whether a given influence diagram is well-formed (or regular). The algorithm eliminates nodes in the following order: (1) barren nodes (a chance or decision node with no successors); (2) decision nodes which have no chance nodes between them and the value node; and (3) chance nodes. There are conditions within the algorithm to determine when arc reversals (which amount to applying Bayes's Theorem) are useful or necessary. Shacter and his colleagues have developed a computer program for implementing this algorithm, thus eliminating the need to draw and solve a decision tree. Example As an illustration of the use of influence diagrams for structuring the relationships of variables in a decision model, we apply influence diagrams to a decision concerning an investment in a beach cottage. There are four separate decision variables contemplated, each of which is shown in a rectangle in Figure 7.8: - Should the investment, if it is made, be an individual investment or that of a partnership with a close friend? — If a partnership, should the friend (a skilled contractor) assume the role of general contractor, thus cutting construction costs?
Analytic tactics
175
Figure y.8 Initial beach investment influence diagram. The initial modelstructuring exercise focused on the major impacts of the alternatives. - What should the total value of the investment be and, if a partnership, what is the total cost of the investment? - What rental price strategy should be employed? Note that the decision boxes are all connected by a single thread. Shacter points out that this is critical in a well-formed influence diagram, calling it the condition of "no forgetting." A common approach for constructing an influence diagram is to define both ends of the decision: alternatives and value criteria. When first thinking about this problem the decision-maker synthesized two criteria: (1) the enjoyment derived from staying at the beach cottage when it is not rented and (2) the net present value (NPV) of the investment. Then three chance nodes were defined that connected the decision and value nodes: (1) investment income - income derived by renting the unit; (2) loan payments made on the mortgage; and (3) the resale value of the cottage. Figure 7.8 shows this initial influence diagram. After reviewing this diagram for a few hours, it became clear that additional insight could be gained by defining the components of these three chance nodes. Also, a colleague pointed out that there is no explicit reference to the initial investment required by each alternative. Upon reflection there was no doubt that initial investment was definitely a value criterion, as well as an important variable. The final influence diagram is shown in Figure 7.9. Enjoyment is impacted by the decision variable sole-versus-partnership investment (the owner will be able to visit the cottage more often if it is a single investment), by the total value of the cottage (the nicer the cottage, the more the owner will enjoy it), and the occupancy rate (the lower the rate, the more the owner can enjoy it).
17 6
Practice
Figure y.9 Final beach investment influence diagram. The second iteration of model-structuring led to greater resolution in the model's variables and the identification of new value attributes.
Initial investment is likewise impacted by the single-versus-partnership decision (lower if a partnership) and by the total cost and mortgage size. Finally, NPV is a function of many factors: the initial investment, the loan payments, the investment income, the tax regulations after the property is built, and the resale value of the property. Mortgage rate and size determine the loan payment, and mortgage size is determined by total cost and the partnership considerations. The investment income is determined by the rental strategy and ensuing occupancy rate. The tax regulations were broken into two pieces because of the uncertainty associated with changes in the US tax laws being contemplated in 1985. 7.2.2.2 Cognitive mapping The second approach to structuring which we wish to discuss is called cognitive mapping, as put forward by Colin Eden and his colleagues at the University of Bath. Cognitive mapping has aspects of both problem-structuring and model-structuring, but we include it here since we see it primarily as a tool for structuring a model. In addition, there is a strong similarity to the concept of influence diagrams. Excellent detailed accounts of cognitive mapping are Eden and Sims (1977). Eden etal. (1979), Sims et al. (1981), and Eden et al. (1983). Our discussion should provide the reader with sufficient insight into the aproach to enable him or her to decide whether to pursue this topic in the original references.
Analytic tactics
177
///In some areas the situation has been bad for a long time
/Float policy informally for prior discussion/Using formal reports only/
Backlog in getting things done /Do/Don't/deal swiftly, efficiently, sympathetically with people's problems t
/Try/Do not try/to indirectly assess individual opinions and attitudes
/Attempt/Do not attempt/to inform tenants individually of what doing
Support of committee chairman
/Often conflict between doing what tenant sees and structure maintenance// /Favourable/Unfavourable/reactions of tenants expressed to members
/Favourable/Less favourable/ attitude of members Getting information disseminated, favourably reported in press and journals Likelihood of proposals being accepted
/Not spending enough time because of pressure of week by week things// /Trying/Not trying/to use press for personal public relations
Figure y.io
An exemplary cognitive map. Source: Eden et al. 1979, Figure 7.6.
Cognitive mapping presupposes, as do all decision analysis strategies, that the reality with respect to which a decision-maker must make a decision is a construction of that individual. Thus, when we are worrying about which action to follow, it is our own mental models of the world which are of importance to us in guiding choice. From this starting point it is a short step to seeking a methodology that will allow the decision-maker to establish a formal model of his own thinking. Such a methodology was provided by Kelly's theory of personal constructs (Kelly 1955), and the method of cognitive mapping is strongly based on this theory. The first step is to have the decision-maker identify personal constructs that are clearly important in the decision. A construct is any idea that the decision-maker believes should be taken into account in the decision. No differentiation is made between alternatives, uncertainties, and values. Rather this initial process is analogous to brainstorming in that no criticisms or categorizations are allowed until a later stage. The next step is to encode these ideas. Chapter 4 of Eden et al. (1979),
178
Practice
"Making maps of thinking," gives a detailed account of this step. Using coding methods developed from ideas of Holsti (1976) and Wrightson (1976), Eden and his associates have developed a procedure to represent the constructs and the relationships between them, which they call a cognitive map. Figure 7.10, taken from Eden et al. (1979), illustrates a cognitive map for a government agency dealing with maintenance requests from tenants. The outcome of the process being mapped is acceptance of the maintenance proposals, as shown in the bottom left of the figure. The arrows in a cognitive map indicate causal influence. Note that the constructs in the figure are differentiated by the presence or absence and number of slashes. Alternatives faced by the individuals in the government agency have the following denotation: "/Do/Don't/deal swiftly,. . ." Uncertain outcomes affecting thefinaloutcome also have this denotation: "/Favourable/Unfavourable/reactions . . ." A positive sign at the head of an arrow indicates that there is a direct correlation between the two assertions. A negative sign indicates a negative correlation. Other constructs that define pre-existing conditions are begun by three slashes, as shown at the top of the figure. Constructs with no slashes (e.g. "Support of committee chairman") represent uncertain outcomes that have not been influenced by the decision-maker's alternatives. Finally, constructs that begin with one slash and end with two denote the state of the agency. As the reader can see, the cognitive map serves to identify and categorize the variables needed in conducting a decision analysis. Once a map has been constructed for a problem, it can then be analyzed. Eden and his group have developed a computer program called COPE which helps in the analysis. This analysis process is not the same as we have described in Chapter 3 but embodies the ideas of Eden and his associates. COPE can also be used as a structuring aid, allowing the decision-maker to define the map interactively. The key to the success of this technique is that the analyst uses it to create an interactive interviewing process with the decision-maker or his experts. Eden and his associates, in their writings, stress theflexibilityof their approach, and the need to practice with it to appreciate its capabilities fully. We must stress that, as far as we are aware, Eden and his colleagues do not see their approach as a precursor to decision analysis, as is our implication here. As a result of the insight provided by cognitive mapping, they find that decision-makers are often able to reach a conclusion without further analysis. We see advantages, however, in going on to a more formal decision analysis to evaluate conflicting objectives or allocate scarce resources, once cognitive mapping has been used to structure a model.
Analytic tactics
179
7.2.3 How to get started Since our definition of problem-structuring includes collecting the information necessary to decide on an analytic strategy and at least a generic definition of the alternatives available to the decision-maker, problemstructuring is the first point of discussion with the decision-maker. However, it is possible, and generally a good idea, to make the transition very quickly into model-structuring when the decision-maker has defined his problem well and has a list of alternatives that he is contemplating. Buede (1986) has described two structuring techniques for developing value systems, either of which can be used at the outset of the analysis and both of which are commonly used at some point in the analysis. The two structuring techniques, top-down and bottom-up, differ as to their focus on problem- versus model-structuring, respectively. The top-down method is objective-driven in that the analyst ascertains what the decisionmaker's objectives are within the decision context, and then further synthesizes the problem boundaries and value structure by subdividing the objectives into attributes. This is the initial approach that is recommended when the decision-maker is troubled, but not sure why, and recognizes that he has partially defined alternatives available, but is not sure what a complete alternative would be. This process is characterized by uneven progress, and the aids described in Section 7.2.1 are often quite useful. If the analyst and decision-maker are not happy with the progress that they are making, it is best to remind everyone concerned that the analysis process is iterative, that the problem structure will be revisited, and that more progress will be made by beginning model-structuring, even with a handful of poorly defined alternatives. By making this early transition to model-structuring the first very coarse analysis can be completed and problem-structuring can be revisited, often with a lot more insight into the problem. The bottom-up approach, in contrast, is alternative-driven. When the decision-maker comes prepared with alternatives already defined, the analyst is able to begin the model-structuring phase almost immediately. The analyst should use the alternatives to probe the decision-maker and his experts for the major differences between these alternatives. These differences can then be used to form the attributes in a multi-attribute value hierarchy, as described in Buede (1986), or to structure an influence diagram or cognitive map for model development. A major benefit of the alternative-driven approach is that the decision alternatives provide natural stimuli for the analyst to use in eliciting the "chunks" that the decision-maker uses when thinking about his problems. By chunks we refer to multi-dimensional bundles that these experts have internalized in their problem-solving experiences as being critical. When the analyst can
180
Practice
utilize the decision-maker's language in conducting the analysis, the results will be much more compact and meaningful. De Groot (1965) defined the validity of and value in using chunks with an experiment involving chess masters and novices. Both masters and novices were shown chess-boards that could have been the result of a partial chess game and asked to memorize the positions of the chess pieces. The novices and masters did equally well when the pieces had been assigned randomly. However, when the pieces were placed according to positions that resulted from actual partial games, the masters did markedly better than the novices, often getting all of the pieces correct. When the masters made a mistake, they were more likely to have shifted several pieces en masse to an incorrect location on the board. When probed it became clear that the masters were memorizing the locations of the pieces on the board in several chunks, and it was these chunks that they might misplace. 7.2.4 Requisite decision-modeling Before concluding we should emphasize our subjective view of modeling. As we mentioned in Chapter 3, decision analysis departs from the classical school of probability and adopts a subjective (or Bayesian) definition of probability, partly because this is necessary if uncertainty is to be included in the analysis of decisions. Likewise, values and risk preference are subjective quantities. What about models? Can they be any different? Phillips (1984) defines a decision model as a satisfactory representation of the decision-maker's perception of his problem; satisfactory, that is, in the sense that he believes that it has given him adequate insight to make up his mind. Phillips calls this a requisite decision model. If there is more than one person involved in the decision, which is generally the case, then the requisite decision model "represents the social reality of the shared understanding of the problem." So a requisite decision model does not describe reality, nor does it reflect the values and beliefs of the decisionmakers in some optimal way; it is an artifact of the analysis, dependent on some degree on the alternatives being considered. The process of iterative analysis, addressed next, is used to refine the decision model in the areas most needing attention, as defined by the decision-maker's need to make a decision. EXERCISES FOR 7 . 2
1. Using the discussion of the facilities investment and expansion decision described in Section 8.4.2 and the decision tree variables defined in Figure 8.14, develop an influence diagram for the eight decision variables in the latter figure.
Analytic tactics
181
2. Using the discussion of hurricane-seeding in Section 8.5.2 and any additional information on hurricanes that you can obtain, develop a problem analysis (the five-step process of Kepner and Tregoe) for minimizing the effects of hurricanes. Do not assume that the seeding of hurricanes is the only answer. 7.3 ITERATIVE ANALYSIS
We emphasize the use of iterative analysis more heavily in this book than is usual in texts on decision analysis. This emphasis is the result not only of it being stressed in the schools of decision analysis from which we come, but also more significantly, of the lessons we have learned in applying decision analysis. We, and many of our colleagues, have many stories to tell of analysis that had to be abandoned because the iterative approach was not planned from the start. No matter how well the problem- and model-structuring phase is thought out and executed, it is likely that the first full analysis will reveal changes that should be made to both the problem definition and the model. If the decision-maker is expecting a finished product at this point, the analyst cannot recommend further study unless the effort was viewed as research (which means we were not actually in the real world). It is best to plan for and educate the decision-maker to expect multiple analyses from the start. 7.3.1 Four types of iterative analysis In the previous chapter (Section 6.2) we discussed the analytical cycle proposed for the MODELING strategy (Figure 6.6). Two analyses in Chapter 8 (8.3.2 and 8.4.2) illustrate this approach. This is but one of several iterative approaches to decision analysis that have been discussed and implemented. A second iterative approach is typically used in the CONFERENCING and RATING strategies: start with an overly simple model or value structure; complete the analysis even though everyone knows it will not be accurate; then decide what was especially wrong; make improvements to the model, alternatives, and problem definition; and repeat the analysis; continue until a satisfactory result is obtained. This approach was exemplified by the CONFERENCING analysis in Section 6.5.3. I* n a s proven successful in many other applications for many analysts. A third approach could be described as "prospecting." This is appropriate if the analyst and decision-maker must examine a rich set of alternatives, from which one must eventually be chosen. Rather than evaluate all of the alternatives at once, the alternatives are aggregated into similar categories, and one or two alternatives from each category are
182
Practice
chosen for the initial analysis. At the conclusion of this analysis whole categories of alternatives can be eliminated from consideration on the basis of the performance of the samples. The categories that remain can be sampled more fully and analyzed using the same model or a more detailed model more suited for the refined set of alternatives. This winnowing process can be repeated as many times as needed to reach a preferred alternative. The case study in Section 8.4.3 illustrates this approach to iterative analysis. A fourth approach is to conduct thefirstiteration of the analysis on only part of the problem for the entire set of alternatives. Then succeeding analyses expand the definition of the problem until the entire problem space has been analyzed. This approach has proven to be less successful than the previous approaches in our experience, but still has merit in certain situations. For example, if the decision-maker has authority over only a portion of the problem space and is hoping to convince others in the organization to participate in the analysis process, the approach has considerable value. Since he can only commit the experts in his direct control, the entire alternative set is examined but the model only addresses the problem space covered by his experts. The remainder of the problem space is assumed constant. By exposing the analysis and its process to the peers and superiors of the decision-maker, it is hoped that they will participate in the second iteration. Another solution to the problem of working with a decision-maker having partial authority is to model the whole problem space and make "best guesses" at those parameters for which better expertise exists but is unavailable. Then when the true experts are shown the first analysis and exclaim shock and dismay at the "best guesses" used for their areas of expertise, they are invited to make the necessary improvements in the next phase. This alternative approach falls under the second approach since the improvements made by the "true experts" typically involve more detailed and complex models. Before closing this discussion a fifth approach must be broached. This approach does not rightfully belong under a discussion of iterative analyses but would so belong if the discussion pertained to multiple (or plural) analyses. Decision analysts have discussed the concept of providing a decision-maker with the results of several simple analyses rather than the result of one detailed analysis, whether it was the culmination of several iterations or one large effort. Yet no analyst (to our knowledge) has ever implemented this concept. In an inference application Brown et al. (1986) develop two probability distributions for the same random variable: the number of items mislabeled over a two-year period by a firm selling down garments. One distribution was developed on the basis of a sample of garments offered for sale, the second on the batches of filler material used in the manufacture of the garments. Edwards (1977) has
Analytic tactics
183
reported conducting several analyses of the same social issue, but each analysis was for a different decision-maker. Linstone (1984) addresses multiple perspectives (technology, organization, and people) in a systematic but non-quantitative fashion. One of us has used two analyses in his personal choice of a portable microcomputer and found the results very useful. Brown and Lindley (1986) are exploring this topic from a theoretical perspective. We believe that this void of plural analyses results from the failing that most (all, judging from the void) analysts have: the belief that they can always improve their model's validity (ability to represent the real world). Yet there is a little evidence (Schwenk and Thomas 1983) that decisionmakers who have access to multiple and conflicting analyses do better than decision-makers with only one analysis. This finding coincides with the evolution of accepted elicitation procedures for probability and utility assessments: namely, the notion that more judgments than necessary may be called for, raising the possibility of inconsistency, and providing the opportunity, in its resolution, for improved judgment. 7.3.2 How much analysis is enough? A question which is not often asked, but which is in fact very important to address at the inception of an analysis, is how much analysis should be done. This involves not only a decision of how many resources (cost and manpower) it is reasonable to devote to the analysis, but also when and how to decide that enough analysis has been done. Watson and Brown (1978) and Nickerson and Boyd (1980) have addressed this question as a formal problem in decision-making. The informal principles that should guide such a decision are set out below. The question has to be tied directly to the reasons that the decisionmaker had for undertaking the analysis. Referring back to Section 6.1, we said there that the major reasons for a decision-maker to enlist the help of a decision analyst are that: (1) he has several options that he cannot decide between; (2) different organizational elements are proposing different options and he needs some objective basis to justify a decision; (3) recommendations from across organizational elements need to be combined into coherent alternatives and evaluated; and (4) the decision must be justified to others. To determine whether enough analysis has been done, we can turn the above question into one that the decision-maker can answer. Can he now distinguish among the options with enough understanding to make a decision? By relying on the iterative form of analysis, rather than an all-or-nothing analysis, the analyst is in the position of being able to tell
184
Practice
the decision-maker what new insights the next stage of analysis might provide as he is contemplating his decision about whether he needs more analysis. The other question, of how many resources to commit to an analysis prior to its execution, is more complex. We conduct an analysis because thereby we expect to stand a better chance of making the decision that is in our best interests. If it is clear that no amount of analysis could affect the choice of action, then viewed in terms of that action alone the analysis has no value. If, on the other hand, we can see that the analysis could lead to many different possible recommendations which might be acted on, then it could have considerable value. What is more, analyses can have value beyond improving the chances of acting in our best interests. An analysis can provide a justification for a decision that otherwise might be difficult to argue for. It can improve communication within an organization about the factors leading to a particular decision. It can lead to improved thinking processes when decision-makers are facing subsequent decisions. All of these aspects are difficult to quantify, and we are not in a position to offer a satisfactory formal means to attempt such evaluation. There is, however, a rule of thumb often quoted in the decision analysis community, and attributed to R. A. Howard, that one should spend about 1% of the resources at risk in a decision on analyzing it. At present this seems the best guide available to the decision-maker on how many resources to spend on a decision analysis. 7.4 PROBABILITY ELICITATION
This section first provides a summary of the research findings relevant to the elicitation of probability distributions. Then the Spetzler and Stael von Holstein elicitation process is described in detail. Finally the subject of discretizing an elicited distribution for use in a decision tree analysis is introduced. 7.4.1 Research findings In addition to the research that has delved into the question of whether people can reliably and validly express their perceptions of uncertainty as subjective probabilities (Section 4.2.2), important research has examined how we should ask experts for probabilistic information. For example, if we were eliciting a probability distribution on the year in which the first human settlement would be operational on the moon, we could ask: (1) In what year is there at least a 10% chance (the 0.1 fractile of the probability distribution) that the settlement would be established? (2) What is the probability that the settlement would be established by 2010?
Analytic tactics
185
(3) In what year are there at least 4:1 odds (80% chance) that a human settlement will be established? (4) What are the odds that a settlement will exist in 2010? These and other approaches that we will discuss are all reasonable. But is there any best approach? Or, if not, are there any approaches that are clearly poorer than the others? We will not review all of the studies that have been conducted in this area, but rather discuss the results of two of the most recent, each of which contain good reviews of previous findings. (For the interested reader a thorough review of empirical studies can be found in Wallsten and Budescu (1983).) Seaver et al. (1978) compared five methods using a questionnaire survey of undergraduate and graduate students. This survey was comprised of general knowledge questions (e.g. What was the population of Canada in 1973?). ^ ls important to state that questionnaires are the elicitation mode of last resort because the questionnaire is an open-loop control system; that is, there is no opportunity for (1) the expert to gather additional information on the meaning of the questions or (2) the analyst to tailor the questions based upon the expert's answers. The results of this study are consistent with other studies and probably robust enough to generalize. The five methods that were examined included giving the subject a value of the random variable and asking for (1) the fractile of the probability distribution or (2) the odds; or giving the subject (3) a fractile, (4) odds, or (5) log odds (the logarithm of the odds) and asking for the value of the random variable. The fractile approaches (1 and 3) are the obvious approaches, yet the odds and log odds approaches have shown promise when the estimation of small probabilities is of key importance. Seaver et al. found that the log odds approach was inferior to the other four in producing a valid interquartile range (that is to say, the range between the 0.25 and 0.75 fractiles - half of the real values should fall between these two points). In addition, they, in common with previous researchers, found that the assessed 0.1 and 0.9 fractiles were too tight using the fractile methods, that is, giving the subject a fractile or an odds and asking for the corresponding value of the random variable. The authors conclude by observing that the results support the anchoring and adjustment process hypothesized by Tversky and Kahneman (1974) and that none of the five methods is especially good. They recommend a combination of approaches such as proposed by Spetzler and Stael von Holstein (1975), the topic of our next section. Ludke et al. (1977) examined a wider range of approaches. Senior nursing students and registered nurses were asked to specify probability distributions on six medical variables (e.g. the systolic blood pressure of American males between the ages of 18 and 70). The nurses were given a method to use and whatever materials were needed, such as graph paper.
186
Practice
Two of the variables had right-skewed frequency distributions, two were left-skewed, and two were centrally distributed. The process was repeated at two different times to gauge the reliability of the methods. The five methods were (a) probability (the first approach in the Seaver et al. 1978 study), (b) log odds - the subject was asked to record odds on a logarithmic scale for ten intervals of systolic pressure (less than n o , 110-120,. . ., more than 190), (c) log-log odds - same as log odds except that a logarithmic-logarithmic scale (if logarithms are a good thing, then logarithms of logarithms must be even better), (d) ranking - the subjects developed rank orders for the likelihood of the ten intervals and then rank-ordered the differences between the first and second priority intervals, second and third priority intervals, and so on, in an attempt to measure the gaps between the intervals of blood pressure, and (e) bisection - first a range of systolic blood pressure that would include all American males was established; then a midpoint was defined such that half of the males would fall below and half above; then, for the lower half, a new midpoint (0.25 fractile) was defined again; until finally there were eight likely intervals of systolic blood pressure for American males. Ludke et al. (1977) compared these five methods on three measures: accuracy, reliability, and acceptability. Again no one method was found superior. The ranking method was most accurate on all six medical variables (right-skewed, central and left-skewed) though not best for all individual distributions. (This result supports our experience in using the ranking method to elicit value curves.) Ranking, probability, and log odds were the most reliable. The probability method was most acceptable to the subjects. Again the major message from the research community is that no one method produces valid results by itself, but multiple methods used during an elicitation session yield greatly improved results. Before closing this section we wish to mention "proper scoring rules." A proper scoring rule is a scoring system for rewarding an expert on the basis of his probability distribution, such that his maximum expected reward coincides with his true, subjective probability distribution. Such incentive systems have been implemented for weather forecasters (see Murphy and Winkler (1984)) and work remarkably well. Matheson and Winkler (1976) provide a detailed discussion of proper scoring rules. 7.4.2 A recommended elicitation process This process for eliciting probabilities was first proposed by Spetzler and Stael von Holstein at a professional society in 1972. This elicitation process has five steps: motivating, structuring, conditioning, encoding, and verifying. The "golden rule" of eliciting expert judgments comes through early in their process: do not ask the expert to do mental aerobics
Analytic tactics
187
while answering the elicitation questions. Mental aerobics include transforming the measurement dimension of the variable into one that he is more familar with and then retransforming the answer back to the dimension the analyst prefers. An example of what not to do is the log-log odds approach from the Ludke etal. (1977) study. The expert should deal with the variable in the manner with which he is most comfortable, and the mathematics needed to transform his answers for analysis purposes should be left to the analyst. Howard etal. (1972) (see Section 8.5.2) deal with this transformation very well. Motivating, the initial step in the encoding process, is often the initial introduction of the analyst to the expert. As such, the analyst should familiarize the expert with the purpose of the analysis, the need to capture the probability distribution on the variable in question, and the details of the encoding process so that the expert knows what to expect. In addition, the analyst should ask the expert about any reservations he may have about providing the information, either because he feels his statements will "commit" him to some performance that may not be possible in the future or because he feels the information is too sensitive for revelation to the analyst. In the first case the analyst can try to reassure the expert that the statement of a distribution is less prone to misuse by superiors than is a single-point estimate. In the second case the analyst may want to enlist the support of the decision-maker in assuring the expert that the analyst will treat the information with utmost care; or, if the decision-maker is not willing to make such assurances, a new analytical approach will have to be devised. The analyst must also be aware of any organizational forces operating on the expert that may lead him to falsify his responses for personal or organizational gain. These concerns must also be voiced to the decision-maker. Second, the structuring step considers the dimension on which the variable is to be measured, as well as any details associated with the definition of the variable. After these details have been decided by the expert, the analyst should explain the model structure to the expert, particularly those variables that interact with the one in question and on which the probability distribution will have to be conditioned. Then the expert should be asked to think through these relationships for himself and propose any new variables that need to be included or any changes to the model structure. This is where influence diagrams are of great value. These changes should then be incorporated into the ensuing steps. The conditioning step refers to the information from which the expert draws in making his judgments, not the conditioning of one variable on others, which was covered in the previous step. It is in this step that the analyst encourages the expert to expand his available information set, identifies any anchors being used, and identifies any assumptions that the
188
Practice
expert feels are key. A valuable technique that the analyst can use in this step is to ask the expert how the variable might obtain some extreme values. This exercise helps defeat any excessive anchoring on central values and causes the expert to bring into play new information that he may otherwise have forgotten. The use of these and other scenarios that both the expert and analyst are free to devise are a good way to ensure that the expert feels his probability assignments are robust and not subject to change when new information is considered. We get to the actual elicitation in the next to last step, encoding. Spetzler and Stael von Holstein emphasize starting the elicitation with the extreme values, not the median, to counteract the central bias (people do not generally spread these extreme values out far enough). That is, we always think we know more than we do. Spetzler and Stael von Holstein favor the use of the probability wheel (devised by them and their colleagues who were at the Stanford Research Institute, now SRI, in the late sixties) for encoding. The probability wheel is spinnable with a fixed pointer attached. There are two pieces of cardboard, each of a different color (for example, blue and orange), that can be adjusted so that one color occupies a certain sized pie shape in the wheel and the other color occupies the remaining part of the wheel. The analyst asks the expert if the value of the random variable is more likely to be greater than a certain number or if the pointer is more likely to land in the blue area of the wheel. Depending on the answer of the expert the analyst increases or decreases the blue area of the wheel until the expert feels the two events are about equal. (This concept of the probability wheel has now been implemented in a decision tree software package.) Most discussants of probability elicitation stress the use of several forms of questions, such as the first four from Seaver et al. (1978). The ranking method that was examined by Ludke et al. (1977) could also be used. Just as the use of several formats for this synthesization process is important, the collection of redundant information is also key. In fact, the multiple forms of questions are often used to obtain the same information. As we mentioned above, this overspecification of judgments often leads to inconsistencies, resulting in improved overall judgments. A final important point that any experienced elicitor would make is that the analyst should be especially attentive to cues the expert gives when he remembers a new piece of information that changes his thought pattern for responding to questions. When this happens the information should be discussed and, if valid, previous reponses should be reviewed and changed, as appropriate. The last step, verifying, involves asking the expert if he really believes his replies. Questions such as "Would you make a decision based upon this distribution?" or "Would you gamble your money in a game of
Analytic tactics
189
fortune based upon this distribution?" can be posed. Also, some of the elicitation questions can be repeated. This discussion has focused upon the elicitation of a continuous probability distribution. Yet there are times when the random event is discrete, e.g. new tax laws will be passed. The elicitation process is unchanged for these discrete events. In fact, the reader should have noticed that the above process for continuous variables was transformed into a series of discrete values to obtain points on the cumulative probability distribution. All of the above steps of motivating, structuring, conditioning, encoding, and verifying apply. However, the ability to use multiple forms of questions and to overspecify the judgments is limited with discrete events. Every discrete event has a negation, and the sum of the probabilities for these two events must equal one. So, if the expert states that the probability that new tax laws will be passed is 0.7, then he ought also to feel that the probability is 0.3 that new tax laws will not be passed. This form of verification should be used extensively with discrete events. 7.4.3 Discretizing continuous distributions The elicitation process described above produces a series of points for the cumulative probability on a continuous random variable. These points are commonly connected by a curved line to yield a continuous distribution. Alternatively, one could fit a polynomial or a "named" distribution (e.g. normal or exponential) to the points. All of these common uses yield a continuous distribution. Yet the typical form of analysis involves a decision tree that requires a discrete (e.g. three-point) approximation of this distribution. Prior to 1983 a number of somewhat arbitrary approaches had been developed for discretizing a distribution. In 1983 two methods were reported based on very different techniques. Keefer and Bodily (1983) give a good discussion of the ad hoc approaches and propose an approach based upon work done by Pearson and Tukey (1965). Miller and Rice (1983) propose another method based upon a mathematical approximation technique, gaussian quadrature. The gaussian quadrature technique appears to perform better based on a limited overlap of test results reported in the two articles. In addition, Miller and Rice describe how to adapt their method for approximating an assessed distribution, rather than a named distribution, that is being used as an approximation. EXERCISE FOR 7 . 4
1. (a) Conduct a probability elicitation session with a fellow student using the probability assessment procedure outlined in this
190
Practice section. For a random variable use the number of times that the pointer of a (fair blue/orange) probability wheel will land in the blue section if the probability wheel were spun five times and the blue area represented 40% of the wheel's area. (b) Calculate the actual distribution of "blue spinners" and compare it with the distribution that was assessed. Which, if any, biases can be identified in the assessed distribution? (c) Now have your colleague assess your distribution with the random variable changed to a probability wheel with 70% of its area blue. Again the wheel is spun five times. Have your colleague compare your assessed distribution with the calculated distribution and identify any biases that are evident. (d) Now you and your colleague should critique each other's elicitation techniques, identifying strengths and weaknesses compared with the procedure described in this section. (e) Now each of you conduct a second assessment for the same random variable, a fair blue/orange probability wheel spun five times with 35% of its area blue. 7.5 VALUE AND UTILITY ELICITATION
As we did in the previous section on probability elicitation, we begin this section with a presentation of relevant research findings for value and utility elicitation. Value elicitation begins with some common definitions of objective, goal, and attribute. Then the topics of value functions, additive value weights, hierarchical additive value weights, and utility functions are presented. 7.5.1 Research Conducting research related to values is inherently difficult because values are truly subjective; there is no right answer. In examining elicitation methods for probabilities, we can use general knowledge questions for which there is a recorded right answer. We have no such convenient recourse when it comes to determining the accuracy of value assessments. In addition, the research that has been done has not really examined some of the more important assessment methods. Schoemaker and Waid (1982) is one of the more recent and comprehensive examinations of value weight assessment. They examined five distinct methods (and several variations within some of the methods): the holistic-regression approach of social judgment theory (see Section 3.5.4), the direct trade-offs between
Analytic tactics
191
attributes, the analytic hierarchy process (AHP) of paired comparisons followed by eigenvalue analysis (see Section 3.5.6), the allocation of points, and the equal weighting of standardized attributes. Their experiment involved the use of college students in the role of a college admissions officer evaluating applicants on four variables. Their findings were that different approaches did indeed produce different results. Accuracy of the assessments was interpreted to be the predictive ability of the weights to reproduce the subject's initial holistic rank order as well as some strength of preference judgments. Three of the methods performed well here; point allocation, social judgment theory's holistic-regression approach, and direct trade-offs. Equal weighting was worst. The students were most comfortable with the more complex approaches (AHP and direct trade-offs). A second research effort, conducted by Hershey, Kunreuther, and Schoemaker (1982), demonstrated five sources of bias in the elicitation of utility (risk preference) functions. First they found that asking the decision-maker for his certainty equivalent of a lottery yielded a greater expression of risk-seeking than when the decision-maker was asked for a probability that equated a given lottery with a stated certainty equivalent. Also, systematic bias in the answers can be introduced by the choice of probability and outcome levels used in the lottery. Third, there is a zero point effect; that is, gains are treated substantially differently from losses. Finally, assumptions about risk transference and difference in the framing of questions yield results that are not consistent with the axioms of utility theory. 7.5.2 Definitions Before talking about values and utilities we need to establish what certain terms mean. We will use the word objective to mean a preferred direction of movement along one or more value or utility variables that we define for the analysis. Goal is taken to be a threshold value on a given value variable which we wish to achieve. A goal may or may not be reached; and we can have many goals, some of which will be more important than others. Finally an attribute is a dimension for measuring value or utility and is therefore equivalent to a value variable. We will sometimes use the word criterion as a substitute for attribute. (Other writers have used the word factor as a substitute for criterion.) A value system therefore will consist of one or more objectives, each of which will have at least one attribute associated with it. In developing a value (or utility) system Keeney and Raiffa (1^76) proposed five properties that define a good value system: completeness, operability, decomposability, lack of redundancy, and minimal size.
192
Practice enjoyment I time spent
T cottage luxury
I ocean access I walking time
I view
Figure y.u Enjoyment attributes. This simple value hierarchy illustrates the clustering of similar attributes. Completeness can be thought of in terms of both the objectives and the attributes. A complete set of objectives is one that spans all issues of concern to the decision-maker. The set of all attributes is complete when they adequately indicate the degree to which all objectives are met. Attributes are operable when they provide a meaningful characterization for the decision-maker of the alternatives being considered; that is, the alternatives can be translated into the value dimensions of the attributes. The independence between preference and uncertainty for a given attribute is the definition of decomposability (required by the axioms). The definition of non-redundant attributes is that a set of attributes provides non-overlapping measures of the alternatives being considered (no double counting). Finally, there is no reason to expand the set of attributes any further than is necessary to meet the above properties. The dimensions by which we choose to define an attribute can take many forms. The most obvious are continuous value variables such as net present value and initial investment from the beach cottage investment example in Section 7.2.2.1. These variables can be measured in monetary units. Enjoyment, the third value variable shown in Figure 7.9, is much harder to quantify. There are times when it is best to equate outcomes to alternatives and measure the value to the decision-makers by making the value scale consist of discrete points representing the alternatives. Section 8.4.3 describes an example of the use of this direct assessment process. However, it is often best to deal with smaller "chunks" than would be encompassed by the attribute "enjoyment" when using direct assessment. Figure 7.11 displays a value hierarchy for enjoyment. Time spent at the beach is linked to the occupancy rate of the cottage and the partnership option. More is clearly better here and time can be measured in weeks per year, so it is a discrete variable that can be considered continuous for all practical purposes. Cottage luxury is another element of enjoyment. The cottage value could be considered a proxy for this attribute; or, if the analysis options were specified in terms of specific cottage plans, direct assessment could be used. Finally, ocean access is broken into walking time (a continuous variable measured in minutes) and view (a discrete variable). Ocean view could be directly assessed. Another distinction is often made between subjective attributes and
Analytic tactics
193
objective attributes. Objective attributes are ones for which the attribute quantity can be measured via an objective process. Examples of objective attributes are those used in the energy production analysis described in Section 6.3.2. The dimensional scale of subjective attributes is defined and measured by the decision-maker or his experts, e.g. the enjoyment attributes above. Objective attributes are often required for analyses that must be justified outside the decision-maker's organization. Subjective attributes are usually more meaningful to decision-makers who are able to decide without outside justification. 7.5.3 Assessing additive value functions Previous discussions have examined processes for defining attributes (Section 7.2.3) and the determination that additive value functions are suitable (Section 3.1.2.4). Before illustrating two methods that can be used in the assessment of value functions, the reader must accept and understand our nomenclature for dealing with this problem. For this discussion we will assume that a cardinal (or interval) scale is adequate for the analysis so that we may choose any two points on the value scale for each function. (A cardinal scale is unique up to positive linear transformations, a + bx. If a ratio scale were required, then the zero value point would have to be found and labeled as such. One other point could then be chosen, yielding a scale unique for positive multiplication, bx.) For each attribute X,-, let xj and xf be convenient upper and lower limits, respectively, for the variable x,. These could be the best and worst conceivable values for the variable; or they could be numbers bounding the alternatives to be considered by the analysis; or they could have some other convenient interpretation. For our purposes we will set x] equal to the highest value point of xf- and xf equal to the lowest value point, and define V;(xJ) equal to 1.0 and V;(xf) to 0.0. Note that, for some variables such as net present value of the beach cottage, more is better so v,•(*,•) will increase with x,-. For other variables such as initial investment in the beach cottage, more is worse so f,-(x,-) will decrease as x, increases. Finally, there are variables for which value rises and then falls; for these variables x] will be in the interior of the range of x* being considered for the analysis. If occupancy rate had been chosen as one of several proxy attributes for both enjoyment and net present value, then it would exhibit this property. Finally we will define the total value for all attributes X to be: V{x) =
where w{ is the value weight to be applied to the value function of x,-. (The interpretation of wi is provided in the next section.) If we normalize the
194
Practice
weights so that they add to i.o, then this convention produces v(x*) equal to i.o and v(x°) equal to o.o. Fishburn (1967) gives twenty-four methods for assessing value functions. We present two of the most commonly used and most effective methods here, and the interested reader should consult Fishburn (1967) or Keller and Farquhar (1974) for more detail. The first method that we will examine is the bisection (or midpoint) method, which is directly analogous to the probability assessment method of the same name. The initial step in assessing a value function with any method is to determine the end points, x] and xf. The upper bound on value is the easiest to define with the decision-maker; when the alternatives have not been totally defined, it can be defined as (1) the point at which he is satiated and does not care about additional improvements or (2) the point beyond which it is no longer practical to go, as a result of resource or technological or other constraints. When the alternatives are known, it could be equal to the best alternative for this particular attribute. Establishing the lower bound with the decision-maker is the point at which the analyst must use as much of his influence on the decision-maker as possible because the decisionmaker will want to set this value higher on the scale of Xi than he should. Remember that setting a cardinal value of zero to xf is establishing xf as the base point from which measurement begins and does not say that xf has truly zero value. However, setting the value of xf to zero does establish that any value of #, below xf will not be considered, and an alternative associated with it is being ruled out of the analysis, xf is often called the threshold value of *,. So an alternative with extremely high value scores on all attributes except x, will be ruled out if its value on *, is below xf. The decision-maker must be reminded of these trade-offs whenever a value for xf is being established. Once xf and x] are established and defined to have value 0.0 and 1.0, respectively, the decision-maker is asked during the application of the bisection method to determine which value of the continuous variable x, is halfway between these two points in value; that is, the improvement from xf to x™ is just as attractive as the improvement from x™ to xj. A very important point needs to be made here about the best and worst ways to ask this and all other assessment questions. The worst way is to ask the decision-maker to state the value of x, such that v^x?) equals 0.5. Almost all decision-makers are new to quantifying their preferences and are overwhelmed by their choices. They could pick any value and do not know where to start thinking about the answer to such a question. Helping them is amazingly easy. Instead of asking which value of x, is the midpoint of value between xf and xj9 ask "Is xf4 (a specific value of xj) the midpoint?" Now the decision-maker only has to decide yes or no. The major secret to assessment is to ask yes/no questions or ask for ordinal
Analytic tactics
195
Figure 7.12 Bisection method. The bisection method should be used to find at least the five points shown above. More points can be assessed if needed.
preferences (as illustrated by the next method). After participating in the assessment process for a while, the decision-maker will begin to internalize the quantification procedure and volunteer answers to the more general question, "What is the midpoint?" Given our notation, if x™ is the midpoint of value for *,, then v&xT) = 0.5. We can now repeat this process to find the midpoints for the invervals [xf, x?1] and [x™, xj]. These points would be defined to be 0.25 and 0.75 on the value scale, respectively. Often these three points will be enough to sketch a reasonable value curve, as shown in Figure 7.12. Additional points can be assessed, if needed. There are difficulties with this method in practice. The foremost one is a bias people seem to exhibit that can best be explained as follows. Having defined xf and x], the decision-maker knows he wants to achieve x] and stay as far away from xf as possible. When asked whether potential midpoints for the value function are the real midpoint, he is drawn to say the midpoint is close to xj. We call this the goal-directed bias. Yet most value preferences exhibit a property that economists call "decreasing returns to scale" or "decreasing marginal returns." That is, if three equal-sized units of Xi are defined between xf and x\ using the dimension of X{ rather than any value scale, then the first improvement from xf is
Practice
196 Value Least f important 1
Intermediate importance
Most important
Ax ;
Figure 7.13 curves.
Ax ;
Ax..
Decreasing returns to scale. This is the most common shape for value
worth more than the second, which is worth more than the third (see Figure 7.13). The mathematical interpretation of decreasing returns to scale is that the value function i/,-(x,-) is a concave function, meaning that any straight line connecting two points of the curve will be below the curve. If this property is operative, then x™ should be closer to xf than to Xj. Often a detailed explanation of "decreasing returns to scale" helps because most decision-makers are familar with this concept in practical terms, if not in theory. Yet they must be constantly reminded as new value curves are assessed. Other potential shapes for value curves are shown in Figure 7.14: constant returns to scale, increasing returns to scale, and an S-curve (increasing then decreasing returns to scale). Once the decisionmaker is used to the bisection method, we can ask him to choose which curve is most appropriate for a given variable as a way to speed the process along. The analyst must constantly be on guard for this goaldirected bias because it invariably reappears after the analyst thinks he has cured the decision-maker with his articulate explanations. The bisection method must be modified for use with discrete value variables. Now we must find the value of x, that is closest to being the midpoint between xf and xj. If there are a lot of discrete points between xf
Analytic tactics
Constant returns to scale (linear)
Figure y.14
Increasing returns to scale (convex)
197
S-curve
Other shapes for value functions.
0.25--
0.0 0.0
2.0
4.0
6.0
10.0
Walking time (minutes)
Figure 7.15 Value function for "walking time." The bisection method is used to assess the value curve for walking time.
and x;\ then clearly the bisection method will work quite well. If, however, there are only two or three intermediate points, then the bisection method quickly turns into the second method we will discuss. First, however, we will look at an application of the bisection method to the beach cottage attribute "walking time to the beach." Defining the top and bottom points of the value scale we find that one half minute is the
Practice
i98 Value Second approach
1.0 -
0.8 v, (xf)
v, (x?)
First approach
Figure y.i6 Differences method. The first approach of the differences method adjusts the value of x, until equal increments of value are found. The second approach directly assesses the incremental values of the initial XjS.
quickest time (the cottage is located right on the beach) and ten minutes is the time for the cottage furthermost from the beach. As shown in Figure 7.15 the decision-maker responds that 7.5 minutes is his value midpoint between these two extremes. He cites the increasingly difficult job of carrying beach chairs and coolers back and forth to justify this assessment. Nine minutes is his midpoint between 7.5 and ten minutes; and 5.25 minutes between 0.5 and 7.5. Plotting these points as shown in Figure 7.15 enables us to draw a smooth curve between them. We then check to be sure that 3.5 minutes would be his midpoint between 0.5 and 5.25. The second method for eliciting value functions is analogous to the ranking method that was examined by Ludke et al. (1977) for probability assessment (Section 7.4.1). This approach works for both discrete and continuous value variables, which is a big advantage. If the variable is discrete, then all the discrete positions between xf and x] are used. If not, then the analyst picks three to six equally spaced values of x?, say, xf, xf, xf, and xf. Now we construct a preference order on the value differences for thefiveintervals [xf, xf], [xf, xf],. . . [xf, xj]. We must point out that there has been a great deal of contention in the literature on whether it is
Analytic tactics
199
plausible to suppose that people can synthesize an order on value differences. There are no requirements in the axiom systems leading to the validity of additive value systems for this to be true. Yet our experience has been that real decision-makers can and are quite willing to rank-order these value differences. As evidence we present our success with the "decreasing returns to scale" argument for overcoming the goal-directed bias. The decision-maker recognizes that thefirstincrement of X; above xf is more valuable than the last interval of x, leading to xj. So responses to questions such as "Is the increment in value going from xf to xf more desirable than the increment between xf and *,*?" can be synthesized. Once we have assessed a rank order on the five increments for Xj presented above, we have two options. First we could redefine the increments in such a way that the new increments would be closer to five equally valuable increments. This redefinition would go on until the increments were equally valuable. Then for the five-increment approach we would have the 0.2,0.4, 0.6, and 0.8 value points defined, as shown in Figure 7.16. A second option would be to obtain a value scale for the increments directly, either by direct assessment or, even better, by the balance-beam procedure that we will describe for assessing value weights. It is important to realize that a value scale on differences is a ratio scale; zero value for a difference is really zero value. Figure 7.16 also illustrates this second option. We have a preference for the second option, scaling the value differences directly, because it can be done reasonably well; and the first option gets rather tedious, especially for the decision-maker. To continue our example, let us now apply this ranking method to the attribute for time spent at the beach. The bottom of the value scale is established to be one week per year. The decison-maker's utility satiates at six weeks per year since he would be unable to take any more time away from his job. It is easy for the decision-maker to rank-order the differences in value between the first and second weeks, second and third, and so on. Thefirstdifference is most valuable, the second second, etc. Using a direct assessment technique we set the smallest interval (five to six weeks) equal to an arbitrary 10 points. We will normalize the scale when we are finished. We then ask the decision-maker what the second worst interval (four to five weeks) is worth if the worst is worth 10 points of value. He responds that this second worst interval is worth twice as much, 20 points. Likewise, he says that three to four weeks is worth 3 5 points, not quite twice as much as four to five weeks. 65 points is assigned to the second best interval, two to three weeks, again not quite twice as much as the one before. Finally he says the best interval one to two weeks is worth 90 points. These points total 220, which is used in the normalization. The value curve is plotted in Figure 7.17.
Practice
2OO
1.0
0.8
0.6
0.4
0.2
0.0*-
3
4
Time spent (wks/yr)
Figure y.iy Value function for "time spent." The second approach of the differences method is used to assess the value curve for time spent at the beach.
7.5.4 Assessing additive value weights There are many approaches for assigning weights to value functions, ranging from equal weighting to deliberate comparisons across the attributes. The SMART approach (Section 6.4) and the percent distribution approach (distribution of 100 points across the attributes) are straightforward, but not guaranteed to produce valid results unless the decision-maker has experienced the more deliberate procedures described below. For example, our experience has found that the point allocation and other direct assignment methods usually produce weights that are too close together compared with those produced by the more deliberate procedures. Below we describe the numeraire and balance-beam approaches, which in our opinion are more reliable. Before describing these two elicitation procedures, let us first acquaint the reader with the difference between importance weights and swing weights. Importance weights are used to reflect the general importance of one attribute over another, without regard to the particular decision application, or, more specifically, the difference between the worst and best value points of each attribute. When this difference from worst to best is not explicitly referenced in assessing weights, we obtain some general notion of importance, which is subject to great variation and
Analytic tactics
201
argument among decision-makers. The correct concept of value weight is the swing weight, in which the decision-maker is explicitly comparing the swing in value (worst to best) of the attributes in question. These weights reflect well-defined quantities, and the decision-makers can have meaningful discussions about what their values should be. The swing weight should also be changed when the range from worst to best for a given attribute is changed. (Throughout the remainder of this book, the swing weight concept will be implied when the word "weight" is used in a value sense.) Most procedures for assessing the value weights of an additive value function begin by establishing a rank order on the attributes in terms of the swing in importance from worst to best. If there are n attributes and we define n vectors to be: then we must ask the decision-maker which vector he favors the most, second, . . ., and last. Again this is best asked via questions like 'is x(/) preferred to x(;)?", especially when n exceeds six or seven. Once the ranking is established, we can proceed in several different ways. For ease of discussion let us reorder the attributes so that Xi is the first ranked attribute, X2 is the second ranked attribute, and so on. The numeraire method works well if the value variables are continuous. Then using Xi as the numeraire, there must be some value of x\, less than xj, say x\, such that we are indifferent between the vectors: and
x< 2 >.
If this is the case, then wx vi(x\) = w2
and hence which can be read from the value function. Repeating this procedure with the third ranked attribute, then the fourth and so on, yields n — 1 equations relating the weights to each other. Finally we can use the normalization equation: W1 + W2 + . . . + wn = 1.0 to solve a set of n linear equations. Often, however, in practice it is best to overspecify the weights by asking more than n — 1 questions because trade-offs across disparate attributes are very difficult to synthesize. By
202
Practice
overspecifying the set of equations, inconsistencies in the decisionmaker's judgments can be found. We are not trying to show the decisionmaker how irrational he is because he was inconsistent. Rather we are providing new ways to help the decision-maker synthesize the tough choices before him. This numeraire process depends on the validity of the value functions. By showing the decision-maker inconsistencies in his weight judgments we are providing a check on the value functions, an error in which may be the cause of the inconsistency. To illustrate the numeraire assessment approach let us return to the enjoyment hierarchy in Figure 7.11 and assess the weights for the attributes under "ocean access." The decision-maker says that the swing in value for "walking time" is more important than that for "view." We refer back to the value curve for "walking time" (Figure 7.15) and ask if the "view" swing is as important as a change in walking time from ten to five minutes. He synthesizes: "view" is more important than that. We finally narrow the weight of "view" to be equivalent to a change in walking time from ten to four minutes (0.85 on the value curve). We now have two equations and two unknowns: + ^32 = 1.0 = 0.85 where w$\ refers to the relative weight of "walking time" (relative, that is, to the other node at that level, "view"), w32 to "view," w3 to "ocean access," w2 to "cottage luxury," and W\ to "time spent." Making the substitution we find that the relative weight for "walking time" is 0.54 and the "view" relative weight is 0.46. A second approach, which was derived from the conjoint process described in Kuskey et al. (1981), we will call the balance-beam approach. Here the value functions are not used at all. Rather the vector x (1) of the most important attribute is compared to a composite vector of the second and third most important attributes, x (23) : X(23) = (*?, *£, *3, X%, . . ., X°n). We often use a balance-beam analogy to provide the decision-maker with an intuitive understanding of our questions. Since one cannot always expect to find equalities in this process of comparing one improvement from worst to best with two such improvements, the analyst searches for inequalities that bracket the vector x (1) . For example, suppose the decision-maker replies that he prefers the composite of the second and third most important attributes to x (1) , but prefers x (1) to the composite of the third and fourth most important attributes. We now have a set of inequalities that bracket the value of W\:
Analytic tactics W\ < W\ >
203
Wi + W$ W3 + W4.
Typically, after the decision-maker has synthesized many inequalities of this kind, we can use them to obtain numerical expressions of the weights. This is done by assigning an arbitrary value (such as 10) to the least important weight, and using the inequalities as well as additional quantitative judgments of the decision-maker to determine the weights of the more important attributes. Once this is done the normalization equation can be used to rescale the weights. Section 8.6.3 provides an example of how this approach is applied. This approach has even been used to assess value functions and weights, simultaneously, for discrete value variables. 7.5.5 Constructing hierarchical additive value weights We describe the use of value hierarchies for problem- and model-structuring in Section 7.2.3. Now we must deal with assessing weights for the various levels of the hierarchy. Just as we talked about top-down and bottom-up structuring, there is also top-down and bottom-up assessment of the hierarchical weights. The bottom-up approach more accurately captures the swing weights of the hierarchy. Once the value functions have been assessed for all of the bottom-level attributes, and value weights have been assessed at the bottom levels of the hierarchy, then single attributes from each of the bottom-level groupings are chosen (often the most important) and compared with each other. If the hierarchy is large, then these comparisons may be done in several subsets, and then finally single attributes from each subset sampled to complete the comparisons. Usually the number of levels in the hierarchy determines the number of samples that must be compared before the top-level attributes can be weighted. To illustrate the bottom-up assessment approach let us return to the enjoyment hierarchy in Figure 7.11 and use the weights assessed above with the numeraire approach. First we distinguish between relative weights and cumulative weights in the following manner. The cumulative weight for "walking time," W31, is defined to be the product of the relative weights above it, W3W31. This is illustrated in Figure 7.18. Now to assess the relative weights at the top of the hierarchy, we choose one of the bottom-level nodes of "ocean access" and "cottage luxury." The decision-maker rank-orders the swings in value for these attributes as follows: (1) "time spent," (2) "cottage luxury," and (3) "walking time." Now we use the numeraire approach (picking "time spent" as the
204
Practice enjoyment i
time^i spent
cottage"* luxury
ocean " 3 access
I
31
walking" time W3l = w3w3l
view"' 32 W32 = w3w32
Figure 7.18 Relative and cumulative weights. Cumulative weights are the product of all relative weights from the given level in the tree to the top of the tree. numeraire) to assess the other two weights. The change between the best and worst points on the "cottage luxury" scale is considered equivalent to the change of one to three weeks at the beach (0.7 on the value curve, see Figure 7.17). "Walking time" was measured at 0.6 on the numeraire's value curve by comparing the change from worst to best walking times to a change from two to six weeks of time spent. Now we have four equations and four unknowns: W2 = 0.7 wu W31 = 0.6 wu W31 = W3W31 = 0.54 w3 (cumulative weight equation), W\ + W2 + W3 = 1 (normalization equation). Solving for W\ we find it to equal 0.36, making w2 equal to 0.25, and W3 equal to 0.39, making "ocean access" the most important factor at the top of the hierarchy. Solving for W31, we find it to be 0.21. This high ranking of "ocean access" is due to the fact that "walking time' was considered to be 60% of "time spent" and that "walking time" was only slightly more valuable than "view." This result can be used as a validity check on the results to ensure that the calculated weights make sense to the decisionmaker. 7.5.6 Assessing utility functions As we have discussed before (see Section 3.3) a utility function quantifies the decision-maker's risk preference for a specific decision. The analytic strategy section on INTROSPECTING (6.3) defined an approach for assessing a multi-attributed utility function. In this section we will focus on the assessment of a utility function on one attribute. Many decision analysts would then convert the multi-attributed values of the alternatives into value scores on this one attribute (using it as a numeraire) and then calculate the utilities for the alternatives using the single utility function. We shall use the beach cottage investment decision to illustrate the
Analytic tactics
205
$190,000
$100,000
$190,000 $145,000$100,000
Figure 7.19 Risk preference lotteries. The two most common risk-preference assessment questions are (1) asking for the certainty equivalent (X) of a lottery and (2) asking for the indifference probability (p). 1.0 -
0.8 -
0.0 100
140 160 Wealth ($000)
190
Figure 7.20 Risk preference function. Risk aversion is typified by a concave function as shown above. A straight line would indicate risk neutrality (expected value) and a convex function signifies risk-seeking preferences.
206
Practice
elements associated with the assessment of a utility function. Suppose we had completed a probabilistic analysis of the investment and found that the minimum net present value of the cottage was $20,000 (US) and the maximum was $120,000. This would be all that we needed to know if the decision-maker had accepted the delta property for utility functions (Section 3.3.3). However, let us not presume this. Rather we have to start with the minimum and maximum NPVs of wealth for the decision-maker. If the decision-maker starts with an initial wealth of $100,000 and puts up $20,000 to achieve a $20,000 NPV, then the minimum wealth position is $100,000 (100,000 — 20,000-1-20,000). Alternatively, if he puts up $30,000 in a partnership and achieves the maximum NPV (after splitting with his partner), then this maximum wealth position is $190,000 (100,000 — 30,000+ 120,000). So we need to assess a utility function over NPV of wealth from $100,000 to $190,000. While this investment is almost surely an increase in the decision-maker's wealth, he must consider other uses of his $20,000 to $30,000 over the time frame of the beach cottage investment to be able to justify it. Putting the money in the bank could also increase his wealth position. Since a utility function measures risk preference, the only way to assess one is by asking the decision-maker a series of lottery questions. We start by assigning the following values: u( 100,000) = 0.0 and #(190,000) = 1.0.
Figure 7.19 depicts two lottery questions that we could pose to the decision-maker to help synthesize this utility function. The first lottery asks for the wealth position that would make the decision-maker indifferent to afifty-fiftylottery over the best and worst wealth positions possible. Suppose his answer to this question is $130,000. The second question asks for a probability, p, of $190,000 that is equivalent to $145,000, assuming that the starting wealth is obtained with 1 — p. His answer here is 0.7. These two answers are plotted on the curve in Figure 7.20. If this curve is correct, the decision-maker will say that he is indifferent between a wealth of $150,000 and a fifty-fifty lottery over wealths of $190,000 and $130,000. Likewise, he would be indifferent to $120,000 and a fifty-fifty lottery over $145,000 and $100,000. Can you explain why? EXERCISES FOR 7 . 5
1.
Place yourself in a position where you have to buy a house. Suppose that you have been talking to a real-estate agent and given this agent a price range with which you are comfortable. Unfortunately you are called out of town on an emergency before you can look at any of the
Analytic tactics
207
houses. However, you do have time to develop a value structure for a friend of yours to use in evaluating the houses: House value
I
I
I
I
Yard size
Bedrooms I •
Utility expenses per year
• Number
1 Size of master bedroom
(a) Define your units of measurement for each attribute. (b) Determine your minimum and maximum thresholds for each attribute. (c) Use both the bisection and the value difference ranking methods to assess your value curves for each of the four attributes (one method for each attribute). Which method did you use for assessing the value curve for number of bedrooms? Why? (d) Use both the numeraire and balance-beam methods to assess the value weights for this four-attribute hierarchy. Compare the weights derived from the numeraire assessment to those assessed with the balance-beam method. Are there any conclusions or trends? Which set of weights do you believe most accurately reflects your preferences? (e) Now define at least five houses in terms of these four attributes. Be sure that no house is at the bottom of every value function or at the top of every value function. First, develop a holistic rank order of preference for each of these five houses. (Which house do you favor most, all attributes considered? Second, third, fourth, and fifth?) Next, use the weighted value curves to derive a composite value for each house using both sets of value weights. Now you have a holistic rank order, value scores for the numeraire method, and value scores for the balance-beam method. Do the rank orders implied by both sets of value scores agree with your holistic rank order? If not, which rank order do you have most confidence in at this point? Often decision-makers see results at this point that cause them to want to change their value weights and/or curves. Do you? 2.
Use the following bottom-up weight assessments to compute all of the relative and cumulative weights for the hierarchy shown below:
2O8
Practice
Objectives
i Attributes
411
I
I
01
02
i
r
412
421
422
423
4221
4222
0.6
0.4
Bottom-up weights 1. 2.
0.4
3.
0.7
4.
0.5
0.4
0.2
0.3 0.5
3. Develop the equations needed to compute the relative weights for a value hierarchy from a set of bottom-up weights. 4. Using the following lotteries, assess your own risk preference function for money over the interval [o, 10,000]. Use whatever unit of currency is familiar to you. X\ x2 x3 x4
~ ~ ~ ~
{o, 10,000; 0.5} {o, X\\ 0.5} {xu 10,000; 0.5} {x2,x3; 0.5}
Graph the assessments. Are you happy with the results? Modify any of the assessments and sketch the resulting risk preference function. 5. Repeat the fourth exercise with the following lotteries. 5,000 2,500 7,500 5,000
~ ~ ~ ~
{o, 10,000; pi} {o, 5,000; p2} {5,000,10,000; p3} {2,500, 7,500; p4}
Compare the results from this exercise with those of exercise four. Are they the same? If not, do they support the research results discussed in Section 7.5.1?
Analytic tactics P(u(x))
1.0--
0.0
Figure 7.21 Stochastic dominance. If more of x is better, then A is dominated by both B and C. J.6
SENSITIVITY ANALYSIS
Any discipline that utilizes quantitative analysis can benefit from an examination of variations in the solution as input parameters are varied within a reasonable range. Some disciplines, such as linear programming, have well-defined procedures for conducting sensitivity analyses while others are more ad hoc. Decision analysis has some standardized approaches to sensitivity analysis; but, as yet, no general purpose heuristics, equivalent to slack variables in linear programming, have been developed to aid the analyst. Sensitivity analysis is conducted during early iterations in decision analysis so that the analyst and decision-maker can determine where additional modeling and assessment efforts are needed to improve the quality of the analysis. These sensitivity analyses are usually simple variations of one parameter at a time. Then, late in the analysis the sensitivity analysis helps the decision-maker understand the implications of his judgments and determine whether any alternatives are dominated by the others. We call one alternative dominated if there are no circumstances for a defined set of parameter variations such that the alternative in question is preferred. Often this concept of dominance is used in relation to the final probability distribution over utility (or value) for the alternatives, and is
Practice
2IO
Table
Beach cottage results
7-i
NPV
Initial investment $
$
Value
0 0
190,000 170,000
100
57 85
170,000 155,000 110,000 100,000
Value
• Enjoyment value
Total value
Alternatives i 2
3 4 5 6
30,000 30,000 27,000 24,000 20,000 20,000
100 100
0.3
100 90
70
75
70
61 11 0
70
7* 39
0.3
0.4
78 78
15 0
59
30
called stochastic dominance. Figure 7.21 illustrates this stochastic dominance, in which alternative A is dominated by B but the distributions for alternatives B and C cross over, so neither can be said to dominate the other. An example of stochastic dominance can be found in Section 8.2.2. Stochastic dominance is a more robust examination of the optimum policy than just looking at the expected value. However, it holds all inputs to the analysis fixed; therefore, the conclusion of stochastic dominance may well change if some of the inputs are changed. As a result stochastic dominance is usually only used late in an analysis when the decisionmaker is quite comfortable with his model structure, probabilities, and values. Policy region analysis, on the other hand, systematically varies the values of probabilities, values or both to determine in which cases one alternative is preferred. We classify policy region analyses as one-way, two-way and three-way. (We could expand this to four or more if there were ways of displaying or visualizing more than three dimensions in this three-dimensional world.) A one-way policy region analysis varies one parameter and holds the rest constant; or, if the parameter is a probability or value weight, then its peers are held in constant proportions but raised or lowered as a group to satisfy the constraint that their sum must equal 1.0. In order to illustrate these various forms of policy region analyses, we will return to the beach cottage decision. Suppose an analysis of six alternatives had yielded the results shown in Table 7.1. The three value attributes are listed. Both dollar and value results are listed for the initial investment and NPV attributes. Note that the value scale on NPV is linear and that the value scale on initial investment exhibits decreasing returns to scale. The far right column shows the results for value weights of 0.3 for initial investment, 0.3 for NPV, and 0.4 for enjoyment. Figure 7.22 illustrates a one-way policy region analysis as the weight of initial
Analytic tactics
0.1
0.2
211
"0.3
0.4
0.5
Initial investment
Figure 7.22 One-way policy region analysis (proportional). The weight of "initial investment" is varied with the weights of the other two value variables being kept in the same proportion so that all three weights add to 1.0.
0.7
0.3 0.4 Initial investment 0.7
0.6
0.5
0.4
0.3
0.2
0.1
Enjoyment
Figure j.23 One-way policy region analysis (NPV-constrained). This sensitivity analysis is conducted with the weight of "NPV" held at 0.3.
Practice
212
0.2
0.4 0.6 Initial investment
0.8
Figure 7.24 Two-way policy region analysis. While all three weights are varied in this policy region analysis, it is only considered a two-way analysis since the sum of the three weights must equal 1.0. investment is varied between 0.0 and 0.5. The decision-maker might justify a weight of zero since the initial investment is already included in the NPV calculation. On the other hand this initial outlay may be very disruptive of the decision-maker's lifestyle so he may wish to weight it very heavily. The crossover point between alternatives 1 and 4 occurs at an initial investment weight of 0.28. Note that alternative 3 is very close to this crossover point. Options 2, 5, and 6 are clearly dominated in this one-way policy region analysis. Another approach to the above one-way sensitivity analysis would be to vary the weights of initial investment and enjoyment, while keeping the weight of NPV constant at 0.3. Figure 7.23 depicts this sensitivity analysis. The linear curves for each alternative were found by setting the weight of initial investment equal to w and the weight of enjoyment equal to 0.7 — w. So the equations for the alternatives are: Options 1
Values 100
—
IOOW
2
86.4 -
90W
3 4 5 6
75.9- iSw 67.3 + i$w 13.8+ Ssw IOOW
Analytic tactics
213
The only difference between these two forms of one-way sensitivity analysis was the assumption of how to vary the other two weights in proportion to each other. Figure 7.24 uses a two-dimension x — y plot to depict the results of simultaneously varying the three weights (a two-way policy region analysis because the third weight can be uniquely determined from the values of thefirsttwo). Policy region analysis gets it name from this figure because the areas labeled 1, 3, 4, and 5 depict those areas where alternatives 1, 3, 4, and 5 are preferred, respectively. The weights for initial investment and enjoyment are on the x-axis and y-axis, respectively. Lines drawn parallel to the line connecting the (o, 1) and (1,0) points represent different values of the weight of the third variable, the weight of NPV. If NPV is weighted zero, then the (o, I ) - ( I , o) line is the space of weights for initial investment and enjoyment. For NPV weighted at zero, alternative 1 is preferred when the weight of enjoyment is greater than 0.74 (initial investment is weighted below 0.26); alternative 4 is preferred when the weight of enjoyment is between 0.42 and 0.74; and alternative 5 is preferred when enjoyment is weighted below 0.42. The dashed line corresponds to the baseline weight for NPV, 0.3. The dotted line corresponds to a weight of 0.6 for NPV. As can be seen from this figure, alternatives 1 and 4 account for nearly all of the policy region, and the baseline weights occur reasonably close to the boundary between these two regions. Alternative 3 does obtain a small portion of the policy region. This policy region analysis should be used in conjunction with the previous one-way policy region analysis that found alternatives 1, 3 and 4 to be close contenders near the baseline weights. These policy region analyses suggest that alternatives 1 and 4 are nearly equal in value, and that more weight on initial investment favors alternative 4. Also, NPV and enjoyment are so correlated that, if we hold the weight of initial investment constant, the preferred option may change from 4 to 1 to 3 as we vary the weights of enjoyment and NPV, but their values will be so close that the decision-maker would be equally happy with any of the preferred alternatives. The analyses suggest that alternatives 2 and 6 can be discarded. The above examples only focused on the value weights. However, similar sensitivity analyses (including three-way policy region analyses) are done with probabilities or a combination of probabilities and value weights. Such illustrations are shown in Section 8.6. EXERCISE FOR
J.6
Suppose you are helping a friend buy a house. Using the value structure from the exercises at the end of Section 7.5, your friend has evaluated five equally expensive houses on three attributes; yard, size, bedrooms, and
214
Practice
operating expenses. Your friend's value assignments for the five houses appear below: Houses I 2
3 4 5
Yard
5
40 40 90 3O
Bedrooms
Expenses
50 40
90 60 90
10
40 60
10
Helping your friend assess his value weights yields 0.2 for yard, 0.5 for bedroom, and 0.3 for expenses. The result is that the first house has the highest value, but the fifth house is close and houses four and one seem attractive when you think about either the yard or the expenses. (a)
Plot the two-way policy region analysis for these weights to help your friend determine the sensitivity of choice to the weights assessed.
(b)
Your friend says that any of the above weights could be off by as much as 0.1. Are any of the houses besides the first one preferred in this region?
8 APPLICATIONS
8.1 GENERAL INTRODUCTION
The primary purpose of this chapter is to use real applications of decision analysis to illustrate the analytic skills described in the previous chapter, "Analytic tactics." So this chapter has the same organization of sections as Chapter 7, with the exception that problem- and model-structuring are each allocated a separate section here to emphasize the difference between the two and the importance of structuring in general. Our secondary purpose with this chapter is to provide examples of real decision analyses across the decision-making spectrum: commerce/ industry, public policy, regulations, and the professions (e.g. the medical field). A guide that relates the applications presented in this chapter to these decision-making areas is presented below: Commerce/industry 8.2.2 New product and facilities Hertz and Thomas 1983 planning 8.4.2 Facilities investment and Spetzler and Zamora expansion 1984 8.6.2 Merchandizing Dyer and Lund 1982 Medical Fryback and Keeney 8.3.3 Indexing trauma severity 1983 8.7.2 Cardiac catheterization Macartney et al. 1984 Public policy Project appraisal Howard et al. 1972 8.5.2 Seeding hurricanes Kirkwood 1982 8.7.3 Nuclear power plant siting Evaluation of programs 8.2.3 Selection of trajectories Dyer and Miles 1976 8.4.3 System design Buede and Choisser 1984 215
2i 6
Practice
Resource allocation 8.5.3 Magnetic fusion choices 8.6.3 Marine Corps program development Regulatory 8.3.2 Emission control strategies
8.2
North and Stengel 1982 Kuskey et al. 1981 North and Merkhofer 1976
PROBLEM-STRUCTURING
8.2.1 Introduction This section provides two examples of problem-structuring; that is, the definition of the problem space, including boundaries, decision-makers, and alternatives. The new product and facilities planning analysis described by Hertz and Thomas (1983) illustrates the use of a financial model to help eliminate inferior alternatives. The search for an appropriate analytic methodology is examined in the trajectory selection case study of Dyer and Miles (1976). Dyer and Miles recognized the importance of matching the methodology and its required subjective inputs to the decision-makers and experts available within the organization. Inattention to this point can doom the best formulated model. 8.2.2 Hertz and Thomas (1983) — new product and facilities planning The disguised ABC corporation was considering a change in strategy in the egg carton market, to which "decision and risk" analysis was applied as an aid for the decision-makers. For several years ABC had been developing "Egg 'N Foam," a plastic product which would compete with paper egg cartons; and strategic options ranged from abandoning the product to considerable expansion. Before reasonable alternatives could be specified, the analysts engaged in significant problem definition and investment analysis. The first step was to construct a simple investment flow chart (see Figure 8.1), which sized the egg carton market, ABC share price, ABC revenue, manufacturing costs, overhead, investment, and the resulting profit and return on investment. From this, more detailed flow charts were developed, assumptions (e.g. market share) stated, data (e.g. forecasts of total egg production) gathered, and a deterministic model built. One of the major uses of the model was a sensitivity analysis of plant operating characteristics. Different equipment combinations and plant sizes were examined, resulting in the definition of two primary plant sizes — small (one extruder and two thermoformers) and super (two extruders
Applications
217
Total egg carton market
ABC foam share
X
ABC selling price
ABC revenue ABC manufacturing costs
ABC total overhead \ Profit
I
Return on investment
t
Investment
Figure 8.1. Initial Egg'N Foam flow chart. Investment flow model used to structure alternatives. Source: Hertz and Thomas 1983, Figure 1. and four thermoformers). The analysis also found that the two most sensitive factors in determining investment return were demand and the caliper (or thickness) of the Egg 'N Foam. This analytical approach to problem-structuring provided the options that were then analyzed. These four investment alternatives were (see Figure 8.2): (1) superplant/fast, (2) superplant/slow, (3) small plant/fast, and (4) small plant/slow. The results of the probabilistic decision analysis are shown in Figure 8.3, which demonstrates that the fast superplant alternative stochastically dominates the other three alternatives. 8.2.3 Dyer and Miles (1976) - selection of trajectories In 1973 Dyer and Miles used decision analysis in a group decision context to help a Science Steering Group (SSG) within the US National Aeronautics and Space Administration (NASA) select a pair of trajectories to be used for the launch of two spacecraft in 1977. These spacecraft were to swing by Jupiter in 1979 and Saturn in late 1980 or early 1981 on their
Practice
218
Key: MW = Midwest SE = Southeast SW = Southwest NE = Northeast
Super plant-strategy Number of thermoformers
Total investment = $8 million 1968
1969
1970
1971
1972
1973
1974
1975
1976
Small plant strategy Number of thermoformers Fast ISW
SW SW 1
'
MW
MW
I—
1
r—
J
Slow , __ IMW
IMW
SE
ISE
ISE
INE
Total investment = $8 million
J
1968
1969
1970
ISW I
1971
1972
1973
1974
1975
1976
Figure 8.2. Egg'N Foam investment strategies. Alternatives that were the result of the problem-structuring process. Source: Hertz and Thomas 1983, Figure 3.
Applications
219
Probability /1.0 Strategy 4
Strategy 2 /
Strategy 3/
Strategy 1
Figure 8.3. Graph of cumulative probability density functions for each strategy. Strategy 1, the fast implementation of the superplant, stochastically dominates the other strategies. Source: Hertz and Thomas 1983, Figure 5. way out of the solar system. The Science Steering Group was composed of the team leaders of specialized science teams, which are listed in Table 8.1 along with their primary investigations. The selection process was governed by the following guidelines: (1) The process should focus on obtaining a trajectory pair compatible with both the science requirements and the mission constraints. (2) The process should be compatible with the Project resources allocated for mission analysis. The existing Project management structure and science interfaces should be used. (3) The process should not divert the efforts of the SSG from other Project activities, nor should it create dissension among the SSG members. (4) The process should be conceptually simple, and any documentation presented to the SSG should be essentially self-explanatory. These guidelines are typical of the constraints placed upon most applications for real clients. In discovering how the various science teams valued trajectories, the authors found that it was absolutely necessary to evaluate the trajectories as a pair. For some scientific investigations the critical information was of such value that the two trajectories provided redundancy in the collection process that was vital. However, for other investigations it was not possible for one trajectory to obtain all of the information; thus, each
220
Practice Table 8.1 The MJSyy science investigations'1
Science team
Abbreviation
Radio science
RSS
Infrared radiation
IRIS
Imaging science
ISS
Photopolarimetry
PPS
Ultraviolet spectroscopy
UVS
Cosmic ray particles
CRS
Low energy charged particles
LECP
Magnetic fields Plasma particles
MAG PLS
Planetary radio astronomy Plasma waves b
PRA PWS
Primary measurements Physical properties of atmospheres and ionospheres. Planet and satellite masses, densities, and gravity fields. Structure of Saturn rings. Energy balance of planets. Atmospheric composition and temperature fields. Composition and physical characteristics of satellite surfaces and Saturn rings. Imaging of planets and satellites at resolutions and phase angles not possible from Earth. Atmospheric dynamics and surface structure. Methane, ammonia, molecular hydrogen, and aerosols in atmospheres. Composition and physical characteristics of satellite surfaces and Saturn rings. Atmospheric composition including the hydrogen-to-helium ratio. Thermal structure of upper atmospheres. Hydrogen and helium in interplanetary and interstellar space. Energy spectra and isotopic composition of cosmic ray particles and trapped planetary energetic particles. Energy spectra and isotopic composition of low energy charged particles in planetary magnetospheres and interplanatary space. Planetary and interplanetary magnetic fields. Energy spectra of solar-wind electrons and ions, low energy charged particles in planetary environments, and ionized interstellar hydrogen. Planetary radio emissions and plasma resonances in planetary magnetospheres. Electron densities and local plasma wave-charged particle interactions in planetary magnetospheres.
a The broad spectrum of complex scientific phenomena requires an evaluation methodology that systematically captures inputs from each science team. b Not selected by NASA at the time of this study. Source: Table 1 of Dyer and Miles 1976.
trajectory provided unique results. Naturally, detailed modeling could capture all of these valuation considerations, but not within the above guidelines. An additive multi-attribute utility model was considered for the selection methodology and rejected for the following reasons: In the first place, it is not an easy task to identify the appropriate set of utility-independent attributes of a trajectory pair. Even if such sets of attributes exist, they may be different for the different science teams. Further, the level of effort required to identify these attributes for each science team and to construct the utility functions seemed inconsistent with the guideline to develop a concep-
Applications
221
tually simple process that could be presented to the SSG in a self-explanatory document. Also, at this early stage in the Project the scientists might be reluctant or unable to provide such detail concerning their preferences. They might feel that the disclosure of this information, even if correctly stated, could prove to be disadvantageous in future negotiations with the Project for resources and in decisions affecting the science investigations These reasons for rejecting a multi-attribute analysis apply more to the INTROSPECTING strategy than to the RATING strategy. However, the final reasons concerning the disclosure of information, if truly held by the SSG and science teams, would be adequate to rule out any multi-attribute formulation for the selection of trajectory pairs. Thirty-two trajectory pairs were identified for consideration through an iterative process with the individual science teams. These trajectory pairs spanned the range of feasible alternatives, met at least the minimum requirements of each science team, and were of interest to at least one science team. The evaluation scheme that the authors devised for this selection process consisted of a comparison of the trajectory pairs by the science teams on an individual basis, followed by a collective choice analysis. As part of the science team comparison, each team rank-ordered and then scaled the thirty-two trajectory pairs. The scaling process used lotteries consistent with von Neumann—Morgenstern utility theory to achieve a cardinal (interval) scale. Each trajectory pair was compared by each science team to a lottery between the most and least favored pairs. Then, in order to measure the value difference between the least and most favored trajectory pairs across science teams, each team was asked to assign a probability that made the team indifferent between its least favored pair and a lottery between its most favored pair and a "no-data" trajectory pair (the Atlantic Ocean Special). The larger this final probability is, the smaller the value difference is between the least and most favored trajectory pair. The authors openly stated that this last judgment did not achieve the desired information. The analysts attempted to get the science teams to revise these judgments, but were "only partially successful." It is recognized that the science teams can game the process by assigning a probability that is lower than they truly believe, thus overstating the value of the difference between the least and most favored pairs. The authors state that it was not appropriate to elicit this information from the science teams, and ultimately relied upon the project scientist for the information. Table 8.2 provides the rank and utility data for the ten science teams for the thirty-two trajectory pairs. Numerous collective choice rules were examined with the information collected from the science teams, ranging from rank sum, addition of
Table 8.2 Ordinal rankings and cardinal utility function values
load* I 2
3 4 5 7 8 9 10 11
13 15 17 18
19 20 21 22
23 24 25 26 27 28
29 30 31 32 33 34 35 36
IRIS
CSS
Trajectory Rank
Utility
Rank
Utility
O.87I O.875 O.925 6.O O.948 O.6OO 9.O O.943 O.6OO O.6OO 3 2 . O O.75O O.6OO 2O.O O.886 O.6OO I 2 . O O.93O O.887 I I.O O.935 O.6OO 13.O O.928 O.652 18.O O.915 O.887 29-5 O.869 O.6OO 4.0 O.966 8.0 O.944 O.6OO O.6OO 26.5 O.873 O.772 10.0 O.937 O.6OO 14.5 O.925 O.6OO 29.5 O.869
Rank
!3-5
6.O O.772 28.O 6.O O.772 24.5 2O.5 0.6OO 14-5
22.O
2O.5
26.O
20.5 20.5 2O.5 20.5 3-o 20.5 8.O 3.0 2O.5 20.5 2O.5 6.O 20.5 20.5 20.5 O.6OO 24-5 3 . 0 O.887 17.0 2O.5 O.6OO 7.0 20.5 O.6OO 2.0 1.0 20.5 O.66O I.O I.OOO 23.0 2O.5 O.6OO 5.0 2O.5 O . 6 0 0 22.0 2O.5 O . 6 0 0 3-o 2O.5 O.6OO 19.0 2O.5 O.6OO 26.5 2O.5 O.6OO 16.0 2O.5 O.6OO 21.0 2O.5 O.6OO 31.0
4.0 28.5 32.O 10.0 16.0 24.0 7.0
22.0 16.0 J 3-5 10.0 10.0 31.0 3-o 10.0 O.875 16.0 O.919 26.0 O.947 28.5 O.996 10.0 I.OOO 30.0 0.876 26.0 0.957 19.0 I.O 0.878 5.0 0.987 0.890 19.0 6.0 0.873 0.921 19.0 2.0 0.883 0.813 22.0
Utility
uvs
PPS
ISS Rank
O.85O 2 2 . O O.75O 25.O O.96O 28.5 O.7OO 2O.O O.67O 17.O O.6OO 32.O 5.O O.9OO O.8OO I3.O O.72O 8.5 O.92O 3-5 O.75O 28.5 O.8OO 18.5 O.85O 6-5 O.9OO 15.0 O.9OO 28.5 O.62O 22.0 O.97O 31.0 O.9OO 18.5 O.8OO 25.0 O.7OO 28.5 O.67O 25.0 O.9OO 11.0 O.65O 16.0 O.7OO 14.0 O.77O 6.5 I.OOO 12.0 O.94O 8.5 O.770 22.0 O.93O 10.0 O.77O 3-5 2.0 O.98O 1.0 O.75O
Utility
O.78O O.65O O.72O O.8lO O.52O O.58O O.75O O.63O O.52O
O.55O O.5IO
O.58O O.53O
Rank
Utility
Rank
Utility
O.82O O.79O O.9IO O.970 4.O O.92O I8.O O.8lO I3.5 O.830 26.0 0.760 19-5 O.8OO 28.5 O.72O J 3-5 0.830 9 . 0 O.84O 13.5 O.83O 22.0 O.79O 26.0 O.76O 32.0 O.63O 22.0 O.79O 31.0 O.640 30.0 O.67O
24.5
O.85O
14.O
O.7OO
15.O
O.5OO
5-5 28.5 28.5 I3.O 32.O 2I.O
O.9IO
7.0
O.83O I9.O O.45O *7-5 O.48O 26.0 O.9OO 6.0 O.43O 25.0 O.74O 7.o O.6IO 16.0 O.6OO 24.0 O.49O 31.0 O.32O 30.0 O.65O 20.0 O.77O 14.0 O.560 17-5 O.5OO 29.0 O.58O 27.5 O.44O II.O O.63O 23.0 O.87O 22.0 O.3OO 32.0 O.96O 5.0 O.98O 1.0 I.OOO 4.0
O.48O O.35O O.35I
1.0
I.OOO
O.84O O.85O O.93O O.83O O.84O O.76O O.86O
O.84O 2 8 . O O.84O 2 7 . O O.88O 6.O O.8OO
3O.O
O.86O 9-5 O.89O 2 . 0 O.96O 5-5 O.9IO 31.0 O.8lO 17.5 O.87O 24.5 O.85O 13.0 O.88O 13.0 O.88O 21.0 O.860 28.5 O.84O 5-5 O . 9 I O 5-5 O . 9 I O 28.5 O.84O 1 . 0 O.97O 17-5 O.87O I 7-5 O.87O 24.5 O.85O 3.0 O.93O 17-5 O.87O
II.O I9.O 2O.O 26.O 3I.O l6.O
8.0
O.9OO
13.0 O.88O 9-5 O.89O O.83O 21.0 O.86O O.83O 13.0 O.88O O.77O 14.5 O.85O O.72O O.8OO
9.O
24.O 25.O 22.0 29.O I7.O 8.O 32.O 5.O 3.O I.O
13.O O.72O
PIS
MAG
Utility
9.0 O.53O O.7OO 7.0 3.0 O.6lO O.64O 13-5 9.0 O.75O O.69O 26.0 6.0 O.72O O.55O 28.5 O.7IO 19-5 O.8lO 13-5 O.82O 13.5 I.OOO 24.0
O.52O
LECP
Rank
O.55O I7.O O.53O 22.0 5.O O.52O 2.0 O.57O O.6OO O.5OO
CRS
Utility
Rank
17.5
Rank
Utility
PRA Rank
26.5 O.6OO 19.O II.O O.75O I8.O 7.0 O.9OO 24.O 7.O O.9OO 25.O 6.0 O.8O0 i-5 I.OOO O.36O 26.5 O.6OO 26.0
O.546 0-547
9.0
O.556 0.552 O.558
O.7OO
17.5 17.5 O.4OO 17-5 O.26O 32.0 O.28O 26.5 O.43O II.O O.49O
O.5IO
O.485 O.3OO
O.35O O-575 0.405 0.408 O.25O
O.95O I.OOO O.97O
O.485
17-5 26.5 26.5 26.5 7.0 26.5 26.5 26.5 II.O i-5 4.0 II.O
O.7OO O.7OO O.7OO O.5OO O.6OO O.75O O.7OO O.6OO O.6OO O.6OO O.9OO O.6OO O.6OO O.6OO
O.75O I.OOO
O.95O O.75O
13.0 7.0 27.0 28.0 22.0 8.0 29.0 30.0 16.0 31.0 23.0 20.0 32.0 1.0 2.0
3.0 II.O
2.0
O.99O
4.0
O.95O
4.0
10.0 O.75O 21.0 4.0 0-975 3.0 18.0 O.62O 13.0 21.0 O.59O 12.0 15.0 O.67O 8.0 12.0 O.78O 9.0 23.0 O.57O 10.0
O.42O
17-5 4.0
O.7OO
12.0 5.0 14.0 15.0 17.0 10.0 21.0
2.O
O.99O
Utility
O.98O O.57O O.573
17-5 17.5
O.95O O.7OO O.7OO
II.O
O.75O
O.595 17-5 O.594 26.5
O.7OO O.6OO
O.6OO
O.IO9 O.IO8 O.6OO O.IO7
O.IO6 O.IO5
O.543 O.557 O.IO4 O.IO3
O.549 O.IO2
O.542 O-545 0.101 1.000 0.990 0.980 0.554 0-970 0.553 0.960 0.551 0.550 0.548 0.555 0.544
Note: "In order to reach a recommendation the results from each science team must be aggregated across the 12 trajectory pairs. Note that 6 , 1 2 , 1 4 , and 16 are missing from this list of 36. Source: Table II of Dyer and Miles 1976.
Table 8.3 Collective choice rankings and valuesa Form
Rank sum
Additive
Nash 0.6 or 0.8
P'
X' weighting
1.0
Rank
Trajectory pairs
Value
Rank
I
O.822
2
2
O.797 O.795
3 4 6
35
4 O.719 5 O.683 6 O.678 7 O.655
17 8
8
O.622
10
9
O.6ll
8
10
10
0.605
12
31 29
26 27
5 2-5
3
1
5 7
Value
O.887 O.875 O.889 O.856 O.791 O.822
O.757 O.738 O.755 O.728
1.0
Rank I
3 2
Rank
O.724 O.692 O.7IO
I
O.9OI
I
3
3
2 .
O.884 O.896
4 8
O.871 O.84I
O.64I O.555 5 O.597
8
O.511
11
O.475 O.488 O.514
Value
1.0
Value
4 6
9 7
1.0
5 O.851 6.5 O.846 10.5 O.836 10.5 O.836 6.5 O.846
Rank
2
or 2.0
Value O.871 O.852 O.87O
O.833 4 6.5 O.765 O.8OO 5 6-5 O.765
10
8 11
O.746 O.756 O.744
1.0
Rank
or 2.0
Value
I
O.691
3
O.638 O.676 O.596
2
4
8 5
7 10 11
6
O.5O2
O.535 O.517 O.487 O.486 O.519
Note: "Several aggregation schemes were examined. The results for the top ten trajectory pairs are shown here. Source: Table III of Dyer and Miles 1976.
0.6 or 0.8
pit,
1.0 or 2.0 Rank
Value
Value
O.887 O.865 2 O.877 O.839 4 12 O.813 O.776 O.822 9 5 0.804 O.745 5-5 O.833 8 8 O.823 £0.5 O.725 10 O.82O 7 O.746 O.7O6 5-5 O.833 1 4 I
3
O.884 O.86O O.878 O.848
Rank
Rank
Value
1-5
I
O.892
3 i-5 4 6
3 4
O.874 O.886 O.856
11
O.829
2
O.836 O.839 9-5 O.83O 9.5 O.83O 5 O.843 7 6
Applications
225 a
Table 8.4 Ordinal rankings for preferred trajectory pairs Collective choice ranking
Science team ordinal rankings
Additive Rank sum 31 29 26 2-7
5
2-5
35 17
8 10
X,= I.O
1
2
2
3
3 4 5 6 7 8 9 10
1
4 6 5 7 10
8 12
RSS IRIS ISS PPS UVS CRS LECPMAG PLS PRA 20.5 20.5 20.5 20.5 20.5 20.5 20.5 20.5 20.5 3.0
3.0 5.0 2.0 1.0
9.0 7.0 21.0 4.0 20.0 11.0
5.0 19.0 10.0 30.0
28.5 28.5 2.0
13-5 10.0 24.0
8.5 6.5 11.0
16.0 17.0 25.0
6.0 9.0 7.0 3.0 4.0
8.0 3.0
4.0
3.0
4.0
5.0
2.0
2.0
4.0
4.0
17-5 17-5
3.0
1.0
1.0
4.0
i-5 4.0
3.0
13.0
6.0
6.0 5.0
i-5 11.0
6.0
9.0 14.0
17-5 17-5
7.0 24.0
17-5
10.0 8.0 9.0 7.0
1.0
5.0
13-5 13.0 6.5 13-5 2-4-5 5.0 13-5 21.0 8.5 19.5 2 . 0
12.0 9.0 11.0 20.0
2.0
9.0
!7-5
1.0
1.0
Note: a Like any good iterative analysis, this methodology winnowed a large list of alternatives down to the top three. By returning to the problem-structuring phase an improvement to trajectory pair 26 was synthesized. Source: Table IV of Dyer and Miles 1976.
cardinal values with various weighting schemes, and a multiplicative rule based upon a Nash bargaining model. Table 8.3 summarizes these results (the interested reader should consult the original article for a detailed discussion of these collective choice rules). The analysis results were presented to the SSG during a two-day meeting in October 1973. Table 8.4 displays the top ten pairs in terms of the ranks assigned by each science team. The Radio Science Team (RSS) wanted at least one trajectory to be occulted at Saturn along the major axis of Saturn's rings, which would provide a complete radio attenuation profile of the ring structure. An optimum occultation and a good encounter of Saturn's major satellite (Titan) were impossible; the satellite encounter was determined to be the more important. The discussion was finally directed to trajectory pairs 31, 29, and 26, which were the top three of all collective choice rules. The Imaging Science Investigation (ISS) team leader expressed an aversion for pair 29. Ultimately pair 26 was chosen and minor improvements were made to it for the RSS team. So the collective choice methodology devised by the analysts during problem definition focused the attention of the decision-makers to three trajectory pairs out of thousands possible and thirty-two which were explicitly analyzed.
Practice
226
25
1 =
23
21
I
75
19
17
0
0.10
I
0.20
0.30
0.40
$/lb Sulfur emitted
Sulfur oxide pollution cost, per unit of emission
Figure 8.4. Illustration of total cost comparison. Since the analysis was sensitive to the pollution cost that society attached to sulfur oxide emissions, this cost was made a variable and each alternative was evaluated in terms of its total electricity cost. Source: North and Merkhofer 1976, Figure 3.
8.3
MODEL-STRUCTURING
8.3.1 Introduction The first article in this section highlights the deterministic phase (the model-building phase) of the MODELING strategy. North and Merkhofer (1976) present a flow chart of submodels to examine emission control strategies for power-plants. The details of some of these submodels are also explored to demonstrate various modeling approaches. Also discussed here are the sensitivity analysis results that permit the authors to determine which parameters require probability assessments in the next phase. Fry back and Keeney (1983) describe the construction of an index for trauma severity in the second example, and thus illustrate many modeling decisions faced by the analyst as he constructs a measurement tool for the decision alternatives. 5.3.2 North and Merkhofer (1976) - analyzing emission control strategies In support of a study by a committee of the National Academy of Sciences North and Merkhofer modeled sulfur oxide emissions and their effects as
Process technology
Capital cost, Costs and benefits
efficiency (sulfur removal during combustion) Health effects model
Demand for electricity
Plant loading
Electric power plant
Emissions from combus tion process
Contain ment of emissions: Removal of partic ulates and stack gas scrubbing
EmisPollusions Exposure Dispersion Dosage tant released model model concenfrom to biota tration site: Population (Including chemical at risk) SO2 SO2 Dosage to changes S0 4 so 4 in atmosPartic material Acid rain phere) ulates washout
Ecological damage model
Damage to
Material damage model
Damage to
Aesthetic effects model Fuels availability
Health cost model
Mortality morbidity
Ecological costs model
biota
Material costs model
materials
Perceived degradation
Aesthetic costs model
Health costs
Biota costs
Material costs
Aesthetic costs
Byproduct production, cost elements for containment
Price per
unit given energy content, sulfur content
Total pollution related costs
Electric produc tion cost elements
Model for computing costs of electricity production and emission reduction
Costs of electricity, including emission control
Net overall costs and benefits V
Figure 8.5. Overview of pollution model. The model that was developed to calculate total electricity cost included health, ecological, material, and esthetic costs. Source: North and Merkhofer 1976, Figure 2.
Practice
228
0 0
0.05 0.10 0.15 0.20 0 25 0.30 0.35 0.40 $/lb SOX 0.10 0.20 0.30 0.40 0.50 0.60 0.70 0.80 $/lb S Sulfur oxide pollution cost, per unit of emission
Figure 8.6. Total social cost versus pollution cost, existing coal-fired plant (non-urban). The preferred alternatives are those with the lowest total cost for a given value of the sulfur oxide pollution cost. Note that the preferred alternatives change at 19 cents and 53 cents, the crossover points. Source: North and Merkhofer 1976, Figure 4.
a way of analyzing emission control strategies. This analysis, begun in the fall of 1974, was to summarize the information available at the time and to determine the value of reducing the uncertainty in the context of regulatory decisions by the US government on sulfur oxides, a major category of air pollution. The analysis examined representative coal-fired power-plants, which are the most important sources of sulfur oxide emissions in the eastern United States. Examples of the alternatives available to the power-plants were: a tall stack and intermittent control systems, coal preparation to reduce its sulfur content, premium-priced low-sulfur coal, and a flue-gas desulfurization (FGD) process. The result of this analysis was to be a comparison of control alternatives in terms of the total cost of electricity, as a function of sulfur oxide pollution cost (per unit of emission). Figure 8.4 illustrates this concept for two types of coal. Using the MODELING strategy, a model composed of numerous submodels was developed to compute both of these costs (total electricity and pollution). Figure 8.5 depicts this model, where the left-hand side is oriented to the cost of electricity and the right-hand side to the social costs of emissions. The authors reported their results for three representative power-plants:
Applications
229
(1) an existing coal-fired plant in a remote non-urban location, (2) a coal-fired plant planned for construction in the near future in a remote non-urban location, and (3) an oil-burning plant, originally designed to burn coal, which may be reconverted to coal, located in an urban area of the east coast. When necessary these submodels were dependent upon the three powerplants, and in some cases unique models were developed for each plant. The left-hand side of the model in Figure 8.5 was used to compute the total cost of electricity, using a parametric value of pollution cost, for the three representative power-plants. Total electricity cost was calculated for each of alternative pollution control strategies. Figure 8.6 illustrates the results for the existing, rural coal-fired plant. The control alternatives were evaluated in a similar manner for the other two representative plants. The detailed submodels on the right side of Figure 8.5 were next used to develop probability distributions on the per unit pollution cost so that a recommendation could be drawn from an analysis as presented in Figure 8.6. The details of the submodels cannot be repeated in full here. The following extract from the description of the dispersion model illustrates the types of modeling decisions faced by the analysts, although all of the submodels are quite different. The [dispersion] model was based on the following assumptions: (1) In a period shortly following the emission, the sulfur oxides become uniformly distributed from the ground to a mixing layer height. The height of plume then remains constant. (2) The emissions are uniformly distributed over an arc of constant size, so that the width of the plume expands in direct proportion to the time since emission, or (with constant wind velocity) in direct proportion to the distance downwind traveled by the plume. (3) The emissions travel down the plume uniformly distributed in a "box" whose length is the distance traveled by the wind per unit time, whose height is the height of the mixing layer, and whose width is the distance perpendicular to the wind direction subtended by an angle of constant size. Thus, the width of the box grows in direct proportion to time, and the concentration of pollutants decreases inversely with time. The following assumptions were made about the chemical reactions of the sulfur oxides. (1) Sulfur dioxide to sulfate oxidation takes place according to a first order rate reaction. The rate may change considerably between comparatively clean rural air and pollutant-laden urban air. (2) Sulfur dioxide is removed to the ground at a constant deposition velocity, beginning at the time of emission. (This assumption will overestimate the amount of sulfur dioxide removal to ground for a plume from a tall stack that travels many miles before touching ground, and it will underestimate the
Practice
230
Probability density function for pollution cost (Rural case)
0.05
0.10
0.15
0.20
0 25
0.30
0.35
0.40 $/lb SOX
0.10
0.20
0.30
0.40
0.50
0.60
0.70
0.80 $/lb S
Sulfur oxide pollution cost, per unit of emission
Figure 8.7. Results for existing coal-fired, non-urban plant. Source: North and Merkhofer 1976, Figure 1.
(3)
sulfur dioxide removal from a shorter stack where the plume is in contact with the ground for substantial time before the plume is dispersed uniformly up to the inversion or mixing layer height.) Suspended sulfate is removed to the ground at a constant deposition velocity, beginning with the time of emission.
The resulting model was used to conduct sensitivity analyses on such parameters as oxidation rate, sulfur dioxide deposition rate and mixing layer height. Using the results of the sensitivity analyses, a probability distribution was developed. This distribution became an input to the exposure model. The result of this modeling process yielded probability distributions of the per unit pollution costs of the three representative power-plants. Figures 8.7 through 8.9 plot these distributions on the initial analyses of control strategies. The probabilitiy distribution for the old reconverted urban plant is quite different from the other two. Note that in these figures the same control strategies (high-sulfur coal, low-sulfur eastern coal, and the flue-gas desulfurization process) are preferred for varying values of the per unit emission cost. The crossover per unit emission cost is always 19 cents and the highest crossover point varies from 37 cents to 59 cents. The
Applications
2-31
Probability density function for pollution cost (Rural case)
21
J_ 0
0.05
0
0.10
I
0.10
0.15
0.20
0 25
0.30
0.35
0.40 $/lb SOX
0.20
0.30
0.40
0.50
0.60
0.70
0.80 $/lb S
Sulfur oxide pollution cost, per unit of emission
Figure 8.8. Results for new coal-fired, non-urban plant. Source: North and Merkhofer 1976, Figure 12.
o
6.5
«
6.0
Probability funct for pollution cost (Urban case)
5.5 0.05 0.10
0.15
0.20
0 25
0.30
0.35
0.40
0.45 0.50
$/lbSO x
0.10 0.20
0.30
0.40
0.50
0.60
0.70
0.80
0.90
$/lbS
1.00
Sulfur oxide pollution cost, per unit of emission
Figure 8.9. Results for old reconverted urban plant. Source: North and Merkhofer 1976, Figure 13.
232
Practice Table 8.5 Expected value of resolving uncertainty, 620 MW representative plant, 80% load factor Low-sulfur coal available
Remote plant (New plant) (Retrofit) Urban plant (Retrofit)
Low-sulfur coal not available
Million kW h
Million $/yr
million kW h
million $/yr
0.48 0.53
3-4 3-7
0.38 0.34
2.7 2.4
0.40
2.8
0.19
Estimated value of resolving uncertainty on sulfur oxide emissions to ambient sulfate relationship and dose response relationship for health effects: Nationwide: ~- $250 million/year Source: Table II of North and Merkhofer 1976.
coal preparation alternative is always close to the lower crossover point, but never is preferred. These generalizations are valid only for the figures presented here. Table 8.5 summarizes the expected value of resolving the uncertainty represented in these probability distributions. The modeling approach adopted for this analysis has two key elements worth noting. First, a nationwide policy and an estimate for eliminating uncertainty concerning pollution costs were generated by examining three representative types of coal-fired power-plants. Second, a detailed mathematical model based upon the best scientific assumptions available was used to generate baseline results, as well as probability distributions for the per unit cost of pollution. 8.3.3 Fryback and Keeney (1983) — constructing an index of trauma severity The modeling done by Fryback and Keeney (1983) in constructing an index of trauma severity is totally different from that done by North and Merkhofer (1976), yet it is an appropriate and demonstrably useful approach for many decision problems. The authors defined a trauma severity index (TSI) to be a measure for use in "evaluating medical facilities or medical procedures, allocating funds, creating or appraising operating policies of emergency medical facilities, and even triage." The TSI should represent both mortality (death) and morbidity (disability and discomfort) threats to patients. One physician served as the trauma expert. The TSI was scaled from zero to one so that "a o level corresponds to a completely healthy individual and the 1 to an individual who will surely
Applications
2-33
Respiration rate Ventilation Collapsed lung
Pulse rate — Circulation
Systolic Blood pressure '—
Diastolic
Central nervous system Trauma severity
Internal organ injury
—
Renal functioning
—
Muscular/skeletal
1
—
Inhalation injury
1 st degree
Skin
2nd degree
Burns
3rd degree
Figure 8.io. Hierarchy of trauma severity concerns. The key in model-structuring for trauma severity was to distinguish between trauma causes and concerns in treatment, where they were different, and to organize the treatment concerns into a logical hierarchy. Source: Fryback and Keeney 1983, Figure 3. die very soon." The numbers in between represent mortality and morbidity in the following way: 0.5 corresponds to (1) an individual who has a fifty-fifty chance of dying from the injury or living with no resulting discomfort or disability or (2) an individual with a 40% chance of dying and a 60% chance of living with some degree of discomfort and/or disability. So the TSI will always be greater than or equal to the probability of dying from the injury associated with the trauma, and the difference between this probability and the TSI is a measure of the morbidity. In developing the TSI it became important to distinguish between the
Practice
2-34
Table 8.6 Trauma concerns and measures" Concern
Measures
Range
Xi = Ventilation
Respiration rate Collapsed lung
0-50/min 0-100%
Xz = Circulation
Pulse rate Systolic pressure Diastolic pressure
0—180/min 0-250 mm Hg 0-140 mm Hg
X 3 = Central nervous system
Glasgow coma score
X 4 = Internal organ
X 5 = Renal function
Worst
i8±6 0
0 100
60-100 90—150 60-100
0 0 0
3-15
15
3
Number needing surgery (liver, spleen, pancreas, kidneys, bowel, major vessel)
0-7
0
7
Urine output
0-250 ml/hr
30-80
0
X 6 = Muscular/skeletalI Number of major fractures 0-8 X 7 = Burns
Percent 1st degree Percent 2nd degree Percent 3rd degree Inhalation injury
0-100% of body 0-100% of body 0-100% of body 0 = no, 1 = yes
0
8
0 0 0 0
Best
100 100 100 1
Note: 'The measures and best-to-worst ranges for each concern were synthesized as a prelude to the quantification of the trauma index. Source: Table 1 of Fryback and Keeney 1983.
cause of the trauma (injury) and the concern in treating the trauma. In many cases these may be the same, but in very extensive injuries there may be more than one concern. The concerns were developed along with measures (both quantitative and qualitative) that would be used to construct the index. Figure 8.10 shows the hierarchy of concerns, and Table 8.6 displays the concerns, measures, and ranges. The development of this construct for a TSI required several hours of interacting with the expert. During the process of developing this construct of trauma severity it was also learned that there are two other factors, not related to the injury, that impact the severity of the trauma and which would have to be included in the TSI for it to be at all accurate. These factors are the victim's age and his or her health status, e.g. diabetes or chronic respiratory problems. These two factors do notfitinto the hierarchy shown in Figure 8.10 in any obvious way. Factors such as this are often called "conditioning variables" and discrete variations of them are placed above the hierarchy in Figure 8.10 so that it is repeated several times for each variation of the conditioning variables. However, this approach was not appropriate here since two to ten discrete representations of age and health status would not provide enough accuracy for the possible ranges
Applications
2-35
Figure 8.11. Severity adjustment factor due to age. This adjustment factor was applied to the entire multi-attributed trauma index. Source: Fryback and Keeney 1983, Figure 3.
Table 8.7 Severity adjustment factors due to other diseased Age (A) Disease combinations (D) None HT* COPDC ASHD^ DM' HT + COPD HT + ASHD HT + DM COPD + ASHD COPD + DM ASHD HT + COPD + ASHD HT + COPD + DM HT + ASHD + DM COPD + ASHD + DM HT + COPD + ASHD + DM
4 mo
zyr
30 yr
55 yr
0 0.2
0 0.2
0.9
0.4 0.6 0.8
0.4 0.85 0.8
0.4 0.6 0.9 0.7
1.0
0.55
0.55
1.0
0
0
0.9 0.9 0.9 0.9
0.8 0.6
1.0 .
0.2
.0.75
1.0
0.98 0.98
.
Notes: "Note that the adjustment factor for other diseases is also age-dependent. h HT — hypertension. C COPD = chronic obstructive pulmonary disease. ''ASHD = arteriosderotic heart disease. T)M = diabetes mellitus. Source: Table 3 of Fryback and Keeney 1983.
75 yr 0
236
Practice
of these variables. The approach taken by the authors was to develop an adjustment factor for the elicited TSI that incorporated the effects of age and other health conditions. The following equation adjusts the elicited TSI (Sr): S = S'[l-s(A,D)]+s(A,D) where: D) = 0.2 s(A) + 0.32 s(D \ A) - 0.064 s(A) s{D | A). The function s(A) is the age adjustment, shown in Figure 8.11. The age of twenty-five was considered optimal for surviving or recovering from a trauma. s(D | A) is the age-dependent adjustment for health status. Table 8.7 presents the adjustments for diseases such as hypertension and diabetes. Note that this adjustment was so complicated that it was discretized, as conditioning variables often are. Also note that these adjustments are made to the TSI once the trauma concerns have been aggregated, thus assuming that age and health status have no differential effect as a function of concern. An example of differential effect would be that age significantly affects burn-related traumas but not circulation traumas. This index as well as other trauma indexes have been compared with holistic ratings given by doctors to trauma patients and all correlate in the 0.6 to 0.7 range. This TSI is the most complete and complex of the indexes. Further evaluation of the indexes is ongoing. The key elements of this modeling effort were (1) the linkage of a numerical index to an intuitive interpretation of both the probability of dying from the injury and the degree of discomfort and/or disability that might result, (2) the integration of the effects that many different types of injury or illness might have on trauma into one common measure, and (3) the modification of the result to account for two non-injury-related variables, age and health status. 8.4 ITERATIVE ANALYSIS
8.4.1 Introduction This section illustrates two possible types of iterative analysis. First Spetzler and Zamora (1984) use the MODELING strategy to examine a facilities investment and expansion decision. The second example by Buede and Choisser (1984) demonstrates how the CONFERENCING strategy can be used to winnow the set of alternatives via a series of sequential analyses.
Applications
237
State variables
System model
Preferences
Raw material costs Plant efficiency Market prices Impurities Cost growths
Plant operations and financial model
Time preference model
Profitability measure
1
Present value meter Initial plant investment decision Plant expansion decision Timing decisions Plant capacity Decision variables
I
J
J_
I
I
Figure 8.12. Facilities investment deterministic model. This model mimics the schematic for the deterministic model of the MODELING strategy (see Figure 6.2.). Source: Spetzler and Zamora 1984, Figure 2.
8.4.2 Spetzler and Zamora (1984) - facilities investment and expansion The authors present a somewhat disguised analysis of an investment in a $20 million facility with a $5 million potential expansion for a major US corporation. This analysis, conducted in the early 1970s, had to address several major areas of uncertainty: The process involved a chemical reaction that was difficult to control. The major product from the facility would be a brightener, but by utilizing new processing techniques a valuable by-product could be produced. The exact yields of both products were uncertain. Minute quantitites of impurities in the raw materials used in the process could greatly affect the amounts of brightener and by-product produced. Market factors, including costs of materials and sales prices of products, were not known exactly. The parameters involved in taking the process from the pilot model up to a full-scale production facility were uncertain. Specific parameters involved included the efficiency of the new plant and the additional design costs. Some government regulatory effects could be expected in the areas of pollution control of the new plant and pricing controls on the by-product, but neither the extent of the regulations nor the cost imposed could be determined beforehand.
This analysis followed the iterative cycle described as part of the MODELING strategy: deterministic phase, probabilistic phase, and informational phase.
238
Practice Table 8.8 Deterministic results for facilities investment1 PV range (millions of dollars)
Base case
From
To
From
To
Change inPV (millions of dollars)
36.00 (lb/ton) $0.27
0.00 (lb/ton) $0.15
80.00 (lb/ton) $0.35
-30
35
65
—
Tested range
i. By-product production i. Market price of brightener in 1980 3. Raw material cost growth 4. Raw material costs 5. Impurities in raw material 6. Cost multiplier on investment 7. Brightener price growth after 1980 8. 9. 10. 11.
Cost multiplier on operations expenses Cost multiplier on maintenance expenses By-product price growth Water reclamation costs
12. By-product price, 1970 13. Plant efficiency 14. Brightener production 15. Market price of brightener in 1974
5.00% (%/yr) $7.00 ($/ton) 4.00 (lb/ton) 90.00% 4.00% (%/yr)
0.00% 8.00% (%/yr) (%/yr) $2.00 $18.00 ($/ton) ($/ton) 2.00 6.00 (lb/ton) (lb/ton) 70.00% 125.00% 6.00% 2.00% (%/yr) (%/yr) IIO.O% 80.00% 150.00% IOO.OO% 70.00% 120.00% $0.06 $0.03 $0.01 $0.03 $0.04 $0.02 ($/gal.) ($/gal.) ($/gal.) $0.60 $0.50 $0.40 75.00% 50.00% 110.00% 50.00 45.00 40.00 (lb/ton) (lb/ton) (lb/ton) $0.25 $O.I 5 0.30
45
57
-7
40
47
~9
35
44
—
10
30
40
—
12
25 28
37 33
2
32 30
29 28
3 4
31
17 17
3
29
26
0
26
4
30
26 26
4
28
24
12
-5 3
16. Government regulation costs
$0.02
$0.01
$0.03
3
2-5
22
17. Market price of other by-products
$0.10
$0.07
$0.12
4
2-4
20
96.00 (lb/ton)
4
*4
20
18. Other by-products produced
84.00 (lb/ton)
\VflD) 7O.OO
(lb/ton)
Note: "The sensitivity analysis on each state variable in the deterministic model produced these results. Source: Table 2 of Spetzler and Zamora 1984.
The deterministic phase saw the development of a system model, with the variables categorized as state and decision variables, and a time preference model (see Figure 8.12). The time preference model calculated discounted cash flows for each alternative: (1) build initial production plant, (2) consider an expansion if the production plant is successful, and (3) build production plant and expansion simultaneously. Over half of the decision analytic effort was consumed in this deterministic phase and produced the following results: the initial production plant was worth $15 million in present value, the plant followed by the expansion dropped to $5 million, and the simultaneous plant/expansion was valued at a negative $30 million. A sensitivity analysis of state variables examined eighteen variables as shown in Table 8.8. The first
Applications
10
239
12
14
16
18
Figure 8.13 (a). Distributions of individual judgments. Source: Spetzler and Zamora 1984, Figure 5.
8
10
12
X-dollars per ton
Figure 8.i3(b). Probability distribution for raw material costs. This distribution is a consensus of the experts whose individual distributions are shown in Figure 8.13(a). Source: Spetzler and Zamora 1943, Figure 4. Table 8.9 Combined effect of discount rate and risk aversion on the certain equivalent1 Certain equivalent risk aversion level
Risk neutral Discount rate
(millions of dollars)
1
2
7%
25.6 11.7 4 .8
16.9 8.S 3-3
17
1
2.0
|
10%
13%
(millions of dollars)
O.I
3 -46.6 - 15-5 -8.3
Note: This two-way sensitivity analysis suggests that the facilities investment should be undertaken. Source: Table 5 of Spetzler and Zamora 1984.
Nodes
begin expansion
build plant
Figure 8.14. Facilities investment decision tree. This decision tree depicts the state variables that were considered critical as a result of the deterministic sensitivity analysis. Source: Spetzler and Zamora 1984, Figure 8.
• Decision node • Chance node
Applications
241
Table 8.10 Value of perfect information" Millions of dollars Total perfect information Single variables only: By-product Impurities Raw material costs Plant efficiency
11.0
6.2
3-9
0.3 0.0
Note: ^There were high values associated with collecting additional information on two of the state variables. Source .-Table 7 of Spetzler and Zamora 1984.
seven of these variables were considered crucial because in each case the decision would be changed at the lower end of the variable's range (to no investment at all). In addition, variable thirteen — plant efficiency — was made a crucial state variable for the probabilistic phase. The major activity of this probabilistic phase was encoding probability distributions on the eight crucial state variables. Risk preference for the decision-maker was also encoded, and further sensitivity analyses were conducted. The probability encoding process for a given state variable first involved eliciting distributions from individuals with the necessary expertise. Then these individuals were gathered together, shown each others' distributions, and asked to discuss the differences in order to develop a consensus distribution. The individual and consensus distributions for raw material costs are shown in Figures 8.13(a) and (b), respectively. Each of these distributions was discretized into three levels (see Section 7.4.3) and a profit lottery was calculated via the decision tree shown in Figure 8.14. A sensitivity analysis on discount rate and risk aversion level (see Table 8.9) was conducted for this client for presentation to his superiors. The three risk aversion levels were based upon the exponential utility curve, and were defined for certainty equivalents of 31.5, 20.4 and o millions of dollars for the lottery in which there was a 0.8 chance of getting $50 million and a 0.2 chance of losing $20 million. The numbers in the box in Table 8.9 were considered to be the reasonable range for a corporation the size of this particular client. Finally the informational phase examined the value of perfect information on the crucial state variables. Table 8.10 presents the results of this analysis. Clearly, the uncertainty on by-product quantities and impurities was the most critical in terms of its effect on the decision. The conclusions of this three-staged analysis were:
Table 8.11 Initial WWDSA alternatives" Family 1 Hierarchical emphasis on backbone Independent voice/data nets Upgrade baseline per commercial trends Common channel signaling (CCS) in backbone Gateways at backbone 1 A: Minimum cost Reliance on commercial resources Eliminate some current facilities iB: Enhanced hardening Facilities hardening Switch relocations Subsidize commercial facilities for hardening/restoral iC: Flexible access nodes Multirate switch capability Reroute and reallocate traffic
Family 2 Extend backbone functions to access nodes Mutually supportive voice/data nets 2A: Fixed facility interfaces CCS to access level Access tandeming Multirite switches Enhanced control for restoration Data/voice backbone interconnects Improved secure voice key distribution New 16 Kb/s voice algorithms 2B: Enhanced tactical interoperability (2A plus) Improved gateways/traffic management 2C: Enhanced secure data (2B plus) End-end encryption for data
Note: These three families of alternatives were the result of several months of engineering design.
Family 3 Fully distributed, integrated voice/data nets Modularized switch family Front-end encryption for all traffic Common signal structure (standards) Distributed uniform connectivity CCS to local travel Enhanced wideband transmission Enhanced system control Fully interoperable 3A: Fixed plant Heavy reliance on commercial facilities Wideband links - mostly terrestial Built-in contingency capability 3B: DoD - furnished satellites Wideband links - mostly satellites GFE earth terminals Highly mobile assets Flexible reconstitution
Applications
243
{
Circu
£
Clear voice Secure voice Data, hard copy
1— Performance -
r-#Userj - 4 - Switch features L Special features
.-Effectiveness
£
lnterceptability Spoofing/TRFC analysis Key protection Physical attack
lility E lectronic attack
•— Responsiveness -
r- Interoperability - -\- Adaptability L- Evolutionary changes 1 - Life cycle
r-Cost
* - 1 rnplementation
1
1- R & D
LProcu, L-O&w
Manning
r-Cost - 4- Schedule L Technical - Status Quo investments - Continuity of operations
Figure 8.15. WWDSA value tree. This value tree had to be compact to be useful in the conference setting. (1) The venture has an expected Present Value of $12 million, based on present information and a discount rate of 10%. (2) Adjustment for risk still indicates a positive Present Value. (3) The value of information, which measures the maximum anyone would be willing to pay to reduce the uncertainty in a variable, indicates large uncertainties as to the amount of by-product produced and the expected level of raw-material impurities. Research should be directed towards reducing these uncertainties. (4) The total value of information is large enough to merit postponing the investment in the production plant. (5) The decision is highly sensitive to both time and risk attitudes. These implications should be made clear to corporate policy makers and their preferences applied to the results of this study. This example demonstrated the iterative approach used within the MODELING strategy to define the key uncertainties faced by the decision-maker. 8.4.3 Buede and Choisser (1984) - system design This analysis, conducted in the early 1980s for the US Defense Communications Agency, focused on the design specification of a Worldwide Digital System Architecture (WWDSA) for 1995. The CONFERENCING strategy was used to gather engineers, system-users, and policy-makers together for a three-staged examination of WWDSA alternatives. Each
Practice
244 100
34®
80 -
2C® 25®
60 -
40 -
20 -
0
1C®
i
20
i 40
i 60
80
100
Implementation
Figure 8.16. Plot of initial WWDSA evaluation results. The family one options did not provide acceptable performance, while the implementation problems of family three eliminated it from consideration. Family two was a very attractive compromise. BL indicates the baseline alternative. stage included the definition and evaluation of increasingly detailed alternatives. The evaluation results of one stage were used to guide the definition of alternatives for the next stage. Table 8 . n provides the initial definition of the WWDSA alternatives. Family one (three alternatives) was an extension of the existing architectures, family two (three alternatives) was the next generation of communication capabilities which would be achievable with low risk by 1995, and family three (two alternatives) pushed the state of the art expected to exist in 1995. The multi-attribute evaluation structure for evaluating these alternatives was developed in three iterations for the first-stage analysis, and then stayed constant for the remainder of the analysis. Figure 8.15 presents this structure. The two highest-level criteria, effectiveness and implementation, were subdivided three times before the evaluators felt confident that the important dimensions were defined well enough for the evaluation of the alternatives. A three-step relative scoring procedure was used in this multi-attribute value analysis: (1) the alternative on a given attribute that provided the greatest effectiveness (or required the least implementation) was scored at 100;
Applications 100
80
#
:tiveness
IC
IS®
C®
LU 40
s® 1®
20 -
2^® n
i 20
i
i 40 60 Implementation
i 80
i 100
Figure 8.iy. Plot of intermediate WWDSA results. This plot demonstrated to the decision-maker that the satellites (S) option was the best single addition to the basic 2.A option, followed by interoperability (IS) and communications security (ISC). (2)
the alternative on the same attribute that provided the least effectiveness (or required the most implementation) was scored at o; and (3) the remaining alternatives were scored on a cardinal (interval) scale between o and 100 for the given attribute. Then the weights assessed to the attributes measured the differences in value from the least to the most attractive alternative. The results of the first stage of analysis are plotted in effectiveness and implementation space in Figure 8.16. This display proved to be most useful to the policy-makers who found it hard to weight effectiveness versus implementation. The status quo (or baseline) alternative which was included in this first analysis was dominated by two of the family one alternatives, indicating that some change was advisable. Family two was chosen as the preferred alternative due to its good balance between effectiveness and implementation. In the second stage of the iterative analysis three incremental improvements were defined for the basic architecture implied by family two: interoperability with the systems of other countries (I), communications security (C), and extensive use of satellites (S). Thus the second stage examined a total of eight alternatives: 2A - the basic architecture, and
Practice
246
100
100
80
1 60 100
i
|
0
40
LU
20
80
20 40 60 80 Implementation Final evaluation results
20 40 60 80 Implementation
g 60
100
Intermediate evaluation results
I 40' LU
20 0
20 40 60 80 Implementation
100
Initial evaluation results
Figure 8.18. WWDSA iterative evaluation of alternatives. This sequence of plots illustrates the winnowing of alternatives. The design designated by the hexagon remained the same through all three analyses.
seven enhancements I, C, S, IC, IS, SC, and ICS. The resulting analysis produced the following additions of capabilities to the basic architecture as implementation was traded off for effectiveness (see Figure 8.17): satellites (S), interoperability (IS), and communications security (ISC). Policy-makers then instructed the design team to use features of these four alternatives to create even more detailed architectures for the third stage of evaluation. As Figure 8.18 shows these alternatives resulted in nearly equal effectiveness/implementation trade-offs. This final figure depicts the sequential processes of design definition followed by evaluation. This entire analysis was completed in only three two-day meetings, one two-day meeting for each stage. The analysts utilized a specially equipped conference room, having a rear-projected, large-screen video monitor for displaying computer output to the group and ample white-board space for outlining the model. This illustration shows how iterative analysis can be used to winnow a large set of alternatives to smaller and smaller subsets.
100
Applications
247
8.5 PROBABILITY ASSESSMENT
8.j.i Introduction In the now classic example of decision analysis by Howard, Matheson, and North (1972) on the hurricane seeding decision we see the adroit use of probabilistic modeling that enables the use of data in defining some probability distributions and definition of the remaining key uncertainties that the experts must address. In addition the authors describe the use of Bayes's theorem during their assessment sessions so as to give the experts as much insight as possible into the meaning of the probabilities for which they are being asked. The analysis of magnetic fusion choices by North and Stengel (1982) highlights an early use of influence diagrams. In this example the influence diagrams are primarily used during the probability assessment sessions to remind the experts about what information is available and what assumptions are being made. 8.J.2 Howard, Matheson, and North (1972) - hurricane seeding Beginning in 1970 Howard, Matheson, and North explored areas in which decision analysis could be usefully applied to problems faced by the US Environmental Science Service Administration. Hurricane modification became the focus of their efforts. As part of problem-structuring the decision to seed a hurricane with massive amounts of silver iodide for the express purpose of reducing peak wind speed was segmented into a two-stage process: (1) a policy decision to lift the prohibition against hurricane seeding, and (2) a tactical decision to seed a particular hurricane a few hours before it reaches a coastal area. The analysis reported on by the authors addresses the policy decision. As a result specific geographical and meteorological details that would prove important in the tactical decision were not considered; rather, a representative severe hurricane was examined using the best available scientific data and judgment. The actual decision tree representing this policy decision is very simple (Figure 8.19) — the government either decides to seed the hurricane or not, and there is some uncertainty about the sustained surface wind speed which causes property damage. Injuries and loss of life were not explicitly included in the analysis because they were felt to be dependent upon the issuance and effectiveness of storm warnings. However, there was a second important effect that had to be considered in the government's
248
Practice Decision alternatives
Resolution of uncertainty Change in maximum sustained surface wind
Consequences
Property damage Government responsibility Seed
Do not seed
Property damage Government responsibility
Figure 8.19. Hurricane seeding decision tree. This simple decision tree describes a complex policy decision. Source: Howard et al. 1972, Figure 2.
decision to seed hurricanes; namely, the responsibility that the government was willing to assume for property damage. This issue is summarized by the authors: Suppose that, if the hurricane is not seeded, the surface wind intensifies as shown by the curve w(t) in [Figure 8.20] and that, if the hurricane is seeded, the behavior of the wind is that shown by the curve w'(t). The effect of the seeding has been to diminish the wind, thus reducing property damage, yet the wind speed w'(t\) when the hurricane strikes land at time t\ is higher than the wind speed when the seeding was initiated at time t§. Even if the decision maker were certain of w(t\) and w'{t\), he would still have a difficult choice. If he chooses not to seed, the citizens may have more property damage. On the other hand, if he chooses to seed, the citizens may not perceive themselves as better off because of his decision. Instead, they may perceive only that the storm became worse after the seeding and they may blame the decision maker for his choice. The trade off between accepting the responsibility for seeding and accepting higher probabilities of severe property damage is the crucial issue in the decision to seed hurricanes.
Applications
249
^
w(t.)
*(t)
Without seeding w'(t) With seeding
— w(t)
Seeding initiated
Fig. 1. Maximum sustained winds over time.
Landfall
Figure 8.20. Maximum sustained winds over time. Seeding the hurricane may appear to cause an increase in winds, but may have actually reduced the impact of the hurricane. Source: Howard et al. 1972, Figure 1.
The policy decision to permit the seeding of selected hurricanes had been structured and modeled as succinctly as possible. Yet there remained the difficult question concerning how to represent existing data and knowledge about the effects of seeding on maximum sustained surface winds. One approach would have been to use existing data to construct a probability distribution on wind speed and use that in the "do not seed" branch, and then to assess directly from experts the distribution of wind speed when seeding had occurred. If this seeded distribution were conditional on the unseeded wind speed, then one would assess multiple conditional distributions. The authors chose not to pursue this approach. Rather they used a sequence of conditional distributions that best represented the data and the knowledge of experts. In fact, the authors used data on the central pressure changes of hurricanes and a scientifically proposed linear relationship between central pressure changes and wind speed changes to produce a normal probability distribution for wind speed changes during a twelve-hour period that was normal with a mean of zero and a standard deviation of 15.6%. Note that the authors choose to deal with wind speed changes rather than actual wind speed. It is a change in wind speed that the seeding is supposed to cause; therefore this is the impact that the decision tree analysis should address. Since there was considerable uncertainty regarding the effects of seeding, the authors formed three hypotheses: H1, the beneficial hypothesis — the average effect of seeding is to reduce the maximum sustained wind speed. H2, the null hypothesis - seeding has no effect on hurricanes. H3, the detrimental hypothesis - the average effect of seeding is to increase the maximum sustained wind speed.
250
Practice
If Hi were true the following information was used to produce the beneficial result: (1) the average reduction would be 15% with a standard deviation of 7%, and (2) the effect of seeding the individual average hurricane would not differ from the average but would have a standard deviation about the average of 7%. Combining these two normal distributions yields a normal distribution for the nominally seeded hurricane with a mean equal to — 15 % of the average unseeded hurricane and a standard deviation of 10%. Assuming that natural wind changes and seeded wind changes are independent, the probability of wind speed during the twelve-hour period for the beneficial hypothesis is normal with a mean of 85% and a standard deviation of 18.6%. The null hypothesis resulted in a normal distribution with a mean of 100% and a standard deviation of 15.6%. The detrimental distribution was normal with a mean of 110% and a standard deviation of 18.6%. The actual distribution given current information could not be defined until the three hypotheses had been assigned probabilities:
p(w') = J p(w' I Hi) p(Hi) i=\
In assisting the experts in assigning these probabilities, the authors used some recent experimental results. Hurricane Debbie had been seeded twice in 1969 and wind speed reductions of 31% and 15% were observed over twelve-hour periods. Bayes's theorem can be used to relate the (prior) probabilities of the three hypotheses before the experiments to (posterior) probabilities after the experiments:
where u is the experimentally determined wind speed resulting from one seeding, and p(u) = ]T p(w' = u I Hi) p(Ht). This formula would have to be applied twice for the two experiments, assuming independence. If the prior probabilities over the three hypotheses were equal, then the posterior probabilities are 0.81 for Hi, 0.15 for H2, and 0.04 for H3. In fact the authors found that the experts believed that prior to the Debbie experiment "seeding was unlikely to have any effect but that, if seeding did have an effect, it was more likely to be a reduction in wind speed than an increase." So using the information
Applications CHANGE IN MAXIMUM SUSTAINED WIND PROBABILITIES ASSIGNED TO OUTCOMES
PROPERTY
GOVERNMENT
DAMAGE
COST
TOTAL COST
(millions of dollars)
(percent of property damage)
(million! of dollars)
$335.8
•60%
$503.7
191.1
46.7
46.7
16.3
16.3
336J
101.1
100.0
46.7
46.7
16.3
16J
Figure 8.21. Results for nominal hurricane. The expected losses from seeding were shown to be less than from not seeding for this set of assessments. Source: Howard et al. 1972, Figure 6. that prior to Debbie H i was more likely than H3, and after Debbie H i and H2 are equally likely, the authors solved for the following results: /?(Hi) = 0.15 p(Hz) = 0.75 p(H3) = 0.10
p(Hi I u) = 0.49 p(H2 I u) = 0.49 p(H3 I u) = 0.02
This formulation of the uncertainty regarding the seeding of hurricanes provided for ways to use the existing data that had been collected as well as the data from the modeling activities associated with hurricanes. Also it provided a great deal of feedback to the experts who were left with the judgments concerning the validity of the three hypotheses. Probability assessment can be much more than just asking an expert for numbers. The seeded and unseeded probability distributions were ultimately
252
Practice
discretized and associated with estimates of property damage and costs for government responsibility as shown in Figure 8.21. As can be seen from the figure there is some advantage to seeding. Further analyses were done to calculate the value of perfect information concerning the three hypotheses and the value of another seeding experiment. The key point to this illustration is the use of creative modeling to simplify the probabilistic elicitation of the experts and to maximize the use of available empirical data. 5.5.3 North and Stengel (1982) — program choices in magnetic fusion In the late 1970s North and Stengel conducted an analysis of four magnetic fusion program choices for the US Department of Energy via the Los Alamos Scientific Laboratory. The four alternatives included two medium-cost options. (1) nominal development pace of the tokamak concept, including the construction of a major engineering test facility, (2) nominal development pace of the mirror concept, including the construction of a major engineering test facility; one high-cost option: (3) accelerated development of the tokamak and mirror concepts, including an engineering test facility for the mirror concept and an engineering prototype reactor for the tokamak concept: and one low-cost option: (4) delay construction of major test facilities by using existing or smaller new facilities. The authors devoted considerable attention to problem- and modelstructuring in this effort, and the interested reader is directed to their article. The analytic structure developed for this problem not only had to distinguish between the tokamak and mirror concepts, but also between other magnetic fusion and non-fusion energy-producing concepts during the first half of the twenty-first century. The decision tree developed for this analysis depicts these considerations (see Figure 8.22). Note that the probabilities of mirror performance are dependent upon the long-run performance of the tokamak concept, and that the probabilities for the performance of other fusion concepts are dependent upon the performance of both tokamak and mirror concepts. The authors recommended modeling the various fusion reactor concepts and assigning probabilities to the model inputs to obtain these distributions. However, their analysis effort did not contain the resources or time to permit this. So they constructed a representation of a fusion reactor using an influence diagram (see Figure 8.23). The authors' description of this representation is:
NEAR-TERM FACILITYDECISION
LONG-RUN TOKAMAK PERFORMANCE
LONG-RUN MIRROR PERFORMANCE
LONG-RUN OTHER MFE CONCEPT PERFORMANCE
LONG-RUN COMPETING TECHNOLOGIES PERFORMANCE
LONG-TERM MFE R&D STRATEGY I
COMPLETION OF DEVELOPMENT
COMMERCIAL IZATION OUTCOME I
I Tokamak I Nominal
Good 2010
Mirror / Nominal
Good
Moderate
Good
Moderate
Poor
Moderate
Poor
Poor
I
Commercialization
Good
Poor
<
2015 Accelerated , No Nominal 2050 Abandon
i Accelerated
Delay
Figure 8.22. Decision tree structure for magnetic fusion R & D . This decision tree included n o t only t h e long-run R & D uncertainties but also a subsequent R & D decision to pursue development of the technologies for possible commercialization. Source: North a n d Stengel 1982, Figure 1.
Commercialization
PHYSICS
DESIGN
COMPONENTS
BLOCKS
Plasma behaviour Particle confinement Energy confinement Instability thresholds Scaling laws Burn behaviour
Magnetic materials and power supplies Fields from superconductors Storage/transfer of energy
First wall materials Resistance to radiation Resistance to thermal stress
1
First wall construction and maintenance
Energy removal and BOP electric generation
Figure 8.23. Influence diagram for assessing probability distributions on fusion reactor outcomes. This influence diagram enabled the assessment of probabilities at the component level while key principles of physics and design concepts were kept in mind. Source: North and Stengel 1982, Figure 2.
REACTOR
Applications
255
Starting from the right side of the figure, the probability of performance (cost/kWh) levels in the overall reactor is conditioned on the level of performance of two blocks of components: the thermonuclear island (defined to be the reacting plasma and its magnetic confinement system) and the physical containment and energy transfer, which includes the first-wall, blanket and tritium breeding system, and the balance of [the] plant for electricity generation. Each of the components should be regarded as conditional on a given design concept, which specifies reactor geometry, timing, materials, and the procedures for construction, operation, and maintenance. The design concept in turn depends upon physics and engineering parameters, and a given design may imply tradeoffs that change the difficulty of constructing viable component systems. The actual assessment of probabilities is carried out on component performance, but we ask the expert whose judgment is being assessed to be specific in his or her assumptions about the design concept and the physics and engineering that motivates this design. Careful definition of the design is essential when performance probabilities for different components are being assessed by different experts. The influence diagram structure of [Figure 8.23] is intended to be sufficiently generic to apply to a range of different design concepts for both tokamaks and mirror fusion reactors. The performance of the overall reactor is built up from the assessment of components, with the two blocks (thermonuclear island, physical containment and energy transfer) as an intermediate stage. We have chosen the components so that for a given design concept the probabilities to be assigned to component performance are reasonably assumed to be independent. Without the assumption of independence, assessment of probabilities on eight components would be a formidable task. The assessment of reactor performance probabilities will obviously depend on the state of information available. We have arranged the diagram in [Figure 8.23] to show on the left-hand side important areas of uncertainty in plasma physics, magnetic materials and power supplies, and first-wall materials. For our analysis we wished to assess probabilities based not only on present knowledge, but on knowledge that might be obtained by the early 1980s in the event of successes on the experimental fusion reactors now under construction. We therefore began by asking the expert subject to assess probabilities on component performance for a mature technology fusion reactor based on his or her present state of knowledge. We then asked the subject to assume that successful results have been obtained on the experimental machines now under construction, and we asked the subject to consider what changes these results would imply in knowledge of physics and engineering and, hence, what changes in the design are assumed for a mature technology reactor.
The probabilities for tokamak and mirror performance were assessed using this influence diagram structure. The other fusion concepts were not as well defined, so the probabilities of success were directly assessed. This direct assessment approach was used for the remaining probabilities as well. The costs and benefits for each alternative were measured in dollars.
256
Practice
Costs included the research and development costs for a particular alternative. Benefits were defined to be the dollar savings realized by using a specific magnetic fusion reactor alternative rather than the most competitive alternative. The final results suggested that the variation between alternatives was much less than the variation resulting from uncertainty for a particular option. The authors conclude: "This would suggest the first decision is not extremely critical in determining the overall outcomes of magnetic fusion R & D." The discount rate (time preference) was also found to be highly critical since the development was spread over fifty years and the real pay-offs were seventy-five years in the future. The use of influence diagrams to keep the experts apprised of the questions being asked and the state of information to be assumed is highlighted in this analysis. 8.6 VALUE ASSESSMENT
8.6.1
Introduction
The first application that illustrates value assessment is by Dyer and Lund (1982) and examines merchandizing packages for Amoco Oil Company. This analysis illustrates the creative transformation of subjective judgments concerning sales margin into value weights. Also included in this analysis is a generic specification of decision areas comprising any merchandizing strategy. This systematic approach is often useful in the problem-structuring phase when the decision space is large. The resource allocation analysis for the US Marine Corps by Kuskey, Waslov, and Buede (1981) demonstrates the balance-beam assessment technique described in Section 7.5.4. 8.6.2 Dyer and Lund (1982) — merchandizing package In 1977 Dyer and Lund began an analysis to find a better way to market gasoline, motor oil, tires, batteries, and accessories for the marketing managers of Amoco Oil Company. The analysis focused on full-activity service stations, which, in addition to selling gasoline, also have service bays and do repairs and sell tires, batteries, and accessories (TBA). The study lasted two years, resulted in a strategy of guaranteed car repair called "Certicare," and received a second place award for applications of management science from the College on the Practice of Management Science (CPMS) of The Institute of Management Science (TIMS). A total of eleven new strategies were defined via a "Christmas tree" representation of twenty-four separate decision areas (Figure 8.24). Each
Applications
2-57
Current
Strategy decision areas
level
Product TBA brand
(Compet | - | Private j -
iNarrowhH Med h
Extent of Amoco accessory line fMedl
TBA quality
I
Amoco atlas
fMTTTT—
-Pyi
T
Price/prom'n to dealer/jobber
fCompet h-| High \-
Price to Dir/Jbr-TBA
1 High |-
- Motor oil
[Nol— H Yes |
Brackets - TBA - Motor oil Dir/Jbr incentives - TBA
| No|— H Yes | [Non el—fUowl
- Motor oil |Nor H—ILowl Tie-ins to Dir/Jbr
[MedJ—
II IMedl— H High | |Noneh [30 daysHHExt'dl
Terms - TBA - Motor oil
f30 dayshHExt'dl
Promotion to Customer Media adv. (M$) - TBA - Motor oil - Total Cust. incentives - TBA - Motor oil Tie-ins to cust. (incl. gasoline)
JNonej— - f C u r r l |Noneh ICurr.h iNonel— H Lowh fNoneT- -TLo^V | None |-
Product/repair guarantee Organization/distribution Sales organization (TBA) Salary/commission GO/reg support/training Dir organization (TBA) TBA distribution/whsg
Figure 8.24. The Christmas tree of merchandizing alternatives. This clever problem-structuring concept for organizing alternatives aided the winnowing of alternatives prior to the serious analysis. Source: Dyer and Lund 1982, Figure 3. of the twenty-four decision areas is a row in the Christmas tree, and one box in each row is chosen for a given strategy. This representation enabled the marketing managers to think of new ways of doing business - a systematic and creative approach to problem structuring. In determing the effect of any strategy four significant dimensions had to be examined: event, product, retail outlet, and marketing channel. A children's toy set for building wooden structures was used to communicate the variations in the first three dimensions for the reseller market channel (Figure 8.25). Moving from front to back in the model, each "slice" represents a person and whether they buy or sell - Amoco sells, dealer buys, dealer sells, and customer
Practice
z58
Figure 8.25. Structural model of merchandizing alternatives. This children's toy set enabled the experts to visualize the model strcture and how their assessments would be used. Source: Dyer and Lund 1982, Figure 1. buys. Across the length of the model, each "slice" represents a product — tires, batteries, accessories, motor oil, and gasoline. The final dimension, the layers in the model, represents the type of outlet - full-facility stations, self-service stations, and specialty stores like tire stores that were being tested at the time of this study. Thus each knob is at an intersection uniquely designating a person and his action regarding a specific product at a particular retail outlet, and represents one of the objectives to be influenced by a marketing strategy. The sticks linking the knobs illustrate the major interactions among the objectives that we sought to quantify with the judgmental model. The short sticks entering each knob on a 45 degree angle represent the impacts of strategy on the objectives of marketing.
Another model was developed for the jobber channel. The analysis of marketing strategies proceeded along two tracks defined by the following equations: (8.1) i
i
k
I
(8.2)
Applications
259
EveHltS 4
3
2
1
Amoco sells
Dealer buys
Dealer sells
Customer buys
Normal selling effort
Sales history
Normal selling effort Recent ' Amoco purchases
,
$150 )
V
J
15% /
\
»( $1,000 )
V
20%/
\
V
J
* { $5,000 )
J
^ j ^ 20%j$30
^ y ^ 25%f$250
Strategy
Strategy
5% /
^-r-^ 30%|$1,500 Strategy
X 30%
H$100,OOOJ
V
J
* Related
v
*-r^ product 10%j$10,000 sales Strategy
Figure 8.26. Allocation of sales margins. The process for selling a 5-quart oil change is shown here with the relevant sales margins. Source: Dyer and Lund 1982, Figure 2. where i represents the events, / the products, k the outlet types, / the channels, d the decision areas of the strategy, and a the alternative being evaluated. The weights in the first equation were developed via judgmental models that allocated the sales margins, as shown in Figure 8.26. Four events are required for the sale of a 5-quart motor oil change. Circles representing these events correspond to the knobs in the Tinker Toy Model. Percentage impacts of the factors on the events are shown on the arrows. $100,000 sales margin is allocated to strategy impacts according to the percentages. Illustrating this judgmental model for only / and /, let pi = the percentage contribution of the merchandizing strategy at event h
prr_x = the percentage contribution of event r to event r — 1, and ra; = the adjusted sales margin produced by product;. Then w\) = p\ mh Wij =
p\pi,\mh
W3j = Plp2,ip3amh
a n dS O O n
'
The authors felt that this conversion of sales margin allocations into weights provided a much more meaningful interpretation of multiattribute weights for the marketing managers than was possible with the typical abstract interpretation of importance. The weights in Equation (8.2) were assessed in the typical fashion. The results of the analysis are plotted in Figure 8.27. This plot uses two criteria - cash flow and return - to discriminate between the alternatives. Four alternatives were ruled out because they had negative or near zero cash flows. The best strategy was considered questionable because it created a relationship between the dealers and Amoco that raised con-
z6o
Practice
Questionable legality
Return on merchandising costs, A Margin/A Cost
Figure 8.27. Evaluation of merchandizing strategies (relative to current). This chart depicts the recommended alternative and the reasons for discarding the most competitive options. Source: Dyer anbd Lund 1982, Figure 5. cerns about the independence of the dealers. Certicare was considered the next best balance between cash flows and return. These results were supported by further detailed studies and sensitivity analyses. The major point of this illustration is the creation of a value model that incorporates judgments that are easily made by the decision-makers, rather than using the brute force approach commonly employed. The problem-structuring concepts employed here are also worth noting. 8.6.3 Kuskey, Waslov, and Buede (1981) - Marine Corps's POM development The Program Objectives Memorandum (POM) is a US Department of Defense document that establishes how the military services plan to allocate their fiscal resources over a five-year period that begins about sixteen months from the date of the document. Each service (Army, Navy, and Air Force) begins preparing this document about eight months prior to the document's date. The US Marine Corps prepares its POM as part of the Navy's. Decision analysts began assisting the Marine Corps in assigning priorities to their resource initiatives or programs in 1977 for the POM whose first year was 1979 and have been expanding the scope of programs considered and analytic assistance ever since. This application of decision analysis is distinctive for its twin emphases on the judgmental and group aspects of decision-making. The analysis
Applications
261
process emphasizes collecting specialized expert judgments from across the organization and then combining these judgments via a sampling process so that a consistent set of judgments across areas of expertise can be used as the basis of priorities. The implementation of this process on a yearly basis has resulted in the formation of a new committee of Marines that spans the expertise of specialized line organizations and addresses issues of integration. The resource allocation process begins as each line organization defines its programs, each of which requires funding during all or part of the five year period under consideration. This definition includes estimates of funding in each of the five years. These programs are (1) typically increments of a major effort that is partially guaranteed of funding and (2) independent of the other programs both in terms of funding and benefit. Ultimately the Marine Corps will be given several funding levels (minimum, basic, and enhanced), each of which have yearly expenditure constraints. As part of the POM priorities the Marine Corps must identify which programs fall within the five-yearly constraints of the minimum funding level, the basic level, and the enhanced level. At the end of the POM process the funding estimates of some of the programs are adjusted so that all five constraints are met for each of the three levels. During the prioritization process, however, these funding estimates are considered fixed. Originally the prioritization process was a negotiation process in which the line managers (or sponsors) of programs met and argued about which programs deserved to be funded at each of the three levels on the basis of need or "benefit." Decision analysts established a three-phased process in which: (1) the sponsors scaled the relative benefits of their own programs, (2) the Program Evaluation Group (PEG), a newly formed group of Marines with expertise across all sponsors, assigned a benefit scale to a sample of programs from each sponsor's area, and (3) the sponsor scales were merged into a single benefit scale and the programs were prioritized on the basis of their benefit-to-cost ratios (where cost is the total five-year cost in constant dollars). After this cost-benefit list of priorities is established, some adjustments are made to accommodate such things as guidance received from the Department of Defense. The benefit elicitation process, which is of interest here, is derived from conjoint measurement as developed in Krantz et al. (1971) and summarized by the authors. The process yields a ratio scale and is thus useful for eliciting the weights for an additive, multi-attribute value function. As an example of this process suppose a given sponsor has nine programs labeled A, B , . . . I. Also suppose the sponsor has established a benefit rank order for these programs consistent with the above order.
z6z
Practice
The analyst presents the group [of experts] with choices between individual items and combinations of items, beginning at the top of the rank-order with item A. Item A is compared to the combination of items B and C, and the group must decide whether they would prefer the purchase of A alone, or the purchase of both B and C (on the basis of benefit alone). Suppose the group decides A > (B + C) (A is preferred to both B and C). The analyst then makes the selection of A less attractive by offering B and C and D together versus A. Questions about A continue until an indifference point is reached, e.g. A = (B + C + D), or the preference reverses, e.g. (B + C + D) > A. Suppose, on the other hand, that (B + C) > A. Then, the analyst makes the preferred side less attractive by offering B and D versus A, or C and D versus A. Again, the comparisons continue until an indifference point or a reversal is found. . . . The ease of the subsequent numerical assignment will depend on the number of preference relations elicited and the tightness of interval about each item. With respect to the latter criterion, the analyst should strive to bound [the value of each program] as narrowly as possible to reduce the size of solution space. For example, the preference relation (D + E + F) > B > (C + D) provides more information about [the value of B] than does the relation A > B > (E + F). Suppose that in the example, the analyst elicited the following preferences: (1) (B + C) > A (2) A > (C + D) (3) B = (C + D) (4) C > (D + E) (5) (D + E + F) > C (6) D > (E + F) (7) (E + F + G) > D (8) E = (F + G) (9) (H + I) > F (10) (H + I) > G . . . In order to derive [the value scale for A through I], we first fix the value of I arbitrarily at 10. Then we determine the ratio of value between H and I by asking, "How many initiatives like I would it take to provide the same importance as H?" Suppose the response is 2, so that (I + I) = H . . . . The analyst still has a great deal of flexibility in assigning numbers that solve the inequalities, and one might reduce the solution space by presenting further comparison questions. Typically, however, one discovers the best comparison to ask by going directly on to the assignment of numerical values. The analyst relates the items' intervals of value that are implied by the preference relations, and the participants are asked to select the benefit value. In the example, we know that 20 < v(G) < 30, and the group decides that v(G) = 25. Now, 25 < v(F) < 30, and the group decides that v(¥) = 26. v(E) is calculated to be 51, so 77 < v(D) < 102. Here, the group has a wider range of values from which to choose, and decides that D is closer in value to (E + F + G) and assigns it
Applications
263
a benefit of 100. Subsequently, they let v(C) = 160, v(B) = 260, and v(A) = 300. . . . The analyst uses the numerical scale to suggest new comparisons.
This analytical process has been used to prioritize many types of programs, such as research and development efforts, procurement initiatives, operations and support projects, and packages involving additional work-years. In addition, a decision support system (Section 6.6.2) has been developed for use by the Marine Corps for managing the cost and benefit data and producing the reports necessary for the POM submission to the Navy and the Department of Defense. This discussion demonstrates a powerful value elicitation technique that has not previously been reported in the literature. 8.7 SENSITIVITY ANALYSIS
8.7.1 Introduction This section illustrates the use of two-way and three-way sensitivity analyses at the conclusion of an analysis to highlight the impact of the important judgments made by the decision-maker. The first example is from the medical field and examines the appropriateness of cardiac catheterization on infants. The authors, Macartney et al. (1984) set up a decision tree of alternatives, uncertainties, and outcomes and then use two- and three-way sensitivity analyses to examine the crossover points on key probabilities. The second case study examines nuclear plant siting. Here Kirkwood (1982) uses a three-way sensitivity analysis to define the impacts of two key probabilities as well as the result of varying the value weight on the socioeconomic factors. 8.7.2 Macartney, Douglas, and Spiegelhalter (1984) - cardiac catheterization Macartney, Douglas, and Spiegelhalter used a decision tree analysis to examine three alternative treatments for sick infants with coarctation of the aorta. The decision tree (Figure 8.28) includes the following uncertainties: (1) the patient may or may not have the disease (D 4- and D — ), (2) the patient may or may not have the disease confirmed by the cardiac catheterization (T 4- and T — ), and (3) the patient may or may not survive (L -I- and L — ). Table 8.12 presents the key probabilities used in this analysis. As can be seen in Figure 8.28 the authors assigned a value of zero to death and a value of one to survival. (The difficult question of valuing survival in
264
Practice
•
Decisions
O
Probabilities
#
Death
(A) Survival
Figure 8.28. Catheterization decision. This simplification of the catheterization decision still contains many probabilities. Note the clever use of probability multipliers (h and g) that parameterize the effects of the operation after catheterization and the effects of catheterization with no disease, respectively. Source: Macartney et al. 1984, Figure 1. more detail has been dealt with elsewhere. See Howard (1984b), JonesLee (1976), and Pliskin et al. (1980)). Figure 8.29 presents a two-way sensitivity analysis of the results to two independent probabilities, so there is no need to insure that their sum is one or less. As the mortality of catheterization increases, the procedure becomes less and less attractive. The major point of interest is where the decision crossover points are for low probabilities of mortality. To examine the effect of a third variable, p — the survival probability after the treatment is adminstered, a series of plots (Figure 8.30) are
Table 8.12 Definition of catheterization variables" Probability Literal interpretation Probability of survival given that the disease is present and operated on Probability of survival given that the disease is absent, and yet the patient is operated on for that disease Probability of survival given that the disease is present, but no operation is performed Probability of survival given that the disease is absent and no operation for that disease is carried out Sensitivity of cardiac catheterization for the disease 1 — (specificity of cardiac catheterization for the disease) Relative reduction in probability of survival of operation, owing to cardiac catheterization having been carried out Relative risk of catheterization in absence of the disease compared with that in its presence Probability of disease being present Probability of dying between onset of catheterization and operation, given that the disease is present
Statistical notation
Variables for Coarctation model
p(L + I D + , O + , C - )
0.9
p(L + I D + , O + , C - )
0.6
Given t h a t n o catheterization is p e r f o r m e d
p(L+|D + ,O + , C - )
p(L+ I D + , O + , C - )
0.95
p(T+|D + ,C + )
0.99
p(T + I D - , C + ) I D±, O + , C + p{L + j D±, O + , C -
0.95
p(L-|D-,C +) p(L- I D + , C + ) p(D + )
p(L-
I D + , C-)
Notes: dThis table presents the major variables and their values. The complements to these probabilities are not presented but can be calculated as 1 — p. L + , patient survives; L — , dies; D + , has disease; D — , does not have disease; O + , operation carried out; O — , operation not carried out; C + , catheterization carried out; C - , catheterization not carried out; T + , catheterization indicates disease present; T - , catheterization indicates disease not present. Source: Table i of Macartney et al. 1984.
Practice
266 80
Neither catheterise nor operate Operate without catheterisation
0
10
2 0 3 0 4 0 5 0 6 0 70 8 0 9 0 K X ) Probability of disease being present (•/.)
Figure 8.29. Two-way policy region analysis of catheterization decision. This figure presents the results of the most basic policy region analysis for this problem. Source: Macartney et at. 1984, Figure 2. presented for the four combinations of coarctation or not and surgery or not. As can be seen here the point at which the two curves intersect at mortality rates near 50% moves both vertically and horizontally. Yet the decision crossovers at low mortality rates are very stable except for the second condition — coarctation is absent yet the patient is operated on for coarctation. The variations in survival probability are changed as the coarctation state and treatment are varied. 8.7.3 Kirkwood (1982) - nuclear power plant site selection The topic of nuclear power plant siting has received significant attention by decision analysts due to the public interest it has received. This paper presents generic results from an extensive analysis of alternative sites. The decision tree used to measure uncertainty for this analysis is shown in Figure 8.31. The initial decision node is self-explanatory. There are several possible water sources for the two sites, and the first chance node represents the uncertainty that water source A is available for use by the reactor for cooling. Besides using natural water sources for cooling, dry cooling technologies could be available. Finally the subsequent decision node addresses how much (in thousands of acre-feet per year) of the three water sources (A, B, and C) are used, depending on the availability of source A and dry cooling technology. To measure values and risk the nine attributes in Table 8.13 were defined. These factors cover system cost, population risks, biological
70 60
oL
50
80 85 90 95 100
40 UOIJf
30
tBO OE
lorta lity i
3
^^
10 0
E
- p = 0.99 1 - p = 0.9 j - p = 0.8 §
20
!
(a) Coarctation is present and operated on.
/
.
i
i
i
i
i
i
(b)
80 70 60 50
(b) Coarctation is absent but operation is performed anyway.
40 30 20 10 10 20 30 40 50 60 70 80 90 100
0 80
•(c)
70 60
A
-
g 30
to (A
iac cai
0)
•o
3
ality
*s •M
o
r
10
-
5
-
50 40
15
20
o
80 85 90 95 100
V
r
(c) Coarctation is present but no operation is performed.
03 | 1 o.oi 1
- P = 0.1
10 0 80 • t o )
15
r
70
10
-
60
5
50
40 30 20
/
o
*&.
i
80 85 90 95 100
(d) Coarctation is absent and no operation is performed.
0 99 I — P = 0.951 p= 09 1
—
10 0
10 20 30 40 50 60 70 80 90 100 Probability of disease being present (%)
Figure 8.30. Three-way sensitivity analysis of catheterization. This figure shows how the bottom half of Figure 8.29 changes as a third dimension is varied, (a) Coarctation is present and operated on. (b) Coarctation is absent but operation is performed anyway, (c) Coarctation is present but no operation is performed, (d) Coarctation is absent and no operation is performed. Source: Macartney et al. 1984, Figure 3.
268
Practice I I
Source used
Utility (base case)
pa = Probability that water from surface. Source A is available pd = Probability that dry cooling technology is available
0.38
Figure 8.31. Decision tree for nuclear plant siting. This decision tree makes a subsequent decision explicit. Source: Kirkwood 1982, Figure 2.
Applications
269
Table 8.13 Attributes for nuclear plant siting Valuation measure X1: levelized system cost X2: site population X3: biological impact at plant site X4: biological impact at water source X5: environmental impact of corridors X&: socioeconomic effects at plant site
Xy: short-term socioeconomic effects at water source Xg: long-term socioeconomic effects at water source
X9: loss of potentially irrigable land
2
Definition Levelized annual cost in year o dollars. This includes water-sensitive costs, site-sensitive costs, and baseline reference costs. Site population factor. A 13-point constructed scale. A 3-point constructed scale. A weighted sum of the distances for the electrical transmission and water supply pipeline corridors. 0 Annual population growth rate due to peak-year construction activities is less than 15%. This indicates that there are population centers near the site with existing infrastructure to serve as a base for the new population influx. 1 Annual population growth rate due to peak-year construction activities is more than 15%. This indicates that there are no existing population centers of significance: boom-bust development is virtually certain to occur. The value of production directly affected by the withdrawal of irrigation due to diversion of water to the power plant, in year o dollars. For each water source, the scale used is: (aggregate annual personal income per annual quantity of water consumption) x (quantity of water supplied to the power plant from the water source) in levelized year o dollars. Weighted square miles of cropland that could potentially be retired from use at a candidate water source, using the following weighting factors on area: 1 cropland currently under irrigation 1 land highly suitable for irrigation 0.7 land moderately suitable for irrigation
Note: ^This analysis includes a mix of subjective and objective, continuous and discrete value attributes. Source: Table 2 of Kirkwood 1982.
Practice
270
Site 1 prefered
5k
Site 2
"«S
prefered
1
w
0
i
i
i
I
i
i
i
i
i
0.2 0.4 0.6 0.8 1.0 Dry cooling probability (pd)
Figure 8.32. Three-way sensitivity analysis for nuclear plant siting. This clever plot shows the impact of various socioeconomic pricing policies. Source: Kirkwood 1982, Figure 3(c).
impacts, socioeconomic impacts, environmental impacts on electrical and watt pipeline corridors, and economic impacts on farming operations. The inclusion of the socioeconomic factors was questioned, however: so as part of the sensitivity analysis the value of socioeconomic costs was varied from the baseline (a dollar of socioeconomic costs equaled a dollar of system cost) to no value being placed on the socioeconomic factors. Figure 8.32 shows the policy regions for sites 1 and 2 as functions of (1) the probability that water source A is available and (2) the probability that dry cooling technology is available. The shaded region shows the area that is impacted by the value placed upon socioeconomic factors. So this is a three-way sensitivity analysis that examines both probabilities and values.
DECISION SYNTHESIS APPRAISED
In this closing chapter we will appraise in three ways the decision synthesis that we have described in the other chapters. First we will discuss how we can know whether to trust the recommendations of a decision analysis should the synthesized decision be taken, or ought we to be cautious about the limitations of our analysis? This is the study of how a decision synthesis may be validated, and it will be discussed in Section 9.1. This will involve an account of how sensitivity analysis may be used to improve the validity of an analysis and, in a rather different argument, discussion showing that, if decision synthesis is constructive, validity is less of a problem. Secondly, in Section 9.2, we shall discuss what needs to be done to make decision synthesis a yet more effective procedure for enhancing decision-making. We shall review the research areas that, at the time of writing, seem to be the most important, and speculate on the likely changes that we might see in the practice of decision analysis in the next few decades. Finally, we conclude the book with a discussion of the ethics of doing decision synthesis. 9 . 1 THE VALIDATION OF DECISION ANALYSIS
Whenfirstintroduced to the ideas of decision theory, many people find the structure and consistency provided by the theory very attractive, but then find great difficulty in applying the ideas to practical problems. Questions new users ask themselves often include: - A m I sure that the subjective probabilities that I have used are correct? - Does my utility function really look like that? - Have I structured the problem properly - are all the areas of concern adequately represented in the problem? - Am I convinced that the attributes that I have used satisfy preference independence? Questions like these make us doubt the validity of our analysis; it is essential to ask, therefore, how we may know whether the analysis is valid 271
zjz
Practice
or not. We shall give two answers to this question; the first will be to urge the use of sensitivity analysis, and the second will be to suggest that the problem itself is founded on a misapprehension of what decision analysis is set up to do. We have already discussed sensitivity analysis in this book, particularly in sections 7.6 and 8.7. We described methods for discovering whether the answers to an analysis depend at all strongly on the actual numbers that we use to represent our beliefs and judgments. We concluded that, if the results are not very sensitive to the inputs, then the recommended decisions can be adopted with some confidence. If the results are sensitive to the inputs, then we must be careful to put more effort into eliciting inputs accurately. Several observations should be made about this view of validation. First, sensitivity analysis can be costly, and as a consequence is often not done in practice as carefully as might be desirable. Not many texts that we are familiar with in management science observe that in practice analysis has to be done on a limited budget, but our experience is that this fact is one of the most important determinants of the type of analysis that is applied to a practical problem. The analyst always has to ask whether it is worthwhile doing more analysis; this is particularly true of sensitivity analysis. If not much sensitivity analysis has been applied to a decision analysis, however, we have to place a question mark over its validity. Second, it is fair to say that the art of doing a sensitivity analysis on a decision model of the kind we have been discussing in this book is far from fully worked out. The straightforward computation of how the answers vary with one, two or even three of the input parameters can be satisfactorily achieved, as we showed in Chapter 7. What is not so clear, and in fact remains an open research question, is how to explore sensitivities when all the input parameters can vary at once, particularly if those sensitivities are to be mapped back into the original judgments which led to the numerical inputs in the first place. Suppose we discover that a certain combination of probability assessments could lead to a different optimal choice. Imagine that those assessments had been produced by asking a set of interrelated questions, comparing the likelihoods of different events. The decision-maker will be interested in knowing what answers he would need to give to the complete set of elicitation questions to produce a combination of numerical assessments that switches the optimal decision. What is needed, and to our knowledge is not currently available, is a protocol for carrying out this complicated kind of sensitivity analysis. What would be even more desirable would be a computer program, maybe framed as an expert system, to aid the analyst in this process. Thirdly, so far we have assumed that the structure of the model is given;
Decision synthesis appraised
273
that is, the shape of the decision tree, the number and definition of attributes, the form of the utility function, and so on, are known with confidence. This is clearly not true for many practical problems, and, in fact, the sensitivity of the answers of our analysis to these aspects may well be more significant than sensitivity to parameters such as the weights in a utility function, or the probabilities on branches of a decision tree. At present we have little idea how to carry out a formal sensitivity analysis with respect to these structural aspects of a decision analysis; all we can do is to try a few different formulations to see if the answers differ. There is, however, an alternative answer to the problem of the validity of a decision analysis. It is a rather radical one, namely that the problem itself is ill-formed. When we ask whether a decision model is valid, we are asking whether the model is a true representation of the real world. This is the question familiarly asked of engineering models (Does this structural model represent the stresses in a bridge accurately enough? Does that control model represent sufficiently well the relationship between the controls on a chemical reactor and its performance?). It is also asked in traditional areas of operational research (Is this linear program a true representation of the movement of oil products in a refinery?). A presupposition for these questions is that a model is valid if it represents an objective reality closely enough for us to be confident that deductions from the model will be valid deductions about reality. But, if we try to apply this way of thinking to decision models of the type that we have described in this book, we have to assume the existence of an objective reality that we are attempting to model. What is this reality? Is it some property of our brains that we are trying to reveal? Is there a true preference structure, or set of probability distributions, lurking in our subconscious, which we are trying to tease out, despite our shortcomings as measuring instruments? This view has undoubtedly been prevalent amongst decision analysts in the past, and follows on naturally from the world-view of the engineer or the scientist. As Phillips (1984) powerfully argues, however, there is an alternative interpretation of what is going on when a decision analysis is done. We are not so much revealing our hidden, but finely tuned, perceptions of value and uncertainty; we are, rather, creating a structure which satisfies certain principles of logic and consistency that we wish to adopt in our decisionmaking (and which we call rationality), and which is consistent with those judgments that we can articulate confidently. Phillips icalls this "requisite decision modelling," by which he means that a moldel is valid once it appears to the decision-maker to be requisite for the decision in hand. Such a model appears to represent all the issues of importance to the decision-maker. We would go a little further than Phillips, since we can imagine situations where a requisite model might be able to produce
274
Practice
implications that the decision-maker cannot accept and yet he is not aware of these implications since by chance they have not been made. Our definition of a valid decision model is one where all the implications that could be made from the model are consistent with what the decisionmaker knows about his beliefs and value judgments. Usually we will \\ f know whether or not this is true for a particular model. In consequence, the problem of validation, which almost disappears in Phillips's interpretation of requisite decision models, is still a problem. How do we know whether our decision model is consistent with all the judgments we feel confident in making? Notice that, despite the validation problem being still with us, it is not nearly as severe as before. We do not have to agonize over whether we really prefer option A to option B; if we feel unable to articulate a preference between them, and yet, as a consequence of our other judgments and the principles of rationality that we wish to adopt, the model tells us that A should be marginally preferred to B, then we will be happy with that synthesized judgment. In this regard the model is valid, not because we know that we prefer A to B, but because there is no articulable preference between the options that clashes with the implications of our model. Thus we still have a validation problem but it is less significant. It is solved by looking carefully at as many implications as possible that can be drawn from the model, and asking whether the decision-maker is prepared to live with them. This is a subtly different kind of sensitivity analysis. No longer do we seek to find what possible set of judgments could lead to different conclusions; rather, we ask whether the implications of a particular set of judgments lead to contradictions with other intuitive judgments. To summarize, we recognize problems in validating decision models. Clearly sensitivity analysis, with respect both to the parameters of a model and to its structure, is necessary if we interpret a decision model as an attempt to describe a real entity within our brain. But, even if we adopt the constructive view of decision models outlined above, this kind of sensitivity analysis can point up the inconsistencies between the implications of our model and our intuitive judgments. In any event, it is clearly necessary to devote considerable effort to ensure that, if we use a decision model as the basis for action, it is not misrepresenting our beliefs and value judgments. 9 . 2 RESEARCH DIRECTIONS
Few students of decision theory and its applications would claim that the subject was closed and fully worked out. We see it developing extensively over the next decades, particularly as the potential of cheap and powerful
Decision synthesis appraised
275
computers is harnessed by effective software. It is not only the computer programs that need to be developed, however. Further research work is needed on the psychology of decision-making, on the formation of decisions within organizations, and on the theoretical underpinnings of decision theory, as well as in other areas of practice. In this section we shall give a review of what we believe to be the most important of these research topics. We base our views not only on our personal knowledge of decision synthesis, in theory and practice, but also on other authors' reviews of research needs (Keeney 1982, Winkler 1982, US National Research Council 1983 and Cohen et al. 1985). 9.2.1 Research on how decisions are made Before attempting to improve decision-making we should understand how it is done at present. We reviewed some of the work psychologists have done on how people make decisions in Chapter 4. That work was concerned both with whether people conform to the norms of decision theory and, if they do not (and many do not!), with identifying which norms they do conform to. Despite this extensive body of research, there is still a need to determine what people do when they make a decision, and to relate this understanding to other areas of cognitive psychology, such as studies of attention, perception, and inference. Many important decisions take place in a social context, however, and we gave a brief introduction to studies of organizational decisionmaking in Chapter 5. What further research is needed on this topic? First, we need to have a better notion of how decisions actually emerge in an organization. Secondly, we need to discover how the introduction of decision-aiding procedures into an organization affects the individuals within that organization and its organizational dynamics. There is no point in trying to ensure that decisions are made more in accord with normative principles of consistency, if the overall effectiveness of the organization diminishes due to the disaffection of its members, or for some other reason. Finally, we should discover to what extent decision-aiding procedures can be manipulated by unscrupulous members of an organization to achieve their personal goals at the expense of those of the organization. Research in some of these areas is only just beginning, but we judge it to be very important. If the "information revolution" does come about, as many commentators are currently predicting, the concept of decision-aiding through the use of expert systems or similar computer programs will become widespread; if we do not know what the consequences of introducing such systems in an organization are, there are very likely to be some highly undesirable consequences!
Practice
9.2.2 Research on the foundations of decision theory As we saw in Chapters 2 and 3, the rationale for decision analysis starts with some sensible postulates for what rationality means, and then, using these as axioms, deduces that the sensible action is the one which maximizes a utility or value function. On the other hand, as we saw in Chapter 4, most people do not naturally make decisions in conformity with these axioms. We may then ask what scope there might be for alternative decision theories, where the axioms of behavior are more in accord with what people actually can do than with what they would like to do. Can we, for example, develop a value theory which does not require a complete preference order over alternatives? Is there a theory of uncertainty which does not require the decision-maker to make arbitrarily fine judgments of relative likelihood? There has been considerable interest in these questions over the last few decades. For example, a most interesting development has been suggested by Fishburn (1982, 1983). He has constructed a normative theory of decision-making based on axioms which are both weak enough to accommodate the judgments that people can actually make and strong enough to have normative appeal (see Bell and Farquhar (1986) for a discussion of this theory). It might well be easier to construct a decision aid based on these norms of rational decision-making, related as they are to the natural judgmental abilities of humans, than to base an aid on the conventional theory. Such decision aids remain to be developed, however. 9.2.3 Research on problems of applying decision theory In the absence of alternative decision theories, however, decision synthesis is concerned with taking the classical decision theory which we described in Chapter 3 and applying it to real decision problems. There are many areas within this general focus where research is needed; we highlight two of these. Structuring As we discussed in each of the last three chapters, when we apply decision analysis in a new context the first thing we have to do is to structure the problem and then to structure a model to fit the problem. A glaring gap in our knowledge is how to do this properly. It is of course done each time an analysis is performed, but each analyst will have his own procedure; there is no agreed protocol for structuring. Each of the four reviews of research in this subject that we listed above gives prominence to this point; it is clearly widely perceived as a weak point in decision synthesis. There are several different aspects to the structuring problem.
Decision synthesis appraised
277
First, there is the problem of discovering decision options. Very often a decision-maker is aware of a problem but not of what may be done about it. Before analyzing alternative actions, or options, they must first be generated. There is, at present, no formal protocol for doing this. Of course, usually a few options spring to mind, and a detailed study of the situation may suggest some more - indeed, it has sometimes been argued that one of the important advantages of any formal approach to decision-making is that the process often suggests options which have not previously been considered. There are some procedures which have been put forward for option generation, such as lateral thinking or brainstorming, but research is still needed to derive formal protocols for option generation. Next is the problem of structuring attributes. What is it that makes one option more attractive than another? In some cases, of course, there is no problem here. If, for example, we are quite clear that all we are interested in is our final asset position, this will be the single attribute of interest. It is usual, however, for there to be many conflicting attributes and, moreover, for it to be unclear how to describe them, let alone find a variable with respect to which each option may be measured. Consider, for example, the decision on where to site a nuclear waste repository. We will obviously be concerned with potential radiation damage, and the costs of building and maintaining the repository, as well as all the other environmental factors. But how should these be characterized? Take the esthetics of the repository, for example. We may well be concerned that the repository does not intrude into a fine natural landscape; but what precisely is our concern, and how should it be measured for different possible repository sites? There is at present no procedure for determining what an appropriate set of attributes should be. Research is currently under way here, and we believe it to be an important area. A last aspect of the structuring problem that we shall mention concerns the creation of value hierarchies. It is often convenient to group attributes in a hierarchical structure, for ease of thinking about the problem and to simplify the elicitation of values. Since there are clearly many ways to go about structuring a hierarchy, we must ask what are the best ways to do this. Once again, this is an open and important research area. Assessing probabilities A very early concern of those engaged in applying decision theory to practical problems was how to elicit subjective probabilities as inputs to a decision model. To some extent good procedures have been discovered, but the subject is by no means closed. Winkler (1982) goes into the open research questions in this area in some detail. He suggests that the following areas need further attention:
278
Practice
(1)
the assessment of multivariate probability distributions — that is, for random variables which are statistically dependent; (2) the conditions in which probabilities can be more easily elicited by indirect than direct methods; (3) the assessment of probabilities for rare events; (4) how to handle the varying degrees of confidence a decision-maker may have in different probability assessments; (5) the development of interactive computer programs for the more efficient elicitation of probabilities; (6) effective methods for approximating empirical distributions to insure analytic tractability; (7) methods for sensibly using the information of people with expert knowledge in formulating a probability. We believe all these to be important research areas. The lack of efficient procedures, in some situations, for representing uncertainty with probabilities stands in the way of more widespread use of decision synthesis. Assessing values We described, in Section 7.5, some methods for the elicitation of values, expressed either as value functions or as utility functions. Readers will be clear from our account that the subject is not closed. We particularly need to find better ways of eliciting weights for an additive value function, since this formulation is so much used in practice. The ideas of overspecification and of pairwise comparison (see our discussion of the analytic hierarchy process in Section 3.5.6) need to be developed to fit into the normative framework of multi-attribute value analysis. The goal is a procedure for weight elicitation which people can use naturally for drawing out judgments in an accessible way and which has a proper intellectual basis. There are many other topics in the elicitation of values which require research, but we shall leave this section by simply observing the need for a simple procedure to test whether the common assumption of additivity in a value function is valid in a particular case. Determining the type of analysis We have touched elsewhere in this book on the decisions an analyst has to take on what kind of analysis to do, and how much to do. Questions that analysts have to decide include: whether a simple analysis is adequate, or whether it is necessary to be more complicated; whether it is proper to assume that a value function is additive, or a utility function linear; whether a decision-maker should be forced into giving coherent probabilities, and, if so, by what method. At present there are no satisfactory protocols for guiding the analyst in answering such questions. Although research to construct such protocols
Decision synthesis appraised
279
is difficult, it is essential for the satisfactory development of decision synthesis. 9.2.4 Research into methods for communicating analyses The final area of research which we will discuss here is perhaps the most important. We may have a superb bundle of techniques for analyzing a decision problem and synthesizing a recommendation; but, if the decision-maker does not understand the analysis or believe its results, not much has been achieved. Indeed, if the results are believed without the argument being understood, the consequences could be disastrous, particularly if the analyst has not properly captured the decision-maker's perceptions. We identify the following important research topics in this area: (1) the use of animated computer graphics to assist in the explanation of decision analyses; (2) the use of interactive computer programs for structuring decision problems, constructing value and utility functions, eliciting weights, and eliciting probabilities; (3) the construction of visual representations of an analysis which fit well into the organizational setting - for example, an aid which can enhance the deliberations of a committee, the members of which may have different goals. 9.2.5 Implications for decision synthesis Readers who have studied our list of research topics might well conclude that so much needs to be done that decision synthesis must necessarily be inadequate at present. That this is not so is evidenced by studies of the extent to which decision synthesis is used (see Ulvila and Brown (1982), for example). Many decision-makers clearly find the procedures of decision analysis very satisfactory, despite the shortcomings of the methodology which the list of research topics displays. Although an imperfect analysis is not necessarily better than no analysis, in our experience it usually is! We should recognize that the purpose of analysis is to serve decision-makers. This is achieved by constructing analyses which the decision-maker understands and finds helpful. We believe that more decision-makers will find the methods of decision synthesis useful if the research program outlined above is followed through successfully. 9.3 THE ETHICS OF DECISION ANALYSIS
Some readers may be surprised at our discussion of ethics in a book such as this. Decision analysis is a procedure for guiding people who wish to
280
Practice
abide by its precepts because they see them as an ideal to which they wish to conform. As we have stressed several times, neither we, nor the community of decision theorists, nor anything in the theory itself requires that anybody should use decision analysis. Is it not, then, ethically neutral? If decision analysis were simply carried out by individuals working on their own problems, then this might well be true. In fact, however, most analyses are done by an analyst for somebody else, either another individual or, more commonly, an organization. It is here that the ethical problems arise for the analyst; how should he or she ensure that in doing analysis the interests of others are not prejudiced? This issue has not received a very great deal of attention in the literature, but an interesting discussion is given in Fischhoff (1980b). He makes an analogy between decision analysis and psychoanalysis, which are both activities whose purpose is to provide people with cognitive help. Fischhoff points out that any attempt to shape and direct the lives of others cannot be totally disinterested; the analyst will only with great difficulty be able to avoid bringing his own prejudices into the subject's decision problem. Even if he does, there are aspects of decision analysis which make hidden assumptions. The process may, for example, encourage people to believe that there is a "true" solution to their decision problem, which they are unable to recognize because of personal inadequacies; the truth might be, on the contrary, that their difficulty in deciding has a much deeper cause, which the process of decision analysis fails to bring to light. It is improper, if this occurs, for the decision analyst to move into a therapeutic counseling role without a proper understanding of, and training in, counseling methods. Most decision analyses are not done, however, for individuals, but for organizations. In this context the analyst must be wary that the analysis is not biased by his or her personal views. Analyses can, of course, be biased while representing the facts truthfully. Advocacy in a law court has this property, and it is even more true of political speeches. One of the advantages claimed for decision analysis, however, is that it promotes consistency and clarity; in the public arena the decision analyst has a duty to construct an analysis which is capable of illuminating public debate, rather than proving the case for one side or the other. There is the further problem that many decision analyses are commissioned by public bodies. These bodies are limited in the amount of money they can spend on an analysis, and often the analyst will wish to spend more effort on getting a good analysis than can be afforded. The ethical problem here for a consultant who makes a living at decision analysis is whether to turn down an under-resourced job. Moreover, once into an analysis, it might be that the sponsor, a public official, will insist that the analysis is carried out in a certain way, perhaps to support a particular agreement. We
Decision synthesis appraised
281
would respond to this by endeavoring to construct a model where different viewpoints could be represented within the same structure (see, for example, Watson (1985)). Ethical dilemmas of this nature cannot always be escaped in this way, however. As Fischhoff (1980b) points out, codes of professional ethics are established for psychotherapy; what we now need is a comparable one for decision analysis. Those of us who, in seeking to improve decisionmaking, are fascinated by the intellectual framework and practical procedures that decision synthesis offers will be enabled, by such a code, to set about our task confident of the ethical validity of what we do.
REFERENCES
Ackoff, R.L., 1979a ' T h e future of operational research is past," Journal of the Operational Research Society, 30, 93—104. 1979b "Resurrecting the future of operational research," Journal of the Operational Research Society, 30, 189-99. Adelman, L., 1984 "Real-time computer support for decision analysis in a group setting: another class of decision support system," Interfaces, 14, 75-83. Albrow, M., 1970 Bureaucracy, London: Macmillan. Alemi, F., Cline, B., Gustafson, D.H., and Bosworth, K., 1981 Use of Decision Analysis in Setting Rules: Application in Program Management, Madison: Centre for Health Systems Research and Analysis, University of Wisconsin. Allais, M., 1953 "Le Comportement de Phomme rationnel devant le risque: critique des postulats et axiomes de l'ecole americaine," Econometrica, 21, 503-46. Allison, G.T., 1971 Essence of Decision: Explaining the Cuban Missile Crisis, Boston: Little, Brown. Alpert, M. and Raiffa, H., 1982 "A progress report on the training of probability assessors," in Kahneman, D., Slovic, P., and Tversky, A., eds., Judgment Under Uncertainty: Heuristics and Biases, Cambridge University Press. Arrow, K.J., 1951 Social Choice and Individual Values, New York: Wiley. Barclay, S. and Peterson, C.R., 1976 Multi-attribute Utility Models for Negotiations, Technical report 7 6 - 1 , McLean, VA: Decisions and Designs, Inc. Barron, F.H., von Winterfeldt, D., and Fischer, G.W., 1984 "Empirical and theoretical relationships between value and utility functions," in Borcherding, K., Brehmer, B., Vlek, C , and Wagenaar, W.A., eds., Research Perspectives on Decision Making under Uncertainty, Amsterdam: North-Holland. Baumol, W.J., 1972 Economic Theory and Operations Analysis, Englewood Cliffs, NJ: Prentice-Hall. Behn, R.D. and Vaupel, J.W., 1982 Quick Analysis for Busy Decision Makers, New York: Basic Books. Bell, D.E., 1977 "A decision analysis of objectives for a forest pest problem," in Bell, D.E., Keeney, R.L., and Raiffa, H., eds., Conflicting Objectives in Decision, New York: Wiley. 1979a "Consistent assessment procedures using conditional utility functions," Operations Research, 17, 1054-66. 282
References
283
1979b "Multi-attribute utility functions: decompositions using interpolation," Management Science, 25, 744-53. Bell, D.E. and Farquhar, P.H., 1986 "Perspectives on utility theory," Operations Research, 34, 179-83. Belton, V. and Gear, A.E., 1983 "On a shortcoming of Saaty's method of analytic hierarchies," Omega, 11, 228-30. Bennett, P.G., 1977 "Toward a theory of hypergames," Omega, 5, 749-51. Bennett, P.G. and Dando, M.R., 1979 "Complex strategic analysis: a hypergame study of the fall of France," Journal of the Operational Research Society, 30, 23-32. Bennett, P.G., Dando, M.R., and Sharp, R.G., 1980 "Using hypergames to model difficult social issues: an approach to the case of soccer hooliganism," Journal of the Operational Research Society, 31, 621-35. Bernoulli, D., 1738 "Specimen theoriae novae de mensura sortis," Commentarii academiae scientiarum imperalis Petropolitanae, 175—92; translated in Sommer, L., 1984, Econometrica, 22, 23-6. Bodily, S.E., 1985 Modern Decision Making, New York: McGraw-Hill. Boothroyd, H., 1984 "The deliberative context of systems analysis," in Tomlinson, R. and Kiss, I., eds., Rethinking the Process of Operational Research and Systems Analysis, Oxford: Pergamon Press. Borel, E., 1938 "Applications aux jeux de hasard," in Traite du calcul des probabilites et de ses applications, Paris: Gauthier-Villars. Brams, S.J., 1978 The Presidential Election Game, New Haven, CT: Yale University Press. 1979 Applied Game Theory, Wurzburg: Physica-Verlag. 1980 Biblical Games: A Strategic Analysis of Stories in the Old Testament, Cambridge, MA: MIT Press. Brock, H.W., 1980 "The problem of 'utility weights' in group preference aggregation," Operations Research, 28, 176-87. Brown, R.V., 1970 "Do managers find decision theory useful?" Harvard Business Review, 48, 78-89. 1984 "Risk-benefit analysis - a response," Omega, 12, 425-6. Brown, R.V. and Lindley, D.V., 1986 "Plural analysis: multiple approaches to quantitative research," Theory and Decision, 20, 133-54. Brown, R.V., Kahr, A., and Peterson, C.R., 1974 Decision Analysis for the Manager, New York: Holt, Rinehart, and Winston. Brown, R.V., Laskey, K.B., and Chinnis, L., 1986 Probabilistic Estimates of Mislabeled Down Garments for Sale by Sears/Kellwood in 19771-78, Technical Report 85-8, Falls Church, VA: Decision Science Consortium, Inc. Buchanan, B.G. and Shortliffe, E.H., 1984 Rule-based Expert Systems, Reading, MA: Addison-Wesley. Buede, D.M., 1979 "Applied decision analysis: engineering science versus clinical art," in Proceedings of the International Conference on Cybernetics and Society, New York: IEEE. 1982 "Decision conferences at Decisions and Designs, Inc.," presented at the World Future Society, Washington, DC, 18 July. 1986 "Structuring value attributes," Interfaces, 16, 52-62.
284
References
Buede, D.M. and Choisser, R.W., 1984 "An aid for evaluators of system design alternatives," Defense Management Journal, 32-8. Buehring, W.A., 1975 "A model of environmental impacts from electrical generation in Wisconsin," Ph.D. dissertation, Department of Nuclear Engineering and Institute of Environmental Studies, University of Wisconsin. Bunn, D.W., 1984 Applied Decision Analysis, New York: McGraw-Hill. Checkland, P., 1981 Systems Thinking, Systems Practice, Chichester: Wiley. 1983 "OR and the systems movement: mappings and conflicts," Journal of the Operational Research Society, 34, 661-75. Chen, K., Mathes, J.C., Jarboe, K., and Wolfe, J., 1979 "Value oriented social decision analysis: enhancing mutual understanding to resolve public policy issues," IEEE Transactions on Systems, Man, and Cybernetics, SMC-9,
567-80. Clough, D.J., 1984 Decisions in Public and Private Sectors, Englewood Cliffs, NJ: Prentice Hall. Cohen, L.J., 1977 The Probable and the Provable, Oxford: Clarendon Press. Cohen, M.D., March, J.G., and Olsen, J.P., 1972 "A garbage can model of organizational choice," Administrative Science Quarterly, IJ, 1-25. Cohen, M.D., Huber, G., Keeney, R.L., Levis, A.H., Lopes, L.L., Sage, A.P., Sen, S., Whinston, A.B., Winkler, R.L., von Winterfeldt, D., and Zadeh, L., 1985 "Research needs and the phenomena of decision-making and operations," IEEE Transactions on Systems, Man, and Cybernetics, SMC-15, 764-75Condorcet, Marquis de, 178 5 Essai sur Vapplication de Vanalyse a la probability des decisions rendues a la pluralite des voix, Paris. Crawford, D.M., Huntzinger, B.C., and Kirkwood, C.W., 1978 "Multiobjective decision analysis for transmission conductor selection," Management Science, 14, 1700-9. Dawes, R.M., 1976 "Shallow psychology," in Carroll, J.S. and Payne, J.W., eds., Cognition and Social Behavior, Hillsdale, NJ: Erlbaum. 1977 "Predictive models as a guide to preference," IEEE Transactions on Systems, Man, and Cybernetics, SMC-y, 355-7. 1979 "The robust beauty of improper linear models in decision making," American Psychologist, 34, 571-82. Dawes, R.M. and Corrigan, B., 1974 "Linear models in decision making," Psychological Bulletin, 81, 95—106. Dawid, A.P., 1982 "The well-calibrated Bayesian," Journal of the American Statistical Association, yy, 605—10. De Bono, E., 1970 Lateral Thinking: Creativity Step by Step, London: Harper and Row. De Finetti, B., 1974 The Theory of Probability, London: Wiley. De Groot, A., 1965 Thought and Choice in Chess, The Hague: Mouton. DeGroot, M.H., 1970 Optimal Statistical Decisions, New York: McGraw-Hill. De Montgolfier, J. and Bertier, P., 1978 Approche multicritere des problemes de decision, Suresnes: Editions Hommes et Techniques. Dempster, A.P., 1968 "A generalisation of Bayesian inference," Journal of the Royal Statistical Society, B30, 205—47.
References
285
Diesing, P., 1962 Reason in Society', Urbana, IL: University of Illinois Press. Dreyfus, S.E., 1984 "The risks! and benefits? of risk-benefit analysis," Omega, 12,335-40.
Dyer, J.S. and Lund, R.N., 1982 "Tinker toys and Christmas trees: opening a new merchandizing package for Amoco Oil Company," Interfaces, 12, 38-52. Dyer, J.S. and Miles, D., 1976 "An actual application of collective choice theory to the selection of trajectories for the Mariner Jupiter/Saturn project," Operations Research, 24, 220-43. Dyer, J.S. and Wendell, R.E., 1985 A Critique of the Analytic Hierarchy Process, Working paper 84/85-4-24, University of Texas at Austin. Ebbesen, E.B. and Konecni, V.J., 1980 "On the external validity of decisionmaking research: What do we know about decisions in the real world?" in Wallsten, T.S., ed., Cognitive Processes in Choice and Decision Behavior, Hillsdale, NJ: Erlbaum. Eden, C. and Sims, D., 1977 Problem Definition Between Clients and Consultants, Centre for the Study of Organizational Change and Development, University of Bath. Eden, C , Jones, S., and Sims, D., 1979 Thinking in Organisations, London: Macmillan. 1983 Messing About in Problems, Oxford: Pergamon Press. Edwards, W., 1954 "The theory of decision making," Psychological Bulletin, 51,330-47. 1961 "Behavioral decision theory," Annual Reviews of Psychology, 12, 59-731968 "Conservatism in human information processing," in B. Kleinmuatz, ed., Formal Representations of Human Judgement, New York: Wiley. 1971 "Social utilities," Engineering Economist, Summer Symposium Series, 6. 1977 "How to use multi-attribute utility measurement for social decisionmaking," IEEE Transactions on Systems, Man, and Cybernetics, SMC-y, 326-40. 1983 "Human cognitive capabilities, representativeness, and ground rules for research," in Humphreys, P., Svenson, O., and Vari, A., eds., Analysing and Aiding Decision Processes, Amsterdam: North-Holland. Einhorn, H J . and Hogarth, R.M., 1981 "Behavioral decision theory: processes of judgment and choice," Annual Reviews of Psychology, 32, 53-88. 1982 A Theory of Diagnostic Inference, Center for Decision Research, University of Chicago. Ellsberg, D., 1954 "Classic and current notions of 'measurable utility'," Economic journal, 64, 528-56. 1961 "Risk, ambiguity and the Savage axioms," Quarterly Journal of Economics, 75, 643-9. Elster, J., 1979 Ulysses and the Sirens: Studies in Rationality and Irrationality, Cambridge University Press. Fine, T.L., 1973 Theories of Probability, New York: Academic Press. Fischhoff, B., 1975a "Hindsight: thinking backward?" Psychology Today, 8 (11),
71-6.
286
References
1975b "Hindsight =£ foresight: the effect of outcome knowledge on judgement under uncertainty," Journal of Experimental Psychology: Human Perception and Performance, 1, 288-99. 1980a "For those condemned to study the past: reflections on historical judgment," in Shweder, R.A. and Fiske, D.W., eds., New Directions for Methodology of Behavior Science: Fallible Judgment in Behavioral Research, San Francisco: Jossey-Burs. 1980b "Clinical decision analysis," Operations Research, 28, 28-43. 1983 "Reconstructive criticism," in Humphreys, P., Svenson, O., and Vari, A., eds., Analysing and Aiding Decision Processes, Amsterdam: NorthHolland. Fischhoff, B., Slovic, P., and Lichtenstein, S., 1980 "Knowing what you want: measuring labile values," in Wallsten, T.S., ed., Cognitive Processes in Choice and Decision Behavior, Hillsdale, NJ: Erlbaum. Fischhoff, B., Lichtenstein, S., Slovic, P., Derby, S.L., and Keeney, R.L., 1981 Acceptable Risk, Cambridge University Press. Fischhoff, B., Goitein, B., and Shapiro, Z., 1982 "The experienced utility of expected utility approaches," in Feather, N., ed., Expectations and Actions: Expectancy-value Models in Psychology, Hillsdale, NJ: Erlbaum. Fishburn, P.C., 1967 "Methods of estimating additive utilities," Management Science, 13,455-55. 1970 Utility Theory for Decision Making, New York: Wiley. 1976 "Cardinal utility: an interpretive essay," Revista Internazionale di Scienze Economiche e Commerciali, 13, 1102-14. 1982 "Nontransitive measurable utility," Journal of Mathematical Psychology, 26, 31-67. 1983 "Transitive measurable utility," Journal of Economic Theory, 31, 293-317. Fraser, N.M. and Hipel, K.W., 1980 "Metagame analysis of the Poplar River conflict," Journal of the Operational Research Society, 31, 377-85. Freeling, A.N.S., 1980 "Fuzzy sets and decision analysis," IEEE Transactions on Systems, Man, and Cybernetics, SMC-10, 341-54. Fryback, D.G. and Keeney, R.L., 1983 "Constructing a complex judgmental model: an index of trauma severity," Management Science, 29, 869-83. Gardiner, P.C. and Edwards, W., 1975 "Public values: multiattribute utility measurement for social decision making," in Kaplan, M.F. and Schwartz, S., eds., Human Judgment and Decision Processes, New York: Academic Press. Geoffrion, A.M., 1983 "Can MS/ORi evolve fast enough?" Interfaces, 13, 10-25.
Goldberg, L.R., 1970 "Man versus model of man: a rationale, plus some evidence, for a method of improving on clinical inferences," Psychological Bulletin, 73, 422-34. Guttentag, M. and Snapper, K., 1974 "Plans, evaluations and decisions," Evaluation, 2, 58-74. Hacking, I., 1975 The Emergence of Probability, Cambridge University Press. Hall, R.H. and Quinn, R.E., eds., 1983 Organizational Theory and Public Policy, Beverly Hills, CA: Sage.
References
287
Hammond, K.R., 1986 "Generalization in operational contexts: What does it mean? Can it be done?" IEEE Transactions on Systems, Man, and Cybernetics, SMC-16, 428-3 3. Hammond, K.R., Stewart, T.R., Brehmer, B., and Steinmann, D.O., 1975 "Social judgment theory," in Kaplan, M.F. and Schwartz, S., eds., Human Judgment and Decision Processes, New York: Academic Press, 272-307.
Hammond, K.R., Rohrbaugh, J., Mumpower, J., and Adelman, J., 1980 "Social judgment theory: applications in policy formation," in Kaplan, M.F. and Schwartz, S., eds., Human Judgment and Decision Processes in Applied Settings, New York: Academic Press, 1-29. Harsanyi, J.C., 1977 Rational Behaviour and Bargaining Equilibrium in Games and Social Situations, Cambridge University Press. Hershey, J.C., Kunreuther, H.C., and Schoemaker, P.J.H., 1982 "Sources of bias in assessment procedures for utility functions," Management Science, 28, 936-54Hertz, D.B. and Thomas, H., 1983 "Decision and risk analysis in a new product and facilities planning problem," Sloan Management Review, Winter, 17-31. Hogarth, R.M., 1980 Judgment and Choice, New York: Wiley. 1986 "Generalization in decision research: the role of formal models," IEEE Transactions on Systems, Man, and Cybernetics, SMC-16, 439—49. Holsti, O.R., 1976 Operational Code Belief Systems: A Codebook, Durham, NC: Duke University Press. Hoos, I.R., 1972 Systems Analysis in Public Policy: a Critique, San Francisco: University of California Press. Howard, N., 1970 The Arab-Israeli Conflict: A Metagame Analysis, Technical report, Department of System Design, University of Waterloo, Canada. 1971 Paradoxes of Rationality: Theory ofMetagames and Political Behavior, Cambridge, MA: MIT Press. Howard, R.A., 1968 "The foundations of decision analysis," IEEE Transactions on Systems, Science and Cybernetics, SSC-4, 211-19. 1975 "Social decision analysis," Proceedings of the IEEE, 63, 359—71. 1980 "An assessment of decision analysis," Operations Research, 28, 4—27. 1984a "The evolution of decision analysis," in Howard, R.A. and Matheson, J.E., eds., The Principles and Applications of Decision Analysis, Menlo Park, CA: Strategic Decision Group. 1984b "On fates comparable to death," Management Science, 30, 407-22. Howard, R.A. and Matheson, J.E., 1981 Influence Diagrams, reprinted in Howard, R.A. and Matheson, J.E., eds., The Principles and Applications of Decision Analysis, Menlo Park, CA: Strategic Decision Group, 1984. eds., 1984 The Principles and Applications of Decision Analysis, Menlo Park, CA: Strategic Decision Group. Howard, R.A., Matheson, J.E., and North, D.W., 1972 "The decision to seed hurricanes," Science, IJ6, 1191-202. Hurst, E.G., Ness, D.N., Gambino, T.J., and Johnson, T.H., 1983 "Growing DSS: a flexible evolutionary approach," in Bennett, J.L., ed., Building Decision Support Systems, Reading, MA: Addison-Wesley.
288
References
Janis, I.L., 1972 Victims of Groupthink, Boston: Houghton Mifflin. Janis, I.L. and Mann, L., 1977 Decision Making: A Psychological Analysis of Conflict, Choice, and Commitment, New York: Free Press. Jannsson, D.S. and Taylor, S.H., 1978 "Search activity in social agencies: institutional factors that influence policy analysis," Social Services Review, 52,
189-201.
Jones-Lee, M., 1976 The Value of Life, University of Chicago Press. Kahneman, D. and Tversky, A., 1972 "Subjective probability: a judgment of representativeness," Cognitive Psychology, 3, 430—54. 1973 "On the psychology of prediction," Psychological Review, 80, 237-51. Kahneman, D., Slovic, P., and Tversky, A., eds., 1982 Judgment Under Uncertainty: Heuristics and Biases, Cambridge University Press. Keefer, D.L., 1978 "Allocation planning for R&D with uncertainty and multiple objectives," IEEE Transactions on Engineering Management, EM-25, 8-14. Keefer, D.L. and Bodily, S.E., 1983 "Three-point approximations for continuous random variables," Management Science, 29, 595—609. Keelin, T.W., 1981 "A parametric representation of additive value functions," Management Science, 27, 1200-8. Keen, P.G.W. and Scott Morton, M.S., 1978 Decision Support Systems: An Organizational Perspective, Reading, MA: Addison-Wesley. Keeney, R.L., 1977 "The art of assessing multiattribute utility functions," Organizational Behavior and Human Performance, 19, 267-3 I O 1981 Issues in Evaluating Alternative Standards, San Francisco: WoodwardClyde Consultants. 1982 Decision Analysis: State of the Field, Technical Report 82-2, San Francisco: Woodward-Clyde Consultants. Keeney, R.L. and Raiffa, H., 1976 Decisions with Multiple Objectives: Preferences and Value Tradeoffs, New York: Wiley. Keller, L.R. and Farquhar, P.H., 1974 Decisions Under Certainty: Assessment Methods for Value Functions, Irvine: Graduate School of Management, University of California. Kelly, G.A., 1955 The Psychology of Personal Constructs, New York: Norton. Kepner, C.H. and Tregoe, B.B., 1981 The New Rational Manager, Princeton, NJ: Princeton Research Press. Kirkwood, C.W. 1982 "A case history of nuclear power plant site selection," Journal of the Operational Research Society, 33, 353-63. Kraemer, K.L. and King, J.L., 1983 Computer Supported Conference Rooms, Irvine: Public Policy Research Organization, University of California. Krantz, D.H., Luce, R.D., Suppes, P., and Tversky, A., 1971 Foundations of Measurement, New York: Academic Press. Kuskey, K.P., 1980 "Decision conferences," ORSA/TIMS conference, Washington, DC, May. Kuskey, K.P., Waslov, K.A., and Buede, D.M., 1981 Decision Analytic Support of the United States Marine Corps' POM Development: A Guide to the Methodology, Final Report PR 81-6-158, McLean, VA: Decisions and Designs, Inc.
References
289
Kyburg, H.E. and Smokier, H.E., 1964 Studies in Subjective Probability', New York: Robert E. Krieger. Layard, R. (ed.), 1972. Cost-Benefit Analysis, Harmondsworth: Penguin. Lee, W., 1971 Decision Theory and Human Behavior', New York: Wiley. Lichtenstein, S. and Fischhoff, B., 1980 "Training for calibration," Organizational Behavior and Human Performance, 26, 149—71. Lichtenstein, S. and Slovic, P., 1971 "Reversals of preferences between bids and choices in gambling decisions," Journal of Experimental Psychology, #9, 46-55. 1973 "Response-induced reversals of preferences in gambling: an extended replication in Las Vegas," Journal of Experimental Psychology, 101,16-20. Lichtenstein, S., Fischhoff, B., and Phillips, L.D., 1982 "Calibration of probabilities: the state of the art to 1980," in Kahneman, D., Slovic, P., and Tversky, A., eds., Judgment Under Uncertainty: Heuristics and Biases, Cambridge University Press. Lindblom, C.E., 1959 "The science of muddling through," Public Administration Review, 19, 79—99. 1979 "Still muddling, not yet through," Public Administration Review, 39, 517-26. Lindley, D.V., 1973 Making Decisions, New York: Wiley. 1982 "Scoring rules and the inevitability of probability," International Statistical Review, 50, 1-26. Lindley, D.V., Tversky, A., and Brown, R.V., 1979 "On the reconciliation of probability assessments," Journal of the Royal Statistical Society, A142,
146-80. Linstone, H.A., 1984 Multiple Perspectives for Decision Making, New York: North-Holland. Luce, R.D. and Raiffa, H., 1957 Games and Decisions, New York: Wiley. Luce, R.D. and Tukey, J.W., 1964 "Simultaneous conjoint measurement: a new type of fundamental measurement," Journal of Mathematical Psychology, 1, 1-27.
Ludke, R.L., Stauss, F.F., and Gustafson, D.H., 1977 "Comparison of five methods for estimating subjective probability distributions," Organizational Behavior and Human Performance, 19, 162-79. Macartney, F., Douglas, J., and Spiegelhalter, D., 1984 "To catheterise or not to catheterise? An approach based on decision theory," British Heart Journal, 51,330-8. McGrew, A.G. and Wilson, M.J., eds., 1982 Decision Making: Approaches and Analysis, Manchester University Press. Madey, G.R. and Dean, B.V., 1984 "A model for the selection of an R&D portfolio within the firm," unpublished paper, Case Western Reserve University, December. Majone, G. and Quade, E.S., eds., 1980 Pitfalls of Analysis, New York: Wiley. Mamdani, E.H. and Gaines, B.R., 1981 Fuzzy Reasoning and Its Applications, New York: Academic Press. March, J.G., 1978 "Bounded rationality, ambiguity, and the engineering of choice," Bell Journal of Economics, 10, 587-680.
290
References
Marshall, A., 1890 Principles of Economics, London: Macmillan. Matheson, J.E., 1983 "Decision analysis practice: examples and insights," in Howard, R.A. and Matheson, J.E., eds., The Principles and Applications of Decision Analysis, Menlo Park, CA: Strategic Decision Group. Matheson, J.E. and Winkler, R.L., 1976 "Scoring rules in probability assessment and evaluation," Management Science, 22, 1087-96. Meehl, P.E., 1954 Clinical Versus Statistical Prediction: A Theoretical Analysis and a Review of the Evidence, Minneapolis: University of Minnesota Press. Miller, A.C. and Rice, T.R., 1983 "Discrete approximations of probability distributions," Management Science, 29, 352-62. Miller, A.C, Merkhofer, M.W., and Howard, R.A., 1976 Development of Automated Aids for Dc^sion Analysis, Menlo Park, CA: Stanford Research Institute. Mintzberg, H., 1979 The Structuring of Organizations: A Synthesis of the Research, Englewood Cliffs, NJ: Prentice-Hall. Moore, P.G. and Thomas, H., 1976 The Anatomy of Decisions, Harmondsworth: Penguin. Morris, W.T., 1977 Decision Analysis, Columbus, OH: Grid. Murphy, A.H. and Winkler, R.L., 1977 "Can weather forecasters formulate reliable probability forecasts of precipitation and temperature?" National Weather Digest, 2, 2—9. 1984 "Probability forecasting in meteorology," Journal of the American Statistical Association, 79, 489—500. Nickerson, R.C. and Boyd, D.W., 1980 "The use and value of models in decision analysis," Operations Research, 28, 139-55. North, D.W. and Merkhofer, M.W., 1976 "A methodology for analyzing emission control strategies," Computers and Operational Research, 3, 185-207. North, D.W. and Stengel, D.N., 1982 "Decision analysis of program choices in magnetic fusion energy development," Management Science, 28, 276-88. O'Connor, M.F., 1985 "Methodology for corporate crisis decision-making," in Andriole, S.J., ed., Corporate Crisis Management, Princeton, NJ: Petrocelli Books. Owen, D.L., 1979 "The concept of influence and its use in structuring complex decision problems," Ph.D. dissertation, Stanford University. Pareto, V., 1927 Manuel d'economie politique, second edition, Paris: Giard. Pearson, E.S. and Tukey, J.W., 1965 "Approximate means and standard deviations based on distances between percentage points of frequency curves," Biometrika, 52, 533-46. Peterson, C.R. and Beach, L.R., 1967 "Man as an intuitive statistician," Psychological Bulletin, 68, 29-46. Phelps, R.H. and Shanteau, J., 1978 "How much information can an expert use?" Organizational Behavior and Human Performance, 21, 209-19. Phillips, L.D., 1982 "Requisite decision modelling: a case study," Journal of the Operational Research Society, 33, 303-12. 1983 "A theoretical perspective on heuristics and biases in probabilistic
References
291
thinking," in Humphreys, P., Svenson, O., and Vari, A., eds., Analysing and Aiding Decision Processes, Amsterdam: North-Holland. 1984 "A theory of requisite decision models," Ada Psychologica, 56, 29-48. Phillips, L.D. and Edwards, W., 1966 "Conservatism in a simple probability inference task," Journal of Experimental Psychology, 72, 346-57. Phillips, L.D., Hays, W.L., and Edwards, W., 1966 "Conservatism in complex probabilistic inference," IEEE Transactions on Human Factors in Electronics, HFE-j, 7-18. Pitz, G.F. and Sachs, N.J., 1984 "Judgment and decision: theory and application," Annual Reviews of Psychology, 35, 139-63. Pliskin, J.S., Shepard, D.S., and Weinstein, M.C., 1980 "Utility functions for life years and health status," Operations Research, 28, 206-24. Pugh, D.S., ed., 1971 Organisation Theory, Harmondsworth: Penguin. Pugh, D.S., Hickson, D.J., Hinings, C.R., Macdonald, K.M., Turner, C , and Lupton, T., 1963 "A conceptual scheme for organisational analysis," Administrative Science Quarterly, 289-315. Pugh, D.S., Hickson, D.J., and Hinings, C.R., 1964 Writers on Organisations, Harmondsworth: Penguin. Raiffa, H., 1968 Decision Analysis, Reading, MA: Addison-Wesley. 1982 The Art and Science of Negotiation, Cambridge, MA: Bellknap. Ramsey, F.P., 1931 "Truth and probability," in Braithwaite, R.B., ed., The Foundations of Mathematics and Other Logical Essays, London: Kegan Paul. Rawls, J., 1971 A Theory of Justice, Cambridge, MA: Bellknap. Ross, S.M., 1976 A First Course in Probability, New York: Macmillan. Roy, B., 1973 "How outranking relation helps multiple criteria decisionmaking," in Selected Proceedings of a Seminar on Multi-criteria Decisionmaking, Chapel Hill, SC: University of South Carolina Press. 1977 "Partial preference analysis and decision-aid," in Bell, D.E., Keeney, R.L., and Raiffa, H., eds., Conflicting Objectives in Decision, New York: Wiley. 1985 Methodologie multicritere d'aide a la decision, Paris: Editions Libraire Economica. Saaty, T.L., 1980 The Analytic Hierarchy Process, New York: McGraw-Hill. 1986 "Analytic foundation of the analytic hierarchy process," Management Science, 32, 841-55. Sage, A.P., 1981 "Behavioral and organizational considerations in the design of information systems and processes for planning and decision support," IEEE Transactions on Systems, Man, and Cybernetics, SMC-11, 640-78. Sassone, P.G. and Schaffer, W.A., 1978 Cost-Benefit Analysis: A Handbook, New York: Academic Press. Savage, L.J., 1954 The Foundations of Statistics, New York: Wiley. Scharlig, A., 1985 Decider sur plusieurs criteres, Lausanne: Presses Polytechniques Romandes. Schlaifer, R., 1969 Analysis of Decisions Under Uncertainty, New York: McGraw-Hill. Schoemaker, P.J.H., 1980 Experiments on Decisions Under Risk: The Expected Utility Hypothesis, Boston: Nijhoff.
292
References
Schoemaker, P.J.H. and Waid, C.C., 1982 "An experimental comparison of different approaches to determining weights in additive utility models," Management Science, 28, 182-96. Schwenk, C.R. and Thomas, H., 1983 "Effects of conflicting analyses on managerial decision making: a laboratory experiment," Decision Science, 14, 467-82. Seaver, D.A., Von Winterfeldt, D., and Edwards, W., 1978 "Eliciting subjective probability distributions on continuous variables," Organizational Behavior and Human Performance, 21, 379-91. Sen, A.K., 1970 Collective Choice and Social Welfare, Edinburgh: Oliver and Boyd. Shacter, R.D., 1986 "Evaluating influence diagrams," Operations Research, 34. Shafer, G., 1976 A Mathematical Theory of Evidence, Princeton University Press. Shanteau, J. and Phelps, R.H., 1977 "Judgment and swine: approaches and issues in applied judgment analysis," in Kaplan, M.F. and Schwartz, S., eds., Human Judgment and Decision Processes in Applied Settings, New York: Academic Press. Shortliffe, E.H., 1976 Computer Based Medical Consultation: MYCIN, New York: Elsevier. Simon, H.A., 1957a Models of Man, New York: Wiley. 1957b Administrative Behavior: A Study of Decision-Making Processes in Administrative Organization, second edition, New York: Macmillan. i960 The New Science of Management Decision, New York: Harper and Row. 1969 The Science of the Artificial, Cambridge, MA: MIT Press. 1983 Reason in Human Affairs, Oxford: Blackwell. Sims, D., Eden, C , and Jones, S., 1981 "Facilitating problem definition in teams," European journal of Operational Research, 6, 360-6. Slovic, P. and Lichtenstein, S., 1971 "Comparison of Bayesian and regression approaches to the study of information processing in judgment," Organizational Behavior and Human Performance, 6, 649—744. Slovic, P. and Tversky, A., 1974 "Who accepts Savage's axiom?" Behavioral Science, 19, 368-73. Slovic, P., Fischhoff, B., and Lichtenstein, S., 1977 "Behavioral Decision Theory," Annual Reviews of Psychology, 28, 1—39. Slovic, P., Lichtenstein, S., and Fischhoff, B., 1986 "Decision making," in Atkinson, R.C., Herrnsten, R.J., Lindzey, G., and Luce, R.D., eds., Stevens' Handbook of Experimental Psychology, second edition, New York: Wiley. Smallwood, R.D. and Morris, P.A., 1980 "A task force decision analysis," Operations Research, 28, 60-80. Smith, B.L.R., 1966 The RAND Corporation: Case Study of a Non-profit Advisory Corporation, Cambridge, MA: Harvard University Press. Spetzler, C.S. and Stael von Holstein, C.A., 1975 "Probability encoding in decision analysis," Management Science, 22, 340-52. Spetzler, C.S. and Zamora, R.M., 1984 "Decision analysis of a facilities investment and expansion problem," in Howard, R.A. and Matheson, J.E.,
References
293
eds., The Principles and Applications of Decision Analysis, Menlo Park, CA: Strategic Decision Group. Sprague, R.H. and Carlson, E.D., 1982 Building Effective Decision Support Systems, Englewood Cliffs, NJ: Prentice-Hall. Stael von Holstein, C-A.S. and Matheson, J.E., 1979 A Manual for Encoding Probability Distributions, Menlo Park, CA: SRI International. Steinbruner, J.D., 1974 The Cybernetic Theory of Decision, Princeton University Press. Stigler, G.J., 1950 "The development of utility theory," Journal of Political Economy, 58, 307-27, 373"9^. Sugden, R. and Williams, A., 1978 Practical Cost-Benefit Analysis, Oxford University Press. Tinker, T. and Lowe, T., 1984 "One-dimensional management science: the making of a technocratic consciousness," Interfaces, 14 (2), 40-56. Tocher, K.D., 1977 "Planning systems," Philosophical Transactions of the Royal Society of London, A287, 425-41. Tomlinson, R. and Kiss, I., eds., 1984 Rethinking the Process of Operational Research and Systems Analysis, Oxford: Pergamon Press. Tversky, A. and Kahneman, D., 1971 "The belief in the 'law of small numbers'," Psychological Bulletin, 76, 105-10. 1973 "Availability: a heuristic for judging frequency and probability," Cognitive Psychology, 5, 207-32. 1 974 "Judgment under uncertainty: heuristics and biases," Science, 185, 1124-31.
1980 "Causal schemas in judgments under uncertainty," in Fishbein, M., ed., Progress in Social Psychology, Hillsdale, NJ: Erlbaum. 1981 "The framing of decisions and the psychology of choice," Science, 211, 453-8. Ulvila, J.W. and Brown, R.V., 1982 "Decision analysis comes of age," Harvard Business Review, 60, 130-41. Ulvila, J.W. and McDonagh, M., 1978 United States—Philippine Military Base Negotiations, Case study, Cambridge, MA: Harvard Business School. Ulvila, J.W. and Snider, W.M., 1980 "Negotiation of tanker standards: an application of multi-attribute value theory," Operations Research, 28, 81-95. Ungson, G.R. and Braunstein, D.N., eds., 1982 Decision Making: An Interdisciplinary Enquiry, Boston: Kent. US National Research Council, 1983 Research Needs for Human Factors, Washington, DC: National Academy Press. Van Gundy, A.B., 1981 Techniques of Structured Problem Solving, New York: Van Nostrand Reinhold Company. Vari, A. and Vecsenyi, J., 1984 "Designing decision support methods in organisations," in Borcherding, K., Brehmer, B., Vlek, C , and Wagenaar, W.A., eds., Research Perspectives on Decision Making Under Uncertainty, Amsterdam: North Holland. Von Neumann, J. and Morgenstern, O., 1944 The Theory of Games and Economic Behavior, Princeton University Press.
294
References
Von Winterfeldt, D. and Edwards, W., 1986 Decision Analysis and Behavioral Research, Cambridge University Press. Waddington, C.H., 1973 OR in World War 2, London: Elek Science. Wallsten, T.S., ed., 1980 Cognitive Processes in Choice and Decision Behavior, Hillsdale, NJ: Erlbaum. Wallsten, T.S. and Budescu, D.V., 1983 "Encoding subjective probabilities: a psychological and psychometric review," Management Science, zy, 151—73. Watson, S.R., 1981 "Decision analysis as a replacement for cost/benefit analysis," European journal of Operational Research, 7, 242—8. 1985 Representing Value judgements in the Evaluation of Management Options for Nuclear Waste, Technical report, Cambridge Decision Analysts. Watson, S.R. and Brown, R.V., 1978 "The valuation of decision analysis," journal of the Royal Statistical Society, 141 A, 69-78. Watson, S.R., Weiss, J.J., and Donnell, M.L., 1979 "Fuzzy decision analysis," IEEE Transactions on Systems, Man, and Cybernetics, SMC-y, 1—9. Weatherford, R., 1982 Philosophical Foundations of Probability Theory, London: Routledge and Kegan Paul. Weber, M., 1947 The Theory of Social and Economic Organization, translated by A.M. Henderson and T. Parsons, Oxford University Press. Wiggins, N. and Kohen, E.S., 1971 "Man versus model of man revisited: the forecasting of graduate school success," journal of Personality and Social Psychology, 19, 100-6. Winkler, R.L., 1982 "Research directions in decision making under uncertainty," Decision Sciences, 13, 517—33. Winkler, R.L. and Murphy, A.M., 1973 "Experiments in the laboratory and the real world," Organizational Behavior and Human Performance, 10, 252-70. Wright, G.N., 1984 Behavioral Decision Theory, Harmondsworth: Penguin. Wrightson, M.T., 1976 "The documentary coding method," in Axelrod, R., ed., Structure of Decision, Princeton University Press. Zadeh, L.A., 1983 "The role of fuzzy logic in the management of uncertainty in expert systems," Fuzzy Sets and Systems, 11, 199—227. Zeleny, M., 1982 Multiple Criteria Decision Making, New York: McGraw-Hill.
INDEX
Ackoff, R. L., 14 Adelman, L., 160 air pollution, 228 Albrow, M., 102 Alemi, F., 112 Allais's paradox, 51, 90 Allison, G. T., 105 Alpert, M., 86 analytic hierarchy process, 76, 191 axiomatization, 77 analytic strategies, 3, 123 analytic tactics, 3, 163 anchoring heuristic, 86 Arrow, K. J., 108 Arrow's impossibility theorem, 108 audit trail, 161 availability heuristic, 86 Barclay, S., 117, 150, 160 Barron, F. H., 21 base-rate fallacy, 85 Baumol, W. J., 40 Bayes's theorem, 83 Beach, L. R., 83 Behn, R. D., 5 belief functions, 31 Bell, D. E., 65, 67, 276 Belton, V., 78 Bennett, P. G., 111 Bentham, J., 20 Bernoulli, D., 19, 36, 37 Bertier, P., 74 biases, cognitive, 34 Bodily, S. E., 5, 168, 172, 189 Boothroyd, H., 15 bootstrapping, 96 Borel, E., 109 bounded rationality, 103 Boyd,D.W., 183 Brams, S. J., 111 Braunstein, D. N., 102 Brier scoring rule, 89
Brock, H. W., 109 Brown, R. V., 5, 53, 182, 183, 279 Buchanan, B. G., 31 Budescu, D. V., 185 Buede, D. ML, 125, 151, 152, 153, 154, i55> 157. *79i 2I 5> *3^, 243, M^, 260
Buehring, W. A., 137 Bunn, D. W., 5 bureaucracy, 102, 103 calibration, 87 improvement, 89 cardiac catheterization, 263 Carnap, R., 30 causality, 169 certainty equivalence, 45 chance node, 3 6 Checkland, P., 15, 16 Chen, K., 116 child welfare, 146 Choisser, R. W., 215, 236, 243 clairvoyance, 55, 129 Clough, D.J., 5, 103 cognitive limitations, 87, 99 cognitive mapping, 16, 176 Cohen, L. J., 32 Cohen, M. D., 104, 275 coherence, 84 collective choice, 106 Condorcet, Marquis de, 107 CONFERENCING, 125, 148, 150, 151 conflicting objectives, 18 and risk preferences, 62 conjoint measurement, 21, 160 COPE, 178 Corrigan, B., 96, 97 cost—benefit analysis, 69 enthusiasm declining, 115 Crawford, D. M., 143, 144 Dawes, R. M., 96, 97, 99
z96
Index
Dawid, A. P., 88 DeBono, E., 166 De Finetti, B., 30, 33 DeGroot, A., 180 De Groot, M. H., 33 De Montgolfier, J., 74 Dean, B. V., 160 decision analysis, 20 criticisms of, 5 2 in organizations, 101, i n iterative, 125 value of, 183 validation of, 271 ethics of, 279 decision analysis cycle, 127 decision conferencing, 113 decision node, 3 6 decision support system, 119, 126, 155 decision synthesis, 3, 98, 106, 156, 279 appraised, 271 decision theory imperialist view, 69 normative, 82 descriptive, 82 applied to organizations, 106 decision tree, 3 6 folding back, 44 coalescence, 170 decision-making proper subject of study, 2 improvement, 2 rational, 9 difficulties, 9 process, 9 bad, 10, 82 good, 10, 82 psychology of, 81 lexicographic, 96 public, 117 and computers, 119 Decisions and Designs Inc., 125, 151, 157 delta property, 46 Dempster, A. P., 31 DEVELOPING, 125, 155, 157, 159 Diesing, P., 11 dominance, 23 Douglas, J., 263 Dreyfus, S. E., 53 Dyer, J. S., 78, 215, 216, 217, 256 Ebbesen, E. B., 98 Eden, C , 16, 176 Edwards, W., 5, 21, 81, 83, 87, 98, 116, 125, 145, 146, 147, 182 efficient frontier, 23, 24, 117 Einhorn, H. J., 81,98 ELECTRE computer programs, 75
Ellsberg, D., 20, 21, 90 Elster, J., 12 expected utility, failure as a descriptive theory, 90 factory location, 151 Farquhar, P. H., 194, 276 Fine, T. L., 30 Fischhoff, B., 70, 86, 87, 89, 96, 98, 280, 281
Fishburn, P. C , 21, 27, 194, 276 Fraser, N. M., 111 free will, 1 Freeling, A. N. S., 31 Fryback, D. G., 215, 226, 232 fuzzy set theory, 31 Gaines, B. R., 31 game theory, 109 Gardiner, P. C., 145 Gear, A. E., 78 Geoffrion, A. M., 14 goal directed bias, 195 Goldberg, L. R., 96 group decision-making, normative theories, 106 group preferences, 109 group utility function, 109 groupthink, 105, 149 Guttentag, M., 146 Hacking, I., 30 Hall, R. H., 105 Hammond, K. R., 72, 73, 99 Harsanyi, J. C., 109 Hershey, J. C., 191 Hertz, D. B., 215, 216 heuristics and biases, 84 hindsight bias, 86 Hipel, K. W., i n Hogarth, R. M., 81, 96, 98, 99 Holsti, O. R., 178 Hoos, I. R., 14 Howard, N., 111 Howard, R. A., 1, 5, 24, 66, 116, 125, 127, 130, 168, 184, 187, 215, 247, 264 Huntzinger, B. C , 143 hurricane seeding, 247 Hurst, E.G., 156 hypergame theory, 111 hypothetical gamble, 41 incoherence, resolution of, 34 incrementalism, 104 influence diagrams, 168, 256 as analysis tools, 174
Index information value of, 53 value of perfect, 55, 241 value of imperfect, 55, 59 insurance, 47 interval scale, 27 INTROSPECTING, 125, 135, 137, 143 iterative analysis, 181, 236, 246 Janis, I. L., 105, 106 Jannsson, D. S., 104 Jones-Lee, M., 264 judgment psychological research, 95 biases in, 96 of livestock, 97 research in, 97 in a criminal court, 98 Kahneman, D., 84, 85, 87, 185 Keefer, D. L., 160, 189 Keelin, T. W., 67 Keen, P. G. W., 126, 155 Keeney, R. L., 5, 21, 24, 26, 27, 63, 64, 65, 66, 109, 113, 125, 135, 138, 139, 140,
141,
142,
191,
215,
226,
232,
*75 Keeney's theorem, 63 Keller, L. R., 194 Kelly, G. A., 177 Kepner, C. H., 164, 165 Keynes,J. M., 30 King,J. L., 114, 126 Kirkwood, C. W., 65, 143, 215, 266 Kiss, I., 14 Kohen, E. S., 96 Konecni, V. J., 98 Kraemer, K. L., 114, 126 Krantz, D. H., 21, 27, 160, 261 Kunreuther, H. C , 191 Kuskey, K. P., 125, 157, 202, 216, 256, 260
Kyburg, H. E., 33 labile values, 98 Laplace, P., 30 Las Vegas gamblers, 93 lateral thinking, 166 Layard, R., 70 Lee, W., 5 Lichtenstein, S., 83, 88, 93 Lindblom, C. E., 104 Lindley, D. V., 5, 29, 33, 34, 69, 183 linear additive model, equal weights, 97 linear programming, 71 Linstone, H. A., 15, 183 log-log odds, 186, 187
297
log odds, 186 Lowe, T., 14 Luce, R. D., 5, 21, 109 Ludke, R. L., 185, 186, 188, 198 Lund, R. N., 215, 256 Macartney, F., 215, 263 McDonagh, M., 118 McGrew, A. J., 105 Madey, G. R., 160 magnetic fusion choices, 252 Majone, G., 14 majority voting, 107 Mamdani, E. H., 31 Mann, L., 105 March, J. G., 11 Marshall, A., 20 Matheson, J. E., 5, 66, 132, 168, 186, 247 Meehl, P. E., 97 Merkhofer, M. W., 168, 216, 226 metagame theory, 111 Mexican electrical system, 132 Miles, D., 215, 216, 217 Mill, J. S., 20 Miller, A. C., 168 , 189 Mintzberg, H., 102 MODELING, 125, 127, 132 money pump, 69 Moore, P. G., 5 Morgenstern, O., 20, 37, 49, 50, 109 Morris, P. A., 133 Morris, W. T., 5 multi-attribute utility theory Keeney's approach, 63 criticisms of, 65 the Stanford approach, 65 multi-attribute value analysis, 244 multi-criterion decision-making, as an academic movement, 70 multi-criterion problem, choice of approach, 78 Murphy, A. H., 83, 88, 89, 186 mutual preference independence, 26 MYCIN, 31 n-person game, n o Nash bargaining model, 225 negotiations, 117 Nickerson, R. C., 183 North, D. W., 216, 226, 247, 252 nuclear power plant site selection, 266 nuclear waste management, 115 O'Connor, M. F., 126 operational research, 13 criticisms of, 14, 15 organizational theory, 101
Index outranking, 73 criticisms of, 75 Owen, D. L., 168, 170, 172 Panama Canal negotiations, Ulvila, J. W., 118 Pareto, V., 20 Pareto-optimality, 24, 160 Pascal, B., 29 Pearson, E. S., 189 perceptions, modeling, 16 personal constructs, 177 Peterson, C. R., 83, 117, 125, 150, 151, 160, 164, 167 Phelps, R. H., 97 Phillips, L. D., 63, 83, 87, 114, 180, 273 Pitz,G.R, 81,95,98 Pliskin, J. S., 264 plural analysis, 182 policy region analysis, 210 political science, 105 preference independence, 26, 64 preference structure, 23 pricing out, 24, 66 prisoner's dilemma, 109 probability meaning of, 30 alternatives to, 31 upper and lower, 31 axioms, 32 axiomatization of subjective, 39 conditional, 58 overspecification, 59 human abilities in assessing, 82 conservatism in assessment, 83 heuristics and biases in assessment, 84 elicitation, 184, 247 elicitation of discrete, 189 probability theory, 29, 83 relative frequency theory, 30 subjective theory, 30 probability wheel, 188 Program Objectives Memorandum, 157, 260 programmed decisions, 103, 112 public administration, 101 public policy analysis, 115 Pugh, D. S., 102 Quade, E. S., 14 Quinn, R. E., 105 Raiffa, H., 5, 21, 24, 26, 27, 29, 52, 63, 64, 65, 66, 86, 109, i n , 117, 125, 135, 160, 191 Ramsey, F. P., 30, 33 RAND corporation, 14
RATING, 125, 145, 146 rationality meaning, 11 as consistency, 12 scientific, 13 Rawls,J., 108 Reichenbach, H., 30 relevance systems, 166 representativeness heuristic, 8 5 requisite decision modeling, 180, 273 research directions, 274 the foundations of decision theory, 276 structuring, 276 assessing probabilities, 277 assessing values, 278 determining the type of analysis, 278 methods for communicating analyses, 279 Rice, T. R., 189 risk aversion, 45 coefficient of, 47 risk neutral, 46 risk preferences, 21, 35 risk premium, 45 risk seeking, 46 Ross, S. M., 35 Roy, B., 73, 74 Saaty, T. L., j6, 77 Sachs, N.J., 81,98 Sage, A. P., 12, 104, 119 Sassone, P. G., 70 Savage, L.J., 30, 33, 39 Schaffer, W. A., 70, Scharlig, A., 74 Schlaifer, R., 5 Schoemaker, P. J. H., 79, 90, 95, 190, 191 Schwenk, C. R., 183 scoring rules, 34 proper, 186 Scott Morton, M. S., 126, 155 Seaver, D. A., 185, 188 Sen, A. K., 108 sensitivity analysis, 209, 263 Shacter, R. D., 168, 174 Shafer, G., 31 Shanteau, J., 97 Shortliffe, E. H., 31 Simon, H. A., 11, 103 Sims, D., 176 Slovic, P., 81, 83, 92, 93 Smallwood, R. D., 133 SMART, 145, 165 Smokier, H. E., 33 Snapper, K., 146 Snider, W. M., 117, 118, 151 social decisions, 116
Index social judgment theory, 72, 190 social psychology, 101, 105 Spetzler, C. S., 136, 184, 215, 236 Spiegelhalter, D., 263 SRI, 132, 188 St Petersburg paradox, 19, 36 Stael von Holstein, C. A., 136, 184, 185 Steinbruner, J. D., 12 Stengel, D. N., 216, 247, 252 Stigler, G. J., 20, 21 stochastic dominance, 210 structuring problem-, 163, 216 model-, 163, 164, 168, 226 top down, 164, 179 bottom up, 164, 179 substitution principle, 3 8 Sugden, R., 70 swing weights, 146 systems analysis, 13 soft, 16 tanker safety, 150 Taylor, S. H., 104 Thomas, H., 5, 183, 215, 216 Tinker, T., 14 Tocher, K.D., 52, 53 Tomlinson, R., 14 transmission conductor selection, 143 trauma severity index, 23 2 Tregoe, B. B., 164, 165 Tukey,J.W., 21, 189 Tversky, A., 84, 85, 87, 92, 93, 185 Ulvila,J. W., 53, 117, 150, 151, 279 uncertainty, 28 human abilities in assessing, 82 Ungson, G. R., 102 utility theory, 19, 37 functions, 19 cardinal 20, 21 ordinal, 20, 21 expected, 37 maximizing expected, 39 axiomatization of, 39 convex, 45 linear, 46 exponential, 46 for non-monetary variables, 49 criticisms, 50 multi-attribute, 63 interpersonal comparisons, 108 utility assessment, 135, 137, 204
299
utility independence, 64 value assessment, 256 value functions, 21 definition, 22, 24 additive, 25, 193 non-additive, 26 multiplicative, 27 descriptive power, 96 elicitation, 190 midpoint method for elicitation, 194 value difference method for elicitation, 198 balance-beam method for elicitation, 199 hierarchies, 203 value preferences, 21, 26 Van Gundy, A. B., 164, 165 Vari, A., 119
Vaupel,J.W., 5
Vecsenyi, J., 119 Venn, J., 30 Von Mises, R., 30 Von Neumann, J., 20, 21, 37, 49, 50, 109 Von Winterfeldt, D., 5 Waddington, C. H., 13 Waid, C. C , 79, 190 Wallsten, T. S., 95, 185 Waslov, K. A., 157, 256, 260 Watson, S. R., 31, 70, 115, 183, 281 weather forecasting, 88 Weatherford, R., 30, 31 Weber, M., 102, 103 weight assessment test of methods, 79, 200 in a hierarchy, 203 Wendell, R. E., 78 Wiggins, N., 96 Williams, A., 70 Wilson, M. J., 105 Winkler, R. L., 83, 88, 89, 186, 275, 277 Wisconsin electrical energy production, 137 Wright, G. N., 5 Wrightson, M. T., 178 Xerox Corporation, 133 Zadeh, L. A., 31 Zamora, R. M., 215, 236 Zeleny, M., 71 zero-sum game, n o
