HANDBOOKS IN INFORMATION SYSTEMS VOLUME 1
Handbooks in Information Systems Advisory Editors
Ba, Sulin
Editor
Andrew B. Whinston
University of Connecticut
Duan, Wenjing The George Washington University
Geng, Xianjun University of Washington
Gupta, Alok
Volume 1
University of Minnesota
Hendershott, Terry University of California at Berkeley
Rao, H.R. SUNY at Buffalo
Santanam, Raghu T. Arizona State University
Zhang, Han Georgia Institute of Technology
Amsterdam – Boston – Heidelberg – London – New York – Oxford – Paris – San Diego San Francisco – Singapore – Sydney – Tokyo
Economics and Information Systems
Edited by
T. Hendershott University of California
Amsterdam – Boston – Heidelberg – London – New York – Oxford – Paris – San Diego San Francisco – Singapore – Sydney – Tokyo
Elsevier Radarweg 29, PO Box 211, 1000 AE Amsterdam, The Netherlands The Boulevard, Langford Lane, Kidlington, Oxford OX5 1GB, UK
First edition 2006 Copyright r 2006 Elsevier B.V. All rights reserved No part of this publication may be reproduced, stored in a retrieval system or transmitted in any form or by any means electronic, mechanical, photocopying, recording or otherwise without the prior written permission of the publisher Permissions may be sought directly from Elsevier’s Science & Technology Rights Department in Oxford, UK: phone (+44) (0) 1865 843830; fax (+44) (0) 1865 853333; email:
[email protected]. Alternatively, you can submit your request online by visiting the Elsevier web site at http://elsevier.com/locate/permissions, and selecting Obtaining permission to use Elsevier material Notice No responsibility is assumed by the publisher for any injury and/or damage to persons or property as a matter of products liability, negligence or otherwise, or from any use or operation of any methods, products, instructions or ideas contained in the material herein. Because of rapid advances in the medical sciences, in particular, independent verification of diagnoses and drug dosages should be made Library of Congress Cataloging-in-Publication Data A catalog record for this book is available from the Library of Congress British Library Cataloguing in Publication Data A catalogue record for this book is available from the British Library ISBN-13: 978-0-444-51771-5 (this volume) ISBN-10: 0-444-51771-5 (this volume) ISSN: 1574-0145 (series)
For information on all Elsevier publications visit our website at books.elsevier.com
Printed and bound in The Netherlands 06 07 08 09 10
10 9 8 7 6 5 4 3 2 1
Contents Introduction
xv
CHAPTER 1 Diffusion of Information and Communication Technologies to Businesses C. Forman and A. Goldfarb
1
1. 2.
Introduction ICT diffusion and its impact
1 4
2.1. 2.2.
4 8
3.
ICT adoption and organizational characteristics 3.1. 3.2. 3.3. 3.4. 3.5.
4.
5. 6.
Diffusion modeling The impact of ICT diffusion Adoption, internal firm organization, and organizational change Adoption and firm boundaries Adoption and size Technical infrastructure Adoption, assimilation, intra-firm diffusion, and usage
9 9 14 17 18 19
Geographic differences in adoption
21
4.1. 4.2. 4.3.
23 24 24
Adoption of ICT across urban and rural areas Evidence of how ICT use influences location patterns Future research
Trade-offs between organization and environment
25
5.1. 5.2.
26 28
Co-invention Strategic issues in technology adoption
Network effects
30
6.1. 6.2. 6.3.
32 33 35
Theoretical literature on direct and indirect network externalities Evidence of network effects of any kind in ICT Evidence of positive network externalities in ICT
7. Internet diffusion across countries 8. Conclusion Acknowledgments References
37 42 43 43
CHAPTER 2 Economics of Data Communications P. Afe`che
53
1. 2.
Introduction Problem definition and discussion framework
53 56
2.1. 2.2..
57 59
Service demand Service contracts: QoS guarantees and service designs
v
vi
Contents 2.3. 2.4. 2.5. 2.6.
3.
4.
6.
7.
63 67 67 71
Pricing guaranteed services
73
3.1. 3.2. 3.3. 3.4. 3.5. 3.6.
73 75 78 78 80 81
Features, modeling and overview Basic pricing and allocation principles Incomplete information and adaptive pricing More differentiation Optimal dynamic pricing Conclusions and directions
Pricing best effort services 4.1. 4.2. 4.3. 4.4. 4.5. 4.6.
5.
Service production Distinctive features in comparison to telephony Pricing decisions Discussion framework
Features, modeling and overview Basic pricing and allocation principles Incomplete information and adaptive pricing More differentiation Optimal dynamic pricing Conclusions and directions
83 83 89 95 100 105 105
Pricing flexible bandwidth-sharing services
106
5.1. 5.2. 5.3. 5.4. 5.5. 5.6.
106 109 111 115 115 117
Features, modeling and overview Basic pricing and allocation principles Incomplete information and adaptive pricing More differentiation Optimal dynamic pricing Conclusions and directions
Discussion
118
6.1. 6.2. 6.3.
118 124 126
Auctions versus posted prices Flat-rate versus usage-based pricing Providing QoS: overprovisioning versus control
Conclusions
128
7.1. 7.2.
128 129
Research directions Further topics and readings
References
129
CHAPTER 3 Firms and Networks in Two-Sided Markets D. F. Spulber
137
1. 2.
Introduction Firms in two-sided markets
137 140
2.1. 2.2. 2.3.
142 143 147
3. 4.
Firms in two-sided markets: matchmaking Firms in two-sided markets: market making Information systems and platforms
Networks in two-sided markets
151
3.1. 3.2.
151 154
Transactions on networks in two-sided markets Basics of network theory
Assignments of buyers and sellers in a network: costly communication
156
Contents 4.1. 4.2. 4.3.
5. 6.
7. 8.
157 161 165
Networks and the core in a two-sided market
166
5.1. 5.2.
166 167
The core with homogenous products The core with differentiated products
Stable assignments in a decentralized two-sided market: costly computation
168
6.1. 6.2.
169 172
Stable assignments with homogenous products Stable assignments with differentiated products
Firms and stable assignments in a centralized two-sided market
173
7.1. 7.2.
174 179
Firms and stable assignments with homogenous products Firms and stable assignments with differentiated products
Matchmaking and market making by a firm using double auctions 8.1. 8.2.
9.
Assignments with homogenous products Assignments with differentiated products Second-best assignments
vii
Market making by a firm using double auctions for homogenous products Matchmaking and market making by a firm using double auctions for differentiated products
181 182 183
Two-sided markets in random networks
188
9.1. 9.2.
188 193
Search and random assignments Markets and random networks
10. Conclusion Acknowledgments References
196 197 197
CHAPTER 4 Organization Structure T. Marschak
201
1. 2.
Introduction Goals, mechanisms, and informational costs: the ‘‘incentive-free’’ case, where individuals obey the designer’s rules without inducement 2.1. 2.2. 2.3. 2.4. 2.5. 2.6.
2.7.
3.
Two general frameworks for judging the organization’s actions How the organization finds its current action when incentives are not an issue Finite approximations of mechanisms whose message spaces are continua The dynamics of a mechanism Constructing an informationally efficient mechanism Finding a best action rule (outcome function) once a mechanism has conveyed information about the environment to each person: the methods of the Theory of Teams Designing an organization "from scratch": choosing its members, what each observes, and the speak-once-only mechanism that they use
201 204 204 207 230 233 236
238 240
Models in which the designer is concerned with incentives as well as Informational costs
264
3.1.
265
The message-space size required for implementation of a goal
viii
Contents 3.2. 3.3.
Models in which the organization’s mechanism is partly designed by its self-interested members, who bear some of the informational costs Networks of self-interested decision-makers, who bear the network’s informational costs
Organizational models in which the primitive is a ‘‘task’’, ‘‘problem’’, ‘‘project’’, or ‘‘item’’ 5. Concluding remarks References
266 275
4.
278 280 281
CHAPTER 5 Open Source Software: The New Intellectual Property Paradigm S. M. Maurer and S. Scotchmer 285 1. 2.
3.
4.
5. 6.
Introduction Incentives for R&D
285 287
2.1. 2.2. 2.3. 2.4. 2.5. 2.6. 2.7.
288 290 290 293 295 295 296
Intellectual property and open source Own use Complementary goods and services Signaling Education Achieving network externalities and denying them to others Social psychology
Stability and organizational issues
300
3.1. 3.2. 3.3. 3.4. 3.5.
300 301 302 304 305
Who contributes, and how much? Who pays? Why licenses? Why leadership? Network effects
Efficiency implications
306
4.1. 4.2. 4.3. 4.4. 4.5. 4.6.
306 307 308 308 310 310
Disclosure of code Meeting users’ needs Deadweight loss and pricing Training and using programmers Free riding Modularity and the organization of the research effort
Open source and proprietary software
312
5.1. 5.2.
312 314
Competition between open source and proprietary software Market segmentation
Limitations and extensions
315
6.1. 6.2.
315
Limits to open source software Beyond software: drug discovery, geographic information systems, and Wikipedia
7. Conclusion Acknowledgments References
315
318 319 319
Contents
ix
CHAPTER 6 Information, Search, and Price Dispersion M. R. Baye, J. Morgan, and P. Scholten
323
1. 2.
Introduction Theoretical models of price dispersion
323 331
2.1.
332 333 336 338 343 343 346 348 351 352 354 357 357 358 359
2.2.
2.3. 2.4.
3.
Search-theoretic models of price dispersion 2.1.1. The Stigler model 2.1.2. The Rothschild critique and Diamond’s paradox 2.1.3. The Reinganum model and optimal sequential search 2.1.4. Remarks on fixed versus sequential search 2.1.5. The MacMinn model 2.1.6. The Burdett and Judd model Models with an ‘‘Information Clearinghouse’’ 2.2.1. The Rosenthal model 2.2.2. The Varian model 2.2.3. The Baye and Morgan model 2.2.4. Models with asymmetric consumers 2.5.5. Cost heterogeneities and the Spulber model Bounded rationality models of price dispersion Concluding remarks: theory
Empirical analysis of price dispersion
360
3.1. 3.2.
360 363 363 365 367 369 370
3.3.
Measuring price dispersion Price dispersion in the field 3.2.1. Dispersion and the ‘‘benefits’’ of search 3.2.2. Dispersion and the ‘‘cost’’ of search 3.2.3. Dispersion and the number of sellers 3.2.4. Dispersion and price persistence Concluding remarks: empirics
Acknowledgments References
371 371
CHAPTER 7 Behavior-Based Price Discrimination and Customer Recognition D. Fudenberg and J. M. Villas-Boas 377 1. 2.
Introduction Monopoly
377 379
2.1.
380 380 382 382 383 384 386 387 389
2.2.
2.3.
Two-period model Base model No customer recognition Customer recognition and behavior-based price discrimination The role of commitment Overlapping generations of consumers No constant prices in equilibrium Price cycles in equilibrium Long-lived consumers
x
Contents
2.4.
3.
398
3.1.
399 400 403 404
3.3. 3.4.
5.
391 393 396
Competition
3.2.
4.
Long-term contracts Relationship to durable goods and bargaining Two-good monopoly Two periods, short-term contracts Analysis of the two-period model under the MHR assumption Discrete distributions Welfare Infinite lived firms, overlapping generations of consumers, and short-term contracts Long-term contracts Switching costs
404 406 408
Behavior-based pricing with multiple products, and product design
413
4.1. 4.2. 4.3. 4.4.
414 417 418 420
Upgrades and buybacks with an anonymous second-handmarket Upgrades and buybacks with non-anonymous consumers Endogenous innovation Endogenous location choice in duopoly
Related topics: privacy, credit markets, and customized pricing
422
5.1. 5.2. 5.3.
422 426 429
Privacy Credit markets Customized pricing
6. Conclusion Acknowledgements References
431 433 433
CHAPTER 8 Information Technology and Switching Costs Pei-yu Chen and L. M. Hitt
437
1. 2. 3.
Introduction Switching cost: definition and measurement issues Switching costs, competition, and firm strategy
438 440 444
3.1. 3.2. 3.3. 3.4.
444 444 445 448 449 450
4. 5.
Switching costs and competition Endogenous switching costs Switching costs in information-intensive markets Empirical evidence Switching costs in software and other ‘‘high-tech’’ markets Switching costs in online markets
Endogenous switching costs and firm strategy in information-intensive markets A framework for managing switching costs
451 455
5.1. 5.2. 5.3.
455 457 461
Introduction A model of customer retention Measuring switching costs
6. Conclusion Acknowledgments References
464 466 467
Contents
xi
CHAPTER 9 The Economics of Privacy K.-L. Hui and I.P.L. Png
471
1. 2. 3.
Introduction ‘‘Free market’’ approach Within-market consequential externalities
471 475 476
3.1. 3.2.
477 480
Non-productive information Productive information
4. Cross-market consequential externalities 5. Direct externalities 6. Property rights 7. Regulation 8. Empirical evidence 9. Future directions Acknowledgment References
481 483 485 487 489 492 493 493
CHAPTER 10 Product Bundling X. Geng, M. B. Stinchcombe, and A. B. Whinston
499
1. 2.
Introduction Bundling for price discrimination: the case of two products
499 502
2.1. 2.2.
503 503 504 504 504 505 505 508 510
2.3. 2.4. 2.5.
3. 4.
The base model Issues to be considered in monopoly bundling 2.2.1. Pure bundling, mixed bundling, and the combinatorial issue 2.2.2. Bundling products and bundling buyers 2.2.3. Monitoring purchase 2.2.4. Posted price and auction Pure bundling Mixed bundling Extension: bundling complements or substitutes
Bundling for price discrimination: the case of many products
510
3.1. 3.2.
512 513
The integer programming approach Pure or simple mixed bundling of many products
Bundling as competition tools
515
4.1. 4.2.
517
4.3. 4.4.
A monopoly market plus a competitive market A monopoly market plus a potentially duopoly market: the additive valuations case A monopoly market plus a potentially duopoly market: the complements case Duopoly bundling
5. Concluding remarks Acknowledgment
517 519 520
522 523
xii
Contents
CHAPTER 11 Dynamic Pricing in the Airline Industry R. P. McAfee and Vera L. te Velde
527
1. Airline pricing 2. Existing literature 3. Dynamic price discrimination with price commitment 4. Continuous time theory 5. Efficiency in the Gallego and van Ryzin model 6. Efficiently allocating limited capacity under uncertainty 7. The log normal case 8. Options and interruptible sales 9. Actual airline pricing patterns 10. Research projects and mysteries 11. Conclusion Appendix References
529 529 534 536 541 541 546 548 551 556 561 561 567
CHAPTER 12 Online Auctions A. Ockenfels, D. H. Reiley, and A. Sadrieh
571
1. 2.
Why do information systems make auctions (even) more popular? Single-object auctions: theory and experiments
571 573
2.1. 2.2. 2.3.
573 575 577
3.
4. 5.
Reserve prices, minimum bids, and shill bids
582
3.1. 3.2.
584 587 587 590 591
Theoretical considerations Empirical and experimental observations 3.2.1. Entry and revenue effects of public and secret reserve prices 3.2.2. Auction fever 3.2.3. Shill bids
Late and incremental bidding The buy-now option
594 599
5.1. 5.2.
600
5.3. 5.4.
6. 7.
Standard auction mechanisms and models Bidding behavior and auction outcomes in theory Bidding behavior in controlled laboratory and field experiments
Explaining the buy-now option with risk-aversion Explaining the buy-now option with impatience and other transaction costs Explaining the buy-now option with a sequence of transaction opportunities Empirical and experimental evidence
601 602 603
Parallel markets and other outside options Multi-item auctions
605 608
7.1. 7.2. 7.3.
608 610 612
Standard multi-unit auction mechanisms Bid shading and demand reduction in multi-unit auctions Complementarities and combinatorial auctions
Contents 8.
xiii
Design of online auctions
614
8.1. 8.2. 8.3.
614 616 620
The advantages of long, open auctions Controlling the pace of bidding Design aspects in multi-unit auctions
Acknowledgements References
621 622
CHAPTER 13 Reputation Mechanisms C. Dellarocas
629
1. 2. 3.
Introduction Signaling and sanctioning role of reputation mechanisms Reputation in game theory and economics
629 633 635
3.1. 3.2. 3.3. 3.4.
635 638 640 642
4.
5.
Basic concepts Reputation dynamics When is reputation bad? Other extensions to the basic theory
New opportunities and challenges of online mechanisms
643
4.1. 4.2. 4.3. 4.4. 4.5.
643 645 647 648 649
Eliciting sufficient and honest feedback Exploring the design space of feedback mediators Coping with cheap online identities Understanding the consequences of strategic manipulation Distributed reputation mechanisms
Empirical and experimental studies
651
5.1. 5.2.
651 654
Empirical studies and field experiments Controlled experiments
6. Conclusions: opportunities for IS research References
655 657
Subject Index
661
This page intentionally left blank
INTRODUCTION
Technological advances in processing and communicating information have begun what may be the largest transformation of the economy since the industrial revolution. The dramatic improvements in information technology (IT) and information systems (IS) decrease the cost of activities and makes activities previously too expensive or cumbersome attractive.1 As IT adoption proliferates both demand- and supply-side externalities increase, making the technology cheaper and more valuable to additional users, spurring further adoption and development of new applications. This virtuous circle has enabled the dramatic growth of applications and businesses. The economic models upon which many of these businesses are based on are old, e.g., auctions trace back to the Egyptians, but the ability to easily connect vast parts of the population to businesses and other users anywhere in the world makes raise new issues in new settings and for far wider ranges of applications. At the beginning of the Internet boom, the popular press promulgated the view—often the view of firms either supplying the technology, using the technology, or funding the firms using and supplying the technology—that the Internet would cause everything to be different; with the primary difference being that competition between firms would be enhanced by creating frictionless markets and that information would be free.2 One of the first and best attempts to deflate this hype was Information Rules by Carl Shapiro and Hal Varian. This book succeeded in bringing to bear in lucid text some of the academic literature on price discrimination and damaged goods, customer switching costs and lock in, and network externalities and the associated compatibility issues and standard wars. It is my hope that this volume will help further our scholarly understanding on these and other topics, which will be discussed further. 1 The distinction between IS and IT is subtle and much debated, with IT often defined as actual information technology and IS as its implementation in complex activities and systems of activities. This volume focuses more on the economics of the IS uses, but these closely depend on the details of the economics of the underlying IT. See Footnote 3 in Dan Spulber’s chapter for a more detailed industry definition of information systems. 2 Table 1 in Dan Spulber’s chapter in this volume provides an excellent summary of how reducing communication and computation costs can affect the way buyers and sellers interact.
xv
xvi
Introduction
Information has risen to prominence in economics, for example, the 2001 Nobel Prize or Fig. 1 in Baye, Morgan, and Scholten in this volume. However, economists often model technology in abstract ways. In contrast, IS researchers often focus on the details of the technology without fully exploring the implications of the systems’ capabilities for economic interaction. This volume attempts to bridge these approaches by highlighting areas where IT is changing the importance of different economic forces and illustrating the economics that lay behind the use of many types of information systems. While the areas of economics and other related disciplines impacted by the growth and improvements in IS are too numerous to catalogue and examine in a single volume, this book’s chapters survey many of the most significant issues and our progress on them. The chapters in this volume focus on various individual interrelated subjects regarding the economics of information systems: the adoption and diffusion of information technologies; the pricing of data communications; the ways firms organize and transform themselves as information is better captured and disseminated within and across firms and customers; the means and tactics firms use to compete with each other; the manner in which firms interact with and distribute goods to customers; the methods and mechanisms for anonymous and infrequent interactions between users, firms, and customers in far reaching locations; and the type and use of information on customers and their behavior. These issues span areas of economics and disciplines within business schools. The chapters that follow flesh these areas out in detail. With the chapters spanning such a broad topic from a variety of perspectives, there are many ways to structure and order the chapters. I choose simply to start with two chapters studying the aspects of the economics of the IT infrastructure: diffusion, adoption, and pricing of information and communication technologies. The next three chapters examine the impact IT is having on markets (networks of buyers and sellers), organization of firms, and methods of innovation. The remaining chapters focus on how IT is transforming firm competition through the frictions in competition, firms’ knowledge of their customers, and improvements in the mechanisms by which prices are adjusted and information is collected. In Diffusion of Information and Communication Technologies to Businesses, Chris Forman and Avi Goldfarb survey the literature on the adoption and diffusion of information and communication technologies with a focus on technologies that facilitate communication within and across firm boundaries. These technologies improve the monitoring and coordination capabilities of organizations. The chapter examines work on adoption costs and benefits and how these are influenced by firms’ internal organization and firms’ external environment. For the internal aspects, they examine how organizational characteristics, firm boundaries, internal decision-making rights, and individual incentives influence adoption. For the external
Introduction
xvii
factors, they examine firms’ geographic location—both rural versus urban and across countries, the role of network effects, and the interaction between the internal and external influences. In Economics of Data Communications, Philipp Afe`che surveys core aspects and roles of pricing data transport services with a focus on three fundamentally different types of service contracts: guaranteed services, congestion-prone best effort services, and adaptive bandwidth sharing. The discussion is organized according to a unifying framework that compares and contrasts the key features of service demand and delivery, develops the basic pricing principles for socially optimal allocation, studies issues of service differentiation and customer incentives, and considers iterative and dynamic price mechanisms. The chapter then examines the value of auctions for data transport services and closes with a discussion of the relative merits and challenges of alternative quality of service approaches. After the first two chapters on the economics of the adoption and pricing of information communications technology, the book moves to three chapters covering IS and markets, organizations, and innovation at high levels. In Networks and Two-Sided Markets, Daniel Spulber represents decentralized and centralized two-sided markets using network theory to shed light on decentralized mechanisms in which consumers transact directly and centralized mechanisms operated by one or more firms acting as intermediaries. After an excellent introduction—as with many of the chapters, I recommend readers to study the introductory sections for less technical background on the topics; this chapter’s introduction is particularly exceptional, the chapter explores the implications of costly communication for the design and efficiency of market mechanisms for allocations of homogeneous and differentiated goods in buyer–seller networks. Firms’ employ IS to improve communication between buyers and sellers and to improve computation through centralized market mechanisms. Firms charging subscribers for admission to a communications network provide centralized allocation mechanisms that can increase economic efficiency and reduce transaction costs relative to decentralized exchange. The chapter also presents how the network can be used via double auctions to link buyers and sellers. Finally, the chapter considers the connection between search models and random networks and compares the efficiency of buyer– seller search with centralized assignments. In Organizational Structure, Thomas Marschak examines formal models of organizations that regularly acquire information about a changing environment in order to optimize their actions. Each member of the organization privately learns about particular aspects of the new environment. The organization operates a mechanism to process this information and act on it. The mechanism has various informational and agency costs and balances these costs against the benefits in performance. As costs drop, due to improved IT, the properties of good mechanisms, and hence the structure of the organizations that adopt them change.
xviii
Introduction
In Open Source Software: The New Intellectual Property Paradigm, Stephen Maurer and Suzanne Scotchmer study how a new form of innovation and intellectual property is developing in information systems: open source methods for creating software which rely on voluntarily revelation of code. Open source incentives differ from other intellectual property paradigms, leading to different types of inefficiencies and different biases in R&D investment. Open source remedies a defect of intellectual property protection—the lack of requirements or encouragement to disclose source code. The lack of disclosure in proprietary innovation hampers interoperability in complex systems. The chapter analyzes developers’ incentive to participate in open source collaborations, studies when open source is most likely to predominate, and evaluates the extent to which open source may improve welfare compared to proprietary development. As noted above, the Internet was claimed to move us into a frictionless utopia where the ‘‘law of one price’’ would truly apply. In Information, Search, and Price Dispersion, Michael Baye, John Morgan, and Patrick Scholten establish that remarkably little progress has been made toward this idealistic state. The chapter provides a unified treatment of various search models that have been proposed to rationalize price dispersion in markets for homogeneous products: sequential search, fixed sample search, and clearinghouses. These models reveal that reduction or elimination of consumer search costs do not always reduce or eliminate price dispersion. The links the authors draw between the models is a significant contribution. The chapter further connects the search literature to mechanism design by showing how auction tools can simplify and even generalize existing results. The chapter concludes with an overview of the burgeoning empirical literature, which suggests that price dispersion in both online and offline markets is sizeable, pervasive, and persistent. Just as IT changes consumers’ ability to gather information about firms and their prices, information technologies and the Internet allow firms to keep, gather, and process more information about their past customers. This increase in information has led to the proliferation of customer relationship management practices in most industries. In Behavior-Based Price Discrimination and Customer Recognition, Drew Fudenberg and Miguel Villas-Boas examine how when firms have information about consumers’ previous purchases, they can use it to offer different prices and/or products to consumers with different purchase histories—practice ‘‘behavior-based price discrimination’’ (BBPD). Throughout the chapter firms’ commitment problem arises: although having more information helps extract more surplus with its current prices, consumers may anticipate this possibility and alter their initial purchases. A second theme is that more information may lead to more intense competition between firms, creating a potential prisoner’s dilemma where each firm would gain from practicing BBPD, but industry profits fall when all practice it. The chapter also surveys the literature on firm competition, short- and long-term contracts
Introduction
xix
between firms and customers, and firms’ optimal product lines under numerous variations. One of the most important aspects of multi-period interactions between firms and customers is how consumers’ past purchases impact their future ones, which implicitly produce one type of demand-side externality: switching costs. In Information Technology and Switching Costs, Pei-yu Chen and Lorin Hitt investigate how firms can influence consumer-switching costs with a focus on ‘‘information-intensive’’ markets which often have significant standardization and compatibility issues. The chapter presents a formal definition of switching costs, clarifies some general points about switching costs, and reviews some theoretical and empirical studies of IT and switching costs. The chapter highlights how switching costs arise endogenously in high-tech and information-intensive markets while discussing instruments for influencing these costs. Finally, the chapter provides a discrete choice model for managing and estimating customer switching costs. While IT could benefit consumers by more precisely identifying their needs, it can also be used to price discriminate (as in BBPD) or to exclude individuals with less attractive characteristics. Furthermore, organizations sell customer information to third parties, subjecting their customers to their information being used ‘‘against’’ them more broadly. In The Economics of Privacy, Ivan Png and Kai-Lung Hui examine the issue of who should control what information is tracked and stored through an economic analysis of privacy. The chapter begins with the ‘‘free market’’ critique of privacy regulation. Because welfare may be non-monotone in the quantity of information—due to the cost of information or some consumers being priced out of the market when it is socially efficient for them to consume—there may be excessive incentive to collect information. This result applies to both non-productive and productive information and is exacerbated when personal information is exploited across markets. Furthermore, the ‘‘free market’’ critique does not apply to overt and covert collection of information that directly causes harm, for example, a flood of unsolicited promotions. The chapter then reviews research on property rights and the challenges in determining their optimal allocation to examine whether or not individuals might voluntarily or be paid to reveal their personal information. IT enabling more flexible pricing policies has increased price discrimination in many ways. In Product Bundling, Xianjun Geng, Maxwell Stinchcombe, and Andrew Whinston study product bundling, especially for information goods whose low marginal cost and flexibility facilitates bundling. While bundling can reduce integration, transaction, and distribution costs, the straightforwardness of this argument has limited its presence in the literature. Therefore, this chapter focuses on bundling for price discrimination and bundling as a competition tool. Price discrimination arises most easily where buyer valuations over two products are negatively correlated. In general, bundling benefits a seller when it reduces valuation
xx
Introduction
heterogeneity and if marginal cost is low. However, these are not necessary conditions and the chapter explores when bundling emerges in a broader range of cases. Research on using bundling as a competition tool falls into two categories: entry deterrence (tying) and product differentiation. While the literature on using bundling for entry deterrence focuses on how a seller can fend off all rivals, the literature on using bundling for product differentiation asks the question of when two or more ex-ante homogeneous sellers can coexist and both reap positive profits using bundling. Another aspect of pricing impacted by information technologies’ ability to track and analyze information is dynamic pricing, also known as yield management or revenue management, which is most useful when the products expire at a point in time and where the capacity is fixed. Airlines are a natural instantiation of these characteristics and in the 1980s airline pioneered the use of complex IS to constantly monitor and vary their prices. In Dynamic Pricing in the Airline Industry, Preston McAfee and Vera te Velde go beyond surveying of yield management research to expand a common existing model to a more standard case. Then, by examining the efficient allocation, rather than the profit-maximizing allocation, the chapter shows that many of the conclusions attributed to profit-maximization are actually consequences of dynamic efficiency. The chapter proposes the perspective of selling options and suggests that airlines should sell two kinds of tickets: a guaranteed use ticket and a ticket that can be delayed at the airline’s request. Finally, airline pricing data is used to generate stylized facts about the determinants of pricing, facilitating the evaluation of different models. The prior chapters on firms’ pricing decisions implicitly assume variants on posted price schemes. Improvements in IT also enhance the attractiveness of more general selling mechanism such as auctions. Auctions have the advantage of price discovery and the disadvantage of higher transaction costs. IS improve both sides of the trade-off in auctions favor by reducing the transaction costs of auctions and raising the value of price discovery by increasing the number of potential participants. In Online Auctions, Axel Ockenfels, David Reiley, and Abdolkarim Sadrieh provide an overview of some of the theoretical, empirical, and experimental research on online auctions. The chapter first compares theoretical single-object-auction results to experimental findings and empirical observations online. The chapter then focuses on auction design details: public and secret reserve prices including minimum bids and shill bids, late and incremental bidding, and the buy-now option. The buy-now option creates an outside option for bidders, which is also provided by parallel auctions and other outside options by other sellers. The chapter also discusses multi-object auctions, although these are both theoretically and empirically more difficult than the single-object auctions. Finally, the chapter concludes with general remarks on the design of online auctions. When large numbers of buyer and sellers interact with one infrequently, reputation and brand become weaker and the adverse selection and model
Introduction
xxi
hazard problems become more acute. Auctions are the most extreme example of this, but it is a widespread phenomenon. Online reputation mechanisms using the Internet’s bi-directional communication capabilities mitigate these concerns by allowing individuals to share opinions and experiences on a wide range of topics. In Reputation Mechanisms, Chrysanthos Dellarocas surveys our progress in understanding the limitations inherent in these mechanisms. The chapter discusses how technology-based reputation mechanisms differ from traditional word-of-mouth networks, for example, the global reach and anonymity of online interactions. The chapter combines an overview of relevant work in game theory and economics as well as insights from computer science, marketing, and psychology to evaluate the design, evaluation, and use of reputation mechanisms in online environments. I am very pleased with this volume. The authors deserve congratulations for their outstanding work. I want to thank all of them for this and for their forbearance in the delays in the publication process. I believe that the volume will provide a survey of our current state of knowledge in theses areas and frame the most fruitful directions for future research. Terrence Hendershott
This page intentionally left blank
Terrence Hendershott, Ed., Handbooks in Information Systems, Vol. 1 Copyright r 2006 by Elsevier B.V.
Chapter 1
Diffusion of Information and Communication Technologies to Businesses
Chris Forman Tepper School of Business, Carnegie Mellon University, Pittsburgh, PA, USA
Avi Goldfarb Rotman School of Management, University of Toronto, Toronto, ON, Canada
Abstract We survey the literature on the adoption and diffusion of information and communication technologies (ICT) in businesses. We identify two key dimensions that have been the focus of most of the literature. First, research can be categorized as focusing on ICT adoption costs or ICT adoption benefits. Second, research can be categorized as focusing on how adoption is influenced by either the internal organization of the firm or by the external environment. Major themes are highlighted as opportunities for future research.
1
Introduction
The invention of new technology is only the first step in economic progress. To contribute to economic growth, new technology must first be used in the course of productive economic activity. However, new technology often diffuses unevenly across economic agents. Such variance in adoption rates is important, as it influences the rate of technological progress. It may also be a source of sustainable competitive advantage (Mata et al., 1995). Research on information and communication technology (ICT) diffusion to businesses seeks to understand why firms adopt ICT at different rates. Researchers attempt to identify and measure characteristics of the organization and its environment that influence the barriers to ICT investment 1
2
C. Forman and A. Goldfarb
and the benefit of ICT investment to the organization. By investigating how business characteristics shape ICT investment decisions through their impact on (unobservable) costs and benefits of adoption, this literature is complementary to business-value research, which seeks to identify how observable ICT investment decisions influence observable gross benefits. This line of research is important to understanding the fundamental question in the information systems literature of ‘‘Why do some organizations succeed with their information technology investments while others do not?’’ (Dhar and Sundararajan, 2004). This review examines research on the diffusion of ICT to businesses. Because of the breadth of this field, we limit the scope of our review in several ways. First, we focus upon recent research on ICT that facilitates communication within and across firm boundaries. Internet technology is one common example of such technology, but it is not the only one.1 Information technologies with data processing functions but no communications functions are not considered. By lowering the costs of communications, ICTs improve the monitoring and coordination capabilities of organizations. Moreover, they will be most susceptible to network effects: the value of the technology will be increasing in the number of other users. A large theory literature has studied how the communications capabilities of ICT will alter the organization of firms (Marschak, 2006), their interrelationships (Spulber, 2006), and their location (Gaspar and Glaeser, 1998). Owing largely to data constraints, empirical testing of these hypotheses has only been possible over the past 15 years. This review will examine recent advances made in this empirical literature, and highlight areas for future research. Moreover, we will focus attention upon ICT that involves significant costs of adoption at the organizational level.2 Thus, our review generally excludes technologies such as cell phones that have significant infrastructure costs but which involve little idiosyncratic adaptation costs at the organizational level to be used successfully. As in any review article, the content of our discussion will be shaped by the extant literature. Thus, our review will include detailed discussion on EDI but less on newer technologies such as RFID for which the literature is presently small. Last, we examine ICT diffusion through the lens of economic analysis. Economic analysis offers a rich theoretical framework for understanding the diffusion of ICT. Moreover, the increasing availability of micro data has led to rapid growth in this field. We consider this review to be complementary to other excellent review articles in the information systems (IS) literature that have focused on the diffusion of IT through an organizational lens (Fichman, 1992, 2000), or to research in economics that has reviewed the 1 Other examples include electronic data interchange (EDI) and automated clearing house (ACH) technology in banking, both of which involve communication that does not occur over the Internet backbone. 2 The main exception is Section 7, in which we examine country-level diffusion of Internet technology.
Ch. 1. Diffusion of Information and Communication Technologies
3
literature on the diffusion of other technologies (Hall and Khan, 2003). By focusing on the economic analysis of ICT diffusion to businesses, we address a growing area of the literature that has not recently been reviewed. Table 1 provides an overview of some of the major questions in ICT diffusion research and our framework for addressing them in this paper. Research in this area can be classified as exploring either how characteristics of the organization or of its external environment shape the (net) benefits of ICT adoption. Further, as noted above, some characteristics influence primarily the costs and some influence primarily the benefits of adoption (and some influence both). In Table 1, we classify some of the major questions in diffusion research along these two dimensions, and then indicate with section numbers where research on these questions can be found in the paper. Following the introduction, we begin Section 2 by describing the empirical models that are commonly used in the economic analysis of ICT diffusion, and provide a summary of recent work that examines the economic outcomes from ICT investment. Sections 3–7 constitute the main body of the paper. In Section 3, we describe how organizational characteristics influence the rate with which firms adopt ICT. Some research in this area focuses on the adaptations needed by organizations to successfully implement ICT, the costs of these adaptations, and how firms overcome them. Other research examines the relationship between ICT adoption, organizational characteristics, and individual incentives. In particular, an evolving body of work seeks to understand how the location of firm boundaries and internal decision-making rights influence the speed of ICT adoption. Sections 4–7 review literature that examines how an organization’s external environment influences the decision to adopt ICT. Section 4 describes how the geographic location of a firm affects the speed of ICT adoption. In particular, we emphasize recent research that examines whether the communications capabilities of ICT imply that the marginal benefit to ICT adoption is highest in rural areas. Section 5 explores research into the extent to which internal and external factors jointly influence an organization’s decision to adopt ICT. Section 6 shows how network effects shape the pattern of ICT diffusion. Though prior literature has begun to identify the role of network effects on the diffusion of new technologies, new data sets offer opportunities to draw a tighter link between theory and evidence than has previously been possible. In Section 7, we review research on cross-country differences in the speed of Internet technology diffusion. Here we change focus somewhat from the micro-level determinants of firm adoption to macro-level factors that shape differences in country-level ICT diffusion. Moreover, in this section we focus primarily on the diffusion of Internet technology because of the important policy implications of this research.3 Section 8 concludes. 3 Because of the different unit of analysis (country versus organization), cross-country differences in Internet use are not included in the framework in Table 1.
4
C. Forman and A. Goldfarb
Table 1 Summary of research on ICT diffusion How adopter characteristics influence the net benefits of internet adoption Source of variation in adopter characteristics Internal organization
External environment
Adoption costs
Adoption benefits
How do firm boundaries influence the speed of ICT adoption? (see Section 3.2) How does firm size influence the speed of ICT adoption? (see Section 3.3) How do prior investments influence the speed of new ICT adoption? (see Section 3.4) How does co-invention shape the diffusion of ICT? (see Section 5.1) Is ICT use a complement or substitute for urban agglomeration? (see Section 4.1) How does co-invention shape the diffusion of ICT? (see Section 5.1) Do network effects shape the adoption of ICT? (see Section 6.2) How do network externalities shape the adoption of ICT? (see Section 6.3)
How does ICT influence the optimal location of decisionmaking rights within firms? (see Section 3.1) Which organizational characteristics are complementary with ICT? (see Section 3.1) How does ICT investment influence firm boundaries? (see Section 3.2) How does firm size influence the speed of ICT adoption? (see Section 3.3) Is ICT use a complement or substitute for urban agglomeration? (see Section 4.1) How does ICT use influence the location decisions of firms? (see Section 4.2) Do network effects shape the adoption of ICT? (see Section 6.2) How do network externalities shape the adoption of ICT? (see Section 6.3)
Note: Table presents major research questions in the literature on ICT adoption classified along two dimensions: variation in adopter characteristics and how adopter characteristics influence the net benefits to adoption
2
ICT diffusion and its impact
2.1 Diffusion modeling Before examining the economics of ICT diffusion, it is necessary to give a brief overview of diffusion research methodology. An excellent literature
Ch. 1. Diffusion of Information and Communication Technologies
5
# of Adopters
review on diffusion modeling in economics is Stoneman (2002). In this section, we provide an overview of diffusion modeling, highlighting some of the common themes that will appear in our analysis of ICT diffusion. Adoption is the individual-level decision to use a new technology. Diffusion is the aggregation of a number of adoption decisions. Rogers (1995, p. 5) defines it as ‘‘the process by which an innovation is communicated through certain channels over time among the members of a social system.’’ Diffusion research is then concerned with finding patterns across a large number of adoption decisions. The earliest economic models of diffusion were epidemic models. These models assumed that the diffusion of new technology is like that of an infectious disease. Non-adopters adopt a new technology when they come into contact with adopters and learn about the new technology. Over time, the number of users increases, leading to an increased probability of any given non-adopter learning about the technology. This increases the rate of diffusion. As more people adopt, the number of non-adopters declines, which decreases the rate of diffusion. This pattern of diffusion leads to the common S-shaped curve on the rate of technology diffusion with respect to time (see Fig. 1). The first modern technology diffusion study by Ryan and Gross (1943). They used an epidemic model to study the diffusion of hybrid corn to Iowa farmers and found that social networks matter. Methods for measuring epidemic effects developed around the same time in sociology and economics. Sociologists Coleman et al. (1957) were among the first to use epidemic diffusion models outside of an agricultural setting. Their work examined physician choices of new drug prescriptions. Epidemic models are commonly used to help forecast the rate of aggregate technology diffusion. Bass (1969) uses an epidemic model to help predict the
Time
Fig. 1.
The typical pattern of diffusion.
6
C. Forman and A. Goldfarb
rate at which a product will diffuse.4 The central themes of these models— communications and social networks—are also prominent in recent economic research on technology diffusion. In Section 6.2, we discuss papers that have examined how these themes may have influenced the diffusion of personal computers (Goolsbee and Klenow, 2002), use of online grocery services (Bell and Song, 2004), and early networks such as BITNET (Gurbaxani, 1990). As noted above, in epidemic models technology spreads through interpersonal contact and information dissemination. These models do not explicitly model the adoption decisions of individual users, nor do they allow for differences in the costs and benefits of adoption by different members of the population. As a result, these models omit many important aspects of economic behavior. Later models explicitly include these elements. Probit (or rank) models emphasize population heterogeneity. Pioneered by David (1969), the most basic model assumes that the entire population has perfect information about the technology. Individuals (or firms) adopt the technology when the net benefit of adopting is positive. Since the probit model is the one most commonly used in economic diffusion modeling, it is worthwhile to consider it further. In general under the probit model, an establishment i will adopt a new ICT at time t if the following conditions hold NBðxi ; tÞ Bðxi ; tÞ Cðxi ; tÞ40 0
NBðxi ; tÞ4NBðxi ; t0 Þ=ð1 þ rÞt 1 8t0 at
(1) (2)
where NB is the net benefit of adoption, B the gross benefit of adoption, and C the cost of adoption. All functions represent the present value of profits discounted to time t. We let xi be a vector of firm characteristics that influence the value of adopting the new technology. These equations say that a firm will adopt if two conditions hold—first, that the expected benefits less expected costs (or net benefits) are positive and second, that the net benefits of adopting at time t are greater than the net benefits of adopting any other time t0 at: A technology diffuses throughout the population because either the benefits of adopting are increasing over time (due, e.g., to improvements in the technology as a result of technological change), @Bðxi ; tÞ=@t40; or because the costs of adoption are declining, @Cðxi ; tÞ=@t40: Most diffusion papers are unable to separately identify benefits and costs and instead identify @NBðxi ; tÞ=@xi ; the change in the net benefits to adoption that occur as a result of the changes in firm characteristics. As Hall and Khan (2003) note, due to high sunk costs of adoption, adoption is usually an absorbing 4 Fichman (2000) refers to epidemic models used to forecast the rate, pattern, and extent of technology diffusion as ‘‘diffusion modeling studies.’’ For surveys of such studies, see Mahajan et al. (1990) and Mahajan and Peterson (1985).
Ch. 1. Diffusion of Information and Communication Technologies
7
state. That is, we rarely observe organizations ‘‘unadopting’’ a new technology, and analysts rarely worry about this decision in their econometric modeling. The basic probit model underlies any diffusion modeling that explicitly considers agents’ trade-offs between the costs and benefits of adopting, and it is the workhorse for many of the models we discuss in Sections 3–6.5 However, in contrast to epidemic models, the probit model examines exclusively how internal firm factors shape the benefits to adoption and assigns no role to the behavior of other users. Clearly this may be too limiting. Recent economic models of ICT diffusion have extended the probit model to allow a role for other users’ behavior. In addition to epidemic and probit models, Karshenas and Stoneman (1993) consider two ways in which other users’ behavior may influence technology adoption, which they term ‘‘stock and order effects.’’ Stock models argue that if new technologies are cost reducing, they will increase the output that a firm produces. As a result, increasing adoption of new technologies eventually decreases the profits of adopters and non-adopters alike. Under certain conditions, the difference in profits between adopters and non-adopters declines over time (Reinganum, 1981), leading to decreasing net benefits from new technology adoption. Stock models assume that the profits among all adopters are identical—as are the profits among all non-adopters. In contrast, order models assume that the benefits of new adoption decrease monotonically with the number of prior adopters. Despite this, early adopters continue to benefit disproportionately from the technology (Fudenberg and Tirole, 1985). The intuition behind why early adopters enjoy higher profits than later adopters is that there may be first-mover advantages to adoption, due, for example, to the ability of early movers to capture scarce inputs such as labor (Ireland and Stoneman, 1985). Order and stock effects are examples of negative network externalities. That is, the benefits of adopting a new technology are declining in the number of other users. However, the benefits of adopting a new technology can also increase as others adopt, generating positive network externalities.6 Positive network externalities can further be categorized as direct or indirect. The telephone provides an example of a direct positive network externality: the value of adoption relies explicitly on the value of communicating with other users. Alternatively, positive network externalities may be indirect, as in the case of video game consoles such as the Sony Playstation: the value of adoption increases in the adoption by other users because of the increased availability of a complementary good (games). Recent research 5 In Section 7, the unit of analysis is frequently the country, making it more difficult to identify the underlying economic model that is generating the observed empirical pattern. 6 The description at the beginning of Section 6 provides further details on positive network externalities.
8
C. Forman and A. Goldfarb
has attempted to understand how direct and indirect network externalities shape user adoption behavior; however, the econometric identification issues in this research are daunting. A finding of a statistical correlation between one user’s adoption and another’s may reflect network externalities. Alternatively, it may reflect unobserved characteristics that are common across users and which increase the value to adoption. In Section 6.3, we discuss the conditions under which these models are identified. The main review of diffusion research in fields other than economics is Rogers (1995). Emphasizing communications and sociology, Rogers focuses on the role of communications networks in technology diffusion. He details the process through which innovations move from one population to another and discusses the role of five key factors in the individual decision to adopt: relative advantage, complexity, compatability, trialability, and observability. He emphasizes that these factors are only relevant after informative contact with the innovation, and much of this work focuses on the roles of different communications networks in initiating this contact. This contact is achieved by a ‘‘change agent.’’ The change agent brings an innovation into a new social network and can effectively communicate its benefits. Managers aiming to generate technology adoption should think of themselves as change agents. In their emphasis on how characteristics of the adopter or technology shape the value of adopting, the five key factors are similar to the emphasis on heterogeneous adoption benefits and costs in the probit model. Moreover, Rogers’ emphasis on communication and change agents is analogous to the importance of information transmission in epidemic models of diffusion. 2.2 The impact of ICT diffusion There is considerable evidence that ICT investment has a positive impact on firm performance. One line of research has demonstrated using microeconomic production theory that ICT investment had a large impact on the company behavior and productivity in the late 1990s. For example, Jorgenson and Stiroh (2000), Oliner and Sichel (2000), and Baily and Lawrence (2001) credit ICT with the rapid growth of the US economy in the late 1990s. Stiroh (2002a) argues that this productivity acceleration was broad based and finds an increase in productivity related to ICT use in nearly twothirds of industries from 1995 to 2000. Baily and Lawrence (2001, p. 308) claim about the 1990s, ‘‘In particular, there has been a substantial structural acceleration of total factor productivity outside of the computer sector. And there is clear supportive evidence of an acceleration of productivity in service industries that are purchasing [ICT].’’ In another study of ICT and productivity growth in the 1990s, Brynjolfsson and Hitt (2003) use firm-level data to find that substantial long-term productivity gains result from ICT
Ch. 1. Diffusion of Information and Communication Technologies
9
use.7 Although these studies focus on the role of generic information technology investment, recent work has demonstrated a link between computer networking and acceleration in establishment-level productivity (Atrostic and Nguyen, 2002; Stiroh, 2002b). Another line of work has examined how adoption of ICT improves business processes. This ‘‘process-oriented’’ framework (Barua and Mukhopadhyay, 2000) examines the relationship between ICT and intermediate process-level variables that are more closely related to ICT investment than the measures of output, sales, or value added that are traditionally used in production theory studies. For example, Mukhopadhyay and several co-authors have examined the impact of ICT investment on supply chain performance (Srinivasan et al., 1994; Mukhopadhyay et al., 1995; Mukhopadhyay and Kekre, 2002). Hubbard (2003) shows that on-board computers increase efficiency in trucking. Athey and Stern (2002) document the role of ICT in improving emergency health care response outcomes. Recent research has argued that the link between IT adoption and firm performance will depend on usage. Using data on hospitals, Devaraj and Kohli (2003) argue that ICT use is a much better predictor of performance than ICT adoption at the firm level. Many firms adopt a technology on the surface, but, unless it is frequently and properly used, it will not have a positive impact and may even have a negative one. We review recent research that has examined IT usage separately from IT adoption in Section 3.5. In summary, ICT has had an important impact on productivity at both the micro- and macro-levels. The rest of this paper details patterns in ICT diffusion. We first look at how organizational characteristics influence technology adoption. We then examine the external environment in Sections 4–7. 3
ICT adoption and organizational characteristics
The decision to adopt a technology can be influenced by factors relating directly to the firm. In this section we consider how organizational characteristics influence ICT adoption (see Table 2). 3.1 Adoption, internal firm organization, and organizational change Bresnahan and Trajtenberg (1995, p. 84) define general purpose technologies (GPTs) as ‘‘enabling technologies, opening up new opportunities rather than offering complete, final solutions.’’ Several authors have argued that ICT is a GPT (e.g., David, 1990; Bresnahan and Greenstein, 1996; 7 For firm-level evidence on the productivity benefits of ICT investment from an earlier time period, see Brynjolfsson and Hitt (1996).
10
C. Forman and A. Goldfarb
Table 2 Summary of research on ICT adoption and organization characteristics (Section 3) Open question
Papers
Results
How does ICT influence the optimal location of decisionmaking rights within firms? (see Section 3.1)
Gurbaxani and Whang (1991), George and King (1991), Marschak (2004), Hubbard (2000), Barua et al. (1995, 1996), Barua and Whinston (1998), Hitt and Brynjolfsson (1997), Bresnahan et al. (2002), Dunlop and Weil (1996),
Which organizational characteristics are complementary with ICT? (see Section 3.1)
Mendelson and Pillai (1998), Dewan et al. (1998), Banker et al. (2003), Forman (2005)
How does ICT investment influence firm boundaries? (see Section 3.2)
Gurbaxani and Whang (1991), Malone et al. (1987), Clemons et al. (1993), Brynolfsson et al. (1994), Hitt (1999), Dewan et al. (1998), Kraemer et al. (2002), Baker and Hubbard (2003, 2004) Forman and Gron (2005), Gertner and Stillman (2001)
ICT can lead both to centralization and decentralization of decisionmaking rights, depending on the technology and organization characteristics. To date, cross-industry empirical work has suggested that decentralization predominates ICT adoption is more valuable for firms that operate in dynamic business environments, firms with higher coordination costs, and firms that encourage participatory behavior by value chain partners Decreases in coordination costs engendered by ICT lead to less vertical integration. However, decreases in monitoring costs may lead to more vertical integration
How do firm boundaries influence the speed of ICT adoption? (Section 3.2) How does firm size influence the speed of ICT adoption? (see Section 3.3)
How do prior investments
Kimberly and Evanisko (1981), Eveland and Tornatzky (1990), Attewell (1992), Hannan and McDowell (1984), Charles et al. (2002), Forman et al. (2002, 2006), Astebro (2002, 2004), Banker et al. (2003), Hubbard (2000), Kauffman et al. (2000) Tornatzky and Fleischer (1990), Bresnahan and
Decreases in vertical integration lead to increases in frictions that slow ICT adoption, other things equal Size is positively correlated with adoption. However, most studies are unable to identify the theoretical explanation for this empirical result
Firms that have made more recent ICT investments or
Ch. 1. Diffusion of Information and Communication Technologies
11
Table 2. (Continued ) Open question
Papers
influence the speed of new ICT adoption? (see Section 3.4)
Greenstein (1996), Forman (2005), Zhu and Kraemer (2005)
How does intra-firm diffusion differ from inter-firm diffusion? (see Section 3.5)
Fichman and Kemerer (1997, 1999), Attewell (1992), Astebro (1995, 2004), Battisti and Stoneman (2003), Battisti et al. (2004), Copper and Zmud (1990), Goldfarb and Prince (2005), Kraut et al. (1998)
How do individuals use ICT once it has been adopted by an organization? (see Section 3.5)
Examples include Davis (1989), Davis et al. (1989), Szajna (1996), Venkatesh and Davis (2000), others
Results that have more experience with ICT will adopt faster, ceteris paribus. However, if these investments are embedded they may lead to slower adoption There is considerable evidence that the pattern of ICT usage differs from that of adoption. However, theoretical development and empirical testing of the reasons behind these differences are at an early stage The TAM model has focused on how perceived usefulness and ease of use predict longrun usage
Note: First column describes major questions that explore how organizational characteristics influence ICT adoption. Second column describes some of the major papers that have addressed these issues. Third column describes a generalization of the results of these papers.
Harris, 1998; Forman et al., 2005).8 A GPT is an enabling technology that can be deployed in a multitude of ways in many sectors of the economy. However, adapting these general solutions to the needs of individual industries and idiosyncratic users often poses great challenges for organizations. A large case study literature has detailed the challenges of implementing general ICT systems to fulfill organizational needs. This case study literature has shown there to be significant technical development risks when implementing new ICT systems in firms (e.g., Kemerer and Sosa, 1991). Moreover, organizations face significant challenges in adapting business processes and organizational routines to new ICT systems (e.g., Attewell and Rule, 1984; Scott Morton, 1991; Davenport, 1998). Finally, new ICT systems may require a different skill set than may have been previously available, requiring either additional education or skills (Autor et al., 2002; Levy and Murname, 2004), or they may also be ‘‘de-skilling,’’ requiring workers to be less skilled than previously. Despite this rich case study 8 Examples of other historical GPTs include the steam engine (Rosenberg and Trajtenberg, 2004) and the dynamo (David, 1990).
12
C. Forman and A. Goldfarb
literature, there remains relatively little empirical adoption research in this area that applies an economic perspective. The main work in this area has, instead, explored theories of organizational change.9 New information technology often changes the organization of work within firms. Beginning with Leavitt and Whisler (1958), a long-running question within the IS field is how improvements in information processing capabilities change the optimal location of decision rights within organizations. Improvements in information processing capabilities can decrease monitoring costs, leading to lower agency costs when delegating decisionmaking rights. However, lower costs of information transmission can also lower the costs of centralizing information within the organization by improving the quality and speed of information processing (Gurbaxani and Whang, 1991). A rich case study and small sample empirical literature has found support for both hypotheses within the context of information processing IT (George and King, 1991). Marschak (2004) formalizes many of these ideas in an economic model. However, only recently have there been large-scale empirical studies that examine the relationship between ICT adoption and decision rights within firms. Hubbard (2000) pursues this debate in the context of trucking carriers’ decisions to adopt electronic monitoring technologies. He notes that monitoring ICT can create two benefits for organizations. First, it can lower agency costs. For example, on-board computers in the trucking industry can monitor drivers’ speeds. Thus, such technologies increase the value of decentralization, all else equal. Second, monitoring ICT can improve the resource-allocation decisions of managers, for example, by monitoring the relative locations of trucks. By lowering information transmission costs, these technologies can also increase the value of centralization. Hubbard shows that both types of monitoring technologies are adopted by trucking firms. Moreover, the likelihood that each is adopted will depend on the relative values of centralization and decentralization across different carriers. Other research has argued that ICT is more valuable when adopted by firms that use innovative organizational and human resource practices, such as self-managing teams, flatter organizational hierarchies, and broader job descriptions that involve decentralization. Milgrom and Roberts (1990) argue that investments in new technology may have indirect effects that increase the value of utilizing such innovative practices. They say that groups of activities such as new technology investment and organizational variables are complements when an ‘‘increase in the levels of any subset of activities implies that the marginal return to increases in any or all of the remaining activities increases.’’ Researchers in the IS literature have argued that there exist complementarities between IS design, worker incentives, 9 For recent exceptions, see the discussion on research on co-invention in Section 5.1. See also Doms et al. (1997) for an econometric examination of the relationship between worker skills and adoption of computer-automated design, numerically controlled machines, and programmable controllers.
Ch. 1. Diffusion of Information and Communication Technologies
13
and organizational characteristics (e.g., Barua et al., 1995, 1996; Barua and Whinston, 1998). A number of recent papers have empirically tested the assertions of Milgrom and Roberts (1990). Hitt and Brynjolfsson (1997) and Bresnahan et al. (2002) use a large cross-industry study to show that decentralized decision rights, innovative human resource practices, and workplace investments in human capital are complementary with ICT investment. Brynjolfsson et al. (1997) develop a tool for managers to understand the indirect effects between ICT investments and organizational practices, and how these interactions should shape investment in new ICT. Dunlop and Weil (1996) study the use of ICT and modular assembly (team production) in the apparel industry. They find that modular production is adopted in conjunction with new communication systems like EDI to reduce lead times and inventories. Empirical research has tested a number of other assertions about which organizational characteristics are complementary with ICT adoption. For example, ICT adoption may be more valuable when coordination costs are high or when real-time coordination is especially valuable. Mendelson and Pillai (1998) show that firms operating in dynamic business environments need to process information more rapidly. These firms are more likely to adopt real-time communication technologies such as pagers and videoconferencing, and they are more likely to have EDI connections. Dewan et al. (1998) show that firms with higher coordination costs arising from diversification or vertical integration have greater ICT investments. Banker et al. (2003) find that manufacturing plants that employ customer and supplier participation practices adopt EDI more rapidly. Forman (2005) shows that firms that are geographically decentralized adopt Internet technology more rapidly. As noted by Athey and Stern (2003), there are significant challenges in testing theories about complementarities solely by looking at investment behavior. As a result, several researchers have sought to test theories relating ICT to human resource practices through an ICT-productivity or businessvalue approach. Research in this literature generally examines how ICT, organizational practices, and their interaction influence labor productivity (e.g., Black and Lynch, 2001; Bresnahan et al., 2002; Hempell, 2003; Bertschek and Kaiser, 2004) and stock market returns (Brynjolfsson et al., 2002).10 These papers generally proceed by regressing outcome measures on ICT, organizational practices, and interactions of these variables. Research in this area has explored how ICT and labor practices influence outcomes, and provided evidence that the marginal returns to each increase with the presence of one of the others. 10 For recent reviews, see Barua and Mukhopadhyay (2000) and Brynjolfsson and Hitt (2000). Corrado et al. (2005) provides an overview of recent work that attempts to measure the value of organizational investments.
14
C. Forman and A. Goldfarb
Empirical work on ICT adoption and organizational change remains a fertile area of research. Despite a rich theoretical and case study literature in this area, the empirical literature remains small. Hypotheses in this area typically focus on long-run effects. However, data sets that allow for testing these long-run hypotheses are rare. New research has used ICT adoption decisions to examine the influence of medium- to long-run organizationlevel decisions on short-run ICT adoption decisions. This allows for the identification of complementarities in the short run. In the long run, these complementarities may influence organizational decisions and outputs. Research in this area has proceeded along two independent lines. Large cross-industry studies such as Bresnahan et al. (2002) have attempted to identify complementarities between ICT investment and organizational characteristics that are common to multiple industries. Single-industry studies such as Hubbard (2000) are able to make very precise statements about the interactions between incentives and ICT in an industry, but at the cost of generalizability. Research in this area should continue along these separate paths to more fully identify the relationships among ICT, incentives, organizational design, and outcomes. 3.2 Adoption and firm boundaries In the section above, we described a set of papers that analyze the relationship between ICT investment and the location of decision-making authority within a firm. ICT investment can influence firm organization in another way, by altering the costs and benefits of locating economic activities outside the boundaries of the firm. One can classify the costs of operations into internal coordination costs, external coordination costs, and production costs (Gurbaxani and Whang, 1991). Production costs refer to the actual physical costs of producing a good or service. Internal coordination costs refer to the costs of managing activities within the boundaries of the firm. External coordination costs represent the search costs of identifying suppliers, the costs of writing contracts, and potential transaction costs arising from opportunistic behavior by upstream and downstream partners (Williamson, 1975). All else equal, market-based exchange should have lower production costs but higher coordination costs (Malone et al., 1987). Investments in ICT can lower the costs of internal coordination; this will decrease the costs of managing large enterprises, enterprises that are geographically dispersed, enterprises that are diversified, and enterprises that are vertically integrated. However, ICT investments lower the costs of market transactions, as well, by lowering the communication and coordination costs required to undertake arm’s-length transactions with external parties and by lowering the risks of opportunism on the part of trading partners (Clemons et al., 1993). Malone et al. (1987) argue that the marginal impact of ICT investment will be greater on external coordination
Ch. 1. Diffusion of Information and Communication Technologies
15
costs than on internal coordination costs. This leads to the conclusion that ICT investments facilitate more market transactions and less vertical integration. Empirical work that has researched the relationship between ICT investment and vertical integration across multiple industries has generally supported the assertion that ICT investment will lead to more market-based transactions. Research in this area usually proceeds by examining whether changes in the dollar value of ICT spending are associated with an increase or decrease in vertical integration. Brynjolfsson et al. (1994) use Bureau of Economic Analysis investment data to show that greater ICT investments are associated with a significant decline in average industry firm size. Hitt (1999) examines these hypotheses empirically at the firm level using a panel of 549 large firms. He explores how ICT investments influence vertical integration and internal diversification. He argues that a negative relationship between ICT investment and vertical integration implies that ICT decreases external coordination costs, while a positive relationship between ICT and diversification implies that ICT decreases internal coordination costs. He shows that ICT investment leads to a significant decline in vertical integration and a smaller increase in diversification. Moreover, he shows that increases in vertical integration lead to less ICT investment.11 Other papers have explored this relationship from the other direction: do firms with greater internal and external coordination costs have greater ICT investments? Dewan et al. (1998) show that vertical integration is negatively related to the level of ICT investment, and that ICT therefore has a larger impact on external coordination costs than on internal coordination costs. They also demonstrate that diversified firms will have greater levels of ICT investment. Thus, their results are consistent with the results of Malone et al. (1987) and Hitt (1999). Kraemer et al. (2005) also find that firms that are more global are more likely to adopt business-to-business electronic commerce. This result may reflect higher coordination costs among global firms, but it may also reflect the effects of stronger competition or external pressure from trading partners. While cross-industry studies have consistently found that ICT investment is associated with less vertical integration, a set of recent single-industry studies have found less consistency in this relationship. These studies have generally examined the adoption of industry-specific ICTs, as opposed to the dollar value of ICT spending. By focusing on a single industry, these studies are able to precisely examine how ICT investments influence incentives and coordination costs among firms within a particular industry environment. However, as always, this precision comes at a potential loss of generalizability. 11 Acemoglu et al. (2004) also examine the role of general technology investment and vertical integration. They find a negative relationship between intensity of R&D and investment and vertical integration.
16
C. Forman and A. Goldfarb
Building on the work of Hubbard (2000), Baker and Hubbard (2003, 2004) examine the role of ICT adoption on the organization of firms in the trucking industry. They argue that while some margins of ICT investment primarily lower coordination costs, other margins of ICT investment also lower monitoring costs, improving agents’ incentives. The effects of ICT on monitoring and coordination create differing predictions on how ICT will influence firm boundaries based on the margin of ICT investment and the characteristics of the firm. Baker and Hubbard (2003) examine the decision of shippers—purchasers of trucking services—to own their own trucking fleets or to contract for carrier services. They show that adoption of certain kinds of ICT leads to more shipper ownership of trucks (vertical integration) by improving monitoring capabilities (and thereby lowering agency costs associated with complex job designs). In contrast, they also find that other margins of ICT investment enhance coordination capabilities that improve the comparative advantage of for-hire trucks, thereby leading to less shipper ownership of trucks. Baker and Hubbard (2004) investigate how ICT investment influences the decision of truckers to own their own trucks. They show that driver ownership of trucks declines with adoption of on-board computers that improve monitoring capabilities. These on-board computers decrease the agency costs of trucker ownership by lowering monitoring costs. Some recent single-industry studies have also examined how vertical integration influences the short-run decision to adopt new ICTs. In contrast to studies that look at the impact of ICT investment on firm boundaries, these papers have consistently found vertical integration and ICT adoption to be complements. This literature has generally focused on the contracting costs of making interorganizational ICT investments. For example, less vertically integrated firms find interorganizational Internet investments less valuable because downstream partners may appropriate some of the surplus generated by the new investment. Moreover, ICT investments such as ecommerce that create new distribution channels may be harder to implement when retailers and producers are different firms because of potential channel conflict. In other words, while long-run studies argue that ICT investments reduce the frictions associated with market transactions, shortrun studies demonstrate that such frictions can have significant effects on the speed of ICT adoption. Forman and Gron (2005) investigate how vertical integration influences adoption of electronic commerce applications in the insurance industry. In their context, electronic commerce influences the distribution relationship between insurers and insurance agents. They argue that insurers that are vertically integrated with their agents will have lower transaction costs of adopting electronic commerce technologies. Their work demonstrates that such insurers adopt e-commerce technologies faster than similar insurers who are not vertically integrated.
Ch. 1. Diffusion of Information and Communication Technologies
17
Gertner and Stillman (2001) examine how vertical integration influences the incentives of apparel firms to offer their products online. In their setting, transaction costs and channel conflict each play a role in increasing the costs of Internet adoption for apparel firms that do not own the distribution channel. They provide evidence that vertically integrated apparel retailers who also own retail outlets such as the Gap started to sell products online sooner than non-integrated apparel companies like Nautica.12 In all, the current literature on ICT investment and firm boundaries draws on a rich array of theoretical perspectives—transaction costs economics, property rights theory of the firm, principal agent theory—and arrives at a broad array of conclusions. Although cross-industry research suggests that, on average, ICT investment may lead to smaller, less vertically integrated firms, single-industry studies suggest there may be substantial heterogeneity in the effects of ICT investment on firm boundaries across industries and technologies. More research is needed to identify the link between ICT investment, agent incentives, and organizational outcomes. Moreover, prior research has identified some differences in the short- and long-run relationship between ICT and firm boundaries. To the extent possible, future research should explore how this relationship changes as organizations adopt, implement, extend, and maintain new ICT innovations. 3.3 Adoption and size Prior research in the diffusion of innovations literature has consistently shown a positive relationship between organization size and innovativeness (Rogers, 1995). The most common reasons offered for this relationship are economies of scale (Kimberly and Evanisko, 1981), slack resources (Eveland and Tornatzky, 1990), access to outside resources (Attewell, 1992), and ability to bear adoption risks (Hannan and McDowell, 1984). Recent work on ICT adoption has continued to show a positive relationship between firm size and adoption. Large-scale descriptive studies on Internet use have shown that even as late as 2000, adoption of basic Internet technologies such as e-mail varies with establishment size, and that small establishments rarely adopt complex technologies such as e-commerce (e.g., Census, 2002; Charles et al., 2002; Forman et al., 2002).13 Academic research has also shown a positive relationship between size and ICT adoption (e.g., Hubbard, 2000; Kauffman et al., 2000; Astebro, 2002; Banker et al., 2003; Forman et al., 2006). 12 Helper (1995) and Lane (1991) have also found a positive relationship between relationship ‘‘closeness’’ between suppliers and buyers and general (non-IT) technology adoption. These papers focus on how tighter relationships between upstream and downstream firms increase the stability of demand and thereby decrease the risks of new technology adoption. 13 A similar relationship has also been found in the adoption of knowledge management practices. See, for example, Kremp and Mairesse (2004).
18
C. Forman and A. Goldfarb
Measures of size are typically included as a control in recent research on firm adoption of ICT. However, the theoretical reasons for why size influences adoption are not widely understood. Empirical research has been unable to inform theory because of the difficulty in separately identifying the various explanations for this phenomenon. This prevents researchers from making strong statements about why size influences technology adoption. One notable exception is Astebro (2002, 2004), who demonstrates that faster adoption of computer-aided design (CAD) and computer numerically controlled (CNC) machine tools among large manufacturing plants is due to the large non-capital investment costs such as learning that are required to use these technologies. Future work should seek to further understand how and why size influences technology adoption. 3.4 Technical infrastructure Some research has also emphasized how prior IT investments influence the value of adopting new ICT. Tornatzky and Fleischer (1990) list technological context as one of the key aspects of a firm’s adoption decision in their TOE framework.14 Because prior work has found that IT investments can also capture organizational differences (e.g., Bresnahan and Greenstein, 1996; Forman, 2005), we include technical infrastructure under organizational characteristics in our framework. Technical infrastructure can influence the value of new technology adoption because compatibility with new innovations influences the costs of adoption. Rogers (1995) lists compatibility as one of his five key factors influencing an individual’s decision to adopt new technology. However, prior investments in hardware and software can also proxy for an organization’s overall technology competence (Bharadwaj, 2000) or technological sophistication (Raymond and Pare´, 1992). Technological sophistication reflects the number and diversity of information technologies used by organizations, and is a key component of IT sophistication (Raymond and Pare´, 1992). Iacovou et al. (1995) note that organizations with high levels of IT sophistication are less likely to feel intimidated by technology and are more likely to have access to the technological and managerial resources necessary to adopt new technologies. Other empirical research has also demonstrated that IS departments with higher levels of technical competence or more recent infrastructure investments are more likely to adopt new ICT (e.g., Zhu et al., 2003; Forman, 2005; Zhu and Kraemer, 2005). However, Forman (2005) and Zhu et al. (2006) have shown that if such investments are specific to prior generations of ICT, they may in fact slow adoption.15 14 The TOE framework identifies three key components that influence an organization’s technology adoption decision: technological context, organizational context, and environmental context. 15 Prior investments in related technologies can also create switching costs that can influence IT vendor choice. For more on empirical research in this area, see Chen and Forman (2006), Chen and Hitt (2006), Forman and Chen (2005), and Greenstein (1993).
Ch. 1. Diffusion of Information and Communication Technologies
19
3.5 Adoption, assimilation, intra-firm diffusion, and usage The adoption of new ICT involves four separate decisions that may occur at separate time periods. One decision is simply whether to adopt ICT at all. This is the decision most commonly studied in the literature on ICT diffusion, and is often labeled inter-firm diffusion (Battisti and Stoneman, 2003, 2005). The second decision involves which capabilities of an innovation to use—variously labeled infusion (Cooper and Zmud, 1990), assimilation (Fichman and Kemerer, 1997, 1999), and depth of adoption (Astebro, 2004). The third decision refers to the rate at which new technology displaces old within the organization, and has been labeled intra-firm diffusion (Battisti and Stoneman, 2003, 2005; Astebro, 2004). The fourth decision is the individual-level long-term decision of how often to use the technology.16 Recent research has explored the relationship between the first two decisions, i.e., the extent to which an organization may not fully assimilate or deploy all of the features of an ICT innovation once the organization adopts it. In most survey-based research, organizations report whether they have a particular application installed. However, patterns of assimilation may differ systematically from that of installation if there exist significant post-investment costs of using new ICT and if these costs are unknown or uncertain ex ante to potential adopters (Fichman and Kemerer, 1999). Empirical work examining process innovations such as software development tools or CAD/CNC tools has demonstrated that these differences exist and can be significant (Fichman and Kemerer, 1999; Astebro, 2004). As a result, some argue that researchers should focus on technology assimilation rather than adoption (Fichman and Kemerer, 1997, 1999). For example, Fichman and Kemerer (1997) show how related and unrelated knowledge and learning economies influence the assimilation of software process innovations, while Zhu and Kraemer (2005) demonstrate how technology competence, financial commitment, competitive pressure, and regulatory support influence e-business use. Because assimilation is a newer concept, little is known about how the factors influencing assimilation differ systematically from those influencing adoption. Astebro (2004) shows that plant size and learning costs influence adoption of CAD/CNC tools more than they do assimilation. Cooper and Zmud (1990) show that task technology fit plays an important role in understanding whether an organization adopts manufacturing resource planning but is less successful in explaining assimilation. The third area of research on ICT diffusion examines the rate with which new technology displaces old within an organization, termed intra-firm diffusion. Research in technologies as diverse client/server computing 16 These distinctions are only relevant if assimilation costs, intra-firm diffusion costs, and usage costs are not perfectly anticipated by the firm. If firms have perfect foresight, then the issues relating to these factors can be mapped back into the first decision on firm-level adoption.
20
C. Forman and A. Goldfarb
(Bresnahan and Greenstein, 1996; Ito, 1996), CNC tools (Battisti and Stoneman, 2003, 2005), electronic mail (Astebro, 1995), and videoconferencing (Kraut et al., 1998; Tucker, 2005) have shown a significant lag between initial adoption and widespread diffusion within an organization. Depending upon the innovation, the internal spread of new ICT innovations may be driven by individual user adoption decisions or by organization-level capital stock adjustment decisions. This distinction has important implications for the factors driving intra-firm diffusion, and for the modeling technology used by the econometrician. User studies in this area have examined how social networks and network externalities influence user decisions to adopt new communication technologies (Astebro, 1995; Kraut et al., 1998; Tucker, 2005).17,18 In contrast, studies that examine firm-level capital adjustment patterns have emphasized adjustment costs (Bresnahan and Greenstein, 1996; Ito, 1996) as well as the importance of available complementary technologies and managerial techniques (Battisti and Stoneman, 2005). The fourth area of research relates to individual-level long-term usage within the firm. Behavioral and psychological approaches are particularly important in this area. In particular, much of this research draws upon the ‘‘technology acceptance model (TAM),’’ based on the theory of reasoned action from social psychology (Davis, 1989; Davis et al., 1989). The TAM model predicts that perceived usefulness and perceived ease of use is key to predicting long-run usage. The idea is that factors that influence behavior, such as user characteristics and system design, do so indirectly through attitudes and subjective norms. The TAM has proven to be a robust model that is frequently employed to study user acceptance of information technology—as of January 2005, the Social Science Citation index reported 511 citations for Davis et al. (1989). It has also inspired several extensions. Szajna (1996) improves on the details of the model and provides further tests. Venkatesh and Davis (2000) extend the TAM model to explain perceived usefulness and ease of use in terms of social influence and cognitive instrumental processes, naming their model TAM2. Kim and Malhotra (2005) show that belief updating, self-perception, and habit help explain usage behavior when added to the TAM. Most research on assimilation, intra-firm diffusion, and usage has been able to demonstrate that there are significant differences in the factors influencing adoption versus these various subsequent decisions. Despite the challenging data requirements, recent gains have been made in this area. Goldfarb and Prince (2005) show differences in Internet adoption and usage patterns at the household level. Their work demonstrates that the 17
We discuss the role of social networks in technology adoption further in Section 6. A related literature on media richness theory has explored user choice of different communication media, and how this choice is shaped by communication needs and the characteristics of the media. See, for example, Daft and Lengel (1984), Daft et al. (1987), Markus (1994), and Hinds and Kiesler (1995). 18
Ch. 1. Diffusion of Information and Communication Technologies
21
demographic characteristics of early adopters are very different from those of heavy users. Battisti and Stoneman (2003) note, however, that our knowledge of this process is much less developed than our knowledge of inter-firm diffusion. As a result, this is an important area for future research. 4
Geographic differences in adoption
In the previous section, we described how ICT can reshape the nature of contractual relationships along the value chain. Until recently, a somewhat less-explored notion is how ICT can alter the geographic dispersion of economic activity.19 The open question is whether ICT leads to more or less concentration in economic activity, i.e., whether ICT is a complement or substitute to urban agglomeration. Research in this literature commonly examines whether ICT adoption and use is more or less common in cities. Less commonly, research in this literature has also examined whether ICT use leads to clustering or dispersion in the location decisions of economic agents (see Table 3).20 One school of thought argues that ICT reduces the costs of performing isolated economic activities, particularly in rural settings, even when deployment costs are high. In this view, ICT decreases the costs of coordinating economic activity that occurs across long distances within and between firms. These distance-related coordination costs may be in addition to those arising from communication across firm boundaries (Section 3.2). For example, these costs may arise due to time lags inherent in transporting physical goods across long distances, or due to the costs of face-to-face communication among geographically separated individuals. This hypothesis argues that the gross benefits for ICT adoption will be decreasing in the size or density of a firm’s location, other things equal (Cairncross, 1997; Forman et al., 2005). There may be several potential explanations for this hypothesis. First, while all business establishments benefit from an increase in capabilities, establishments in rural or small urban areas derive the most benefit from overcoming the disadvantages associated with a remote location. Second, establishments in rural areas lack substitute data communication technologies for lowering communication costs, such as fixed private lines. Third, advanced tools such as groupware, knowledge management, web meetings, and others also may effectively facilitate collaboration over distances. These alternative explanations all lead to the same empirical prediction: that ICT and urban agglomeration will be substitutes. 19 This research builds upon the seminal work of Griliches (1957), who examined economic factors shaped the geographic variance in hybrid seed adoption. 20 Kolko (2002) refers to increasing clustering of economic activity as concentration, while shifts in economic activity away from places where it has traditionally been concentrated as convergence.
22
C. Forman and A. Goldfarb
Table 3 Summary of research on geographic differences in adoption (Section 4) Open question
Papers
Results
Is ICT use a complement or substitute for urban agglomeration? (see Section 4.1)
Forman et al. (2005), Gaspar and Glaeser (1998), Duranton and Puga (2004), Sinai and Waldfogel (2004), Charlot and Duranton (2006), Kolko (2000)
How does ICT use influence the location decisions of firms? (see Section 4.2)
Kolko (2002), Fitoussi (2004)
Empirical work has shown that ICT can be either a complement or substitute. Depending on the technology, the benefits of ICT use can either be increasing or decreasing in location size. Cities often contain complementary resources that can increase the net benefits of ICT use. ICT use leads industries and firms to become more evenly distributed geographically.
Note: First column describes major questions that explore how geographic location influences ICT adoption. Second column describes some of the major papers that have addressed these issues. Third column describes a generalization of the results of these papers.
A second school of thought argues that ICT will lead to increasing concentration of economic activity. There are two reasons why ICT may lead to increases in concentration. First, increases in the size or population density of a location may increase the marginal benefit to electronic communication (Gaspar and Glaeser, 1998). This view argues that improvements in electronic communications will increase the prevalence of face-to-face meetings, thereby increasing the value of locating in cities.21 Moreover, increases in location size will increase the availability of complementary products and services that increase the net benefits of ICT investment. For example, urban areas may offer (1) availability of complementary information technology infrastructure, such as broadband services;22 (2) labor-market thickness for complementary services or specialized skills; and (3) knowledge spillovers and earlier access to new ideas (Duranton and Puga, 2004).23 Each of these 21 This view is consistent with that of IS researchers who study how different types of communication media have different levels of information richness (Daft and Lengel, 1984; Daft et al., 1987). Media such as face-to-face communication, e-mail, and telephone communication differ in terms of feedback capability, communication channels, utilization, source, and language (Bodensteiner, 1970; Holland et al., 1976). As a result of these differing capabilities, these media may be used to transmit different kinds of information. 22 By 1998, almost all but the poorest and most remote geographic areas were serviced by dial-up internet service providers (Downes and Greenstein, 2002). Yet, broadband access was disproportionately an urban technology (U. S. Department of Agriculture, 2001; Crandall and Alleman, 2002). 23 These are closely related to the three major reasons given for industrial agglomeration (e.g., Marshall, 1890; Krugman, 1991).
Ch. 1. Diffusion of Information and Communication Technologies
23
concepts leads to the same empirical prediction: ICT use and urban agglomeration will be complementary. 4.1 Adoption of ICT across urban and rural areas Adoption papers in this literature generally proceed by examining differences in ICT use across urban and rural areas. However, as noted above, differences in location may affect the value of ICT in several ways. A major challenge for empirical papers in this literature is to identify how these competing forces simultaneously shape geographic variation in ICT adoption and use. Using a large survey of Internet use among US firms, Forman et al. (2005) show that, on average, firms in large cities adopted Internet technology faster than those in small cities or rural areas. However, they demonstrate that this pattern is due in part to the disproportionate presence of ICTintensive firms in large locations. Controlling for industry composition, they find a very different relationship between location size and ICT adoption. They show that use of Internet technology for basic purposes like e-mail or web browsing is more likely in rural areas than in urban areas, other things equal. This is particularly true for technologies that involve communication between establishments, which are associated with ending economic isolation. However, use of frontier Internet technologies is more common in cities, even with industry controls. This is particularly true for Internet technologies used for within-establishment communication. They argue that this pattern is consistent with better complementary resources in cities to help overcome the co-invention costs for complex Internet technology. Sinai and Waldfogel (2004) examine how location shapes the use of Internet technology among individuals. Like Forman et al. (2005), they find evidence of both complementarity and substitutability between Internet use and cities. In particular, they provide evidence of increasing availability of local online content in large cities. Individuals use the Internet more when there is more local content; however, controlling for content, individuals in large markets use the Internet less. On balance, there is no systematic relationship between Internet use and geographic location. In contrast, examining individual use of electronic communication technologies among French firms, Charlot and Duranton (2006) find increasing use of all electronic communication technologies within cities. This is consistent with evidence found in Gaspar and Glaeser (1998). Research that has examined investment in Internet infrastructure among suppliers of Internet services has also found a complementary relationship between city size and ICT investment. Kolko (2000) shows that domain name registrations are especially prevalent in medium and large cities. Greenstein (2000) and Downes and Greenstein (2002) show that, in the mid-1990s, one important barrier to Internet adoption was local availability. Internet access was simply not available in isolated areas. By 1998,
24
C. Forman and A. Goldfarb
however, almost all areas had access, and by 2000 almost all had more than one provider in their local market. Availability ceased to be a barrier to adoption. Augereau and Greenstein (2001) find evidence of an urban bias in the adoption of high-speed enabling Internet technology—56 K modem and ISDN—among Internet service providers (ISPs) in the 1990s. This bias likely reflects the impact of larger local market demand and greater local competition on the incentives for suppliers of Internet services to invest in new ICT. 4.2 Evidence of how ICT use influences location patterns A related question asks how the adoption of ICT influences the location decisions of firms. Given that widespread interorganizational communications are a relatively new phenomenon, it has been difficult to test this hypothesis using current data because insufficient time has elapsed to understand how ICT use influences firm location decisions. However, a small number of papers have begun to investigate this important question. Kolko (2002) defines employment convergence as the tendency for an industry to become more uniformly distributed geographically over time. He shows that while there exists an overall trend toward employment convergence, ICT-intensive industries exhibit slower convergence than others. However, he also demonstrates that slower convergence is not due to ICT usage per se, but rather is because ICT-intensive industries tend to value more highly the complementary resources found in cities. In particular, ICT-intensive industries hire more highly educated workers who are disproportionately found in cities. Controlling for labor market effects, he finds that the direct effect of ICT use is to speed convergence. Fitoussi (2004) argues that ICT adoption might allow firms to relocate employees to remote locations. Alternatively, it might make local assets even more important if ICT and face-to-face communication are complements. Based on a sample of Fortune 1000 manufacturing firms, he shows that there is unlikely to be massive relocation due to the advent of the Internet. He also finds that Internet use does induce cost saving through reduced communication costs. 4.3 Future research The relationship between ICT adoption and firm location remains a fruitful area of research. Although the studies above have provided some useful findings on this subject, they have only scratched the surface. These papers demonstrate a variety of different relationships between ICT use and location size, depending on the use of the technology, its complexity, and the importance of complementary resources that depend on market scale. More research is needed on different technologies in different settings in order to understand exactly what features of a location shape ICT use.
Ch. 1. Diffusion of Information and Communication Technologies
25
Future work should also investigate how ICT use affects the long-run location decisions of firms and the agglomeration of economic activity. For example, inexpensive communications may mean that establishments relocate from high-cost, high-density areas to low-cost, low-density areas. These remain open questions, however. Further work should compare the location decisions in industries where interorganizational ICT use is prevalent with those in other industries. This will help complete the picture of how the Internet affects geographic variance in productivity and employment. Furthermore, future work should continue with regard to how pooled local resources influence ICT investment decisions. These resources have the potential to significantly alter co-invention costs by providing access to highly skilled resources that firms may not have internally. For example, one open question is how the thickness of local labor markets and thirdparty service firms influences the ability of firms to overcome co-invention costs. A further question is when do firms rely on external channels to overcome co-invention costs and when do they choose to use internal resources. Forman et al. (2006) represent one step toward addressing this question. Finally, increasing use of ICT may eventually decrease the costs of using externally sourced resources such as ICT services firms. This premise lies behind much of the recent movement to ICT and business-process offshoring. Surprisingly, to date there has been no systematic empirical work that has examined this issue. This represents an important area for future research. Research on urban/rural differences in technology usage has important public policy implications. Rural areas are often the recipients of telecommunications subsidies. The argument for subsidizing rural areas relates to the high fixed costs of building telecommunications infrastructure in lowdensity areas. If there are positive externalities to having the entire nation online, then subsidizing rural areas may make sense. However, the results of Forman et al. (2003) suggest that firms in rural areas have already adopted the Internet in large numbers. Further subsidies would simply provide money to firms that would adopt anyway. Advocates of subsidies need to provide more compelling evidence that adoption lags in rural areas. 5
Trade-offs between organization and environment
Many adoption decisions depend on the interaction of factors that are external and internal to the firm. For example, adapting GPTs to the idiosyncratic needs of organizations often relies on complementary inputs that can be obtained from inside the organization or from external sources (Bresnahan and Greenstein, 1996). Moreover, strategic decisions relating to rival actions and the optimal timing of investment relate to both internal and external factors. This section explores all of these issues (see Table 4).
26
C. Forman and A. Goldfarb
Table 4 Summary of research on trade-offs between organization and environment (Section 5) Open Question
Papers
Results
How does co-invention shape the diffusion of ICT? (see Section 5.1)
Bresnahan and Greenstein (1996), Forman (2005), Forman et al. (2005, 2006), Borzekowski (2004) Dos Santos (1991), Kumar (1996), Benaroch and Kauffman (1999, 2000), Taudes et al. (2000), Fichman (2004)
High co-invention costs lead to slower adoption of new ICT
Does real options analysis lead to better decisions for when to adopt ICT? (see Section 5.2)
Real options analysis can lead to better timing of adoption decisions
Note: First column describes major questions that explore how organization and environment influence ICT adoption. Second column describes some of the major papers that have addressed these issues. Third column describes a generalization of the results of these papers.
5.1 Co-invention Bresnahan and Greenstein (1996) argue that GPTs require complementary investments (‘‘co-invention’’) to adapt general technologies to the idiosyncratic needs of organizations. Such co-invention can involve great time and expense and may involve both organizational change as well as technical adaptation of the organization. Because these changes arise within the boundaries of the organization, co-invention theory draws upon some of the same ideas on organizational characteristics that were discussed in Section 3. However, co-invention theory emphasizes the role of third parties in enabling the transformations necessary for new technology adoption. As a result, research drawing upon co-invention theory spans both the organizational and environmental perspectives on technology adoption. In their study of the firm adoption of client/server networking technology, Bresnahan and Greenstein (1996) demonstrate that co-invention costs can be a significant barrier to technology adoption. They further demonstrate that co-invention costs are highest among high-value users because ICT is most embedded in the business processes and legacy investments in such organizations. As a result, high-value users may be slow to adopt new ICT innovations such as client/server. They use this result to explain the slow conversion of many data-processing centers from mainframe to client/ server technology. Though Bresnahan and Greenstein (1996) is unique as an econometric study that shows how co-invention can be used to explain firm-level ICT adoption, other studies have used co-invention to explain macrodiffusion patterns. For example, Forman (2005) argues that the rapid diffusion of Internet technologies such as access to the World Wide Web and e-mail can be explained by their low co-invention costs. In contrast,
Ch. 1. Diffusion of Information and Communication Technologies
27
technologies such as consumer electronic commerce or business-to-business integration require substantial co-invention, and have consequently diffused more slowly. A technology that requires significant co-invention to be useful is likely to face resistance. Therefore, for the technology to diffuse within a firm, management must be an effective change agent. Rogers (1995, pp. 346–354) argues that effective change agents have high social status and technological competence but are otherwise similar to their target group. He maintains that managerial attempts to drive technology adoption must adhere to these principles. Co-invention, therefore, implies that ICT adoption and usage will be particularly driven by effective management. Co-invention can be accomplished either through innovative activity by users or by third parties. For example, third parties such as ICT outsourcing firms or ISPs may have economies of scale advantages because of their ability to spread the fixed costs of innovation across multiple clients (Ang and Straub, 1998; Greenstein, 2001). Use of third-party resources may be less costly in cities since such locations may have thicker labor markets for complementary services or specialized skills such as outsourcing. In such cases, thicker markets lower the price of obtaining workers to perform development activities in-house and/or lower the price of obtaining coinvention services such as contract programming. Forman et al. (2006) show that, if third-party resources are less costly in cities, adoption of frontier technology will be faster, other things equal. Wheeler (2005) shows that computer adoption is increasing in the employment of a firm’s local industry even controlling for the population of a city, suggesting that these third-party resources may be industry specific. Firms may utilize internal or external channels or both when adapting new ICT. In other words, these channels substitute for one another (Forman et al., 2006). Forman et al. show that organizations with little ICT capabilities but that are located in a city are at least as likely to adopt new ICT as similar organizations with greater capabilities but that are located in smaller areas. Thus, a variety of papers have demonstrated how co-invention costs shape the diffusion of new information technology across firms, industries, and locations. While these papers have demonstrated the importance of co-invention to new technology adoption, relatively little work has explored how firms overcome co-invention barriers. One reason is the stringent data requirements required for such analysis. For example, to identify how ICT outsourcing influences co-invention costs, the econometrician faces the daunting task of identifying two simultaneous discrete decisions: outsourcing and adoption. In the absence of quality instruments, such analysis requires a time series of data, preferably in a single-industry setting. One such paper is Borzekowski (2004), which examines the joint decision to outsource ICT and adopt Internet technology by credit unions. Borzekowski demonstrates the importance of controlling for unobserved adopter heterogeneity, which
28
C. Forman and A. Goldfarb
has a significant impact on the outsourcing decision. He demonstrates that, controlling for buyer type, the decision to outsource has little effect on the Internet adoption decision. Despite the wealth of case study evidence, there is still much research to be done in this area. With the exception of Bresnahan and Greenstein (1996), little work has examined how co-invention shapes the diffusion of other ICT. The co-invention necessary to adopt Internet technology is likely to differ substantially from that in the transition from mainframe to client/server. For example, a current challenge for many firms is the electronic integration of vertical supply chains (e.g., Reddy and Reddy, 2001). Besides the obvious technical challenges of enabling communication between heterogeneous software systems, a major challenge to these integration efforts are incentive problems related to the misuse of this information by supply-chain partners. Co-invention is also likely to differ across industries and, as noted above, may be based on the location of the firm and its external environment. In other words, more work needs to explore how co-invention shapes the diffusion of ICT across different industries and locations. In addition, more empirical work is needed to understand how firms undertake co-invention. When do firms decide to develop new ICT projects in-house, and when do they rely on third parties? Do spillovers play an important role in obtaining new ideas necessary for co-invention? How do a firm’s co-invention activities evolve after initial adoption? 5.2 Strategic issues in technology adoption Firms often face strategic trade-offs in the decision of when to adopt a new technology. A rival firm’s adoption may influence its own adoption. New strategic opportunities may depend on technology adoption choices. For example, Debruyne and Reibstein (2005) show that retail investment brokers adopt e-commerce in response to similar rivals. In particular, all else equal, a particular broker is much more likely to adopt e-commerce in the quarter after its closest rival adopts.24 In another exploration of the strategic value of adoption, Sadowski et al. (2002) examine the Internet adoption decisions of 264 small and medium Dutch enterprises. They use a rank (logit) framework to show that adoption is not based on current strategic factors such as intensity of competition, but on potential strategic opportunities relating to future communication requirements. Management adopts the technology in anticipation of future needs. One recent area of research in the ICT investment and adoption literature has used real options analysis to understand the optimal timing of ICT investment decisions. These studies argue that traditional financial 24 In Section 6, we consider additional ways in which one user’s adoption may influence the adoption decisions of others.
Ch. 1. Diffusion of Information and Communication Technologies
29
analysis of IT investment projects such as NPV is inadequate for IT projects with inherently high uncertainty (McGrath, 1997). The value of such options will depend upon internal organizational and external factors. Dos Santos (1991) makes an early case in the IS literature for why NPV analysis undervalues ICT investment projects and provides a numerical example of how real options analysis can improve ICT investment decisions. Kumar (1996) shows that, unlike financial options, the value of real options can go up or down with the variance of second-stage projects. Benaroch and Kauffman (1999, 2000) provide a case for using real options analysis to model the timing of ICT investment decisions and show how the Black–Scholes and binomial pricing models can be used for this purpose. Further, they apply an approximation of the Black–Scholes model to study a real investment decision, the Yankee 24 banking network’s decision to provide point-of-sale (POS) debit card network to member firms. They further discuss the assumptions needed to use models designed for the pricing of financial instruments within the setting of an ICT investment decision, and demonstrate the robustness of real options analysis to changes in these assumptions. Other research has used Margrabe’s (1978) formula for valuing the exchange of one asset for another to evaluate IT investment decisions. Kumar (1996, 1999) has used the Margrabe formula to value decision support systems (DSS). Taudes et al. (2000) uses the Margrabe formula to examine the decision to adopt SAP R/3 at a central European manufacturing firm. Schwartz and Zozaya-Gorostiza (2003) argue that application of the Black–Scholes model involves strong assumptions that are sometimes not consistent with reality. For example, the Black–Scholes model only applies to European options, while most real options are American options since managers have discretion over when to exercise the option. Further, Black– Scholes assumes there is an underlying tradable asset, while most IT investment projects do not typically involve tradable assets. Schwartz and Zozaya-Gorostiza develop new models for valuing IT investments. They develop separate models for when managers wish to value IT projects in which they develop an IT asset and for when they wish to acquire an IT asset, depending upon the time it takes to start benefitting from an IT asset after the initial investment. Fichman (2004) develops a conceptual model that draws from both the real options and ICT adoption perspectives. Drawing upon four streams of research from the innovation literature—technology strategy, organizational learning, innovation bandwagons, and technology adaptation—he discusses the factors that lead to increases and decreases in real options value. One potential area for future research could be to test some of these assumptions. This would require a broader cross-section analysis than has generally been used, however, and would be challenging because of the detailed data required to execute options-pricing analysis.
30 6
C. Forman and A. Goldfarb
Network effects
The term ‘‘network effect’’ has been used to describe a number of different phenomena in the literature. Researchers have used ‘‘network effects’’ to refer to three distinct concepts: direct network externalities, indirect network externalities, and social network effects (see Table 5). In the economics literature, a ‘‘network effect’’ is another name for a positive externality. For this reason, we will refer to this type of network
Table 5 Summary of research on network effects (Section 6) Open question
Papers
Results
How do network externalities shape technology adoption? (theory work) (see Section 6.1)
Farrell and Saloner (1986a, 1986b), Katz and Shapiro (1986), Riggins et al. (1994), Wang and Seidmann (1995), Nault and Dexter (1994), Dai and Kauffman (2006), others
Do network effects shape the adoption of ICT? (see Section 6.2)
Goolsbee and Klenow (2002), Goldfarb (2006), Bell and Song (2004), Gurbaxani (1990), Iacovou (1995), Chwelos et al. (2001), Premkumar and Ramamurthy (1995), Premkumar et al. (1994), Hart and Saunders (1997, 1998), Bertschek and Fryges (2002), Forman (2005) Saloner and Shepard (1995), Kauffman et al. (2000), Gowrisankaran and Stavins (2004), Augereau et al. (2004).
When network externalities are present, the private benefits of adoption differ from the social benefits. This can lead to a variety of suboptimal equilibrium outcomes, including ‘‘excess inertia’’ and ‘‘excess momentum’’ The probability of adopting by a given date is positively related to the proportion of firms in the peer group that have already adopted. However, most research has been unable to identify whether this empirical pattern is caused by network externalities, knowledge spillovers, or other ‘‘bandwagon effects’’ Larger network size increases the speed of adoption when there exist common standards, other things equal. When standards are not set and can be used to differentiate, adoption may exhibit negative network externalities
How do network externalities shape the adoption of ICT? (see Section 6.3)
Note: First column describes major questions that explore how network effects influence ICT adoption. Second column describes some of the major papers that have addressed these issues. Third column describes a generalization of the results of these papers.
Ch. 1. Diffusion of Information and Communication Technologies
31
effect as a ‘‘network externality.’’ There are two types of positive externalities: direct and indirect. The simplest example of a technology that exhibits direct network externalities is the telephone. If only one person owns a telephone, it has no value. The benefits of using a telephone are increasing in the number of other users. An indirect network externality exists when increasing consumption of a good leads to the provision of complementary goods.25 For example, more people adopt a video game system when more firms produce content for it, and similarly more firms produce content for a system when more people adopt it. Another example of this type of network externality is a local Yellow Pages directory. More people use it if there are more advertisers, and more people advertise if there are more users (Rysman, 2004). While software and operating systems often display such indirect network effects related to complementary inputs such as compatible software and user skills (e.g., Gandal, 1994; Shapiro and Varian, 1999), information-processing ICT does not exhibit direct network effects. This is a key difference between informationprocessing IT and ICT. In sociology and communications, ‘‘network effects’’ usually refer to the communication of ideas through social ties. The rapid diffusion of Hotmail e-mail is an example of social network effects. New customers learned about the product from friends through e-mail. Rogers (1995) gives a detailed literature review. These models are driven by the importance of personal interaction in learning about a new technology. As mentioned in Section 2.1, epidemic models of diffusion rely on communication of ideas through social ties, though modern work generally attempts to measure social network effects using probit models. Manski (1993) discusses the difficulties of econometrically identifying social network effects separately from underlying similarities among people in the same communication network or from positive externalities. Recent literature on ICT diffusion has focused on identifying some type of network effect, but has been unable to separately identify social network effects from network externalities. In this section, we first give a brief review of the theoretical literature on network externalities. We then discuss papers that examine network effects that may arise either from social network effects or from network externalities. These papers do not separately identify the source of network effects. Our focus in this section will be on empirical work that identifies any type of network effect using revealed preference data on adoption decisions; however, we also mention some papers that use a survey-based approach. Finally, we describe the small number of papers that have identified positive network externalities in ICT diffusion using data on adoption decisions.
25
Rohlfs (2001) provides numerous case studies of direct and indirect network externalities.
32
C. Forman and A. Goldfarb
6.1 Theoretical literature on direct and indirect network externalities A large theoretical literature has developed showing the implications of network externalities. This literature has shown how network externalities can lead to under-adoption of a new technology or adoption of a technologically inferior product, and has examined at length how network externalities can influence supplier strategies, including the decision of whether to produce compatible or incompatible products. This literature is far too extensive to survey here; for comprehensive reviews, see Farrell and Klemperer (2004) and Spulber (2006). Instead, we briefly review early results on how network externalities influence technology adoption, as these are the results that have been empirically tested most frequently. We also note several theory papers in the IS literature that have examined adoption of interorganizational systems (IOS). These latter works provide an interesting contrast with empirical work on the adoption of IOS, as we shall see below. When network externalities are present, user adoption influences the utilities of past and future users of a technology; however, these externalities are not internalized. In other words, the private benefits of adoption differ from the social benefits. This diversion of private benefits from social ones can engender a number of equilibrium outcomes that are not social welfare maximizing. For example, it can create ‘‘excess inertia’’ when an industry is trapped in an inferior standard, and can also create ‘‘excess momentum’’ when users move to a new standard that ‘‘strands’’ users of the existing standard (Farrell and Saloner, 1986a,b). Katz and Shapiro (1986) show that sponsored technologies have a strategic advantage over technologies that are not sponsored and may be adopted even when they are inferior.26,27 A number of theory papers in the IS literature have looked at the role of network externalities in adoption of EDI and other IOS. A common theme is the presence of negative network externalities: increases in network size decrease the value of adopting EDI. These negative externalities are similar to what Karshenas and Stoneman (1993) refer to as stock effects.28 Riggins et al. (1994) show that in buyer-driven networks, buyers may provide a subsidy in the second period of a two-period game to encourage marginal suppliers to adopt. These latter-stage subsidies may distort first-period adoption incentives for suppliers, leading to slower network growth and 26 A sponsor is an entity with property rights to a technology and who may make investments to promote it. 27 Choi and Thum (1998) extend Katz and Shapiro (1986) by examining how waiting alters their conclusions. Further, they find that consumers do not appropriately value network effects and adopt too early. Au and Kauffman (2001) extend Choi and Thum’s (1998) work to explain the adoption of electronic billing and find that, due to network externalities, agents may adopt a new technology too early even when the next technology may be superior. 28 Karshenas and Stoneman (1993) show that stock effects play little role in explaining the diffusion of CNC machine tools in the UK.
Ch. 1. Diffusion of Information and Communication Technologies
33
lower buyer profits. Wang and Seidmann (1995) also examine the impact of competitive externalities in the adoption of EDI systems. Like Riggins et al. (1994), they argue that supplier benefits from adopting EDI will be decreasing in the number of other suppliers who adopt. Also, a buyer’s profits will be increasing in the number of suppliers that adopt; however, the marginal profit from supplier adoption decreases monotonically with the number of adopters. They show that buyers will offer a price premium to suppliers that adopt EDI, while prices offered to non-adopters will fall from their ex ante levels. This leads to a concentration in the production of upstream inputs, a result that is consistent with empirical evidence showing that EDI use tends to decrease the number of suppliers that are used. They further examine conditions under which buyers may require sellers to adopt or provide a subsidy to encourage adoption. Other research has examined how network externalities influence the adoption of other kinds of IOS. Nault and Dexter (1994) examine how franchise agreements influence electronic network size and franchise incentives for investment. Parthasarathy and Bhattacherjee (1998) find that the presence of indirect network externalities can reduce the likelihood that an adopter of online services will eventually discontinue use of the service. While traditional theoretical work that has examined the adoption of IOS has focused on the adoption of proprietary EDI applications, new businessto-business applications based on Internet protocols create new decisions for buyers adopting IOS. Buyers can choose to adopt extranet systems that are based on Internet protocols but which maintain the ‘‘one buyer to many sellers’’ characteristics of traditional EDI systems. Alternatively, they can opt to join electronic markets that lower the search costs of identifying low prices but which may also provide lower incentives for suppliers to make non-contractible investments. Dai and Kauffman (2006) examine this tradeoff, finding that a buyer’s decision on an e-procurement approach will depend upon the importance of four factors: (1) lower search and operation costs; (2) the importance of information sharing between suppliers; (3) the extent of competition in the supplier market; and (4) the desired levels of supplier relationship-specific investments. 6.2 Evidence of network effects of any kind in ICT Empirical research in network effects faces two inherently challenging identification problems. First, observed statistical correlations in user behavior in cross-sectional regressions may be the result of an underlying relationship between one user’s adoption decision and another’s, or may simply reflect common unobserved factors that increase the likelihood that both users adopt. For example, if an econometrician observes that two firms in the same location adopt ICT, this result may be due to network effects or to unobserved lower adoption costs in that location. In the
34
C. Forman and A. Goldfarb
absence of long panels of data with sufficient cross-sectional heterogeneity, these alternative explanations are not usually separately identified. Second, even when network effects are themselves identified, it is often the case that the source of the network effects is not. In this case, the term ‘‘bandwagon effects’’ is often used. Bandwagon effects may be the result of network externalities, social network effects, or even competitive effects. Stated succinctly, the bandwagon hypothesis argues that ‘‘the probability of adoption by a firm at a given date is positively related to the proportion of firms in the industry that have already adopted’’ (Jensen, 1982). Goolsbee and Klenow (2002), Bell and Song (2004), and Goldfarb (2006) examine the impact of network effects on Internet adoption by consumers. Goolsbee and Klenow (2002) use instrumental variables estimation to examine the importance of local spillovers such as learning and network externalities on consumer home PC adoption. They show that these spillovers are connected to Internet usage and argue that this provides evidence of network effects in adoption. Though they are able to demonstrate that network effects exist, they are unable to show whether they are the result of direct externalities related to the use of -email or the Internet, or whether they are related to learning spillovers. In other words, they are unable to separately identify network externalities from social network effects. Goldfarb (2006) is also able to identify network effects without separately identifying these phenomena. He shows that the impact of prior university attendance on Internet use is much higher for people who attended university in the mid-1990s than for others. This is not true of other computing technologies such as word processing. Universities may have taught students to use the Internet, suggesting a social network effect. Alternatively, network externalities may be the driving factor: students may have an extra benefit from using the Internet because they know more people online. Bell and Song (2004) find strong neighborhood effects in the adoption of online grocery services, but they do not separately identify social network effects from spurious correlation due to the fact that people with similar preferences often choose to live in the same neighborhoods. A number of papers have examined network effects using a survey approach. Iacovou et al. (1995) develop a model using Rogers’ (1995) diffusion theories to understand how organizational readiness, external pressure, and perceived benefits influence EDI adoption and examine the applicability of this framework using seven case studies. They show that external pressure from trading partners plays an important role in determining whether an organization adopts EDI. Chwelos et al. (2001) extend this model and use it to examine EDI adoption by a group of over 300 Canadian purchasing managers. They break the effects of external pressure into four constructs: competitive pressure, dependency on trading partner, trading partner power, and industry pressure. They find that external pressure plays a significant role in explaining EDI adoption; in particular, they find competitive pressure and trading partner power-influenced adopter intentions. This research supported
Ch. 1. Diffusion of Information and Communication Technologies
35
findings in earlier work on EDI adoption that found a role for competitive pressure and trading partner power (e.g., Premkumar and Ramamurthy, 1995; Hart and Saunders, 1997, 1998; Premkumar et al., 1994).29 Surprisingly, despite the extensive body of work that has examined how network effects and competitive pressure has influenced EDI adoption, relatively little work has examined how such factors influence adoption of Internet technology by firms. Currently, firms are migrating from traditional EDI to Internet-enabled supply-chain applications (Saloner and Spence, 2002). Like EDI, these applications automate the transfer of information between trading partners, saving on labor costs and decreasing error rates. However, by integrating with existing enterprise applications, they also allow for the possibility of real-time knowledge of production and inventory levels of trading partners. A small number of papers have begun to investigate the role of network effects on Internet-enabled IOS. Bertschek and Fryges (2002) find that bandwagon effects play a role in the adoption of business-to-business electronic commerce among 3,000 German firms, while Forman (2005) finds they influence adoption of advanced business applications in a sample of over 6,000 US firms.30 Neither is able to identify between competing explanations for bandwagon effects nor to control for unobservable differences in preferences across industries or locations. Zhu et al. (2006) use survey methods to show that network effects significantly influence firm decisions to adopt Internet-based IOS. There exists a disconnect between the theory and empirical literatures on how competitor adoption influences the speed with which organizations adopt IOS. While theory work emphasizes how negative network externalities reduce the incremental benefit of adopting IOS for later adopters, empirical work focuses on the role of bandwagon effects, generally finding a positive relationship between competitor adoption and the speed with which an organization adopts IOS. Researchers that wish to reconcile these findings will need to identify network effects from unobserved heterogeneity.
6.3 Evidence of positive network externalities in ICT A handful of studies have separately identified network externalities from social network effects. These studies have examined adoption of enabling network infrastructure that is subject to strong network externalities but which is unlikely to be subject to social network effects. 29 For a recent example of how external pressure may influence EDI adoption as viewed through a sociological lens, see Teo et al. (2003). They show that mimetic pressures, coercive pressures, and normative pressures all have a significant influence on firm decisions to adopt EDI. 30 Another exception is Lee and Cheung (2004), who find that environmental factors are one of the key drivers of Internet retailing.
36
C. Forman and A. Goldfarb
As noted above, network externalities arise when the value for participating in the network increases in network size. Saloner and Shepard (1995) examine how network size increases the speed with which commercial banks adopt proprietary ATM technology during 1971–1979. Using the number of bank branches as a proxy for network size, they find that banks with many ATM branches adopt ATM technology earlier than banks with fewer branches (controlling for the number of depositors), suggesting the presence of network effects. Kauffman et al. (2000) examine banks’ decisions to join the Yankee 24 electronic banking network. In contrast to Saloner and Shepard (1995), they examine how the potential size of an interorganizational banking network influences the decision to adopt a new ATM technology. They demonstrate that, other things equal, larger potential network size increases the speed of adoption; they also show that the banks with a larger investment in proprietary network technology adopt more slowly because of lower net benefits from an interorganizational network. While the papers above attempt to identify how potential network size influences the adoption of new ICT through network externalities, a second class of papers seeks to explicitly measure the externality that arises from user adoption. Rigorously identifying whether such network externalities exist is difficult for the reasons described above. As Gowrisankaran and Stavins (2004) note, time series data are inadequate because price and costs are decreasing over time, while quality is increasing. Use of cross-sectional data also presents problems since local differences may be caused by unobservable heterogeneity in supply or preferences. In general, identification is only possible when the analyst has long panels with sufficient crosssectional heterogeneity. Gowrisankaran and Stavins (2004) utilize a panel data set to demonstrate that network externalities influence commercial banks’ adoption of automated clearing house (ACH) electronic payments systems. In their application, direct network effects are likely the most important. For ACH to work, bill payers and payees have to coordinate on a method of payment, and their banks must support that method. They use three separate reduced-form analyses to demonstrate the presence of network effects: (1) fixed-effects estimation, (2) instrumental variables, and (3) a quasi-natural experiment using the adoption decisions of small, remote branches of banks. Tucker (2005) examines how network externalities shape intra-firm adoption of videoconferencing technology. Her very detailed data enable her to measure communication patterns within a firm, and to show that network effects arise from externalities related to communication with other firm employees, rather than spillovers related to learning. Network externalities arise only for adoption by employees that communicate with likely adopters frequently. Moreover, the adoption decisions of employees who are ‘‘information brokers’’ that communicate directly with many other employees are most important in shaping others’ adoption decisions. Moreover, she
Ch. 1. Diffusion of Information and Communication Technologies
37
finds that the influence of network externalities varies across individuals. Tucker’s research is the first in the economics literature to empirically identify such heterogeneity in the causes of network externalities and how they influence adoption behavior. Network externalities have also played a role in the adoption of enabling Internet technologies. Augereau et al. (2004) show how network externalities can influence users’ decision of which standard to adopt. They examine the adoption of competing 56 K modem technologies by ISPs in 1997. Prior to standardization by the International Telecommunication Union (ITU), these model standards were incompatible. Adoption and choice of 56 K modem technology by ISPs was influenced by network effects: the value of adopting a particular model technology was increasing in the number of consumers with 56 K modems and the particular technology they had adopted; however, the value of standard adoption was decreasing in the number of competitors offering the same standard. Network effects in this market operate at the local level because of flat-rate local pricing of telephone service in the US. Augereau et al. demonstrate that network effects have a significant impact on consumers’ choice of model technology in this market. Further, they show that despite these network effects, local ISP markets failed to standardize, as ISPs chose to differentiate their offerings rather than conform to a common standard. In other words, while consumer adoption of a 56 K modem standard exhibited positive externalities, firm adoption exhibited negative externalities. Research on network externalities has important implications for public policy. The positive network externalities associated with ICT imply that the private benefits to adoption may be lower than the overall welfare effect of an individual adoption decision. Subsidies then can lead to improved efficiency. Implementation of this idea, however, is difficult. If people who receive the subsidy would have adopted anyway, then the tax distortions from the subsidy may outweigh the benefits in overcoming the externality. Therefore, while there is a theoretical argument for government policy to subsidize ICT, it is difficult to implement in practice.
7
Internet diffusion across countries
A number of studies have sought to understand cross-country differences in the rate of Internet diffusion. These studies argue that if Internet use has a substantial positive impact on the rate of productivity growth, then crosscountry differences in Internet use can exacerbate existing income inequalities between countries. However, some authors have argued that adoption of ICT such as the Internet can offer the opportunity for late-industrializing countries to catch up with richer countries (e.g., Kagami and Tsuji, 2001; Steinmuller, 2001) (see Table 6).
38
Table 6 International diffusion of internet technology (Section 7) Papers
Results
What is the relationship between per capita income and Internet diffusion?
Comin and Hobjin (2003), Beilock and Dimitrova (2003), Chinn and Fairlie (2004), Kiiski and Pohjola (2002), Hargittai (1999), Wong (2002), Dewan et al. (2005) Coming and Hobjin (2003), Beilock and Dimirova (2003), Oxley and Yeung (2001), Chinn and Fairlie (2004), Hargittai (1999), Kenney (2003), Wallsten (2003)
Per capita income is positively correlated with Internet diffusion
What is the relationship between government institutions and Internet diffusion?
What is the relationship between country-level education and Internet diffusion? What is the relationship between existing telecommunications infrastructure and Internet diffusion? What is the relationship between English language use and Internet diffusion? (How) can developing countries use ICT to catch up to developed countries?
Chinn and Fairlie (2004), Kiiski and Pohjola (2002), Dewan et al. (2005) Beilock and Dimitrova (2003), Chinn and Fairlie (2004), Kiiski and Pohjola (2002), Hargittai (1999), Dewan et al. (2005) Hargittai (1999), Kiiski and Pohjola (2002), Gandal (2006) Antonelli (2003), James (2003)
Internet diffusion has been faster among countries that are democratic, those with political freedom, and that have rule of law. Telecommunications and regulatory policy also play a role Internet use will diffuse more quickly to countries with higher education Internet diffusion is faster in countries with better telecommunications infrastructure. The relationship between Internet diffusion and telecommunications prices is less clear There is no clear consensus in the literature
There is no clear consensus in the literature.
Note: First column describes major questions that explore the international diffusion of Internet technology. Second column describes some of the major papers that have addressed these issues. Third column describes a generalization of the results of these papers.
C. Forman and A. Goldfarb
Open question
Ch. 1. Diffusion of Information and Communication Technologies
39
One of the most persistent findings in all of these studies is that Internet use is correlated with per capita income: Internet technology diffuses fastest in rich countries (e.g., Hargittai, 1999; Kiiski and Pohjola, 2002; Wong, 2002; Beilock and Dimitrova, 2003; Comin and Hobijn, 2003; Chinn and Fairlie, 2004; Dewan et al., 2005). This is true even when one compares differences in adoption rates within groups of industrialized countries such as the OECD (Kiiski and Pohjola, 2002) and geographic regions such as Asia (Wong, 2002). Antonelli (2003) provides one explanation for this finding, arguing that new technological change may be ‘‘biased’’ in that it may increase the productivity of inputs with relatively low factor prices in the country of origin. Antonelli argues that ICT fits this profile, as it is a skilled labor and capital-intensive technology, i.e., unskilled labor saving. Thus, he argues it is much better suited for developed countries such as the US than for developing countries. Antonelli argues that ICT is complementary with many of the managerial and organizational changes listed in Section 3.1 and that have been undertaken in the US As a result, he argues that new innovations in ICT work to increase the competitive advantages of developed nations such as the US. James (2003) argues that ICT produced with poor countries in mind can overcome this difficulty. He argues that much of the current framework for thinking of ICT adoption is inappropriate for developing countries. Several studies have also sought to understand how a country’s governmental institutions can encourage or discourage the diffusion of new ICT such as the Internet. Beilock and Dimitrova (2003) and Comin and Hobijn (2003) argue that ICT diffuses more quickly to democratic countries and those with political freedom, while Oxley and Yeung (2001) argue that rule-of-law plays a major role. Hargittai (1999), Kenney (2003), Wallsten (2003), and Chinn and Fairlie (2004) show that telecommunications and regulatory policy are also important in explaining crosscountry diffusion rates. Some studies have argued that Internet use will diffuse more quickly to countries with higher education (Kiiski and Pohjola, 2002; Chinn and Fairlie, 2004; Dewan et al., 2005). Telecommunications infrastructure also influences the rate of Internet diffusion (Beilock and Dimitrova, 2003; Chinn and Fairlie, 2004). Kiiski and Pohjola (2002) argue that Internet access prices are an important determinant of the rate of Internet diffusion. In contrast, Hargittai (1999) finds they play little role. Dewan et al. (2005) show that telephone costs influence the rate of Internet diffusion; this is particularly true at low levels of penetration. Another factor that may influence Internet use is language. Since most web sites are in English, English language countries may have an advantage in Internet use. Several cross-country studies of the Internet have explored the role of English language use in diffusion (e.g., Hargittai, 1999; Kiiski and Pohjola, 2002); however, these studies have generally been unable to
40
C. Forman and A. Goldfarb
uncover a systematic role for language. The role of English in Internet adoption remains an open question.31 Overall, the empirical evidence supports the view that, in the short run at least, Internet technology has diffused fastest to industrialized nations with greater income, more open political institutions, and with moredeveloped telecommunications infrastructures. At this aggregate level of analysis, these studies suggest that there is an international digital divide between rich and poor country use of the Internet. Some authors argue, however, that this divide may be narrowing over time (Dewan et al., 2005). One challenge faced by researchers studying the cross-country digital divide is the difficulty in controlling for unobserved cross-country differences. Countries differ in complex political, cultural, and economic ways that are impossible to control for in any regression. Developed and developing countries are particularly different. Hypothesized relationships are difficult to identify because covariates may pick up other unobserved differences across countries. Some researchers have attempted to address the problem of unobserved country-level differences by comparing countries within similar global regions or demographics. Kiiski and Pohjola (2002) examine OECD countries, while Kraemer and Dedrick (2002) and Wong (2002) show there exists significant variation in Internet use among Asian countries.32 While such studies must still face some residual inter-country differences, they do allow for easier comparisons among countries than studies that are not so restricted. Other studies have sought to understand how income and other factors affect the use of Internet technology at different points along the diffusion path (Dewan et al., 2005). By examining how Internet use is shaped among Internet-intensive countries and non-Internet-intensive countries, this research is able to compare Internet use among countries that are more similar, removing some of the problems of unobserved heterogeneity. Another strategy that has been used to understand the factors influencing country-level Internet investment is to pursue case studies on individual countries. For example, Brousseau and Chaves (2004) compare the diffusion of Internet-based e-commerce technology in France to that in Denmark and Germany. One finding in their paper is that the existence of a prior e-commerce technology that preceded the Internet influenced the speed with which French businesses and individuals adopted Internet technology and the manner in which they used Internet technology. Tachiki 31 Other studies have examined how national cultural differences impact ICT adoption. For example, Png et al. (2001) demonstrate in a multinational study that high uncertainty avoidance lowered the speed of frame relay adoption. 32 Kauffman and Techatassanasoontorn (2005) have found evidence of multinational regional contagion in the diffusion of wireless technology.
Ch. 1. Diffusion of Information and Communication Technologies
41
et al. (2004) study the diffusion of e-commerce in Japan.33 They find that keiretsu adopted business-to-business technologies quickly but were slower than small- and medium-sized enterprises at adopting business-to-consumer technologies. Finally, many studies have addressed country-level differences by shifting the unit of observation from the country to the organization or, in some cases, to the individual. Some studies have examined how factors such as telecommunications infrastructure, regulatory policy, and education influence an organization’s decision to adopt ICT in a single-country setting. Gandal (2006) uses this strategy to examine the role of language on ICT diffusion. He studies Internet use in Quebec, whose population speaks both English and French. He finds that Internet use among French-speaking 20–45-year-olds in Canada is significantly lower than that of similar English speakers. Several US studies have also examined how factors such as education and government policy influence Internet adoption. For example, Goldfarb (2006) examines the role of education on Internet use within the US, while Goolsbee (2000) shows how high local sales taxes in the US made online purchasing more appealing. While single-country econometric studies represent a useful way of examining the factors influencing Internet penetration, thus far such studies have used primarily data from developed countries. However, some research has argued that the theoretical models created for industrialized countries (and the associated empirical results) may not apply to developing countries because of differences in culture, regulatory climate, and economic environment (e.g., Austin, 1990; Rosenzweig, 1994; Xu et al., 2004). To overcome this difficulty, several studies have examined organizationlevel adoption of Internet technology using mixed samples of firms from developed and developing countries (Zhu et al., 2002; Xu et al., 2004; Zhu and Kraemer, 2005). These studies confirm the importance of the local regulatory environment on e-business. Moreover, they also find a role for organizational factors, such as firm size and scope, technology competence, and competitive pressure. More studies that include developing country data are needed. In summary, there is considerable evidence of variance in Internet use across countries, suggesting the presence of what some authors have titled an international digital divide. Though differences in income explain most of this variance, a variety of other factors have been found to explain the residual not explained by income differences. Identifying the causes and consequences of this divide is difficult because of unobserved country-level heterogeneity and because the frameworks created for understanding diffusion in rich countries may not apply in developing countries. A number of approaches have been used to overcome these difficulties, including focused 33 For other examples of country-level case studies of e-commerce diffusion, see Kraemer et al. (2002), Tan and Ouyang (2004), Tigre (2003), Wong and Ho (2004).
42
C. Forman and A. Goldfarb
empirical approaches that look at similar countries or that look at Internet use within a particular country, and case studies that are able to explain in detail how the Internet is used within a country. Considerable progress has been made, but much more work needs to be done to resolve the conflicting evidence in the current literature. 8
Conclusion
In this review, we have examined how internal organizational factors and external factors such as geography and network effects influence business adoption of ICT. Although research on diffusion has a long and rich history, the communication capabilities of ICT have led to a number of new research topics. In particular, the rapidly developing communication capabilities of ICT over the past 20 years have given rise to new ways that ICT can influence the economic and geographic relationships between firms. Adoption research provides one lens through which to understand who benefits most from new capabilities. It examines how factors such as organizational investments, economic relationships, and geographic location shape the returns to new ICT adoption. Such studies form a useful complement to other approaches, such as business-value studies, that also explore the economic effects of ICT. We have explored the diffusion of ICT from a variety of perspectives; however, it is apparent that research has only scratched the surface on many of these topics. As noted above, one reason is that many of these questions have been driven by new developments in ICT. Tests of a technology’s ability to reduce the costs of geographic isolation were only relevant with the lower costs of communication engendered by ICT in general, and the Internet in particular. Moreover, the micro data sets necessary to test many of these hypotheses have only recently become available: by its very nature, adoption research requires detailed data on a heterogeneous pool of economic agents. Our understanding of ICT adoption will continue to improve as more micro data sets become available. This review has highlighted a number of issues that require additional exploration. While there is an understanding that internal factors matter, there is little work that provides a deep understanding of how co-invention occurs in ICT adoption. Moreover, additional research is needed to understand the relationship between ICT investment and the economic organization of firms. Research in this area would complement existing business-value work on the same topic. Moreover, additional research is needed to develop an understanding of the intensive margin of ICT investment beyond adoption. The core remaining questions about the impact of location on adoption relate to the effect of Internet use on isolated areas. Does Internet access substitute for cities? Do we see an increase in telecommuting? How does new ICT influence the location decisions of firms?
Ch. 1. Diffusion of Information and Communication Technologies
43
The core problem with respect to network effects involves separately estimating network externalities from social network effects. This is an excellent topic for future research. Further research is also needed to understand cross-country differences in the diffusion of Internet technology: additional empirical and case study research that examines Internet diffusion within countries other than the US would be especially helpful in this regard. Finally, there are many opportunities to study how government policy influences ICT diffusion. From India to the United States, numerous government policies have been implemented over the last 10 years that aim to encourage ICT adoption. Solid empirical work estimating the impact of these policies will provide us with an important understanding of what works and why.
Acknowledgments We thank Shane Greenstein and Terry Hendershott for comments and advice, and Kristina Steffenson McElheran for outstanding research assistance. All opinions and errors are ours alone.
References Acemoglu, D., P. Aghion, R. Griffith, F. Zilibotti (2004). Vertical integration and technology: theory and evidence. NBER Working Paper #10997. Ang, S., D. Straub (1998). Production and transaction economies and IS outsourcing: a study of the U.S. banking industry. MIS Quarterly 22, 535–552. Antonelli, C. (2003). The digital divide: understanding the economics of new information and communication technology in the global economy. Information Economics and Policy 15, 173–199. Astebro, T. (1995). The effect of management and social interaction on the intra-firm diffusion of electronic mail systems. IEEE Transactions on Engineering Management 42, 319–331. Astebro, T. (2002). Noncapital investment costs and the adoption of CAD and CNC in U.S. metalworking industries. RAND Journal of Economics 33, 672–688. Astebro, T. (2004). Sunk costs and the depth and probability of technology adoption. Journal of Industrial Economics 52, 381–399. Athey, S., S. Stern (2002). The impact of information technology on emergency health care outcomes. RAND Journal of Economics 33, 399–432. Athey, S., S. Stern (2003). An empirical framework for testing theories about complementarity in organization design. Working Paper, Northwestern University. Atrostic, B.K. S.V. Nguyen (2002). Computer networks and U.S. manufacturing plant productivity: New evidence from the CNUS data. Working Paper #02-01, Center for Economic Studies, U.S. Census Bureau. Attewell, P. (1992). Technology diffusion and organizational learning: the case of business computing. Organization Science 3, 1–19. Attewell, P., J. Rule (1984). Computing and organizations: what we know and what we don’t know. Communications of the ACM 28, 1184–1192. Au, Y. R. Kauffman (2001). Should we wait? Network externalities, compatibility, and electronic billing adoption. Journal of Management Information Systems 18 47–63.
44
C. Forman and A. Goldfarb
Augereau, A., S. Greenstein (2001). The need for speed in emerging communications markets: upgrades to advanced technology at internet service providers. International Journal of Industrial Organization 19, 1085–1102. Augereau, A., S. Greenstein, M. Rysman (2004). Coordination vs. differentiation in a standards war: 56 K modems. NBER Working Paper # 10334. Austin, J.E. (1990). Managing in developing countries. Free Press, New York. Autor, D., F. Levy, R.J. Murname (2002). Upstairs, downstairs: computer-skill complementarity and computer-labor substitution on two floors of a large bank. Industrial and Labor Relations Review 55, 432–447. Baily, M.H., R.Z. Lawrence (2001). Do we have a new economy?. American Economic Review 91, 308– 312. Baker, G.P., T.N. Hubbard (2003). Make versus buy in trucking: asset ownership, job design, and information. American Economic Review 93, 551–572. Baker, G.P., T.N. Hubbard (2004). Contractibility and asset ownership: on-board computers and governance in U.S. trucking. Quarterly Journal of Economics 119, 1443–1479. Banker, R., I. Bardhan, H. Chang, S. Lin (2003). Impact of manufacturing practices on adoption of plant information systems. Proceedings of the 24th International Conference on Information Systems, pp. 233–245. Barua, A., C.H. Lee, A.B. Whinston (1995). Incentives and computing systems for team-based organizations. Organization Science 6, 487–504. Barua, A., C.H. Lee, A.B. Whinston (1996). The calculus of reengineering. Information Systems Research 7, 409–428. Barua, A., T. Mukhopadhyay (2000). Information technology and business performance: past, present, and future, in: R.W. Zmud (ed.), Framing the Domains of IT Management, Pinnaflex, Cincinnati, pp. 65–84. Barua, A., A.B. Whinston (1998). Decision support for managing organizational design dynamics. Decision Support Systems 22, 45–58. Bass, F.M. (1969). A new product growth for model consumer durables. Management Science 15, 215–227. Battisti, G., A. Canepa, P. Stoneman (2004). The diffusion of e-business activities in the UK: productivity, profitability, externalities and policy. Working Paper, University of Warwick. Battisti, G., P. Stoneman (2003). Inter- and intra-firm effects in the diffusion of new process technology. Research Policy 32, 1641–1655. Battisti, G., P. Stoneman (2005). The intra-firm diffusion of new process technologies. International Journal of Industrial Organization 23, 1–22. Beilock, R., D.V. Dimitrova (2003). An exploratory model of inter-country internet diffusion. Telecommunications Policy 27, 237–252. Bell, D.R., S. Song (2004). Neighborhood effects and trial on the internet: evidence from online grocery retailing. Working Paper, Wharton. Benaroch, M., R.J. Kauffman (1999). A case for using real options pricing analysis to evaluate information technology project investments. Information Systems Research 10, 70–86. Benaroch, M., R.J. Kauffman (2000). Justifying electronic banking network expansion using real options analysis. MIS Quarterly 24, 197–225. Bertschek, I., H. Fryges (2002). The adoption of business-to-business e-commerce: empirical evidence for German companies. Discussion Paper No. 02-05, Centre for European Economic Research. Bertschek, I., U. Kaiser (2004). Productivity effects of organizational change: econometric evidence. Management Science 50, 394–404. Bharadwaj, A. (2000). A resource-based perspective on IT capability and firm performance: an empirical investigation. MIS Quarterly 24, 169–196. Black, S.E., L.M. Lynch (2001). How to compete: the impact of workplace practices and information technology on productivity. Review of Economics and Statistics 83, 434–445. Bodensteiner. W.D. (1970). Information channel utilization under varying research and development project conditions: an aspect of inter-oganizational communication channel usages. Unpublished doctoral dissertation, University of Texas at Austin.
Ch. 1. Diffusion of Information and Communication Technologies
45
Borzekowski, R. (2004). In through the out door: the role of outsourcing in the adoption of internet technologies by credit unions. Working Paper, Board of Governors of the Federal Reserve System. Bresnahan, T., E. Brynjolfsson, L. Hitt (2002). Information technology, workplace organization, and the demand for skilled labor: firm-level evidence. Quarterly Journal of Economics 177, 339–376. Bresnahan, T., S. Greenstein (1996). Technical progress in computing and in the uses of computers. Brookings Papers on Economic Activity. Microeconomics 1996, 1–78. Bresnahan, T., M. Trajtenberg (1995). General purpose technologies: ‘engines of growth’?. Journal of Econometrics 65, 83–108. Brousseau, E., B. Chaves (2004). Diffusion and impact of e-commerce. The French specific path. Working Paper, Center for Research on Information Technology and Organizations (CRITO), University of California, Irvine. Brynjolfsson, E., L.M. Hitt (1996). Paradox lost? Firm-level evidence on the returns to information systems spending. Management Science 42, 541–558. Brynjolfsson, E., L.M. Hitt (2000). Beyond computation: information technology, organizational transformation, and business performance. Journal of Economic Perspectives 14, 23–48. Brynjolfsson, E., L.M. Hitt (2003). Computing productivity: firm-level evidence. Review of Economics and Statistics 85, 793–808. Brynjolfsson, E., L.M. Hitt, S. Yang (2002). Intangible assets: computers and organizational capital. Brookings Papers on Economic Activity 2002, 137–199. Brynjolfsson, E., T.W. Malone, V. Gurbaxani, A. Kambil (1994). Does information technology lead to smaller firms?. Management Science 40, 1628–1644. Brynjolfsson, E., A. Renshaw, M. Van Alstyne (1997). The matrix of change: a tool for business process reengineering. Sloan Management Review 38, 37–54. Cairncross, F. (1997). The Death of Distance. Harvard University Press, Cambridge. Census (2002). Detailing tabulations of manufacturing e-business process use in 2000. Survey Release from United States Department of Commerce. Available at www.census.gov/epcd/www/ebusiness.htm. Charles, S., Ivis, M., Leduc, A. (2002). Embracing e-business: does size matter? Research Report, Statistics Canada. Charlot, S., G. Duranton (2006). Cities and workplace communication: some French evidence. Urban Studies 43, 1365–1394. Chen, P., C. Forman (2006). Can vendors influence switching costs and compatibility in an environment with open standards?. MIS Quarterly 30, 541–562. Chen, P., L. Hitt (2006). Information technology and switching costs, in: T. Hendershott (ed.), Handbook on Economics and Information Systems, Elsevier, Amsterdam. Chinn, M.D., R. Fairlie (2004). The determinants of the global digital divide: a cross-country analysis of computer and internet penetration. NBER Working Paper #10686. Choi, J.P., M. Thum (1998). Market structure and the timing of technology adoption with network externalities. European Economic Review 42, 225–244. Chwelos, P., I. Benbasat, A.S. Dexter (2001). Research report: empirical test of an EDI adoption model. Information Systems Research 12, 304–321. Clemons, E., S.P. Reddi, M. Row (1993). The impact of information technology on the organization of production: the ‘‘move to the middle’’ hypothesis. Journal of Management Information Systems 10, 9–35. Coleman, J., E. Katz, H. Menzel (1957). The diffusion of an innovation among physicians. Sociometry 20, 253–270. Comin, D., B. Hobijn (2003). Cross-country technology adoption: making the theories face facts. Federal Reserve Bank of New York Staff Reports No. 169, June. Cooper, R.B., R.W. Zmud (1990). Information technology implementation research: a technological diffusion approach. Management Science 36, 123–139. Corrado, C., J. Haltiwanger, D. Sichel (2005). Measuring Capital in the New Economy. University of Chicago Press, Chicago.
46
C. Forman and A. Goldfarb
Crandall, R.W., J.H. Alleman (2002). Broadband: should we regulate high-speed internet access? AEIBrookings Joint Center for Regulatory Studies, Washington, DC. Daft, R.L., R.H. Lengel (1984). Information richness: a new approach to managerial behavior and organization design. Research in Organization Behavior 6, 191–233. Daft, R.L., R.H. Lengel, L.K. Trevino (1987). Message equivocality, media selection, and manager performance: implications for information systems. MIS Quarterly 11, 355–366. Dai, Q., R.J. Kauffman (2006). To be or not to B2B? An evaluative model for e-procurement channel adoption. Information Technology and Management 7, 109–130. Davenport, T. (1998). Putting the enterprise into the enterprise system. Harvard Business Review July– August, 121–131. David, P.A. (1969). A contribution to the theory of diffusion. Stanford Center for Research in Economic Growth, Memorandum No. 71, Stanford University. David, P.A. (1990). The dynamo and the computer: an historical perspective on the modern productivity paradox. American Economic Review 80, 355–361. Davis, F.D. (1989). Perceived usefulness, perceived ease of use, and user acceptance of information technology. MIS Quarterly 13, 319–340. Davis, F.D., R.P. Bagozzi, P.R. Warshaw (1989). User acceptance of computer technology: a comparison of two theoretical models. Management Science 35, 982–1003. Debruyne, M., D.J. Reibstein (2005). Competitor see, competitor do: incumbent entry in new market niches. Marketing Science 24, 55–66. Devaraj, S., R. Kohli (2003). Performance impacts of information technology: is actual usage the missing link?. Management Science 49, 273–289. Dewan, S., D. Ganley, K.L. Kraemer (2005). Across the digital divide: a cross-country analysis of the determinants of IT penetration. Journal of the Association of Information Systems 6, article 10. Dewan, S., S.C. Michael, C. Min (1998). Firm characteristics and investments in information technology: scale and scope effects. Information Systems Research 9, 219–232. Dhar, V., A. Sundararajan (2004). Should the core information systems curriculum be structured around a fundamental question. Proceedings of the Twenty-Fifth International Conference on Information Systems, pp. 1021–1022. Doms, M., T. Dunne, K. Troske (1997). Workers, wages, and technology. Quarterly Journal of Economics 112, 253–290. Dos Santos, B.L. (1991). Justifying investments in new information technologies. Journal of Management Information Systems 7, 71–90. Downes, T., S. Greenstein (2002). Universal access and local internet markets in the U.S.. Research Policy 31, 1035–1052. Dunlop, J.T., D. Weil (1996). Diffusion and performance of modular production in the U.S. Apparel Industry. Industrial Relations 35, 334–355. Duranton, G., D. Puga (2004). Micro-foundations of urban increasing returns: theory, in: J.V. Henderson, J.-F. Thisse (eds.), Handbook of Regional and Urban Economics, Vol. IV. Elsevier, Amsterdam, pp. 2063–2117. Eveland, J.D., L. Tornatzky (1990). The development of technology, in: L. Tornatzky, M. Fleischer (eds.), The Processes of Technological Innovation, Lexington Books, Lexington. Farrell, J., P. Klemperer (2004). Coordination and lock-in: competition with switching costs and network effects. Working Paper, Oxford University. Available at www.paulklemperer.org. Farrell, J., G. Saloner (1986a). Standardization, compatibility, and innovation. RAND Journal of Economics 16, 70–83. Farrell, J., G. Saloner (1986b). Installed base and compatibility: innovation, product preannouncements, and predation. American Economic Review 76, 940–955. Fichman, R.G. (1992). Information technology diffusion: a review of empirical research, in: Proceedings of the Thirteenth International Conference on Information Systems, Dallas, pp. 195–206. Fichman, R.G. (2000). The diffusion and assimilation of information technology innovations, in: R.W. Zmud (ed.), Framing the Domains of IT Management, Pinnaflex, Cincinnati, pp. 105–128.
Ch. 1. Diffusion of Information and Communication Technologies
47
Fichman, R.G. (2004). Real options and it platform adoption: implications for theory and practice. Information Systems Research 15, 132–154. Fichman, R.G., C.F. Kemerer (1997). The assimilation of software process innovations: an organizational learning perspective. Management Science 43, 1345–1363. Fichman, R.G., C.F. Kemerer (1999). The illusory diffusion of innovation: an examination of assimilation gaps. Information Systems Research 10, 255–275. Fitoussi, D. (2004). So close and yet so far: information technology and the spatial distribution of customer service. Proceedings of the Twenty-Fourth International Conference on Information Systems, pp. 642–653. Forman, C. (2005). The corporate digital divide: determinants of internet adoption. Management Science 51, 641–654. Forman, C., P. Chen (2005). Switching costs and network effects in the market for routers and switches. Working Paper, Tepper School of Business, Carnegie Mellon University. Forman, C., A. Goldfarb, S. Greenstein (2002). Digital dispersion: an industrial and geographic census of commercial internet use. NBER Working Paper #9287. Forman, C., A. Goldfarb, S. Greenstein (2003). The geographic dispersion of commercial internet use, in: S. Wildman, L. Cranor (eds.), Rethinking Rights and Regulations: Institutional Responses to New Communication Technologies, MIT Press, Cambridge, MA, pp. 113–145. Forman, C., A. Goldfarb, S. Greenstein (2005). How did location affect adoption of the commercial internet? Global village vs. urban leadership. Journal of Urban Economics 58, 389–420. Forman, C., A. Goldfarb, S. Greenstein (2006). Understanding the inputs into innovation: do cities substitute for internal firm resources? Working Paper, Tepper School of Business, Carnegie Mellon University. Forman, C., A. Gron (2005). Vertical integration and information technology adoption: a study of the insurance industry. Proceedings of the 38th Hawaii International Conference on System Sciences. Fudenberg, D., J. Tirole (1985). Preemption and rent equalization in the adoption of a new technology. Review of Economic Studies 52, 383–401. Gandal, N. (1994). Hedonic price indexes for spreadsheets and an empirical test for network externalities. RAND Journal of Economics 25, 160–170. Gandal, N. (2006). The effect of native language on internet use. The International Journal of the Sociology of Language, forthcoming. Gaspar, J., E. Glaeser (1998). Information technology and the future of cities. Journal of Urban Economics 43, 136–156. George, J.F., J.L. King (1991). Examining the computing and centralization debate. Communications of the ACM 34, 62–72. Gertner, R.H., R.S. Stillman (2001). Vertical integration and internet strategies in the apparel industry. Journal of Industrial Economics 49, 417–440. Goldfarb, A. (2006). The (teaching) role of universities in the diffusion of the internet. International Journal of Industrial Organization 24, 203–225. Goldfarb, A., J. Prince (2005). Internet adoption and usage patterns are different: implications for the digital divide. Working Paper, University of Toronto. Goolsbee, A. (2000). In a world without borders: the impact of taxes on internet commerce. Quarterly Journal of Economics 115, 561–576. Goolsbee, A., P. Klenow (2002). Evidence on learning and network externalities in the diffusion of home computers. Journal of Law and Economics 45, 317–343. Gowrisankaran, G., J. Stavins (2004). Network externalities and technology adoption: lessons from electronic payments. RAND Journal of Economics 35, 260–276. Greenstein, S. (1993). Did installed base give an incumbent any (measurable) advantages in federal computer procurement?. RAND Journal of Economics 24, 19–39. Greenstein, S. (2000). Building and delivering the virtual world: the commercial internet access market. Journal of Industrial Economics 48, 391–411.
48
C. Forman and A. Goldfarb
Greenstein, S. (2001). Technological mediation and commercial development in the early internet access market. California Management Review 43, 75–94. Griliches, Z. (1957). Hybrid corn: an exploration in the economics of technological change. Econometrica 25, 501–522. Gurbaxani, V. (1990). Diffusion in computing networks: the case of BITNET. Communications of the ACM 33, 65–75. Gurbaxani, V., S. Whang (1991). The impact of information systems on organizations and markets. Communications of the ACM 34, 59–73. Hall, B., B. Khan (2003). Adoption of new technology, in: D.C. Jones (ed.), New Economy Handbook, Academic Press, Amsterdam. Hannan, T., J. McDowell (1984). The determinants of technology adoption: the case of the banking firm. RAND Journal of Economics 15, 328–335. Hargittai, E. (1999). Weaving the western web: explaining differences in Internet connectivity among OECD countries. Telecommunications Policy 23, 701–718. Harris, R. (1998). The internet as a GPT: factor market implications, in: E. Helpman (ed.), General Purpose Technologies and Economic Growth, MIT Press, Cambridge, pp. 145–166. Hart, P., C.S. Saunders (1997). Power and trust: critical factors in the adoption and use of electronic data interchange. Organization Science 8, 23–42. Hart, P., C.S. Saunders (1998). Emerging electronic partnerships: antecedents and dimensions of EDI use from the supplier’s perspective. Journal of Management Information Systems 14, 87–111. Helper, S. (1995). Supplier relations and adoption of new technology: results of survey research in the U.S. auto industry. NBER Working Paper #5278. Hempell, T. (2003). Do computers call for training? firm-level evidence on complementarities between ict and human capital investments. Centre for European Economic Research (ZEW), Discussion Paper #03-20. Hinds, P., S. Kiesler (1995). Communication across boundaries: work, structure, and the use of communication technologies in a large organization. Organization Science 6, 373–393. Hitt, L.M. (1999). Information technology and firm boundaries: evidence from panel data. Information Systems Research 10, 134–149. Hitt, L.M., E. Brynjolfsson (1997). Information technology and internal firm organization: an exploratory analysis. Journal of Management Information Systems 14, 81–101. Holland, W.E., B.A. Stead, R.C. Leibrock (1976). Information channel/source selection as a correlate of technical uncertainty in a research and development organization. IEEE Transactions on Engineering Management 23, 163–167. Hubbard, T.N. (2000). The demand for monitoring technologies: the case of trucking. Quarterly Journal of Economics 116, 533–560. Hubbard, T.N. (2003). Information, decisions, and productivity: on-board computers and capacity utilization in trucking. American Economic Review 93, 1328–1353. Iacovou, C.L., I. Benbasat, A.S. Dexter (1995). Electronic data interchange and small organizations: adoption and impact of technology. MIS Quarterly 19, 465–485. Ireland, N., P. Stoneman (1985). Order effects, perfect foresight, and intertemporal price discrimination. Recherche Economique de Louvain 51, 7–20. Ito, H. (1996). Essays on investment adjustment costs. Ph.D. Thesis, Stanford University. James, J. (2003). Bridging the Global Digital Divide. Edward Elgar, Cheltenham UK. Jensen, R. (1982). Adoption and diffusion of an innovation of uncertain profitability. Journal of Economic Theory 27, 182–193. Jorgenson, D., K. Stiroh (2000). Raising the speed limit: U.S. economic growth in the information age. Brookings Papers on Economic Activity, pp. 125–211. Kagami, M., M. Tsuji (2001). The ‘IT’ revolution and developing countries: late-comer advantage. Institute of Developing Economies, Japan External Trade Organization, Tokyo. Karshenas, M., P.L. Stoneman (1993). Rank, stock, order, and epidemic effects in the diffusion of new process technologies: an empirical model. RAND Journal of Economics 24, 503–528.
Ch. 1. Diffusion of Information and Communication Technologies
49
Katz, M.L., C. Shapiro (1986). Technology adoption in the presence of network externalities. Journal of Political Economy 94, 822–841. Kauffman, R.J., J. McAndrews, Y. Wang (2000). Opening the ‘‘Black Box’’ of network externalities in network adoption. Information Systems Research 11, 61–82. Kauffman, R.J., A.A. Techatassanasoontorn (2005). Is there a global digital divide for digital wireless phone technologies? Journal of the Association of Information Systems 6, article 12. Kemerer, C.F., G.L. Sosa (1991). Systems development risks in strategic information systems. Information and Software Technology 33, 212–223. Kenney, M. (2003). The growth and development of the internet in the United States, in: B. Kogut (ed.), The Global Internet Economy, MIT Press, Cambridge. Kiiski, S., M. Pohjola (2002). Cross-country diffusion of the internet. Information Economics and Policy 14, 297–310. Kim, S.S., N. Malhotra (2005). A longitudinal model of continued is use: an integrative view of four mechanisms underlying postadoption phenomena. Management Science 51, 741–755. Kimberly, J.R., M.J. Evanisko (1981). Organizational innovation: the influence of individual, organizational, and contextual factors on hospital adoption of technological and administrative innovations. Academy of Management Journal 24, 689–713. Kolko, J. (2000). The death of cities? The death of distance? Evidence from the geography of commercial internet usage, in: I. Vogelsang, B.M. Compaine (eds.), The Internet Upheaval, MIT Press, Cambridge. Kolko, J. (2002). Silicon mountains, silicon molehills, geographic concentration and convergence of internet industries in the US. Information Economics and Policy 14, 211–232. Kraemer, K.L., J. Dedrick (2002). Information technology in Southeast Asia: engine of growth or digital divide?, in: C.S. Yue, J.J. Lim (eds.), Information Technology in Asia: New Development Paradigms, Institute of Southeast Asian Studies, Singapore. Kraemer, K.L., J. Dedrick, D. Dunkle (2002). E-commerce in the United States: Leader or one of the pack? Working Paper, CRITO, University of California, Irvine. Kraemer, K.L., J.L.Gibbs, J. Dedrick (2005). Impacts of globalization on e-commerce use and firm performance: a cross-country investigation. The Information Society 21, 1–18. Kraut, R.E., R.E. Rice, C. Cool, R.S. Fish (1998). Varieties of social influence: the role of utility and norms in the success of a new communication medium. Organization Science 9, 437–453. Kremp, E., J. Mairesse (2004). Knowledge management, innovation and productivity: a firm level exploration based on French manufacturing CIS3 data. NBER Working Paper #10237. Krugman, P. (1991). Geography and Trade. MIT Press, Cambridge. Kumar, R.L. (1996). A note on project risk and option values of investment in information technologies. Journal of Management Information Systems 13, 187–193. Kumar, R.L. (1999). Understanding DSS value: an options perspective. Omega 27, 295–304. Lane, S. (1991). The determinants of investment in new technology. American Economic Review 81, 262– 265. Leavitt, H.J., T.L. Whisler (1958). Management in the 1980’s. Harvard Business Review November– December, 41–48. Lee, M., C. Cheung (2004). Internet retailing adoption by small-to-medium sized enterprises (SMEs): a multiple-case study. Information Systems Frontiers 6, 385–397. Levy, F., R. Murname (2004). The New Division of Labor. Princeton University Press, Princeton. Mahajan, V., E. Muller, F. Bass (1990). New product diffusion models in marketing: a review and directions for research. Journal of Marketing 54, 1–26. Mahajan, V., R. Peterson (1985). Models for Innovation Diffusion. Sage, Beverly Hills, CA. Malone, T., J. Yates, R. Benjamin (1987). Electronic markets and electronic hierarchies. Communications of the ACM 30, 484–497. Manski, C.F. (1993). identification of endogenous social effects: the reflection problem. Review of Economic Studies 60, 531–542. Margrabe, W. (1978). The value of an option to exchange one asset for another. Journal of Finance 22, 177–186.
50
C. Forman and A. Goldfarb
Markus, M.L. (1994). Electronic mail as the medium of managerial choice. Organization Science 5, 505– 527. Marschak, T. (2004). Information technology and the organization of firms. Journal of Economics and Management Strategy 13, 473–515. Marschak, T. (2006). Organizational structure, in: T. Hendershott (ed.), Handbook on Economics and Information Systems, Elsevier, Amsterdam. Marshall, A. (1890). The Principles of Economics 8th ed. Macmillan, London, 1920. Mata, F., W. Fuerst, J. Barney (1995). Information technology and sustained competitive advantage: a resource-based analysis. MIS Quarterly 19, 487–505. McGrath, R.G. (1997). A real options logic for initiating technology positioning investments. Academy of Management Review 22, 974–996. Mendelson, H., R. Pillai (1998). Clockspeed and informational response: evidence from the information technology industry. Information Systems Research 9, 415–433. Milgrom, P., J. Roberts (1990). The economics of modern manufacturing: technology, strategy, and organization. American Economic Review 80, 511–528. Mukhopadhyay, T., S. Kekre (2002). Strategic and operational benefits of electronic integration in b2b procurement processes. Management Science 48, 1301–1313. Mukhopadhyay, T., S. Kekre, S. Kalathur (1995). Business value of information technology: a study of electronic data interchange. MIS Quarterly 19, 137–156. Nault, B.R., A.S. Dexter (1994). Adoption, transfers, and incentives in a franchise network with positive externalities. Marketing Science 13, 412–423. Oliner, S.D., D.E. Sichel (2000). The resurgence of growth in the late 1990s: is information technology the story?. Journal of Economic Perspectives 14, 3–22. Oxley, J., B. Yeung (2001). E-commerce readiness: institutional environment and international competitiveness. Journal of International Business Studies 32, 705–724. Parthasarathy, M., A. Bhattacherjee (1998). Understanding post-adoption behavior in the context of online services. Information Systems Research 9, 362–379. Png, I., B. Tan, K.-L. Wee (2001). Dimensions of national culture and corporate adoption of IT infrastructure. IEEE Transactions on Engineering Management 48, 36–45. Premkumar, G., K. Ramamurthy (1995). The role of interorganizational and organizational factors on the decision mode for adoption of interorganizational systems. Decision Science 26, 303–336. Premkumar, G., K. Ramamurthy, S. Nilakanta (1994). Implementation of electronic data interchange: an innovation diffusion perspective. Journal of Management Information Systems 11, 157–186. Raymond, L., G. Pare´ (1992). Measurement of information technology sophistication in small manufacturing businesses. Information Resources Management Journal 5, 4–16. Reddy, R., S. Reddy (2001). Supply Chains to Virtual Integration. McGraw-Hill, New York. Reinganum, J.F. (1981). Market structure and the diffusion of new technology. Bell Journal of Economics 12, 618–624. Riggins, F.J., C.H. Kriebel, T. Mukhopadhyay (1994). The growth of interorganizational systems in the presence of network externalities. Management Science 40, 984–998. Rohlfs, J. (2001). Bandwagon Effects in High-Technology Industries. MIT Press, Cambridge. Rogers, E. (1995). The Diffusion of Innovations 4th ed. Free Press, New York. Rosenberg, N., M. Trajtenberg (2004). A general purpose technology at work: The Corliss steam engine in late nineteenth century United States. Journal of Economic History 64, 61–99. Rosenzweig, P. (1994). When can management science research be generalized internationally?. Management Science 40, 28–39. Ryan, B., N.C. Gross (1943). The diffusion of hybrid seed corn in two Iowa communities. Rural Sociology 8, 15–24. Rysman, M., (2004). An empirical analysis of payment card usage. Working Paper, Boston University. Sadowski, B.M., C. Maitland, J. van Dongen (2002). Strategic use of the internet by small- and mediumsized companies: an exploratory study. Information Economics and Policy 14, 75–93. Saloner, G., A. Shepard (1995). Adoption of technologies with network effects: an empirical examination of the adoption of automated teller machines. RAND Journal of Economics 26, 479–501.
Ch. 1. Diffusion of Information and Communication Technologies
51
Saloner, G., A.M. Spence (2002). Creating and Capturing Value: Perspectives and Cases on Electronic Commerce. Wiley, New York. Schwartz, E., C. Zozaya-Gorostiza (2003). Investment under uncertainty in information technology: acquisition and development projects. Management Science 49, 57–70. Scott Morton, M. (1991). The Corporation of the 1990s: Information Technology and Organizational Transformation. Oxford University Press, New York. Shapiro, C., H.R. Varian (1999). Information Rules: A Strategic Guide to the Network Economy. Harvard Business School Press, Boston. Sinai, T., J. Waldfogel (2004). Geography and the internet: is the internet a substitute or a complement for cities?. Journal of Urban Economics 56, 1–24. Spulber, D. (2006). Networks and two-sided markets, in: T. Hendershott (ed.), Handbook on Economics and Information Systems, Elsevier, Amsterdam. Srinivasan, K., S. Kekre, T. Mukhopadhyay (1994). Impact of electronic data interchange technology on JIT shipments. Management Science 40, 1291–1304. Steinmuller, W.E. (2001). ICTs and the possibilities of leapfrogging by developing countries. International Labour Review 140, 193–210. Stiroh, K. (2002a). Information technology and the U.S. productivity revival: what do the industry data say?. American Economic Review 5, 1559–1576. Stiroh, K. (2002b). Are ICT spillovers driving the new economy?. Review of Income and Wealth 48, 33–57. Stoneman, P. (2002). The Economics of Technological Diffusion. Blackwell Publishers, Oxford, UK. Szajna, B. (1996). Empirical evaluation of the revised technology acceptance model. Management Science 42, 85–92. Tachiki, D., S. Hamaya, K. Yukawa (2004). Diffusion and impacts of the internet and e-commerce in Japan. Working Paper, Center for Research in Information Technology and Organization (CRITO), University of California Irvine. Tan, Z., W. Ouyang (2004). Diffusion and impacts of the internet and e-commerce in China, electronic markets. The International Journal of Commerce and Business Media 14, 25–35. Taudes, A., M. Feurstein, A. Mild (2000). Options analysis of software platform decisions: a case study. MIS Quarterly 24, 227–243. Teo, H.H., K.K. Wei, I. Benbasat (2003). Predicting intention to adopt interorganizational linkages: an institutional perspective. MIS Quarterly 27, 19–49. Tigre, P.B. (2003). E-commerce readiness and diffusion: the case of Brazil. Working Paper, CRITO, University of California, Irvine. Tornatzky, L.G., M. Fleischer (1990). The Processes of Technological Innovation. Lexington Books, Lexington. Tucker, C. (2005). Network effects and the role of influence in technology adoption. Working Paper, MIT Press. U.S. Department of Agriculture (2001). Farm computer usage and ownership report, 07.30.01. Available at http://usda.mannlib.cornell.edu/reports/nassr/other/computer/ Venkatesh, V., F.D. Davis (2000). A theoretical extension of the technology acceptance model: four longitudinal field studies. Management Science 46, 186–204. Wallsten, S. (2003). Regulation and internet use in developing countries. Working Paper, AEI and Brookings Institution, Washington, DC. Wang, E.T.G., A. Seidmann (1995). Electronic data interchange: competitive externalities and strategic implementation policies. Management Science 41, 401–418. Wheeler, C.H., 2005. Technology and industrial agglomeration: evidence from computer usage. Working Paper #2005-16A, Federal Reserve Bank of St. Louis. Williamson, O. (1975). Markets and Hierarchies: Analysis and Antitrust Implications. Free Press, New York. Wong, P. (2002). ICT production and diffusion in Asia: digital dividends or digital divide?. Information Economics and Policy 14, 167–187. Wong, P., Ho (2004). E-commerce in Singapore: impetus and impact of globalization. Working Paper, CRITO, University of California, Irvine.
52
C. Forman and A. Goldfarb
Xu, S., K. Zhu, J. Gibbs (2004). Global technology, local adoption: a cross-country investigation of internet adoption by companies in the United States and China. Electronic Markets 14, 13–24. Zhu, K., K.L. Kraemer (2005). Post-adoption variations in usage and value of e-business by organizations: cross-country evidence from the retail industry. Information Systems Research 16, 61–84. Zhu, K., K.L. Kraemer, V. Gurbaxani, S. Xu (2006). Migration to open-standard interorganizational systems: network effects, switching costs, and path dependency. MIS Quarterly 30, 515–539. Zhu, K., K.L. Kraemer, S. Xu (2002). A cross-country study of electronic business adoption using the technology-organization-environment framework. Proceedings of the Twenty-Third International Conference on Information Systems, pp. 337–348. Zhu, K., K.L. Kraemer, S. Xu (2003). Electronic business adoption by European firms: a cross-country assessment of the facilitators and inhibitors. European Journal of Information Systems 12, 251–268.
Terrence Hendershott, Ed., Handbooks in Information Systems, Vol. 1 Copyright r 2006 by Elsevier B.V.
Chapter 2
Economics of Data Communications
Philipp Afe`che University of Chicago, Chicago, IL, USA
Abstract This chapter reviews research on data transport pricing decisions for a single provider who pursues one of two economic performance objectives: welfare maximization or profit maximization. It largely focuses on studies that consider these issues in the context of mathematical models of data transport service demand and production. The survey aims to provide a broad roadmap for important questions, modeling approaches and results, and to outline open questions. To this end it develops an unifying framework that classifies data transport service contracts based on the types of their quality of service (QoS) promises into guaranteed, best effort and flexible bandwidthsharing services. It highlights and compares their key features and discusses their pricing decisions along common dimensions: basic pricing and allocation principles; incomplete information and adaptive pricing; more differentiation; optimal dynamic pricing; and conclusions and directions. The survey then examines benefits and challenges of auctions versus posted prices; the debate on flat-rate versus usage-based pricing; and the merits and challenges of alternative QoS designs and their delivery mechanisms.
1
Introduction
The last few years have witnessed a revolution in the market for data communications, marked by the transformation of the Internet from an academic and research network with restricted access to a commercial platform used by millions of individuals and organizations to communicate, access information and conduct business electronically. Key enablers of this growth have been the development of cost-effective optical network technologies, the widespread acceptance of Internet protocols as the standard for transporting any kind of digitized information, and large investments in fibre infrastructure and startup companies during the e-commerce boom of 53
54
Philipp Afe`che
the late 1990s. Over the past decade, the number of Internet service providers has grown from dozens to thousands and the worldwide Internet population from 40 million to about one billion people who use an increasingly diverse set of applications ranging from basic email, real-time audio and video to online auctions and distributed computing. This increase in the number of users and the growth in usage per capita, spurred by the emerging deployment of broadband access connections, have resulted in dramatic traffic growth and network capacity expansion. Annual data traffic and bandwidth growth rates, roughly 1000% in 1995, remain substantial at close to 100% and Internet traffic recently surpassed voice traffic in volume, cf. Odlyzko (1999a, 2003). However, this enormous growth in data traffic volume and variety has also created substantial problems, most notably the persistence of intermittent network congestion leading to poor service quality in the form of transmission delays and data losses. The ensuing debate has centered around several problem causes and challenges. Flat connection fees that only depend on the access capacity but not on actual usage are prevalent in the consumer and parts of the interconnection market, contributing little to discourage overuse at peak times. The Internet was designed to offer undifferentiated ‘‘best effort’’ service to all applications; it is ill-equipped for tailoring service quality attributes and guarantees to specific applications or users, making it difficult to offer preferential treatment to those who value and are willing to pay for it. This absence of strict economic and technological usage controls has led many to view Internet congestion as a classical ‘‘tragedy of the commons.’’ Capacity levels may be too low in certain network segments and excessive in others, and limitations of currently deployed technologies constrain the ability of providers to offer premium services, leaving them to compete in a commodity market which hurts their revenues. These challenges have raised many questions and sparked much research on how to better manage data communication services and the infrastructure to deliver them. Can congestion simply be eliminated by deploying ample capacity to simultaneously accommodate all users? Or, is there a need for better control mechanisms to regulate usage and allocate scarce capacity among competing traffic flows? How should differentiated services be designed and priced to increase customer value and provider profits? What are the relative strengths and weaknesses of flat and usage-based pricing schemes? How should price and resource allocation mechanisms be designed to account for the decentralized nature of the Internet whose performance depends on the decisions of many independent self-interested parties with conflicting incentives and private information? This chapter surveys research on these questions and offers an outlook on future directions. Scope. The economics of data communications is a vast subject area. This survey must therefore be selective. It makes no attempt to discuss all the important topics, models and questions that might have been included in a chapter with this title. Its scope is defined as follows.
Ch. 2. Economics of Data Communications
55
The operation of a data communications network involves three basic functions: transport or communication of data among interconnected network nodes, storage and processing at these nodes. This survey focuses on data transport services at the exclusion of storage and processing services such as those offered by application service providers. Although the Internet accounts for much of the growth in data communications, the discussion is only in part specific to the Internet. For the most part it emphasizes economic principles that are valid for a range of data network technologies. The survey is very much focused on the economics of traffic sources and resource allocation mechanisms that are particular to modern data communication networks; it is not concerned with voice communications over traditional telephone networks. While this chapter focuses on the economics, not the technology, of data transport services, it does outline and compare the resource allocation principles that are of particular importance for their economic analysis, but it ignores implementation details. From an economics perspective managing data transport services can be viewed as a complex resource allocation problem that involves a range of decentralized decisions by independent and self-interested agents. Providers decide how to design, price and produce these services taking into account the demand characteristics that drive customers’ purchase and usage decisions. This chapter surveys various aspects of this resource allocation problem, focusing on various dimensions of usage-based pricing decisions for a single provider who pursues one of two economic performance objectives: welfare maximization or profit maximization. Research on data transport service pricing draws on tools and contributions from several disciplines, including economics, computer science and electrical engineering, information systems, management science/operations research, and applied probability. These efforts are also characterized by a striking variety in terms of their objectives (optimization of a given system or design of new pricing architectures), methodologies (mathematical modeling and analysis, simulation or empirical study) and abstraction levels (stylized models of generic systems or detailed representations of specific systems). This survey largely focuses on research that studies pricing issues in the context of mathematical models of data transport service demand and production. To a lesser extent, the survey also outlines implementation issues and architectural proposals for pricing and service design. The survey does not discuss, but references a few empirical analyses of data transport service demand and pricing. Goals and approach. This chapter aims to provide a broad roadmap rather than an in-depth tutorial. The main goals are to identify and compare important questions and modeling approaches, existing answers and proposals, and outline open questions. To this end the survey emphasizes the development of an unifying framework that classifies economic analyses of data transport services based on the types of QoS guarantees they make and the pricing decisions they consider. It uses this framework to discuss and
56
Philipp Afe`che
compare the partly disjointed research streams through a common lens. The discussion is mainly conceptual in nature, but mathematical notation is used selectively where doing so may help clarify the key ideas. Organization. The chapter is organized as follows. Section 2 defines the problem and develops the discussion framework. It categorizes data transport service contracts based on the types of QoS guarantees they make into guaranteed, best effort and flexible bandwidth-sharing services, and characterizes the salient features of service demand and production that shape their analysis. It then describes major dimensions of pricing decisions, outlines for each dimension the relevant issues for this survey, and outlines the framework for their discussion. The core Sections 3–5 focus on one of the three service types, but follow the same organization outlined in the framework: features, modeling and overview; basic pricing and allocation principles; incomplete information and adaptive pricing; more differentiation; optimal dynamic pricing; and conclusions and directions. The discussion in each of the core sections focuses on the paradigm of economic optimality. Section 6 goes beyond the optimality criterion and considers implementation issues in pricing and service design, including the benefits and challenges of auctions versus posted prices; the debate on flat-rate versus usage-based pricing; and the merits and challenges of alternative QoS designs and their delivery mechanisms. Section 7 concludes and outlines some overall research directions. 2
Problem definition and discussion framework
From an economics perspective managing data transport services can be viewed as a complex resource allocation problem that involves a range of decentralized decisions by independent and self-interested agents. Providers decide how to design, price and produce these services taking into account the demand characteristics that drive customers’ purchase and usage decisions. This chapter surveys various aspects of this resource allocation problem, focusing on various dimensions of usage-based pricing decisions for a single provider who pursues one of the economic performance objectives: welfare maximization or profit maximization. Welfare maximization or economic efficiency, also referred to as net value maximization or social optimization, serves as the key benchmark for any allocation since it captures the sum of customer surplus and provider surplus. It is also the relevant optimization criterion for the provider of an internal data network who should maximize the overall value to the entire organization rather than her own profit. Profit maximization is the key optimization criterion for a commercial provider with external customers. Users or customers—this survey uses the terms interchangeably—generate demand for data transport services through a variety of applications such as email, file downloads, distributed computing and e-commerce transactions. These applications are quite heterogenous in terms of their
Ch. 2. Economics of Data Communications
57
traffic source properties and quality of service requirements, which specify data transfer performance targets or preferences. Data transport services are sold in the context of service contracts that specify three sets of attributes: QoS parameters or metrics that the provider guarantees, constraints on traffic parameters that the user must meet, and a tariff that determines the user’s charge. The provider ‘‘produces’’ services with the desired attributes by means of a communication network. It can be viewed as a service factory consisting of a resource network with the capacity and connectivity to carry traffic and control mechanisms to manage traffic and resource allocation. To organize the discussion, this section develops a simple framework that classifies economic analyses of data transport service contracts based on the types of QoS guarantees they make and the pricing decisions they consider. Sections 2.1–2.3 lay the groundwork for the framework. They describe three types of service contracts, guaranteed, best effort and flexible bandwidthsharing services, and characterize the salient features of service demand and production that shape their analysis. Along the way they introduce key concepts and terminology. Section 2.4 highlights differences between modern data communications and traditional telephone networks. Section 2.5 discusses the role of pricing, describes major dimensions of pricing decisions and outlines, for each dimension, the relevant issues for this survey. Section 2.6 outlines the organizing framework for discussing these decisions. 2.1 Service demand Users and their applications generate traffic sources that drive the demand for data transport services. Key demand characteristics are the properties of traffic sources, their QoS requirements, and user preferences over the quantity, quality and time of service. Traffic sources The properties of traffic sources determine their bandwidth demand and the relationship between the traffic mix a network can carry and the QoS it can deliver. As detailed below the data flow generated by a traffic source is typically broken into packets or cells. Based on their data flow pattern one can broadly distinguish between bulk transfer, constant bit rate (CBR) and variable bit rate (VBR) sources. Bulk transfers are triggered by web file downloads, network print jobs or email transmissions. Each generates a data ‘‘burst’’ at a single point in time which is fully characterized by its data volume. CBR sources are generated by certain voice and video applications; each is characterized by its duration or holding time and constant data rate which equals its bandwidth requirement. CBR source models are also representative of leased virtual circuits or dialup access connections that need a fixed amount of capacity over a given time horizon. VBR sources are generated by interactive voice, audio and video applications. They exhibit a
58
Philipp Afe`che
range of traffic patterns and are characterized by their duration, mean and peak data rate, and measures of their ‘‘burstiness’’ or data rate fluctuation. See Kelly (1996) and Kumar et al. (2004) for specific source models. QoS requirements Customers and their applications have QoS requirements that are expressed in terms of any combination of three key performance metrics: throughput (bandwidth or data rate), delay (or latency), and loss. These QoS requirements may depend both on the technical features of the hardware and software involved and on user preferences. For example, order entry processing transactions may be time-sensitive due to technologically imposed time-outs but also because their users require low delays to maintain their productivity. Based on the nature of their QoS requirements one can roughly distinguish between elastic applications that have flexible QoS preferences and real-time applications that require relatively strict QoS bounds, cf. Shenker (1995). Elastic applications are inherently (technologically) tolerant of delay, delay variation (jitter) and throughput limitations: quality degradation due to network congestion hurts performance but does not compromise the integrity of the transmitted data. When transmissions slow down they keep working, just not as quickly as usual—performance ‘‘degrades gracefully.’’ Examples include bulk transfers such as email, web file downloads or streaming audio that can adapt its compression and data rate to available resources. Depending on the desired level of interactivity, elastic applications may differ in their delay tolerance: transactions initiated during an interactive remote login session are more delay-sensitive than interactive bulk file transfers, which in turn are more sensitive than asynchronous (non-interactive) bulk file transfers such as electronic mail. Delay tolerant text-based transmissions must be loss- and error-free whereas audio or video transmissions may tolerate some data loss. Real-time applications require relatively strict QoS bounds such as minimum throughput, maximum delay, or maximum packet loss ratio. Violating these bounds compromises the interactive nature of these applications and may distort their data signals to the point where they become incomprehensible. Examples include interactive CBR or VBR voice and video applications. User preferences User preferences are commonly modeled through utility functions that measure their willingness to pay as a function of service attributes such as the quantity, quality and time of service. In making their service purchase and usage decisions users act as self-interested agents who maximize their expected utility minus their cost of service for a given transmission or over some time horizon. This utility-maximizing behavior determines inverse demand functions that map service attributes to prices. An alternative modeling
Ch. 2. Economics of Data Communications
59
approach is to specify user utility implicitly via demand functions that specify demand depending on the quality, time and price of service. This survey uses utility functions to illustrate model features and certain results. The heterogenous nature of data communications demand has given rise to a range of demand models. This survey does not discuss empirically derived demand models. See Edell and Varaiya (1999), Altmann et al. (2000), Varian (2002) and Beckert (2005) for empirical analyses of demand for Internet-based data transport services. Utility functions may have any combination of the following attributes and features. Modeling details are discussed later in conjunction with the respective studies. Throughput—quality or quantity: Throughput is a measure of quantity per unit time. Depending on the context, it measures the throughput of data units belonging to an individual flow (e.g., bits, packets or cells of a file downloaded per second) or that of entire individual flows (e.g., files transmitted per second). In the former case throughput may be more naturally interpreted as a quality attribute, in the latter case it may equally well be understood to measure service quantity per unit time. Delay or loss quality: These QoS metrics are typical utility function attributes in settings with elastic applications; they capture the users’ delay or loss sensitivity. Congestion externalities and network load: An important distinction is between models with and without (negative) congestion externalities. In models with congestion externalities, appropriate for services without QoS guarantees, a user’s utility depends not only on her own service quantity but also on (all) other users’ consumption, captured by some measure of network load, since more traffic overall leads to congestion and reduces the quality of transmissions. This chapter does not consider positive network externalities where user utility increases in the number of potential connection partners. Rohlfs (1974) and Oren and Smith (1981) study pricing for communication services with network externalities. Westland (1992) and Giridharan and Mendelson (1994) study settings with network and congestion externalities. Time of service: In models that capture non-stationary demand, utility functions also depend on when a particular transmission occurs. The vast majority of studies consider stationary demand. Market segmentation: Models of heterogenous user populations typically segment users into classes or types based on their traffic source properties and/or utility function attributes. 2.2 Service contracts: QoS guarantees and service designs As noted above data transport services are bought and sold via contracts between customers and providers. A contract specifies three sets of service attributes: QoS parameters or metrics that the provider guarantees, constraints on user traffic parameters, and a tariff that determines the user’s
60
Philipp Afe`che
charge. This survey uses the term ‘‘contract’’ broadly to refer to any agreement between a user and the network that specifies the terms for performing a transmission. A contract may apply to services at various levels of resource and time aggregation, e.g., from sending a single file at a particular point in time to transmitting multiple files over a fixed time horizon. The heterogenous nature of traffic sources and QoS requirements calls for a variety of data transport services, each tailored to suit a particular kind of application. This survey categorizes data transport service contracts into those for guaranteed, best effort and flexible bandwidth-sharing services, based on the type of QoS guarantees they offer and the traffic constraints they impose. It is important to distinguish between the contract that defines the attributes of a service and the resource network and control mechanisms used to produce it. This section focuses on the QoS and traffic attributes of these contract types. Section 2.3 describes how these services are produced. The discussion of tariff and pricing issues starts in Section 2.5. Guaranteed services Guaranteed services specify QoS parameters the network promises to deliver provided the user’s traffic profile conforms to certain traffic parameters. QoS parameters specify the maximum tolerable congestion level in terms of deterministic or probabilistic traffic flow performance bounds such as minimum bandwidth, maximum delay, jitter or packet loss rate. Common traffic parameter constraints concern the peak rate, mean rate and burstiness. QoS guarantees can be fixed (static) or flexible (dynamic) over the contract lifetime. In this survey the term ‘‘guaranteed services’’ refers exclusively to contracts with fixed guarantees. They are discussed separately (Section 3) from ‘‘flexible bandwidth-sharing services’’ (Section 5) that are introduced below. The two rely on different control mechanisms and pricing paradigms. Fixed guarantees are best suited for real-time applications with strict QoS requirements. Examples of such guaranteed services include CBR and real-time VBR in the Asynchronous Transfer Mode (ATM) framework and the Guaranteed Quality services in the Integrated Services (IntServ) architecture proposal of the Internet Engineering Taskforce (IETF). Flexible bandwidth-sharing services Flexible bandwidth-sharing services offer dynamic bandwidth guarantees and give users some control over and the ability to adapt to their allocation while their transmissions are in progress. At each point during a transmission the contract guarantees a certain peak data rate to the user, in response to changing demand and supply conditions, which represents the user’s flexible QoS guarantee and traffic parameter constraint. Flexible bandwidth-sharing services are best suited for elastic applications whose performance easily adapts to data rate fluctuations. Examples include the
Ch. 2. Economics of Data Communications
61
Internet’s Transmission Control Protocol (TCP) which supports email and the World Wide Web, and the Available Bit Rate (ABR) service in the ATM framework. Best effort services Best effort services make no explicit quality guarantees. The network promises to provide the best quality it can to each customer, attempting to deliver data as quickly and error-free as possible without committing to any quantitative bounds. In turn, best effort services do not constrain users’ traffic inflow other than through the capacity of their access link. Users do not request permission before transmitting and may do so at their maximum rates. They may experience delays or losses during congestion periods since performance is determined not only by the network itself but also by other users’ offered load. While the network does not guarantee QoS for individual flows, the notion is that users estimate or receive information on aggregate performance statistics such as the delay distribution or packet loss ratio across all flows. Best effort services are suited for elastic applications that are somewhat tolerant of delay and loss or if the capacity is so abundant that performance degradation is unlikely to reach critical levels. Examples are the Internet’s Internet Protocol (IP) and User Datagram Protocol (UDP), the Unspecified Bit Rate (UBR) service in the ATM framework and the Best Effort service in the IntServ Architecture proposal. Loss-sensitive applications that do use best effort service, e.g., streaming media that use UDP, may also provide their own mechanisms to adjust their sending rates and avoid data loss. Remarks: This survey introduces the term ‘‘flexible bandwidth-sharing services’’ to highlight the most striking features of TCP-like services from the users’ perspective. It adopts a literal definition of best effort services and distinguishes them from ‘‘flexible bandwidth-sharing services,’’ even though the Internet’s TCP transport service is often considered part of the ‘‘best effort’’ Internet. As understood here, best effort services offer no quality guarantees whatsoever and impose no constraints on the traffic that users may send into the network, irrespective of congestion conditions. Unlike TCP, they have no built-in traffic flow control and feedback mechanisms for avoiding congestion. Thus, users have little information on precisely when and how to adjust their sending rates to match fluctuations in bandwidth availability. The Internet transport service provided by the UDP protocol is a typical example: packets may be delayed, lost or arrive out of order without notice. By contrast, the Internet transport service provided by the TCP protocol guarantees zero packet loss by retransmitting lost packets and dynamically controls the peak rates at which competing connections are allowed to transmit via adaptive flow control and feedback mechanisms. Its distinctive features are closed-loop control, on fast time scales and at the level of individual flows. The distinction between best effort
62
Philipp Afe`che
and flexible bandwidth-sharing services made here is useful, since the respective pricing and resource allocation decisions occur on different time scales that call for somewhat different analytical frameworks and perspectives. In the case of flexible bandwidth-sharing services the fast feedback loops afford users (or agents on their behalf) some control over their transmissions while they are in progress. In the case of best effort services the notion is that feedback loops between sending rates and congestion levels are relatively slow—so slow that users have no control over a given individual flow through the network once it is in progress. Thus, users make all decisions concerning an individual flow or a collection of such flows covered by their contract at the start of the respective transmissions, based on the contract parameters and performance information available at that time. In this sense best effort service contracts only have static parameters. Since the network does not make any strict QoS guarantees per user, the notion is that users, prior to sending data, estimate or receive information on aggregate QoS statistics, e.g., the delay distribution or packet loss ratio over all flows. One can think of these estimates as the expected QoS metrics of a given best effort service. Service designs In this chapter the term ‘‘service design’’ refers to the key features that characterize a set of service classes offered by a provider. The survey broadly distinguishes among service designs based on three dimensions: 1. Number of service classes. Under uniform service all traffic sources— whether homogenous or heterogenous—are given identical or no QoS guarantee(s) and subjected to the same traffic constraints. The canonical example is a single class of best effort service that offers no QoS guarantees and imposes no constraints on traffic inflows. Differentiated services differ in at least one QoS metric or traffic constraint. For example, multiple classes of guaranteed service designed to support heterogenous traffic sources may share the same QoS guarantee, e.g., the packet loss probability, but differ in their traffic parameter constraints, e.g., the peak data rate. Services may also be differentiated in terms of the type of QoS guarantee with respect to a single QoS metric such as delay. In multi-link networks, which have received less attention than the single link case, otherwise identical service classes may also differ in their source-destination or route. 2. Type of QoS guarantee per service class. Each class may offer guaranteed, best effort, or flexible bandwidth-sharing service. The majority of papers consider service designs with a single type of QoS guarantee for all classes. 3. Metrics per service class. Based on the metrics of the different classes this survey distinguishes between service designs with bandwidth differentiated services, which only differ based on bandwidth (or
Ch. 2. Economics of Data Communications
63
throughput) related QoS metrics or traffic parameter constraints but not based on delay or loss, and delay or loss differentiated services . 2.3 Service production A communication network can be viewed as a service factory that produces and delivers services with the desired attributes. It is a collection of interconnected resources and control mechanisms. Resources provide the capacity and connectivity to carry traffic, and control mechanisms manage traffic demand and resource allocation. There are many valuable references on the technology, operation, analysis and control of data communication networks. A seminal work on communication networks is Kleinrock (1964). Recent textbooks include Walrand and Varaiya (2000), Tanenbaum (2003) and Kumar et al. (2004). Courcoubetis and Weber (2003) provide a conceptual overview of network services and technology. Gevros et al. (2001) discuss congestion control mechanisms for best effort services. This section describes the structural and functional characteristics of data communication networks. It focuses on the conceptual principles that are of particular importance for their economic analysis and ignores implementation details. It first outlines the main features of the resource network, its cost structure and basic operating principles. It then describes the network control mechanisms that are instrumental in delivering services with certain QoS metrics. The section then compares guaranteed, best effort and flexible bandwidth-sharing services in terms of their reliance on these control mechanisms for service delivery. Resource network Network resources are grouped into nodes, computers where traffic originates and terminates and switches or routers that forward traffic, and connecting links. Nodes and links have buffers in which data packets queue in the presence of congestion. Network design and investment decisions determine the numbers, types and capacities of these resources and their interconnections. The performance of a network crucially depends on its capacity. The processing capacity or bandwidth of a resource measures its maximum throughput or transmission rate per unit time, in packets, cells or bits per second (bps). The capacity along a network route is determined by the bottleneck, the resource with the smallest capacity. The buffer size or storage capacity is an important factor in determining at what point congestion translates to data loss as opposed to queueing delays. Systems without buffers are referred to as loss systems, those with buffers as queueing or queueing-loss systems. The network topology describes the pattern of links connecting the nodes. Large networks such as the Internet consist of multiple interconnected networks at different hierarchical levels: end-users are connected via access links to Internet Service Providers (ISPs) which in turn interconnect through bilateral and multilateral agreements.
64
Philipp Afe`che
Network cost structure The network cost has several components: the cost for setting up, operating and maintaining the network resources and technologies, the cost of capacity expansion and the connection cost for access lines and customer premises equipment. An important network cost characteristic is that the fixed cost is a major percentage of total cost. As a result, for a given network capacity level the direct marginal cost of a transmission is essentially zero. However, transmissions cause an indirect congestion cost if their network use reduces the service quality of other users, e.g., by increasing their delay or data loss. The congestion cost is negligible at low network utilization but very significant when traffic demand levels approach the network capacity. Basic operating principles The fundamental functions multiplexing, switching and routing, define the context for network control mechanisms. Multiplexing and switching specify the principles for allocating bandwidth to multiple individual traffic sources such as voice conversations, email or streaming videos. Routing determines the data path through the network. Traditional telephone networks significantly differ from data communication networks in how they perform these functions. Telephone networks use circuit switching and deterministic multiplexing. When a call request arrives the admission control function decides whether to accept or reject it by comparing the network’s resource availability with the call’s requirements. Circuit switching establishes for each accepted voice call a dedicated physical circuit, fixing a route and reserving a fixed bandwidth amount equal to the data flow’s peak rate for the circuit lifetime. All data then travels along this connection. Deterministic multiplexing manages the static bandwidth shares at each link by letting active circuits only transmit in their pre-assigned time slots, regardless of need. This works great for voice and other CBR sources. Since resource needs of individual flows are constant over time, it is easy to simultaneously deliver QoS guarantees and attain high resource utilization by performing admission control and reserving matching bandwidth. However, circuit switching can waste a lot of capacity if used for bursty traffic sources with large ratios of peak to mean rate. Data communication networks such as the Internet use packet or cell switching and statistical multiplexing to mitigate this problem. Packet switching divides data flows into packets that travel individually to their destination where they are reassembled. Statistical multiplexing dynamically allocates bandwidth at each link to competing data packet streams based on their probability of need, QoS and fairness considerations. Packet switching networks are connectionless if packets of the same flow are routed independently of each other, or connection-oriented if they all travel along the same a priori established route. In packet networks the term ‘connection’ refers to a set of
Ch. 2. Economics of Data Communications
65
logically linked end-points for data exchange. In contrast to circuit switched phone networks, traditional packet networks neither use admission control nor dedicate fixed bandwidth for a data flow or connection during its entire lifetime. Rather they allocate bandwidth on demand when and where needed by a packet. Packet switching therefore achieves higher bandwidth utilization for bursty traffic compared to circuit switching, but it is vulnerable to quality degradation in the form of data delay and loss when the network is congested. A range of mechanisms, including admission control, are designed to control congestion in modern packet networks. Network control mechanisms There is a daunting array of network technologies that implement control mechanisms in the hardware and software of nodes and at all layers of the network architecture. This survey describes a selection of these mechanisms at a conceptual level, focusing on their role in delivering certain service performance attributes and in creating differentiated services. It classifies these control mechanisms into demand regulation or supply allocation mechanisms depending on whether they regulate the traffic inflow or the allocation of a resource network with fixed capacity to admitted traffic flows. Their key functions and characteristics are as follows. Demand regulation mechanisms include admission control, traffic shaping and policing and flow control. (This chapter ignores content replication, caching and data compression techniques that reduce the potential demand for data transmissions.) Admission control decides whether to accept or reject a connection request with certain traffic constraints and QoS requirements after checking whether the network has the necessary resources. It operates on the time scale of connection interarrival times. Traffic policing and shaping mechanisms ensure that the data flow of an admitted connection conforms to contractually established traffic parameter constraints. They operate on individual packets and cells and are commonly implemented via leaky bucket descriptors. Nonconforming traffic may be shaped to conform by delaying cells or allowed into the network but dropped if the network becomes congested. Flow control mechanisms regulate traffic inflow by controlling the sending rates of whatever sources choose to be active, rather than via the number of traffic sources as does admission control. Flow control dynamically manages bandwidth demand and sharing via fast feedback loops between network nodes that send congestion indication signals and end user applications which adapt their sending rates accordingly. It operates on the time scale of round trip signal propagation delays which are shorter than connection interarrival times, but typically longer than packet queueing delays. Supply allocation mechanisms considered here include link partitioning, routing, resource reservation and packet scheduling. (This survey distinguishes between supply allocation for given capacity and capacity expansion.) Link
66
Philipp Afe`che
partitioning slices the bandwidth of a link into multiple parallel channels. Routing assigns individual data packets or entire data flows to channels and links from source to destination. Data flows may be assigned to resources using a combination of exclusive access via reservations and shared access via packet scheduling and routing. Resource reservation a priori fixes a route along which it dedicates a set of buffers and channels to an individual data flow; it is used at connection setup in conjunction with admission control. Packet scheduling disciplines allocate resources dynamically: they regulate the access of multiple individual data flows to a shared set of buffers and channels by specifying the order in which queued data packets are transmitted. Common packet scheduling disciplines are first-in-first-out (FIFO); strict or absolute priority disciplines, where higher priority traffic waiting for transmission always gets exclusive bandwidth access; and processor sharing disciplines where all priority levels get some bandwidth access based on their relative priority index. Variations of the latter include egalitarian, discriminatory and generalized processor sharing, which is an idealized model of Weighted Fair Queueing. Guaranteed, best effort and flexible bandwidth-sharing services Guaranteed services rely on admission control, traffic shaping and policing for demand regulation. In terms of supply allocation tools their analysis typically assumes fixed link partitions and routes, which are assigned to individual data flows via reservations and/or packet scheduling mechanisms—FIFO if connections share and priority disciplines if they differ in their QoS requirements. These supply allocation mechanisms are insignificant from the perspective of users since the network guarantees their transmission performance. A key performance issue for pricing is to quantify the bandwidth requirement of each connection. Guaranteed services rely on open-loop control and require extensive network state information at the start of a transmission contract to ensure strict flow-level QoS guarantees. Best effort services, unlike guaranteed services, hardly rely on technological demand regulation mechanisms, except through the bandwidth constraints of users’ access links. They do not have built-in flow control and feedback mechanisms for congestion avoidance. Pricing is the only direct demand regulation tool, but demand is also regulated indirectly based on customers’ response to and anticipation of congestion. Best effort services may rely on a range of (reservationless) supply allocation mechanisms to manage QoS performance, cf. Gevros et al. (2001). This survey considers the role of packet scheduling, link partitioning and routing mechanisms in controlling congestion and creating quality differentiated services. These tools may play a larger role in a future Internet with multiple classes of best effort or ‘‘better-than best effort’’ service, somewhat similar to the Differentiated Services (DiffServ) architecture proposal of the IETF. A key performance issue for pricing is to quantify the QoS of each class in terms of
Ch. 2. Economics of Data Communications
67
queueing delay or data loss. Best effort services rely on open-loop and slow user-driven closed-loop control to try and meet certain average class-level QoS metrics over longer time periods. Flexible bandwidth-sharing services rely only on flow control for demand regulation and congestion avoidance. In terms of supply allocation their analysis typically assumes fixed routes of nonpartitioned links, which are assigned to data flows without resource reservation via FIFO packet scheduling. Flow control leads users to share the link bandwidth so their cumulative data rates equilibrate around the amount of available capacity, keeping queueing delays small. The result is a set of services with arbitrary quality differentiation in terms of their bandwidth allocations. Hence, the relationship between bandwidth and quality of each flow is immediate. Flexible bandwidth-sharing services rely on fast closed-loop control, while transmissions are ongoing to deliver flexible flow-level bandwidth guarantees. 2.4 Distinctive features in comparison to telephony The context of data communications differs in important ways from that of ‘‘plain old telephone service’’ which has been studied extensively. On the demand side, the traffic properties and QoS requirements of voice conversations are uniform, simple and stable, while those of data sources are heterogenous, complex and keep evolving as new applications emerge. The demand for data transport is therefore less well understood than telephony demand. For example, even users may not fully know their traffic source properties ex ante. On the supply side, the need for data networks to simultaneously carry a wide variety of traffic types with a multitude of QoS requirements has spawned new types of service contracts along with new technologies and control mechanisms that use fundamentally different resource allocation principles, including flow control, statistical multiplexing and discriminatory packet scheduling disciplines. These differences have posed new challenges and generated new research on the economics of data communication services and on closely related issues of operational network performance, specifically, on the bandwidth requirements of traffic sources and on the relationship between traffic demand, network capacity, control mechanisms, and QoS. 2.5 Pricing decisions The preceding sections have introduced and characterized three fundamentally different types of data transport service contracts and the features of service demand and production that shape the context for their analysis. This section discusses the role of pricing, identifies major dimensions of pricing decisions and outlines for each dimension the relevant issues for this survey.
68
Philipp Afe`che
The role of pricing in managing data transport services From an economics perspective the performance of a data communications network is measured by its total surplus, i.e., total customer utility minus total cost, and its division among customers and providers, which depends on the payment flow from the former to the latter. Utility and cost in turn depend on the specifics of network deployment and allocation to customer traffic. Pricing plays a key role in optimizing network deployment and allocation. Early debates on Internet congestion considered and rejected several nonpricing approaches, including voluntary user restraints and administratively mandated and technologically enforced usage quotas (cf. MacKie-Mason and Varian, 1995a). These approaches are inappropriate because of key features of the data communications environment: users and the provider(s) are independent self-interested decentralized decision makers with conflicting incentives and they have private information—users about their preferences, the provider(s) about network characteristics. As a result, neither users nor the provider(s) are well-equipped to make all allocation decisions. The use of pricing serves several functions that achieve a more appropriate distribution of decision making. Prices generate revenues for providers that allows them to cover costs and make profits. They shape the incentives of providers to invest in capacity. They shape the incentives of customers to choose certain quantities, types and times of service. This regulates the amount and composition of traffic admitted to the network over time and its allocation to bandwidth via specific routes or priority classes. As such pricing is an important economic congestion control tool that complements the technological network controls discussed above. It also makes network control more scalable, since a single or a few prices are sufficient to control the traffic demand of millions of users. By observing user reactions to different price levels providers may also gain important demand information that allows them to further improve their profits and/or social surplus. Pricing decisions and issues Managing transport services involves service design, pricing and production decisions. Reflecting the emphasis in the literature, this survey primarily focuses on pricing decisions but also considers capacity decisions. As noted earlier the chapter focuses on the case of a single provider with one of two performance objectives: welfare maximization or profit maximization. Pricing defines a set of tariffs for the offered set of service classes. A tariff for a given service class is a function that specifies the user’s total payment depending on two sets of usage attributes, the expected or actual bandwidth consumption per service unit and the quantity of service units. Depending on the context a service unit may be defined as the transmission of a data packet, a bulk transfer, a CBR or a VBR traffic source. In each case, one can quantify the expected or actual bandwidth consumption based on a priori known or measured parameters such as packet or file size, peak rate,
Ch. 2. Economics of Data Communications
69
mean rate, duration and network route that are inherent to the source and/ or specified by the service contract. Virtually all pricing analyses assume a communication network with a given structure and set of control mechanisms. This section categorizes pricing analyses further along several dimensions and outlines, for each dimension, the relevant issues for this survey. The number of network links and the information structure characterize the decision environment. The usage-sensitivity of tariffs, price and service differentiation, dynamic tariff fluctuations, and posted prices versus auctions refer to important issues in structuring and computing tariffs. Number of network links. The studies considered here assume networks with a fixed number of resources and topology. One can distinguish between services based on whether they are delivered over a single link or a multi-link network. In a multi-link network the bandwidth consumption and price of a service unit also depends on the set of links along its route. The studies of multi-link networks considered here typically assume a given set of fixed routes. Information structure. Pricing and capacity decisions hinge on demand information about utility functions and traffic source properties, and service usage decisions require information about QoS metrics such as delay or network load. Many pricing studies assume complete information about all these characteristics. Studies with incomplete information assume at least some of these characteristics to be the private information of users or the provider. One can distinguish between several cases of incomplete information. Cases where the provider has no or incomplete information on demand functions or users have no or incomplete system QoS information motivate the study of adaptive and auction-based price mechanisms. Cases where the provider has incomplete information on traffic source parameters or only aggregate information but no information on QoS preferences of individual users motivate the design of incentive-compatible tariffs that discriminate based on actual usage characteristics or service quality attributes. Usage-sensitivity of tariffs. A fundamental question is whether tariffs should at all be usage-sensitive. ‘‘Flat-rate pricing’’ has been and still is the prevalent tariff structure in the Internet retail market. A user pays a flat periodic subscription fee that only depends on the bandwidth of her access link. Thus her charge is related to her maximum sending rate but independent of actual usage, quality or time of service. Like much of the research activity on data pricing this survey, including the following discussion of the framework, focuses on usage-based pricing. It briefly turns to the question of flat-rate versus usage-based pricing in Section 6.2. Price and service differentiation. A central research issue focuses on how to tailor tariffs and services to users with heterogenous preferences and traffic properties. Most studies consider tariff design for a given service design, defined by a fixed number of service classes with a single type of QoS guarantee for all classes, and delivered by prespecified control mechanisms.
70
Philipp Afe`che
A few of these papers investigate the economic value of differentiation by comparing optimally priced systems with different numbers of service classes, or jointly optimize tariff design and the number of service classes. Sections 3–5 consider these studies by type of QoS guarantee. Section 6.3 briefly outlines the merits and challenges of alternative QoS designs and their delivery mechanisms. For each type of QoS guarantee the discussion in this survey distinguishes between pricing studies for uniform service or bandwidth differentiated services and those for delay or loss differentiated services. In general, tariffs may discriminate based on service class, usage and customer attributes. This survey does not consider tariffs that discriminate solely based on customer attributes (their preferences, identity, business versus home, etc.). It focuses on tariffs that are linear in the quantity of service units (e.g., throughput) and that may discriminate based on service class, and the expected or actual bandwidth consumption per service unit. (See Wilson, 1993, for a classic treatment of nonlinear pricing.) An important issue in price and service differentiation arises when the provider does not have complete information about demand characteristics. The case where the provider has incomplete information on traffic source parameters is particularly relevant for guaranteed services. The case where the provider has aggregate information but no information on QoS preferences of individual users is particularly relevant for delay or loss differentiated best effort services. Both cases call for the design of incentive-compatible tariffs that take into account users’ choice behavior. Dynamic tariff fluctuations. This survey distinguishes between static pricing and two types of mechanisms with dynamic tariff fluctuations, adaptive pricing and optimal dynamic pricing. Adaptive pricing mechanisms have the objective of finding the optimal static equilibrium prices and allocation in environments with no or incomplete information about demand functions. They resemble the classical tatonnement process. Price fluctuations serve the purpose of learning demand, and the analysis focuses on how to adjust prices to reach an equilibrium for a given stationary demand environment and issues of convergence speed and stability. Optimal dynamic pricing mechanisms seek to optimally adjust prices to the state of network load or demand to improve performance over static (state-insensitive) pricing. In contrast to adaptive pricing, here the equilibrium prices are dynamic, not static, over the time horizon, and demand functions are typically known. Research issues focus on characterizing the optimal price policy for a given service design and on quantifying the benefits of dynamic over static pricing. This survey further distinguishes between two forms of dynamic pricing. Load-sensitive dynamic pricing seeks to optimally adjust prices based on the network load to improve performance over static load-insensitive prices. The analysis typically considers stationary known demand functions that are independent of the time of service. Demand-sensitive dynamic pricing is closely related to peak-load pricing that has been important for telephone tariffs. It seeks to optimally
Ch. 2. Economics of Data Communications
71
adjust prices based on the state of demand at different times of service, to reduce traffic peaks and improve performance over static demand-insensitive prices. The analysis typically considers nonstationary known demand functions that depend on the time of service, e.g., the time of day. In an empirical study Edell et al. (1995) show that time of day prices may move 33% of traffic from peaks to less busy times. Posted prices versus auctions. The majority of papers analyze centralized pricing where the provider designs and posts tariffs. A fundamentally different approach is via auctions where pricing decisions are at least partly decentralized: users submit bids and the provider determines prices and allocations based on bids and previously announced payment and allocation functions. Starting with the ‘‘smart market’’ proposal of MacKie-Mason and Varian, (1995a), which was inspired by Vickrey’s (1961) second-price sealedbid auction, auction-based pricing approaches have received considerable attention in data pricing research, for services with different types of QoS guarantees. An important motivating factor for these efforts is the demandrevealing property of the classic second-price auction which allows the provider to attain the efficient allocation even if she has no or incomplete information on demand functions. A reasonable question to ask is whether this demand-revealing property is preserved in the data communications context and under what conditions. Section 6.1 surveys auction-based approaches and considers this question. 2.6 Discussion framework After describing major dimensions and issues of pricing decisions this section outlines the framework for their discussion. The core Sections 3–5 focus on the paradigm of economic optimality. Section 6 goes beyond the optimality criterion and also considers some implementation issues in pricing and service design. Guaranteed, best effort and flexible bandwidth-sharing services Sections 3–5 consider usage-based pricing decisions by type of QoS guarantee, for guaranteed, best effort, and flexible bandwidth-sharing services, respectively. These sections share the following organization. Not all parts are equally relevant for all service types but the common structure aims to provide an unifying framework for comparing the different research streams. 1. Features, modeling and overview. This introductory part reviews the main contract features and model characteristics that shape the context for their analysis, and outlines the issues discussed in the section. 2. Basic pricing and allocation principles. This part considers environments with complete information. It focuses on static socially optimal
72
Philipp Afe`che
3.
4.
5.
6.
pricing for fixed capacity but also discusses the relationship between socially optimal prices and capacity levels. Throughout it assumes a fixed service design that reflects in some sense the ‘‘basic level of service differentiation’’ for each service type: a single service class (best effort) or multiple bandwidth differentiated services (guaranteed services or flexible bandwidth-sharing services). The discussion emphasizes the single link case but also outlines how the results extend to a multilink network, typically with fixed routes. Where applicable this part also outlines how the results extend to a profit-maximizing monopoly. Incomplete information and adaptive pricing. This part considers issues that arise due to incomplete information. All three contract types consider the case where the provider has no or incomplete information on demand functions and study adaptive price mechanisms to find the right prices. The case where users have no or incomplete system QoS information is relevant for best effort and flexible bandwidth-sharing services since the network does not a priori commit to a certain QoS level, which calls for user adaptation to QoS changes. More differentiation. This part discusses work that goes beyond the basic forms of price and service differentiation considered in Part 2. Issues include pricing for service designs with delay or loss differentiated services; pricing for service designs with two different types of QoS guarantees (e.g., best effort and strict guarantees) and incentivecompatible tariff design under incomplete information on traffic source parameters (for guaranteed services) or on individual users’ QoS preferences (for multi-class best effort services). Optimal dynamic pricing. This part surveys studies of load- and demand-sensitive dynamic pricing. Maybe in part due to the mathematical challenges involved in their analysis, these pricing approaches have received comparatively little attention. Conclusions and directions. This part summarizes and suggests directions for future research.
Discussion After considering each contract type in isolation the survey then turns to broad questions that concern all contract types. 1. Auctions versus posted prices. This part surveys auction-based approaches and discusses their benefits and challenges, including their potential for demand revelation depending on the level of resource and time aggregation of the service units sold, dynamic effects and the presence of delay externalities. 2. Flat-rate versus usage-based pricing. This part outlines the debate on flat-rate versus usage-based pricing, outlines some results on two-part tariffs that combine a flat and a usage-sensitive component, and characterizes the
Ch. 2. Economics of Data Communications
73
fundamental trade-off between optimality and implementability criteria that should be considered in the design and evaluation of any pricing mechanism. 3. Providing QoS: overprovisioning versus control. This part outlines the merits and challenges of alternative QoS designs and their delivery mechanisms. 3
Pricing guaranteed services
3.1 Features, modeling and overview Features Guaranteed service contracts specify QoS parameters the network promises to deliver, traffic parameters that constrain the user’s traffic profile, and a tariff that determines the user’s charge. Guaranteed services are suited for real-time applications with CBR or VBR traffic sources. QoS parameters express the maximum tolerable congestion level in terms of deterministic or probabilistic traffic flow performance statistics such as minimum throughput rate; average, maximum delay or delay variation (jitter) and packet loss probability. This section focuses on static QoS parameters that are agreed upon at contract inception and remain fixed over its lifetime. Traffic parameters may include metrics such as the peak rate, mean rate and burstiness of the source. In the basic case all traffic parameters are a priori known to the provider and statically defined at the start of the contract, serving as the basis for tariffs and network control decisions. If some traffic properties are a priori not known, contracts may give users more flexibility in choosing traffic parameters and tariffs that depend not only on a priori known (declared) traffic parameters but also on a posteriori information based on actual traffic measurements, cf. Kelly (1994) and Courcoubetis et al. (2000a, 2000b). Since the network strictly guarantees quality there are no congestion externalities: a user’s net utility function depends only on the attributes of her own service contracts—quantities, QoS and traffic parameters, and tariffs, but not on other users’ network usage. The network uses several control mechanisms to ensure the promised quality levels. Among these the ones directly visible to users are admission control, which determines whether the network has the required resources to meet contractual QoS parameters for traffic with a certain profile, and traffic policing, which holds admitted calls to contractual traffic parameters. A key issue in managing guaranteed services is to estimate the bandwidth consumption of a source at each link along its route. This measure of resource consumption forms the basis for admission control, resource reservation and pricing decisions. Pricing can regulate demand for connection requests and the traffic properties of accepted connections. The control mechanisms operate at the level of individual data flows which requires considerable complexity and ‘‘intelligence’’ in the core of the network.
74
Philipp Afe`che
Modeling There is a fixed set of CBR or VBR traffic source types, each with its own statistical properties, pre-specified QoS requirements and traffic parameters. Typically there is a fixed set of service contracts, each tailored to one of the traffic source types. A customer’s utility is a function of the quantities of traffic contracts she buys. The bandwidth requirement of an active CBR source equals its constant data rate. The bandwidth requirement of an active VBR source is much more complex; it depends not only on its own traffic parameters and QoS requirements but also on the traffic mix and control policies on the link, and on link and buffer capacity levels. The effective bandwidth of a traffic source maps these multiple factors into a one-dimensional measure of resource consumption for the connection. In some cases the traffic properties of a VBR source, and hence its effective bandwidth, may not be fully known a priori but only a posteriori after actual usage measurements. Pricing analyses of guaranteed services consider primarily two environments. Studies of VBR sources typically focus on a static environment with a given connection mix, since the effective bandwidths of VBR sources are highly dependent on the traffic mix. Studies of CBR sources typically consider a dynamic environment where individual connections arrive and depart over a time horizon much longer than their own duration. These dynamics are easier to analyze for CBR than for VBR sources since their resource consumption is invariant to the overall traffic mix. The key network control mechanisms experienced by the user are admission control coupled with resource reservation and traffic policing. Other network control mechanisms, including routing at connection acceptance and dynamic bandwidth allocation through scheduling of admitted data packets at network nodes, are essentially invisible to users since the network guarantees quality. Overview Section 3.2 first reviews the notion of effective bandwidth, a key measure of bandwidth consumption for traffic sources with guaranteed service. It then discusses the basic principles for optimal static pricing in a static environment for bandwidth differentiated services: the connections they serve have different but fully known traffic characteristics, a single common QoS parameter and are given FIFO service. The analysis focuses on social optimization for the single-link case and discusses extensions to a monopoly and a multi-link network. In this static environment the socially optimal linear prices for guaranteed service classes are proportional to the effective bandwidths of their connections, whereas the monopoly prices also depend on the demand elasticity. The section then outlines the difference to static analyses of dynamic environments, where socially optimal static prices are not simply proportional to effective bandwidths but also depend on the blocking probability. Section 3.3 reviews work on iterative adaptive price
Ch. 2. Economics of Data Communications
75
mechanisms that are designed to attain the socially optimal bandwidth allocation when the network lacks information on utility functions. Section 3.4 surveys several cases with more service differentiation compared to the basic amount of service differentiation considered in Section 3.2: First, multiple delay or loss differentiated QoS classes supported by priority-service at network nodes; second, settings where some traffic source properties are a priori not known to the provider who therefore designs a menu of multi-part tariffs to elicit this information from users. Each tariff is linear in measured usage parameters of the connection, time and data volume in the simplest case, with coefficients that vary based on the user’s a priori declared traffic parameters. Third, the case where a provider offers guaranteed and best effort service over the same infrastructure. Section 3.5 discusses analyses of load-sensitive and demand-sensitive dynamic pricing. Section 3.6 summarizes and outlines directions for future research. 3.2 Basic pricing and allocation principles Consider the pricing problem for a single capacitated link in a static environment with a fixed mix of potential connections. How to allocate the link to different types of traffic sources and at what prices? Solving this problem requires two steps. First, the network needs to understand the resource requirements of traffic sources that request connections so it can characterize the admission region, defined as the set of feasible contract quantities that it can simultaneously offer without violating the QoS constraints. Second the network needs to choose the desired contract mix within the admission region and prices that support the corresponding demand. Resource consumption and admission region Consider a provider offering K types of service contracts, each tailored to a traffic source with certain statistical properties and QoS parameters, and let x ¼ (x1,x2, y, xK) denote the number of active contracts of each class. The admission control problem is to ensure that the traffic mix always remains within the admission region. The admission control problem is simple in the case of CBR sources such as certain voice and video applications (leased virtual circuits or dialup access connections may also be modeled as CBR sources): the network simply reserves a bandwidth amount equal to the constant source data rate for each admitted contract. Traffic streams do not interact, the QoS of a traffic stream directly corresponds to its resource requirement—defined at any point in time by a known scalar quantity, and the resource consumption of a traffic mix is additive in those of its component streams. However, the admission region can be difficult to evaluate for VBR traffic sources due to their stochastic and heterogenous nature, especially if QoS constraints are multi-dimensional. When traffic sources have randomly fluctuating bandwidth requirements the network reserves
76
Philipp Afe`che
for each source less than its peak data rate and uses statistical multiplexing for multiple sources to dynamically share the link capacity. This dramatically increases the complexity of the admission control problem: traffic sources now interact through the sharing mechanism, and the relationship between traffic source properties, QoS requirements and the acceptable capacity utilization level is far from obvious. For example, to provide acceptable QoS to bursty sources with tight delay and loss requirements it may be necessary to keep the required average utilization of a link below 10%, while for constant rate sources the average utilization may be close to 100%. The concept of effective bandwidth, based on the seminal paper of Hui (1988), has given rise to elegant and powerful solutions to this resource measurement problem. It provides a one-dimensional measure of resource usage which adequately represents the trade-off between sources of different types taking proper account of their varying statistical characteristics and multi-dimensional QoS requirements. The main result on effective bandwidths for the single-link case is outlined as follows. (See Kelly, 1996, and Chapter 4 in Courcoubetis and Weber, 2003, for reviews of effective bandwidths.) Suppose that the K traffic classes are multiplexed over one FIFO link with bandwidth C and buffer size B. The traffic sources may have different, a priori fully known, statistical properties but share a single common QoS parameter, e.g., that the cell loss probability of any source not exceed 10–8. The main result defines and characterizes the effective bandwidth of traffic source k by a function ak (s, t) and the link’s effective capacity as a function C ðs; tÞ: The parameters s and t depend on the link capacity C, the buffer size B, and on the operating point x of traffic sources and their statistical properties and QoS parameter. Based on these quantities the admission control problem simplifies to verifying whether a traffic mix x satisfies the linear constraint K X
xk ak ðs; tÞ C n ðs; tÞ.
(1)
k¼1
Thus, effective bandwidths can be interpreted as defining a local linear approximation to the boundary of the admission region at an operating point x. Socially optimal prices in static environments The basic problem of socially optimal static linear pricing of guaranteed services for VBR sources hinges entirely on the characterization of their effective bandwidths. Since the effective bandwidth of a VBR source is sensitive to the traffic mix, most effective bandwidth models assume a fixed set of contract types or service classes. Under this assumption the characterization of resource consumption by effective bandwidths reduces the analysis of static pricing for such contracts to a classic multi-product pricing problem with linear capacity constraints.
Ch. 2. Economics of Data Communications
77
For illustration, let U (x) denote the total system utility as a function of the traffic mix x. Assume for now that it is known to the provider and that the direct marginal cost of a contract is zero. Let b denote the Lagrange multiplier of the constraint (1). The problem of finding the socially optimal allocation of contracts is ! K X max UðxÞ þ b C ðs; tÞ xk ak ðs; tÞ . (2) x0;b0
k¼1
At the socially optimal traffic mix x ; the marginal utility of each contract type must be equal to the product of its effective bandwidth by the shadow price of the admission region constraint @Uðx Þ ¼ b ak ðs; tÞ; k ¼ 1; 2; . . . ; K. (3) @xk Suppose, a linear pricing scheme that charges a unit price Pk per class-k contract. Given these prices users demand contracts up to the point where their marginal utility equals the prices @UðxÞ ¼ Pk k ¼ 1; 2; . . . ; K. (4) @xk Hence, if the socially optimal traffic mix is on the boundary of the admission region, i.e., the constraint (1) is binding, then b 40 and the socially optimal unit prices of contracts are proportional to their effective bandwidths Pk ¼ b ak ðs; tÞ;
k ¼ 1; 2; . . . ; K.
(5)
Network. The single-link social optimization problem (2) naturally generalizes to the multi-link network case by adding for each link one capacity constraint of the form (1). Price and capacity decisions. The joint problem of socially optimal pricing and capacity investment is discussed in Section 4.2 together with the case of best effort services. Profit-maximizing prices in static environments Of course the profit-maximizing prices in a static environment also depend on the demand elasticity, not just on resource shadow prices and effective bandwidths. The respective analysis and results follow immediately from the classical multi-product monopoly problem with linear technology. Optimal static prices in dynamic environments The principle for socially optimal pricing in static environments given by (5) ceases to hold in dynamic environments where connections arrive and depart over time. Specifically, the socially optimal static linear prices depend not only on the effective bandwidths of individual sources, but also on the system’s blocking probability which measures the fraction of time the
78
Philipp Afe`che
system spends on the boundary of the admission region. See Kelly (1991) and Ross (1995) for theoretical and computational issues in the analysis of blocking probabilities and their approximations. Courcoubetis and Reiman (1999) and Paschalidis and Tsitsiklis (2000) study the problem for a single link, and Paschalidis and Liu (2002) tackle the network case. They all model the service provider as a multi-server loss system and consider the case of CBR sources, which is more tractable since the effective bandwidth of each source is independent of the traffic mix. Courcoubetis and Reiman (1999) consider static pricing under revenue and welfare maximization using an asymptotic analysis that simplifies the calculation of blocking probabilities. 3.3 Incomplete information and adaptive pricing The analysis discussed so far assumes that the provider knows the customer demand functions. Low and Varaiya (1993) and Thomas et al. (2002) relax this assumption. They propose and study decentralized tatonnementstyle iterative adaptive price mechanisms to attain the socially optimal allocation in a static environment (with a fixed set of possible connections) when the network does not know the customer demand functions. Rather than relying on the notion of effective bandwidths, they assume that the relationships between QoS and resource requirements are explicitly given as opposed to derived from first principles based on traffic source properties. Low and Varaiya (1993) model ATM networks as pools of capacitated transmission links and buffers. They rule out resource sharing across connections through statistical multiplexing. They specify the QoS constraints of each service class in terms of a minimum bandwidth and a ‘‘burstiness curve’’ that captures acceptable bandwidth-buffer size substitutions. The socially optimal unit prices for buffer and link capacity equal their shadow prices. Their decentralized algorithm has the network set resource prices and users respond with resource requests in iterative fashion. Thomas et al. (2002) consider more general QoS resource allocation relationships and a market mechanism for a competitive economy with three price-taking agent types, users, resource providers and service providers, and an auctioneer who updates prices based on the observed aggregate resource supply and demand. Their distinctive contribution is to prove convergence of their algorithm in a finite number of iterations to a resource allocation that is arbitrarily close to the socially optimal solution. Similar iterative mechanisms for settings with congestion externalities are discussed in Section 4.3. 3.4 More differentiation The pricing analysis of Section 3.2 focuses on bandwidth differentiated services. It assumes that all traffic sources share a single QoS bound, that the traffic properties of each source are a priori fully known, and that the
Ch. 2. Economics of Data Communications
79
network only offers guaranteed services. This section outlines work that relaxes these assumptions. Delay or loss differentiated services: multiple QoS parameter constraints If there are multiple constraints on QoS parameters that differ by traffic source the admission region can be reasonably well approximated by multiple linear constraints of the form (1), one for each QoS constraint represented at that link. In the network case there is one set of such constraints for each link. Effective bandwidth characterizations span a range of models and service disciplines. Zhang (1993), Elwalid and Mitra (1995), de Veciana and Walrand (1995), Kulkarni and Gautam (1997), Zhang (1997), and Berger and Whitt (1998a,1998b) study effective bandwidths for systems that implement service differentiation through priority scheduling disciplines. Elwalid and Mitra (1992) and Kulkarni et al. (1995) consider an alternative priority mechanism with loss priorities: it uses FIFO scheduling but rejects or discards low priority data packets if the workload is above a threshold. It appears that the pricing of such differentiated guaranteed services with two or more QoS bounds has not received significant attention. However, from the perspective of economics what matters most about these service designs is their output, i.e., the characterization of resource consumption for each traffic mix, which determines the admission region and the optimal allocation and prices. The implementation details of these network control mechanisms are hidden from users; all that matters to them is that their QoS requirements be met. A priori unknown traffic parameters In practice the network may not a priori have the necessary information on traffic source properties with given QoS parameters to compute their effective bandwidths. It must instead rely on traffic parameters announced at contract inception which raises the following questions: how to estimate effective bandwidths for admission and pricing decisions, and how to structure price tariffs to set the right user incentives for truthfully declaring their best estimate of traffic properties? For given QoS parameters users’ traffic parameter choices affect both the value and the effective bandwidth of their transmissions. Charging contracts based only on a priori information or only on a posteriori measurements is problematic. Charging only based on a priori estimates of their average effective bandwidths, using historic data for similar traffic, gives users the incentive to use the maximum that their contractual traffic parameters allow. This in turn leads the operator to calculate and charge users the maximum effective bandwidths possible given their contract parameters, which discourages users with a lower effective bandwidth from buying such contracts. At the other extreme, charging only based on a posteriori measurements may leave the network with too much of the risk associated with the opportunity cost of
80
Philipp Afe`che
admitting contracts with high a priori, but low actual effective bandwidth and revenue. Kelly (1994) and Courcoubetis et al. (2000a, 2000b) develop the following solution to this incentive issue for sources with a priori known (static) traffic parameters specified in the contract such as the peak rate, and a priori unknown and measured (dynamic) traffic parameters such as the mean rate. They construct charges based on the effective bandwidth computed from the static parameters and measurements of the dynamic parameters. Users are offered a menu of multi-part tariffs. Each tariff is linear in measured usage parameters of the connection with coefficients that depend both on the static parameters and on users’ declared estimates of the dynamic traffic parameters. This scheme gives rise to charges proportional to effective bandwidths and is incentive-compatible, giving risk-neutral users who minimize their expected charge the incentive to truthfully report their expected value of their a priori unknown traffic statistic. In the simplest case this approach gives rise to time-volume charging whereby the total charge is a linear function of the measured connection time and data volume with coefficients that depend on the known static peak rate and the user’s declared mean data rate. Guaranteed and best effort service Wang et al. (1997) and Maglaras and Zeevi (2005) study pricing for the case where a provider offers guaranteed service (for real-time CBR sources) and best effort service over the same infrastructure. 3.5 Optimal dynamic pricing Dynamic pricing studies of guaranteed services appear to have focused on CBR sources so far. Load-sensitive dynamic pricing Paschalidis and Tsitsiklis (2000) and Paschalidis and Liu (2002) derive and compare the performance of the optimal static and load-sensitive dynamic prices under revenue and welfare maximization. They consider a market with a fixed set of service classes, each defined by its fixed bandwidth requirement, an exponentially distributed holding time, and a pricesensitive Poisson arrival process. They assume that demand is stationary, i.e., the arrival rates only depend on prices, not on the time of service. The provider is assumed to know all demand characteristics. Paschalidis and Tsitsiklis (2000) model the provider as a single-node multi-server loss system who chooses class-specific prices. They show that in an asymptotic regime of a large system with many relatively small users, the performance of optimal dynamic load-sensitive prices is closely matched by suitably chosen class-dependent but static, i.e., load-insensitive, prices. Paschalidis and Liu (2002) extend this result to a network with fixed routing and allow
Ch. 2. Economics of Data Communications
81
for demand substitution across service classes. This result is quite intuitive: as the number of users and the capacity increase while the utilization remains constant, the law of large numbers takes over, eliminating statistical fluctuations so that loss probabilities become small; this in turn reduces the value of optimizing prices based on network load. This result suggests that, at least in settings with many small users, time-of-day prices that adjust to slow changes in average network load may match the performance of prices that fluctuate more rapidly based on instantaneous changes in network load. These studies assume that prices for a contract are fixed during its lifetime, even though prices may vary across contracts based on congestion levels. In contrast, Semret and Lazar (1999) analyze a proposal to price loss system services in a stationary demand environment using load-sensitive spot prices and a derivative market mechanism. To avoid being dropped before service completion, a user must either continuously pay the spot price, which fluctuates based on auctions held among recent requests, or purchase a hold option up-front which gives her the right to buy service for a fixed price at any time up to a specified future date. They characterize the equilibrium spot price and the reservation fee as the fair market price of the hold option. Demand-sensitive dynamic pricing Wang et al. (1997) develop a procedure to determine optimal demandsensitive dynamic prices. They assume given time-varying demand functions that are known to the provider. They consider an integrated-services network which offers multiple guaranteed service classes to CBR sources and a single class of best-effort service. Requests for guaranteed service are lost if not admitted immediately while those for best-effort service are queued until they are transmitted. 3.6 Conclusions and directions The economic analysis of guaranteed services crucially hinges on the ability to adequately quantify key measures related to resource consumption: the effective bandwidths of individual traffic sources, and the admission region and blocking probabilities of the network. Much of the technical mechanisms and analysis underlying how the network delivers a set of QoS requirements is hidden from the users’ decision problem; they only care about what the service class attributes are. Pricing in static environments closely follows the classic analysis of multi-product pricing with linear capacity constraints, the main challenge is to characterize the effective bandwidth of each ‘‘product.’’ Hence, the socially optimal prices of connections in a static environment with perfect traffic source information are proportional to their effective bandwidths whereas the corresponding monopoly prices also depend on demand elasticities. In the absence of
82
Philipp Afe`che
perfect a priori traffic source information, tariffs with components based on both a priori declared and ex post measured traffic source properties set the right incentives for users to declare their best estimate of traffic parameters. Studies of static and load-sensitive dynamic pricing in dynamic environments have so far focused on CBR sources and suggest that the relative benefit of load-sensitive dynamic pricing may be minor in settings with many small users. Several issues appear to have received little attention so far and to merit further research. Dynamic environments of VBR sources. It appears that pricing studies for VBR sources have so far focused on static environments. Determining the optimal (static or dynamic) prices for VBR sources in dynamic environments faces the additional challenge that their effective bandwidths depend on the traffic mix, whereas the bandwidth requirements of each CBR source is independent of the traffic mix. One approach is to define an ‘‘average’’ effective bandwidth for each VBR source and to apply the analytical techniques for loss networks with CBR sources to those with VBR sources. Are there suitable approximations of such ‘‘average’’ effective bandwidths for use in dynamic environments? If so, how good are these approximations relative to more precise approaches that re-compute the effective bandwidths as the traffic mix changes? What are the pricing implications of different effective bandwidth characterizations? Value of load-sensitive dynamic pricing. Paschalidis and Tsitsiklis (2000) and Paschalidis and Liu (2002) show that the value of load-sensitive pricing over static pricing is negligible in an asymptotic regime with many small users. Under what conditions does load-sensitive pricing generate a significant performance improvement? Demand-sensitive dynamic pricing. Existing analyses, e.g., Wang et al. (1997), assume that the nonstationary demand rate is a function of time t and of prices only at that time, but does not depend on prices at other times. In other words, customers are nonstrategic with respect to the time of their transmission request. How should prices be set to smooth demand peaks when customers substitute over time? Analyses of peak-load pricing with substitution effects has received some attention outside the data communications context, e.g., by Bergstrom and MacKie-Mason (1991) for a static environment and by Shy (2001) for a dynamic environment. Service design. Much of the analysis of guaranteed services takes as a starting point a given set of pre-defined traffic sources and service classes. However, customers may choose to aggregate and split traffic. What are the incentives of customers for doing so and how should providers define and price service bundles of multiple connections? Courcoubetis and Weber (2003, p. 212) observe that the incentive to split traffic may arise in certain situations and that it can be discouraged by adding a fixed charge per connection. However, the broader issue of how to design the mix and pricing of service classes appears to have received little systematic attention.
Ch. 2. Economics of Data Communications
4
83
Pricing best effort services
4.1 Features, modeling and overview Features Congestion occurs whenever demand for resources exceeds their supply. Since data networks face significant demand variability in terms of the number of active users and their data transmissions, they are prone to intermittent congestion even if on average underutilized. The guaranteed services just discussed commit to strict quantitative QoS bounds for each individual transmission flow before it starts which the network then ensures by tightly controlling congestion via admission control, resource reservation and related mechanisms. In contrast, best effort services make no such quality promises and do not constrain users’ traffic inflow to the network. As a result the QoS performance of a given user’s transmission is determined not only by her own behavior and the network control mechanisms, but also depends on other users’ offered load—users create and are subject to congestion externalities. Congestion may lead to performance degradation in the form of data delays, losses and throughput limitations that reduce customer utility. Best effort services are suited for elastic applications which tolerate this QoS degradation. Recall from Section 2.2 that this survey adopts a literal definition of best effort services and distinguishes them from flexible bandwidth-sharing services that are discussed in Section 5. As understood here, best effort services offer no quality guarantees whatsoever and impose no constraints on the traffic that users may send into the network, irrespective of congestion conditions. They have no built-in traffic flow control and feedback mechanisms for avoiding congestion. Thus, users have little information on precisely when and how to adjust their sending rates to match fluctuations in bandwidth availability. As a result there can be a significant mismatch between a user’s sending rate and her realized bandwidth at any given time, which translates into delays and/or losses. The Internet transport service provided by the UDP protocol is a typical example: packets may be delayed, lost or arrive out of order without notice. It is used for transmissions where more control is not worthwhile, e.g., for small bursts of data or for high-speed multimedia applications where resending data is of no value. By contrast, flexible bandwidth-sharing services offer dynamic bandwidth guarantees and give users some control over and the ability to adapt to their allocation, while their transmissions are in progress. The Internet transport service provided by the TCP protocol is a prime example. It guarantees zero packet loss by retransmitting lost packets and dynamically controls the peak rates at which competing connections are allowed to transmit via adaptive flow control and feedback mechanisms. Its distinctive features are closed-loop control, on fast time scales and at the level of individual flows: individual end user applications receive congestion signals from network
84
Philipp Afe`che
nodes every few tens of milliseconds that vary with the state of packet queues and adapt their sending rates correspondingly quickly to track changes in their bandwidth allocations. Keeping in mind that the current Internet—including the TCP protocol— is often broadly referred to as a ‘‘best effort network,’’ the distinction between best effort and flexible bandwidth-sharing services made here is useful, since the respective pricing and resource allocation decisions occur on different time scales that call for somewhat different analytical frameworks and perspectives. In the case of flexible bandwidth-sharing services the fast feedback loops afford users (or agents on their behalf) some control over their transmissions while they are in progress. In the case of best effort services the notion is that feedback loops between sending rates and congestion levels are relatively slow—so slow that users have no control over a given individual flow through the network once it is in progress. Thus, users make all decisions concerning an individual flow or a collection of such flows covered by their contract at the start of the respective transmissions, based on the contract parameters and performance information available at that time. In this sense, best effort service contracts only have static parameters. Since the network does not make any strict QoS guarantees per user, the notion is that users, prior to sending data, estimate or receive information on aggregate QoS statistics, e.g., the delay distribution or packet loss ratio over all flows. One can think of these estimates as the expected QoS metrics of a given best effort service. Best effort services—unlike guaranteed services—hardly rely on technological demand regulation mechanisms, except through the bandwidth constraints of users’ access links. Pricing is therefore the only direct demand regulation tool. Demand is also regulated indirectly based on customers’ response to and anticipation of congestion. Best effort services may also rely on a range of supply allocation mechanisms to manage QoS performance, cf. Gevros et al. (2001). This survey considers the role of packet scheduling, link partitioning and routing mechanisms in controlling congestion and creating quality differentiated services. These tools may play a larger role in a future Internet with multiple classes of best effort or ‘‘betterthan best effort’’ service, somewhat similar to the DiffServ architecture proposal of the IETF. Modeling The following generic model of congestion externalities illustrates the analysis framework for best effort services. It is worth noting that several model features and results presented here are developed in congestion pricing studies that are not specifically targeted to data communications. Nevertheless they consider a collection of issues and models that are quite relevant in this setting. For now consider a single congestible resource with capacity K and N users (or user segments), indexed by n ¼ 1,2, y, N. Let xn be user n’s average usage
Ch. 2. Economics of Data Communications
85
or transmission rate, i.e., the number of requests per unit time, and let x denote the vector of usage rates. Depending on the context, the usage rates x may measure the throughput of bits, data packets or cells, or the throughput of higher level data flows such as email messages or file downloads. Let s¯n denote the average size of user n’s data files or packets. For now suppose that all file or packet sizes are i.i.d., with average resource requirement equal to one unit of capacity ð¯sn ¼ 1Þ and let X :¼ SN n¼1 xn be the average system load or resource requirement per unit time. An important issue in the economic analysis of best effort services is how users forecast the QoS performance of their transmissions and based on what network state information. For now assume that users lack dynamic system state information and instead forecast or receive an announcement of aggregate performance statistics. Further assume that the expected utility of user n, denoted by the differentiable function un (xn, X, K), depends on the system performance only through the average system load X and capacity K. It is increasing and concave in her own usage rate xn ð@un =@xn 40 and @2 un =@x2n o0Þ: In the presence of congestion cost the utility decreases in the load X ð@un =@X o0Þ and increases in the capacity K ð@un =@K40Þ: This reflects the negative quality impact of congestion and the fact that in the presence of demand variability, congestion effects such as queueing delays or data loss occur intermittently even if the system is on average underutilized, i.e., XoK. In summary, customer segmentation or differentiation may by along several attributes: user (segment) n is characterized by an (typically Poisson) arrival process for transmission requests at rate xn that depends on prices and congestion levels, a random packet or file size with mean s¯n and an utility function un (xn, X, K). Congestion may impact QoS and customer utility via three effects: data delay, loss and throughput or bandwidth limitations. Based on these attributes one can identify the following types of utility models. Delay-sensitive utility. Most studies focus on delay (response or throughput time) as the quality attribute linking congestion and customer utility. It measures the time from request to completion of a packet or flow transmission, including time in buffers and in transmission. Delay adversely affects the value of interactive applications such as online data bases and also the value of time-sensitive information. Following Naor (1969) the canonical delay cost model assumes that utility is additive in service values and delay cost, i.e., a transmission of user n with value u generates utility u – dn (t) where the delay cost function dn (t) increases in the total delay t. The delay cost functions are typically assumed to be linear: dn (t) ¼ dnt, where dn>0 is a constant marginal delay cost. Dewan and Mendelson (1990), Van Mieghem (2000) and Afe`che and Mendelson (2004) also consider increasing convex delay costs. An important question is how users estimate the delay t. For now assume that users lack dynamic system state information and instead compute their expected utility based on their forecast or the provider’s announcement of the steady-state delay distribution. This is a common assumption in many queueing models that focus on static equilibrium
86
Philipp Afe`che
analyses: the system state evolves dynamically but users’ transmission decisions are static. This time-averaging sacrifices the ability to study the dynamic relationship between congestion and pricing but yields tractability and insights into system properties that are pertinent on longer time scales. In this case, a user’s expected utility per unit time has the form un ðxn ; X ; KÞ ¼ V n ðxn Þ xn Dn ðX ; KÞ;
n ¼ 1; 2; . . . ; N,
(6)
where Vn (xn) is the value of usage rate xn under delay-free service and Dn(X, K) is the expected delay cost of a transmission (the time average in steady-state), which increases in the average load ð@Dn =@X 40Þ and decreases in the capacity ð@Dn =@Ko0Þ: The expected delay cost is defined as Dn ðX ; KÞ :¼ E½d n ðT n ðX ; KÞÞ where the random variable Tn is the steadystate delay and represents the QoS. In the case of uniform or undifferentiated service, i.e., if all transmissions are treated the same, Tn ¼ T for all n. The analysis of usage and pricing decisions hinges on a quantitative understanding of the relationship between the QoS performance of each class and the characteristics of traffic demand (arrival processes and resource requirements of transmission requests), network resources (number, topology and capacity of links and buffers) and control mechanisms (e.g., scheduling disciplines such as FIFO, strict priorities, processor sharing, etc.). For example, many studies that consider delay-sensitive customers focus on queueing systems with infinite buffer size. For a resource modeled as an M/M/1 system with infinite buffer and FIFO service, the delay distribution is PrfTðX ; KÞ tg ¼ 1 expftðK X Þg
(7)
and the expected utility under linear additive delay costs satisfies un ðxn ; X ; KÞ ¼ V n ðxn Þ xn Dn ðX ; KÞ ¼ V n ðxn Þ xn d n
1 . K X (8)
Under the common assumptions that the delay cost functions dn (t) are unbounded and buffer size limitations are insignificant it is optimal to underutilize the system since the expected delay cost is unbounded in the utilization and the expected utility is negative at close to full utilization, i.e., Dn ðX ; KÞ ! 1 and un ðxn ; X ; KÞ ! 1 as X ! K: The additive delay cost model (6) is inadequate if the delay cost and value of a transmission are correlated, for example, as in financial or industrial markets where delays in the execution of electronic trades may deflate the investor’s expected profit, or when delays in video transmissions reduce the image quality. Afe`che and Mendelson (2004) introduce a generalized delay cost structure that captures this interdependence by augmenting the standard additive model with a multiplicative component. In this model, a transmission with value u and delay t has utility u qn ðtÞ d n ðtÞ where
Ch. 2. Economics of Data Communications
87
the delay discount function qn (t) is decreasing and qn(0) ¼ 1. The expected delay discount function satisfies Qn ðX ; KÞ :¼ E½qn ðTðX ; KÞÞ 1: It decreases in the load X and increases in the capacity K. The expected utility in this model is given by un ðxn ; X ; KÞ ¼ V n ðxn Þ Qn ðX ; KÞ xn Dn ðX ; KÞ.
(9)
As discussed below the structure of delay costs can significantly impact the efficiency of a monopoly. Loss-sensitive utility. Only a few studies focus on loss as the quality attribute linking congestion and customer utility, which occurs in systems with finite buffers. Loss may measure packet loss, which may impact audio and video transmission quality (text does not tolerate data loss), or blocking of higher level service requests such as dialup connections. Marbach (2004) models a system where loss is naturally interpreted as packet loss. Courcoubetis and Reiman (1999) consider a system where loss represents blocking. The system serves transmissions with a fixed bandwidth requirement over a random holding time. A service request is blocked whenever its bandwidth requirement exceeds the supply. If all transmissions have equal resource requirements then they face the same steady-state blocking probability B(X, K), which is the QoS measure in this context. It increases in the average load X and decreases in the capacity K. Expected utility per unit time is of the form un ðxn ; X ; KÞ ¼ f n ðxn ; BðX ; KÞÞ;
n ¼ 1; 2; . . . ; N
(10)
where the function fn increases in the offered usage rate xn and decreases in the blocking probability B. The effective usage rate of user n is xn ð1 BðX ; KÞÞ: For example, if there are K circuits without buffers, arrivals of transmissions follow a Poisson process with rate K and each requires one circuit for an exponentially distributed time, then the blocking probability satisfies Erlang’s classic formula BðX ; KÞ ¼
ðX =KÞK =K! . K P k ðX =KÞ =k!
(11)
k¼0
Studies with delay- or loss-sensitive utility functions operate on the premise that congestion may be significant on the time scales where customers make their usage decisions. This is natural for the best effort service context as defined above where congestion avoidance mechanisms are inherently lacking. Throughput-sensitive utility. The work discussed in Section 5 takes a different approach. It hinges on the premise that congestion is and can be maintained at low levels by letting users dynamically adapt their sending rates, while transmissions are in progress in response to rapid congestion feedback signals. These studies typically model the throughput or bandwidth
88
Philipp Afe`che
as the only explicit utility function attribute. User n has utility un(xn) per unit time where xn is naturally interpreted as the throughput of a flow in packets or cells. Delay or losses play a role as a basis for congestion feedback signals, but they are not explicit attributes of utility functions. User decisions. Economic analyses of best effort services consider various user transmission decisions that mainly depend on the service design and the availability of system state information. The vast majority of studies assume that users do not strategize over the arrival times of their transmission demands; random arrival processes are exogenous to the models, with arrival rates depending on prices and expected QoS levels. The basic user decision is whether or not to buy transmission service and if so, how much. In studies of differentiated services (two or more service classes), users also get to choose among all service menu options unless the provider can distinguish among users and so limit their options. In systems where users lack dynamic state information they make static transmission decisions based on the forecast of steady-state QoS statistics; in static equilibrium analyses once for an infinite horizon, in adaptive pricing studies repeatedly—once in each of successive demand periods. If users do have dynamic state (queue length) information they make transmission decisions repeatedly on a case-by-case basis. Overview The remainder of this section mostly focuses on delay as the explicitly modeled utility function attribute, reflecting the emphasis in the literature, although some results also hold in settings where congestion leads to data loss. Most of the references assume the above described additive delay cost model, but to provide an unified framework the discussion presents the results for generic utility functions un ðxn ; X ; KÞ: Section 4.2 discusses the basic principles of socially optimal and profit-maximizing static pricing and capacity decisions for uniform service, assuming that users have perfect steady-state delay forecasts and the provider knows demand. It starts with the single-link case and transmissions with identical resource requirements and later relaxes these assumptions. Section 4.3 considers two approaches that go beyond the static framework to handle users’ and the provider’s information constraints: providing congestion information for users and iteratively updating price and delay forecasts over successive demand periods. Section 4.4 discusses decisions involved in creating differentiated best effort services, including how many service classes to offer (service design), how to allocate bandwidth to these classes (service production), and how to price and target these classes to users (pricing), considering users’ choice behavior in designing the price-QoS menu. Section 4.5 outlines studies on optimal dynamic pricing, in particular on load-sensitive pricing, and observes the relative sparsity of work in this area. Section 4.6 concludes with suggestions for future research.
Ch. 2. Economics of Data Communications
89
4.2 Basic pricing and allocation principles The following discussion considers static pricing for uniform (undifferentiated) service. It builds on the above model framework. It starts with the case of a single link and transmissions with identical resource requirements and later relaxes these assumptions. The mathematical analysis assumes that users do not have dynamic system state information—they make static decisions based on static forecasts of system performance—but several results apply in settings where they do. Socially optimal congestion prices The basic principle for socially optimal congestion pricing is simple and well known: charge customers their congestion externality, i.e., the marginal congestion cost they impose on the system. This ‘‘internalizes the externality’’ and thereby aligns the system and individual user incentives, yielding the socially optimal allocation. The system’s social welfare rate, W, is defined as the rate of aggregate utility minus capacity cost C(K), which is assumed to be increasing and convex W ðx; KÞ :¼
N X
ui ðxi ; X ; KÞ CðKÞ.
(12)
i1
The problem of finding the socially optimal allocation for fixed capacity level K is max x0
W ðx; KÞ
s:t: X K.
(13)
Let lðx; K; bÞ ¼ W ðx; KÞ þ bðK X Þ be the Lagrangian where b is the shadow price of the capacity constraint. The socially optimal usage rates x and shadow price b satisfy N X @un ðxn ; X ; KÞ @ui ðxi ; X ; KÞ þ b ; ¼ @xn @X i¼1
b ðK X Þ ¼ 0; X K.
b 0,
n ¼ 1; 2; . . . ; N,
(14) (15) (16)
The marginal utility of a transmission equals its marginal congestion externality cost plus the resource shadow price. Even if the network is operated by a single provider, the solution of (14)–(16) cannot be imposed in centralized fashion since usage decisions and information about customer preferences are typically decentralized. The basic objective of congestion pricing is to achieve the socially optimal usage rates in such a decentralized environment in which self-interested customers make their own usage decisions. The following discussion focuses on the structural properties of the
90
Philipp Afe`che
socially optimal prices. Section 4.3 considers mechanisms for computing these prices in the presence of informational constraints on utility functions and network load. Suppose a linear pricing scheme that charges each user a unit price p per transmission. Given the price p and taking other user demands and the capacity as fixed, user n solves max un ðxn ; X ; KÞ pxn ; xn0
n ¼ 1; 2; . . . ; N.
(17)
She increases xn up to the point where her marginal utility equals the price plus the incremental congestion cost she imposes on her own traffic @un ðxn ; X ; KÞ @un ðxn ; X ; KÞ ¼p ; @xn @X
n ¼ 1; 2; . . . ; N.
(18)
If the social planner and customers know all the utility functions, the social planner can choose prices that induce users to choose the socially optimal usage rates x : If there are many users (N is large) whose individual demands are small relative to the total load X, as in the Internet, the socially optimal unit price is (approximately) the same for all users and satisfies p ¼
N X @ui ðx ; X ; KÞ i
i¼1
@X
þ b
(19)
where the first term on the right-hand side is the marginal congestion externality and the second is the resource shadow price at the socially optimal usage vector. Combining (18) and (19) yields @un ðxn ; X ; KÞ @un ðxn ; X ; KÞ ¼ p p @xn @X N X @ui ðxi ; X ; KÞ þ b . ¼ @X i¼1
ð20Þ
If users are relatively small, then (14) and (20), the optimality conditions of the system and those of individual users, agree and p supports x as a Nash equilibrium: no user n has an incentive to unilaterally choose a usage rate other than xn given the price p and the anticipated expected delay cost resulting from all other usage rates xn ; and the ex-ante anticipated expected delay costs are correct ex-post given the equilibrium usage rates. As discussed below the Nash equilibrium concept makes strong assumptions on information availability. It is insightful to compare (19) for the cases with and without congestion externalities. For the case with congestion externalities ð@un =@X o0Þ; assume that un ðxn ; X ; KÞ 0 as X ! K for all n. In this case, the capacity constraint is not binding at the optimal allocation, b ¼ 0; and the optimal
Ch. 2. Economics of Data Communications
91
price (19) equals the marginal congestion externality. For example, in the case of linear additive delay cost as in (6) the socially optimal price satisfies p ¼
N X @ui ðx ; X ; KÞ i
i¼1
@X
¼
N X i¼1
xi d i
@E½TðX ; KÞ . @X
(21)
Edelson and Hildebrand (1975), Mendelson (1985), Dewan and Mendelson (1990), Ha (1998), Afe`che and Mendelson (2004) are among studies that derive this relationship for FIFO scheduling, Ha (2001) and Maglaras and Zeevi (2003) consider (egalitarian) processor-sharing systems, and MacKieMason and Varian (1995b) consider a generic nondiscriminatory scheduling discipline. Papers that characterize the socially optimal static price for systems where customers observe the queue length include Naor (1969), Yechiali (1971, 1972), Knudsen (1972), Edelson and Hildebrand (1975) and Lippman and Stidham (1977). In the absence of congestion externalities ð@un =@X 0Þ the optimal price equals the resource shadow price, i.e., p ¼ b ; which is positive only if it is socially optimal to fully utilize the network. This corresponds to the case of guaranteed services; compare with (5) in Section 3 and set the effective band-widths ak ðs; tÞ ¼ 1: Thus, the socially optimal congestion price is similar to a resource shadow price in the absence of congestion cost, except that it is positive at lower than full capacity utilization. Variations. The above analysis finds that a single linear usage price supports the socially optimal allocation, assuming that users (i) decide how much to consume, (ii) are so small that the marginal congestion cost they cause for theirPown traffic is negligible relative to that imposed on all users ð@un =@X N i¼1 @ui =@X Þ and (iii) have transmissions with identical resource requirements. This result still holds in the limiting case of a continuum of users that have unit demand. Simply interpret un ðxn ; X ; KÞ as the aggregate utility of user segment n where @un ðxn ; X ; KÞ=@xn is the utility of that group’s marginal user at given load and capacity, and the congestion cost term @un =@X vanishes from the individual user’s optimality condition (18). If individual users account for a significant fraction of network load and are large enough to affect congestion levels, then a single price will not align individual and system incentives. Based on (14) and (18) users should only be charged the congestion cost they impose on all other users pn ¼
X ian
@ui ðxi ; X ; KÞ ; @X
n ¼ 1; 2; . . . ; N,
(22)
which differs across users except if they are identical. If users are small but consume services with different resource requirements, e.g., different file sizes, then congestion externalities, and hence the
92
Philipp Afe`che
socially optimal prices also vary across customers. For example, if customer n requires on average s¯n units of resources per transmission andPthe utility depends on the usage vector only through the total load X ¼ N ¯n n¼1 xn s then the socially optimal prices based on (19) are ! N X @ui ðxi ; X ; KÞ ; n ¼ 1; 2; . . . ; N. (23) pn ¼ s¯n @X i¼1 If the provider cannot a priori tell apart different users, then prices should be aPfunction of actual data volumes. For example, given the tariff p ðsÞ ¼ sð N i¼1 @ui ðxi ; X ; KÞ=@X Þ where s is the actual data volume transmitted, the expected price of user n is pn as in (23). Time scales for congestion price fluctuations. The above analysis computes the optimal static price based on the average congestion impact of transmissions. This raises several issues pertaining to the time scale for price changes. For one, it may be desirable for prices to more accurately reflect the differential congestion impact of different transmissions, even while keeping the average price constant. Gibbens and Kelly (1999a) propose and show how to charge each packet individually its sample path shadow price, which is the exact amount of congestion it inflicts on the system at each resource. They show conditions for when the rate of sample path shadow prices so derived equals the average congestion impact. Another issue is that the computation of optimal congestion prices also requires information on users’ utility functions, specifically on their sensitivity to delay or other QoS metrics. In the setting of Gibbens and Kelly (1999a) the cost of congestion is incurred by and known to the system, unlike here where it is borne by users and likely unknown to the system. Thus, even if sample path shadow prices correctly reflect the congestion impact of individual packets, the problem of correctly evaluating the resulting cost impact on users still remains. The adaptive price mechanisms discussed in Section 4.3 may help the provider learn the utility functions and find the right prices iteratively. Congestion prices may also fluctuate under optimal dynamic price policies, discussed in Section 4.5. Network. Mendelson and Whang (1994) and Masuda and Whang (2002) study the natural network extension of this problem, finding that for fixed capacity the socially optimal congestion prices are additively separable in the node prices and that they induce the desired usage rates and routes if customers can choose their routes. Socially optimal congestion prices and capacity investment Now consider the joint problem of choosing the efficient use and capacity level. Let x ðKÞ; X ðKÞ; p ðKÞ and b ðKÞ be the socially optimal usage vector, network load, unit price and resource shadow price, respectively, as a function of capacity, and let LðKÞ :¼ W ðx ðKÞ; KÞ þ b ðKÞ ðK X ðKÞÞ
Ch. 2. Economics of Data Communications
93
be the maximum value of the Lagrangian as a function of capacity. The marginal change in social welfare as a function of capacity satisfies @W ðx ðKÞ; KÞ þ b ðKÞ @K N X @ui ðxi ; X ðKÞ; KÞ ¼ C 0 ðKÞ þ b ðKÞ. @K i¼1
L0 ðKÞ ¼
ð24Þ
In the absence of congestion cost utility is independent of capacity ð@ui =@K 0Þ and p ðKÞ ¼ b ðKÞ; so it is optimal to expand capacity if and only if the price exceeds the marginal capacity cost L0 ðKÞ ¼ p ðKÞ C 0 ðKÞ.
(25)
If there is a congestion cost, then b ðKÞ 0 and the marginal value of capacity depends on the impact of network load X and capacity K on customer utility. If the utility depends on X and K only through the average excess capacity K–X, as in (8) for an M/M/1 system, then @ui =@K ¼ @ui =@X and (25) holds, i.e., the optimal price equals the marginal capacity cost (Mendelson, 1985; Dewan and Mendelson, 1990). If the utility depends on X and K only through the average utilization X/K, then @ui =@K ¼ @ui =@X X =K; so (19) and (24) yield X ðKÞ (26) C 0 ðKÞ, K which implies that the optimal price at the socially optimal capacity exceeds the marginal cost: expanding capacity increases welfare if and only if the price, multiplied by the utilization, exceeds the marginal capacity cost (MacKie-Mason and Varian, 1995b). Network. Masuda and Whang (2002) investigate the capacity expansion problem for a decentralized system with general network topology. They show that the optimal capacity investment principles carry over to a decentralized network with fixed demand and routing whereas this need no longer hold under routing choice; they propose a modified capacity analysis for such systems. Korillis et al. (1995) consider pricing and capacity decisions for decentralized parallel link networks where customers have routing choices; they show that the optimal decentralized solution coincides with that under centralized control. In summary, the socially optimal congestion price plays a dual role: it measures the social cost of increased usage at any given capacity level and serves as a signal to determine whether to increase capacity. L0 ðKÞ ¼ p ðKÞ
Perfect competition MacKie-Mason and Varian (1995b) find that in a competitive market in which many small providers charge connection fees and usage prices the equilibrium yields the social welfare maximizing usage and capacity levels.
94
Philipp Afe`che
Free usage It is well known that the absence of usage prices leads to resource overuse because users do not internalize the congestion externality. Studies that derive this classic result under FIFO scheduling for the case of unobservable system state include Mendelson (1985) and for the case of observable system queue include Naor (1969), Knudsen (1972), Yechiali (1971, 1972), Lippman and Stidham (1977). Two exceptions to this result are noteworthy. First, Hassin (1985) shows that social optimality can be achieved without usage fees if the service discipline is last come first served preemptive resume (users can observe the queue length). The key customer decision under this mechanism is when to renege (leave the system without obtaining service) rather than whether to request service, a decision made by the customer who is last in queue and who therefore imposes no externality. As a result, the last customer’s incentive is aligned with that of the system: the queue only grows as long as the last customer’s service value exceeds her expected delay cost. While this mechanism is probably not of practical importance for data communication services, it nicely highlights the link between the service discipline and the distortion under individual optimization. Second, Giridharan and Mendelson (1994) show that in the presence of both negative (congestion) and positive (network) externalities usage levels may be close to socially optimal even if usage is free, since the positive at least partially offset the negative externalities. See Westland (1992) for a further study of socially optimal pricing under congestion and network externalities. Monopoly price and capacity The pricing and capacity decisions of a profit-maximizing monopoly provider compare as follows to the socially optimal ones. The profit-maximizing prices in the case of users with identical resource requirements—the analog of (19), typically do depend on users through their demand elasticity. If the provider cannot tell apart individual users, she can only charge a single price that may result in some users transmitting more and others less traffic than what is optimal if the provider can price discriminate. The respective analysis follows from the classic multi-product monopoly analysis with the direct cost replaced by the delay cost functions. For a fixed capacity level and linear pricing a monopoly serving a single user segment will typically charge a higher price and reduce usage (quantity) compared to socially optimal levels, consistent with the standard result in the absence of congestion externalities. Naor (1969) and Mendelson (1985) establish this result for the case of customers with identical delay cost under the additive delay cost structure given by (6). However, Edelson and Hildebrand (1975) give counterexamples that show this property need not hold if customers differ in their cost of delay. Afe`che and Mendelson (2004) provide analytical conditions on the delay cost structure and the price elasticity for when the monopoly price exceeds, equals or is lower than the socially optimal price.
Ch. 2. Economics of Data Communications
95
Under linear pricing a monopoly typically restricts capacity below the socially optimal level. Moreover, since the optimal capacity utilization is below 100% in the presence of congestion, (25) and (26) imply that the provider incurs a loss at the socially optimal price and capacity levels if the capacity cost function is linear (or more generally, not ‘‘too convex’’). Mendelson (1985), Dewan and Mendelson (1990) and Maglaras and Zeevi (2003) observe this point. Dewan and Mendelson (1990) study how the budget deficit at the socially optimal capacity depends on the delay cost function. This implies that internal data communication services with the goal of maximizing the overall net value to the organization should not be evaluated as profit centers or not use linear pricing. There are various approaches to mitigate the problem of inefficient use and capacity investment under a monopoly provider. Under Ramsey pricing linear prices are chosen to maximize social welfare subject to a constraint on the provider’s profit, usually taken to be a balanced-budget constraint. Ramsey pricing is mostly relevant for regulated firms. See Mitchell and Vogelsang (1991) for a survey of Ramsey pricing in traditional telecommunication markets. Another approach is to use a two-part tariff of the form Pðxn Þ ¼ a þ p xn where the usage-insensitive access or subscription fee a serves to recover the cost and the linear usage charge p regulates congestion. If users are homogenous then the monopoly can extract all consumer surplus and maximizes total welfare by choosing the socially optimal p equal to the marginal congestion cost, and by setting a to extract the remaining surplus. This is the classic result of Oi (1971). In the more realistic case of heterogenous customers, the monopoly can typically not extract all surplus. In this case the optimal usage price is below the marginal congestion cost if the marginal customer’s bandwidth demand exceeds that of the average customer, e.g., demand for low-value video vs. high-value email, and vice versa (see MacKie-Mason and Varian, 1995b). The provider can typically improve upon a single two-part tariff by offering a more general non-linear tariff P(x) or a menu of multiple ðM NÞ two-part tariffs Pm ðxn Þ ¼ am þ pm xn from which customers select the option that suits them best. Masuda and Whang (2006) identify conditions where the optimal menu of tariffs with a simple ‘‘fixed-up-to’’ structure perform as well for a monopoly as any nonlinear pricing schedule. See Mitchell and Vogelsang (1991) and Wilson (1993) for surveys of nonlinear pricing. 4.3 Incomplete information and adaptive pricing The static posted price mechanisms of the preceding section involve two steps. The provider first determines the optimal capacity and usage rates and the prices that induce customers to choose these rates. Each customer then determines her utility-maximizing usage given the posted prices and her expected delay cost, which depends on system properties and other users’ anticipated usage rates. The notion that an optimal allocation can be
96
Philipp Afe`che
supported as a Nash equilibrium in user decisions assumes that the provider and users are omniscient and have unbounded rationality. To compute the optimal price and capacity levels the provider must have complete information on demand (the number, preferences and traffic characteristics of customers) and system performance characteristics (the relationship between usage levels and QoS metrics such as delay). To determine her utilitymaximizing transmission rates each user must have complete information on system parameters and other users’ preferences; they need this information to form rational expectations about equilibrium usage rates and delay cost since the network does not guarantee the QoS. By contrast, customers do not require this information if the provider delivers QoS guarantees to each user regardless of overall network conditions. These information requirements pose significant implementation challenges. The rapidly evolving data communications market makes it difficult for providers to accurately estimate utility functions, particularly delay-sensitivity and other quality preferences, except perhaps in internal networks with known users. It is also unrealistic to require from customers the ability to accurately forecast their congestion cost over their usage horizon. While the optimal congestion prices can be formulated in theory, it is not clear how or even whether they can be computed and implemented in practice. A couple of approaches that may help overcome these informational constraints have been considered. One is to use auction mechanisms, where users submit price bids before the provider determines payments and allocations. Section 6.1 discusses auctions and their potential for revealing demand information. Two approaches that have mainly been considered for posted price mechanisms include providing congestion information for users and adaptive pricing and delay forecasting. The former focuses on improving the information available for any single user-network interaction, the latter focuses on adaptation based on information gained through repeated interactions. Congestion information for users This approach focuses on giving users the required forecast information on system state and/or performance statistics. In models where users do not have dynamic queue length information the assumption that users can perfectly forecast steady-state delay statistics on their own can alternatively be interpreted as the provider posting information on the system’s (unconditional) steady-state delay distribution. The only requirement for this alternative interpretation is that the ex ante information be consistent with the ex post realized performance. If users do receive queue-length information before submitting a transmission, they can improve their delay forecast accuracy for any given transmission by basing their QoS estimate on the conditional delay distribution. Studies that consider various questions for this observable queue case include Naor (1969), Yechiali (1971,1972), Knudsen (1972), Edelson and Hildebrand (1975), Lippman and Stidham (1977), and Afe`che and Ata (2005) for uniform FIFO service; Balachandran (1972), Adiri
Ch. 2. Economics of Data Communications
97
and Yechiali (1974), Dolan (1978) and Alperstein (1988) for strict priority scheduling; Hassin (1985,1986) for last-come-first-served preemptive repeat; Altman and Shimkin (1998) for processor sharing; and Maglaras and Zeevi (2005) for (egalitarian) processor sharing of a randomly fluctuating amount of bandwidth. Most of these papers study static price policies, just like the one considered above, but they allow customers to react to a given set of prices based on dynamic congestion information. Section 4.5 considers studies of optimal dynamic price policies. Hassin (1986) studies whether making dynamic queue-length information available to customers increases or reduces the total surplus and the provider revenue, compared to the case where customers do not have this information. The answer is ambiguous and depends on the system utilization. Maglaras and Zeevi (2005) show that in large scale systems, i.e., that process large volumes of data and have proportionally large processing capacity, real-time delay information increases system revenues and the resource utilization rate while simultaneously decreasing the average delay experienced by the users. Adaptive pricing and delay forecasting This approach focuses on users and the provider adapting their decisions based on information they gain over repeated interactions, assuming users cannot observe the current queue length before deciding whether or not to transmit. Users adjust their delay forecast and demand based on their past delay experience and past prices, and the provider learns about demand and adjusts prices based on the observed user response. Analyses of this approach examine the stability of dynamic adaptation algorithms under alternative update rules for prices and/or delay forecasts. In Stidham (1992) and Rump and Stidham (1998) the provider does know user demand attributes and fixes a price. Both studies focus on users updating their delay cost expectation and arrival rate based on past experience, in Stidham (1992) the provider also updates the capacity. Their basic approach is similar to the classic tatonnement process. Time is partitioned into successive periods; in each period users update their delay forecast and base their usage decisions on the updated forecast. Demand is assumed to be stationary for a sufficient number of time periods so that the system converges to steady-state before conditions change. Gupta et al. (1997) and Masuda and Whang (1999) analyze how the socially optimal prices and usage rates can be attained in a resource network through an adaptive algorithm when both the provider and customers have imperfect information. The provider does not know the demand functions and customers do not know the system delay. Their basic approach is the same as in Stidham (1992) and Rump and Stidham (1998), but the provider also updates prices, in addition to customers adjusting their usage rates. The pricing scheme is linear with node-specific unit prices for each transmission. Low and Varaiya (1993) and Thomas et al. (2002) also study such adaptive mechanisms, but they focus on guaranteed services where each
98
Philipp Afe`che
user’s utility is independent of others’ consumption and user decisions only depend on prices. Gupta et al. (1997) and Masuda and Whang (1999) consider a capacityconstrained queueing network that serves multiple delay-sensitive user classes, each characterized by a network route and an aggregate utility function with a linear delay cost parameter. The provider does not know the utility functions but knows delay cost parameters. The studies differ in several assumptions. Gupta et al. (1997) consider delay cost parameters drawn from a distribution, generic transmission time distributions and multiple predefined priority classes at each node with priority class-dependent prices, which requires simulation and approximations to evaluate delays. Masuda and Whang (1999) consider an identical delay cost parameter for all users, FIFO service at each node and assume Poisson arrivals and exponential transmission times which yields analytical expected steady-state delay formulae. As a benchmark the studies characterize the socially optimal arrival rates and the prices and expected delays that support these arrival rates as a stochastic equilibrium (cf. Stahl and Whinston, 1994), which is defined by two properties. Best-reply, i.e., the arrival rates maximize users’ expected utility given the posted prices and anticipated delays, and accurate expectation, i.e., the anticipated delays are the correct ex-ante expected delays given the resulting arrival rates. (Every stochastic equilibrium is a Nash equilibrium and vice versa. The concepts differ in their informational assumptions.) The following measurement-based rules govern price and delay forecast updates. The provider updates prices using exponential smoothing. For example, for a single node with FIFO service ^ pðt þ 1Þ ¼ a pðtÞ þ ð1 aÞ pðxðtÞÞ,
(27)
where a 2 ½0; 1 is the exponential smoothing parameter, p (t) and x (t) are the ^ period-t posted price and usage rates, and the period-t price estimate pðxðtÞÞ evaluates the marginal delay cost, the right-hand side of (21), at the actual time-average usage rates. The price adjusts upwards (downwards) if the prior period price is lower than (exceeds) the marginal delay cost estimate. In each period users choose their transmission rates based on the posted prices and their delay forecasts, and the resulting usage rates and delays are used for the next round of updates. The studies differ somewhat in how users forecast delays. In Gupta et al. (1997) the network estimates and announces the next period expected delays based on current time-average estimates. In Masuda and Whang (1999) users estimate the next period expected delays based on their estimate of next period arrival rates and the analytical steady-state delay formulae. For example, for a single M/M/1 node with FIFO service ^ þ 1Þ ¼ X^ ðt þ 1Þ ¼ b X^ ðtÞ þ ð1 bÞ X ðtÞ and Tðt
1 , K X^ ðt þ 1Þ (28)
Ch. 2. Economics of Data Communications
99
^ þ 1Þ are the arrival rate and expected delay estimates where X^ ðt þ 1Þ and Tðt for period t+1, X (t) is the actual current period average arrival rate and b 2 ½0; 1 is an exponential smoothing parameter. The following insights on the dynamic network behavior and its economic performance emerge under these update rules. Gupta et al. (1997) develop and run several simulation models that show prices and arrival rates generally converging quickly, also in scenarios where demand alternates between low and high states (Gupta et al., 1999), to what they suggest are approximately optimal levels. Their simulations also show that customer and total benefits of priority pricing over uniform pricing and free access may be significant. Masuda and Whang (1999) focus on the issue of network stability. They show that in the absence of exponential smoothing (a ¼ b ¼ 0) the network may be unstable and continuously oscillate between low- and high-utilization periods, with the low-utilization periods reminiscent of Yogi Berra’s famous quote, ‘‘Nobody goes there any more since it is so crowded.’’ They establish analytical conditions on the demand function, delay cost and smoothing parameters for the network to be locally stable at the socially optimal equilibrium, and they prove that a single node M/M/1 ‘‘network’’ is stable for a sufficiently large smoothing parameter. These mechanisms resemble the adaptive flow control and pricing mechanisms considered for flexible bandwidth-sharing services in Section 5, e.g., Kelly et al. (1998). A main difference is that there flow rates and prices are updated perhaps every tens of milliseconds whereas here these updates occur on a longer time scale, perhaps on the order of hours or days. The setting here implicitly assumes that users react slowly to perceived congestion, inasmuch as the system operates long enough at a particular set of load values for steady-state to be (approximately) achieved in each time period. These adaptive pricing mechanisms have the potential to dramatically reduce the information requirements: the provider need not know the demand functions and users need not know each others’ preferences. In addition, prices can be calculated in a decentralized manner at each network node based on local flow rate and delay measurements. However, several theoretical and practical issues require further study. For one, the convergence behavior and its sensitivity to the network characteristics are not fully understood. Under what conditions is the system globally stable, and if there are multiple equilibria which one will be reached? Kelly et al. (1998) and Thomas et al. (2002) prove that their adaptive algorithms converge. However, they consider systems without congestion externalities that have unique socially optimal allocations; systems without these features need not have the same stability properties. The system considered by Stidham (2003) considers a generic delay cost model for a network with undifferentiated best effort service that serves users who differ in their delay sensitivity. His analysis suggests that the total system benefit function may have multiple stationary points and that adaptive algorithms may converge
100
Philipp Afe`che
to a local rather than global maximum, depending on the starting point. Stidham (2003) develops this result for a continuous-time adaptive flow control algorithm very similar to Kelly et al. (1998), but his model framework is general enough to suggest similar stability issues are likely to be prevalent in the setting considered in this section. Another set of issues concern the convergence speed of such adaptive mechanisms. How long does it take for prices to converge in stationary and nonstationary environments? How do the answers depend on the update period length and update rules? How should these mechanisms be modified and how would they perform if the provider lacks information not only on the value functions, as in Gupta et al. (1997) and Masuda and Whang (1999), but also on delay cost parameters and other quality preferences? Gupta et al. (2000) contribute in this direction: they propose a nonparametric statistical technique for estimating users’ delay cost parameters based on their observed choices and show via simulation that substituting these estimates for the ‘true’ parameters in the price updates only results in a minimal efficiency loss. Afe`che and Ata (2005) analytically characterize the optimal dynamic pricing policy under Bayesian updating for a stylized single-node system that serves customers with unknown delay cost parameter distribution in a stationary environment. They characterize the time to learn and the probability that the provider eventually learns the correct distribution. From a practical perspective, customers may find it difficult to plan communication service purchases given the price fluctuations inherent in these adaptive approaches. It may be useful to run such mechanisms in representative test markets to determine ‘‘appropriate’’ prices for low- and high-demand conditions which are then to be used in actual markets. 4.4 More differentiation The discussion in this section has so far focused on uniform or undifferentiated service—the network treats all transmissions the same. For uniform service differentiated pricing is required for social optimality only if transmissions have different resource requirements, e.g., as in (23), whereas the profit-maximizing pricing scheme may involve price discrimination even for homogenous transmissions. If users are heterogenous in terms of their traffic characteristics and/or QoS preferences it may be beneficial to offer them a menu of delay or loss differentiated service classes and to differentiate prices not only based on resource requirements but also in function of the QoS. Such differentiation typically attains a more efficient resource allocation; it also allows the provider to increase her revenue by better segmenting the market and giving users the option to pay more for higher QoS. Chao and Wilson (1987) and Wilson (1989) are classic references on priority service in the economics literature.
Ch. 2. Economics of Data Communications
101
Service differentiation and best effort service The Internet presently offers only a single class of best effort service. While best effort was conceived with nondiscrimination in mind ‘‘service differentiation and best effort service are conceptually orthogonal’’ (Gevros et al., 2001). In the context of the best effort service model it is possible to differentiate services by defining two or more traffic classes that are treated differently within the network, have qualitatively different QoS characteristics (e.g., average delay or loss ratios) tailored to elastic applications with different QoS preferences and are priced accordingly. Analyses of differentiated best effort services consider a range of control mechanisms for achieving differential QoS metrics, including link partitioning, routing and packet scheduling mechanisms. The notion is to have only a few classes, each chosen by multiple individual flows, in contrast to guaranteed services where QoS is managed for each individual flow via admission control, resource reservation and traffic enforcement mechanisms. In differentiated best effort services the QoS of each service class is not expressed in terms of strict performance bounds that are pre-defined for each individual flow, as in guaranteed services, but rather in terms of expected performance metrics averaged over all flows in a service class. Depending on how bandwidth is assigned to a service class its QoS metrics may depend on all flows within the network or only on its own flows. Proposals A couple of QoS and pricing proposals build on the notion of a multiclass best-effort network. The DiffServ architecture proposal of the IETF aims to provide statistical QoS guarantees for traffic aggregates, as opposed to strict guarantees for individual data flows. As such it can be viewed as augmenting the basic elements of a multi-class best effort network—traffic is classified into a few categories and all packets within a class are handled the same—with demand regulation mechanisms that enforce certain traffic inflow constraints and help achieve statistical class-level QoS targets. DiffServ does not by itself say anything about the assignment of priorities to different traffic classes and pricing. Odlyzko (1999b) proposes an approach called Paris Metro Pricing (PMP) for Internet pricing, with the name due to its resemblance to the pricing structure of the Paris Metro. The concept is to partition the Internet into several logically separate ‘‘channels’’ for different service classes with higher prices for using the better provisioned and therefore (allegedly) less congested classes. PMP is similar to DiffServ in that it considers multiple best effort service classes, but unlike DiffServ, it does not consider the use of technological demand regulation mechanisms to attain some (class-level) QoS targets. In PMP pricing is the primary demand regulation mechanism and it focuses on how to price and assign bandwidth to the different classes whereas DiffServ does not.
102
Philipp Afe`che
Issues The economic analysis of multi-class best effort services should address several interrelated provider decisions: how many service classes to create (service design), how to allocate bandwidth to these classes (service production), and how to price and target these classes to users (pricing). An important consideration for these provider decisions is how much information she has on user preferences. In particular, if the provider only has aggregate information about user preferences, but no information on QoS preferences of individual users, then she cannot distinguish among them ex ante; all users can and will choose among all options on the provider’s service class menu in line with their own self-interest. This behavior gives rise to incentive-compatibility constraints, which the provider must consider in designing her price-QoS menu. The vast majority of papers consider pricing decisions for a given service design, defined by a fixed number of service classes, and for given supply allocation mechanisms at a single network link, for example: how to price two service classes delivered over a single server with a strict nonpreemptive priority discipline? A few of these papers investigate the economic value of differentiation for given supply allocation mechanisms by comparing optimally priced systems with different numbers of service classes, or jointly optimize tariff design and the number of service classes. Hardly any papers jointly optimize or compare alternative combinations of service design, service production and pricing. Number of service classes Adiri and Yechiali (1974), Alperstein (1988), Cocchi et al. (1993), Gupta et al. (1997), Bajaj et al. (1998), Odlyzko and Fishburn (1998), Van Mieghem (2000), Afe`che and Mendelson (2004), Afe`che (2004) and Zhang et al. (2006) among others study the economic benefit of two or more service classes compared to uniform service. All demonstrate revenue and/or total surplus gains as a result of adding a class, in various settings. The marginal benefit from an additional class typically drops (Afe`che and Mendelson, 2004), suggesting that a few classes may generate most of the benefits. However, ‘‘more differentiation’’ need not always be beneficial: Afe`che (2004) shows that pooling different user segments in a single class may increase revenues if users have private information about their preferences. Bandwidth allocation to service classes In multi-class best effort services the specification of bandwidth allocation has a significant impact on the user-perceived quality and therefore plays an important role in pricing studies. Bandwidth allocation occurs at multiple links along a route. Reflecting the emphasis in the literature this survey focuses on a single link ‘‘network’’. Bandwidth can be allocated to service classes using a combination of two basic methods: reserving for each class exclusive access to a dedicated (set of) channel(s); or letting two or
Ch. 2. Economics of Data Communications
103
more classes share a common pool of communication channels according to certain packet scheduling and routing mechanisms. Thus, exclusive access results in a static and shared access yields a dynamic channel allocation. Exclusive access is typically simpler to manage but offers less flexibility and has inferior delay performance compared to shared access with appropriate scheduling and routing control. Studies of price and service differentiation have so far focused either on link partitioning and routing issues for multi-channel systems, or on priority scheduling policies for single-channel systems; the joint problem of pricing, routing, and priority scheduling for multi-channel systems has hardly received any attention. Priority scheduling mechanisms play a major role in delivering differentiated services using pooled capacity. The most common are strict or absolute priority disciplines, where higher priority traffic waiting for transmission always gets exclusive bandwidth access, and processor sharing disciplines where all priority levels get some bandwidth access based on their relative priority index. The majority of pricing studies focus on strict priority disciplines, which minimize the system’s average delay cost in single-node systems and are therefore more efficient than processor-sharing disciplines when users are delay-sensitive. However, processor-sharing systems typically exhibit a lower delay variance across service classes, which makes them attractive for fairness objectives. As noted above most papers consider pricing for a given bandwidth allocation scheme. Exceptions include Odlyzko and Fishburn (1998) who compare the revenues for dedicated and pooled allocations of two service classes; Van Mieghem and Van Mieghem (2002) who consider generalized strict priority and processor-sharing disciplines for a single channel; and Afe`che (2004) who jointly optimizes over prices and scheduling policies for a provider who serves two customer segments over a single channel. Multiple channels: link partitioning and routing. An appealingly simple approach is to partition the capacity of a link into multiple channels and statically allocate each service class to a dedicated channel pool with FIFO queueing at each channel pool. This corresponds to the basic design for Odlyzko’s (1999b) PMP proposal. A major question is how much capacity to allocate for each class. Odlyzko assumes a monopolistic setting. Gibbens et al. (2000) address the issue of pricing and capacity allocation also for a duopoly market. Other studies consider how to route multiple service classes that share access to multiple channels with given capacities. Bell and Stidham (1983) and Bradford (1996) characterize the equilibrium behavior of customers in such multi-server queueing systems. One channel: processor sharing. As noted above processor sharing systems with differentiated services have received relatively little attention in pricing studies. Haviv and van der Wal (1997) consider equilibria for users’ priority choice problem in a discriminatory processor sharing system. Van
104
Philipp Afe`che
Mieghem and Van Mieghem (2002) study and compare generalized strict priority and processor-sharing disciplines. Maglaras and Zeevi (2005) study profit-maximization for a system with guaranteed and best effort service where best-effort traffic shares the dynamically fluctuating leftover capacity not tied up by guaranteed service. One channel: strict priorities. The majority of papers consider systems with strict or absolute priorities. Bohn et al. (1994) propose how to implement priority levels based on existing Internet protocols without considering pricing and incentive issues. Cocchi et al. (1993) simulate example networks with strict priority service and different traffic types, and they determine prices that make all users better off than under uniform service. The following studies focus on social optimization for the linear delay cost case. Kleinrock (1967) first studied priority pricing by ignoring customer incentives. Marchand (1974), Ghanem (1975), Dolan (1978), and Mendelson and Whang (1990) focus on and provide a thorough understanding of the incentive-compatible and socially optimal price-scheduling mechanism: the socially optimal and incentive-compatible scheduling policy is the traditional cm rule—known to minimize the system’s average delay cost rate— and each customer class is charged her externality. If customer types have private information on their service requirements, e.g. file size, the socially optimal charging scheme consists of a static a priori payment and an ex post payment based on the actual file size. Gupta et al. (1997) consider a network with priorities (see Section 4.3). Van Mieghem (2000) generalizes the analysis of Mendelson and Whang (1990) to the case of convex delay cost, and for the same cost structure Van Mieghem and Van Mieghem (2002) study and compare pricing for generalized strict priority and processor-sharing disciplines. Interestingly, the design of revenue-maximizing and incentive-compatible price-scheduling mechanisms has only received limited attention so far. Rao and Petersen (1998) consider incentive-compatible pricing for welfare and revenue-maximization but implicitly assume, as opposed to optimize over, the scheduling policies. Afe`che (2004) jointly optimizes over prices and scheduling policies for a revenue-maximizing provider who serves two customer segments. He shows that the revenue-maximizing and incentivecompatible scheduling policies have novel features and may significantly differ from those under social optimization. In particular, one such policy involves strict priorities, but with the insertion of optimal strategic delay and another reverses the priority ranking compared to the socially optimal ranking. Afe`che’s (2004) analysis assumes linear delay cost. Yahalom et al. (2006) extend his treatment by allowing for convex delay cost and show that the use of strategic delay may also be optimal in this case. Papers that study auctions for strict priority service include Balachandran (1972), Glazer and Hassin (1985), Lui (1985), Hassin (1995), Stahl (2002), Afe`che and Mendelson (2004), and Kittsteiner and Moldovanu (2005); they are discussed in Section 6.1.
Ch. 2. Economics of Data Communications
105
The above studies all measure QoS by delay. Marbach (2004) considers static pricing and bandwidth allocation for static loss priorities for the case where QoS is measured by throughput and packet loss. 4.5 Optimal dynamic pricing Optimal dynamic pricing policies have received little attention, maybe in part due to the mathematical challenges inherent in their analysis. Load-sensitive dynamic pricing Low (1974) and Chen and Frank (2001) characterize optimal dynamic policies where prices depend on the queue length, which is visible to users. Demand-sensitive dynamic pricing Demand-sensitive dynamic pricing appears not to have received any explicit attention. Time-of-day pricing could be studied in a similar framework as discussed in Section 3.5 by considering an exogenous time dependent arrival process and ignoring intertemporal effects. However, this fails to capture users’ choices of when to transmit as they anticipate price and delay fluctuations over time. The analysis of this issue is likely quite complicated due to the presence of congestion externalities. 4.6 Conclusions and directions The basic principle for socially optimal pricing is to charge customers the marginal congestion cost they impose on the system. This price also serves as a signal for optimal capacity expansion. For uniform service differentiated pricing is socially optimal only if transmissions have different resource requirements whereas the profit-maximizing pricing scheme may involve price discrimination even for homogenous transmissions. If users are heterogenous in terms of traffic characteristics and/or QoS preferences it may be beneficial to offer delay or loss differentiated best effort service classes with appropriately differentiated prices. Doing so involves decisions on service design and bandwidth allocation to the service classes. If the provider has no information on QoS preferences of individual users, then she must consider incentive-compatibility constraints that reflect users’ choice behavior in designing her price-QoS menu. In this case, it may be optimal for a profit-maximizing provider to ‘‘pool’’ multiple segments in a single service class or, conversely, to increase the amount of differentiation between classes relative to the socially optimal levels. An important issue in the economics of best effort services is the information available to users and the provider. Users have to form QoS forecasts since the network inherently provides neither quality guarantees nor
106
Philipp Afe`che
congestion feedback signals, and the provider needs to understand users’ utility functions, and in particular their QoS sensitivity. Static equilibrium analyses assume that users have perfect steady-state delay forecasts and the provider knows demand. Two approaches that go beyond this static framework include providing congestion information for users and iteratively updating price and delay forecasts over successive demand periods. Several issues appear to have received little attention so far and to merit further research. Service design. Most best effort service pricing studies take as a starting point a given set of service classes and supply allocation mechanisms. Since users of best effort services are inherently flexible in terms of their QoS requirements, there is a research opportunity for jointly optimizing pricing, service design and supply allocation mechanisms. Afe`che (2004) illustrates this approach for a single server system with two customer segments and develops a stepwise solution approach that can be used to study systems and markets with more general characteristics. Loss as QoS. Virtually all studies consider delay as the only utility function attribute, whereas data loss has hardly received any attention as a measure of QoS. While some best effort applications, certainly text-based ones, may not be loss-tolerant, others may be, and it would be interesting to identify similarities and differences in the analysis and results for delay- and loss-tolerant applications. Network. The majority of studies consider a single-link ‘‘network.’’ While some results naturally extend to multi-link networks, e.g., the optimal prices for a given transmission may be additive in the prices at each link along its route, issues relating to service differentiation in a network and routing control call for further study. Value of load-sensitive and demand-sensitive dynamic pricing. The prevalent static equilibrium analysis framework generates a lot of insights into structural properties of prices and capacity levels, but it sacrifices the ability to study the dynamic relationship between user decisions, congestion and pricing. The analytical difficulty notwithstanding, it would be interesting to shed more light on the structural properties and the value of optimal dynamic policies. 5
Pricing flexible bandwidth-sharing services
5.1 Features, modeling and overview Features The guaranteed and best effort service contracts considered so far typically have only static parameters: guaranteed or expected QoS parameters, traffic parameter constraints and tariffs are fixed at the start of the contract. Users make all decisions concerning an individual flow or a collection of flows covered by their contract at the start of the respective transmission; once a
Ch. 2. Economics of Data Communications
107
user submits a flow she has no further control over its progress through the network. This section turns to services that offer dynamic bandwidth guarantees and give users some control over and the ability to adapt to their allocation while their transmissions are in progress. This flexibility is ideally suited for elastic applications such as email, Web browsing, or streaming video that can adapt their data rates to match the available bandwidth while achieving a graceful degradation in the perceived quality of service. As further discussed below, the survey uses the phrase flexible bandwidth-sharing services to emphasize its focus on a particular type of bandwidth allocations that determine the shares of a given set of concurrent flows based on user utility functions. While one can think of contracts with other dynamic guarantees, in practice the peak rate is the most common one. The Internet transport service provided by the TCP protocol is the most prominent example. It guarantees zero packet loss by retransmitting lost packets and dynamically controls the peak rates at which competing connections are allowed to transmit via adaptive flow control and feedback mechanisms. The network nodes transmit congestion signals to end user applications which adapt their sending rates accordingly. The distinctive features of this approach are closed-loop control, on fast time scales and at the level of individual flows: individual end user applications receive feedback signals every few tens of milliseconds and adapt their sending rates correspondingly quickly to track changes in their bandwidth allocations. In contrast to this approach the guaranteed services discussed above use open-loop control—admission decisions are based solely on a priori available traffic and network information—and the self-optimizing behavior of users of best effort services allows at best for much slower feedback between demand and congestion levels. Bandwidth sharing mechanisms generally pursue multiple allocation objectives: efficiency (using all available bandwidth to the fullest), feasibility and congestion avoidance (small packet delays and losses) while achieving and maintaining a certain fairness in the bandwidth shares attributed to different flows. There are several definitions of fairness (cf. Mo and Walrand, 2000, Massoulie´ and Roberts, 2002) and the relationship between fairness and network performance is a topic of intense research activity (cf. Roberts, 2004.) The body of work reviewed here is inspired by adaptive flow control procedures as implemented by TCP. To embed these procedures in an economic environment it associates congestion signals with per-packet prices, endows users with utility functions and lets them adapt their data rates to maximize their utility net of payments. The approach proposes and analyzes the potential of such distributed adaptive price and flow control algorithms for achieving proportionally fair bandwidth sharing among competing elastic flows. The proportional fairness criterion is of particular interest from an economics perspective since it is closely related to welfare maximization when utility functions depend on throughput only.
108
Philipp Afe`che
This proposal takes a novel approach to quality of service differentiation and provisioning. A key feature is that it delivers a set of arbitrarily differentiated and evolving services that are defined by users based on their personal needs—hence the term flexible bandwidth-sharing services used here. The approach assumes that capacity is adequate and queueing delays are small in comparison with propagation delays, and that a simple network with a single packet class provides differentiation through algorithms located at the end nodes, in line with the ‘‘end-to-end principle.’’ This approach stands in contrast with conventional service differentiation solutions for guaranteed and best effort services, which operate on the premise that congestion can be significant on the time scales where customers make their usage decisions. They offer a fixed set of service classes with pre-defined quality levels and prices, and they require intelligence embedded within the network and sophisticated control mechanisms such as admission control, resource reservation and discriminatory packet scheduling disciplines, to manage quality in the presence of potentially significant queueing delays. Modeling There is a discrete economy with a fixed set of users, each generating a single rate adaptive flow over a fixed network route. Like much of the work on fairness and flow control, bandwidth sharing is mostly considered in a static regime where users share network resources for the transfer of infinitesized documents. In other words, the number of users and their preferences are constant throughout the horizon, and it is long enough for the system to converge to the desired bandwidth allocation. This setting is useful when the network carries large file transfers and the number of flows changes infrequently. A user’s utility at a given time only depends on her data rate or bandwidth at that time, which is the only explicit QoS metric. In contrast to best effort services, here delays or losses typically are not explicit attributes of customer utility but merely act as feedback signals for flow control. Congestion externalities are thus not modeled directly, but congestion impacts a user’s utility indirectly by reducing her future bandwidth. The key network control mechanism is flow control, which describes the dynamic feedback process of price adaptation at routers and sending rate adaptation at end nodes. Once in the network all flows are treated the same; thus services are differentiated only based on bandwidth, and this differentiation is the result of different sending rates. The fundamental results model traffic sources as deterministic fluid flows and ignore details of specific flow control algorithms, packet scheduling and congestion signalling. Studies of implementation issues also consider stochastic fluctuations, packet level dynamics and operational details. In its original form this approach uses non-discriminatory FIFO packet scheduling.
Ch. 2. Economics of Data Communications
109
Overview Section 5.2 discusses static bandwidth sharing in a static regime with complete information. It introduces the weighted proportional fairness criterion, which roughly specifies that the network allocate flow rates in proportion to how much users choose to pay. Under appropriate weights (user payments) this sharing criterion yields the socially optimal bandwidth allocation. Section 5.3 discusses distributed adaptive flow control and price algorithms that are designed to attain the socially optimal proportionally fair flow allocation when the network has no or incomplete information on utility functions and users are unaware of capacity constraints and congestion. Section 5.4 outlines extensions of this approach to networks with more than a single packet class serving delay- or loss- sensitive flows, to bandwidth sharing among real-time applications that have flexible peak rates but are also subject to a strict QoS bound, and to distributed admission control via congestion signals. Section 5.5 observes that this work has so far focused on sharing under a static regime with a fixed number of flows and outlines basic questions that may arise if one wants to investigate optimal dynamic pricing policies in dynamic environments where the number of flows is in flux. Section 5.6 concludes with suggestions for future research. 5.2 Basic pricing and allocation principles Socially optimal allocation and prices Consider the social welfare maximization problem in the basic model where a fixed set of users each transmit a single long-lived data flow over a capacitated network. Let L be the set of network links, and R denote the set of routes, each comprising a collection of links. Users are associated with fixed routes. User r A R gets utility ur (xr) per unit time when sending at rate xr, and x is the vector of flow rates. Following Shenker’s (1995) notion of elastic applications, utility functions are increasing and concave in throughput. The system incurs a congestion cost, e.g., measured by the cumulative packet loss rate at all resources. Let Cl (yl) denote the expected packet loss rate at resource l when its total flow rate is yl ¼ Ss:lAs xs where Cl ( ) is convex, strictly increasing and differentiable. (The congestion cost functions could be chosen to represent hard capacity constraints.) This network model is relatively coarse in that it captures only fluid flow rates and abstracts from packet details and queue lengths. All packets are treated equally by the network. The system objective is to maximize total welfare W over flow rates x ! X X X SYSTEM : max W ðxÞ :¼ (29) ur ðxr Þ Cl xs : x0
r2R
l2L
s:l2s
110
Philipp Afe`che
At the socially optimal flow rates x ; the marginal utility of each user equals the marginal congestion cost ! X X u0r ðxr Þ ¼ (30) C 0l xs ; r 2 R. l2r
s:l2s
The main challenge in solving SYSTEM and finding x is that neither the network nor the users have the required information. The network does not know the utility functions ur and users know neither each other’s utilities nor the capacity constraints or congestion cost functions Cl. Proportionally fair bandwidth sharing However, it follows from convex optimization theory that SYSTEM may be decomposed into subsidiary optimization problems, one for each user and one for the network, that are linked by the appropriate resource shadow prices and information exchange between the optimizing agents. Consider the following decomposition. Given the network sets linear prices Pr per flow unit, user r determines the total payment br (Pr) per unit time that maximizes her net utility: USERr ðPr Þ : max ur ðbr =Pr Þ br .
(31)
br 0
Individual users are assumed too small to anticipate the effect of their actions on the packet loss rate and related prices, hence they simply maximize their utility minus payment rate. Clearly br (Pr) satisfies: u0r ðbr =Pr Þ ¼ Pr .
(32)
The network in turn takes the vector of user payments b per time unit as given and computes the weighted proportionally fair allocation of flow rates x (b): ! X X X NETWORKðbÞ : max (33) br log xr Cl xs , x0
r2R
l2L
s:l2s
obtaining the prices Pr per flow unit on route r as the sum of the respective resource shadow prices: ! br X 0 X ¼ Cl xs ¼ Pr . (34) xr l2r s:l2s This decomposition, due to Kelly (1997), only imposes local information requirements on the network and users. Kelly (1997) proves existence of a social welfare maximizing equilibrium, i.e., there exist vectors P ; b and x with xr ¼ br =Pr that simultaneously solve USERr (Pr) for all r, NETWORK(b) and SYSTEM. The proportional fairness criterion specifies, loosely speaking, that the network allocate flow rates in proportion to how
Ch. 2. Economics of Data Communications
111
much users choose to pay. It is appealing for a couple of reasons. First, it yields the socially optimal bandwidth allocation under appropriately chosen user payments b : As noted above this survey refers to this bandwidth allocation as flexible since it takes into account user utility functions. Specifically, unlike under max–min fairness, it does not just depend on the number of flows and their network routes. (Mo and Walrand, 2000, define the notion of a-fairness that encompasses all these fairness objectives.) It also differs from the predefined bandwidth sharing rules for single-node egalitarian processor sharing systems discussed in Section 4, cf. Altman and Shimkin (1998), Ha (2001), and Maglaras and Zeevi (2003). Second the weighted proportionally fair allocation arises naturally as a result of existing flow control algorithms embedded in the Internet, making it feasible to solve the network problem in a distributed fashion, as discussed next. 5.3 Incomplete information and adaptive pricing Price and flow adaptation While the decomposition of (31) and (33) only requires locally available information there is no guarantee for iterative solutions of the user and network problems to converge to the welfare maximizing equilibrium. The network problem is mathematically tractable but difficult to implement by a centralized processor, even one that could reliably handle the computational complexity, since the required information exchanges with users are asynchronous and vulnerable to random link delays and failures which differ from user to user. One should also keep in mind that the socially optimal bandwidth allocation changes as soon as the number of flows in the system or one of their utilities changes, requiring a new set of iterations. Kelly et al. (1998) propose to control the system with distributed rate and price control algorithms through a feedback loop between end users and resources (routers, switches) so that the overall network reacts intelligently to perturbations. Golestani and Bhattacharyya (1998) and Low and Lapsley (1999) study related approaches. Each end user r chooses a payment rate br(t) and a data rate xr(t) at time t. Resource l marks a proportion of packets pl ðyl ðtÞÞ ¼ C 0l ðyl ðtÞÞ with feedback signals, where pl (yl (t)) can be interpreted as the resource shadow price, the marginal increase in expected packet loss rate at link l in response to an infinitesimal increase in the total flow rate. The packet marks are sent back to users at the rate they produce them; they can be viewed P as prices and/or congestion signals. User r receives marks at a rate of xr ðtÞ l:l2r pl ðyl ðtÞÞ and adjusts her instantaneous flow rate according to ! X dxr ðtÞ pl ðyl ðtÞÞ ; r 2 R (35) ¼ kr br ðtÞ xr ðtÞ dt l2r
112
Philipp Afe`che
where kr>0 is a constant gain parameter. Thus a user increases or decreases her sending rate depending on whether the rate of packet marks falls short of or exceeds her willingness to pay. In addition to adjusting their flow rates xr (t), users also update their willingness to pay br (t) to maximize their net utilities at the implied prices, either continuously or on a slower time scale. Kelly et al. (1998) establish stability of this algorithm at a proportionally fair and socially optimal allocation. First, for given fixed payment vector b, the system (35) has an unique stable point of flow rates x ðbÞ which solves NETWORK(b) and to which all trajectories x (t) converge. Second, if users also adjust their payment rates br (t) in response to fluctuations in their data rate xr (t) to continuously track the optimum to their local net utility maximization problems, then all data rate trajectories x (t) converge to the socially optimal allocation x ; the solution of SYSTEM. They also extend these results to the case where users have routing choices, and for the dual of this algorithm where shadow prices vary gradually, and flow rates are given as functions of the shadow prices. The adaptive nature of this mechanism bears some resemblance to those for best effort services discussed in Section 4.3, cf. Gupta et al. (1997) and Masuda and Whang (1999). A main difference is that here flow and price adaptation decisions occur on a much shorter time scale, perhaps tens of milliseconds versus minutes, hours or days. The stylized model described so far only captures the macroscopic fluid flow characteristics and ignores microscopic details of packet level marking and flow control algorithms. Several issues need to be addressed for an implementation to achieve the desired macrobehavior. Delays and randomness in feedback signals The basic stability results assume that the negative feedback is instantaneous. A number of studies, e.g., Kelly et al. (1998), Johari and Tan (2001), Vinnicombe (2002), Massoulie´ (2002) and Kelly (2003) consider how queueing or propagation delays in the communication between end users and resources and stochastic fluctuations in packet loss or marking affect the stability and speed of convergence to the system equilibrium, depending on the sensitivity of end-user responses to congestion signals, the gain parameters kr, and of congestion marks to resource load, the derivatives of the marking functions pl ( ). Packet marking System stability requires that the packet marking rate at each resource equal its shadow price where both are functions of the total flow rate, i.e., pl ðyl ðtÞÞ ¼ C 0l ðyl ðtÞÞ for all l A L. Given the stochastic nature of traffic loads at resources, the congestion cost randomly fluctuates from packet to packet. The challenge is to design packet marking algorithms that result in a rate of packet marks that indeed equals the resource shadow price. Gibbens and Kelly (1999a) propose to charge each packet individually the exact amount
Ch. 2. Economics of Data Communications
113
of congestion cost it inflicts on the system at each resource, defined by its sample path shadow price: it equals one if and only if deleting the packet from the sample path of arrivals at that resource would result in one less packet loss. They show for Poisson traffic that the packet marking rate under this principle precisely equals the resource shadow price. The sample path shadow price is intuitively appealing but hard to measure in practice. By definition it can only be determined once packet loss has occurred, at which point some of the packets contributing to this loss have already left the queue and routers do no longer have information to track them. Furthermore, signalling congestion only after packets are dropped limits the ability to anticipate or prevent packet losses, especially in the presence of communication delays. These considerations have led to proposals for congestion marking algorithms that generate marks before packet losses occur based on locally available queue length information at routers. These algorithms use either actual statistics (e.g., Random Early Marking by Lapsley and Low, 1999, a variant of Random Early Detection by Floyd and Jacobson, 1993; see also Floyd, 1994, on Explicit Congestion Notification) or simulations (the virtual queue algorithm of Gibbens and Kelly, 1999a). Wischik (2001) analyzes under what conditions marking algorithms achieve proportional fairness taking into account the impact of traffic burstiness. As long as queueing delays are small relative to signal propagation delays—the premise of the approach described here—randomness associated with whether or not a particular packet is marked under a given algorithm becomes less relevant than the proportion of packet marking since user adaptation decisions occur on the time scale of propagation delays. Results suggest that simple algorithms yield marking rates that equal or closely approximate the resource shadow prices, inducing the desired system stability. Flow control The adaptation algorithm (35) specified by Kelly et al. (1998) raises a couple of issues, on its consistency with user behavior and on its potential for implementation in the current Internet. What if users do not voluntarily adapt their rates according to (35)? This could occur even if the algorithm were implemented at end nodes. Ganesh et al. (2006) and Wu and Marbach (2003) show that even if users behave selfishly, choosing their net benefit maximizing transmission rates at each time step based on adaptive price estimates and unconstrained by flow rate algorithms, then the system still converges to the equilibrium allocation of Kelly et al. (1998). Another point concerns the assumption that a single user does not anticipate the effect of its own actions on prices. This is appropriate if individual data rates are small relative to the total flow rate as in the case of a network with a large user population. Gibbens and Kelly (1999a) observe that a user who accounts for a significant fraction of the network load and who does anticipate her impact on prices will tend to
114
Philipp Afe`che
reduce its data rate, relative to the case where she does not, since she internalizes the congestion cost she imposes on her own flow. How can the adaptation algorithm specified by (35) be implemented based on Jacobson’s (1988) rate control algorithm TCP that operates in the current Internet? Crowcroft and Oechslin (1998) propose MulTCP, a TCP variation in which users set certain parameters so that it attains weighted proportional fairness. Gibbens and Kelly (1999a) explore how various TCP-like algorithms and packet marking mechanisms can support heterogenous applications by tailoring them to a mix of rate adaptive real-time traffic and to file transfers, and they compare their mechanisms to those of the Internet. Further comparisons of TCP-like flow control algorithms via distributed network simulations and games are developed by Key et al. (1999) and Key and McAuley (1999). Kelly (2001) proves that MulTCP maximizes system welfare for particular utility and cost functions. He also shows that if users have routing choices, then the equilibrium allocation under a TCP-like algorithm suffers from two problems that do not occur in the stylized system model described above: it need not be pareto-efficient and the addition of a network link may in fact decrease welfare, an instance of Braess’ paradox, see Braess (1968). Intelligent user agents In the context of this bandwidth sharing approach a user will typically find it optimal to frequently adapt her data rate, to respond to randomly changing demand and supply conditions and also perhaps as a means to learn her true utility function. The burden of these decisions may be prohibitive, prompting studies on how intelligent agents can perform these tasks on behalf of users, e.g., see Courcoubetis and Weber (2003, pp. 250–254). Price uncertainty and bandwidth contracts Flexible contracts and their inherent fluctuations in prices and bandwidth availability may not appeal to customers who cannot or do not wish to absorb this uncertainty, creating a market opportunity for bandwidth futures contracts that offer the use of a pre-specified amount of bandwidth over some future time interval at a given price. Since bandwidth cannot be stored—capacity available at a given time is either used or lost forever, bandwidth contracts cannot be hedged in the same way as storable commodities such as silver, i.e., by purchasing the underlying asset for sale at a later date. The pricing of bandwidth derivative contracts differs from the traditional analysis of financial derivatives. Kenyon and Cheliotis (2001) describe the special features of bandwidth in comparison with other commodities, primarily electricity, and propose a method for constructing spot prices. Upton (2002) prices forward bandwidth contracts in a market for short term single-link data transfers over a time scale of seconds to hours and Keppo (2005) analyzes the pricing of bandwidth derivatives under network arbitrage conditions. The main task in pricing such contracts is to
Ch. 2. Economics of Data Communications
115
determine the expected spot price over the relevant time horizon. Anderson et al. (2006) take a different perspective: they propose a contract and balancing mechanism for a set of users to choose how much bandwidth to buy up-front over a fixed contract horizon at given unit prices, but at quantities that depend on the expected time-average spot price which fluctuates in function of usage-sensitive packet marks. The packet marks serve as a basis for computing balancing payments that users make or receive at the end of the contract. 5.4 More differentiation The approach discussed above focuses on bandwidth differentiated services (throughput is the only attribute of QoS, customer utility and service differentiation), a single packet class in the network, and on long-lived rateadaptive elastic flows. The following studies consider extensions. Key and Massoulie´ (1999) and Gibbens and Kelly (1999a) consider a mix of users with rate-adaptive traffic and utility functions that only depend on throughput, as modeled above, and users with finite-size file transfers whose utility is a function of total transfer time, the ratio of data volume by throughput. Alvarez and Hajek (2002) and Gibbens and Kelly (2002) study marking mechanisms for priority scheduling policies for two packet classes, which may be beneficial in the presence of significant queueing delays and if some users value the ability to choose among delay or loss differentiated service classes. These studies make no attempt to optimize performance; they focus on the differences in throughput-, loss- and delay-performance for given marking schemes. Gibbens and Kelly (1999b) describe how a network can accommodate competing rate-adaptive traffic and nonadaptive traffic such as traditional telephone calls requiring low packet loss by using packet marks to make distributed admission control decisions. Courcoubetis and Weber (2003, p. 250) discuss how to apply ideas from the approach discussed here to allocate and price bandwidth to flows of real-time applications that have flexible peak rates but are also subject to a strict QoS bound. 5.5 Optimal dynamic pricing Recall that the analysis in this section focuses on a static regime with a fixed set of users with constant utility functions who transfer infinite-sized flows over fixed routes. The weighted proportional fairness criterion specifies the desired allocation for this fixed set of flows and the dynamic element of the approach shows how to attain this equilibrium via decentralized price and flow rate adaptation in the presence of incomplete
116
Philipp Afe`che
information. While bandwidth sharing and flow control has until recently typically been studied in this static context, in reality bandwidth sharing takes place in a dynamic stochastic environment: finite-sized documents arrive at random instants and depart when their transfer is complete. However, the topic of bandwidth sharing in such dynamic environments has only recently received research attention, and the study of optimal dynamic pricing seems completely open. Therefore the following discussion merely outlines basic questions that may arise if one wants to investigate optimal dynamic pricing in the context of dynamic bandwidth sharing. The topic of bandwidth sharing in dynamic stochastic networks has recently become the subject of intense research, e.g., Massoulie´ and Roberts (2000), de Veciana et al (2001), Bonald and Massoulie´ (2001), Roberts (2004). However, unlike the approach discussed above, these papers ignore the time scale on which flow control processes converge to equilibrium, assuming that the new bandwidth allocation is immediately attained as the set of ongoing flows changes. They focus instead on the dynamic effects occurring on the slower time scale of flow arrivals and departures, considering the stability and throughput and response time performance of networks that operate under different fairness rules. It is unclear how much insight can be gained on the interaction between these effects that occur on different time scales. However, several questions may be worth investigating for such stochastic dynamic environments. What is the right criterion for user-perceived QoS performance in such settings? Key and Massoulie´ (1999) and Roberts (2004) among others argue that the utility for the transfer of fixed-size documents may be more reasonably a function of response time than flow throughput at any given time. In the single-node case strict priority scheduling disciplines often outperform those that aim for ‘‘fairness’’ in terms of total system delay cost. Unfortunately, it is not clear which sharing criterion optimizes overall response time performance in a network, but max–min fair sharing has been shown to outperform proportional fair sharing in some settings, e.g., Massoulie´ and Roberts (2000). The issue of utility function formulation aside, how does the network behave under the proposed flow control and pricing mechanisms in a dynamic environment where flows arrive and terminate over time? Should the mechanisms be modified for such dynamic environments, and if so, how? Considering proportional fairness and alternative fairness criteria, how long does convergence to successive equilibrium bandwidth allocations take on average relative to the time scale of flow arrivals and departures? How does this adaptive ‘‘rebalancing’’ process affect operational and economic performance? What should be the relationship between flow level prices that users might pay prior to requesting a document and the charges they might be charged while a transmission is in progress via packet marks? Anderson et al. (2006) consider this latter question in the static context with a fixed number of flows.
Ch. 2. Economics of Data Communications
117
5.6 Conclusions and directions Sharing bandwidth among a fixed set of long-lived flows in a proportionally fair way with appropriate weights maximizes total system welfare if throughput is the only QoS attribute of user utility. The proportionally fair allocation is inherently flexible in that it takes into account user utility functions and creates bandwidth-differentiated services with just the right amount of granularity: one ‘‘class’’ for each user. Distributed adaptive flow control and price algorithms are shown to attain this equilibrium bandwidth allocation even when the network has no or incomplete information on utility functions and users are unaware of capacity constraints and congestion. The approach assumes that capacity is adequate and queueing delays are small in comparison with propagation delays, a simple network with a single packet class and differentiation provided by the algorithms located at the end nodes. The approach is appealing since it offers users flexibility, is based on local information and leverages simple and largely existing flow control mechanisms. The uncertainty in prices and the complexity of end node decisions due to the price fluctuations that are inherent to this approach pose some challenges, but they may be overcome by designing appropriate bandwidth markets and user agents. Several issues appear to have received little attention so far and to merit further research. Dynamic stochastic environments. The issues considered here have not been studied in dynamic stochastic environments where the optimal bandwidth allocations and the respective prices fluctuate as flows arrive and depart. (See the discussion in Section 5.5.) Pricing user congestion cost. In contrast to the models for best effort services in Section 4, here a user’s utility function only depends on her own throughput. The congestion cost, represented by the functions Cl, l A L, is a known system cost for the network. Therefore the socially optimal prices are immediate from the packet loss rate and the corresponding packet marks. By contrast, if users are also sensitive to congestion via QoS metrics such as packet loss or delay (see Section 5.4), then the congestion cost is not known to the network and may vary across users. How to translate congestion signals into prices in this case to attain a socially optimal network allocation? A useful reference point are Gupta et al. (1997), Masuda and Whang (1999) and Stidham (2003) who consider adaptive pricing of best effort services for delay-sensitive customers, i.e., whose utility is an explicit function of delay. Stidham (2003) introduces user heterogeneity with respect to delay sensitivity in the model of Kelly et al. (1998). He shows that this introduces a fundamental nonconvexity into the congestion-cost functions. As a result, there are typically multiple stationary points, and the rate-control algorithm may converge to a local rather than global maximum, depending on the starting point.
118
Philipp Afe`che
Flow level versus customer level decisions. The analysis discussed above focuses on customers’ flow level payment and usage decisions. How do these decisions relate to their overall subscription and usage decisions? Profit maximization. The body of work discussed here focuses on social welfare maximization whereas the Internet is operated by profit-maximizing providers. How should a profit-maximizing provider who does not know user preferences design prices and packet marking algorithms? What is the impact of competition? How should interconnected networks share payments based on congestion marks and how can these marks be propagated among networks with different technologies? 6
Discussion
After considering each contract type in isolation the survey now discusses broad pricing and service design issues that concern all contract types: the benefits and challenges of auctions versus posted prices; the debate on flatrate versus usage-based pricing; and the merits and challenges of alternative QoS designs and their delivery machanisms. 6.1 Auctions versus posted prices This survey has so far focused on posted price mechanisms. A fundamentally different approach to selling data communication services is via auctions. Customers submit bids and the provider determines prices and allocations based on their bids as a function of previously announced rules. See Krishna (2002) for a recent treatment of auction theory and Ockenfels, Reiley and Sadrieh (2006) for a survey of online auctions. Vickrey’s (1961) second-price sealed-bid auction inspired the ‘‘smart market’’ proposal of MacKie-Mason and Varian (1995a) which received considerable attention in the data pricing community. An important motivating factor for the smart market is the demand-revealing property of the classic second-price auction that allows the provider to attain the efficient allocation even if she has no information on demand functions. This section considers the benefits and challenges of the smart market proposal and other auction-based approaches that differ in the auction format they study (first-price or second-price auction) and the objects that users bid on (data packets, network resources, guaranteed services or best effort priority services.) The discussion also considers under what conditions the demand-revealing property of the second-price auction, which is only known to hold in static settings, may be preserved in dynamic data communications settings. An exhaustive discussion of auction formats and issues is beyond the scope of this survey. See Courcoubetis et al. (2001) for an example of a descending bandwidth auction for a multi-link network. They propose a version of a Dutch auction and discuss several implementation and economic performance issues in comparison to other auction formats.
Ch. 2. Economics of Data Communications
119
The second-price auction and the smart market proposal In the classic second-price auction a single indivisible unit is sold to bidders with independent private values and is awarded to the highest bidder at the second highest bid. Thus the winner pays the social cost or surplus loss that her use inflicts on other bidders, measured based on the reported valuations. This payment rule implies the celebrated demand-revealing property of the second-price auction: it is a dominant strategy for each bidder to bid her true preferences (regardless of others bidders’ actions) and so the equilibrium bids yield the efficient allocation. The Vickrey auction and the Vickrey-Clarke-Groves mechanism extend the second-price auction to multiple identical and nonidentical units, respectively, and share its demand revelation and efficiency properties (Clarke, 1971; Groves, 1973; see Loeb, 1977, for a comparison of these mechanisms.) The resulting prices also reflect the marginal value of capacity which send the correct signal about efficient capacity investments. MacKie-Mason and Varian (1995a) propose to repeatedly execute at each network node packet-level Vickrey auctions in short successive time intervals: in each time interval the bandwidth available for transmitting K data packets is allocated to the K highest-bid packets present at that time. The smart market proposal has some appeal, in part due the attractive theoretical properties of the underlying second-price auction, and also because it allows prices to be determined immediately based on user bids rather than gradually in response to user demand, as in adaptive pricing approaches such as those studied by Low and Varaiya (1993), Gupta et al. (1997), Kelly et al. (1998), Masuda and Whang (1999) and Thomas et al. (2002). However, the nature of data communications demand poses significant challenges for the implementation of the smart market or indeed any auction-based approach. Resource and time aggregation The basic smart market proposal envisions no aggregation: each auction sells individual units of packet transmission capacity at a network node over a short time horizon. However, users assign value to successful end-toend transmissions of complete data flows, e.g., email messages or database queries, rather than to their component data packets whose values are interdependent. Since complementary packets may be routed through different nodes and arrive over multiple bidding periods, it is not feasible to elicit valuations and run bandwidth auctions at the packet and node level. MacKie-Mason (1997) and Lazar and Semret (1998) apply the smart market approach at a higher aggregation level, allowing users to bid on service units or resource bundles with more easily quantifiable values. In MacKieMason (1997) users bid for rates of service units over longer demand periods, each service unit for end-to-end delivery of an entire data stream over a network with certain QoS guarantees. Users submit utility functions over their allocation and the provider determines the allocation and payments using the Vickrey–Clarke–Groves mechanism. In contrast to the packet-level
120
Philipp Afe`che
auctions, here users bid on resources only indirectly, via their bids for services with QoS guarantees. In Lazar and Semret’s (1998) ‘‘progressive second price auction’’ users with elastic demand and fixed routes directly bid for shares of resource quantities along their routes. Their analysis applies to any infinitely divisible resources where consumption is additive, but the notion that bidders know their valuations of resource quantities may be most plausible if the auction is for bulk bandwidth for aggregate flows like virtual paths. Lazar and Semret (1998) propose and analyze an iterative variation of the second price auction. They prove existence of a truthful and efficient A-Nash equilibrium where all players bid at prices equal to their marginal resource valuations. This is weaker than the dominance result but it comes at the benefit of a minimal message space: unlike in the standard format, users do not bid their entire utility function at once, but rather a price-quantity pair in response to the (observable) bids already in competition for the resource. Convergence to the equilibrium is shown experimentally. Dynamic effects The properties of the Vickrey auction and indeed most auction results are developed for static one-shot environments: all bidders and units for sale are present at the start of the auction and all units are allocated at once. In contrast, data communications demand is dynamic: it occurs as a random sequence of asynchronous requests whose resource requirements partially overlap in time. The static auction results need not hold in this dynamic environment due to several potentially significant intertemporal effects: the arrival and departure of bidders over time; their cost of delay between service request, allocation and delivery; complementarities between resource requirements over time; and spill-overs of unserved demand from one auction to the next. As a case in point, Milgrom and Weber (2000) show that in a sequential second-price auction where bidders strategize over time and the selling prices are revealed after each auction, bidding one’s true value is a dominant strategy only in the last auction, but not in earlier stages—there is an option value of winning at a later auction and lower price, and unlike in the one-shot case the equilibrium bids depend on the entire value distribution. Therefore, mechanisms such as the smart market that run repeated one-shot Vickrey auctions over nonoverlapping time periods inherit its demand-revealing and efficiency properties only if intertemporal effects are insignificant or assumed away. One may argue that the extent of resource and time aggregation has an impact on how closely the static results approximate those for repeated one-shot auctions: intertemporal effects are fundamental to bidding strategies in packet-level second-by-second auctions but perhaps less important, e.g., in auctions that sell contracts for daily usage rates. This reflects the trade-off between tractability and optimality inherent in the aggregation level: the more aggregation, the more restrictions on bidding strategies and the fewer the opportunities to finetune bids and allocations.
Ch. 2. Economics of Data Communications
121
Several papers consider auctions for priority service at a single resource among dynamically arriving and departing delay-sensitive bidders. With the main exception of Balachandran (1972) who characterizes the equilibrium of state-dependent bidding strategies, these papers (discussed below) assume that bidders cannot observe the queue length, which reduces the environment to a static auction with delay externalities. Dynamic auctions have received increased attention in recent years. Broad issues include what information is available to bidders on competing bidders and their history, how often bids may be updated and based on what events (e.g., a new arrival) and when and how to allocate units. For examples, see Etzion et al. (2006), Caldentey and Vulcano (2006), Pai and Vohra (2006) who consider settings outside the data communications context where a fixed number of units are sold over a fixed time horizon to dynamically arriving bidders. Delay externalities The smart market approach does not explicitly consider externalities: a bidder’s valuations only depend on her own consumption, and if users bid on services (as opposed to resources) the network guarantees quality. However, if the auction is for services without quality guarantees and congestion is significant, then externalities are relevant and each bidder’s utility also depends on the consumption of other bidders. Several papers study auction-based service allocations with delay externalities in the context of single-resource single- or multi-class queueing models that offer ‘‘best effort’’ service to delay-sensitive customers. Whang (1990) studies the Clarke–Groves mechanism for FIFO service. Glazer and Hassin (1985), Lui (1985), Hassin (1995), Stahl (2002), Afe`che and Mendelson (2004) and Kittsteiner and Moldovanu (2005) study first-price auctions for priority service and Stahl (2002) also considers a second-price priority auction. The first-price priority auction is not demand-revealing and requires from bidders common knowledge about all demand and supply factors. Furthermore, in the presence of externalities the resulting allocation is only efficient with an appropriately chosen reserve price which in turn depends on the demand distribution (Stahl, 2002; Afe`che and Mendelson, 2004). An exception is the case of preemptive priorities where the optimal reserve price is zero (see Hassin, 1995; Afe`che and Mendelson, 2004.) Thus, the first-price auction may only have an information advantage over a posted price mechanism if bidders are better informed about each other’s demand than the provider. Whether a second-price priority auction is demand-revealing likely depends on its specific design. In general the demand-revealing property of the static Vickrey–Clarke–Groves mechanism extends to settings with externalities, provided that bidders submit preferences over all users’ consumptions (MacKie-Mason and Varian, 1994; Jehiel and Moldovanu, 2005). Extending this static result to a queueing setting raises two questions. First, how to handle the fact that a queueing system is inherently dynamic? The
122
Philipp Afe`che
following common assumptions reduce the analysis to that of a static auction: A1. bidders do not observe the queue length (except in Balachandran, 1972) and do not choose their arrival time; A2. each bidder submits a single bid and cannot renege, i.e., withdraw her bid; A3. the Poisson arrival rate, value and delay cost distributions are common knowledge; A4. bidders form rational expectations about the delay distribution of the system that is assumed to be in steady-state. Assumptions A1–A4 allow bidders to compute their delay and payoff distribution as a function of all bid strategies, like in the static case. Second, under what conditions is a second-price auction satisfying assumptions A1 and A2 demand-revealing and efficient? Stahl (2002) presents examples where a second-price priority auction does not have a pure-strategy bid equilibrium, unlike in the classic static case. However, it is important to emphasize that Stahl (2002) has bidders submit preferences only over their own transmission, rather than the overall usage rates, as required for demand-revelation in the presence of externalities (MacKie-Mason and Varian, 1994; Jehiel and Moldovanu, 2005). His finding therefore is not inconsistent with static auction results and the apparent discrepancy follows from restricting the bid strategy space in the presence of externalities. Nevertheless, his conclusion that ‘‘efficiency arguments [from static auctions] cannot be used to justify the implementation of simple y second price auctions in dynamic stochastic environments’’ is certainly valid. First, truly dynamic auctions, as opposed to static ones fitted to dynamic resource allocation problems, indeed need not inherit known properties from static auctions. Second, submitting preferences over all consumptions in the presence of delay externalities requires bidders to understand how their delay distribution depends on overall usage rates, capacities and the priority mechanism—hardly a realistic requirement. What may work instead, at least in principle, is for users and the provider to contribute information. Each user declares her utility as a function of only her own quantity and delay, regardless of how the delay relates to overall usage, and the provider determines the allocation using her knowledge of the relationship between usage and delay performance. Revenue considerations The demand-revealing and efficiency properties of second price auctions do not address the issue of how to maximize provider revenues. The key is that these auctions need not generally be the revenue-maximizing format in multiunit settings. Choosing the revenue-maximizing auction format and reservation price typically does require knowledge of the demand distribution. Computational and accounting overhead Theoretical questions of demand-revelation, efficiency and revenue maximization aside, auctions may impose an unreasonably high overhead for computation, communication and accounting. Bidders’ utility functions may be complex and their strategies sophisticated, particularly if they bid
Ch. 2. Economics of Data Communications
123
on multiple services or resources in a dynamic setting. The inherent price uncertainty also makes it harder to plan usage compared to a quote-based scheme. The provider’s tasks of tracking bids and determining the optimal allocation and payments is computationally more intense than under a posted price scheme, particularly in a second price auction where each winner’s payment depends on all other bids. Conclusions In assessing the value of an auction design compared to a posted price mechanism for data communication services one should trade off its potential for demand revelation and efficiency gains with the complexity of the bidding mechanism. The potential efficiency gains of an auction may be large enough to offset the overhead in rapidly changing nonrepetitive demand environments, since this implies a large information gap between users and the provider, and for units with significant transaction values where inefficiencies are costly. These requirements suggest that auctions are only viable if they sell units with a minimum level of resource and time aggregation, e.g., not individual packet transmissions over seconds, but larger bandwidth chunks over days or weeks, probably sold to larger retail or business customers. Such aggregation leads to less frequent auctions, each for fewer but more bundled units; this sacrifices some optimality by restricting the bid strategies but also increases the transaction values and reduces the overall overhead. The following are to be considered in evaluating the demand revelation potential of a particular auction design. First, the ideal outcome of ‘‘complete’’ demand revelation of users with no information on each others’ demand is only known to hold in the static Vickrey auction—it extends neither to other static formats such as the first price auction nor to dynamic auctions. Second, the performance of the static Vickrey auction may be a good approximation of the outcome in repeated one-shot auctions with negligible intertemporal effects, as may be the case if each auction exhibits significant time aggregation. Each demand period in a repeated auction should be long enough to ensure significant transaction values but short enough so the allocation can adapt to changing demand. Third, while any ‘‘truly dynamic’’ auction mechanism is likely to lack the complete demand revelation property of the static Vickrey auction, an ‘‘appropriately’’ designed dynamic auction may still have a partial demand revelation advantage over a fixed price scheme in environments where the provider cannot keep up with demand changes (cf. Whang, 1990). Indeed, even if the optimal ‘‘truly dynamic’’ auction mechanism could be identified, it would likely be very complicated, involving state and timedependent bids (Gupta et al., 2006), and require from users an unrealistic amount of information on each others’ preferences. Fourth, whether users have enough information on their own preferences, a necessary condition for demand revelation even in the static Vickrey auction, depends in part on the type of resources or services auctioned.
124
Philipp Afe`che
6.2 Flat-rate versus usage-based pricing Like much of the research activity on data pricing this survey has focused on usage-based pricing. This activity has been stimulated by the prevalence of Internet congestion and by the development of many service technologies that can implement sophisticated resource allocation and QoS differentiation mechanisms and that raise challenging research questions. In practice, usage-based pricing is far from universal. Traditional packetswitched networks (e.g., based on the X.25 protocol) have typically charged based on data volume and some Internet Service Providers have been offering usage-based pricing for high bandwidth access links in recent years (cf. Songhurst, 1999, p. 8). However, flat-rate pricing has been and still is the prevalent tariff structure in the Internet retail market, whereby a user pays a flat periodic subscription fee that only depends on the bandwidth of her access link. Thus, her charge is related to her maximum sending rate but independent of actual usage, quality or time of service. Flat-rate versus usage-based pricing The co-existence of these two polar tariff structures raises a simple question: what are the relative benefits and challenges of flat-rate versus usagebased pricing? Flat-rate pricing has many attractive features. It is cheap to implement and operate, leads to simple and predictable user charges and provider revenues, and is quite popular with customers, which has stimulated the growth of the Internet. On the downside, flat-rate pricing typically results in a socially inefficient allocation and market segmentation with light users subsidizing heavy users. Usage-based pricing, if appropriately designed, is superior to flat-rate pricing in terms of economic performance, giving the provider a more powerful tool to control congestion, segment the market and increase revenues. This was also confirmed empirically by Internet Demand Experiment, a market and technology trial conducted at the University of California at Berkeley (cf. Edell and Varaiya, 1999). In general, the potential economic benefit of usage-based pricing must be weighed against the potential drawbacks, including user adoption problems, a higher cost and complexity of implementation, and the requirement for more information about user preferences and network performance. There are also arguments in favor of two-part tariffs that combine a flat and a usage-sensitive component. For settings with congestion costs, MacKie-Mason and Varian (1995a) argue for an efficient two-part tariff with subscription fees at levels that cover the infrastructure cost and usage-based charges that depend on congestion. MacKie-Mason and Varian (1995b) show for the monopoly case that a two-part tariff mitigates the problem of inefficient use and capacity investment that arises under usage-based pricing only. They also compare competitive market equilibria under two-part tariffs and connection only fees. They argue that the absence of a usage-sensitive
Ch. 2. Economics of Data Communications
125
tariff component may result in an equilibrium with lower usage but higher congestion than when usage is also priced. Essegaier et al. (2002) study for access services without congestion costs the optimal choice among flat-rate, usage-based and two-part tariffs as a function of the capacity level and user heterogeneity. They show that two-part tariffs are not always optimal. Optimality versus implementability The debate on flat-rate versus usage-based pricing points to a fundamental trade-off between optimality and implementability criteria that should be considered in the design and evaluation of any pricing mechanism. Optimality loosely refers to the economic performance of a pricing mechanism in terms of total surplus and its distribution among providers and customers. Implementability depends on several criteria on the side of the network provider: the availability, cost-effectiveness, simplicity and scalability of the technical infrastructure required to implement network control, usage measurement and billing mechanisms; and the availability of the necessary demand and network congestion information to determine tariffs and charges. The implementability of a price mechanism also depends on the adoption by customers who typically want tariffs and usage decision processes to be simple and charges predictable, based on metrics they can understand and control. This trade-off highlights the importance of striking a good balance between optimality and implementability in designing a pricing scheme in practice. For example, a mechanism that price discriminates at the level of each individual packet may be theoretically optimal, but hardly implementable due to prohibitive implementation constraints and costs. Based on these insights Shenker et al. (1996) call for a shift away from the ‘‘optimality paradigm’’ and focus instead on structural and architectural pricing issues, including edge pricing at the network access points instead of in the core network, multicast and receiver charging. They also assert that computing the true congestion costs and obtaining detailed knowledge of actual congestion conditions along the entire path of a packet is not feasible and propose approximations to compute the price based on the expected congestion cost. This proposal laid the foundation for Clark’s (1997) expected capacity model which was later expanded into the DiffServ model discussed below. Conclusions The benefits and challenges of flat-rate pricing or any usage-based pricing scheme must be evaluated based on optimality and implementability criteria. Implementability criteria may not be as ‘‘quantifiable’’ as economic performance measures—at least some criteria have hardly been explicitly modeled so far, but they may be the decisive factor in choosing a pricing scheme. Implementability issues also imply that usage-based pricing and service differentiation schemes should not only be considered with the sole
126
Philipp Afe`che
objective of economic optimality; a significant improvement over flat-rate pricing for uniform service may already be sufficient to justify their deployment, especially if their implementation is relatively simple. 6.3 Providing QoS: overprovisioning versus control The optimality and implementability criteria discussed in the previous section imply that not only differentiated price schemes but also the QoS delivery mechanisms that support them should be relatively simple. In this sense the issue of implementability versus optimality is related to the debate on overprovisioning versus control in providing QoS. Generally QoS performance depends on demand characteristics and on ‘‘service production’’ which is a function of capacity and network control mechanisms. The two extreme approaches for providing good QoS are overprovisioning (a lot of excess capacity in the presence of very little control, e.g., a simple best-effort network) and control (sophisticated control mechanisms in the face of little excess capacity.) Overprovisioning The proponents of overprovisioning argue that the cost of excess capacity has already been incurred for the network backbone, making it possible to quickly and cheaply expand the capacity of this important network segment. By some measures networks are so lightly utilized that complex QoS technologies seem to bring no value. Data networks are overall lightly utilized on average: The Internet backbones exhibit average utilizations of 10–15% and peak utilizations of 25%. Private line networks have average utilizations of 3–5% and peak utilizations of 15–25%. For comparison, U.S. long distance voice networks have 33% and 70% average and peak utilizations, respectively, (Odlyzko, 1999a). However, parts of the Internet are highly congested, in particular: public peering points, network access points, access links that aggregate traffic into the backbone, and transatlantic links between the US and the rest of the world. Global overprovisioning is considered an economically prohibitive luxury for the foreseeable future (Gevros et al., 2001). The fact that capacity does not come cheaply throughout the network infrastructure casts doubt on the universal validity of the overprovisioning argument. Bandwidth bottlenecks may well persist in certain network segments and may also occur intermittently in transition periods between successive capacity upgrades. Even if there were excess capacity throughout the network, it is questionable whether a competitive equilibrium with excess industry capacity is sustainable in the long run. Control: guaranteed services At the opposite end of the spectrum, QoS can be delivered by guaranteed services (as discussed in Section 3), exemplified by the IntServ architecture
Ch. 2. Economics of Data Communications
127
proposal of the IETF and by the ATM, Resource Reservation Protocol and Frame Relay standards. These approaches rely on sophisticated control mechanisms including admission control, reservation and enforcement based on per-flow state information within network routers. The challenge is scalability since the network core must maintain state information for each flow, which becomes a problem when the number of flows is high such as in the Internet. A possibility is to deploy these approaches at the network access level where the number of connections may be relatively small and bandwidth may be tight, which is when the value of service guarantees is large. Moderate solution: differentiated services This approach can be viewed as a moderate solution between overprovisioning and control. In contrast to guaranteed services the DiffServ architecture proposal of the IETF does not try to distinguish among individual data flows. Instead it classifies packets into one of a few categories. All packets within a class are then handled (e.g., scheduled and routed) in the same way, with the same QoS parameters. There is no per flow information at routers. As such the DiffServ model can be viewed as augmenting the basic elements of a multi-class best effort network (as discussed in Section 4) with demand regulation mechanisms that enforce certain traffic inflow constraints and help achieve statistical class-level QoS targets. Thus, complexity is reduced at the expense of control. A key advantage of such an approach is that it is more scalable than guaranteed services since the network only needs to distinguish among aggregate flows that belong to the different service classes, not among individual flows. It creates differentiated services with the least amount of network control. This approach should probably be viewed as increasing social welfare compared to no service differentiation, without attaining economic optimality. A downside of this approach is that it does not offer absolute QoS guarantees at the individual flow level, but rather average class-level guarantees. It is also not entirely clear exactly how distinctive levels of quality of service are to be achieved. DiffServ does not by itself say anything about the assignment of priorities to and the pricing of different traffic classes. Odlyzko’s (1999b) PMP proposal for Internet pricing is to partition the Internet into several logically separate ‘‘channels’’ for different service classes with higher prices for using the better provisioned and therefore (allegedly) less congested classes. One implication of these approaches is that the provider must monitor and adjust the capacity allocation of individual service classes if she is to maintain the desired average quality levels. Conclusion Extreme solutions such as guaranteed QoS require excessively complex micromanagement that needs a lot of overhead, whereas overprovisioning may not be feasible at all times and throughout the network. Simpler and
128
Philipp Afe`che
more effective solutions to QoS provisioning should draw on the benefits of network control mechanisms and capacity expansion in a way that mitigates their respective drawbacks. For example, a broad service segmentation into 3–4 classes likely provides much of the benefits of price and service differentiation without the prohibitive cost and complexity of tailoring a service class to each individual connection. Such service differentiation and the requisite control mechanisms should play a larger role when and where network bandwidth is tight and expensive.
7
Conclusions
This chapter has surveyed research on data transport pricing decisions for a single provider who pursues one of two economic performance objectives: welfare maximization or profit maximization. It has largely focused on studies that consider these issues in the context of mathematical models of data transport service demand and production. The discussion is organized according to an unifying framework that classifies data transport service contracts based on the types of their QoS promises into guaranteed, best effort and flexible bandwidth-sharing services. The survey highlights and compares the key features of each service type, provides a broad roadmap for important questions, modeling approaches and results, and outlines open questions. It discusses pricing decisions for each service type along common dimensions: basic pricing and allocation principles; incomplete information and adaptive pricing; more differentiation; optimal dynamic pricing; and conclusions and directions. The survey then examines benefits and challenges of auctions versus posted prices; the debate on flat-rate versus usage-based pricing; and the merits and challenges of alternative QoS designs and their delivery mechanisms. 7.1 Research directions Sections 3–5 suggest research directions for each service type. The suggestions listed here refer to issues that appear relevant for all service types. Revenue & profit maximization. The majority of studies consider socially optimal pricing decisions; profit-maximization has received less attention so far, but would benefit from more attention in the future. Clearly in commercial data communications markets providers are primarily interested in their own profits, not in the surplus generated for the entire system. Service design. Most studies consider pricing for a given service design; the problem of jointly designing services and prices is important and interesting. This includes decisions on the number of service classes and the types of QoS guarantees to offer, and on the supply allocation mechanisms to deliver them.
Ch. 2. Economics of Data Communications
129
Service plans and bundles. In a similar vein, most analyses focus on determining the price for an individual connection or transmission, and consider tariffs that are linear in the number or rate of transmissions. This ignores potentially interesting bundling and nonlinear pricing options for users that have demand for larger service quantities over longer horizons. Similarly, it might be interesting to study the bundling of data transport services with complementary services such as information content, data hosting and computing applications (See Geng, Stinchcombe and Whinston, 2006, for a survey of product bundling.). Integrated services. The vast majority of studies consider networks that offer a single type of QoS guarantee. Only a handful of studies consider a mix of service types, e.g., best effort and strict QoS guarantees. There appears to be potential for more work along these lines. Integration of empirical demand models into pricing studies. A number of papers report on empirical analyses of demand for Internet-based data transport services, including Edell and Varaiya (1999), Altmann et al. (2000), Varian (2002) and Beckert (2005). However, these empirically derived demand models appear not to be used in other pricing studies. Ideally, more studies would build on such empirical models. 7.2 Further topics and readings A number of issues have not been considered in this survey. Songhurst (1999) discusses experimental and implementation issues. Courcoubetis and Weber (2003) provide a discussion of interconnection and regulatory issues, traffic splitting and multicasting; they also offer more discussion of some issues considered here. Gupta et al. (2006) offer another survey of data transport pricing. Hassin and Haviv (2003) survey the economics of queueing systems, an area which is closely related to the pricing of best effort services.
References Adiri, L., U. Yechiali (1974). Optimal priority-purchasing and price decisions in non-monopoly and monopoly queues. Operations Research 22, 1051–1066. Afe`che, P. (2004). Incentive-compatible revenue management in queueing systems: optimal strategic delay and other delay tactics. Manufacturing & Service Operations Management, to appear. Afe`che, P., B. Ata (2005). Revenue management in queueing systems with unknown demand characteristics. Working Paper, University of Chicago, Chicago, IL. Afe`che, P., H. Mendelson (2004). Pricing and priority auctions in queueing systems with a generalized delay cost structure. Management Science 50, 869–882. Alperstein, H. (1988). Optimal pricing policy for the service facilities offering a set of priority prices. Management Science 34, 666–671. Altman, E., N. Shimkin (1998). Individual equilibrium and learning in processor sharing systems. Operations Research 46, 776–784.
130
Philipp Afe`che
Altmann, J., B. Rupp, P. Varaiya (2000). Quality matters: some remarks on internet service provisioning and tariff design. Telektronikk 96, 20–25. Alvarez, J., B. Hajek (2002). On the use of packet classes in communication networks to enhance congestion pricing based on marks. IEEE Transactions Automatic on Control 47, 1020–1026. Anderson, E., F. Kelly, R. Steinberg (2006). A contract and balancing mechanism for sharing capacity in a communication network. Management Science 52, 39–53. Bajaj, S., L. Breslau, S. Shenker (1998). Is service priority useful in networks?. SIGMETRICS Performance Evaluation Review 26, 66–77. Balachandran, K. (1972). Purchasing priorities in queues. Management Science 18, 319–326. Beckert, W. (2005). Estimation of stochastic preferences: an empirical analysis of demand for internet services. Review of Economics and Statistics 87, 495–502. Bell, C., S. Stidham (1983). Individual versus social optimization in the allocation of customers to alternative servers. Management Science 29, 831–839. Berger, A., W. Whitt (1998a). Effective bandwidths with priorities. IEEE/ACM Transactions on Networking 6, 447–460. Berger, A., W. Whitt (1998b). Extending the effective-bandwidth concept to networks with priority classes. IEEE Communications Magazine 36, 78–84. Bergstrom, T., J. MacKie-Mason (1991). Some simple analytics of peak-load pricing. Rand Journal of Economics 22, 241–249. Bohn, R., H. Braun, K. Claffy, S. Wolff (1994). Mitigating the coming internet crunch: multiple service levels via precedence. Technical Report. San Diego Supercomputer Center, University of California, San Diego. Bonald, T., L. Massoulie´ (2001). Impact of fairness on internet performance. Proceedings of the ACM SIGMETRICS pp. 82–91. Bradford, R.M. (1996). Pricing, routing, and incentive compatibility in multiserver queues. European Journal of Operational Research 89, 226–236. Braess, D. (1968). U¨ber ein Paradoxon aus der Verkehrsplanung. Unternehmensforschung 12, 258–268. Caldentey, R., G. Vulcano (2006). Online auction and list price revenue management. Working Paper, New York University, New York. Chao, H.-P., R. Wilson (1987). Priority service: pricing, investment, and market organization. The American Economic Review 77, 899–916. Chen, H., M. Frank (2001). State dependent pricing with a queue. IIE Transactions 33, 847–860. Clark, D. (1997). A model for cost allocation and pricing in the internet, in: L. McKnight, J. Bailey (eds.), Internet Economics, MIT Press, Cambridge, MA, pp. 215–252. Clarke, E. (1971). Multipart pricing of public goods. Public Choice 11, 17–33. Cocchi, R., S. Shenker, D. Estrin, L. Zhang (1993). Pricing in computer networks: motivation, formulation, and example. IEEE/ACM Transactions on Networking 1, 614–627. Courcoubetis, C., M. Reiman (1999). Pricing in a large single link loss system, in: P. Key, D. Smith (eds.), Teletraffic Engineering in a Competitive World, Elsevier, pp. 737–746. Courcoubetis, C., F. Kelly, R. Weber (2000a). Measurement-based usage charges in communication networks. Operations Research 48, 535–548. Courcoubetis, C., F. Kelly, V. Siris, R. Weber (2000b). A study of simple usage-based charging schemes for broadband networks. Telecommunication Systems 15, 323–343. Courcoubetis, C., M. Dramitinos, G.D. Stamoulis (2001). An auction mechanism for bandwidth allocation over paths. Proceedings of the 17th International Teletraffic Congress. Courcoubetis, C., R. Weber (2003). Pricing Communication Networks: Economics, Technology and Modelling, Wiley, West Sussex, England. Crowcroft, J., P. Oechslin (1998). Differentiated end-to-end internet services using a weighted proportionally fair sharing TCP. ACM Communications Review 28, 53–67. Dewan, S., H. Mendelson (1990). User delay costs and internal pricing for a service facility. Management Science 36, 1502–1517. Dolan, R. (1978). Incentive mechanisms for priority queuing problems. Bell Journal of Economics 9, 421–436.
Ch. 2. Economics of Data Communications
131
Edell, R., N. McKeown, P. Varaiya (1995). Billing users and pricing for TCP. IEEE Journal of Selected Areas of Communication 13, 1162–1175. Edell, R., P. Varaiya (1999). Providing internet access: what we learn from INDEX. IEEE Network 13, 18–25. Edelson, N.M., D.K. Hildebrand (1975). Congestion tolls for poisson queuing processes. Econometrica 43, 81–92. Elwalid, A., D. Mitra (1992). Fluid models for the analysis and design of statistical multiplexing with loss priorities on multiple classes of brusty traffic. Proceedings of IEEE INFOCOM, 415–425. Elwalid, A., D. Mitra (1995). Analysis, approximations and admission control of a multiservice multiplexing system with priorities. Proceedings of IEEE INFOCOM, 463–472. Essegaier, S., S. Gupta, Z.J. Zhang (2002). Pricing access service. Marketing Science 21, 139–159. Etzion, H., E. Pinker, A. Seidmann (2006). Analyzing the simultaneous use of auctions and posted prices for on-line selling. Manufacturing and Service Operations Management 8, 68–91. Floyd, S. (1994). TCP and explicit congestion notification. ACM Computer Communication Review 24, 10–23. Floyd, S., V. Jacobson (1993). Random early detection gateways for congestion avoidance. IEEE/ACM Transactions on Networking 1, 397–413. Ganesh, A., K. Laevans, R. Steinberg (2006). Congestion pricing and non-cooperative games in communication networks. Operations Reasearch, to appear. Geng, X., M. Stinchcombe, A. Whinston (2006). Product bundling, in: T. Hendershott (ed.), Handwork on Economics and Information Systems, Elsevier, North Holland. Gevros, P., J. Crowcroft, P. Kirstein, S. Bhatti (2001). Congestion control mechanisms and the best effort service model. IEEE Network 15, 6–26. Ghanem, S. (1975). Computing central optimization by a priority pricing policy. IBM Systems J 14, 272–292. Gibbens, R., F. Kelly (1999a). Resource pricing and the evolution of congestion control. Automatica 35, 1969–1985. Gibbens, R., F. Kelly (1999b). Distributed connection acceptance control for a connectionless network. Proceedings of the 16th International Teletraffic Congress. Gibbens, R., F. Kelly (2002). On packet marking at priority queues. IEEE Transactions on Automatic Control 47, 1016–1020. Gibbens, R., R. Mason, R. Steinberg (2000). Internet service classes under competition. IEEE Journal on Selected Areas in Communications 18, 2490–2498. Giridharan, P., H. Mendelson (1994). Free-access policy for internal networks. Information Systems Research 5, 1–21. Glazer, A., R. Hassin (1985). Stable priority purchasing in queues. Operations Research Letters 4, 285–288. Golestani, S., S. Bhattacharyya (1998). A class of end-to-end congestion control algorithms for the Internet. Proceedings 16th International Conference on Network Protocols. Groves, T. (1973). Incentives in teams. Econometrica 41, 617–631. Gupta, A., B. Jukic, D. Stahl, A. Whinston (2000). Extracting consumers’ private information for implementing incentive compatible Internet traffic pricing. Journal of Management Information Systems 17, 9–29. Gupta, A., D. Stahl, A. Whinston (1997). A stochastic equilibrium model of internet pricing. Journal of Economic Dynamics and Control 21, 697–722. Gupta, A., D. Stahl, A. Whinston (1999). The economics of network management. Communications of the ACM 42, 57–63. Gupta, A., D. Stahl, A. Whinston (2006). Pricing traffic on interconnected networks: issues, approaches, and solutions, in: S. Majumdar, I. Vogelsang, M. Cave (eds.), Handbook of Telecommunications Economics, Volume 2: Technology Evolution and the Internet, Elsevier, Amsterdam. Ha, A. (1998). Incentive compatible pricing for a service facility with joint production and congestion externality. Management Science 44, 1623–1636. Ha, A. (2001). Optimal pricing that coordinates queues with customer-chosen service requirements. Management Science 47, 915–930.
132
Philipp Afe`che
Hassin, R. (1985). On the optimality of first come last served queues. Econometrica 53, 201–202. Hassin, R. (1986). Consumer information in markets with random product quality: the case of queues with balking. Econometrica 54, 1185–1196. Hassin, R. (1995). Decentralized regulation of a queue. Management Science 41, 163–173. Hassin, R., M. Haviv (2003). To Queue or not to Queue. Kluwer, Boston. Haviv, M., J. van der Wal (1997). Equilibrium strategies for processor sharing and queues with relative priorities. Probability in the Engineering and Informational Sciences 11, 403–412. Hui, J.Y. (1988). Resource allocation for broadband networks. IEEE J. Selected Areas of Communication 6, 1598–1608. Jacobson, V. (1988). Congestion avoidance and control. Proceedings ACM SIGCOMM, 314–329. Jehiel, P., B. Moldovanu (2005). Allocative and informational externalities in auctions and related mechanisms. Working paper, University of Bonn, Bonn, Germany. Johari, R., D. Tan (2001). End-to-end congestion control for the Internet: delays and stability. IEEE/ ACM Transactions on Networking 9, 818–832. Kelly, F. (1991). Loss networks. Annals of Applied Probability 1, 319–378. Kelly, F. (1994). On tariffs, policing and admission control for multiservice networks. Operations Research Letters 15, 1–9. Kelly, F. (1996). Notes on effective bandwidths, in: F. Kelly, S. Zachary, I. Ziedins (eds.), Stochastic Networks: Theory and Applications, Oxford University Press, pp. 141–168. Kelly, F. (1997). Charging and rate control for elastic traffic. European Transactions on Telecommunications 8, 33–37. Kelly, F. (2001). Mathematical modelling of the internet, in: B. Engquist, W. Schmid (eds.), Mathematics Unlimited – 2001 and Beyond, Springer, Berlin, pp. 685–702. Kelly, F. (2003). Fairness and stability of end-to-end congestion control. European Journal of Control 9, 159–176. Kelly, F., A. Maulloo, D. Tan (1998). Rate control in communication networks: shadow prices, proportional fairness and stability. Journal of the Operational Research Society 49, 237–252. Kenyon, C., G. Cheliotis (2001). Stochastic models for telecom commodity prices. Computer Networks 36, 533–555. Keppo, J. (2005). The pricing of point-to-point bandwidth contracts. Mathematical Methods of Operations Research, to appear 6, 191–218. Key, P., D. McAuley (1999). Differential QoS and pricing in networks: where flow-control meets game theory. IEE Proceedings Software 146, 39–43. Key, P., D. McAuley, P. Barham, L. Laevens (1999). Congestion pricing for congestion avoidance. Microsoft Technical Report. Key, P., L. Massoulie´ (1999). User policies in a network implementing congestion Pricing. Workshop on Internet Service Quality Economics, Cambridge, MA. Kittsteiner, T., B. Moldovanu (2005). Priority auctions and queue disciplines that depend on processing time. Management Science 51, 236–248. Kleinrock, L. (1964). Communication Nets. Dover, New York. Kleinrock, L. (1967). Optimum bribing for queue position. Operations Research 15, 304–318. Knudsen, N. (1972). Individual and social optimization in a multiserver queue with a general costbenefit structure. Econometrica 40, 515–528. Korillis, Y., A. Lazar, A. Orda (1995). Architecting noncooperative networks. IEEE Journal on Selected Areas in Communications 13, 1241–1251. Krishna, V. (2002). Auction Theory. Academic Press, San Diego, CA. Kulkarni, V., N. Gautam (1997). Admission control of multi-class traffic with service priorities in highspeed networks. Queuing Systems 27, 79–97. Kulkarni, V., L. Gun, P. Chimento (1995). Effective bandwidth vectors for multiclass traffic multiplexed in a partitioned buffer. IEEE Journal Selected Areas in Communications 13, 1039–1047. Kumar, A., D. Manjunath, J. Kuri (2004). Communication Networking: An Analytical Approach. Morgan Kaufmann, San Francisco, CA.
Ch. 2. Economics of Data Communications
133
Lapsley, D., S. Low (1999). Random Early Marking: an optimisation approach to Internet congestion control. Proceedings of IEEE ICON. Lazar, A., N. Semret (1998). Design, analysis and simulation of the progressive second price auction for network bandwidth sharing. Technical Report, Columbia University. Lippman, S., S. Stidham (1977). Individual versus social optimization in exponential congestion systems. Operations Research 25, 233–247. Low, D. (1974). Optimal dynamic pricing policies for an M/M/s queue. Operations Research 20, 545–561. Low, S., D. Lapsley (1999). Optimization flow control, I: basic algorithm and convergence. IEEE/ACM Transactions on Networking 7, 861–875. Low, S., P. Varaiya (1993). A new approach to service provisioning in ATM networks. IEEE/ACM Transactions on Networking 1, 547–553. Loeb, M. (1997). Alternative versions of the demand-revealing process. Public Choice 29, 15–26. Lui, F. (1985). An equilibrium queuing model of bribery. Journal of Polish Economy 93, 760–781. MacKie-Mason, J., H. Varian (1994). Generalized vickrey auctions. Working Paper, University of Michigan, Ann Arbor, MI. MacKie-Mason, J., H. Varian (1995a). Pricing the internet, in: B. Kahin, J. Keller (eds.), Public Access to the Internet, MIT Press, Cambridge, MA, pp. 269–314. MacKie-Mason, J., H. Varian (1995b). Pricing congestible network resources. IEEE Journal of Selected Areas in Communications 13, 1141–1149. MacKie-Mason, J. (1997). A smart market for resource reservation in a multiple quality of service information network. Working Paper. University of Michigan, Ann Arbor, MI. Maglaras, C., A. Zeevi (2003). Pricing and capacity sizing for systems with shared resources: approximate solutions and scaling relations. Management Science 49, 1018–1038. Maglaras, C., A. Zeevi (2005). Pricing and design of differentiated services: approximate analysis and structural insights. Operations Research 53, 242–262. Marbach, P. (2004). Analysis of a static pricing scheme for priority services. IEEE/ACM Transactions on Networking 12, 312–325. Marchand, M.G. (1974). Priority pricing. Management Science 20, 1131–1140. Massoulie´, L. (2002). Stability of distributed congestion control with heterogeneous feedback delays. IEEE Transactions on Automatic Control 47, 895–902. Massoulie´, L., J. Roberts (2000). Bandwidth sharing and admission control for elastic traffic. Telecommunication System 15, 185–201. Massoulie´, L., J. Roberts (2002). Bandwidth sharing: objectives and algorithms. IEEE/ACM Transactions on Networking 10, 320–328. Masuda, Y., S. Whang (1999). Dynamic pricing for network service: equilibrium and stability. Management Science 45, 857–869. Masuda, Y., S. Whang (2002). Capacity management in decentralized networks. Management Science 48, 1628–1634. Masuda, Y., S. Whang (2006). On the optimality of fixed-up-to tariff for telecommunications service. Information Systems Research 17, 247–253. Mendelson, H. (1985). Pricing computer services: queueing effects. Communications of the ACM 28, 312–321. Mendelson, H., S. Whang (1990). Optimal incentive-compatible priority pricing for the M/M/1 queue. Operations Research 38, 870–883. Mendelson, H., S. Whang (1994). Pricing for communication network service. Working Paper, Stanford University, Stanford, CA. Milgrom, P., R. Weber (2000). A theory of auctions and competitive bidding, II, in: P. Klemperer (ed.), The Economic Theory of Auctions, Edward Elgar, Cheltenham, UK. Mitchell, B., I. Vogelsang (1991). Telecommunications Pricing: Theory and Practice. Cambridge University Press, New York, NY. Mo, J., J. Walrand (2000). Fair end-to-end window-based congestion control. IEEE/ACM Transactions on Networking 8, 556–567.
134
Philipp Afe`che
Naor, P. (1969). On the regulation of queue size by levying tolls. Econometrica 37, 15–24. Ockenfels, A., D. Reiley, K. Sadrieh (2006). ‘‘Online Auctions.’’ in: T. Hendershott (ed.), Handbook on Economics and Information Systems. Elsevier, North Holland. Odlyzko, A., P. Fishburn (1998). Dynamic behavior of differential pricing and quality of service options for the internet. Proceedings First Intern. Confence on Information and Computation Economies (ICE98) 128–139. Odlyzko, A. (1999a). The current state and likely evolution of the internet. Proceedings in Globecom’ 99, 1869–1875. Odlyzko, A. (1999b). Paris metro pricing for the internet. Proceedings ACM Conference on Electronic Commerce 140–147. Odlyzko, A. (2003). Internet traffic growth: sources and implications, in: B. B. Dingel, W. Weiershausen, A. K. Dutta, and K.-I. Sato (eds.), Optical Transmission Systems and Equipment for WDM Networking II, Proc. SPIE, Vol. 5247, pp. 1–5. Oi, W. (1971). A disneyland dilemma: two-part tariffs for a mickey mouse monopoly. Quarterly Journal of Economics 85, 77–96. Oren, S., S. Smith (1981). Critical mass and tariff structure in electronic communications markets. The Bell Journal of Economics 12, 467–487. Pai, M., R. Vohra (2006). Optimal dynamic auctions. Working Paper, Northwestern University, Evanston, IL. Paschalidis, I.Ch., Y. Liu (2002). Pricing in multiservice loss networks: static pricing, asymptotic optimality, and demand substitution effects. IEEE/ACM Transactions on Networking 10, 425–438. Paschalidis, I.Ch., J.N. Tsitsiklis (2000). Congestion-dependent pricing of network services. IEEE/ACM Transactions On Networking 8, 171–184. Rao, S., E.R. Petersen (1998). Optimal pricing of priority services. Operations Research 46, 46–56. Roberts, J. (2004). A survey on statistical bandwidth sharing. Computer Networks: The International Journal of Computer and Telecommunications Networking 45, 319–332. Rohlfs, J. (1974). A theory of interdependent demand for a communications service. The Bell Journal of Economics and Management Science 5, 16–37. Ross, K. (1995). Multiservice Loss Models for Broadband Telecommunications Networks. Springer, London. Rump, C., S. Stidham (1998). Stability and chaos in input pricing for a service facility with adaptive customer response to congestion. Management Science 44, 246–261. Semret, N., A. Lazar. (1999). Spot and derivative markets in admission control. Proceedings of the 16th International Teletraffic Congress. Shenker, S. (1995). Service models and pricing policies for an integrated services Internet, in: B. Kahin, J. Keller (eds.), Public Access to the Internet, Prentice Hall, Englewood Cliffs, NJ. Shenker, S., D. Clark, D. Estrin, S. Herzog (1996). Pricing in computer networks: reshaping the research agenda. ACM SIGCOMM Computer Communication Review 26, 19–43. Shy, O. (2001). Dynamic peak-load pricing. Working Paper, University of Haifa. Songhurst, D. (1999). Charging Communication Networks: From Theory to Practice. Elsevier, Amsterdam. Stahl, D. (2002). The inefficiency of first and second price auctions in dynamic stochastic environments. Netnomics 4, 1–18. Stahl, D., A. Whinston (1994). A general economic equilibrium model of distributed computing, in: W. Cooper and A. Whinston (eds.), New Directions in Computational Economics, pp. 175–189. Stidham, S. (1992). Pricing and capacity decisions for a service facility: stability and multiple local optima. Management Science 38, 1121–1139. Stidham, S. (2003). Pricing and congestion management in a network with heterogeneous users. Working Paper, University of North Carolina, Chapel Hill, NC. Tanenbaum, A. (2003). Computer Networks 4th ed. Prentice-Hall, Upper Saddle River, NJ. Thomas, P., D. Teneketzis, J. MacKie-Mason (2002). A market-based approach to optimal resource allocation in integrated-services connection-oriented networks. Operations Research 50, 603–616.
Ch. 2. Economics of Data Communications
135
Upton, D. (2002). Modelling the market for bandwidth. Ph.D. Dissertation, University of Cambridge, Cambridge, UK. Van Mieghem, J.A. (2000). Price and service discrimination in queuing systems: incentive compatibility of Gcm scheduling. Management Science 46, 1249–1267. Van Mieghem, J.A., P. Van Mieghem (2002). Price-coupled scheduling for differentiated services: GcM vs. GPS. International Journal of Communication Systems 15, 429–452. Varian, H. (2002). The demand for bandwidth: evidence from the INDEX project, in: R. Crandall, J. Alleman (eds.), Broadband: Should We Regulate High-Speed Internet Access?, AEI-Brookings Joint Center for Regulatory Studies, Washington, DC, pp. 39–56. de Veciana, G., T. Lee, T. Konstantopoulos (2001). Stability and performance analysis of networks supporting elastic services. IEEE/ACM Transactions on Networking 9, 2–14. de Veciana, G., J. Walrand (1995). Effective bandwidths: call admission traffic policing and filtering for ATM networks. Queueing Systems 20, 37–59. Vickrey, W. (1961). Counterspeculation, auctions, and competitive sealed tenders. Journal of Finance 16, 8–37. Vinnicombe, G. (2002). On the stability of networks operating TCP-like congestion control. Proceedings 15th IFAC World Congress on Automatic Control. Walrand, J., P. Varaiya (2000). High-Performance Communications Networks 2nd ed. Morgan Kaufmann, San Francisco, CA. Wang, Q., J. Peha, M. Sirbu (1997). Optimal pricing for integrated-services networks, in: L. McKnight, J. Bailey (eds.), Internet Economics, MIT Press, Cambridge, MA, pp. 353–376. Westland, J. (1992). Congestion and network externalities in the short run pricing of information system services. Management Science 38, 992–1009. Whang, S. (1990). Alternative mechanisms of allocating computer resources under queueing delays. Information Systems Research 1, 71–88. Wilson, R. (1993). Nonlinear Pricing. Oxford University Press, New York, NY. Wilson, R. (1989). Efficient and competitive rationing. Econometrica 57, 1–40. Wischik, D. (2001). How to mark fairly. Statistical Laboratory Research Report. University of Cambridge, Cambridge, UK. Wu, X., P. Marbach (2003). Pricing and rate control in a non-cooperative environment. IEEE Conference on Decision and Control. Yahalom, T., J.M. Harrison, S. Kumar (2006). Designing and pricing incentive compatible grades of service in queueing systems. Working Paper, Stanford University, Stanford, CA. Yechiali, U. (1971). On optimal balking rules and toll charges in the GI/M/1 Queuing Process. Operations Research 19, 348–370. Yechiali, U. (1972). Customers’ optimal joining rules for the GI/M/s queue. Management Science 18, 434–443. Zhang, J. (1993). Performance study of Markov modulated fluid flow models with priority traffic. Proceedings IEEE INFOCOM, pp. 10–17. Zhang, Z. (1997). Large deviations and the general processor sharing scheduling for a two-queue system. Queuing Systems 26, 229–254. Zhang, Z., D.Dey, Y. Tan (2006). Pricing in communication services with delay guarantee. INFORMS Journal on Computing, forthcoming.
This page intentionally left blank
Terrence Hendershott, Ed., Handbooks in Information Systems, Vol. 1 Copyright r 2006 by Elsevier B.V.
Chapter 3
Firms and Networks in Two-Sided Markets
Daniel F. Spulber Kellogg School of Management, Northwestern University, Evanston, IL 60208, USA
Abstract The chapter presents a theory of two-sided markets. Firms create and operate centralized allocation mechanisms both by matching buyers and sellers and by market making. Buyers and sellers have the option of decentralized search, matching, and bargaining. The chapter applies network theory to examine the transaction costs of alternative centralized and decentralized allocation mechanisms. It examines the efficiency of two-sided markets both for homogenous products and for differentiated products. Firms employ information systems to improve communication between buyers and sellers and to improve computation through centralized market mechanisms. Centralized allocation mechanisms offered by firms can increase economic efficiency and reduce transaction costs relative to decentralized exchange. Firms play a critical economic role in establishing the microstructure of markets.
1
Introduction
All markets are two sided—they have a buyer side and a seller side. The term two-sided market refers to a collection of individual buyers and individual sellers such that the buyers on one side of the market can transact only with the sellers on the other side of the market. Two-sided markets have two main modes of organization: decentralized and centralized. In a decentralized market, buyers and sellers match with each other and determine transaction prices. In a centralized market, firms act as intermediaries between buyers and sellers, principally by matchmaking and market making. In this article, I derive and present a number of new results on intermediation by firms in two-sided markets. I compare the relative efficiency of 137
138
D.F. Spulber
decentralized and centralized two-sided markets. I also review a number of key developments in the literature on two-sided markets. The analysis sheds light on the theory of the firm and on market mechanisms. Two-sided markets present economic actors with a critical problem. Buyers and sellers must find each other for the purpose of economic transactions. In a decentralized market, buyers and sellers must search for each other. In a centralized market, firms can provide matchmaking services that improve information exchange between buyers and sellers or more advanced matchmaking services that implement assignments of buyers and sellers.1 Another critical problem that buyers and sellers face in two-sided markets is to determine their terms of trade. In a decentralized market, buyers and sellers must handle their own negotiations through bilateral or multilateral transactions. In a centralized market, firms provide market making services that establish prices and balance supply and demand (see Spulber, 1999). Firms employ information systems for communication and computation functions that are necessary for matchmaking and market making services. Firms use information systems to improve the effectiveness of information exchange between buyers and sellers, to determine efficient assignments of buyers and sellers, and to implement transaction mechanisms such as auctions and posted prices. The result can be greater economic efficiency relative to decentralized exchange. I apply network theory to represent decentralized and centralized twosided markets. I review some of the basics of network theory, which provides highly useful tools for representing buyer–seller interactions and for modeling transaction costs. The study of networks using graph theory covers almost three centuries beginning in 1736 with Leonhard Euler’s celebrated problem of the seven bridges of Ko¨nigsberg (Euler, 1736). Network theory sheds light on decentralized allocation mechanisms in which consumers transact directly. Network theory also helps to explain centralized allocation mechanisms operated by one or more firms acting as intermediaries. The main contributions of the article are as follows. First, I explore the implications of costly communication for the design of market mechanisms. I extend existing results on allocation in buyer–seller networks (Kranton and Minehart, 2000, 2001). I examine markets with homogenous products in which not only buyers have differences in willingness to pay but also sellers have cost differences. In addition, I extend the model of buyer–seller networks to markets in which there are differentiated products. Using network theory, I provide sufficient conditions under which allocations on incomplete networks can be efficient both for markets with homogenous products and markets with differentiated products.
1 An assignment is defined as a set of buyer–seller pairs that matches each buyer to a distinct seller or each seller to a distinct buyer.
Ch. 3. Firms and Networks in Two-Sided Markets
139
Second, I examine the Gale–Shapley stable assignment in a decentralized market and modify the model by adding money transfers between buyers and sellers. I show that the stable assignment with differentiated products and money transfers is unique. The stable assignment with homogenous products is shown to be an efficient assignment. I show that with costly communication due to an incomplete network, the stable assignment need not be efficient. I consider the stable assignment with differentiated products and I give sufficient conditions for the stable assignment to be an efficient assignment. Third, I introduce a firm into the assignment problem. The firm can charge subscribers for admission to a communications network. Once they have subscribed to the network, buyers and sellers experience costless communication so subscribers can attain a stable assignment. I examine how a profit-maximizing firm chooses subscription prices for buyers and sellers. I show the generality of Bo¨hm-Bawerk’s (1891) method of marginal pairs by extending it to a market with differentiated products. Additionally, I show how the method of marginal pairs corresponds to a stable assignment in markets for homogenous products and for differentiated products. Fouth, I present a two-sided Vickrey auction by a monopoly firm in a market with differentiated products. The method is related closely to double Vickrey auctions in markets for homogenous and for differentiated products. Finally, I consider the connection between search models and random networks in two-sided markets. I compare the efficiency of buyer– seller search with centralized assignments. Economists have studied two-sided markets for well over a century. Eugen von Bo¨hm-Bawerk’s (1891) famous market for horses originates the important idea of a two-sided market. Bo¨hm-Bawerk presents what is essentially a double auction 70 years before William Vickrey’s (1961) landmark work. Bo¨hm-Bawerk introduces the term ‘‘two-sided competition,’’ stating that ‘‘The case of two-sided competition is the most common in economic life, as it is the most important in the development of the Law of Price. It demands, therefore, our most careful attention.’’ Rather than characterizing supply and demand as aggregates, he begins with individual buyers and sellers and their subjective valuations. Bo¨hm-Bawerk (1891, p. 213) observes that If all are to exchange at one market price, the price must be such as to suit all exchanging parties; and since, naturally, the price which suits the least capable contracting party suits, in a higher degree, all the more capable, it follows quite naturally, that the relations of the last pair whom the price must suit, or, as the case may be, the first pair whom it cannot suit, afford the standard for the height of the price.
The market clearing price and quantity are determined by the marginal pair who trade, that is, the buyer–seller pair who have the smallest positive difference between the buyer’s value and that of the seller, or by the marginal pair who are excluded from trade.
140
D.F. Spulber
The two-sided market model is developed further by John von Neumann and Oskar Morgenstern (1944). In their classic book Games and Economic Behavior, they apply Bo¨hm-Bawerk’s framework as their main, indeed their only, economic interpretation of general non-zero-sum games.2 Lloyd Shapley (1961) introduces a related multiplayer market game. David Gale and Lloyd Shapley (1962) examine two-sided matching markets including students and universities, marriage partners, and renters and housing. Shapley and Shubik (1972, p. 111) employ the term ‘‘two-sided market’’ and observe that Two-sided market models are important, as Cournot, Edgeworth, Bo¨hm-Bawerk, and others have observed, not only for the insights they may give into more general economic situations with many types of traders, consumers, and producers, but also for the simple reason that in real life many markets and most actual transactions are in fact bilateral—i.e., bring together a buyer and a seller of a single commodity.
Shapley and Shubik (1972) analyze two-sided markets both for homogenous products and for differentiated products.3 The traditional literature on two-sided markets in game theory focused on decentralized exchange and thus has not given sufficient attention to the role of the firm. The literature on market microstructure in economics and finance has highlighted the role of the firm in intermediating exchange (see Spulber, 1996a, b, 1998, 1999, 2002b). The industrial organization literature has begun to extend analyses of product market competition to the consideration of competition between intermediaries (see Stahl, 1988; Gehrig, 1993; Spulber, 1996b, 1999; Fingleton, 1997; Caillaud and Jullien, 2003; Rust and Hall, 2003; Loertscher, 2004, 2005; Shevchenko, 2004; Weill, 2005; Hendershott and Zhang, 2006).
2
Firms in two-sided markets
The concept of a two-sided market is fundamental to models of intermediation (see Spulber, 1996a, b, 1998, 1999, 2002a, b, 2003). Firms act as matchmakers by bringing together buyers and sellers. Firms also act as market makers by establishing and operating the institutions of exchange. 2 von Neumann and Morgenstern (1944, p. 562) observe that ‘‘The size of the transaction, i.e. the number t0 of units transferred, is determined in accord with Bo¨hm-Bawerk’s criterion of the ‘marginal pairs.’’’ 3 A substantial literature examines matching in two-sided markets. Much of this literature extends the work of Gale and Shapley (1962) and Shapley and Shubik (1972). Hylland and Zeckhauser (1979) examine efficient assignments by a social choice mechanism in which individuals report their preference and the mechanisms assigns probabilities to position. Demange (1982), Leonard (1983), and Demange and Gale (1985) examine auction-based allocation mechanisms. For an overview of this literature see Roth and Sotomayor (1990). Two-sided matching markets include workers and employers, particularly at the entry level in such specialized professional markets as law, medicine, and business (see Roth and Xing, 1994). Becker (1973) presents a model of a marriage market in which types are multiplicative and therefore act as complements.
Ch. 3. Firms and Networks in Two-Sided Markets Buyers
141
Sellers
1
1
2
2
3
3 A decentralized market Buyers
Sellers
1
1 Firm
2
2
3
3 A centralized market
Fig. 1. In a decentralized market, buyers and sellers communicate directly and engage in bilateral transactions. In a centralized market, a firm establishes a hub-and-spoke network to provide matchmaking and market making services.
Spulber (1999) examines in a general setting how firms establish markets and act as intermediaries between buyers and sellers. Firms offer potential advantages over decentralized exchange in solving problems associated with both matchmaking and market making. Firms address the problems of costly communication and costly computation by establishing hub-and-spoke networks of transactions. Fig. 1 represents decentralized and centralized transactions. In a hub-and-spoke network of transactions, transaction costs are potentially lower because buyers and sellers transact through the intermediary firm rather than forming bilateral or multilateral transactions.4 In a decentralized market, individual buyers and sellers handle all communication and computation tasks. Allocations in decentralized markets are characterized by constraints on communication and computation. Costly communication is likely to lead to random search and inefficient matching of buyers and sellers. Costly computation is likely to involve asymmetric information and inefficient allocation mechanisms. Individuals thus encounter transaction costs that limit the efficiency of decentralized exchange.
4 Such hub-and-spoke economies are widely recognized in communications, electric power transmission, and transportation. Hub-and-spoke economies also are well known in marketing, Alderson (1954) (see also Townsend, 1978).
142
D.F. Spulber
In a centralized market, firms provide matchmaking and market making services to buyers and sellers. Firms establish and operate information systems that supply buyers and sellers with some of the means to communicate and process information. Firms engage in communication with buyers and sellers to gather information about their characteristics and to provide information about terms of exchange, such as prices and product features. Firms also engage in computation to improve the efficiency of matchmaking and market-making activities. 2.1 Firms in two-sided markets: matchmaking One of the most important problems faced by buyers and sellers is to find the best possible trading partner. By acting as matchmakers, firms help buyers and sellers find each other. Matchmaking firms offer potential advantages over random search by buyers and sellers. Firms address the problem of costly communication by establishing huband-spoke communications networks (see Fig. 1). Such networks effectively connect all buyers to all sellers while offering advantages over pointto-point networks that link buyers and sellers in decentralized exchange. Point-to-point networks that are complete can be costly to establish because they have so many links while incomplete point-to-point networks lead to imperfect communication. The firm that serves many buyers and sellers creates a network that gives its customers access to many suppliers and its suppliers access to many customers (see Fig. 1). Firms provide various types of centralized communications systems that contribute to buyer and seller search activities. Postal systems involve centralized mechanisms for sorting and delivering mail (see Sidak and Spulber, 1996). The hub-and-spoke network also corresponds to a telecommunications network, including a traditional voice telephone system or computer data transmission system. In a hub-and-spoke network, each buyer is linked to every seller and each seller is linked to every buyer. The hub-and-spoke communications network thus provides the same services as a more costly complete point-to-point network. Communications systems generally involve a physical infrastructure and a system for making connections. A postal system not only involves facilities for sorting and delivering mail but also depends on an established system of addresses. Traditional telecommunications systems transmit voice and data through a system of lines and switches. Traditional telecommunications systems also give each user an address in the form of a telephone number and provide guidance through telephone directories. The Internet provides a transmission mechanism as well as a system of addresses for electronic mail and for web sites. The Uniform Resource Locator (URL) specifies both the protocol needed to retrieve information on a web site and the domain name where the information is located. For basic postal, telecommunications, and data transmission systems, subscribers choose
Ch. 3. Firms and Networks in Two-Sided Markets
143
what other subscribers to contact without the help of the firm that supplies communications services. Some firms provide a very basic form of matchmaking in the form of guides to subscribers of communications services. For example, suppliers of telephone directories improve the information available to subscribers of a telecommunications network. Yellow pages in particular allow buyers to identify sellers of goods and services and obtain their telephone number. Internet users require both the communications capability of the Internet to link to web sites and the computational ability of search engines to locate web sites. Internet auction sites, such as eBay offer both communication between buyers and sellers and computation in the form of automated auctions. Firms offer more complex matchmaking services as well. Some matchmaking firms operate in markets with homogenous products, introducing buyers and sellers. Many other matchmaking firms operate in markets with differentiated products, taking into account not just buyer and seller characteristics but also the features of the products or services to be exchanged. For example, real estate agents are matchmakers in the housing market where every house is a differentiated product. Firms that are brokers match buyers and sellers without involving themselves as a party in the economic transactions between buyers and sellers. Table 1, while not exhaustive, gives some idea of the complexity and range of available matchmaking services. 2.2 Firms in two-sided markets: market making Market-making firms aggregate demand and aggregate supply, offering potential advantages over bilateral transactions. Market-making firms also perform critical price adjustment functions, applying computation to calculate posted prices or to operate auctions. As market makers, firms perform dealer functions by buying from sellers and reselling to buyers. For a partial listing of firms that provide market-making services see Table 1. Firms play a critical role in coordinating transactions between buyers and sellers. Buyers and sellers decide whether or not to trade on organized exchanges for financial assets depending on their expectations of market participation. If there are sufficient numbers of buyers and sellers, the market for financial assets is said to offer liquidity, while in product markets, the corresponding idea is immediacy.5 Buyers participate in markets based on their expectations of seller participation and sellers participate in markets based on their expectations of buyer participation. Market makers provide coordination services by standing ready to buy and sell, assuring buyers and sellers of liquidity if there is not a sufficient number of counter parties available. 5
See Clower and Leijonhufvud (1975).
144
D.F. Spulber
Table 1 Partial listing of types of firms that provide matchmaking and market-making services Firms are intermediaries in two-sided markets Matchmakers Market makers Communications intermediaries
Posted price market makers
Telecommunications firms Data networks Postal systems
Retailers Wholesalers Securities dealers, market makers, and specialist firms Mutual funds Banks and insurance companies Commodities firms (metals, energy, grains)
Basic information providers(infomediaries) Search engines and online portals (Yahoo, Google) Directories and yellow pages Publishers of classified advertisements Comparison shopping web sites
Enhanced service matchmakers
Dating and marriage matchmakers Interest group matchmakers Residential and commercial real estate brokers Employment and recruiting agencies Staffing and temporary agencies
Auction firms Auctioneers (real estate, art, antiques, commodities, agricultural products, industrial equipment) Online auctions (eBay) Organized exchanges for securities, currencies, financial assets, commodities Business-to-business market places
Agents and brokers Representative agents (literary, talent, sports) Travel agents and freight transportation brokers Media (book publishers, journals, broadcasters, movie studios) Payments systems (credit cards, check clearing, money transfers) Securities brokers Business brokers (for buyers and sellers of businesses) Technology and intellectual property brokers Commodity and metals brokers Ship brokers Art brokers Consignment, resale, and antique stores
Rubinstein and Wolinsky (1987) consider a random matching model with buyers, sellers, and intermediaries that act as dealers. In a related setting, Yavas (1994a) examines the role of intermediaries that act as matchmakers in two-sided markets where both buyers and sellers search, considering such intermediaries as employment agencies and real estate brokers. In Yavas (1994a), the presence of the matchmaker affects the endogenously chosen search intensity of buyers and sellers. Gehrig (1993) addresses the coordination problem when there is competition between a market maker and a decentralized search market. Buyers and sellers choose between dealing with the market-making intermediary
Ch. 3. Firms and Networks in Two-Sided Markets
145
and direct exchange in the search market based on the price spread offered by the intermediary and expectations about the participation in the search market. Yavas (1996a) considers competition between a market maker and a search market when buyer and seller search intensities are endogenous; Yavas (1992) lets the intermediary choose between being a market maker and a matchmaker. Spulber (2002a) considers competition between a market maker and an intermediary when buyers and sellers make investments that increase the returns from exchange. Extending Spulber’s (1996b) model of competition between intermediaries in a search market, Rust and Hall (2003) consider competition between multiple dealers and a central market maker. Buyers and sellers choose between searching across dealers for the best price and transacting with the market maker at observable posted prices. Hendershott and Zhang (2006), in a model related to Spulber (1996b), examine competition between upstream firms who can sell directly to consumers or through intermediaries. Ellison, Fudenberg, and Mobius (2003) consider how two auctions can compete without one driving out the other from the market. Competition between market makers is a subject that draws increasing interest. Stahl (1988) considers competition between intermediaries engaged in Bertrand competition for inputs and subsequent Bertrand competition for outputs. Spulber (1999, Chapter 3) examines competition between intermediaries in various settings. Spulber (1999, Chapter 3) introduces a model of a Hotelling-type differentiated duopoly in a two-sided market. Because the intermediary firms compete in a differentiated duopoly, the equilibrium bid–ask spread is positive and depends on the relative transport costs on the buyer side and the seller side of the market. Spulber (1999, Chapter 3) also introduces a model of a two-sided market in which both buyers and sellers face switching costs in changing intermediaries and the equilibrium bid–ask spread is also positive. Gehrig (1993) allows Bertrand competition between intermediaries who also compete with the decentralized search market and shows that competition eliminates the bid–ask spread. Loertscher (2004) introduces capacity constraints on intermediaries leading to rationing and positive bid–ask spreads. Spulber (1996b) obtains price dispersion in both bid and ask prices when intermediaries compete in a search setting. Fingleton (1997) examines a model of competition similar to Stahl (1988) that allows direct trade between buyers and sellers. Shevchenko (2004) examines competition between intermediaries in a search setting when market makers can hold inventories of varying size. Weill (2005) examines how competing market makers provide liquidity in financial markets during financial disruptions. The firm as an intermediary can be observed in a wide variety of industry applications. Yavas (1996b) models the matching of buyers and sellers by real estate brokers and compares alternative commission structures (see also
146
D.F. Spulber
Yavas, 1994b). Baye and Morgan (2001) study Internet intermediaries that act as information gatekeepers. Baye et al. (2004) consider Internet comparison sites that perform match-making functions in two-sided markets and their effects on price dispersion across online retailers. Lucking-Reiley and Spulber (2001) examine business-to-business intermediaries that act as dealers and operate online marketplaces. Some other examples of networks and two-sided markets include information services (Caillaud and Jullien, 2003) and yellow pages (Rysman, 2004). Ju et al. (2004) consider oligopolistic market makers in the natural gas industry. In product markets, retailers perform various types of market-making functions. Retailers include online merchants, automobile dealers, supermarkets, discount stores, department stores, general merchandise stores, specialty apparel stores, warehouse clubs, drug stores, convenience stores, and variety stores (United States Census Bureau, 2000). Retailers provide a wide variety of intermediation services including posting prices, market clearing, marketing, inventory holding, selection of suppliers, quality certification, and management of transactions. Retailers employ information systems in intermediation including web sites for communication with customers and suppliers, and data collection and processing through bar coding of merchandise, point-of-sale scanners, and computerized inventory tracking and reordering. Wholesalers also provide market-making functions. Wholesalers primarily include merchant wholesalers that purchase and resell durable and nondurable goods including distributors, jobbers, drop shippers, import/ export merchants, grain elevators, and farm product assemblers. They also include agents such as brokers, commission merchants, import/export agents and brokers, auction companies, and manufacturers’ agents (United States Census Bureau, 2000). Wholesalers also include direct manufacturer– retailer transactions (retail chain stores, warehouse clubs, discount stores, and home center stores), mail order, catalog sales, manufacturer–industrial user transactions, and retail sales to industrial users. In addition, there are also manufacturer’s sales branches, agents, brokers, and commission merchants. Wholesalers also provide a variety of intermediation services as intermediaries between businesses; they distribute goods, manage inventories, communicate price and product information, certify quality, and provide credit. Wholesalers employ information systems in communicating with retailers and manufacturers and many types of information gathering devices for bar coding, electronic data interchange, product tracking, inventory controls, and distribution. Financial firms that provide intermediation services include banks, securities brokerages, mutual funds and insurance companies (United States Census Bureau, 2000). Financial intermediaries provide many types of services including pricing of some financial assets, providing liquidity, risk allocation, allocation of financial assets over time, combining assets to reduce the transaction costs of diversification, supplying information, and
Ch. 3. Firms and Networks in Two-Sided Markets
147
managing transactions. Depository institutions intermediate between borrowers and lenders, setting rates of interest for loans and deposits, screening borrowers for credit worthiness, and monitoring their repayment performance. Securities and commodity brokers provide a range of intermediation services managing complex financial transactions, carrying out trades on the organized exchanges, and supplying investors with information. Insurance companies manage transactions, allocate risk, and intermediate between investors and buyers of insurance contracts. Organized exchanges for securities, commodities, currencies, and other assets provide market-making services. Some organized exchanges are operated by firms. Although many organized exchanges are associations of members, the incorporation of some exchanges such as the New York Stock Exchange, demonstrates that these markets also are owned and operated by firms. 2.3 Information systems and platforms Firms are intermediaries that create and operate markets. Information systems are part of these firms’ production technology. Firms use information systems to communicate with buyers and sellers and to provide computation services needed to carry out transactions. Although information systems are a critical component of transaction technology, information systems are not the same thing as a firm—they are tools used by firms. Information systems contain two essential components: communication, which refers to the exchange of information between individuals, and computation, which refers to the processing of data that is being exchanged. Information systems generally comprise physical networks involving telecommunications and connected computers.6 Information systems must interact with human intelligence and facilitate interaction within economic and social networks. Information technology is part of the firm’s production function for creating transactions. The costs of communication and computation are part of the transaction costs both in decentralized and centralized markets. When communication is costly, buyers and sellers deal with incomplete networks. When computation is costly, buyers and sellers engage in bilateral transactions rather than more complex multilateral transactions such as those represented by the Core. There is a role for firms to provide communication and computation in the form of matchmaking and market–making services. 6 An industry definition of information system states ‘‘1. A system, whether automated or manual, that comprises people, machines, and/or methods organized to collect, process, transmit, and disseminate data that represent user information. 2. Any telecommunications and/or computer related equipment or interconnected system or subsystems of equipment that is used in the acquisition, storage, manipulation, management, movement, control, display, switching, interchange, transmission, or reception of voice and/or data, and includes software, firmware, and hardware. y 3. The entire infrastructure, organization, personnel, and components for the collection, processing, storage, transmission, display, dissemination, and disposition of information.’’ (see Committee T1A1, 2000).
148
D.F. Spulber
Table 2 presents a classification of market mechanisms that is based on the costs of communication and computation. This classification is useful in understanding the role of the firm in two-sided markets. Markets consist of transactions between buyers, sellers, and firms. Markets require the services of information systems to function effectively. Individuals must communicate to find each other, negotiate the terms of transactions, and monitor performance. Individuals must perform computations to choose between trading partners and to evaluate alternative terms of exchange. Market outcomes are affected by costs of communication and costs of computation. Transaction costs and the capabilities of information systems affect the structure and performance of market mechanisms. The classification scheme in Table 2 connects transaction costs and the properties of information systems to market mechanisms. If communication is costless, buyers and sellers can find each other with greatly reduced search costs. Buyers and sellers may still face problems of asymmetric information and the time costs of search. When computation is costless, buyers and sellers face greatly reduced costs of determining the best assignment of buyers and sellers, market-clearing prices or the allocation of goods. If both communication and computation are costless, then in principle decentralized exchange can achieve efficient outcomes by implementing such market mechanisms as auctions or solutions to cooperative games. When communication or computation is costly, there may be advantages to centralized markets. Firms offer customers various types of matchmaking and market-making services through the use of information systems. Firms that provide telecommunications and data transmission are information intermediaries. Additional, firms extensively use communications systems internally to communicate with their customers and suppliers. Firms employ communications systems to automate transactions with their customers and suppliers as well as automating the myriad internal transactions associated production operations, back office record keeping, and organizational management. Electronic commerce refers to the automation of economic transactions through the use of information systems. Electronic commerce lowers transaction costs by enhancing communications and computation in exchange between consumers, between consumers and firms, and between firms. Electronic commerce substitutes capital for labor services in the production of transactions, potentially displaces costly labor services applied to routine commercial tasks including the time that employees spend communicating with customers and suppliers regarding prices, product availability, ordering, billing, and shipping. Moreover, electronic commerce enhances the productivity of labor services in commercial activities such as sales, distribution, and procurement. Firms improve efficiency by linking external transaction systems with their internal computer systems, thus increasing the frequency, rapidity, and accuracy of communication and
Ch. 3. Firms and Networks in Two-Sided Markets
149
Table 2 Classification of allocation mechanisms depending on the costs of communication and computation Computation
Costless computation
Costly computation
Buyers and sellers in a decentralized market can communicate costlessly and can costlessly implement efficient mechanisms for exchange: Core allocations and Walrasian equilibria
Buyers and sellers in a decentralized market communicate over complete networks (due to costless communication) and engage in bilateral exchange (due to costly computation) Firms offer a centralized market alternative to buyers and sellers. Firms provide computation services by offering advanced matchmaking services to assign buyers to sellers by offering market making services to specify terms of trade and clear markets Buyers and sellers in a decentralized market engage in bilateral exchange due to costly computation and transact on incomplete networks or on random networks due to costly communication Firms offer a centralized market alternative to buyers and sellers. Firms provide separate or bundled communication and computation services to buyers and to sellers through matchmaking and market making services
Communication Costless communication
There is no need for firms to provide communication or computation services
Costly communication
Buyers and sellers in a decentralized market can implement efficient mechanisms (auctions and cooperative game solutions) due to costless computation but they transact on incomplete networks due to costly communication Firms offer a centralized market alternative to buyers and sellers. Firms provide communication services (gathering and distributing information, facilitating information exchange) and basic matchmaking services with buyers and sellers handling their own computation activities.
150
D.F. Spulber
allowing links to production and inventory management systems within each organization (see Lucking-Reiley and Spulber, 2001). A platform is a collection of related technological standards. The term ‘‘platform’’ sometimes is used in the economics literature as shorthand to designate firms that act as intermediaries in electronic commerce. Although firms use information systems that obey common technical standards, a platform is not a firm. The firm’s technology and the applicable technology standards are distinct from the firm itself. The use of technological standards or information system platforms to implement the linkage of buyers and sellers through networks is related to the earlier discussions of compatibility of products in a network (see Farrell and Saloner, 1985, 1986; Katz and Shapiro, 1985, 1986, 1994; Liebowitz and Margolis, 1994, 1999; Liebowitz, 2002). Platforms play an important role in information systems, both in communications and in computing. In computers, a platform is a ‘‘reconfigurable base of compatible components on which users build applications’’ and is identified with ‘‘engineering specifications for compatible hardware and software,’’ Bresnahan and Greenstein (1996) (see also Bresnahan and Greenstein, 1999; Greenstein, 1998). For example, IBM devised standards for the personal computer that were adapted by manufacturers of software designers, internal components, such as memory and microprocessors, and peripheral devices, such as printers and monitors. In turn, Microsoft’s standards for the personal computer operating systems are used by designers of software applications that are compatible with operating systems. In communications networks, platforms permit compatible transmission of information in communications and interconnection of transmission equipment (see Spulber and Yoo, 2005). Platforms in telecommunications include computer hardware and software standards for computer-based switching and transmission systems. Platforms in communications include computer software standards such as the Transmission Control Protocol/ Internet Protocol (TCP/IP) used for Internet communications between computers. A network is said to be modular or to exhibit an open architecture if most suppliers of complementary services can gain access to the network. Collections of technical standards exist in many industries where independent producers supply substitute products that are interchangeable and complementary products that must work together. Thus, cameras and film share technological standards that allow the products to be used together, and there are multiple providers of cameras and of film that follow the technical standards. These standards exist in many high-tech industries such as audio systems, video systems, and mobile phones. Platforms exist in many other types of industries in which compatible components are needed including automobiles, aircraft, and industrial machinery. Platforms exist in electronic commerce, in the form of technical standards for the electronic exchange of data between companies. Innovations in communications and computation as applied to business documents avoids the need to translate computer files into paper documents, thereby increasing
Ch. 3. Firms and Networks in Two-Sided Markets
151
the speed and accuracy of transactions. There is a wide-ranging set of standards for electronic data interchange (EDI) on private networks that predates the Internet. Extensible markup language (XML) provides standards for documents and data transmission over the Internet developed by the World Wide Web Consortium. The advantage of document standardization is ease of communication and computation between businesses, including retailers, wholesalers, manufacturers, and parts suppliers. Electronic commerce further enhances communication of transaction information by allowing buyers and sellers to transact with the firm at remote locations and at different times. Thus, the buyers and sellers in an auction on eBay need not be present at the same location and can participate in the auction at different times. This reduces the transaction costs by avoiding the costs of travel and the costs of holding meetings whether those costs would be borne by the firm or its customers and suppliers. Thus, technological change in information processing and communications result in innovations in transaction methods and changes in the organization of firms. Increasing standardization of economic transactions provides additional economic benefits from centralization of markets. 3
Networks in two-sided markets
Graphs are sets of points and lines that connect some of those points to each other. The points are referred to as nodes and the lines are referred to as links. Networks are graphs in which numerical values are assigned to the links.7 In economic networks studied in this article, nodes in the graph represent economic actors. The links that connect the nodes in a graph represent some important aspect of the relationship between those economic actors. The architecture or configuration of the set of nodes and links provides a representation of the market mechanism. The pattern of links can be used to represent decentralized exchange in which buyers and sellers search for each other or centralized exchange with firms as intermediaries, as shown previously in Fig. 1. This section presents some of the basics of graph theory that will be useful in the discussion. 3.1 Transactions on networks in two-sided markets The theory of networks provides a highly useful framework for understanding how markets work. The theory of networks uses the mathematics 7 For an introduction to graph theory see Bolloba´s (1998), Aldous and Wilson (2000), Tutte (2001), Gross and Yellen (1999, 2004), and Diestel (2000). For a more popular treatment see for example Watts (2003). Networks can represent facilities used by firms to provide communication of information, transmission of energy, and transportation of goods and services. Networks also can represent individuals in society and their relationships. There are also many scientific applications using networks in physics, chemistry, biology, and other areas.
152
D.F. Spulber
of graph theory to represent markets as networks. Important developments in the field of graph theory offer a rich set of useful mathematical tools. These tools can be incorporated in economic models to show the interaction of buyers and sellers. These concepts help our understanding of markets and the role of the firm. They also illuminate the critical function of information systems in economic transactions. I apply network theory to understand the performance of markets with and without intermediation by firms. Networks can represent decentralized allocation mechanisms used by buyers and sellers in decentralized markets. Networks also can represent centralized allocation mechanisms established by firms. Techniques adapted from graph theory can be used to study how firms design efficient allocation mechanisms. The results shed light on the rules, organization and institutions of exchange, referred to as market microstructure, for all types of markets including those for products, financial assets, and labor services. Network theory contributes to the theory of the firm by allowing comparisons between decentralized exchange and markets mediated by firms. First, networks can be used to find those connections between buyers and sellers that yield the greatest gains from trade. Second, networks can be used to find the connections between buyers and sellers that have the lowest transaction costs. Third, networks can be used to determine what connections between buyers and sellers are the most efficient. The most efficient connections are those that generate the greatest benefits from transactions net of the costs of transactions. Firms exist when they can provide the most efficient networks in comparison to decentralized networks. Network theory helps to illustrate the complexity of assignments of buyers and sellers. The discussion thus far showed that buyers and sellers in decentralized exchange face two difficult problems. Buyers and sellers must establish the network communication links that permit the selection of an efficient assignment. This may be costly since carrying out an efficient assignment can require establishing a complete network. Second, buyers and sellers must make the complex computations that are required to select an assignment among the many possible permutations. There is an extensive literature on the strategic formation of networks by buyers and sellers (see Dutta and Jackson, 2003) and the references therein.8 Kranton and Minehart (2001) present a theory of buyer–seller networks that has the following features. Products are homogenous and sellers have identical costs equal to zero. Buyers each purchase a single unit and sellers 8 Myerson (1977) introduces networks to cooperative game theory and obtains a constrained Shapley value that depends on available links between players. Jackson and Wolinsky (1996) consider cooperative games in which the value function depends on the structure of the network. They define an equilibrium in the endogenous formation of networks as occurring when no pair of individuals wants to create a link that is absent or to dissolve a link that is present. They contrast efficiency and stability of networks, although it should be pointed out that their equilibrium notion of stability in link formation differs from the notion of stability in matching models (see also Dutta and Mutuswami, 1997).
Ch. 3. Firms and Networks in Two-Sided Markets
153
each offer a single unit. Buyers differ in their willingness to pay for a unit of the good. Buyers can only transact with sellers with whom they are connected in a network. For those buyers and sellers that are connected, sellers hold ascending-bid auctions with the same going price across all sellers. The price rises until there is a subset of sellers for whom demand equals supply. The auctions of these sellers then clear at the current price. If there are remaining sellers, the price rises until all sellers have sold their goods. Kranton and Minehart (2001) show that the equilibrium allocation of goods is efficient since the highest-value buyers receive the goods, subject to constraints imposed by the pattern of links. They further show that the allocation of goods is pair-wise stable in the sense that the surplus that any buyer and seller could obtain by exchanging their goods does not exceed their joint payoffs. They further demonstrate that efficient link patterns are equilibrium outcomes of the game played by buyers and sellers. The present discussion differs from Kranton and Minehart (2000, 2001) in several critical areas. First, they focus on decentralized markets in which buyers and sellers handle their own communication and computation, so that buyers and sellers can establish their own links to communicate and transact with each other. Kranton and Minehart (2001) refer to their process as a ‘‘frictionless model of network competition’’ because they allow buyers and sellers to hold auctions without incurring transaction costs. Since I am interested in transaction costs and the role of the firm, I assume that buyers and sellers face constraints in communication and computation. In my framework, when buyers and sellers are in a decentralized market, they are limited to bilateral exchange. I require the presence of a firm to establish centralized allocation mechanisms such as auctions. Second, while Kranton and Minehart (2000, 2001) focus on homogenous products, I consider both homogenous and differentiated products which have important consequences for efficiency. Third, while Kranton and Minehart (2000, 2001) assume that sellers have equal costs, I allow sellers to have different costs, which creates some difficulties for allocation mechanisms. I do not assume that buyers and sellers can hold auctions on their own. In my framework, centralized matching and operation of auctions require intermediation by firms. Graph theory has many economic applications. Graphs can be applied to determine how buyers and sellers are matched in two-sided markets. The links represent transactions and the properties of those transactions. The values assigned to the links between two economic actors can represent transaction benefits. The value assigned to the link can show the potential gains from trade of a transaction. Those transactions with nonnegative gains from trade are economically feasible. Thus, the value of the links shows what transactions are feasible. The links between economic actors also can represent transaction costs. One can assign numerical values to the links equal to the potential cost of transacting between the two individuals. The presence or absence of links
154
D.F. Spulber
provides a highly useful if somewhat more dramatic type of transaction costs. If a link is present, transaction costs are zero and if a link is absent, transaction costs are infinite. This gives a picture of feasible transactions. Such links represent communication that allows economic interaction between buyers and sellers.9 Such preconditions for trade also include whether or not the two parties are already acquainted, perhaps they are connected through personal, business, or social ties. They may be part of the same social group or business association.10 The links between buyers and sellers can be established endogenously. Suppose that a link between a buyer i and a seller j can be established at a cost kij then, kij represents the transaction cost required before trade can take place. This is the cost of linking buyers and sellers. If links must precede trade, then kij is the ex ante transaction cost. After links are established, then the ex post transaction costs again are either zero or infinite.11 If k is the cost of a link, the total costs of a point-to-point network kmn are replaced by the lower costs of a hub-and-spoke network, k(n+m). 3.2 Basics of network theory A graph is a pair G ¼ (J, L) consisting of a set of nodes J and a set of links L that connect the nodes.12 For example, in Fig. 2, the set of nodes is J ¼ {1, 2, 3, 4} and the set of links is L ¼ {(1, 2), (3, 4), (1, 4)}. A network is a graph with numerical values assigned to the links. The numerical values of the links can represent such things as gains from trade, transaction costs, and transportation capacity. For example, in Fig. 2 the gains from trade associated with the links (i, j) are represented as aij. The interaction of buyers and sellers can be represented using bipartite graphs. In a bipartite graph, the set of nodes is partitioned into two sets, and the set of links has nodes that are in different sets. Let J ¼ (B, S) represent a partition of the set of nodes into those in the set of buyers and those in the 9 The link between two economic agents can represent a physical communication channel, particularly one provided by telecommunications and Internet systems for transmitting voice and data. Communications networks are an important type of information system and they allow buyers and sellers to establish economic transactions. Links also represent pathways for travel and transportation, travel so that buyers and sellers can visit each other and transportation so that trading partners can send goods and services to each other. 10 Granovettor (1985) emphasizes the importance of strong ties in connecting individuals within social groups and the importance of weak ties in connecting social groups with each other. Weak ties can be more important than strong ties for adding value to economic transactions because they help establish critical connections between many individuals. 11 One might suppose that with full information, buyers and sellers could establish the minimum number of links necessary for efficiency, n*. But, if we require information to be transmitted over links, then full information would require a complete set of links for consumers to communicate that information. Thus, if links are channels needed for communication, full information is not possible without a complete set of links. 12 In the mathematics literature, nodes also are referred to as vertices and links also are referred to as edges.
Ch. 3. Firms and Networks in Two-Sided Markets 1
a12 = 5
155
2
a14 = 7
4 Fig. 2.
a34 = 2
3
A network consisting of a graph G ¼ (J, L) with nodes J ¼ {1, 2, 3, 4}, links L ¼ {(1, 2), (3, 4), (1, 4)}, and numerical values assigned to the links.
set of sellers, B[S ¼ J and B\S ¼ null set. Let L be a set of links such that the nodes for each link are not adjacent, that is, one link is in the buyer partition class and the other link is in the seller partition class. For convenience, we write a link (b, s) to indicate that the first node is in the set of buyers and the second node is in the set of sellers. The set of buyers is B ¼ {1, y, n} and the set of sellers is S ¼ {1, y, m}. Define a neighbor of a node as the other node of a link that is connected to that initial node. Let N(v) be the set of all neighbors of the node v. Thus, N(b) is the set of sellers connected to the buyer b. Also, N(B) is the set of neighbors of all buyers in B. If two nodes are neighbors, they are said to be adjacent. Two links are said to be adjacent if they share a node. Two links are said to be independent if they are not adjacent, that is, if the two links do not share a node. A bipartite graph is complete if all nodes in one partition class are adjacent with all nodes in the other partition class. Thus, if a bipartite graph is complete, each buyer is connected with every seller. This would require nm links if there are n buyers and m sellers. Let LC represent the complete set of links for the set of nodes J ¼ (B, S). Then, the complete graph is GC ¼ ((B, S), LC). In a bipartite graph, an assignment of the set of buyers B associates each buyer with a distinct seller in S. Notice that an assignment in a bipartite graph is defined with respect to one of the partition classes. Such an assignment would involve n links and only exists if there are enough sellers, nrm. An assignment is a set of independent links contained in a graph. A necessary and sufficient condition for a bipartite graph to contain such an assignment is Hall’s (1935) Marriage Theorem (see also Diestel, 2000, p. 31). Marriage Theorem (Hall). Let N(A) be the neighbors of A in the set S for ADB. The bipartite graph G ¼ ((B, S), L) contains an assignment of B if and only if |N(A)|Z|A| for all ADB. The theorem states that the graph contains an assignment if and only if every subset of buyers has enough neighbors in the set of sellers. A corresponding necessary and sufficient condition applies to the set of sellers. Clearly, there
156
D.F. Spulber Buyers
Sellers
1
1
2
2
3
3
4
4
Fig. 3.
A bipartite graph that does not contain an assignment.
is a one-to-one matching of buyers and sellers when|B| ¼ |S|, that is, n ¼ m. Fig. 3 shows a bipartite graph that does not contain an assignment. Hall’s Marriage Theorem gives conditions under which, if each buyer is connected to a subset of sellers, each buyer will be able to be matched with a seller with whom that buyer is connected. The condition also can be stated in terms of sellers being connected to buyers. Informally, the theorem considers the number of connections needed for transactions to take place.13 The theorem is useful for assigning buyers to sellers or sellers to buyers. Hall’s Marriage Theorem addresses the case in which all buyers are identical and all sellers are identical. The only issue is the pattern of links that connect them. Kranton and Minehart (2000, 2001) examine an important application of Hall’s Marriage Theorem. In their setting, buyers are no longer identical but instead have different valuations of a good. All of the sellers are identical because they have the same cost of supplying the good to a buyer. Buyers then are assigned to the set of sellers. Their analysis focuses on making sure that members of the set of sellers are allocated to the highest-value buyers. In the next section, I consider a more general case that allows there to be both diverse buyers and diverse sellers. 4
Assignments of buyers and sellers in a network: costly communication
Network theory provides insights into how costly communication affects economic efficiency. Costly communication takes the form of incomplete networks. This section assumes that there is costless computation so that the allocation mechanism performs efficiently. The outcome is inefficient due to communication constraints in the form of missing links between buyers and sellers. When there are missing links, it may not be possible to achieve efficient assignments of buyers to sellers. Consider a market with consumers divided into two distinct groups, a set of buyers B ¼ {1, y, n} and a set of sellers S ¼ {1, y, m}. Let i denote an 13 The traditional story of the Marriage Theorem discusses marriage to someone with whom the woman (or man) is acquainted, depending on the direction of the assignment.
Ch. 3. Firms and Networks in Two-Sided Markets
157
element of the set of buyers and let j denote an element of the set of sellers. To give some idea of the complexity of the problem, consider the number of possible assignments. When these are n buyers and n sellers, there are n! possible assignments. Thus, with 10 buyers and 10 sellers there are over 3.6 million solutions. Moreover, the efficient assignment need not be unique. Choosing an efficient assignment depends on both gains from trade and the costs of trade. In an economy without transaction costs, buyers and sellers will choose only the best transactions, that is, those transactions that maximize gains from trade. When there are transaction costs, the set of efficient transactions changes. If a buyer–seller pair (i, j) is in the set of links L, then buyer i and seller j can transact. Thus, if (i, j) is in the set of links L then the transaction costs for buyer i and seller j equal zero. Conversely, if a buyer–seller pair (i, j) is not in the set of links L, the buyer i and seller j cannot transact. This means that if (i, j) is not in the set of links L, then the transaction costs for buyer i and seller j are infinite. This section applies network theory to determine the set of efficient transactions when there are both transaction benefits and transaction costs. 4.1 Assignments with homogenous products Every buyer wishes to purchase a unit of a homogenous good and every seller can supply a unit of the good. Buyers have valuations {v1, y, vn}, that are arranged in decreasing order, v1>v2>y>vn. Sellers have costs {c1, y, cm} that are arranged in increasing order, c1oc2oyocm. For any link between a node i in the set of buyers and a node j in the set of sellers, we can assign a value equal to vij ¼ max f0; vj cj g.
(1)
This represents the total of gains from trade that buyer i and seller j would obtain from a transaction. This is also referred to as the value of a match. See Fig. 4 for a network in which all gains from trade are positive. An assignment of buyers to sellers matches every buyer i in B to a distinct seller j in S. There is no loss of generality in framing the discussion in terms of assigning buyers to sellers. If the number of buyers exceeds the number Buyers
Sellers v1 – c1
1
1
v1 – c2 v2 – c1 2
v2 – c2
2
Fig. 4. A network with values assigned to the links representing the potential gains from trade that would result from a transaction between pairs of buyers and sellers.
158
D.F. Spulber
of sellers, then one can create dummy sellers with high costs such that there would be no trade between a buyer and such a seller. The benefits of an assignment equal the sum of the gains from trade for any buyer–seller pair in the assignment that has positive gains from trade vi–cj. For any bipartite graph with a set of buyers and a set of sellers, there can be many possible assignments of buyers to sellers. An efficient assignment of B to S is an assignment that maximizes total net benefits over the set of assignments of B to S. Thus, efficiency is defined strictly in terms of gains from trade. When transaction costs are introduced, we will define a constrained efficient assignment as one that maximizes gains from trade subject to feasibility. A feasible assignment of B to S for a graph G is an assignment such that each buyer–seller pair (i, j) in the assignment is in the graph. Thus, a feasible assignment is a subset of the set of links in the graph G such that not all buyers need to be assigned to a seller but each buyer that is assigned is matched to a distinct seller. A constrained efficient assignment is defined to be a feasible assignment that maximizes total net benefits over the set of feasible assignments from B to S. Consider the set of efficient assignments. Let n be the largest integer i such that vi cj where i ¼ j, iAB and jAS. The number n is the market-clearing quantity. Let B ¼ {1, y, n} and S ¼ {1, y, n}. The sets B and S describe the buyers and the sellers that are on the left-hand side of the supply and demand diagram. The excluded buyers i>n and the excluded sellers j>n are those on the right-hand side of the supply and demand diagram (see Fig. 5).14 This outcome corresponds to Bo¨hm-Bawerk’s (1891) method of marginal pairs. Let E be an assignment of B to S. In other words, E is a set of distinct pairs that matches every buyer in B with a different seller in S. The sets B and S have the same number of elements by construction. Any such assignment E is an efficient assignment of B to S. The proof of this result is immediate. For any such E, number the distinct pairs k ¼ 1, y, n. Then, n n X X X ðvk ck Þ ¼ vi cj . k2E
i¼1
(2)
j¼1
The right-hand side equals the maximum net benefits that can be obtained from matching buyers in B and sellers in S. This brings us to our first result. In a market with homogenous products, given any set of buyers B and sellers S, and the subset of buyers B and the 14 We restrict attention to bilateral trades between a buyer and a seller. We thus rule out side payments between more than two parties.
159
Ch. 3. Firms and Networks in Two-Sided Markets Supply
v, c B*
Demand
S*
n* Fig. 5.
m
n
i, j
The set of efficient transactions that maximizes gains from trade.
subset of sellers S as defined previously, an assignment of buyers B to sellers S is efficient if and only if it contains an assignment from B to S. The result holds for the following reason. If an assignment from B to S contains an assignment from B to S then those pairs of buyers and sellers outside of B and S will not trade. In an efficient assignment, the buyer– seller pairs that trade are those on the left-hand side of the supplyand-demand diagram while the buyers and sellers that do not trade are those on the right-hand side of the supply-and-demand diagram. Can buyers and sellers attain an efficient outcome through direct exchange? To answer this question, it is necessary to make explicit what types of transactions are involved in direct exchange. Suppose that there is full information so that the identity and values of buyers and the identity and costs of sellers are common knowledge. Suppose further that any transaction must be a bilateral exchange, which rules out coalitions of more than two players. Suppose further that any pair of players resolves their negotiation by the same bargaining procedure, such as the Nash Bargaining Solution, so that a buyer and a seller split the gains from trade. Given this framework, the highest-value buyer will trade with the lowest-cost seller. There will be at least one trade if v1Zc1. This pairing continues until viocj is reached at which point trade ceases. This approach yields an efficient outcome since it matches the buyers in B with the sellers in S and all other buyers and sellers are inactive. This allows a characterization of the efficiency of any graph that connects the set of buyers and sellers. Hall’s Marriage Theorem can be applied to this task. Let G ¼ ((B, S), L) be any bipartite graph. Define N(i) as the neighbors of iAB that are contained in the restricted set of sellers S. Then, the
160
D.F. Spulber
Marriage Theorem implies that G contains an assignment of B with S if and only if|N(A)|Z|A|for all ADB. I now introduce a useful benchmark to determine how many links are needed to assure that there will be an efficient assignment. All pairs are said to be viable with homogenous products if the lowest-value buyer has a value greater than or equal to the cost of the highest-cost seller, vnZcm. Complete Viability Condition. All pairs are viable and the number of buyers equals the number of sellers. The condition states that all buyer–seller matches are viable, vnZcm and there is no need for rationing, n ¼ m. Later the condition will also be applied to markets with differentiated products. Recall that a graph G ¼ ((B, S), L) contains an efficient assignment if and only if G contains an assignment of B to S. The condition guarantees that the demand curve is everywhere above the supply curve. If the condition holds, then B ¼ B and S ¼ S. The following result shows the connection between the viability condition and efficiency. Proposition 1. Let products be homogenous. Given the Complete Viability Condition, any assignment of B to S is efficient. If the Complete Viability Condition does not hold, then the conditions of Hall’s Marriage Theorem on the graph G are not sufficient to guarantee that there exists an efficient assignment of buyers to sellers. Given the Complete Viability Condition, all assignments contained by the graph G ¼ ((B, S), L) will be efficient. However, if this condition does not hold, then assignments need not be efficient. To illustrate this result, suppose first that the number of buyers and sellers is equal but that demand and supply cross. For example, let there be three buyers and three sellers, with v1>v2>v3 and c1oc2oc3. Assume that v2>c2 and v3oc3. The bipartite graph in Fig. 6 does not contain an efficient assignment. Buyer 2 in the set B ¼ {1, 2} does not have a neighbor in the set S ¼ {1, 2}. Next, consider a graph G ¼ ((B, S), L) in which the number of sellers is greater than the number of buyers, m>n. Suppose that vnZcn. This case Buyers
Fig. 6.
Sellers
1
1
2
2
3
3
Given v1>v2>v3 and c1oc2oc3, and assuming that v2>c2 and v3oc3, the graph contains an assignment but does not contain an efficient assignment.
Ch. 3. Firms and Networks in Two-Sided Markets Buyers
161
Sellers
1
1
2
2
3
3 4
Fig. 7.
The graph contains assignments but does not contain an efficient assignment.
corresponds to the situation in which the market demand function is greater than the market supply function. Then, B ¼ B and SCS so that not every assignment in the graph G is an efficient assignment. Suppose for example that there are three buyers and four sellers. Then, B ¼ {1, 2, 3} and S ¼ {1, 2, 3}. The bipartite graph in Fig. 7 contains assignments but does not contain an efficient assignment. The reason is that buyer 3 is not matched with any seller in the set of efficient sellers S but only with inefficient seller 4. Therefore, a network can fail to be efficient for two reasons. The network may contain assignments, but unless all assignments are efficient, the network may not contain the efficient ones. Secondly, even if all possible assignments are efficient the network may not contain any assignments, that is, the conditions of Hall’s Marriage Theorem may not be satisfied. Without knowing whether or not the Complete Viability Condition is satisfied, guaranteeing efficiency requires the network to have all possible links. Proposition 2. Let products be homogenous. Only a complete bipartite graph, that is one containing all possible links between buyers and sellers, is sufficient to guarantee a priori the existence of an efficient assignment. The proposition imposes a very strong condition. It suggests that information networks must offer a complete set of connections between individuals to guarantee that the set of efficient transactions is attainable. Of course, it may be costly to establish and operate an information network that offers a complete set of connections, from anyone to anyone. There is a tradeoff between the benefits and costs of a complete set of connections. 4.2 Assignments with differentiated products It is even more difficult to assign buyers to sellers (or sellers to buyers) when products are differentiated. With differentiated products, such as a housing market or a more realistic marriage market, buyers differ from each other and sellers differ from each other. The number of efficient
162
D.F. Spulber
assignments is likely to narrow, possibly to only one assignment if such an assignment exists. This means that the need for complete networks is even greater with differentiated products than it is with homogenous products. Consider a market with a set of buyers B ¼ {1, y, n} and a set of sellers S ¼ {1, y, m}. The sellers each offer an indivisible unit of a good or service. The products offered by the sellers are differentiated goods. Also, buyers have different preferences. Thus, the ith buyer derives value from the jth good equal to aij, i ¼ 1, y, n, j ¼ 1, y, m. Sellers have costs cj, j ¼ 1, y, m. The total of gains from trade that buyer i and seller j would obtain from a transaction equals vij ¼ maxf0; aij cj g.
(3)
As before, vij represents the value of a match. All information is contained in the value of the match because products are differentiated. The assignment problem with differentiated products is discussed in Shapley and Shubik (1972) and Shubik (1984, Chapter 8). An efficient assignment is a set of matches that maximizes total surplus. A buyer or a seller can be matched with at most one counterpart. Buyers and sellers also can be viewed as partners in a productive enterprise. Buyers and sellers also can represent firms and workers in a market where each firm is matched with a worker, and where firms have different technologies and workers have different skills. Consider the general assignment problem for either homogenous products or differentiated products, where vij is defined either by (1) for homogenous products of by (3) for differentiated products. Let the graph G be complete. An efficient assignment considers the following linear programming problem, ! XX VðB; SÞ ¼ maxxij (4) vij xij i2B j2S
subject to
X
xij 1;
8 j2S
i2B
X
xij 1;
8 i 2 B and
j2S
xij 0;
8 i 2 B; j 2 S.
This problem is presented in Shapley and Shubik (1972) and Shubik (1984, Chapter 8) (see also Roth and Sotomayor, 1990). There exists a solution to the maximization problem xij that involves only P values of xij equal to either one or zero, see Dantzig (1963, p. 318). Thus, ij vij xij is greater than or P equal to ij vij xij for all xij satisfying the constraints in problem (4). The
163
Ch. 3. Firms and Networks in Two-Sided Markets
P solution allows for someP buyers to be unassigned, j xij ¼ 0; or for some sellers to be unassigned, i xij ¼ 0: Only a mixed group of buyers and sellers can create value, V(A) ¼ 0 if ADB or ADS. The value of any mixed coalition is an assignment of buyers and sellers that maximizes the benefits of the coalition, XX VðAÞ ¼ vij xij ; A B [ S, (5) i
j
where xij solves the linear programming maximization problem (4) for i and j restricted to the set ACB[S. Clearly, the value of the grand coalition consisting of all buyers and sellers equals the value of the set of efficient matches that solve the linear programming problem (4). Note that V ðfi; jgÞ ¼ vij for i in B and j in S. Buyers and sellers are complements in determining the value of a coalition, VðA þ i þ jÞ VðA þ iÞ V ðA þ jÞ V ðAÞ; where iAB and jAS, ACB[S, and ieA, jeA (see Shapley, 1962). The value of a coalition is non-decreasing when adding a buyer or a seller, V(A+i)ZV(A) and V(A+j)ZV(A). The incremental value of a buyer or a seller is non-decreasing in the addition of a player of the opposite type. The additional buyer and seller pair adds at least vijZ0 and possibly more because the entire set of matches for the coalition can be reshuffled and improved. To illustrate the assignment problem with differentiated products, consider the classic example of a housing market due to Shapley and Shubik (1972). There are three sellers each with a house on the market with sellers’ costs are equal to c1 ¼ 18, c2 ¼ 15, and c3 ¼ 19. There are three buyers whose valuations aij are shown in Table 3, where rows refer to buyers and columns refer to sellers. Buyer values net of seller costs are shown in Table 4. The efficient assignment is shown in bold and is given by (1, 3), (2, 1), and (3, 2), which yields a total value of 16. To illustrate the effects of incomplete networks, I establish a benchmark for markets with differentiated products. Suppose that each buyer i has a productivity parameter zi and each seller j has a productivity parameter yj. Suppose that the productivity parameters are in decreasing order for buyers Table 3 Buyer valuations in the Shapley and Shubik housing market
Buyer 1 Buyer 2 Buyer 3
Seller 1
Seller 2
Seller 3
23 26 20
22 24 21
21 22 17
164
D.F. Spulber
Table 4 Buyer valuations net of seller costs in the Shapley and Shubik housing market
Buyer 1 Buyer 2 Buyer 3
Seller 1
Seller 2
Seller 3
5 8 2
7 9 6
2 3 0
and for sellers, z1>z2>?>zn and y1>y2>?>yn. The benefits from a match are given by an additive value function, aij ¼ aðzi ; yj Þ ¼ zi þ yj . Given this additive value function, all pairs are viable with differentiated products if any pair that contains the lowest-value buyer covers the seller’s cost, zn+yjZcj for all j ¼ 1, y, n. Proposition 3. Let products be differentiated and suppose that the value function is additive. If the Complete Viability Condition holds, then any assignment of B to S is efficient. If the Complete Viability Condition does not hold, then the conditions of Hall’s Marriage Theorem on the graph G are not sufficient to guarantee that there exists an efficient assignment of buyers to sellers. Any assignment E is efficient because additivity implies that n n X X X X vij ¼ ðzi þ yj cj Þ ¼ zi þ ðyj cj Þ ði;jÞ2E
ði;jÞ2E
i¼1
(6)
j¼1
The right-hand side equals the maximum net benefits that can be obtained from matching buyers in B and sellers in S. Given the Complete Viability Condition, all assignments contained by the graph G ¼ ((B, S), L) will be efficient. The statement that without the Complete Viability Condition, Hall’s Marriage Theorem is not sufficient to guarantee that there exists an efficient assignment of buyers to sellers follows from similar arguments made for homogenous products. Without knowing whether or not the Complete Viability Condition is satisfied, guaranteeing efficiency requires the network to have all possible links. Proposition 4. Let products be differentiated. Only a complete bipartite graph, that is one containing all possible links between buyers and sellers, is sufficient to guarantee a priori the existence of an efficient assignment. The marriage market is another example of the assignment problem with differentiated products (see Shapley and Shubik, 1972). A useful special case of a marriage market is given by Gary Becker (1973) in which each buyer i
Ch. 3. Firms and Networks in Two-Sided Markets
165
has a productivity parameter zi and each seller j has a productivity parameter yj. Again, suppose that the productivity parameters are in decreasing order for buyers and for sellers, z1>z2>?>zn and y1>y2>?>yn. Also, suppose that all sellers have zero costs. The value of a match is given by a multiplicative value function, aij ¼ aðzi ; yi Þ ¼ zi yi . The value function exhibits complementarity in productivity parameters z and y, azy(z, y)>0. The marginal product of a player’s type increases in the partner’s type. As Becker (1973) observes, an efficient assignment requires matched partners to be identical. Therefore, in Becker’s marriage market, the efficient assignment exists and is unique, and is given by (1, 1), (2, 2), y, (n, n). The aggregate value is the sum of the values generated by matching identical types. A graph G does not contain an efficient assignment unless it contains all of the diagonal pairs (1, 1), (2, 2), y, (n, n). Recall that this is only one assignment out of n! possible assignments. When assignments are constrained, it is necessary to reconsider all of the matches in the assignment. We now examine how to choose a constrained efficient assignment. 4.3 Second-best assignments To represent costly communication, consider how the choice of efficient assignments is constrained by the possible absence of links between some buyer–seller pairs. Define a second-best assignment as an assignment that maximizes benefits subject to the constraint that the network is not complete. Given a bipartite graph G ¼ ((B, S), L), let dij ¼ 1 if the link (i, j) is in G and dij ¼ 0 otherwise. Then, define the weighted value of a match hij by hij ¼ maxf0; dij vij g.
(7)
This applies whether products are homogenous and vij ¼ max{0, vicj}, and it applies when products are differentiated and vij ¼ max{0, aij–cj}. Thus, hij>0 if buyer i and seller j have positive gains from trade and if there is a link between buyer i and seller j. There exists a solution to the maximization problem xij ¼ xP ij (G) that involves only values of xijPequal to either one or zero. Thus, ij hij xij is greater than or equal to ij hij xij for all xij satisfying the constraints in problem (4). Again, the solution allows for some buyers P P to be unassigned, j xij ¼ 0; or for some sellers to be unassigned, i xij ¼ 0: The solution xij ¼ xij(G) is efficient if and only if G contains an efficient assignment. The value function XX xij ðGÞ (8) V L ðB; SÞ ¼ i2B j2S
166
D.F. Spulber
gives total benefits given that L is the set of links. Since the assignment is second best, the following result holds. The total benefit is non-decreasing in the set of links, VL(B, S) rVK(B, S) for LDK. 5
Networks and the Core in a two-sided market
The Core in a two-sided market is the ideal case, because the solution concept implicitly assumes that there are no costs of communication or computation. Players in a cooperative game can engage in the communication and computation that are necessary to examine the value of all possible coalitions and to implement an allocation of value in the Core. Costless communication in a cooperative game also means that all coalitions of buyers and sellers are feasible. Myerson (1977), in contrast, examines the value in cooperative games on networks, where some coalitions are ruled out by the absence of links. 5.1 The Core with homogenous products The Core of the market game with homogenous products corresponds to a market allocation with a uniform price (see Shubik, 1984, p. 228) and Shapley and Shubik (1972). Even if the efficient assignment were unique, there are at least min{n, m} degrees of freedom in the Core, although the dimensionality of the Core cannot exceed min{n, m} since the buyers’ payoffs determine those of the sellers (see Shubik, 1984, p. 199). The Core allocation yields a surplus vi–p for active buyers, a surplus of p– ci for active sellers, and zero for inactive buyers and sellers. The market price lies between the value of the marginal buyer and the cost of the marginal seller. The market price is constrained above by the cost of the highest-cost seller who does not trade if cn+1ovn. The market price is constrained below by the value of the highest-value buyer who does not trade if vn+1>cn. Thus, the market price is in the interval maxfcn ; vnþ1 g p minfvn ; cnþ1 g. This range of prices determines the set of core allocations. Each allocation in the Core corresponds to an allocation determined by one of the prices in this range.15 Because the Core corresponds to an allocation based on a common price, it effectively requires a central market mechanism. The equivalence of market outcomes and outcomes in the Core illustrates the relationship between allocation mechanisms in cooperative games and market allocation mechanisms. 15
For a proof of core equivalence in the assignment game, see Shapley and Shubik (1972).
Ch. 3. Firms and Networks in Two-Sided Markets
167
5.2 The Core with differentiated products Consider next the cooperative assignment game with differentiated products. The discussion in this section is from Shapley and Shubik (1972) and Shubik (1982). Define the characteristic function of the cooperative game, v(A) ¼ V(A), where the value function is given by the linear programming problem (4). The Core of the assignment game with differentiated products consists of a set of payoffs for buyers uiZ0, i ¼ 1, y, n and a set of payoffs for sellers rjZ0, j ¼ 1, y, m such that the following two conditions hold: X X ui þ rj ¼ vðB [ SÞ. (9) i2B
X
j2S
ui þ
i2A\B
X
rj vðAÞ.
(10)
j2A\S
Imputations in the Core correspond exactly to the (nonempty) set of solutions to the dual of the linear programming problem for the assignment games (see Shapley and Shubik, 1972). To find the dual of the linear programming problem (4), find the pair of payoff vectors (u, r) that solve the following problem: X X minu;r ui þ rj (11) i2B
j2S
subject to ui 0; rj 0; and ui þ rj vij ; for all i 2 B and j 2 S. The dual problem has a solution because the primal problem has a solution and the objective functions of the two problems reach the same value, Dantzig (1963, p. 129). Thus, we obtain a solution (u, r) that satisfies the following condition: X X XX ui þ rj ¼ vij xij ¼ vðB [ SÞ. (12) i2B
i2S
i
j
Note that from the constraint of the dual problem, ui þ rj vij , so that no buyer–seller coalition can improve on the imputation, which establishes that it is a core allocation. Any core allocation (u, r) is associated with a vector of prices. Consider a buyer–seller pair for a given core allocation. Recall the definition vij ¼ max{0, aij–cj}. If aijZcj, then buyer i and seller j transact at price pij where the price is determined as follows, pij ¼ cj þ rj ¼ aij ui ;
i ¼ 1; . . . ; n
j ¼ 1; . . . ; m.
(13)
168
D.F. Spulber
Otherwise, the buyer–seller pair receives zero payoffs and no transaction occurs between them. Product differentiation implies that prices may differ for each transaction. This contrasts with the matching of buyers and sellers when products are homogenous, which can be achieved with a uniform market-clearing price. Notice that all payments are made within buyer–seller pairs without the need to make side payments to other buyers and sellers. Just as in the assignment problem, the simplicity of the dual linear programming problem hides the complexity of determining the set of core allocations. The complexity of the assignment problem strongly suggests that an efficient decentralized solution is not available for buyers and sellers. First, the assignment problem presents many challenges that are likely to entail substantial transaction costs in practice. Solving the assignment problem requires information about buyer values and seller costs. Then, it is necessary to assign buyers to sellers by choosing from the set of n! possible assignments or equivalently solving the linear programming problem. Second, buyers and sellers must determine transfer payments that reflect the value of coalitions and alternative assignments.
6 Stable assignments in a decentralized two-sided market: costly computation Consider a decentralized two-sided market. To represent the situation in which buyers and sellers face costly computation, consider a game in which buyers and sellers are restricted to bilateral exchange with no money transfers. The equilibrium solution for such a game is a stable assignment defined by Gale and Shapley (1962). The Gale–Shapley assignment game corresponds to a decentralized market since the stable assignment does not require any central coordination, it can be reached with only bilateral offers and acceptances or rejections. Gale and Shapley (1962) consider a two-sided market in which equal numbers of buyers and of sellers have ordinal preferences over matches. An assignment of buyers to sellers is stable if it does not contain two buyers i1 and i2 who are assigned to sellers j1 and j2, respectively, such that buyer i1 prefers seller j2 to seller j1 and buyer i2 prefers seller j1 to seller j2. Gale and Shapley (1962) show that a stable assignment always exists and that it can be found with a simple algorithm. Sellers can break off a match if they get a better offer. Any buyer that does not have a match, proposes a match to a seller that ranks highest among all sellers that have not yet rejected him. If the seller is not matched, the seller accepts the offer. If the seller is matched, the seller can either accept or reject the buyer’s offer depending on whether or not it is preferred to the seller’s existing match. The buyers can be chosen in any order. The algorithm continues until all buyers are matched with a seller.
Ch. 3. Firms and Networks in Two-Sided Markets
169
The Gale–Shapley assignment game contrasts with that of Shapley and Shubik (1972), who allow for money transfers and who focus on core allocations. As already noted the Core allocation corresponds to a game in which there is costless communication and computation. Recall that in the Core, there need only be money transfers within buyer–seller pairs. In this section, I obtain some interesting insights by combining elements of these two fundamental studies. I allow for the possibility of money transfers. I then consider stable assignments in the two-sided market game. The stable assignment with homogenous products is shown to be an efficient assignment. The stable assignment for a market with differentiated products is analogous to Bo¨hm-Bawerk’s method of marginal pairs for a market with homogenous goods. The outcome is not necessarily in the Core, although I give sufficient conditions under which a stable assignment is in the Core. 6.1 Stable assignments with homogenous products In contrast to Gale and Shapley (1962), consider the assignment game with the possibility of money transfers within any buyer–seller pair. Once buyers and sellers make a match they lose the outside option of other matches. Suppose that buyers and sellers engage in generalized Nash bargaining. Let a represent the buyer’s share of the surplus, so that 1–a is the seller’s share of the surplus, where 0oao1. The generalized Nash bargaining solution specifies a consistent proportional division of surplus across buyer–seller pairs. A match yields monetary payoffs for the two parties i and j equal to avij for buyer i and (1–a)vij for seller j. Thus, buyers and sellers have cardinal as well as ordinal preferences rankings across matches. The values of matches in the assignment game with money transfers treat buyers and sellers symmetrically as Shapley and Shubik (1972) observed. Proposition 5. Given money transfers within each buyer–seller pair and generalized Nash bargaining, there is a unique stable assignment. We know from Gale and Shapley (1962) that with ordinal preferences a stable assignment always exists. With money transfers and generalized Nash bargaining, there can be only one stable assignment.16 Consider the market with homogenous products. As before, let buyer valuations be arranged in decreasing order, v1>v2>y>vn and let seller 16 A B-stable assignment is defined as an assignment that is optimal for the buyer side of the market, that is, all members of the buyer side of the market are as well off as they would be in any other stable assignment. With ordinal preferences, if there is a B-stable assignment, then the B-stable assignment is unique. An S-stable assignment is defined as an assignment that is optimal for the seller side of the market, that is, all members of the seller side of the market are as well off as they would be in any other stable assignment. With ordinal preferences, if there is a S-stable assignment, then the S-stable assignment is unique (see Bolloba´s, 1998).
170
D.F. Spulber
costs be arranged in increasing order, c1oc2oyocm. The best buyer and the best seller wish to trade with each other to obtain the greatest gains from trade. The highest-value buyer will match with the lowest-cost seller, the next-highest-value buyer will match with the next-lowest cost seller, and so on, until there are no more subscriber pairs with non-negative surplus. Therefore, a stable matching takes the form of Bo¨hm-Bawerk’s method of marginal pairs. This implies the following result. Proposition 6. In the market with homogenous products, the unique stable assignment of buyers and sellers is an efficient assignment. Total net benefits of buyers and sellers therefore are maximized by decentralized exchange between buyers and sellers under full information. Even though the stable assignment is efficient, the bilateral allocation of surplus between buyers and sellers does not correspond to a core allocation. In a market mechanism as described by the Walrasian auctioneer or by Bo¨hm-Bawerk’s method of marginal pairs in which the marginal trade sets the price, the law of one price holds.17 However, when buyers and sellers match themselves, every buyer–seller pair (i, j) that completes a trade divides the surplus proportionally and thus trades at a potentially distinct price. Since buyers and sellers split the surplus according to the generalized Nash Bargaining Solution, the exchange price equals pij ¼ ð1 aÞvi þ acj .
(14)
With decentralized exchange, the law of one price does not hold and the allocation of surplus does not correspond to a Walrasian allocation. Although the stable assignment with decentralized matching corresponds exactly to the set of pairs in the Core, the allocation with decentralized matching generally is not a Core allocation because prices will vary across matched pairs. The efficiency of the stable assignment with homogenous products depends on costless communication. Suppose instead that communication is costly, so that the bipartite graph linking buyers and sellers is incomplete. Then, there exist stable assignments on the incomplete graph. Such stable 17 The theory of multiperson cooperative games sometimes portrays equilibrium solutions as decentralized mechanisms. For example, Shubik (1982, p. 127) considers the solutions to games as ‘‘the end results of rational, motivated activity by the players.’’ However, solutions to multiperson cooperative games, such as the Core, also resemble centralized allocation mechanisms. Much coordination goes into finding core allocations. Determining the solution to a game, even under full information, requires an evaluation of the value of all possible coalitions and a calculation of the set of feasible payoff vectors that cannot be improved on by any coalition. The imputations then must be provided to individual players. The complexity of these procedures represents centralized mechanisms rather than multilateral negotiations. The equivalence of core allocations and market outcomes suggests that there exists a fundamental connection between mechanisms needed to discover and implement a market equilibrium and a solution to a cooperative game. It should be emphasized that both the neoclassical market equilibria and the solutions to cooperative games are attained without transaction costs.
Ch. 3. Firms and Networks in Two-Sided Markets Sellers
Buyers v1
1
2 Fig. 8.
171
v1
c2
v2
c1
c1
1
2
The stable assignment on the incomplete network is (1, 1).
assignments need not be efficient even subject to the constraint imposed by the available links.18 Suppose for example that there are two buyers and two sellers. Suppose that buyer 1 is linked to both sellers and suppose that buyer 2 is linked only to seller 1 (see Fig. 8). Then, since v1>v2 and c1oc2, the only stable assignment is for buyer 1 to trade with seller 1. Given generalized Nash bargaining, buyer 1 prefers seller 1 to seller 2 and seller 1 prefers buyer 1 to buyer 2. To follow the Gale–Shapley algorithm, buyer 1 makes an offer to seller 1 that is accepted. Buyer 2 makes an offer to seller 1 that is declined, and the process is complete. The assignment yields value equal to v1–c1. The three other feasible assignments yield v1–c2, v2–c1, and (v1–c2)+(v2–c1). The stable assignment (1, 1) dominates the assignment (1, 2) and the assignment (2, 1). It also dominates the assignment composed of both matches (1, 2) and (2, 1) if v2oc2. However, if v2>c2, the stable assignment is not efficient since v1–c1o(v1–c2)+(v2–c1). This situation corresponds to the Complete Viability Condition. Thus, when the Complete Viability Condition holds, a stable assignment on an incomplete network need not be efficient on that network. It is important to observe that the simple bipartite graph in Fig. 8 satisfies Hall’s Marriage Theorem and therefore contains an assignment. Proposition 7. Consider a market with homogenous products and suppose that the Complete Viability Condition holds. Let the graph G satisfy the conditions of Hall’s Marriage Theorem. Then, the stable assignment on the graph G need not be efficient on that network. Any assignment on the graph is efficient by the Complete Viability Condition, which guarantees that all assignments are efficient. The incompleteness of the graph means that the stable assignment need not be an assignment since it does not necessarily map all buyers into the set of all 18 This differs from Kranton and Minehart’s (2001) result in their auction framework since they have identical sellers.
172
D.F. Spulber
sellers, as in the previous example. Accordingly, the stable assignment need not be efficient, which establishes the result in Proposition 7. 6.2 Stable assignments with differentiated products Consider now the market in which the products offered by the sellers are differentiated goods and buyers have different preferences. Again, suppose that buyers and sellers find their own matches and engage in bilateral exchange. Also, once they are matched buyers and sellers lose the outside option of other matches. As before, the ith buyer derives value from the jth good equal to aij, i ¼ 1, y, n, j ¼ 1, y, m. Sellers have costs cj, j ¼ 1, y, m. The value of a match is vij ¼ max{0, aij –cj}. Assume that buyers and sellers engage in generalized Nash bargaining so that the buyer obtains surplus avij and the seller obtains surplus (1–a)vij. The main insight of the discussion is that a stable assignment with differentiated products can be obtained in a manner that is analogous to Bo¨hm-Bawerk’s method of marginal pairs for the homogenous products case. This corresponds to a sequential process with the buyer–seller pair that has the highest value being matched first, the buyer–seller pair with the next-highest value matched next and so on. However, in contrast to the case of homogenous products, matching by buyers and sellers need not yield an efficient assignment even on a complete network. Proposition 8. In a market with differentiated products, a stable assignment need not be efficient. To see why, consider again the housing market example due to Shapley and Shubik (1972). Recall that the efficient assignment equals (1, 3), (2, 1), and (3, 2), which yields a total value of 16. This is shown in bold in Table 5. The stable assignment, which results from decentralized matching by buyers and sellers, consists of the two matches (2, 2) and (1, 1), with buyer 3 and seller 3 inactive. This outcome is indicated with asterisks in Table 5. Thus, the stable assignment yields a total value of 14 and only two transactions. The stable assignment is not efficient. Becker’s (1973) marriage market provides conditions under which a stable assignment is efficient with differentiated products. The production Table 5 The efficient assignment (in bold) and the stable assignment (with asterisks) in the housing market
Buyer 1 Buyer 2 Buyer 3
Seller 1
Seller 2
Seller 3
5* 8 2
7 9* 6
2 3 0*
Ch. 3. Firms and Networks in Two-Sided Markets
173
function is a(z, y) ¼ zy. Let the zi and yj be arranged in decreasing order, z1>z2>y>zn and y1>y2>y>yn, and let zi ¼ yj for i ¼ j. Let seller costs equal zero. There is only one efficient assignment, which is to match each buyer with a seller of the same type. With decentralized exchange, buyers and sellers will self-select with the highest-value buyer matching with the highest-value seller and so on. Proposition 9. Suppose that buyers and sellers make their own bilateral matches in a market with differentiated products. Then, in Becker’s marriage market, the unique stable assignment of buyers and sellers is also the unique efficient assignment. Consider the market for differentiated products in which the technology is additive a(z, y) ¼ z+y. Then, given the Complete Viability Condition, all assignments are efficient. Since a unique stable assignment exists, it is efficient as well. Proposition 10. Suppose that buyers and sellers make their own matches in a market with differentiated products. Then, given an additive production function and the Complete Viability Condition, the unique stable assignment is efficient. The stable assignment need not be efficient if the network is not complete, as was seen with homogenous products. The same problem arises with differentiated products. Given the conditions of Hall’s Marriage Theorem, the graph contains an assignment. If the production function is additive and the Complete Viability Condition applies, that assignment must be efficient. The stable assignment may not be a complete assignment as was shown in Fig. 8. Seller 1 prefers to deal with buyer 1 instead of buyer 2 since z1>z2 implies that z1+y1–c1>z2+y1–c1. Buyer 1 prefers to deal with seller 1 instead of seller 2 if z1+y1–c1>z1+y2–c2, in which case the stable assignment is (1, 1). However, the assignment (1, 2), (2, 1), which is feasible, yields greater value than (1, 1) since it is an efficient assignment on the complete network by the Complete Viability Condition. This implies the following result. Proposition 11. Consider a market with differentiated products. Suppose that the production function is additive and the Complete Viability Condition holds. The graph G is not complete but satisfies the conditions of Hall’s Marriage Theorem. Then the stable assignment on the graph G need not be efficient on the network. 7
Firms and stable assignments in a centralized two-sided market
Consider a market in which a monopoly firm provides centralized communication using information systems. Buyers and sellers who are the firm’s
174
D.F. Spulber
subscribers find each other and form their own bilateral matches, so that the matching process itself is decentralized. The firm incurs costs in creating and operating the centralized market. Once they become members of the network, buyers and sellers have zero costs of communication. The firm provides communication in the form of a centralized hub-and-spoke communications network that allows every buyer to be linked to every seller. This provides the links described by a complete bipartite graph that connects every buyer with every seller. Once buyers and sellers are members of the network their costs of communication are zero. In addition, the firm provides every buyer and every seller with full information about all other buyers and sellers. The cost savings of establishing a centralized hub-andspoke network reinforces potential efficiency advantages of full information. The firm charges buyers a subscription fee P to participate in the network and charges sellers a subscription fee W to participate in the network. There are two alternative scenarios. In the first scenario, the firm charges the fees only to buyers and sellers after they complete a match based on transactions. In the second scenario, the firm charges the fees to buyers and sellers before they complete a match based on subscription to the network. These two scenarios affect the nature of the equilibrium.19 7.1 Firms and stable assignments with homogenous products The firm reduces but does not eliminate coordination problems for buyers and sellers. Because buyers and sellers have complete information about each other after they become subscribers, they coordinate easily after joining the firm’s network. However, they still face a coordination problem before becoming subscribers because each side must anticipate which buyers and sellers will choose to participate in the network. Buyers and sellers play a coordination game in participation decisions. To illustrate the coordination problem, consider a simple game with a single buyer and a single seller. The buyer has willingness to pay v and the seller has cost c. Assume that they have potential gains from trade, v>c. Suppose that the firm charges a fee after a match is made. Then, the buyer and the seller allocate the surplus from exchange net of the fees charged by the firm. They will become members of the network if the total gains from trade are non-negative, (v–c–P–W)Z0. Suppose now that the firm charges a fee before a match is made, so that the subscriber fees are sunk costs when the transaction occurs. Let buyers and sellers engage in Nash bargaining (a ¼ 1/2) and thus evenly divide the quasirents from exchange so that exchange between the buyer and the seller 19 Caillaud and Jullien (2003) refer to ex ante fees as connection or registration fees and they refer to ex post fees as transaction fees. Their framework differs from the present one since in their model, all buyers are identical and all sellers are identical but there is only one acceptable match for each one.
175
Ch. 3. Firms and Networks in Two-Sided Markets Table 6 A participation game where payoffs are (buyer, seller) Seller
Subscribe
Do not subscribe
(v–c)/2–P, (v–c)/2–W 0, W
P, 0 0, 0
Buyer Subscribe Do not subscribe
gives (v – c)/2. Then, the buyer and seller face a coordination game resembling the battle of the sexes (see Table 6). Then, if the buyer anticipates trading with the seller, the buyer will become a member if (v–c)/2ZP. Also, if the seller anticipates trading with the buyer, the seller will become a member if (v–c)/2ZW. If both these conditions are satisfied, then both subscribing is a Nash equilibrium, although neither subscribing also is a Nash equilibrium. Consider the market with consumers divided into two distinct groups, a set of buyers B ¼ {1, y, n} and a set of sellers S ¼ {1, y, m}. As before, buyers have valuations that are arranged in decreasing order, v1>v2>y>vn, and sellers have costs that are arranged in increasing order, c1oc2oyocm. Only buyers and sellers subscribing to the firm’s service can communicate with each other and form matches. Communication is costless since buyers and sellers are linked by a complete bipartite graph and buyers and sellers have full information about each others’ types. Buyers and sellers form bilateral matches and split the gains from trade in some way. Consider the first scenario in which buyers and sellers make their participation decision and make a match before paying the fee. The modified value of a match between buyer i and seller j is hij ¼ maxf0; vi cj P W g.
(15)
Let n be the largest integer i such that vi P cj þ W ; where i ¼ j, iAB and jAS. The number n is the market-clearing quantity with fees P and W. Let B ¼ {1, y, n} and S ¼ {1, y, n}. The sets B and S describe the buyers and the sellers that are on the left-hand side of the supply-and-demand diagram such that if they are paired their gains from trade exceeds the sum of the two subscription fees. In defining the two sets B and S, buyers and sellers are paired in order, just as in Bo¨hm-Bawerk’s (1891) method of marginal pairs. It is possible that an excluded buyer i>n, if paired with a seller in S, could have gains from trade that exceed the sum of subscriber fees. The same applies to excluded sellers j>n. However, the two sets reflect the best matches. Because buyers and sellers have full information on the firm’s network, buyers and sellers will choose to participate only in the best
176
D.F. Spulber
possible match. Thus, the highest-value buyer trades with the lowest-cost seller, and so on with the remaining buyers and sellers. As in Bo¨hm-Bawerk (1891), buyers and sellers form pairs until the marginal pair is reached. However, in contrast to Bo¨hm-Bawerk, the terms of trade are determined by bilateral bargaining rather than with two-sided competition managed by a central auctioneer. Let R ¼ P+W be the sum of the subscription fees. Proposition 12. Let subscription fees be paid after a match is made. Then, in a market with homogenous products, only buyers in B and sellers in S will choose to become subscribers on the firm’s network. Only the sum of subscription fees matters for participation, not the amounts assigned to buyers and to sellers. Those buyers and sellers that choose to join the network as subscribers correctly expect to find a match. Other buyers and sellers do not participate since they would not find a match. Only the sum of the subscriber fees matters since buyers and sellers anticipate the assignment process on the network, see Fig. 9. The allocation of the fees between buyers and sellers does not affect the participation decision since the total of the two fees is subtracted from the value of the bilateral transaction. A buyer and seller engaged in bilateral exchange bargain over the allocation of rents from exchange, which equals the value to the buyer minus the cost to the seller and minus total fees. The allocation of fees between buyers and sellers also does not affect the firm’s profit. Suppose that the firm has a cost T per subscriber pair. Since n is the volume of
Supply
v, c B**
R{
Demand S** n** Fig. 9.
m
n
i, j
Subscribers to a network when the sum of the buyers’ and sellers’ fee equals R.
Ch. 3. Firms and Networks in Two-Sided Markets
177
transactions, the firm’s profit equals P ¼ n ðP þ W TÞ ¼ n ðR TÞ.
(16)
Consider the second scenario in which buyers and sellers must pay a fee before making a match. The fees become sunk costs that do not affect the quasirents from exchange. After fees are paid and buyers and sellers are members of the firm’s network, the value of a match between buyer i and seller j is the same as in decentralized exchange hij ¼ max{0,vi–cj}. Because buyers and sellers have full information on the firm’s network, buyers and sellers will choose to participate only in the best possible match. Thus, the highest-value buyer trades with the lowest-cost seller, and so on with the remaining buyers and sellers, regardless of the subscription fees. The allocation of the subscription fees does affect the participation decision of buyers and of sellers in the second scenario since buyers and sellers make decisions about participation before paying the fees. The outcome of bilateral exchange affects the participation decision as well. Suppose for example, that buyers and sellers engage in Nash bargaining (a ¼ 1/2) and thus evenly divide the quasirents from exchange. Even division of quasirents implies that exchange between buyer i and seller j gives each party (vi–cj)/2. Then, if buyer i anticipates trading with seller j, buyer i will become a member if and only if (vi–cj)/2ZP. Also, if seller j anticipates trading with buyer i, seller j will become a member if and only if (vi–cj)/ 2ZW. If P is greater than W, then fewer buyers than sellers will find it worthwhile to subscribe based on the division of surplus. Since buyers and sellers are paired in order, the number of sellers that choose to subscribe in equilibrium equals the number of buyers that choose to participate. Therefore, participation is determined by P when P is greater than W. The converse is true when W is greater than P. Therefore, when the buyer and seller evenly split the quasirents from exchange, the higher of the two fees effectively determines participation of both buyers and sellers in equilibrium. Proposition 13. Let subscription fees be paid before a match is made. Then, in a market with homogenous products, if buyers and sellers engage in Nash bargaining, the greater of the buyer and seller fees determines participation in the network. Since the higher of the two fees determines participation with Nash bargaining between buyers and sellers, the firm will choose to set equal subscriber fees for buyers and for sellers, P ¼ W ¼ R/2. Since the higher of the two fees determines participation, the profit-maximizing firm will raise the lower of the two fees to equal the higher one since that increases revenue without affecting participation. From Propositions 12 and 13, it follows that the same level of participation will be observed under the two scenarios for the same total of the fees R. This is because the allocation of subscriber fees is neutral in the first
178
D.F. Spulber
scenario and because the firm chooses equal subscriber fees P ¼ W ¼ R/2 in the second scenario. I now derive the monopoly firm’s choice of buyer and seller fees. For ease of presentation, consider the limiting case in which there is a continuum of buyers and a continuum of sellers both of which are uniformly distributed on the unit interval with unit density. Recall in the first scenario that if fees are paid after a match is made, only the total of the fees matters. In the second scenario, with fees paid before a match is made, the firm will set equal buyer and seller fees when buyers and sellers engage in Nash bargaining. Thus, in either case, a monopoly firm chooses only the total of buyer and seller fees, R. For any R less than one, the marginal buyer–seller pair is such that the market clears, 1 v ¼ c.
(17)
The marginal buyer–seller pair obtains gains from trade equal to the total buyer and seller fee R, v c ¼ R.
(18)
Thus, solving (17) and (18), the marginal pair is given by v ¼ ð1 þ RÞ=2;
c ¼ ð1 RÞ=2.
The volume of transactions is a function of the total of the buyer and seller fee, Q(R) ¼ (1R)/2. A firm with transaction cost T per subscriber pair has profit equal to PðRÞ ¼ ðR TÞð1 RÞ=2.
(19)
Maximizing profit in (19) gives the firm’s profit-maximizing fee, which equals R ¼ (1+T)/2. The firm’s profit is P ¼ (1T)2/8, and the marketclearing volume of transactions is Q ¼ (1T)/4. For any given fee R, consumers’ surplus equals buyers’ benefits minus sellers’ costs and minus the payments to the firm, Z QðRÞ Z QðRÞ D P ðQÞdQ PS ðQÞdQ RQðRÞ CSðRÞ ¼ 0
0
¼ ð1=4Þð1 RÞ2 .
ð20Þ
The firm’s fee R is a transaction cost that has two effects on gains from trade for buyers and sellers. There is a direct cost RQ(R) and there is a deadweight loss due to those efficient transactions that cannot be completed. The deadweight loss to buyers and sellers is the transactions foregone in comparison with a situation in which the transaction cost R equals zero. These two effects are shown in Fig. 10. Total surplus including the firm’s profit equals VðRÞ ¼ CSðRÞ þ PðRÞ ¼ ð1=4Þð1 R2 2T þ 2TRÞ.
(21)
Ch. 3. Firms and Networks in Two-Sided Markets
179
p S(p)
R
I
II
D(p)
Q(R)
Q
Fig. 10. Transaction costs to buyers and sellers due to a positive subscriber fee R (in comparison to the case of R ¼ 0) result in a resource cost (I) and a deadweight loss (II).
Evaluated at the profit-maximizing fee R ¼ ð1 þ TÞ=2; total surplus equals V ¼ ð3=16Þð1 TÞ2 .
(22)
7.2 Firms and stable assignments with differentiated products Consider again a market with differentiated products as represented by Becker’s (1973) marriage market in which each buyer i has a productivity parameter zi and each seller j has a productivity parameter yj. Again, suppose that the productivity parameters are in decreasing order for buyers and for sellers, z1>z2>y>zn and y1>y2>y>yn, and let zi ¼ yj for i ¼ j. Also, suppose that all sellers have zero costs. The value of a match is given by a multiplicative value function, aij ¼ a(zi, yj) ¼ ziyj. Consider the first scenario in which the firm charges buyers and sellers subscription fees after a match is made. The value of a match between buyer i and seller j with differentiated products is hij ¼ maxf0; aij P W g.
(23)
The stable assignment in the marriage market consists of buyer–seller pairs with identical types. Define the sets B and S as consisting of those buyers and sellers for whom a match yields surplus that covers the total of
180
D.F. Spulber
the subscribers’ fees. Let n be the largest n such that aðzi ; yj Þ ¼ R for zi ¼ yj .
(24)
The sets B and S consist of buyers and sellers 1, 2, y , n. Only those buyers and sellers that expect to be matched will subscribe to the network. Buyers and sellers will only choose to participate in the network if they can complete matches that generate sufficient gains from trade to cover the fee R. Proposition 14. Let subscribers’ fees be paid after a match is made. In Becker’s marriage market, only buyers in B and sellers in S will choose to become subscribers on the firm’s network. The sum of subscription fees matters for participation decisions, not the amounts assigned to buyers and to sellers. As with homogenous products, when subscription fees are charged after a match is made, the allocation of subscriber fees between buyers and sellers in the marriage market does not affect participation decisions. Consider now the second scenario in which subscriber fees are charged before buyers match with sellers. The outcome of bilateral exchange affects the participation decision. Suppose that buyers and sellers engage in Nash bargaining and thus evenly divide the quasirents from exchange so that exchange between buyer i and seller j gives each party aij/2. Then, if buyer i anticipates trading with seller j, buyer i will become a member if and only if aij =2 P: Also, if seller j anticipates trading with buyer i, seller j will become a member if and only if aij/2ZW. The effects of unequal fees are more complex with differentiated products than with homogenous products. Changes in the participation decisions of buyers or sellers can fundamentally change the configuration of efficient assignments and thus will affect buyer and seller expectations about participation on the other side of the market However, in Becker’s marriage market, the effects of unequal fees are similar to those in the homogenous products market. Suppose that P is greater than W. Then, since buyers and sellers are matched in decreasing order, only those buyers participate if aij/ 2ZP and the corresponding number of high-value sellers will participate. Thus, the greater of the two subscriber fees determines participation. Proposition 15. Let subscription fees be paid before a match is made. Then, in Becker’s Marriage Market with Nash Bargaining, the greater of the buyer and seller fees determines participation in the network. As in the case of homogenous products, the firm that charges subscriber fees before a match is made will set equal fees with differentiated products, P ¼ W ¼ R/2. The firm raises the lower fee to the level of the higher fee to increase revenue without reducing participation. In either scenario, with differentiated products, the profit-maximizing firm is only concerned with the choice of the total fee R. This is again
Ch. 3. Firms and Networks in Two-Sided Markets
181
because the sum of subscription fees is neutral if they are charged after a match is made, or if they are charged before a match is made, the firm will choose to make them symmetric, P ¼ W ¼ R/2, To examine the firm’s choice of a total fee, consider the marriage market with a continuum of buyers and a continuum of sellers, both of which are uniformly distributed on the unit interval. Since z2ZR for completed transactions, the firm’s volume of transactions equals QðRÞ ¼ 1 R1=2 .
(25)
Note that the marginal buyer and the marginal seller equal z ¼ y ¼ 1–Q(R). The firm’s profit is therefore PðRÞ ¼ ðR TÞð1 R1=2 Þ.
(26)
Arranging terms in the first-order condition, the firm’s profit-maximizing fee R solves 3R 2ðR Þ1=2 T ¼ 0.
(27)
Given the firm’s fee R, consumers’ surplus is the sum of the value of buyer–seller matches, Z
1
ðz2 RÞdz ¼ ð2=3ÞR3=2 R þ 1=3.
CSðRÞ ¼
(28)
1=2
R
Buyers and sellers pay a transaction cost in the form of the sum of the buyer and seller subscriber fees, R. This fee has both a resource cost effect on total buyer and seller benefits RQ(R) and a deadweight loss effect since it prevents efficient matches at the margin. As in the homogenous products case, the direct cost is a transfer to the firm and is viewed as a transaction cost to buyers and sellers. The deadweight loss to buyers and sellers is the transactions foregone in comparison with a situation in which the transaction cost R equals zero. This is shown in Fig. 11. Total surplus including the firm’s profit equals VðRÞ ¼ CSðRÞ þ PðRÞ ¼ 1=3 ð1=3ÞR3=2 þ TR1=2 T. 8
(29)
Matchmaking and market making by a firm using double auctions
Firms offer a combination of communication and computation that takes advantage of both centralized networks and centralized market-making mechanisms. For buyers and sellers that are members of the firm’s network, any buyer can be matched with any seller. The firm handles communication and computation by matching buyers and sellers through double auctions.
182
D.F. Spulber z2
z2
T
II
I
1 Q
1
z
Fig. 11. Transaction costs to buyers and sellers due to a positive subscriber fee R (in comparison to the case of R ¼ 0) result in a resource cost (I) and a deadweight loss (II).
8.1 Market making by a firm using double auctions for homogenous products Consider a market in which the firm assigns buyers to sellers using a double auction. The auction is a modified Vickrey auction. The firm charges a fixed commission R for each completed transaction. The firm has a transaction cost T for each completed trade. This includes the costs of communicating with buyers and sellers, operating the auction, and completing purchases and sales. The firm is profitable if and only if RZT. The firm does not know the types of buyers and sellers in advance, just as buyers and sellers themselves do not know each others’ types. The firm can apply a double-auction mechanism to match buyers with sellers efficiently. Using the double auction mechanism, the firm takes bids from buyers and sellers and uses the bids to clear the market. The firm sells goods to buyers and purchases goods from sellers. There are n potential buyers who choose bids bi, i ¼ 1, y, n. There are m potential sellers who choose bids sj, j ¼ 1, y, m. Buyers and sellers are risk neutral and symmetric. Their types are independently distributed with the uniform distribution on the unit interval. As before, buyers demand at most one unit of the good and sellers supply at most one unit. Suppose that bids received by the firm are numbered in decreasing order for buyers, b1>b2>y>bn, and in increasing order for sellers, s1os2oyosn. Assume that the firm chooses a uniform market-clearing price p. Buyers pay p to the firm and the firm pays p–R to sellers. The firm follows a process for choosing the market-clearing price based on the uniform-price auction.
Ch. 3. Firms and Networks in Two-Sided Markets
183
First, the firm finds the highest i such that bi si þ R. Denote that i by i. If bi þ1 si þ R; then let p ¼ bi+1 and sell to the first i buyers and buy from the first i sellers based on their bids. If that is not feasible but si þ1 bi and si þ1 si þ R; then let p ¼ si þ1 and sell to the first i buyers and buy from the first i sellers. Otherwise, let p ¼ bi and sell to the first i–1 buyers and buy from the first i–1 sellers. For convenience, let Q represent the market-clearing quantity that results from the auction, where Q ¼ i if p ¼ bi þ1 or p ¼ si þ1 ; and Q ¼ i–1 if p ¼ bi : Given the modified uniform-price auction, it can be shown that it is a weakly dominant strategy for buyers to bid their willingness to pay and for sellers to bid their cost, bi(vi) ¼ vi and sj(cj) ¼ cj. When arranged in decreasing order, buyer values are reversed order statistics. When arranged in increasing order, seller costs are standard order statistics. To see why bidding truthfully is a weakly dominant strategy, consider a buyer’s decision. A buyer never will bid above the true willingness to pay because the buyer would make a loss if a unit of the good were received. A buyer never would bid strictly below the true willingness to pay. If the buyer were not to receive a unit of bidding at willingness to pay, then bidding less would not have any effect. If the buyer were to receive a unit bidding at willingness to pay there are two possibilities. If the buyer is marginal then lowering the bid slightly below value will have no effect on the price paid or the likelihood of receiving a unit, while lowering the bid farther below value will result in the buyer not obtaining a unit. An inframarginal buyer does not have any effect on price from small reductions in the bid and does not benefit from becoming the marginal buyer by making large reductions in the bid. Therefore, each buyer bids their true value as a weakly dominant strategy, bi ¼ vi for all buyers. Similar arguments apply to seller bids. Each seller bids his true cost as a weakly dominant strategy, sj ¼ cj for all sellers. Sellers who supply a unit receive the market-clearing price net of the firm’s commission. The firm uses the double auction to match buyers and sellers efficiently by including highvalue buyers and low-cost sellers and by excluding low-value buyers and high-cost sellers. The firm extracts rents from buyers and from sellers, so that the marginal buyer and the marginal seller have positive surplus. Because the firm earns rents, it excludes some buyers and some sellers on the left-hand side of the supply-and-demand diagram. 8.2 Matchmaking and market making by a firm using double auctions for differentiated products The firm in a two-sided market for differentiated products can use double auctions to match buyers with sellers of differentiated products. In addition
184
D.F. Spulber
to being a matchmaker, the firm is also a market maker because the auction mechanism serves to determine the prices at which each buyer–seller pair will transact. In this section, I present a double auction for the two-sided market with differentiated products. The discussion uses the dominant-strategy mechanism due to Vickrey (1961), Clarke (1971), and Groves (1973), and the generalized Vickrey–Clarke–Groves mechanism of Krishna and Perry (1998). An efficient Vickrey–Clarke–Groves mechanism that is incentive compatible and individually rational needs to satisfy a balanced-budget constraint, see Green and Laffont (1977). It has been shown that under some conditions, there may exist an efficient Bayes–Nash mechanism that is incentive compatible and individually rational that satisfies a balanced-budget constraint, d’Aspremont and Ge´rardVaret (1979). Spulber (1988) shows that an efficient Bayes–Nash mechanism will satisfy budget balancing if the gains from trade between buyers and sellers are sufficiently large that they cover information rents used to induce buyers and sellers to tell the truth. I now consider the problem of a firm that is a monopoly intermediary in a two-sided market. Since the firm is maximizing profits there is not the problem of operating with a surplus, but the firm must be profitable to operate. The firm provides buyers and sellers with incentives to participate in the auction and to reveal their true types. The firm is profitable if the rents from exchange between buyers and sellers is sufficient to cover the incentives for truth-telling. The firm extracts rents from buyers and sellers, so that the firm necessarily excludes some matches that would be in the Core. Demange (1982) and Leonard (1983) design a related one-sided Vickrey auction for the two-sided market with differentiated products. They assume that all sellers have identical costs that equal zero, so that in their framework the auction serves to allocate the differentiated products across buyers. Demange (1982) and Leonard (1983) use buyer bids to determine imputations in the Core of the cooperative game, which yields payoffs to both buyers and sellers (see also Roth and Sotomayor, 1990). In their framework, the auction satisfies a break-even constraint since buyer payoffs plus seller payoffs equal total value. In contrast, I allow sellers to have different costs so that sellers must also make bids that reveal their cost information. In the general setting of the two-sided market, sellers have different costs, cj>0, j ¼ 1, y, m. Also, the double auction determines ask prices for buyers and bid prices for sellers. Accordingly, the firm designing the auction earns a profit equal to the difference between payments received from buyers and payments made to sellers. There are n buyers, i ¼ 1, y, n. Buyer types are represented by a vector of benefits that the buyer would obtain from consuming a unit of the various goods j ¼ 1, y, m, ai ¼ (ai1, ai2, y, aim). Seller types are simply
Ch. 3. Firms and Networks in Two-Sided Markets
185
scalar costs cj for sellers j ¼ 1, y, m. Let Hi be the set of possible types for a buyer where Hi is a compact, convex subset of Rm, and let Cj be the set of possible cost types for a seller, where Cj is a closed interval on the positive real line. Buyers report their vector of preference types ai in Hi and sellers report their cost types cj in Cj. Let H ¼ in¼ 1Hi and let C ¼ jm¼ 1Cj. Let the n m matrix of values x represent an allocation of goods. The values xij in the matrix P x equal either zero or one. A buyer receives at most a unit of one good, ixijr1. A seller provides at most a unit of one good P x r1. A good j can be transferred from seller j to buyer i only if aij>cj. i ij Define a benefit function for buyers bi(x) such that bi ðxÞ ¼ aij if for some j in S, xij ¼ 1, and let bi(x) ¼ 0 otherwise. Define a benefit function for sellers bj(x) such that bj ðxÞ ¼ cj if for some i in B, xij ¼ 1 and let bj(x) ¼ 0 otherwise. Let X be a discrete finite set containing all possible allocations of the differentiated products. The firm designs a direct mechanism consisting of an allocation rule Q: H C-X and a payment rule c : H C-Rn+m. The firm obtains the allocation rule Q by calculating the valuations vij ¼ max{0, aij–cj} and choosing x to solve the linear programming problem (4). The firm’s payment rule c consists of a vector of ask prices for buyers p1 , p2 , y, pn and a vector of bid prices for sellers w1 , w2 , y, wm. A buyer i reports his type, which consists of a preference vector ai1, ai2, y, aim and a seller j reports his cost type cj. Then, given the allocation rule Q and the payment rule c, a buyer i receives bi(x)–pi, and a seller j receives wj+bj(x). By the definition of the individual benefit functions, it follows that the total benefit equals the sum of the individual benefit functions X X VðJÞ ¼ bi ðx Þ þ bj ðx Þ. (30) i2B
j2S
Also, define V(J–i) as the total benefits obtained from optimization problem (4) with the set of buyers except for buyer i, B–i and the set of sellers S. Define V(J–j) as the total benefits obtained from optimization problem (4) with the set of buyers B and the set of sellers except for seller j, S–j. Define V(J–i–j) as the total benefits obtained from optimization problem (4) with the set of buyers B and the set of sellers S, excluding the buyer–seller pair (i, j). Given the generalized Vickrey–Clarke–Groves mechanism, the payment for buyer i equals X X pi ¼ VðJ iÞ bk ðx Þ bj ðx Þ þ R=2. (31) k2B; kai
j2s
186
D.F. Spulber
The Vickrey–Clarke–Groves payment to seller j equals X X wj ¼ V ðJ jÞ þ bi ðx Þ þ bk ðx Þ R=2. i2B
(32)
k2s; kaj
Therefore, buyer i receives a benefit from the auction equal to bi ðx Þ pi ¼ V ðJÞ V ðJ iÞ R=2.
(33)
Seller j receives a benefit from the auction equal to wj þ bj ðx Þ ¼ V ðJÞ V ðJ jÞ R=2.
(34)
The buyer and seller payment mechanism is thus c ¼ ðp1 ; . . . ; pn ; w1 ; . . . ; wm Þ,
(35)
where prices depend on the reported types, that is pi(a, c), wj(a, c) with (a, c) in H C. The firm assigns buyer i to seller j to maximize the sum of net benefits. Let E be the firm’s profit-maximizing assignment and let n be the number of matches in the assignment. Assume that the firm chooses R to maximize profit after observing buyer and seller bids. Buyers and sellers choose to participate if they have positive net benefits, so that the mechanism satisfies individual rationality. The mechanism is incentive compatible. Buyer i’s payoff is V(J)–V(J–i)– R/2 so that only the first term depends on the buyer’s reported type ai ¼ (ai1, y, aim). The buyer maximizes the first term by correctly reporting his type. Seller j’s payoff has the same property. Since truth-telling is a weakly dominant strategy with the Vickrey–Clarke–Groves mechanism, the mechanism is incentive compatible. Thus, in equilibrium every buyer and every seller receives their marginal contribution to total benefits (see Krishna and Perry, 1998). A mechanism P is efficient P if the allocation rule Q: H C-X maximizes total benefits ibi(x)+ jbj(x). Krishna and Perry (1998) demonstrate that the Vickrey–Clarke–Groves mechanism maximizes the expected payments of each agent among all mechanisms that are efficient, incentive compatible, and individually rational. The Vickrey–Clarke–Groves mechanism is efficient since it selects an allocation x that yields the maximum benefits V(J) for the set of buyers and sellers J ¼ (B, S). Except for the payment R/2, the payoff to a buyer in the Vickrey double auction, V(J)–V(J–i)–R, exactly equals the maximum payoff to a buyer in the Core since it equals each buyer’s contribution to the grand coalition’s value. This is the same payoff observed in the one-sided Vickrey auction considered by Demange (1982) and Leonard (1983). In the double auction, the payoff to a seller also equals the maximum payoff since it equals their contribution to the grand coalition’s value V(J)–V(J–j), again net of R/2.
Ch. 3. Firms and Networks in Two-Sided Markets
Given the Vickrey–Clarke–Groves mechanism, the firm’s profit is X X pi wj . P¼ i2B
187
(36)
j2S
If buyer i is matched with seller j, the price spread equals pi wj ¼ bi ðx Þ bj ðx Þ ðV ðJÞ V ðJ iÞÞ ðVðJÞ V ðJ jÞÞ þ R.
ð37Þ
By the definition of the benefit functions, note that bi(x)bj(x) ¼ aijcj. The incremental value contributed by a matched pair (i, j) equals aij cj ¼ V ðJÞ V ðJ i jÞ.
(38)
Substituting into the price spread equation gives pi wj ¼ ðV ðJÞ þ V ðJ i jÞ V ðJ iÞ V ðJ jÞÞ þ R.
(39)
The term in parenthesis is positive since buyers and sellers are complements. Thus, the price spread is less than or equal to R for each buyer and seller pair, piwjrR. Summing over buyer–seller pairs in the assignment chosen by the firm, the firm’s profit equals X P¼ ðV ðJÞ þ V ðJ i jÞ V ðJ iÞ VðJ jÞÞ þ n R. ði;jÞ2E
(40) To illustrate the firm’s optimization problem, consider an example with two buyers and two sellers. Assume that a11–c1oa22–c2. a11 c1 40;
a12 c2 o0;
a21 c1 o0;
a22 c2 40.
The optimal assignment is (1, 1) and (2, 2). Then, the total benefit equals V(J) ¼ a11c1+a22c2. The benefit without buyer 1 or without seller 1 equals V(Ji) ¼ V(Jj) ¼ V(Jij) ¼ a22c2 for i ¼ j ¼ 1. The benefit without buyer 2 or without seller 2 equals V(Ji) ¼ V(Jj) ¼ V(Jij) ¼ a11c1 for i ¼ j ¼ 2. The firm’s profit is then P ¼ ða11 c1 þ a22 c2 Þ þ 2R. The firm sets the fee R such that R=2 ¼ a11 c1 . Substituting for the fee R, the firm’s profit equals P ¼ 3ða11 c1 Þ ða22 c2 Þ. The firm is profitable if 3(a11c1)>(a22c2).
188 9
D.F. Spulber
Two-sided markets in random networks
In this section, I consider decentralized exchange between buyers and sellers with both costly communication and costly computation. I examine both costly search and transactions on random networks. The inefficiencies of the random matching process are the manifestation of transaction costs for buyers and for sellers. When communication is costly, buyers and sellers may incur search costs in trying to find trading partners. Before searching, consumers may not know the characteristics of other consumers. The costs of search to consumers can include the costs of travel and the time it takes to search. Also, the search market represents a market with costly computation since the allocation mechanism is decentralized with transactions taking the form of independent, bilateral transactions.20 The search process is inherently uncertain, consumers do not know whether or not they will find a trading partner and they do not know the characteristics of prospective trading partners. The risks involved in the search process represent yet another cost of search. Because search takes time, consumers experience delays in consumption. The loss of future benefits is another cost of search if consumers discount future benefits. In addition to the resources expended in the search process and the costs of risk and time, costly search results in deadweight losses either from inefficient matches or from the failure to make efficient matches. Consumers may not find the best trading partner, settling instead for a less-desirable match that is available so as to avoid further search. If search is sufficiently costly, consumers may choose not to seek any trading partners. Thus, the costs of search include lost gains from trade. 9.1 Search and random assignments A random selection from the set of assignments corresponds to random matching of buyers and sellers. Random matching is one way of representing the transaction costs of consumer search. If there are n buyers and n sellers, then a complete assignment consists of n matchings and there are n! equally likely assignments. Buyers and sellers do not know each others’ identities before they meet to trade. Buyers and sellers are matched only once. After a buyer and seller meet, they observe each others’ type. After they are matched, trade takes place between buyer i and seller j if and only if gains from trade are nonnegative. In the market with homogenous products this requires viZcj, and in the market for differentiated products this requires aijZcj. This implies the following result. Consider either (i) the market with homogenous products or (ii) the market for differentiated products with 20
See Shimer and Smith (2000) for a model of assortative matching with costly search.
Ch. 3. Firms and Networks in Two-Sided Markets
189
additive production function and different costs across sellers. Then, random matching is efficient if and only if the Complete Viability Condition holds. Consider first the market for homogenous products. Random matching is efficient if and only if there is no right-hand side of the supply-and-demand diagram. The demand curve must be everywhere above the supply curve. Otherwise, buyers and sellers on the right-hand side of the supply-anddemand diagram are active in the search market. These low-value buyers and high-cost sellers may be matched with players who would otherwise trade under an efficient assignment, thus displacing efficient matches. The displacement of efficient matches can be viewed as an externality that results from search inefficiencies. Buyers and sellers cannot choose efficient matches due to imperfect information about the types of potential trading partners. Costly search prevents buyers and sellers from visiting multiple trading partners as a way of improving matches. Sufficiency of the Complete Viability Condition is evident since random matching will result in a selection from the set of efficient assignments. To establish necessity, suppose that all pairs are viable vnZcm, but the set of sellers has one more member than the set of buyers. Since one of the buyers can be matched with the highest-cost seller, a better pairing is displaced, which is inefficient. Suppose that there is the same number of buyers and sellers and viZci, i ¼ 1, y, n–1 but vnocn. Then, if buyer 1 is matched with seller n, the match displaces an efficient match. Therefore, the Complete Viability Condition is necessary for random matching to be efficient. So, the Complete Viability Condition is both necessary and sufficient for random matching to be efficient. When the Complete Viability Condition does not hold therefore, random matching is always inefficient in expectation. The demand and supply functions cross, due to the highest-value buyer having a lower value than the high-cost seller’s cost or due to the number of buyers and sellers not being equal. Thus, inefficient matches can occur with positive probability. If the Complete Viability Condition does not hold, random matching can generate too much trade. Consider, for example, a market with four buyers and four sellers. Assume that v1>c4>v2>c3>v3>c2>v4>c1 (see Fig. 12). Supply and demand cross so the Complete Viability Condition does not hold. The efficient number of trades equals 2. At an efficient outcome, buyers v1 and v2 would be paired with sellers c1 and c2 in either combination. Buyers v3 and v4 and sellers c3 and c4 would be inactive. However, there are 4! ¼ 24 possible assignments, so only 4 out of 24 are efficient. With random matching, efficiency occurs with probability 1/6. Otherwise, successful matches with inefficient buyers or sellers are possible. Out of 24 assignments there is only 1 assignment with 4 trades. There are 10 assignments with 3 trades, 12 assignments with 2 trades, and 1 assignment with 1 trade. Thus, the expected number of trades with random matching equals 4(1/24)+3(10/24)
190
D.F. Spulber v, c v1 Supply c4 v2
c2
c3 v3
v4 c1 Demand
Q Fig. 12. A market with four buyers and four sellers.
+2(12/24)+1(1/24) ¼ 59/24. This exceeds 2, which is the number of efficient trades. If buyer types and seller types are chosen randomly from overlapping intervals of buyer values and seller costs, the Complete Viability Condition is not likely to hold. To illustrate the inefficiency of random matching, suppose that there are many buyers and sellers. Let buyer values v be uniformly distributed on the unit interval so that market demand is DðpÞ ¼ Prfv4pg ¼ 1 p.
(41)
Let seller costs c be uniformly distributed on the unit interval so that market supply is SðpÞ ¼ Prfc pg ¼ p. D
S
(42)
Let P (Q) ¼ 1Q and P (Q) ¼ Q be the inverse demand and the inverse supply. The efficient volume of trade is Q ¼ 1/2 and the total benefits with an efficient assignment of buyers to sellers are Z 1 Z 1=2 V ¼ vdv cdc ¼ 1=4. (43) 1=2
0
Ch. 3. Firms and Networks in Two-Sided Markets
191
The total expected value with random matching and a continuum of buyers and sellers equals Z 1Z v 0 V ¼ ðv cÞdcdv ¼ 1=6, (44) 0
0
so that random matching is not efficient. The total expected volume of trade with random matching and a continuum of buyers and seller is Q0 ¼ 1/2, which is the same as the efficient volume. Now compare decentralized search by buyers and sellers with centralized exchange with a network operated by monopoly firm. The firm’s network is more efficient than decentralized search, VZV0 ¼ 1/6 if the firm’s costs are not too large, Tr1(1/3)O8. Consider now the market for differentiated products. Any costly search process under imperfect information introduces randomness in the assignment process. It is straightforward to show that random matching will be inefficient game. Let bij be a set of probabilities such that P for the assignment P bijZ0, ibij ¼ 1, and jbij ¼ 1. Then, for any arbitrary set of probabilities, it follows by optimization that the efficient solution must yield a greater total value than random matching, XX XX vij xij vij bij . i
j
i
j
The preceding inequality is likely to be strict. The maximum value in the linear program is attained for xij taking values of zero or one. Suppose for example that there is a unique efficient matching where xij ¼ 1 for i ¼ j, xij ¼ 0 for i6¼j and n ¼ m ¼ 2. Thus, v11+v22>v12+v21. Then, the expected value of any random matching with 0obijo1 for some ij pair is strictly less than the maximum total value, v11 þ v22 4X 11 ðv11 þ v22 Þ þ ð1 X 11 Þðv12 þ v21 Þ, XX vij bij ; ¼ i
j
since b11 ¼ 1–b12 ¼ 1–b21 ¼ b22. The inefficiency of random matching applies to any set of probabilities including a random process that matches buyers and sellers with equal probability of any match. If the numbers of buyers and sellers are not equal, then any player on the long side of the market has the same probability of not being matched. Let vij ¼ 0 represent the value of matches between players on the long side of the market with ‘‘dummy’’ players that are added to the short side of the market to equalize the number of players on each side. Then, the inefficiency of random matching also applies when the number of buyers differs from the number of sellers. Consider Becker’s marriage market with a continuum of buyers and sellers who have productivity parameters uniformly distributed on the unit
192
D.F. Spulber
interval. When z ¼ y, a(z, y) ¼ z2. Thus, the value of the grand coalition of buyers and sellers is Z 1 V ¼ vðB [ SÞ ¼ aðz; zÞdz ¼ 1=3. (45) 0
Buyers and sellers can attain this outcome if there is full information and costless matching. To attain the outcome requires a network with a complete set of links between all buyers and all sellers. If establishing links is costly, then it will be very costly to establish a complete network so as to attain the efficient assignment. Compare the value generated by efficient matching with that generated by random matching. With random matching all buyer–seller pairs are feasible since they generate nonnegative value. The total gains from trade with random matching equals Z 1Z 1 0 V ¼ zydydz ¼ 1=4: (46) 0
0
The difference between V ¼ 1/3 and V0 ¼ 1/4 represents the inefficiency of random matching in a large market with differentiated products. In a Becker marriage market, all matches generate positive benefits so that random matching performs relatively well. More generally, with seller costs and general differentiated products, there can be matches that do not result in trade, so that random matching need not perform well. To compare random matching with intermediation by the firm, let the firm costs T equal zero. Then, the monopoly commission rate is R ¼ 4/9 and total surplus is VðR Þ ¼ ð1=3Þð19=27ÞoV 0 ¼ 1=4. Since any positive value of costs T will lower total surplus, it follows that centralized matching by the monopoly firm yields lower total gains from trade in comparison with random matching. Recall that random search in the marriage market performs well because all matches yield positive benefits. The monopoly firm also performs less well than random search because of the efficiency losses from monopoly pricing. Centralized matching by the firm offers advantages in comparison with decentralized matching that can be realized by lowering the fee. Suppose that the firm competes with decentralized exchange and suppose that buyers and sellers must decide whether to enter the search market or deal with the firm before observing their type. The firm will be constrained in its pricing by the search market alternative. The highest commission rate that the firm can charge and still attract potential buyers and sellers must yields the consumer surplus from decentralized exchange, CSðRMAX Þ ¼ V 0 ¼ 1=4,
(47)
Ch. 3. Firms and Networks in Two-Sided Markets
193
so RMAX solves 2R3/2–3R+1/4 ¼ 0 and RMAX is approximately 1/10. For the market-making firm to be profitable and to improve economic efficiency, its transaction cost T must be less than RMAX. Then, the marketmaking firm can attract all potential buyers and sellers in a market with differentiated products. Given consumers’ surplus evaluated at the maximum commission CS(RMAX) ¼ V0, the firm’s profit at the maximum commission rate equals PðRMAX Þ ¼ V ðRMAX Þ V 0 .
(48) MAX
, the firm earns a profit by If transaction costs are not too high, ToR improving allocative efficiency relative to the search market. The firm offers the advantage of efficient assignments of buyers and sellers that are its subscribers but at the cost of excluding some buyers and sellers. The search market yields inefficient assignments but all buyers and sellers participate and receive positive value from any match. The firm adds value if the direct costs of making assignments and the deadweight losses from excluding some buyers and sellers are not too high. Hoppe, Moldovanu and Sela (2005) consider a Becker-type marriage market. They compare the total gains from random matching with that from assortative matching under incomplete information when players can engage in costly signaling. For a class of distributions with increasing failure rate, they find that random matching performs better than assortative matching, although for distributions with a decreasing failure rate, the efficiency gains from assortative matching can exceed those under random matching. 9.2 Markets and random networks Mathematicians, scientists, and sociologists have studied extensively the random formation of networks.21 Random networks are of great value in understanding the structure of networks. First, random networks provide a contrast with the design of efficient networks by firms that establish networks. Second, random networks are highly interesting because they suggest patterns of usage that might be observed within an existing network. For example, telephone calls on a telecommunications network within a given period of time activate various nodes and links to constitute a temporary network that reflects traffic patterns on the underlying physical network. Network reliability depends on the supply of network capacity and random demand for that capacity. Even managed networks evolve over time as new facilities are added and old facilities are abandoned. Companies alter their network facilities in response to variations in customer demand, production costs, technology, 21 On random networks, see Solomonoff and Rapoport (1951), Erdo¨s and Re´nyi (1960, 1961), Janson et al. (2000), and Bolloba´s, (2001). For popular introductions, see Baraba´si (2002) and Watts (2003).
194
D.F. Spulber
competitor behavior, and regulation. Thus, demand shocks or cost shocks alter the plans of network designers and introduce randomness in the evolution of the network. Random networks offer some insights into the evolution of designed networks that are subject to random shocks. Networks that offer open access experience randomness in terms of the types of complementary services that are provided by independent suppliers. Innovation by suppliers of complementary services introduces uncertainty about traffic and usage patterns on a network. The rapid growth of electronic commerce applications on the Internet created demand patterns and usage of the network exhibited considerable randomness. For example, consider the impact of search engines on the development of the Internet. Consider a simple random network with three preexisting nodes. Suppose that in a given period of time, there is a probability equal to b that a link forms to connect any pair of nodes where 0obo1. Then, by the end of the period, there are four possibilities. There might be no links, one link, two links, or the full set of three links. The likelihood of these events is as follows: no links (1–b)3, one link 3b(1–b)2, two links 3b2(1–b), and three links b3. These possibilities are shown in Fig. 13. This assumes that there is no distinction between any of the nodes. The probability distribution over the number of links follows the basic Bernoulli distribution. The notion of random networks can be extended to examine the evolution of networks over time. Suppose for example that at most one link can form per unit of time, again with likelihood b. Then, the four configurations shown in Fig. 13 also would depict the growth of the network over time. The network would have no links after the first period with likelihood 1–b and one link with likelihood b. After two periods, the network would have no links with likelihood (1–b)2, one link with likelihood 2b(1–b), and two links with likelihood b2. The likelihood of a link forming in a period thus will determine the speed of network formation. Another extension of this approach is to make the likelihood of links forming depend in some way on the characteristics of the nodes. For example, the nodes might be separated by geographic distance. A link might be more likely to form between two nodes that are closer to each other than
No links
One link
1
3
1
2
(1 – b)3
Two links 1
2
3
3b(1 – b)2
Three links
3
1
2
3b2(1 – b)
3
2 b3
Fig. 13. The likelihood of various configurations of a random network with three links and likelihood b of a link forming between any pair of nodes.
Ch. 3. Firms and Networks in Two-Sided Markets
195
between two nodes that are farther apart. This might describe the formation of business relationships when individuals are separated geographically. The discussion of random networks assumed that the nodes of the network were already established but that the formation of links was random. Instead, one can suppose that the number of characteristics of the nodes are also random. For example, the existence of a node can represent the decision of a consumer to enter or exit a given market. The number of nodes at any given time would be the net result of the entry and exit of individuals up to that time, in a similar way to demographic analysis of the evaluation of a population. Then, networks would reflect both the random number of nodes and the random formation of links between them. Random graphs have many interesting properties. Consider a set of nodes N ¼ {1, y, n}. Suppose that in each time period starting with the first time period, one new link is added on, so that at time t the graph has exactly t links. Suppose that any new link is equally likely. This is known as a random graph process. Such a process provides a description of how a graph might grow over time if the links were to be added randomly. As links are added randomly to a graph, the largest component of the graph has an important feature. Suppose that there is an initial set of n nodes. Then, after t periods of time, where t is not much greater than n/2, a giant component or cluster forms that swallows up other large components. The critical time n/2 is the phase transition time for the random graph process. For example, a graph process beginning with eight unconnected nodes has formed a giant component shortly after only four periods. After more time passes, the giant component swallows up all other components and the graph becomes connected! This means that by adding links randomly to n nodes, a network is very likely to be connected shortly after n/2 time periods.22 Assignments on bivariate graphs can be extended to incorporate uncertainty in the creation of links. Suppose that the numbers of buyers and of sellers are equal, n ¼ m. Let G(n, p) denote the set of all bipartite graphs with n nodes in each of the two parts of the graph. Let p be the probability that any pair of nodes, one from each set, is connected by a link. The probability that a graph G0 in G has y links is given by n Ny y ; where N ¼ . PðG0 Þ ¼ p ð1 pÞ 2 The greater is the likelihood of link formation p, the greater is the expected number of links. Since a random network is likely to be incomplete, assignments on the network are likely to be inefficient. The expected efficiency of the network will therefore be increasing in the probability of link formation. Proceeding 22
This discussion on random graphs draws from Be´la Bolloba´s (2001) and Janson et al. (2000).
196
D.F. Spulber
informally, let V(p) represent the expected gains from trade generated by a random network with probability of link formation p. As p approaches one, the likelihood of a complete network approaches one and decentralized exchange will be more efficient than a market with a firm that centralizes assignments. Suppose in contrast that a firm establishes a network with links to all buyers and sellers. The firm matches buyers and sellers efficiently as long as the surplus from a match exceeds the fee charged by the firm R. Then, the expected total benefits generated by the firm equals V(R), the firm offers an advantage over decentralized exchange if p is sufficiently small.23
10 Conclusion Firms play an essential role in the economy if they can coordinate transactions between buyers and sellers more efficiently than decentralized search and bargaining. Firms employ information systems to improve communication between buyers and sellers and to improve computation through centralized market mechanisms. Whether goods are homogenous or differentiated, firms can employ dominant-strategy auction mechanisms to match diverse buyers with diverse sellers. Such centralized matching mechanisms can offer efficiencies that reduce or avoid some of the resource costs and deadweight losses from search. When communication between consumers requires establishing costly networks, firms can offer some efficiencies from centralized networks. Huband-spoke networks with the firm at the center offer potential benefits relative to costly point-to-point networks that link consumers. There is a trade off between benefits of establishing more links in a decentralized network, which improves the performance of the network, and the costs of the links. By establishing and coordinating buyer networks and seller networks, firms offer potential advantages relative to decentralized exchange over a network of complete links between consumers. Centralized networks created by firms can also offer benefits relative to consumer networks that form randomly. 23 We can also represent random networks as a selection from a set of graphs. Let G(n, Y) be a set of bipartite graphs with n nodes in each set and Y links connecting nodes in one set with nodes in the other. n Suppose that all graphs in the set of such graphs are equally likely. Then, since G(n, Y) has Y elements, each graph occurs with probability 1= Yn : Then, G(n, Y) approaches a complete graph as Y goes to n2. As y increases, G(n, Y) yields a bipartite graph that is more likely to permit an efficient assignment. Let V(Y) be the expected efficiency of decentralized exchange with Y randomly chosen links. Compare this with a firm that establishes a centralized network at a fixed cost of T and matches buyers and sellers efficiently. Then, compare V(R) with g(Y). As Y increases toward n2, the expected benefits of decentralized approach full efficiency. The firm offers an advantage over decentralized assignments of buyers and sellers if Y is sufficiently small. As Y increases, any randomly chosen network will permit more efficient assignments. Thus, if Y is sufficiently large, decentralized exchange is likely to be more efficient than a market with a firm that centralizes assignments.
Ch. 3. Firms and Networks in Two-Sided Markets
197
Economic analysis of network theory suggests that the design of information systems can be enhanced by adding economic efficiency criteria to technical performance criteria. Engineers commonly use network theory to design communications and information systems to satisfy performance criteria such as reliability and transmission capacity. The interaction between information networks and markets is most clearly observed in electronic commerce such as Internet retail transactions and business-tobusiness wholesale transactions. Electronic trading systems in financial markets are another key example of the interaction between information networks and markets. Designers of information systems can benefit from a better understanding of the connections between networks and markets. Such an understanding should yield innovations in information systems that are designed to enhance the efficiency of economic transactions. Additional research is needed to explore the role of the firm as an intermediary. Competition between firms as matchmakers and as market makers should continue to provide a rich area of study. Network theory provides a powerful tool for understanding economic interactions between buyers and sellers that can be applied in a variety of settings. Network theory provides critical insights on how firms help to establish the microstructure of markets. Acknowledgments Daniel F. Spulber is the Elinor Hobbs Distinguished Professor of International Business and Professor of Management Strategy at the Kellogg School of Management, Northwestern University. The support of a research grant from the Searle Fund is gratefully acknowledged. I thank Michael Baye, George Deltas, Yuk-fai Fong, Terry Hendershott, Heidrun Hoppe, Simon Loertscher, and John Morgan for helpful comments. References Alderson, W. (1954). Factors governing the development of marketing channels, in: R.M. Clewett (ed.), Marketing Channels for Manufactured Products, Irwin, Homewood, IL. Aldous, J.M., R.J. Wilson (2000). Graphs and Applications: An Introductory Approach. Springer, New York. Baraba´si, A.-L. (2002). Linked: The New Science of Networks. Perseus Publishing, New York. Baye, M., J. Morgan (2001). Information gatekeepers on the internet and the competitiveness of homogenous product markets. American Economic Review 91, 454–474. Baye, M., J. Morgan, P. Scholten (2004). Price dispersion in the large and in the small: evidence from an internet price comparison site. Journal of Industrial Economics 52(December), 463–496. Becker, G.S. (1973). A theory of marriage: part I. The Journal of Political Economy 81(4), 813–846. Bo¨hm-Bawerk, E. von (1891). Positive Theory of Capital, Translated from German by W.E. Smart. Macmillan, London. Bolloba´s, B. (1998). Modern Graph Theory. Springer, New York.
198
D.F. Spulber
Bolloba´s, B. (2001). Random Graphs 2nd Edition. Cambridge University Press, Cambridge. Bresnahan, T., S.M. Greenstein (1996). Technical progress and co-invention on computing and in the uses of computers. Brookings Papers on Economic Activity: Microeconomics 1996, 1–78. Bresnahan, T., S.M. Greenstein (1999). Technological competition and the structure of the computer industry. Journal of Industrial Economics 47(March), 1–40. Caillaud, B., B. Jullien (2003). Chicken & egg: competition among intermediation service providers. Rand Journal of Economics 34, 309–328. Clarke, E. (1971). Multipart pricing of public goods. Public Choice 2, 19–33. Clower, R., A. Leijonhufvud (1975). The coordination of economic activities: a Keynesian perspective. American Economic Review 65, 182–188. Committee T1A1 [renamed Network Performance, Reliability and Quality of Service Committee (PRQC)] (2000). ATIS Telecom Glossary 2000. Alliance for Telecommunications Industry Solutions, Washington, DC. d’Aspremont, C., L.-A. Ge´rard-Varet (1979). Incentives and incomplete information. Journal of Public Economics 11(February), 25–45. Dantzig, G. (1963). Linear Programming and Extensions. Princeton University Press, Princeton, NJ. Demange, G. (1982). Strategyproofness in the assignment market game, Laboratoire d’Economie de L’Ecole Polytechnique, Paris, Working Paper. Demange, G., D. Gale (1985). The strategy structure of two-sided matching markets. Econometrica 53, 873–888. Diestel, R. (2000). Graph Theory 2nd Edition. Springer, New York. Dutta, B. and M.O. Jackson (eds.) (2003). Networks and Groups: Models of Strategic Formation. Springer, Berlin. Dutta, B., S. Mutuswami (1997). Stable networks. Journal of Economic Theory 76, 322–344. Ellison, G., D. Fudenberg M. Mobius (2003). Competing auctions, Department of Economics, MIT, Working Paper, March. Euler, L. (1736). Solutio problematis ad geometriam situs pertinentis [The solution of a problem relating to the geometry of position], 8 Commentarii Academiae Scientiarum Imperialis Petropolitanae 128,128—140, reprinted in Biggs, N.L., E.K. Lloyd and R.J. Wilson (1976). Graph Theory 1736— 1936. London, Oxford University Press, 3–11. Erdo¨s, P., A. Re´nyi (1960). On the evolution of random graphs. Publications of the Mathematical Institute of the Hungarian Academy of Sciences 5, 17–61. Erdo¨s, P., A. Re´nyi (1961). On the strength of connectedness of random graphs. Acta Mathematica Hungarian Academy of Sciences 12, 261–267. Farrell, J., G. Saloner (1985). Standardization, compatibility, and innovation. Rand Journal of Economics 16, 70–83. Farrell, J., G. Saloner (1986). Installed base and compatibility: innovation, product preannouncements, and predation. American Economic Review 76, 940–955. Fingleton, J. (1997). Competition among middlemen when buyers and sellers can trade directly. The Journal of Industrial Economics, XLV(4), 405–427. Gale, D., L. Shapley (1962). College admissions and the stability of marriage. The American Mathematical Monthly 69(1), 9–15. Gehrig, T. (1993). Intermediation in search markets. Journal of Economics and Management Strategy 2, 97–120. Granovettor, M. (1985). Economic action and social structure: the problem of embeddedness. American Journal of Sociology 91, 451–510. Green, J., J.-J. Laffont (1977). Characterization of satisfactory mechanisms for the revelation of preferences for public goods. Econometrica 45, 427–438. Greenstein, S.M. (1998). Industrial economics and strategy: computing platforms. IEEE Micro 18(May– June), 43–53. Gross, J., J. Yellen (1999). Graph Theory and Its Applications. CRC Press, Boca Raton. Gross, J. and J. Yellen (eds.) (2004). Handbook of Graph Theory. CRC Press, Boca Raton. Groves, T. (1973). Incentives in teams. Econometrica 41, 617–631.
Ch. 3. Firms and Networks in Two-Sided Markets
199
Hall, P. (1935). On representatives of subsets. Journal of the London Mathematical Society 10, 26–30. Hendershott, T., J. Zhang (2006). A model of direct and intermediated sales. Journal of Management & Economics Strategy 15, 279–316. Hoppe, H., B. Moldovanu, A. Sela (2005). The theory of assortative matching based on costly signals. Department of Economics, University of Bonn, Working Paper, November. Hylland, A., R. Zeckhauser (1979). The efficient allocation of individuals to positions. The Journal of Political Economy 87(2), 293–314. Jackson, M.O., A. Wolinsky (1996). A strategic model of social and economic networks. Journal of Economic Theory 71, 44–74. Janson, S., T. Luczak, A. Rucinski (2000). Random Graphs. Wiley, New York. Ju, J., S.C. Linn, Z. Zhu (2004). Price dispersion in a model with middlemen and oligopolistic market makers: a theory and an application to the North American natural gas market. University of Oklahoma, Working Paper. Katz, M.L., C. Shapiro (1985). Network externalities, competition and compatibility. American Economic Review 75, 424–440. Katz, M.L., C. Shapiro (1986). Technology adoption in the presence of network externalities. Journal of Political Economy 94, 822–841. Katz, M.L., C. Shapiro (1994). Systems competition and network effects. Journal of Economic Perspectives 8(Spring), 93–115. Kranton, R.E., D.F. Minehart (2000). Competition for goods in buyer–seller networks. Review of Economic Design 5, 301–331. Kranton, R.E., D.F. Minehart (2001). A theory of buyer–seller networks. American Economic Review 91(June), 485–508. Krishna, V., M. Perry (1998). Efficient mechanism design. Pennsylvania State University, PA, Working Paper, April. Leonard, H.B. (1983). Elicitation of honest preferences for the assignment of individuals to positions. Journal of Political Economy 19(3), 461–479. Liebowitz, S.J. (2002). Rethinking the Network Economy. American Management Association, New York. Liebowitz, S.J., S.E. Margolis (1994). Network externality: an uncommon tragedy. Journal of Economic Perspectives 8, 133–150. Liebowitz, S.J., S.E. Margolis (1999). Winners Losers and Microsoft: Competition and Antitrust in High Technology. The Independent Institute, Oakland, CA. Loertscher, S. (2004). Market-making oligopoly. University of Bern, Working Paper. Loertscher, S. (2005). Horizontally differentiated market makers. University of Bern, Working Paper. Lucking-Reiley, D., D.F. Spulber (2001). Business-to-business electronic commerce. Journal of Economic Perspectives 15(Winter), 55–68. Myerson, R. (1977). Graphs and cooperation in games. Mathematics of Operations Research 2, 225–229. Roth, A.E., M.A.O. Sotomayor (1990). Two-Sided Matching: A Study in Game Theoretic Modeling and Analysis. Cambridge University Press, Cambridge. Roth, A.E., X. Xing (1994). Jumping the gun: imperfections and institutions related to the timing of market transactions. American Economic Review 84(4), 992–1044. Rubinstein, A., A. Wolinsky (1987). Middlemen. Quarterly Journal of Economics 102, 581–593. Rust, J., G. Hall (2003). Middlemen versus market makers: a theory of competitive exchange. Journal of Political Economy 111(2), 353–403. Rysman, M. (2004). Competition between networks: a study of the market for yellow pages. Review of Economic Studies 71(April), 483–512. Shapley, L.S. (1961). Values of Large Games V: An 18-Person Market Game. The Rand Corporation, RM-2860, Santa Monica, CA, November. Shapley, L.S. (1962). Complements and substitutes in the optimal assignment problem. Naval Research Logistics Quarterly 9, 45–48. Shapley, L.S., M. Shubik (1972). The assignment game I: the Core. International Journal of Game Theory 1(2), 111–130.
200
D.F. Spulber
Shevchenko, A. (2004). Middlemen. International Economic Review 45(1), 1–24. Shimer, R., L. Smith (2000). Assortative matching and search. Econometrica 68(2), 343–369. Shubik, M.S. (1982). Game Theory in the Social Sciences, Concepts and Solutions. MIT Press, Cambridge, MA. Shubik, M.S. (1984). A Game-Theory Approach to Political Economy. MIT Press, Cambridge, MA. Sidak, J.G., D.F. Spulber (1996). Protecting Competition from the Postal Monopoly. American Enterprise Institute, Washington, DC. Solomonoff, R., A. Rapoport (1951). Connectivity of random nets. Bulletin of Mathematical Biophysics 13, 107–117. Spulber, D.F. (1988). Bargaining and regulation with asymmetric information about demand and supply. Journal of Economic Theory 44(April), 251–268. Spulber, D.F. (1996a). Market microstructure and intermediation. Journal of Economic Perspectives 10(Summer), 135–152. Spulber, D.F. (1996b). Market making by price-setting firms. Review of Economic Studies 63, 559–580. Spulber, D.F. (1998). The Market Makers: How Leading Companies Create and Win Markets. McGraw Hill/Business Week Books, New York. Spulber, D.F. (1999). Market Microstructure: Intermediaries and the Theory of the Firm. Cambridge University Press, New York. Spulber, D.F. (2002a). Market microstructure and incentives to invest. Journal of Political Economy 110(April), 352–381. Spulber, D.F. (2002b). Transaction innovation and the role of the firm, In: M.R. Baye (ed.), The Economics of the Internet and E-commerce, Advances in Applied Micro-Economics, Vol. 11, JAI Press/Elsevier Science, 159–190. Spulber, D.F. (2003). The intermediation theory of the firm: integrating economic and management approaches to strategy. Managerial and Decision Economics 24, 253–266. Spulber, D.F., C.S. Yoo (2005). Network regulation: the many faces of access. Northwestern University, Working Paper. Stahl, D.O. (1988). Bertrand competition for inputs and Walrasian outcomes. American Economic Review 78, 189–201. Townsend, R.M. (1978). Intermediation with costly bilateral exchange. Review of Economic Studies 55, 417–425. Tutte, W.T. (2001). Graph Theory. Cambridge University Press, Cambridge. United States Census Bureau (2000). 1997 Economic Census, Wholesale Trade, Geographic Area Series. U.S. Department of Commerce, Washington, DC, March. Vickrey, W. (1961). Counterspeculation, auctions and competitive sealed tenders. Journal of Finance 16, 8–37. von Neumann, J., O. Morgenstern (1944). Theory of Games and Economic Behavior. Princeton University Press, Princeton, NJ. Watts, D.J. (2003). Six Degrees: The Science of a Connected Age. Norton, New York. Weill, P.-O. (2005). Leaning against the wind. Department of Finance, University Stern School of Business, New York, Working Paper. Yavas, A. (1992). Marketmakers versus matchmakers. Journal of Financial Intermediation 2(March), 33– 58. Yavas, A. (1994a). Middlemen in bilateral search markets. Journal of Labor Economics 12(July), 406– 429. Yavas, A. (1994b). Economics of brokerage: an overview. Journal of Real Estate Literature 2(July), 169– 195. Yavas, A. (1996a). Search and trading in intermediated markets. Journal of Economics and Management Strategy 5, 195–216. Yavas, A. (1996b). Matching of buyers and sellers by brokers: a comparison of alternative commission structures. Real Estate Economics 24, 97–112.
Terrence Hendershott, Ed., Handbooks in Information Systems, Vol. 1 Copyright r 2006 by Elsevier B.V.
Chapter 4
Organization Structure
Thomas Marschak Haas School of Business, University of California, Berkeley
Abstract This chapter concerns formal models of organizations that regularly acquire information about a changing environment in order to find actions which are appropriate to the new environment. Some or all members of the organization are specialists. Each of them privately learns something about a particular aspect of the new environment. The organization operates a mechanism, which assembles relevant pieces of the specialists’ private observations and uses the assembled information to obtain the desired new action. The mechanism has various informational costs and they are measured in a precise way. The research seeks to characterize mechanisms that strike an appropriate balance between informational cost and the performance of the mechanism’s final actions. As costs drop, due to improved information technology, the properties of good mechanisms, and hence the structure of the organizations that adopt them, may change. The chapter starts by examining research in which the organization’s members reliably follow the mechanism’s rules and so incentives are not an issue. It then turns to research in which each member is self-interested and needs an inducement in order to make the informational efforts that the mechanism requires. A number of unmet research challenges are identified.
1
Introduction
This chapter concerns organizations that acquire information about a changing environment in order to take appropriate actions. The term ‘‘organization’’ is used in many disciplines and is applied to many groups of persons. The term covers government agencies, markets, entire economies, firms, nonprofit institutions, the users of the Internet, the 201
202
T. Marschak
firms in a supply chain, and so on. We shall be concerned with models of organizations where some or all members are specialists: each of them privately learns something about a particular aspect of the organization’s newly changed environment. The organization then adjusts its actions. It seeks new actions that are appropriate to the new environment. It could find them by ‘‘direct revelation’’ (DR)—collecting in one central place all the information about the changed environment that the members have privately collected, and using it to obtain a new action. But that would be wasteful: much of the transmitted information would not be needed. Instead, the organization balances the benefit of appropriate actions against the costs of learning about the current environment, transmitting some of what has been learned, and using the transmitted information to choose new actions. Advances in information technology (IT) may reduce those costs, and that may change the structure of successful organizations. In all the models discussed in this chapter, there is some precise measure of one or more informational costs. The members of our modeled organization might be totally self-interested, or they might behave like loyal and selfless team members (or wellprogrammed robots), choosing their informational efforts and their actions so as to contribute in the best way to a common goal. In either case, we consider some appropriate measure of the organization’s ‘‘gross’’ performance, before informational costs are subtracted. For the self-interested traders in a market, gross performance might be closeness to Pareto optimality or perhaps the sum of individual utilities. For a multidivisional firm whose divisions are loyal and selfless, gross performance might be total profit. That might also be our gross performance measure when each division pursues its own interests. We are concerned with the structure of organizations whose gross performance is high given the informational costs they incur. The research discussed here approaches the problem in the style that is usual in economic theory: a formal model, with well-defined concepts, is studied, in order to find the conditions under which certain interesting conjectures can be shown to hold. Typically, the conjectures arise from certain empirical observations, which suggest loose claims. When it comes to organizations and information, empirical work is hard and scarce, the issues are complex, and the gap between a general empirical claim and a tractable formal model might be very large. Consider, for example, the following loose claim which appears to date back almost 50 years, to the time when dramatic advances in IT first loomed on the horizon: As IT advances, firms (and perhaps other organizations) will become ‘‘flatter’’: middle management will fade away, and decisions will be made centrally, at the top of a flat hierarchy. (That paraphrases the conjectures in Leavitt and Whistler, 1958.) Or consider a much more recent and quite different conjecture: As IT advances, firms will find it advantageous to adopt decentralized modes of organization, in which unit managers are given wider
Ch. 4. Organization Structure
203
authority.1 Finally, consider a classic claim in economics, very much older than the preceding two claims: For organizations whose task is to allocate resources efficiently, the only informationally cheap allocation mechanisms are those that use prices, since the ‘‘amount’’ of information transmitted in such mechanisms is exactly what is needed to find an efficient allocation; in any mechanism that achieves the same thing without prices, the information transmitted exceeds what is needed. The formal modeler who wants to explore these claims faces major challenges. Terms have to be defined precisely, while still capturing informal usage. (How, for example, should we define ‘‘decentralized’’?) The informal claim has to be stated in a precise manner. A way of showing that appropriate (and plausible) assumptions imply the formalized claim has to be developed. When the modeler is finished, his demonstrated propositions may be far from those that existing empirical work can support or refute. Even so, the future empirical study of organizations might be usefully guided by some of the modeler’s results. In the formal models which we consider, the organization uses a mechanism to obtain appropriate new actions once its members have learned about the organization’s newly changed environment. The mechanism uses messages and it has informational costs, which are measured in a precise way. We are particularly interested in informationally cheap mechanisms, and we often take the viewpoint of a ‘‘designer’’ whose task is to construct an informationally cheap mechanism that generates actions meeting the organization’s goals. There is a vast literature on mechanisms, but only a small part of it makes informational costs explicit. That is the part with which we shall be concerned. We interpret the term ‘‘organization structure’’ as referring to the mechanism the organization uses, or to certain properties of the mechanism. Some examples of structural questions: Are the mechanism’s message flows arranged in a hierarchical pattern? Are the actions it generates chosen by one central person or are the action choosers dispersed? Is each external environment variable observed by a single person, who is the sole specialist in that variable, or is each variable observed by several persons?2 We discuss past research as well as new paths.3 A number of unmet Research Challenges are identified. In Section 2, we consider organizations whose members reliably follow the rules of the mechanism which the 1 That conjecture is consistent with the following statement in Bresnahan et al. (2002), which performs a careful analysis of IT adoptions and organizational characteristics in a variety of industries: ‘‘IT use is more likely to be effective in organizations with a higher quality of service output mix, decentralized decision-making, and more skilled workers.’’ See also Bresnahan et al. (2000). 2 There is a literature which refers to the former case as the ‘‘U-form’’ (unitary form) and the latter case as the ‘‘M-form’’ (multidivisional). See, for example, Harris and Raviv (2002) and Creti (2001). 3 Some new (unpublished) results are reported in Section 2.7 (on ‘‘speak-once-only mechanisms’’) and in Section 3.3 (on networks of self-interested decision-makers, who bear the network’s informational costs).
204
T. Marschak
designer has constructed, whatever those rules may be. The designer is not concerned with incentive issues. In Section 3, the members become selfinterested and the designer has to take incentives into account. In Section 4, we look very briefly at some formal models in which the primitive terms are no longer ‘‘external environment’’ and ‘‘organizational action.’’ Section 5 offers a quick retrospective impression. 2 Goals, mechanisms, and informational costs: the ‘‘incentive-free’’ case, where individuals obey the designer’s rules without inducement Consider an organization composed of n persons. The organization confronts a changing environment e ¼ (e1, y ,ek). The set of possible values of each ej is denoted Ej. The set of possible values of e is denoted E and is the Cartesian product E1 y Ek.4 In many settings we will have k ¼ n, and ej will describe person j’s local environment, which he privately observes. At any moment of time, the organization has in force an action a. The action is a vector (a1, y ,am). The set of possible values of each a‘ is denoted A‘. The set of possible values of a ¼ (a1, y ,am) is denoted A and is the Cartesian product A1 y Am.5 In many settings, m will equal n and a‘ will be the action variable for which person ‘ has responsibility. 2.1 Two general frameworks for judging the organization’s actions 2.1.1 First framework: there is a performance function on A E Suppose there is a real-valued function F on A E which measures the organization’s performance, when the current environment is e and the action in force is a. The organization, or its designer, wants F (a, e) to take high values. If e never changes and is known to everyone, then the problem reduces to that of finding, once and for all, an a in A which maximizes F for the perfectly known and unchanging e. We may study that optimization problem, and it may be challenging. But if we do so, we strip away the organizational-design issue, and the question of how information about e might be obtained and who might obtain it. Accordingly, we shall suppose that e varies. We are then in a position to define the organization’s task. Its task is to learn something about the current e, so that it can find an appropriate action a. It seeks to strike a good balance between the costs of learning about e and the value of F (e, a) that it achieves when it chooses an action a that is best, given the information about e that it has collected. 4 There are settings in which one has to relax the requirement that E be the Cartesian product. For example, each ej may be a parameter which identifies a production technology in a location j, i.e., it identifies the input/output bundles that are feasible in that location. It may be that because of externalities between the locations, some k-tuples belonging to the Cartesian product E1 y Ek cannot occur. 5 Again, in some settings one would want to relax the requirement that A be the Cartesian product.
Ch. 4. Organization Structure
205
(Here F has to have properties guaranteeing the existence of a best action.) A mechanism is used by the organization to fulfill its task. We defer, until Section 2.2, formal definitions of ‘‘mechanism,’’ and of the informational costs associated with a mechanism. Note that this framework covers the case where each component of e is a signal, privately observed by some person, about a random variable x, whose probability distribution is common knowledge. The true value of x, which does not become known until after the action has been taken, determines a payoff W (a, x) for every action a. Then we may let F (a, e¯) be the conditional expectation E (W(a, x)|e ¼ e¯). 2.1.2 Second framework: for each environment e in E, there is a satisfactory (goal-fulfilling) set of actions Suppose now that for every e in E, there is a set of satisfactory actions (a subset of A), denoted G (e), where G may be called the goal correspondence. Any action in G (e) meets the organization’s goal for the environment e. The organization, or its designer, asks: Is there a mechanism which generates a satisfactory action for every environment e in E, and what are the informational requirements of such a mechanism? In some settings, any action in G (e) has to meet certain constraints, which are defined by e. In other settings, one starts by defining the performance function F of the first framework. Then an action a belongs to G(e) if and only if F(a, e) is within some given distance, say d, of its maximal value. Thus for any fixed e¯ in E, we have G (e¯) ¼ {a AA:|F (a, e¯)F (a, e¯)|rd for all aAA}. 2.1.3 Three examples First example: an organization that provides health services. Suppose each person iA{1, y , n} is a diagnostician. Each week he examines a group of Ti patients (who are examined by no one else). For the tth patient, he correctly determines a diagnostic category, say dti. Thus the week’s local environment for i is ei ¼ (di1, y , d iT i ). The organization’s action for the week is an assigment of a treatment modality rti to each patient t inspected by diagnostician i. (We do not specify who chooses the modality.) The possible modalities comprise the finite collection Q. So the organizational action a belongs to the set A ¼ fððr11 ; . . . ; r1T 1 Þ; . . . ; ðrn1 ; . . . ; rnT n ÞÞ : rit 2 Q for all ði; tÞg. Let pe (a) denote the proportion of patients who, X weeks later, are found to be ‘‘significantly improved’’ given that the observed diagnostic vector is e and the chosen treatment vector is a. For every (e, a) in E A, the proportion pe (a) is a random variable with a known probability distribution. In our first framework we may have a performance function F, where F (a, e) is
206
T. Marschak
a weighted sum of (i) the total cost, denoted C (a), of the treatment vector a and (ii) the expected value of pe (a). In the second framework, we have a goal correspondence G, which might, for example, assign to every e the set of actions for which cost does not exceed an upper bound C, and at the same time the probability that the proportion of significantly improved patients exceeds a specified lower bound p is at least P (with 0oPo1). Thus, GðeÞ ¼ a : CðaÞ C ; P probfpe ðaÞ p g . Second example: a multidivisional firm. Now person i is the Manager of a production facility which produces product i. The environment component ei is the current week’s cost function for that product: it costs ei (q) to produce the quantity q. The week’s cost function ei is known with certainty as soon as the week starts. There are n action variables a1, y , an, where ai is the current week’s quantity of product i. (We suppose that any positive quantity is feasible.) The products are related, and the price at which the quantity ai of product i is sold depends on the entire vector of quantities a ¼ (a1, y , an). The price is denoted ri (a). For the function F, measuring the week’s performance (profit), we have n X Fða; eÞ ¼ ½ri ðaÞ ai ei ðai Þ. i¼1
It may be informationally costly to gather the information about the current e that is needed in order to find an F-maximizing a. If so, it may be appropriate to consider the second framework. The goal correspondence G might specify, for each e, the set of actions for which profit exceeds a lower bound F. Thus GðeÞ ¼ fa : F ða; eÞ F g. Third example: an exchange economy. This organization’s n members are consumers in an L-commodity economy with trade but no production. Person i has a (privately observed) local environment ei ¼ (Ui, wi), where Ui is a utility function defined on all vectors with L real nonnegative components and wi ¼ (wi1, y , wLi) is the vector of person i’s (nonnegative) initial endowments of the L commodities. An action a specifies trades. It is an nL-tuple a ¼ ((a11, y ,aL1),y,(an1, y ,aLn)), where ai‘ may be positive (an addition to i’s endowment of commodity ‘), negative (a deduction), or zero. Let ai denote (ai1, y , aiL). An action a is feasible P n for ei ¼ (e1, y , en) if ai‘Zwi‘ for all ‘ and the trades balance, i.e., i ¼ 1 a ¼ 0. The set of actions that are feasible for e is denoted Ae. A possible performance function F is as follows: FðeÞ is a value of a which maximizes
n X i¼1
U i ðwi þ ai Þ on Ae :
Ch. 4. Organization Structure
207
That performance function would be appropriate if the designer of resource-allocating schemes is willing to compare utilities across individuals and to take their sum as an appropriate standard. On the other hand, it is more conventional to take Pareto optimality and individual rationality as the goal. A trade vector a is individually rational for e if U i ðwi þ ai Þ U i ðwi Þ for all i. The trade vector aAAe is Pareto optimal for e if 8 > < for all i and all a¯ in Ae ; the following holds : ‘‘U i ð¯ai þ wi Þ4U i ðai þ wi Þ’’ implies that for some j we have > : ‘‘U j ð¯aj þ wj ÞoU j ðaj þ wj Þ’’:
(1)
ð2Þ
Note that for a given environment e ¼ ((U1, w1), y , (Un, wn)), conditions (1) and (2) are restrictions on a alone. To study the informational requirements of schemes that achieve this goal, the second framework is needed since, for a given e, there may be more than one trade a that is feasible (belongs to Ae), individually rational, and Pareto optimal. (We would have to restrict the Ui in order to guarantee uniqueness.) Formally, the goal correspondence is: Gða; eÞ ¼ a 2 Ae : a satisfies ð1Þ and ð2Þ given e . 2.2 How the organization finds its current action when incentives are not an issue We now consider mechanisms which the organization may repeatedly use in order to find its current action. A mechanism requires the transmission of messages. In our first framework, the organization (or its designer) seeks a mechanism which strikes a good balance between the performance measure and the mechanism’s informational costs. In the second framework, it seeks a mechanism which always yields goal-fulfilling actions and is, at the same time, informationally cheap. We start with the assumption that once the mechanism has been chosen, all members of the organization reliably follow its rules. They may be thought of as robots. So we need not worry about designing the mechanism so that each member will want to follow the mechanism’s rules. Incentives are introduced in Section 3. 2.2.1 Decentralized many-step broadcast mechanisms which obtain the organization’s action at the final step Until we reach Section 2.7, where the designer decides who shall observe a given environment variable, we will assume that our n-person organization’s current environment e is a vector (e1, y , en), and the local environment ei is
208
T. Marschak
privately observed by person i. The possible values of ei comprise the set Ei, and the possible values of e comprise the Cartesian product E ¼ E1 y En. There is a set A of possible organizational actions. In our first characterization of a mechanism, we shall suppose that it proceeds in a sequence of steps. At each step each person i broadcasts or announces an individual message to everyone. The vector of n individual messages is simply called a message. Person i’s announcement at a given step (his part of the broadcast message) is a function, denoted fi, of the preceding broadcast message and of ei. But the variable ej, for any j6¼i, does not directly enter the function fi. The privacy of every person is preserved. Others can only learn about the current value of ei indirectly, through the broadcast message. ‘‘Informational decentralization’’ is an alternative term for privacy preservation.6 Suppose that, for a given e, the message m¯ has the property that once it is broadcast, the next broadcast message is again m¯. Then m¯ is called an equilibrium message for e or a stationary message for e. When an equilibrium message, say m, has been reached, the sequence stops and the organization takes the action h (m). The function h: M-A is called the outcome function. Formally, let Mi be i’s individual message space, i.e., the set of individual messages that person i is able to announce. Then the message space (the set of possible messages) is M ¼ M1 y Mn. At step t, person i broadcasts the message mit ¼ fi(mt1, ei), where mt denotes (mt1, y , mnt). There is an initial message m0(e) ¼ (m01(e1), y ,mn0(en))AM. The message m ¼ (m1 , y , mn) is an equilibrium message for e if, for all i, we have mni ¼ f i ¼ ðmn ; ei Þ.
(3)
The quadruple /(M1, y ,Mn), (m01, y ,mn0), (f1, y ,fn),hS is an n-person privacy-preserving (decentralized) broadcast mechanism on E, with action set A, written in dynamic form. The term ‘‘broadcast’’ will often be omitted but understood, until we reach Section 2.2.9, where individually addressed messages are introduced. In many studies one ignores the message-forming functions fi. Moreover, one does not require that a message have n components, one for each person. Instead it suffices, for the purposes of the study, to define, for each e and each i, the set of messages mi (ei) (a subset of M) for which the equilibrium condition (3) is satisfied. That set can be specified without taking the trouble to specify the individual message spaces and the functions fi. Then the individual-message-correspondence form of a decentralized (privacy-preserving) mechanism on the environment set E with action set A is a 6 In Section 3.2, the term ‘‘decentralized’’ is given another meaning, related to incentives: in a ‘‘decentralized’’ organization, each person is free to pursue her own self-interest. In particular, she makes a self-interested choice as to whether or not to obey a proposed mechanism’s rules.
Ch. 4. Organization Structure
209
triple /M,(m1, y , mn),hS. Its elements are as follows. M is the message space. mi is a correspondence from Ei to M (i.e., mi is a function from Ei to the subsets of M); the mi define a correspondence m, from E to M, namely m(e) ¼ m1(e1)\ y \mn(en); a message m in m(e) is called an equilibrium message for e. m has the coverage property7, i.e., for all e in E, the set m(e) is not empty (for every e there is at least one equilibrium message). h (the outcome function) is a function from M to A. One can give the broadcast mechanism /M,(m1, y , mn),hS an interpretation that is sometimes called the verification scenario. In this scenario, we imagine a central agent who broadcasts a sequence of trial message announcements. When a message m is announced, each person i responds by saying YES if he finds, using his private knowledge of ei, that m belongs to the set mi (ei), and he says NO otherwise. The announcements stop when and only when the announcer has announced an m for which all n persons say YES. The organization then takes the action h (m). The message m lies in the set m(e) ¼ m1(e1)\ y \mn(en). In a still further condensed formalism, one does not trouble to identify the individual mi, but merely specifies the correspondence m. Then a mechanism is a triple /M,m,hS. The term ‘‘decentralized’’ or ‘‘privacy-preserving’’ is a restriction on m. It means that there exist correspondences m1, y ,mn such that m(e) ¼ m1(e1)\ y \mn(en), even though we do not identify them. Thus it is understood, without being made explicit, that mAm(e) means that person i has determined that m belongs to mi(ei), using his own private knowledge of the current ei to do so. Now suppose the mechanism designer is given a goal correspondence G from E to A, as in our second framework. Then we say that the mechanism /M, m, hS realizes G if for every e in E; ‘‘m 2 mðeÞ’’ implies ‘‘hðmÞ 2 GðeÞ’’. An important observation is that any goal correspondence can be realized by a Direct Revelation (DR) mechanism. In a DR mechanism each person i reveals his current ei, i.e., his announced message mi belongs to Ei. An action in G (e) is taken once a complete description of the current e is assembled. Formally, we have M¼E mi(ei) ¼ {ei} h(m)AG(e). 7 In an alternative terminology, introduced by Hurwicz (1960) in an early and fundamental discussion of mechanisms, the mechanism is called decisive on E if it has the coverage property.
210
T. Marschak
Fig. 1.
A three-message two-person broadcast mechanism.
The only equilibrium message for e is e itself. A DR mechanism is, of course, informationally costly. On the other hand, it has the merit that it can reach an equilibrium message in just one step.8 Moreover DR mechanisms play a central role in the incentive literature. That literature often confines itself to DR mechanisms, on the ground that any mechanism can be rewritten as a DR mechanism (the ‘‘revelation principle’’). That may be a correct claim, but it is not useful if one seeks mechanisms that are informationally cheap. 2.2.2 A three-message example wherein messages may be visualized as rectangles that cover E The organization has two persons. For person i, the set of local environments is the real interval Ei ¼ [0, 1]. Consider the three-message mechanism portrayed in Fig. 1.
8 Define the initial message to be m0(e) ¼ (m01(e1), y ,mn0(en)) ¼ (e1, y ,en), and let person i’s messageforming rule have the property that
f i ðm; ei Þ ¼ mi if and only if mi ¼ ei . Then the initial message is already an equilibrium message. The message formed at Step 1 just repeats it. At Step 1, the action is taken, and it belongs to G (e).
Ch. 4. Organization Structure
211
The message space is M ¼ {m1, m2, m3}, where mj identifies the rectangle labeled mj. Then 8 for 0 e1 o12; fm ; m g > < 1 3 1 m1 ðe1 Þ ¼ fm1 ; m2 ; m3 g for e1 ¼ 2; > : fm ; m g for 1oe 1: 2
3
8 fm ; m g > < 1 2 m2 ðe2 Þ ¼ fm1 ; m2 ; m3 g > : fm g 3
2
1
for 0 e2 o34; for e2 ¼ 34; for 34oe2 1:
Let the outcome function h be the following: 5 9 11 hðm1 Þ ¼ ; hðm2 Þ ¼ ; hðm3 Þ ¼ . 8 8 8 It is easy to verify that the mechanism /M, (m1, m2),hS so defined realizes the following goal correspondence: 5 GðeÞ ¼ a : ja2ðe1 þ e2 Þj . 8 That is the case because h (mj) is the value of e1+e2 at the center of the rectangle mj. Call the center (ej1, ej2). The largest value taken by the distance jðej1 þ ej2 Þ ðe1 þ e2 Þj; over all e in the rectangle mj, occurs at the ‘‘northeast’’ and ‘‘southwest’’ corners of the rectangle. At those corners the distance equals one quarter of the rectangle’s perimeter. All three rectangles have the perimeter 5/2. The above goal correspondence belongs to a class of goal correspondences Gd, where d>0 and Gd (e) ¼ {a:|a(e1+e2)|rd}. To interpret this class, go back to our first framework. Suppose that aˆ(e) is the organizational action which uniquely maximizes a performance function F, defined on A E, where the set A is the positive reals and (as in our example) E ¼ [0, 1] [0, 1]. Suppose that it is very costly for the action taker to learn the exact current value of e (as he would in a DR mechanism). Instead the action taker only learns the rectangle in which e lies. Having learned that the rectangle is mj, he takes the action h (mj). As a result there is an ‘‘error’’|aˆ(e)h(mj)|at each e in the rectangle mj. It is straightforward to verify that for any correct-action function aˆ, no outcome function h achieves a lower maximum error (with respect to aˆ), on any given rectangle, than the function which assigns to that rectangle an action that is midway between the minimum of aˆ on the rectangle and the maximum.9 So a mechanism which minimizes error on each rectangle will use that outcome function, and such a mechanism minimizes error on all of E. 9 That statement holds as well if the rectangle is a ‘‘generalized’’ one, i.e., it is the Cartesian product of its e1-projection and its e2-projection and may consist of disjoint pieces.
212
T. Marschak
In the Fig. 1 example, aˆ equals e1+e2 and our ‘‘midpoint’’ action is the value of e1+e2 at the rectangle’s center. The maximum error of our mechanism (relative to the true e1+e2) is 5/8. It is natural to ask: Is our value of d, namely d ¼ 5/8, the smallest d such that Gd can be realized by a threerectangle mechanism? Is there some other three-message mechanism in which the maximum error (relative to the true value of e1+e2) over all e in E is less than 5/8? The answer turns out to be NO. The argument which establishes that fact has not yet been generalized to the case of k messages. We do not know, in general, the smallest maximum error (relative to the true e1+e2) that is achievable by a k-message (k-rectangle) mechanism.10 2.2.3 The ‘‘rectangle’’ definition of a broadcast mechanism The preceding example suggests that we can define a decentralized (privacy-preserving) mechanism by specifying a covering of E ¼ E1 ? En, provided that the sets in the covering are generalized rectangles, i.e., each is the Cartesian product of its n projections. Let S be such a covering of E. Its typical element, denoted sm, is a generalized rectangle, i.e., it is a Cartesian product, sm ¼ sm1 ? smn, where smi is a subset of Ei. The collection of possible values of the index m is denoted M. To complete our rectangle definition of mechanism, we only need an outcome function h from M to A. Then a broadcast mechanism is defined by the triple /M, S, hS, with S ¼ {sm}mAM. We obtain the /M,(m1, y , mn),hS specification from the /M, S, hS specification, by letting mi(e) be the set {mAM: eiAsmi}. We obtain the /M, S, hS specification from the / M,(m1, y , mn),hS specification by letting smi be the inverse of mi, i.e., smi ¼ {eiAEi : mAmi(ei)}. The verification scenario provides one way to interpret a broadcast mechanism that is specified in the /M, S, hS form. Imagine a central announcer who displays successive rectangles sm to all n persons. Each responds with YES if he finds that his privately observed ei lies in the projection smi and NO otherwise. When the announcer has found a rectangle sm to which all say YES, then he takes the action h(m).
10 For the general case, one first has to establish that nothing is lost by confining attention to rectangles (such as the three in Fig. 1), which are ‘‘proper,’’ rather than being generalized rectangles (each the Cartesian product of its two projections) consisting of disjoint pieces. We next have to argue that we lose nothing by further confining our attention to proper rectangles of equal perimeter. (Recall that the maximum error on a rectangle, relative to the true e1+e2, equals its quarter-perimeter.) That can be shown for our three-rectangle case and certain other cases, but a general argument, for arbitrarily many proper rectangles, is not available. Finally, we have to calculate the smallest obtainable perimeter when we cover E (the unit square) with k proper rectangles of equal perimeter. For our case (k ¼ 3) that can indeed be shown to be 5/2, as in Fig. 1. For general k, there is no known formula giving the smallest obtainable common perimeter. There is a conjectured formula, and bounds on the distance between that formula and the unknown true one have been obtained. See Alon and Kleitman (1986).
Ch. 4. Organization Structure
213
2.2.4 A broadcast mechanism in ‘‘agreement function’’ form In the verification scenario, as we have described it so far, person i responds to an announced message mAM by inspecting his privately observed ei and saying YES if he finds that m lies in the set mi(e). In a number of settings it is useful to be more explicit about person i’s procedure, by specifying a function which he computes. Let gi be a function whose domain is M Ei and whose range is a finite dimensional Euclidean space. Let person i say YES to the message m if he finds that gi(m, ei) ¼ 0. We may call gi person i’s agreement function. When gi(m, ei) ¼ 0, we may think of person i’s YES announcement as ‘‘agreement with’’ the message m. A mechanism in agreement-function form is a triple /M,(g1, y , gn),hS. We obtain the /M,(m1, y , mn),hS form from the /M,(g1, y , gn),hS form by specifying that mi (ei) is the set {mAM : gi(m, ei) ¼ 0}. We obtain the /M,(g1, y , gn),hS form from the /M,(m1, y , mn),hS form by choosing, for each i, any function gi which takes the value zero if and only if mAmi(ei). Now suppose that M is the Cartesian product of n individual message spaces Mi, where each Mi is a linear vector space, so that subtracting one value of mi from another is well defined. Suppose we have written the mechanism in dynamic form, i.e., we have specified a message-forming rule fi for each person i. Suppose we are interested in the action generated by the mechanism when m is an equilibrium message for e. Then we can rewrite the mechanism in agreement-function form. Let gi express i’s equilibrium condition for the rule fi. That is to say, we define gi ðm; eÞ ¼ f i ðm; ei Þ mi . When (and only when) gi(m, ei) ¼ 0, person i’s response to the announced message m ¼ (m1, y , mn) (in the dynamic version of the mechanism) is to repeat his piece of that announcement. Thus, message m is an equilibrium message for e (in the dynamic version) when and only when all n persons i find that gi (m, e) ¼ 0. 2.2.5 A summary We have identified several different ways of specifying a decentralized (privacy-preserving) n-person broadcast mechanism on the environment set E ¼ E1 ? En, which the n-person organization may use to obtain actions in reponse to a new environment e in E. The alternative specifications are A mechanism with individual messages in dynamic form. This is a triple /(M1, y , Mn), ((m01, y , mn0)), (f1 y , fn),hS. Here mi0 is a function from Ei to Mi and mi0 (ei) is i’s initial individual message when i’s local environment is ei. In each sequence of announced messages, person i forms his next individual message by using the message-forming rule fi. A mechanism with individual message correspondences. This is a triple /M,(m1, y , mn),hS, where mi is a correspondence from Ei to M.
214
T. Marschak
A mechanism in which only an equilibrium message correspondence is identified. This is a triple /M, m, hS where m is a correspondence from E to M, and it is understood that there are (unspecified) individual message correspondences m1, y , mn such that for all e we have m(e) ¼ m1(e1)\ y \mn(en). A mechanism in rectangle form. This is a triple /M,{sm}mAM,hS where each sm is a generalized rectangle in E. A mechanism in ‘‘agreement function’’ form. This is a triple /M,(g1, y , gn),hS, where gi is a function from M Ei to a (finitedimensional) Euclidean space. 2.2.6 An example: a price mechanism for an exchange economy Return now to the n-person L-commodity exchange economy discussed in Section 2.1.3. In the classic (Walrasian) mechanism for obtaining individually rational and Pareto optimal allocations, the typical message, broadcast to all n persons, consists of a nonnegative price vector p ¼ (p2, y , pL), with the price of commodity 1 (the numeraire) being one, and a proposed trade vector ((a11, y , aL1),y,(an1, y , aLn)), whose components may be positive, negative, or zero. proposed trades specified in any message m have the property that PThe n i a ¼ 0 (where ai ¼ (ai1, y , aLi)), or equivalently, for every i and every i¼1 commodity ‘: X j ai‘ ¼ a‘ . (y) jai
The prices and proposed trades in m have the further property that each person i’s budget balances,11 i.e., L X
p‘ ai‘ ¼ ai1 .
(yy)
‘¼2
Let us write the mechanism in agreement-function form. For person i, the local environment is ei ¼ (Ui, wi). Assume Ui to be differentiable and increasing in each of its arguments. The agreement function gi has L1 components, denoted gi2, y , giL. Each corresponds to one of the commodities 2, y , L. All of person i’s functions gi‘ equal an arbitrary nonzero number if i’s proposed trade vector is infeasible with regard to some com¯ i.e., if ai¯ o wi¯ (where, as (yy) specifies, ai1 ¼ S‘L¼ 2 p‘ a‘i). For modity ‘; ‘ ‘ the feasible case, where ai‘Zwi‘ for all ‘, consider the bundle that person 11 That is to say, the amount that i spends on positive additions to his endowment must equal the value of the quantities that he subtracts from his endowment and makes available to others.
Ch. 4. Organization Structure
215
i holds after the proposed trades have taken place, and consider the ratio of person i’s marginal utility for commodity ‘ (at that post-trade bundle) to his marginal utility for commodity 1. The function gi‘ equals that ratio minus the price of ‘. For an equilibrium message, each function gi‘ equals zero. That implies that for the prices p, person i’s bundle (wi1+ai1, y ,wli+aLi) satisfies the first-order condition for utility maximization P L subject ito the constraint that the bundle’s value not exceed ‘ ¼ 1 p‘ w‘ . Let U‘i(x1, y ,xL;ei) denote i’s marginal utility for commodity ‘ (the partial derivative of Ui with respect to x‘) when he consumes the bundle (x1, y , xL) if his utility function is the function Ui specified in ei. Using (y), (yy), we have the following for a message m ¼ (a, p) such that a‘iZw‘i for all ‘: for ‘ ¼ 2, y , L, P U i‘ wi1 2 L‘¼2 p‘ ai‘ ; wi2 þ ai2 ; . . . ; wiL þ aiL ; ei p‘ . gi‘ ðða; pÞ; ei Þ ¼ P U i1 wi1 2 L‘¼2 p‘ ai‘ ; wi2 þ ai2 ; . . . ; wiL þ aiL ; ei In view of condition (y), we can reduce the size of m, by deleting the proposed-trade vector of one person, say person n. In person n’s agreement i rule we replace each a‘n (where ‘ ¼ 2, y , L), with Sn1 j¼1 a‘ : We replace the L L n i commodity-1 term S‘¼2 p‘ a‘ ; with the term S‘¼2 p‘ ½Sn1 j¼1 a‘ : Then the message m is a vector of n(L1) real message variables, namely (n1)(L1) trade variables plus (L1) prices. To complete our definition of the mechanism, we have to provide the outcome function h. We let that function be a simple projection operator, i.e., h(a, p) ¼ a. If we now assume that each utility function Ui is strictly concave, then the mechanism has the coverage property: for every eAE, there exists an equilibrium message (a, p). Moreover the allocation a is feasible, Pareto optimal, and individually rational. Now consider any other mechanism whose equilbrium actions (trades) are also individually rational and Pareto optimal. Under what further restrictions on the rival mechanism can we claim that its message space cannot be ‘‘smaller’’ than that of the mechanism we have just constructed? In particular, if the rival mechanism’s messages are again real vectors, when can we claim that those vectors cannot have fewer than (n) (L1) components? That is a well-studied question. We shall return to it in Section 2.2.8. 2.2.7 Another example: a price mechanism for a firm with managers and a resource allocator12 In this organization, persons 1, y , n1 are Managers and person n is an allocator. Manager j is in charge of nj activities. An activity uses resources and it generates profit. There are L resources and the quantity e‘n of 12
This example is discussed in Ishikida and Marschak (1996).
216
T. Marschak
resource ‘ is available to the firm. Manager j operates each of his nj activities, say activity k, at some level xki (a nonnegative real number). We assume nj>L, j ¼ 1, y , n1. When Manager j’s activity-level vector is xj ¼ (xj1, y , xnjj ), he contributes ej0(xj) to the organization’s profit and he uses the quantity e‘j(xj) of resource ‘, for ‘ ¼ 1, y , L, provided that the Allocator gives him those resource quantities. In each period there are new resource availabilities e‘n, and these become known to the Allocator. For each Manager j, there are also new profit functions ej0 and new resource-requirement functions ej‘, and those become known to Manager j. So the Allocator’s local environment is an L-tuple of resource availabilities, namely en ¼ (en1, y , eLn), and Manager j’s local environment is an (L+1)tuple of functions, namely ej ¼ (ej0,ej1, y , eLj). The organization’s action is an allocation vector y ¼ ((y1, y , yL1),y,(yn1, y , yLn1)), where yj‘ is the quantity of resource ‘ allocated to Manager j. The action y meets the organization’s goal if it permits the managers to choose activity-level vectors that maximize the firm’s total profit subject to the current resource availabilities. So the goal correspondence is G, defined by GðeÞ ¼ y : some nonnegative ðx1 ; ; xn1 Þ satisfies ej‘ ðxj Þ ¼ yj‘ ; j ¼ 1; . . . ; n 1; ‘ ¼ 1; . . . ; L, and maximizes
n1 X
ej0 ðxj Þ
j¼1
subject to
n1 X
)
ej‘ ðxj Þ en‘ ; ‘ ¼ 1; . . . ; L .
j¼1
Now suppose that the sets of possible local environments are as follows. For each Manager j E j ¼ fej : ej0 is strictly concave and differentiable; ej‘ is convex and differentiable; ‘ ¼ 1; . . . ; Lg; þ
while for the Allocator we have E n ¼ RL : Assume that for every e in E ¼ E1 ? En, the set G(e) is nonempty. Then we can construct a mechanism / M, m, hS which uses prices and realizes G on E. To do so, consider a vector p of nonnegative resource prices, (p1, y , pL), and, for each Manager j consider the following local problem: j
find x so as to maximize
ej0 ðxj Þ
L X
p‘ ej‘ ðxj Þ subject to xj 0:
‘¼1
Let Sj (ej, p) denote the set of solutions to that problem. We use the individual-message-correspondence form /M,(m1, y , mn),hS þ to define our mechanism. The message space M is RnL : The typical message,
217
Ch. 4. Organization Structure
broadcast to everyone, is a pair m ¼ (p, y), where p is a price vector and y is a proposed allocation vector. For Manager j ¼ 1, y , n1, define mj ðej Þ ¼ fðp; yÞ 2 M : for some xj in Sj ðej ; pÞ and for all ‘ ¼ 1; . . . ; L, we have ð1Þ ej‘ ðxj Þ yj‘ and ð2Þ p‘ ðyj‘ ej‘ Þ ¼ 0g. For the Allocator, define mn ðen Þ ¼
( ðp; yÞ 2 M :
n21 X
yj‘ en‘
j¼1
and p‘ en‘
n21 X
yj‘
!
) ¼ 0; ‘ ¼ 1; . . . ; L .
j¼1
Finally, the outcome function is a projection operator (just as it was in the exchange-economy price mechanism): h (p, y) ¼ y. It is quickly verified that if (p, y) is an equilibrium message for e (i.e., (p, y)Am1(e1)\ y \mn(e1)), then y, together with some activity-level vector (x1, y , xn1), satisfies the first requirement in our definition of G. Moreover that activity-level vector satisfies the Kuhn–Tucker conditions associated with the maximization described in the second requirement of our definition of G. The equilibrium vector p is the vector of Lagrange multipliers in those Kuhn–Tucker conditions. So we have yAG(e). Under our assumptions on E, a Kuhn–Tucker solution exists for every e in E. That means that for every e in E, there exists an equilibrium (p, y). So our mechanism has the coverage property. We conclude that our price mechanism indeed realizes our goal correspondence G on the environment set E. Its message space has dimension nL. It is natural to ask: Is there another mechanism (with appopriate regularity properties) which also realizes G on E and does so with a message space of dimension less than nL? The next section deals with questions of that sort. 2.2.8 Using the ‘‘uniqueness property,’’ or ‘‘fooling sets,’’ to obtain a useful lower bound to the message-space size of a ‘‘smooth’’ goal-realizing broadcast mechanism The best-known cost measure for a broadcast mechanism is the size of its message space M. Suppose each broadcast message m is a vector of QZ1 real numbers m1, y , mQ, and M is the cube {m: AqrmqrBq, q ¼ 1, y , Q}, where AqoBq for all q. Then the natural size measure for M is its dimension, namely Q. More generally, the dimension of M is a suitable cost measure as long as M is any subset of RQ for which ‘‘dimension’’ is defined. (For example, M may be a differentiable manifold.) Even more general classes of message space have been studied, and message-space size measures have been defined for those classes as well. Whatever the definition, we typically seek to identify the broadcast mechanisms that realize a given goal correspondence G while using the
218
T. Marschak
smallest possible message space. When we do so, however, smuggling of many numbers into a single number is a basic difficulty. If, for example, we start with a mechanism in which each message is a vector of Q>1 real numbers, then we may define a new mechanism, which realizes the same goal correspondence as the original mechanism, but has messages comprised of just a single real number. One way to do this is to let the single real number be a decimal, which encodes the qth of the Q original numbers as a string composed of our decimal’s qth digit, its (Q+q)th digit, its (2Q+q)th digit, and so on. That particular smuggling trick is ruled out if, when writing the mechanism in the message-correspondence form, we require that every set mi(ei) contain an element ti(ei), where ti is a continuous function. A more elaborate smuggling trick uses the Peano ‘‘space-filling-curve’’ mapping. (See, for example, Apostol, 1957, pp. 396–398). That mapping allows us to recover the Q numbers from a single number in a continuous manner. A requirement stronger than continuity (e.g., differentiability) is therefore needed in order to exclude it. If we want to give a nontrivial meaning to the message-space minimality of a particular mechanism within some interesting class of mechanisms, then smoothness requirements on the candidate mechanisms are unavoidable. Now consider the case of a two-person organization. We are given a goal correspondence G from E ¼ E1 E2 to an action set A. Suppose we suspect that no suitably smooth mechanism can realize G with a message space of dimension less than D. Suppose further that E is a subset of a Euclidean space, and that we have found E¯, a subset of E—called a test class of environments—which has dimension D. Suppose further that G has the uniqueness property on E¯. That means that there is no action which is goalfulfilling for all four corners of the ‘‘cube’’ defined by a pair of distinct environments e, e in E¯, i.e., the four environments (e1 , e2 ), (e 1 , e2 ), (e1 , ¯ e2 ) and (e1 , e2 ). Thus for any e , e in E, the following holds: nn n nn nn n n nn If Gðen1 ; en2 Þ \ Gðenn 1 ; e2 Þ \ Gðe1 ; e2 Þ \ Gðe1 ; e2 Þa+; then e ¼ e .
Now note that if a mechanism realizes G on all of E, then, in particular, it realizes G on the test class E¯ Next, recall that a mechanism is privacy preserving: in determing whether or not to agree to a broadcast message m (or determining whether or not m lies in mi(ei)), each person i looks only at his own local environment ei. Thus if a message m¯ is an equilibrium message for both (e1 , e2 ) and (e ¯ is also an equilibrium message for 1 , e2 ), then m (e1 , e2 ) and (e1 , e2 ). So if the mechanism’s action h(m¯) is indeed going to lie in the sets G(e) and G(e), as realization of G requires, then h(m¯) lies in ¯ G(e1 , e 2 ) and G(e1 , e2 ) as well. Since G has the uniqueness property on E, it follows that e ¼ e . So a pair of distinct environments in the D-dimensional test class E¯ cannot share the same equilibrium message. Since M must contain at least one equilibrium message for all e in E¯ (the coverage requirement), we can claim, informally, that M must be at least as large as E¯.
Ch. 4. Organization Structure
219
More precisely, consider any mechanism which realizes G on E and hence on ~ So m~ is a E¯. Consider the restriction of m to E¯ and call that restriction m: ¯ ¼ mðEÞ: ¯ Now ~ EÞ correspondence from E¯ to a subset of M, namely the set mð since E¯ has the uniqueness property for G, the inverse of the correspondence m~ ¯ ! E: ¯ For example t might be the ~ EÞ is a function, which we may call t : mð ¯ even ~ EÞ; Peano mapping, which assigns a member of E¯ to every message in mð ¯ A message m in mð ¯ is an ~ EÞ: ~ EÞ though (E¯) has higher dimension than mð equilibrium message for the environment t(m)AE¯ and for no other environment in E¯. Since every environment in E¯ must have some equilibrium message in m(E¯) (by the coverage property), our function t is onto (it is a surjection). Now suppose our smoothness requirement on the candidate mechanisms is ¯ ¼ ~ be differentiable. If M, and hence mðEÞ that the function t, the inverse of m; ¯ ¯ ~ EÞ; had a smaller dimension than E, then t would be a one-to-one function mð from one set onto a second set having higher dimension. That cannot be the case if t is differentiable. (For example, the Peano mapping, while continuous, is not differentiable, so a mechanism in which t is the Peano mapping from ¯ onto the higher-dimensional set E¯ violates our smoothness requirement.) ~ EÞ mð So we have confirmed our suspicion that D is indeed a lower bound for smooth mechnisms which realize G on all of E. We obtain the same conclusion for other smoothness requirements. Some of them are weaker than our requirement that the function t (the inverse of the message correspondence, restricted to the test class) be differentiable.13 Moreover, there is another way to force M to have a dimension at least as large as E¯. We can impose requirements directly on the correspondence m~ rather than on its inverse. Suppose we require m~ to be locally threaded. That means that for any neighborhood N in E¯, we can find a ~ i.e., vðeÞ 2 mðeÞ ~ continuous function u: N-M, which is a selection from m; for all e in N. The uniqueness property of E¯ tells us that for any two distinct environments e¯, e¯ in N, we have vð¯eÞavðe¯ Þ: It can be shown that this fact rules out a continuous u if M indeed has a smaller dimension than E¯.14 13
In particular, we may use the weaker requirement that t be ‘‘Lipschitz-continuous’’, i.e., there exists we have||t(m0 )t(m00 )||rK ||m0 m00 ||. (Here the symbol||x||, for K>0 such that for all m0 , m00 in mðEÞ; ¯ x ¼ (x1, y , xl), denotes max {|xj|: jA{1, y ,l}}.) The Peano mapping is not Lipschitz-continuous. 14 We may want to consider mechanisms whose message space M is not Euclidean but consists, for example, of certain infinite sequences, or of integer k-tuples, or of preference orderings (as when an environment specifies agents’ individual preference orderings and a message identifies a set of possible environments). That has motivated the study of mechanisms with message spaces that are general topological spaces. Then, instead of comparing dimensions, we use a general topological definition of the statement that one message space is ‘‘at least as large as’’ another. If M is Euclidean, then ‘‘at least as large as’’ reduces to ‘‘having a dimension no smaller than.’’ For example, one may define M to be at least as large as M if and only if there is a subspace of M which is homeomorphic to M. For each such topological definition, we seek an associated smoothness requirement on the message correspondence m used by a G-realizing mechanism, so that a smooth mechanism’s message space is at least as large as a test class E¯ having the uniqueness property for G. One such requirement is ‘‘spot-threadedness’’ of m on the test class. That is a weaker requirement than local threadedness. It means that there is an open set WDE¯ and a function q : W-M such that q(e)Am(e) for all e in W. If a G-realizing mechanism obeys that condition, while its message space and the test class E¯ are both Hausdorff and E¯ is locally compact, then the message space must be at least as large as the test class, where ‘‘at least as large as’’ has the meaning just given. The details are carefully developed in Section 2.3 of Hurwicz (1986), which concerns
220
T. Marschak
The technique extends to n-person mechanisms. Let the symbol e/ei denote the vector obtained from e(e with 1 , y ,en ) when we replace ei ei . Then the statement ‘‘the correspondence G from E ¼ E1 ? En to an action set A has the uniqueness property on the test class E¯ E’’ means that if en ; enn 2 E¯ and Gðen Þ \ \ni¼1 Gðenn =eni Þ a+; then en ¼ enn . It is interesting to note that the idea just sketched was developed independently by computer scientists and economists.15 In the computer science field known as ‘‘communication complexity,’’16 one studies dialogues between n persons that end with one person having enough information to compute a function F of n numbers, each of them privately known by one of the n persons. The dialogue is a sequence of binary strings. The dialogue changes when the privately known numbers change. One wants the worst-case dialogue to be as short as possible. If the function possesses a ‘‘fooling set,’’ then the size of the fooling set provides a lower bound to the length of the worst-case dialogue. In the terminology we have just developed, a fooling set is a set on which F has the uniqueness property. An illustration of the uniqueness technique: resource allocating mechanisms for a class of exchange economies. Let us return to the n-person L-commodity exchange economies discussed in Sections 2.1.3 and 2.2.6. Recall that person i’s local environment is a pair ei ¼ (Ui, wi), where Ui is a utility function and wi ¼ (wi1, y , wLi) is an endowment vector. As before, there is a set Ei of possible local environments ei. Recall also that the action the economy chooses is a trade nL-tuple a ¼ (a1, y , an), where ai ¼ (ai1, y , aLi). Call the action a interior for e if ai‘>wi‘ for all (i, ‘).
(footnote continued) ‘‘a strategy for obtaining minimality results.’’ See also (among others) Mount and Reiter (1974), Walker (1977), and Hurwicz and Marschak (1985). 15 Economic applications of the technique, so as to establish lower bounds to the message-space size required to achieve various resource-allocation goals, include the following papers, in each of which the lower bound is shown to be attainable by a particular mechanism that uses prices: Hurwicz (1997), Mount and Reiter (1977), Osana (1978), Sato (1981), Jordan (1982) (which shows that the only mechanisms that have minimal message space while realizing Pareto-optimality are versions of the competitive mechanism); Chander (1983), Aizpura and Manresa (1995), Calsamiglia and Kirman (1998), Tian (2004), Stoenescu (2004), and Osana (2005). On the other hand, the following papers find that realization of the resource-allocation goal requires a message space of infinite dimension: Calsamiglia (1977) (which permits increasing returns in production), Hurwicz and Weinberger (1990), Manresa (1993), Kaganovitch (2000) (which consider efficient intertemporal resource allocation), and Jordan and Xu (1999) on expected profit maximation by the managers in a firm. 16 See, for example, Lova´sz (1990), Karchmer (1989), and Kushilevitz and Nisan (1997).
Ch. 4. Organization Structure
221
Slightly modify the goal correspondence defined in Section 2.1.3 so that it is now defined by
GðeÞ ¼
a:
Xn i¼1
ai ¼ 0; for the economy defined by e; a is interior, Pareto-optimal and individually rational .
The price mechanism introduced in Section 2.2.6 has a message space of dimension n (L – 1) and realizes G on certain environment sets E ¼ E1 ? En. Can we show n(L1) to be a lower bound for any suitably smooth mechanism which realizes G on E? Yes we can, using the uniqueness technique, provided E contains a suitable test class E¯ whose dimension is n(L1). In the test class most used in the literature,17 utility functions have the Cobb–Douglas form and endowments are fixed. Specifically, if e is in E¯, then person i’s endowment wi is (1, 1, y , 1), and his utility for the bundle Q L vector ai (X1, y , XL) is ‘ ¼ 1X‘ , where the ai are positive numbers such that a1+ ? +aL ¼ 1. Thus each ei is uniquely determined by L1 real numbers and so the set E¯i has dimension L1. The dimension of E¯ is n(L1), which equals the dimension of our price mechanism’s message space. The uniqueness property of G on the set E¯ is readily shown, using the firstorder conditions that characterize the unique n interior trade vectors ai that maximize each person i’s Cobb–Douglas utility subject to the balancing constraint Sai ¼ 0. One then has a choice of several smoothness conditions to be imposed on the candidate G-realizing mechanisms. If the set of messages that are equilibrium messages for the environments in E¯ has dimension less than n(L1), then each of these smoothness conditions rules out a one-to-one mapping from that set of equilibrium messages onto the n(L1)-dimensional environment set E¯. But if the mechanism indeed realizes G on E¯, then such a mapping must exist, by the uniqueness property of G on E¯. So if the entire set E contains our test class E¯, we can rule out18 a smooth broadcast mechanism that realizes G on all of E and has a message space of dimension less than n(L1).19 17
See, among others, Mount and Reiter (1977) and Jordan (1982). For the organization considered in Section 2.2.7 (N1 Managers and an Allocator), a similar argument (given in Ishikida and Marschak, 1996) establishes that no suitably smooth mechanism can realize the goal function G (defined in Section 2.2.7) with a message-space dimension less than nL (the message-space dimension of the G-realizing price mechanism that we constructed). In our nL-dimenj sional test class, the q kth ffiffiffiffiffiactivity for Manager j (who has n oL activities) uses only the resource k and 18
earns a profit of 2ajk xjk when it is operated at level xkj Each member of the test class is defined by an nL-tuple of positive numbers (the numbers akj), so the test class has dimension nL. It is straightforward to show that G has the uniqueness property on that test class. 19 Another interesting setting for dimensionally minimal broadcast mechanisms is the allocation of one or more objects among n persons, each of whom has a private valuation for each object. Consider the case of a single object. Let ei be i’s valuation for the object and let Ei be the interval [0, H], where H>0. Let the action set be 1, y , n, where ‘‘action i’’ is the allocation of the object to person i. Let the goal be allocation to a maximal-value person, i.e., the goal correspondence G is defined by G (e) ¼ {j: ejZei} for all i. Then one G-realizing broadcast mechanism uses messages m ¼ (t, J), where t is a real number and J is an integer in 1, y , n. Person i agrees to m if and only if: (1) i6¼J and eirt or (2) i ¼ J and ei ¼ t. The
222
T. Marschak
2.2.9 Mechanisms in which messages are not broadcast but are individually addressed, and responsibility for each action variable is assigned to a selected person We have assumed thus far that all parts of a given message are broadcast. They are ‘‘seen’’ or ‘‘heard’’ by all of the organization’s n members. It may, of course, be true that while an agreement function gi has the entire broadcast message m as an argument, the function gi is sensitive to only a portion of m, namely the portion that i hears. Even though m is broadcast to everyone, we can interpret ‘‘hearing only a portion of the broadcast message m’’ as ignoring all of m except that portion. Formally, for every person i, we may be able to write every message m as a pair (mi, mi), where mi is the portion of m that i hears, and to write the agreement function gi(m, ei) as gi(mi, ei). Similarly, we may be able to write the set mi(ei) as {m ¼ mi, mi)AM : miAmi(ei)}, where mi is a correspondence from Ei to Mi and Mi is the set of possible values of mi. But if we want to permit messages to be individually addressed, and if we want to study the cost born by person i as he hears and processes the messages he receives, and responds to them by sending further messages to certain other persons, then it is more convenient to extend our previous formalism by introducing network mechanisms.20 That will also have another advantage: it will allow us to be explicit about who is responsible for a given action variable. Our mechanism concept thus far has been silent on this matter. In defining a network mechanism we may again use the agreement-function form, but an agreement function’s domain and range are now different. We start by letting M denote an n-by-n matrix of sets Mij, where Mij is the set of possible messages that i may send to j. The set Mii on the diagonal of M may be empty, or, if it is not, we may interpret the self-addressed messages in Mii as stored information. Moreover, Mik may be empty for some k6¼i. That means that i never sends a message to k. Next, let Mi denote the Cartesian product of the sets in the ith row of M, i.e., Mi ¼ Mi1 ? Min. Let Mi denote the Cartesian product of the sets in the ith column of M, i.e., Mi ¼ M1i ? Mni. Let Pi (M) denote the Cartesian product of the sets (footnote continued) outcome function h is a projection: h (t, J) ¼ J. This mechanism is one way to model a Dutch (descending) auction. Consider a subclass of E ¼ E1 ? En, namely the ‘‘diagonal’’ class E¯ ¼ {eAE:e1 ¼ e2 ¼ y ¼ en}. It is easily seen that G has the uniqueness property on E¯. But E¯ has dimension one, so it does not provide a useful lower bound for mechanisms whose messages are real vectors. The ‘‘auction’’ mechanism, however, uses both real numbers and integers. One has to be careful in choosing a cost measure, and smoothness requirements, for mechanisms of that sort. Much more challenging is the case of several objects, when each person has a valuation for each subset of the set of objects, and each person may be allocated a subset. One seeks a mechanism which finds (at equilibrium) an allocation that maximizes the sum of the valuations. Lower bounds for such mechanisms have been developed by Nisan and Segal (2005). The mechanisms considered again use both real numbers and integers in their messages. The uniqueness technique, using a counterpart of the ‘‘diagonal’’ test class, plays a central role in that study. The ideas are extended to a much larger class of allocation problems in Segal (2004). 20 They are studied in Marschak and Reichelstein (1995, 1998).
Ch. 4. Organization Structure
223
that are in the ith row or the jth column of M. Thus Pi(M) ¼ Mi {Mi/Mii}. We shall say that a message muuAMuu is heard by i if it is received by i (so that u6¼i, u ¼ i), sent by i (so that u ¼ i, u6¼i), or stored by i (so that u ¼ u ¼ i). Then Pi (M) is the set of the possible message vectors that i can hear. We shall speak of a message array mAM. Its typical component is an individually addressed message mijAMij, where mij is a vector of sij real numbers; sij may be zero. The symbol Pi(m) will denote the portion of m that i hears; Pi(m) is an element of the set Pi(M). Let the domain of person i’s agreement function gi be the Cartesian product of Ei with the set Pi(M) n of possible message vectors that i can hear, and let its range be RSj¼1 sij : The statement ‘‘gi (Pi(m), ei) ¼ 0,’’ means that person i finds the message array m to be acceptable: given his current local environment ei, and given that he has received the messages m1i, y , mni, he finds it appropriate to send the messages mi1, y , min. The message array m is an equilibrium message array for the environment e ¼ (e1, y , en) if all persons find it acceptable, i.e., gi (Pi (m), ei) ¼ 0 for all persons i.21 To complete the definition of a network mechanism, we have to specify how the action variables are chosen once an equilibrium message array is found. Let the organization have k action variables, z1, y , zk; let Zj be the set of possible values of zj, j ¼ 1, y , k; and let Z ¼ Z1 ? Zk be the set of possible organizational action k-tuples z ¼ (z1, y , zk). Partition the index set {1, y , k} into n sets (some of them may be empty), namely J1, y , Jn. The (possibly empty) set Ji identifies the action variables for which i is responsible. Those are the variables zr, where rAJi; they comprise a vector zji belonging to the set ZJ i ¼ t2J i Z t : Person i chooses the value of the action variables that are in his charge as a function of what he has heard. So he uses an outcome function hi : Pi ðMÞ ! Z ji : As before, we want the agreement functions to have the coverage property: for every eAE, there exists a message array m which is an equilibrium array for e. If coverage is satisfied, then a triple /M, (g1, y ,gn), (h1, y ,hn)S, whose elements we have just defined, is a (privacy-preserving) n-person network mechanism on the environment set E ¼ E1 ? En with action space Z ¼ Z1 ? Zk. As before, we may be given a goal correspondence G: E-Z, where the set G(e) consists of the organizational actions z which meet a certain goal when the environment is e. As before, we shall say that a given network mechanism realizes G if, for every e in E, the organizational action (h1(Pi (m)), y , hn (Pi(m)) lies in the set G(e) whenever m is an equilibrium message array for e. Note that every network mechanism has a communication graph. Its nodes are the n persons, and there is an edge between i and j if and only if at least one of the sets Mij, Mji is nonempty. 21 We can also write a network mechanism in dynamic form, in message-correspondence form, or in rectangle form, just as we can for broadcast mechanisms.
224
T. Marschak
An example: a network ‘‘price’’ mechanism for a three-plant four-person firm. Consider a four-person firm. Person 4 markets two products. He obtains revenue from the quantities Q1 and Q2 of the products, which are produced, respectively, by Person 1 and Person 2. Person 4’s privately observed local environment is a function, namely the revenue function e4(Q1, Q2). Person 1’s local environment is the cost function e1(Q1). For Person 2, cost depends not only on product quantity but also on the quantity I of an intermediate material used in production (the material is supplied by Person 3). So Person 2’s local environment is the cost function e2(Q2, I). Person 3 produces the intermediate material; his local environment is the cost function e3(I). For Persons 1 and 3, the local-environment set Ei (where i ¼ 1 or 3) is the set of all continuous convex functions from a closed interval [Ai, Bi] (with 0rAioBi) to the positive real numbers. For Person 2, the local-environment set E2 is the set of all continuous convex functions of two variables, from a set [A2, B2] [C, D] (with 0rA2oB2, 0rCoD) to the positive real numbers. For Person 4, E4 is the set of all continuous concave functions from a closed interval [A4, B4] (with 0rA4oB4) to the positive real numbers. The numbers Ai, Bi, C, D stay the same for all the environments e ¼ (e1, y , en). Now consider a network mechanism in which Person 4 sends prices u1 and u2 to Persons 1 and 2, respectively, and 1 and 2 reply with quantities Q1,Q2 that they are willing to supply to 4 at those prices. Similarly, Person 2 sends an intermediate-material price u to 3, who replies with an intermediate-material quantity I that he is willing to supply to 2 at that price. So there are six message variables: m14 ¼ Q1, m23 ¼ u, m24 ¼ Q2, m32 ¼ I, m41 ¼ u1, m42 ¼ u2. Let each of the sets Mij of possible message-variable values be the nonnegative real numbers. For all other i, j combinations let the set Mij be empty. Ignore the empty sets and note that for the typical ¯ 2 ; I¯ ; u¯ 1 ; u¯ 2 ; u¯ ), we have P4(m) ¼ (u1, u2, Q1, ¯ 1; Q message array, say m ¼ (Q Q2), P1(m) ¼ (u1,Q1), P2(m) ¼ (u2, Q2, u, I), and P3(m) ¼ (I, u). Fig. 2 portrays the message flows in this six-message-variable mechanism. The agreement rules of our price mechanism will express the usual conditions for divisional profit maximization. Consider the typical message ¯ 1; Q ¯ 2 ; I¯ ; u¯ 1 ; u¯ 2 ; u¯ Þ: Person 4 agrees with m¯ (he finds that array m ¯ ¼ ðQ g4(p1(m¯),e4) ¼ 0) if and only if his divisional profit e4(Q1, Q2)u1Q1u2Q2 ¯ 1 ;Q ¯ 2 ). He can determine whether or not that is so without is maximized by (Q knowing two components of the array m¯, namely I¯ and u¯ : Person 1 agrees if ¯ 1 ; and does not need to and only if the profit u1Q1e2(Q1) is maximized by Q ¯ ¯ ¯ know U2,Q2 ; I, or u¯ : Person 2 agrees if and only if u2e2(Q2, I) – uI is ¯ 2 ; I¯), and does not need to know u% 1 or Q ¯ 1 : Finally Person 3 maximized by (Q ¯ agrees if and only if uIe3(I) is maximized by I, and does not need to know ¯ 2: ¯ 1 , or Q u¯ 1 u¯ 2 Q The organization’s action variables are Q1, Q2, and I. (To minimize notation, we use the same symbol for the action variable as for the message variable associated with it.) For each action variable, we let the set of
Ch. 4. Organization Structure
Fig. 2.
225
A four-person network mechanism with six individually addressed message variables.
possible values (one of the sets Zk in our general definition) be the nonnegative reals. We have many choices in designing the outcome function. We may, for example, give Person 4 responsibility for the action variables Q1, Q2, while Person 3 has responsibility for I, the remaining action variable. Then we write h4 ðP4 ðmÞÞ ¼ h4 ðQ1 ; Q2 ; u1 ; u2 Þ ¼ ðQ1 ; Q2 Þ; h3 ðP3 ðmÞÞ ¼ h3 ðI; uÞ ¼ I. (The outcome function is simply a projection operator, just as it was in the exchange-economy price mechanism that we considered before introducing network mechanisms.) Under our assumptions on the Ei, our agreement functions have the ¯ 1; Q ¯ 2 ; and I¯ are the actions of an equicoverage property. Moreover, if Q librium message array, then they maximize the firm’s profit. That is to say, our network mechanism realizes the following goal correspondence: ¯ 2 ; I¯ Þ : ðQ ¯ 1; Q ¯ 2 ; I¯ Þ maximizes ¯ 1; Q GðeÞ ¼ fQ e4 ðQ1 ; Q2 Þ e1 ðQ1 Þ e2 ðQ2 ; IÞ e3 ðIÞg: Note that we may, if we wish, reverse the directions of the flows depicted in Fig. 2. We may, for example, let Person 1 send a price to Person 4, who replies with a quantity. Let the agreement rules stay as they were. Then the set of equilibrium messages for any e does not change and hence the set of
226
T. Marschak
actions obtained at equilibrium for any given e does not change. The revised mechanism again realizes G. The costs of a network mechanism. One cost measure is simply the dimension of the entire message space M, the set of possible message Parrays. Since each mij is a vector of sijZ0 real variables, we have dim M ¼ i,j sij. If there are no self-addressed messages (i.e., the sets Mii are empty), and if we think of each of the sij real message variables as requiring a ‘‘pipeline’’ between i and j, then the dimension of M is the total number of pipelines. In the preceding example, there are no self-addressed messages and there are six pipelines. But it is also of considerable interest to study the n individual communication P burdens. Person i’s burden is the number of variables he hears, i.e, fj:M ji a0g sij : That is also the dimension of the set Pi (M), so we may use the symbol dim Pi (M) for i’s burden. Note that if there are no selfaddressed messages, then dim M equals half the sum of the individual burdens, since that sum counts each pipeline twice. In the example, sij is either one or zero and the vector of individual burdens, for 1,2,3, and 4, respectively, is (2, 4, 2, 4). It is natural to ask: Is there another network mechanism which also realizes G, but does so with fewer than six message variables, and with an individual-burden vector that dominates (2,4,2,4)— i.e., one person’s burden is less than in our price mechanism and no person’s burden is higher? Once again a smoothness requirement has to be imposed on the candidate mechanisms, to avoid the smuggling of many numbers into one. We shall define one such smoothness requirement in a general way, for a class of n-person network mechanisms where (as in our three-plant example) (i) each person i’s environment is a real-valued valuation function ei whose arguments are certain action variables and (ii) the mechanism realizes a goal correpondence, in which the goal-fulfilling action variables maximize the sum of the valuation functions. (That is the case in our threeplant example if we define the valuation function for 1,2, and 3 to be the negative of the cost function, while 4’s evaluation function is his revenue function; then the firm’s profit is indeed the sum of the four valuation functions.) We start by considering the k action variables z1, y , zk. Let each set Zr— the set of possible values of the action variable zr—be a closed real interval, so that Z ¼ Z1 ? Zk is a closed cube in Rk : Next we shall say that the action variable zr is one of person i’s concerns if it enters his function ei. A given action variable may be the concern of several persons. (Thus, in our example, Person 4’s concerns are the action variables Q1 and Q2; Person 2’s concerns are Q2 and I.) Let Ai be the index set that identifies i’s concerns, i.e., Ai ¼ {r A {1, y , k}: zr enters the function ei}. Then a given vector zAi specifies a value for each of i’s concerns, and ZAi (a closed cube or closed interval) denotes the set of possible values of zAi : We now fix the concern sets Ai and we consider mechanisms and environment sets for which the
Ch. 4. Organization Structure
227
following is true: 8 The local environment set for person i is E i ¼ > < Ai ~ Ai is an fei : ei is a concave function from Z~ to Rþ g; where Z > : open convex set that includes Z Ai : (y) Now consider the goal correspondence P defined by n X ei ðzAi Þ on Zg: PðeÞ ¼ fz 2 Z : z max imizes i¼1
Consider a subclass of the environment set E, namely E¯ ¼ E¯1 ? E¯n, where each concave valuation function eiAE¯i takes a separable quadratic form. That is to say, if eiAE¯i, then X 1 a a2 Ai a a ei ðz Þ ¼ xi z yi ðz Þ , 2 a2A i
xia
a
,yi are numbers privately observed by i. So we may identify each where local environment ei in E¯i by a point in R2jAi j : (For a finite set H, we let |H| denote the number of elements it contains.) Moreover if eAE¯, then the goalfulfilling action set P(e) has a single element, since there is a unique maxP imizer of ni¼1 ei ðzAi Þ on the cube Z. We now let P(E) denote that unique maximizer. Call an element e of E¯ interior if that unique maximizer is an interior point of Z. Let t (a) denote the set of persons concerned with the action variable za, i.e., t (a) ¼ {i: aAAi}. Assume (to avoid trivial cases) that each set t (a) has at least one member. Write the action k-tuple P(e) as (P1(e), ? , Pk(e)). It is quickly checked that for an interior environment e¯AE¯, we have, for every aA{1, y , k} P a i2tðaÞ xi a P ðeÞ ¼ P a. i2tðaÞ yi We are now ready to define our smoothness requirement. Consider a network mechanism /M,(g1, y ,gn), (h1, y ,hn)S on the environment set defined in (y). Let each message mij in a nonempty set Mij be a vector of real numbers. The mechanism is smooth on the separable quadratic subset E¯ if for some interior environment e¯AE¯, there exists a neighborhood U(e¯) and a continuously differentiable function r: U(e¯)-M, such that for all e 2 Uð¯eÞ we have g1 ðrðeÞ; e1 Þ ¼ 0; . . . ; gn ðrðeÞ; en Þ ¼ 0. Thus the function r identifies an equilibrium message for each e in the neighborhood U (e¯), and that message varies in a continuously differentiable fashion as we move away from e¯. Using a variant of the uniqueness
228
T. Marschak
argument sketched in Section 2.2.8, we obtain a lower bound on each person’s communication burden. The following can be shown: Proposition A. Suppose /M,(g1, y ,gn), (h1, y ,hn)S is a network mechanism on the environment set E defined in (y), and each message mij in every nonempty set Mij is a vector of real numbers. If the mechanism is smooth on the separable quadratic subset E¯ and realizes the goal correspondence P on E, then the number of real message variables that each person hears is at least twice the number of his concerns, i.e., for each person i we have dim Pi(M)Z2|Ai|. If we now return to our four-person three-plant example, we see that the concern-set sizes are (1, 2, 1, 2) for Persons 1,2,3,4, respectively. But (as already noted) the vector of message variables heard is (2, 4, 2, 4). So Proposition A tells us that no smooth mechanism whose equilibrium messages yield actions that maximize the firm’s profit can improve on the Fig. 2 price mechanism with regard to any person’s communication burden. That is a strong result in favor of price mechanisms. Is it confined to situations that do not depart significantly from our example? The answer is not known. In particular, suppose we consider P-realizing mechanisms in which each individual burden need not be minimal but instead a weaker efficiency requirement is met. Call a mechanism efficient in a class if no other mechanism in the class has a lower burden for some person and not higher burdens for the others. The following challenge is unmet. Research challenge # 1. Consider the class of all P-realizing mechanisms on the above environment set E which are smooth on the separable quadratic subclass E¯. If such a mechanism is efficient in that class, is it always possible to write it so that it becomes a price mechanism, where each message variable can be interpreted as a price or a quantity, and the agreement functions express divisional profit maximization? Other fundamental results on network mechanisms concern the size of the overall message space M rather than the individual burdens. These results require a stronger condition than the smoothness we have defined. Call the stronger condition regularity on the separable quadratic subclass E¯.22 For regular mechanisms, there is a useful lower bound to the size of M. For any P-realizing mechanism which is regular on E¯ we can show that X dim M 2 ðjtðaÞj 1Þ. a2f1; ... ;kg
22 For regularity on E¯, we have to add the requirement that the matrix of second partial derivatives of the gi has full rank at (m¯,e¯), where e¯AE¯ is the interior environment in our previous smoothness condition and m¯ ¼ r(e¯); moreover, the rank does not change when we vary e in a neighborhood of e¯ while keeping the message array constant at m¯.
Ch. 4. Organization Structure
229
A mechanism is dimensionally minimal in a certain class if no other mechanism in the class has a smaller value of dim M. It is of particular interest to know when the communication graph of a dimensionally minimal regular P-realizing mechanism is hierarchical, i.e., the graph is a tree. That may have advantages that are related to incentives (e.g., it may facilitate ‘‘control’’). But trees may have communication costs that are higher than needed. Using the above lower bound we can characterize the situations where trees turn out to be dimensionally minimal. Before doing so, note that the communication graph of a network mechanism defines a subgraph for every subset of persons. In particular, there is a subgraph for t (a), the set of persons who are concerned with the action variable za. If the communication graph of the mechanism is a tree, then that subgraph may or may not be a tree as well. The following has been shown. Proposition B. There exists a P-realizing mechanism, that is (i) regular on the separable quadratic subset E¯, (ii) dimensionally minimal among all such mechanisms, and (iii) hierarchical, if and only if there is an n-node tree with the property that for every action variable za, the tree’s subgraph for the persons concerned with that variable is also a tree. Propositions A, B, and further propositions that we do not summarize here, all deal with environments that are valuation functions and with a goal correspondence that requires maximization of their sum. The valuation functions have a ‘‘public good’’ property, since a given action variable may enter several of them. Are there analogues of these propositions for other classes of environment sets and goal correspondences, where there are no public goods? In particular, suppose that person i’s privately observed local environment is a ‘‘revenue’’ function ei whose argument is a ‘‘private’’ action vector zi that does not enter any other ej, where j6¼i. Suppose, however, that all the action variables have to meet a common resource constraint. We require z A C, where C is some subset of a finite-dimensional Euclidean space. Suppose, in particular, that the set C can be written in the form fðz1 ; . . . ; zn Þ : r1 ðzB1 Þ 0; . . . ; rk ðzBn Þ 0g: Here the Bi are index sets and each is a subset of {1, y , n}. The functions r1, y , rk are fixed and known to all persons. Now we may view person i as being ‘‘concerned with’’ those constraint functions rt in which his own action vector enters , i.e., those for which iABt. Consider the goal correspondence P defined by n X P ðeÞ ¼ fz : z maximizes ei ðzÞ on the set Cg. i¼1
Research challenge # 2. Are there propositions characterizing those Prealizing network mechanisms which are regular (in a suitable sense) and are efficient with regard to individual burdens, or are dimensionally minimal? In particular, are there propositions which describe the situations where efficient or minimal mechanisms are hierarchical?
230
T. Marschak
2.3 Finite approximations of mechanisms whose message spaces are continua In most mechanisms studied in the literature, the message space M is a continuum. That is not surprising, since typically the mechanism’s final action maximizes some function, in which actions and environments are the arguments. Maximization is most easily studied with the tools of calculus and those tools deal with continua, not with finite sets. Thus in a classic price mechanism, a message specifies prices and proposed quantities. The first-order conditions for the required maximization can be expressed in a statement about prices and quantities, and that statement holds at the mechanism’s equilibrium messages. The message space is a continuum composed of possible prices and possible quantities. But continua are not realistic. In practice, one cannot send all the messages in a continuum (e.g., all the points in a real interval). Moreover it may take infinite time to find that message in a continuum which satisfies the required equilibrium conditions. If an organization wants to use a continuum mechanism in a practical way, it has no choice but to approximate the continuum mechanism with an appropriate finite mechanism, whose message space is a finite set. The penalty paid for such finite approximation may be an error: the actions generated (at equilibrium) by the finite approximation may differ from the goal-fulfilling actions which the original continuum mechanism generates. If we take the issue seriously, then the following question immediately comes to mind: Will the informational advantages of the original continuum mechanism be reflected in its finite approximation? In particular, if we have found (using the tools of smooth mathematics) a continuum mechanism that realizes a given goal correspondence and does so with minimal message space dimension, are finite approximations to that mechanism superior (in an appropriate sense) to finite approximations of a continuum mechanism which also realizes the goal correspondence but has a higherthan-minimal message-space dimension? Does dimension still matter when we turn from continuum mechanisms to their finite approximations? In fact, for broadcast mechanisms, a theory of finite approximations has been begun23 and several ‘‘dimension still matters’’ propositions have been established. In these propositions, the environment set is a continuum as well as the message space of the mechanism we are approximating. The view taken is that ‘‘nature’’ is able to choose the organization’s current external environment from some continuum of possible environments, but the continuum message space is ‘‘man-made’’ (for analytic convenience) and we are free to replace it with a finite message space. The first step is to define a style of approximation. Suppose we are given an n-person broadcast mechanism L on an environment set E ¼ E1 23
In Marschak (1987) and Hurwicz and Marschak (2003a, b, 2004).
Ch. 4. Organization Structure
231
? En, with action set A, where both M and each Ei are continua. In particular (as in our introductory discussion of mechanisms in D1 Section 2.2.1), let M be the Cartesian product M M Dn : Let Di Di each M be a closed Di-dimensional cube in R ; while each Ei is a closed Ji-dimensional cube in RJ i : Write the continuum mechanism L in agreement-function form: L ¼ /M,(g1,y,gn),hS. Recall that each agrement function gi has M Ei as its domain and M Di as its range. Thus each gi has P Di real-valued components, say gi1, y , gik, y , giDi. Let D denote in¼ 1Di. In a finite approximation to L, our finite message space, denoted MA is the intersection of the D-dimensional continuum message space M with a mesh-A lattice of points, which are spaced 2A apart (A>0). That lattice, denoted SD A, is the D-fold Cartesian product of the set S 2 ¼ f. . . ; 2ð‘ þ 1Þ 2; 2‘ 2; . . . ; 4 2; 2 2; 0; 2 2 , 4 2; . . . ; 2‘ 2; 2ð‘ þ 1Þ 2; . . .g. Next we replace each agreement function gi with a new function gZ i ¼ Z Z2 ðgZ i1 ; . . . ; giDi Þ: Each gik is the following two-valued function: ( for every m 2 M 2 ; gn2 ik ðm; ei Þ ¼
0
if jgik ðm; ei Þj Z;
1
otherwise;
where Z>0 is called the tolerance. Finally, we have to specify the outcome function of our finite approximation. In the simplest approach, we let the outcome function be the original one, i.e., it is the restriction of h, the outcome function in L, to the new finite message space MA (which is a subset of M). Denoting the new outcome function h0, we have h0(m) ¼ h(m). Suppose that our new agreement functions satisfy the coverage requirement: i.e., for every e A E, there exists m A MA such that gZ2 i ðm; ei Þ ¼ 0; all 0 Z2 i. Then the finite mechanism LZ2 ¼ hM 2 ; ðgZ2 ; . . . ; g Þ; h i is called the finite n 1 exact-outcome approximation of L with message mesh A and tolerance Z. To obtain it we have, in effect, rounded off the original functions gi to a specified accuracy. The accuracy is determined by the the mesh A and the tolerance Z. In an alternative approximation of L, we do not require the outcome for the message m to be exactly what it was in L. Rather we place a mesh-n lattice on the action set A, so that our finite mechanism’s action set becomes A\Sna We then choose the outcome to be a lattice point that is closest to the action chosen in L. Suppose there are a real-valued action variables and that A is contained in a closed cube in Ra : Then in the finite rounded-outcome approximation of L with message mesh A, action mesh n and tolerance Z, all elements except the outcome function are the same as
232
T. Marschak
those just defined. The outcome function hn : M 2 ! A \ S an is defined as follows: 8 the element a ¼ ða1 ; . . . ; aa Þ of A \ Sav > > > > < which is closest to hðmÞ; where distance v h ðmÞ ¼ is measured by max ðjhðmÞ ar jÞ > > r2f1;...;ag > > : and ties are broken downward: For both versions, we shall say that our finite mesh-A approximation has the minimal tolerance property if Z has the smallest value that permits coverage. (Such a smallest value can be shown to exist.) If the mechanism L is a finite approximation of the continuum mechanism L, then for any e A E, we define the error at e of L to be the worstcase distance between the (equilibrium) value of an action variable in the continuum mechanism and its value in the approximation. Let h# denote the finite approximation’s outcome function. (This is either the exact outcome of the continuum mechanism or it is a rounded-outcome approximation.) Since there are a real-action variables, the function h# has a realvalued components ha. The error at e of L is supfjh#j ðmÞ2h ¯ ¯ 2 M 2; j ðmÞj : m 2 M; m gi ðm; ei Þ ¼ 0; g2Z ¯ ei Þ ¼ 0; i ¼ 1; . . . ; n; j 2 f1; . . . ; agg. i ðm; The overall error of L is supeAE (error at e of L). We define the cost of the approximation L to be the number of messages in its finte message space. We impose some regularity conditions24 and obtain the following ‘‘dimension still matters’’ proposition. Proposition C. Consider two regular continuum mechanisms L ¼ /M,(g1, y , gn),hS and L ¼ /M,(g1 , y ,gn, hS. Each is a mechanism on the same environment set, namely the compact set E ¼ E1 ? En, and each has an action set which is a subset of Ra : The two message spaces are distinct: M is a D-dimensional subset of RD , while M is a D-dimensional subset of RD , where D>D. Let L be a mesh-A rounded-outcome minimal-tolerance approximation of L, and let L be a mesh-2¯ rounded-outcome minimal-tolerance approximation of L. Suppose that L costs no more than L. Then if A is sufficently small, the overall error of L exceeds the overall error of L: Note that the proposition does not require us to specify goal correspondences realized by the two continuum mechanisms. But it can certainly be applied to two continuum mechanisms which realize the same goal 24 In a regular mechanism each function gik is continuously differentiable. Moreover there exists a number d>0 such that for all dikA[d,d] and for all e in E, there is a unique message m satisfying all the equations gik(m,ei) ¼ dik.
Ch. 4. Organization Structure
233
correspondence. In that case it tells us, informally speaking, that if we want to come close to achieving our goal, then the low-dimensional mechanism L is a better candidate for finite approximation than the high-dimensional mechanism L. We achieve lower overall error, for a given ‘‘budget’’ if we approximate L than if we approximate L. So dimension indeed continues to matter.25 It remains open whether or not there are similar propositions for network mechanisms. Research challenge #3. If two continuum network mechanisms realize the same goal correspondence and have suitable regularity properties, but the second has a higher number of individually addressed message variables, is it better to approximate the first? (Do we achieve a smaller overall error, for a given ‘‘budget,’’ when we do so?) Returning to the case of broadcast mechanisms, a very difficult question remains unaddressed. In constructing finite broadcast mechanisms, we have confined our attention to finite mechanisms which approximate regular continuum broadcast mechanisms. Do we ignore certain efficient finite mechanisms when we do so? Research challenge # 4. Given a goal correspondence G, can there be a finite broadcast mechanism which is NOT an approximation of any regular G-realizing continuum broadcast mechanism but makes better use of a given ‘‘budget’’ (achieves lower overall error with respect to G while not using more messages) than any such approximation? 2.4 The dynamics of a mechanism Return now to the dynamic form of a broadcast mechanism, which started our discussion in Section 2.2.1. The mechanism is a quadruple /(M1, y ,Mn), (m01, y ,mn0),(f1, y ,fn),hS. It defines a difference-equation system, namely: t1 mt ¼ f ððmt1 1 ; . . . ; mn Þ; ei Þ; i ¼ 1; . . . ; n;
with an initial message m0(e) ¼ (m01(e1), y ,mn0(en)). We have been interested thus far in the achievements of the mechanism once it has reached an equilibrium message. A difficult missing piece in the story has to do with the stability properties of the difference-equation system. We would like the action taken once an equilibrium message is reached to meet a specified goal, but we would also like the difference-equation system to display some 25 Another proposition lets us be ‘‘kind’’ to the high-dimension mechanism by permitting its approximation to have the original exact outcomes, while we are ‘‘harsh’’ to the low-dimension mechanism by requiring its outcomes to be rounded off. Even so, it is better to approximate the low-dimension mechanism. This proposition, however, requires the high-dimension mechanism to have a ‘‘projection’’ property: each message is a pair (a, q) and the outcome function is h (a, q) ¼ a.
234
T. Marschak
sort of convergence to the system’s equilibria. Do the mechanism’s informational requirements (e.g., its message-space size) grow if we require stability as well as goal realization? One can construct examples where the answer is Yes.26 Some progress has been made on this question when the difference equations are replaced by differential equations. In particular, Jordan (1995) developed the following new mechanism concept.27 For each person i there is a real-valued ‘‘control message’’ ci, whose possible values comprise a set Ci. There is also a broadcast ‘‘state message’’ m ¼ (m1, y , mq), with q real components, which is continually adjusted as a function of both c ¼ (c1, y , cn) and m. It is not required that q ¼ n. We have the differential-equation system dmj ¼ aj ðc; mÞ; j ¼ 1; . . . ; q; dt where t is a time point. Note that the function aj does not have ei as an argument. The interpretation is that each person i continually adjusts his own control variable ci in a privacy-preserving manner. We do not specify who adjusts a given component of m itself, but that adjustment does not require direct knowledge of the privately observed eis. In the case where q ¼ n and i has responsibility for the message variable mi, we have a complete privacy-preserving scenario: person i observes the entire broadcast message m, adjusts his own part of m, and chooses his control variable as a function of the broadcast message and his local environment. The general question is then as follows. Suppose we are given a particular message correspondence m from E ¼ E1 ? En to the statemessage space M (the set of possible values of m). We are interested in this correspondence because it realizes some goal (i.e., if m A m(e), then there is an action h (m) which lies in a set G (e) of goal-fulfilling actions), but the goal itself is not part of the research question. Instead we ask: How large do the sets Ci have to be if the equilibria of the differentialequation system always lie in the set m(e) and the system has a local stability ci ¼ f i ðm; ei Þ; i ¼ 1; . . . ; n;
26 A two-person example, due to Reiter and discussed in Hurwicz (1986) is as follows. Person i’s local environment is a real-number pair ei ¼ (ei1,ei2). The action set is R: For each e the goal specifies a unique action for all e such that e116¼e12, namely F(e) ¼ [e11e22e21e12]/[e11e21]. If we do not require stability, we can realize the goal with a two-message-variable mechanism. The typical message is a pair m ¼ (m1, m2). The 1 t1 2 t t1 t1 1 t1 2 mechanism’s difference-equation system is mt1 ¼ 2mt1 1 e1, m2 e1,m2 ¼ m1 +m2 e2m2 e2. The outcome function is a projection: h (m) ¼ m1. But this system fails to satisfy the following local stability requirement: for m0 sufficiently close to an equilibrium value of m, the system should converge to that equilibrium. Moreover, if we seek any twomessage-variable difference equation system which realizes F at equilibrium and uses the projection outcome function, we find that if the functions f1, f2 have continuous partial derivatives, then the system is not locally stable. On the other hand, a locally stable F-realizing mechanism with four message variables can be constructed. 27 See also Jordan (1987), Mount and Reiter (1987), and Saari and Simon (1978).
Ch. 4. Organization Structure
235
property for every e? How large, in other words, is the extra informational cost of stability? Several general results characterize the required size of the Ci. One would like applications of these results to classic questions, notably the informational merits of price mechanisms. Consider, once again, the n-person L-commodity exchange economy and the Pareto-optimal message correspondence. If we construct a privacy-preserving dynamic mechanism which uses prices, has suitable regularity properties as well as local stability, and achieves Pareto-optimality at the equilibrium message, then is its total message space (i.e., M C) dimensionally minimal among all dynamic mechanisms with those properties? Much remains to be learned about this question, but for certain reasonable classes of economies, and certain versions of the dynamic price mechanism the answer is Yes. For one such class, Jordan (1995) studies dynamic mechanisms in which the message m specifies current trades, the control variable ci is a vector of i’s ‘‘demand prices’’ (marginal utilities), and the adjustment rules for m (i,.e., the functions aj) adjust the trades so that they are Pareto-improving. It is shown that if we delete the stability requirement for such mechanisms, then a lower bound on the dimension of each C is n (L–1). It is then shown that stability can be achieved without increasing the dimension of C beyond n (L–1). In other types of dynamic mechanism, the control variables are trades as well as prices. It turns out that if such a mechanism is formulated so that privacy is preserved and local stability is achieved, then C has to be very large and the stabilized price mechanism may no longer be minimal among all such mechanisms. Note that for a finite broadcast mechanism we have an upper bound on the time required to achieve equilibrium—namely the time required to announce all the messages in the finite set M, in some predetermined sequence. If M is large, that upper bound is of little interest. We may then want to choose the sequence with care, perhaps by approximating (in some suitable way) the rules of a locally stable mechanism in which the sets M and C are continua. Such approximation remains to be studied. Note also that if we truncate a difference-equation broadcast mechanism /(M1, y ,Mn), (m01, y ,mn0),(f1, y ,fn),hS after T steps, then we have defined a new privacy-preserving broadcast mechanism in which every possible broadcast message describes a possible T-step ‘‘conversation,’’ namely a proposed sequence of announcements m0, m1, y , mT. Person i agrees to the proposed conversation if he finds—given his ei, given the proposed sequence of announcements by others, and given his function fi—that the proposed sequence mi0, mi1, y ,miT is exactly what he would announce. So, using the uniqueness technique discussed in Section 2.2.8, one could study the goal correspondence realized by the T-step truncation and could ask whether there are broadcast mechanisms which realize the same goal correspondence using a smaller message space.
236
T. Marschak
2.5 Constructing an informationally efficient mechanism The informationally efficient (or cheap) mechanisms that have appeared in the literature are, in a sense, historical accidents. They are mechanisms which allocate resources so as to satisfy a goal correspondence that expresses Pareto-optimality, or perhaps profit maximization. Typically each message proposes prices and quantities, and agreement to the message by all persons means that its prices and quantities correspond to the required first-order maximization conditions. With some exaggeration one might say that the literature started by asking: ‘‘What is the precise information-related question to which ‘price mechanisms’ (or ‘competitive mechanisms’) is the answer?’’ That was a natural challenge, given the long history of sweeping but never rigorously defended claims about the price (or competitive) mechanism’s informational merits. But what if prices had not yet been discovered? Imagine looking for lowcost mechanisms among all those that realize a goal correpondence. If the goal required Pareto optimality, then a search for such mechanisms would eventually discover mechanisms that use prices. How might such a search proceed? Two new books, one by Hurwicz and Reiter (2006), and the other by Williams (2007), deal with this fundamental puzzle. It would be futile to attempt any kind of summary here. But we can roughly visualize one of the main issues in the Hurwicz/Reiter agenda by going back to the ‘‘rectangle’’ definition of a mechanism in Section 2.2.3 and the two-person, three-message example in Fig. 1 of Section 2.2.2. Suppose that our environment sets are Ei ¼ [0, 1], i ¼ 1, 2 and that we have an action variable a that takes three values, namely u, v, w. Suppose we have not yet constructed a mechanism which yields (at equilibrium) a value of the action for every e A E ¼ E1 E2. Relabel the three rectangles m1, m2, m3 in Fig. 1 as U, V, W, respectively. Let those rectangles (which overlap at boundary points) define the goal correspondence G that we want to realize. Thus GðeÞ ¼ fa : a ¼ u if e 2 U; a ¼ u if e 2 V ; a ¼ w if e 2 W g. We may call U the level set of the correspondence G for the action u, and similarly for V and W. Formally the level set corresponding to the action a is G1(a) ¼ {e A E: a A G (e)}. Consider the mechanisms which realize G and suppose that we write all of them in rectangle form, so that there is a generalized rectangle sm for every message m, and m is an equilibrium message for all the environments in sm. (Recall that a generalized rectangle is a set of environments e that is the Cartesian product of its E1-projection and its E2-projection.) There are many such mechanisms, but they all use more generalized rectangles (messages) than we need except the
Ch. 4. Organization Structure
237
three-message mechanism defined in Section 2.2.2. In that mechanism there are just three rectangles sm, namely the same three rectangles U, V, W that define the goal correspondence. An inefficient G-realizing mechanism might, for example, needlessly add a fourth message, by dividing the rectangle sm3 ¼ W into two further generalized rectangles. The three-message mechanism defined in Section 2.2.2 is efficient: it covers each level set of our goal correspondence with rectangles sm in such a way that the total number of rectangles sm is minimized. To illustrate further, suppose we modify our example by allowing just the two actions u and w and letting w be goal-fulfilling for all environments that lie in W or V. (The action u remains goal-fulfilling for all e A U.) Now the level set for the action w is no longer a (generalized) rectangle; it is now the union of W and V. Nevertheless, an efficient goal-fulfilling mechanism has to cover that level set (as well as the level set corresponding to u) with generalized rectangles sm and it has to do so in a minimal way. The efficient mechanism will again require three messages. It will cover the level set corresponding to the action w with two generalized rectangles, namely the rectangles W and V. So the search for an efficient goal-realizing privacy-preserving mechanism requires us to inspect the level sets, to find a way of covering each of them with generalized rectangles, to find a way of indexing (labeling) each of our generalized rectangles with an index m, and to do all this while keeping the size of M (the set of values if the index m) as ‘‘small’’ as possible. The basic set-theoretic properties of such a mechanismdesigning algorithm are worked out in the Hurwicz/Reiter book. The algorithm yields efficient mechanisms whatever the goal correspondence may be, whether the action set and the environment sets are finite or are continua. The book of Williams also deals with a mechanism-designing algorithm, but from a very different point of view. Smoothness requirements are imposed on the goal and on the candidate mechanisms. Tools of differential topology are used rather than purely set-theoretic tools. Some of the results imply that the agreement functions of an efficient goal-realizing mechanism can be found by solving an appropriate system of partial differential equations. Once these two books are understood, they may open a massive research agenda for the designers of practical computer-friendly algorithms that construct mechanisms (protocols). It remains to be seen, for example, whether the general results in these books will eventually allow a computer to generate protocols that yield the minimal-length dialogues studied in the computer-science communication-complexity literature (briefly discussed in Section 2.2.8). At present that literature finds bounds on the length of the dialogues but does not tell us how to construct the minimal-length dialogues themselves.
238
T. Marschak
2.6 Finding a best action rule (outcome function) once a mechanism has conveyed information about the environment to each person: the methods of the Theory of Teams The central problem studied in the Theory of Teams (Marschak and Radner, 1972) is the choice of a rule that tells each member of an organization—called a team—what action to choose, given certain information about the organization’s randomly changing environment. The rule has to maximize the expected value of a payoff function whose arguments are the environment and the team action. Mechanisms, as we have defined them, do not appear in the statement of the central problem studied in the Theory of Teams, but they are part of the story which implicitly precedes that problem.28 Here is one version of the central n-person team problem. The team has to chose an action, namely a vector a ¼ (a1, y , an), where ai is the responsibility of person i. Let Ai denote the set of possible values of ai and assume that every n-tuple a in A1 ? An is a possible team action. The team earns a (real-valued) payoff H (a, e), where e ¼ (e1, y , en) is a vector of random local-environment variables. We study a given information structure, specifying what each person knows about a given e. The set of possible values of e is denoted E. Let Zi be a function from E to a signal set Yi. Let E be a subset of a (finite-dimensional) Euclidean space, and similarly for each Yi and each Ai. A probability distribution P on E is given. In the information structure Z ¼ (Z1, y , Zn), person i observes the signal yi ¼ Zi (e) when the environment is e. The signal Zi (e) might be a vector, and ei might be one of its components. In our discussion of privacy-preserving broadcast mechanisms, each ei was automatically known to one person, namely person i. We may interpret Zi (e) as the information about the current e that person i possesses once the mechanism has terminated. Then Zi (e) indeed includes ei, but it also describes the information about the other ej that is revealed to i by the mechanism’s terminal message.29 For a given information structure Z, we consider the possible team action rules a ¼ (a1, y , an), where ai is a function from Yi to Ai. Thus the team action is (a1(y1), y , an (yn)) when the signal vector is y ¼ (y1, y , yn).30 An action rule a^ is H and the information struc team-best for the payoff function ture Z if EH ð^a1 ðZ1 ðeÞÞ; . . . ; a^ n ðZn ðeÞÞÞ; e EHðða1 ðZ1 ðeÞÞ; . . . ; an ðZn ðeÞÞÞ; eÞ 28 In Chapter 8 of The Economic Theory of Teams, there is a discussion of ‘‘networks’’, with a number of examples. One may interpret the network concept developed in that chapter as a mechanism in our sense. 29 Thus we can express an information structure as a message correspondence m, where m lies in m(e) if and only if m ¼ (Z1(e), y ,Zn(e)). 30 Appropriate measurability assumptions have to be made when E is not finite. They guarantee that (i) P implies a probability distribution on the set ai(Z1(E)) for every i and (ii) H((aiZ1(e)), y ,an(Zn(e))),e) has a finite expected value.
Ch. 4. Organization Structure
239
for all action rules a, where e again denotes expectation. A necessary condition for an action rule a^ to be team-best for H, Z is that it be person-byperson-satisfactory (pbps) for H, Z. That means that for every i and every y, the action a^ i (yi) maximizes the conditional expected value E H ð^a1 ðZ1 ðeÞÞ; . . . ; a^ i21 ðZi21 ðeÞÞ; ai ; a^ iþ1 ðZiþ1 ðeÞÞ; . . . ; a^ n ðZn ðeÞÞÞ; eÞ Zi ðeÞ ¼ yi : on the set Ai. If H is differentiable and strictly concave in a for each e, then the pbps condition is sufficient as well as necessary. Consider the case of a team with the linear-quadratic team payoff function W (a, e) ¼ 2a0 ea0 Qa. Here Ai is the real line. The random variables e1, y , en also take values in the real line and they are independently distributed with finite second moments; Q ¼ ((qij)) is an n n symmetric positive definite matrix. The function W is differentiable and strictly concave in a for each e. Accordingly the pbps condition is both necessary and sufficient. A best team action rule is linear. Its coefficients can be found by solving a system of linear equations. That permits the explorations of information structures with interesting organizational properties. (For some explorations, it is also helpful to assume that each ei is normally distributed.) For example in a ‘‘management-by-exception’’ information structure, each person i 6¼ n knows only his own ei. But person n is a manager who learns the value of every ej whenever that ej lies in a specified ‘‘exceptional’’ region. A best team action rule a will take advantage of the manager’s exceptional information. In a variant of this structure, an ‘‘emergency conference’’ of all n persons is called whenever some person j observes an exceptional value of ej. When that happens, all persons learn that exceptional value. We can vary the exceptional regions and in each case we can compute the structure’s ‘‘gross’’ performance, i.e., expected team payoff when a best action rule is used. The gross performance of one interesting structure can be compared to that of another. The cost of each structure, however, needs to be measured in some consistent way if we are to characterize structures whose net performance is high. Unfortunately, it is difficult to obtain similar explicit results about gross performance once we leave the linear-quadratic case. It is difficult even though we remain in the class of payoff functions W that are strictly concave in a for each e, so that the pbps condition is sufficient as well as necessary. We are, after all, performing a search in function space, and that is difficult unless the functions can be parametrized in some convenient way. Nevertheless it would seem plausible that for some class of concave payoff functions, algorithms could be constructed that come close to yielding best team action rules. Research challenge # 5. Construct an algorithm that yields best (or nearly best) team action rules for a wide class of concave functions W (containimg the linear-quadratic function and others as well), and does so for a wide class of information structures and probability distributions on the environment variables
240
T. Marschak
2.7 Designing an organization ‘‘from scratch’’: choosing its members, what each observes, and the speak-once-only mechanism that they use31 So far we have assumed (without comment) that the n members of our organization are already in place, and that we have no choice as to the external variables (the local-environment variables) which a given member observes. That is natural if our organization is an economy and its members are a given collection of consumers and producers. There is a natural privately observed external variable for each consumer; it describes her individual preferences and perhaps her individual resource endowment. For a producer, the natural external variable describes his current technology. Similarly, when modeling a firm with several divisons, it is natural to let the members of the organization be the division managers, who are already in place. For each manager, the natural privately observed external variables are those that characterize his production technology. Once we leave such settings, we may want to enrich our modeling toolkit. We may want to take the view that the organization does not yet exist but is being designed. The designer has a clean slate. He is given external variables e1, y , ek, and a set E which contains their possible values. He is given a set A of possible organizational actions. He is given a goal, which identifies at least one appropriate action in A for each external environment e ¼ (e1, y , ek) in E. But he can choose the following: The size and composition of the collection of persons who make up the organization. The identity of the person who will observe each external variable; some external variables may be observed by more than one person; some persons may not observe any external variable. The speak-once-only mechanism, which the organization uses to find a new action when the environment changes. 2.7.1 Speak-once-only mechanisms: introductory comments In a speak-once-only mechanism, the newly changed external variables are observed by their designated observers. Each observer sends forward to others a message based on those observations and then stays silent. The recipients of those messages send forward messages to others, based on the messages they have received and on their external observations (if they are designated external-variable observers), and then stay silent; the recipients of those messages send forward still further messages and then stay silent. And so on. When all sending has stopped, one member, called the action taker, takes an action based on the messages he has received 31 Some of the ideas in this section grew out of conversations with Ilya Segal. Some of the results in Section 2.7.6 are due to Jeff Phan. The discussion of delay in Section 2.7.10 as well as the result in Section 2.7.7 are largely due to Dennis Courtney.
Ch. 4. Organization Structure
241
(and on his own external observations if he is a designated externalvariable observer). The mechanism realizes a given goal if the action taker’s action is always goal-fulfilling for the environment e which initiates the process. Our speak-once-only requirement is certainly restrictive. For full generality, we would let the designer choose a mechanism in which each sender sends messages to others at each of a sequence of steps. But our clean-slate assumption, where the designer chooses the size and composition of the organization, the observing assigments, and the mechanism used to generate new actions presents a formidable challenge. Confining attention to speak-once-only mechanisms is a reasonable compromise, if one wants to start learning something about the structure of the designer’s chosen organization. To help motivate a research agenda on speak-once-only mechanisms consider the following question32: What might be inefficient about a one person mechanism; wherein a single person observes all the external variables and then finds a goal fulfilling action? A first step in making sense out of this question (and more complicated ones) is to choose some cost measures for a speak-once-only mechanism, so that ‘‘efficiency’’ (and inefficiency) can be defined. First we assume that each external variable ek is real-valued, and that every message in the mechanism is a vector of real numbers. For simplicity we assume that for every external vector e, there is a unique goal-fulfilling action F (e) where F (e) is a real number. Consider the following three cost measures for a speakonce-only mechanism: The number of persons. Each person’s burden, defined as the number of real variables observed or received. The mechanism’s delay, i.e., the total elapsed time until the actiontaker has computed F, given that (i) no one sends message variables or computes F until he has finished doing all the receiving and observing that the mechanism requires of him, and (ii) it takes one time unit for a person to observe or receive one real variable, but no extra time is required for the sender of a message to compute and send it or for the action taker to compute F.
32 One can ask a version of this question, even if one specifies that n, the number of persons, is greater than one and cannot be changed by the designer, and that each external variable is observed by one and only one person. A mechanism in which one person does essentially all the work, would be one in which, say, Person 1 collects full information about e from all the others and thereupon finds the goal-fulfilling action.
242
T. Marschak
2.7.2 The approaches of Radner/Van Zandt and Mount/Reiter Papers by Radner (1993), Radner and Van Zandt (1995), and Van Zandt (1997, 1998, 2003a, b) study a class of speak-once-only mechanisms. The messages transmitted are composed of real numbers, and there is an action taker who acquires enough information from others to compute a realvalued goal function F of external real variables e1, y , ek, each of which is observed by someone. The cost measures are number of persons and delay.33 But the function F has to have the form F ¼ H (H1(e1) * H2(e2) * ? * Hk (ek)), where * is an associative operation. Moreover, any person i, who receives, say, the real numbers u1, y , ur, is only able to compute u1 * u2 ? * ur, where * is the same associative operation. It takes one time unit to perform the operation * and no time is required for communication, so the Radner/Van Zandt work may be viewed as a model of organizational computing. It can be shown that if F is differentiable in the k variables, then no generality is lost if the operation * is constrained to be addition. That is to say, any differentiable F having the above associative form can be rewritten so that it becomes a function of the sum of certain terms H~ i ðei Þ: Nondifferentiable goal functions with the associative property include max (e1, y , ek), where e1, y , ek are real numbers. In the book by Mount and Reiter (2002),34 the goal function F (e1, y , ek) can be any analytic function. The problem is to assemble a minimal-delay F-computing network of processors (persons). Each of them receives inputs from other processors; each of those inputs consists of at most d real numbers. In one time unit a processor computes a certain analytic function of the numbers received and sends the result to one or more other processors. The class of functions that the processors are able to compute is a primitive of the model. If F itself has d real arguments, and is one of the available functions, then the problem is trivial, since a single processor can then compute F itself in one time unit. Instead, one seeks networks (directed graphs) that compute F with minimal delay when the functions available for each processor belong to an interesting class. The network is not required to be a tree, so cycles are permitted. But it is shown (under weak assumptions) that no generality is lost if one confines attention to trees. A tree has no 33 While Radner (1993) and Radner and Van Zandt (1995) consider a ‘‘one-shot’’ situation, where each environment vector e ¼ (e1, y , ek) is processed before a new e appears, papers by Van Zandt (1999, 2003a, b) go on to study the much more challenging situation where a new e arrives before the previous one has been completely processed, and the successive es follow a stochastic process. The current computed action, which is a function of the previous e, is then somewhat obsolete. The penalty due to obsolescence is studied. In particular, Van Zandt (2003b; sketched also in Van Zandt, 1998, Section 3.3) studies the performance of an organization which repeatedly allocates resources and thereby earns, in each time period, a payoff that is a quadratic function of the allocation and of that period’s environment vector. But the information used by the allocator reflects an earlier period’s environment vector, since it takes time for that information to reach him. Results are obtained by exploiting the fact that the mean of k variables is a sum and hence it can be computed by a sequence of associative operations. Early discussions of obsolescence as one of the costs of a mechanism appear in Marschak (1959, 1972). 34 An easily accessible summary of some of the book’s ideas is given in Mount and Reiter (1998).
Ch. 4. Organization Structure
243
cycles and so it is, in our terminology, an F-computing speak-once-only mechanism. While similar models of networks of processors (automota) have been studied by others, the Mount/Reiter research has a major novelty: the possible values taken by the input numbers, by F, and by each processor’s function, can be continua rather than finite sets. The research reported in the book does not explicitly seek efficient combinations of number of processors, individual burdens, and delay. Instead it seeks to characterize the networks (trees) that are capable of computing certain functions F with a given delay, when the processor functions obey key conditions like twice differentiability. Some propositions, for example, concern the number of processors to which each processor sends. There are also results about the relation between the delay for a continuumvalued F and the delay for each of a sequence of finite-valued functions that approximates F. It turns out that the former delay is the limit of the latter sequence of delays. The Mount–Reiter model is a highly innovative way to study the complexity of a given goal function F. Since it does not treat number of processors (persons) and individual burdens as explicit costs, it does not easily lend itself to the study of some of the efficient-organization questions that we shall now consider. Note that if each processor’s function is required to be addition, and if number of processors is a cost element, then the Mount/ Reiter model becomes the Van Zandt/Radner model. Note also that in both models one finds, in many interesting cases, that there is an upper limit to the useful number of persons (processors). Going above that number does not further decrease delay.
2.7.3 A formal definition of a speak-once-only mechanism To define a speak-once-only mechanism, a directed graph has to be specified. It will be useful to modify the conventional terminology of directed graphs to fit our context. A directed graph is defined by a set of nodes and a set of ordered pairs whose elements belong to the set of nodes. We shall say that the first node in the pair sends to the second and the second receives from the first. There is an arc between them. Then node i0 s received-from set, denoted Ri is the set of nodes from which i receives, while i’s sent-to set, denoted Si is the set of nodes to which i sends. We shall call a node j a leaf if it sends but does not receive. We call a node r a root if it receives but does not send. (Usually, the terms leaf and root are reserved for trees, but our graph need not be a tree.) An n-person speak-once-only mechanism on the local-environment sets E 1 RD1 ; . . . ; E k RDk ; with action space A Ra is a pair L ¼ hG; ð~ rkþ1 ; . . . ; ~ rn Þi: Here G is a graph and ~ rkþ1 ; . . . ; ~ rn are vectors of sending functions. If j A {1, y , k}, then ej is an environment variable; its
244
T. Marschak
possible values comprise the set E j RDj : We specify that G is a connected directed graph with nodes denoted 1, y , k, k+1, y , n, where nodes 1, y , k, with kon, are leaves (they correspond to the environment variables), node n (the action taker) is the only root, there are no directed cycles, and there is at least one directed path from every node to the root. for i ¼ k+1, y , n – 1, the vector ~ ri has one sending function, ri‘, for every ‘ in the sent-to set Si. Person i sends Di‘ real variables to person ‘ in Si, so the range of ri‘ is RDi‘ : The domain of ri‘ is the set of possible values of the variables i receives or observes, i.e., the domain is a subset of RSt2Ri Dti ; where Dti Dt if t A {1, y , k}. The remaining vector of functions is ~ rn : It has a single component, denoted simply rn, which yields the organization’s action. The range of rn is A Ra and the domain is a subset of RSt2Rn Dtn : The function rn, and every function rij, where i A {k+1, y , n1}, jASi, satisfies a smoothness requirement, e.g., it is differentiable at all points of its domain. Thus any person i in {k+1, y , n} whose received-from set Ri contains t A {1, y , k} is an observer of the environment variable et. In addition, person i receives a message, say mji, from every j A Ri with j>k. The message mji is a vector with Dji real components. Person i sends a message, say mij, to every person j A Si; that message is determined by the function ri. The action taker n receives message variables, say mjn, from certain persons j>k. His choice of the action a in A is determined by the function rn. For person i inP {k+1, y , n} we shall define i’s individual burden in the mechanism L to be t2Ri Dti : 2.7.4 An illustration, which leads to three questions To illustrate, consider four real and positive local-environment variables. It will be notationally convenient to call them w, x, y, and z. Suppose we want to realize the following goal function F: 1 1 1 1 F ðw; x; y; zÞ ¼ w þ þ x þ þ y þ þ z þ þ ðwxyzÞ. w x y z Here is a two-person mechanism that realizes F. Each person’s burden is 3 and the delay35 is 5. 35 Suppose a new (w, x, y, z) occurs. Three time periods then pass. At that point, Person 1 has completed his observing (processing) of his three assigned variables w, x, y and Person 2 has completed his observing of z. Now two more time periods pass. During the first of them, Person 2 receives (processes) the first of the two message variables which Person 1 sends, and during the second, Person 2 receives (processes) the second message variable. At the end of the fifth period, Person 2 has received (from Person 1) the messages he needs, and is able to take the action F (w, x, y, z).
Ch. 4. Organization Structure
Fig. 3.
245
A two-person mechanism that realizes F .
We can ask the following questions about the Fig. 3 mechanism and other possible speak-once-only mechanisms which also realize F: (1) If we reduce the number of persons to just one, must the burden and the delay of an F-realizing mechanism rise? Our one-person mechanism would be a five-node tree, with four nodes correponding to w, x, y, and z, and the sole person at the root. Clearly the burden of the sole person will be four and the delay will be four. We actually reduce delay when we move to the one-person mechanism, since no time is used for the processing of messages received from other persons.36 (2) Is there a two-person F-realizing mechanism in which neither person has a burden more than two and delay is not more than four? The mechanism would improve on the Fig. 3 mechanism with regard to delay and it would improve on the one-person tree with regard to burden. Such a mechanism would exist if it were possible to aggregate three of the four external variables into one, i.e., if it were possible to write F in the form G (H (w, x, y), z)), where G, H are real-valued functions (with suitable smoothness properties). It is natural to call the function H an aggregator. The single number H (w, x, y) contains 36 That illustrates a deficiency of our definition of delay, where the computing of the final action takes no extra time, once the action-taker has collected the information needed to do so. On the other hand, if we insisted on measuring computing time, then we would need a detailed model of computing, such as those studied in the Van Zandt/Radner and Mount/Reiter work.
246
T. Marschak
all the information about w, x, y that is needed in order to compute the goal function. Consider a mechanism in which Person 1 observes w, x, y and then sends the number H (w, x, y) to Person 2, the action taker, who observes z and is able to compute the action F(w, x, y, z) once he receives the number H. After three periods, Person 1 is done, and the action taker knows z. It takes just one more period for the action taker to learn H. So delay is indeed four. The key issue is whether the functions G, H exist. (3) Is there a two-person F -realizing mechanism in which neither person has a burden more than 3 and one person has a burden less than 3? Additional questions about F can be posed. In trying to answer them, it is clear that we have to restrict the F-realizing mechanisms that we are permitting. Once again the issue of smoothness is inescapable. If we permitted the sort of smuggling of many numbers into one number that we have already discussed (in Section 2.2.8), then no message need ever contain more than one number, and so we would obtain trivial and uninteresting answers to our questions. A workable definition of a smooth mechanism is that its sending functions rij be differentiable on their domains. Weaker and stronger definitions can be explored as well. What is known about efficient speak-once-only mechanisms and what might one hope to learn? To organize the remarks we now make, we shall use the above illustration, and its accompanying three questions. For each question, we consider the tools available to answer it, as well as the answer itself and its possible generalization. 2.7.5 General remarks suggested by Question (1) For a goal function F that is sensitive to all k external variabes, it is obviously true that a one person F-realizing mechanism has a tree structure, the sole person’s burden is k, and the delay is also k. A nontree F-realizing mechanism may improve on the one-person mechanism with regard to delay if the observing of the external variables is split among several persons who do their observing simultaneously, provided that the reduction in observing time exceeds the message-reading time. 2.7.6 General remarks suggested by Question (2) Let us relabel the external variables as x1, y , xm, y1, y , yn. Given a realvalued differentiable goal function F (x1, y , xm, y1, y , yn), the general aggregation question is as follows: Is there some neighborhood on which we can aggregate the m variables x1, y , xm into rom variables, i.e., do there exist a neighborhood U
Ch. 4. Organization Structure
247
Rmþn ; and rom real-valued functions G, H1, y , Hr which are differentiable on U, such that on U we have ðþÞ F ðx1 ; . . . ; xm ; y1 ; . . . ; yn Þ ¼ GðH 1 ðx1 ; . . . ; xm Þ; . . . ; H r ðx1 ; . . . ; xm Þ; y1 ; . . . ; yn Þ? An important contribution to answering this question is a theorem of Abelson (1980). Let Fi denote the partial derivative of F with respect to xi. Abelson’s theorem is as follows: There exists a neighborhood U Rmþn and differentiable functions G; H 1 ; . . . ; H r ðwhere romÞ; which satisfy ðþÞ on U if and only if at most r of the functions F i are linearly independent on U: Checking the linear independence of the Fi, is not, in general, a straightforward matter. But a technique closely related to the Abelson theorem provides answers to the aggregation question in certain cases. To introduce the technique, let us use the following notation: For an n-tuple of nonnegative integers a ¼ (a1, y , an), and for y ¼ (y1, y , yn) 2 Rn ; let the symbol ya denote the array of symbols ya11 yann : (This array will be used to identify partial derivatives of varying orders.) The symbol|a|denotes the sum a1+ ? +an. The symbol D (F, xt, ya) denotes the following partial derivative of order 1+|a|: @1þjaj F . @xt @ya11 . . . @yan n
The symbol D (G, z) denotes the partial derivative @G=@z: goal function F the Hessian 1 . . . DðF; x1 ; yn Þ C .. .. C, . . A DðF; xm ; y1 Þ . . . DðF ; xm ; yn Þ
Now let us associate with the 0 DðF ; x1 ; y1 Þ B .. HðF Þ B . @ and the bordered Hessian
1 DðF; x1 Þ B .. HðFÞ C BHðF Þ B A . @ DðF ; xm Þ 0
248
T. Marschak
For easy reference, note that The m rows of BH may be indexed by the m variables x1, y , xm that are being aggregated. The first of the n+1 columns may be indexed by the function F and the remaining columns by the nonaggregated variables y1, y , yn. We now state two propositions about aggregation.37 We call them Proposition (*) and (**). Proposition (*)38 provides bordered-Hessian conditions that are necessary for the existence of functions G, H1, y , Hr satisfying (+). Proposition (*). Let F be a C3 function. If there exist C2 functions G, H1, y , Hr such that (+) holds on some neighborhood U, then rank BHðF Þ r at every point of U. Now we turn to Proposition (**), which provides conditions that are sufficient for the existence of functions G, H1, y ,Hr satisfying (+). In stating Proposition (**), we consider open subsets of Rmþn of the form W V where W Rm ; V Rn : We denote points in W V by (p, q), where pAW, qAV. Proposition (**). Let F be a Ckþ1 function, kZ2. Suppose that rank BHðF Þ r everywhere on W V Rmþn and rank HðF Þ ¼ r everywhere on W. Then there exists a neighborhood UDW V and Ck functions G, H1, y Hr for which (+) holds. We can successfully apply Proposition (*) to the goal function 1 1 1 1 F ¼ w+x+y+z+wxyz+ w þ x þ y þ z considered in Question 2. Any three variables, say w, x, y, can be aggregated into two real variables, using
37
They are found in Mount and Reiter (1996) and in Appendix B of the book by Mount and Reiter. The proof of Proposition (*) is as follows: If (+) holds on U, then everywhere on U we have
38
DðF ; xi Þ ¼
r X
DðG; H k ÞDðH k ; xi Þ
k¼1
and DðF ; xi yj Þ ¼
r X
DðG; H xyj ÞDðH k ; xi Þ;
k¼1
BHðFÞ since Pthe Hk are independent Pr of the variables yi y ,yn We Pr can therefore write 1 0 r ... k¼1 DðG; H k DðH k ; x1 Þ k¼1 DðG; H k ; y1 ÞDðH k ; x1 Þ k¼1 DðG; H k ; yn ÞDðH k ; x1 Þ B C .. .. .. .. C: So asB . . . . @ A Pr Pr Pr k¼1 DðG; H k ÞDðH k ; xm Þ k¼1 DðG; H k ; y1 ÞDðH k ; xm Þ . . . k¼1 DðG; H k ; yn ÞDðH k ; xm everywhere on 0 1 U, 0each column 1 of BHðF Þ is a linear combination of the r column vectors DðH r ; x1 Þ DðH 1 ; x1 Þ B C B C .. .. B C; . . . ; B C: That implies that the rank of BHðF Þ is at most r at each point . . @ A @ A
DðH 1 ; xm Þ of U. ’
DðH r ; xm Þ
Ch. 4. Organization Structure
249
the C2 functions H 1 ¼ x þ x1 þ y þ 1y þ z þ 1z; H 2 ¼ xyz; G ¼ H 1 þ w þ w1 þ wH 2 : Can they be aggregated into just one real variable, i.e., do there exist differentiable functions G, H such that on every neighborhood in Rþ we have F ¼ G (H (w, x, y), z)? To check this, the relevant BH is 0 1 1 1=w2 þ xyz xy B 1 1=x2 þ wyz wy C @ A. 2 1 1=y þ wxz wx That has rank two on any neighborhood in which (for example) all four variables are neither zero nor one, and no two variables take the same value. Proposition (*) tells us that if the rank exceeds one on a neighborhood, then on that neighborhood we cannot aggregate three variables into one. There is no two-person F-realizing mechanism with differentiable sending functions in which neither person has a burden more than two and delay is not more than four. On the other hand, consider the following goal function F, identical to F except that we now raise the term w1 þ x1 þ 1y þ 1z to the power 2. Thus, 1 1 1 1 2 þ þ þ F ðw; x; y; zÞ ¼ ðw þ x þ y þ zÞ þ ðwxyzÞ þ . w x y z Suppose we now ask whether we can aggregate three variables into two (as we could for F), i.e., do there exist differentiable functions G, H1, H2 such þ that on every neighborhood in R4 we have F ¼ G (H1(w, x, y), H2(w, x, y), z)? Again BH has three rows (indexed by the three variables being aggregated) and two columns (indexed by F and by the remaining variable). Proposition (*) says that if the proposed aggregation is possible then BH has rank at most 2. But that is the case whether or not the proposed aggregation is possible. So Proposition (*) cannot be used to rule out the proposed aggregation. Proposition (**) says that the proposed aggregation is indeed possible if the rank of BH is at most 2 and at some point the rank of H is exactly 2. The latter condition cannot be satisfied, since H has just one column. So Proposition (**) does not tell us that the proposed aggregation is possible. More generally39: The necessay condition of Proposition ðnÞ has no force ðit is automatically satisfiedÞ if mon þ 1 39 The limitations of Propositions (*), (**) (and the extended Proposition (*0 ) which follows) might lead one to explore algebraic approaches. In particular, one might seek counterparts of the Abelson Theorem in which linear independence is replaced by algebraic independence. T functions are algebraically dependent if there is a polynomial, with the T functions as its arguments, which takes the value zero at all points in the functions’ domains. The T functions are algebraically independent if there is no such polynomial. Such a counterpart of the Abelson Theorem might hold for rational goal functions F, i.e., F is the quotient of two polynomials.
250
T. Marschak
ðand hence ron þ 1Þ: The sufficient condition of Proposition ðnnÞ cannot be satisfied if nor We can, however, generalize Proposition (*). The following generalization is more widely applicable than Proposition (*). Proposition (*0 ). Let F be a Ck+1 function, kZ2. Fix an integer ‘Z0. Using our definition of expressions of the form D (F, xj, ya), consider the matrix 1 0 DðF ; x1 ; yað1Þ Þ . . . DðF ; x1 ; yað‘Þ Þ C B .. .. .. C, M¼B . . . A @ að1Þ að‘Þ . . . DðF; xm ; y Þ DðF; xm ; y where, for all tA{1, y ‘}, the symbol a(t) denotes a t-tuple of nonnegative integers with|a(t)|ok. If there exist Ck functions G, H1, y ,Hr such that on some neighborhood F ¼ GðH 1 ðx1 ; . . . ; xm Þ; . . . ; H r ðx1 ; . . . ; xm Þ; y1 ; . . . ; yn Þ, then for all ‘ the matrix 0 DðF ; x1 Þ 1 C B .. M A M B . @ DðF ; xm Þ has rank at most r on that neighborhood.40 While Propositions (*) and (**) did not allow us to resolve the aggregation question for the function F ¼ (w+x+y+z)+(wxyz)+ ðw1 þ x1 þ 1y þ 1zÞ2 ; we can now do so, using Proposition (*0 ). To fit the notation of Proposition (*0 ), we first relabel the four variables as x1, x2, x3, y. Our function is: 1 1 1 1 2 þ þ þ . F ¼ ðx1 þ x2 þ x3 þ yÞ þ ðx1 x2 x3 yÞ þ x1 x2 x3 y 40 The proof is essentially the same as the proof of Proposition (*). The existence of functions G,H1, y , Hr on some neighborhood U implies that each column of M is a linear combination of the column vectors
0 B B @
1 0 1 DðH r ; x1 Þ DðH 1 ; x1 Þ C B C .. .. C; . . . ; B C. . . A @ A DðH 1 ; xm Þ DðH r ; xm Þ
That implies that the rank of M is at most r on U.
’
251
Ch. 4. Organization Structure
We ask whether there exist real-valued C2 functions G, H1, H2 such þthat F ¼ G (H1(x1, x2, x3), H2(x1, x2, x3), y) on all neighborhoods in R4 : To apply Proposition (*0 ), we first have to choose the partial derivatives that will make up the matrix M. It is clear that if we are going to rule out the existence of a given number of aggregators, then we want to make M (and hence M) as large as possible without repeating columns or introducing redundant columns like those consisting entirely of zeros. It turns out that the following matrix M suffices to resolve the aggregation question for our function F: 0
DðF ; x1 Þ
B M ¼ @ DðF ; x2 Þ DðF ; x3 Þ
DðF ; x1 ; y1 Þ DðF ; x1 ; y21 Þ
1
DðF ; x2 ; y1 Þ DðF ; x2 ; y21 Þ C A. DðF ; x3 ; y1 Þ DðF ; x3 ; y21 Þ
If we now choose a suitable neighborhood U and perform a Gauss–Jordan reduction on M for any point in U, we obtain the identity matrix. So M has rank 3 and Proposition (*0 ) tells us that the proposed aggregation cannot occur on U. The neighborhood U must be chosen so that we do not divide by zero during the reduction process. To do so, it suffices to pick a U that does not intersect any of the zero sets of the numerators and denominators of any of the entries of M at any stage of the Gauss–Jordan process. Since those numerators and denominators are always polynomials in x1, x2, x3, y1, it is possible to find such a U.41 While Propositions (*), (**), and (*0 ) are useful for a variety of goal functions, a general difficulty is the absence of a usable necessary AND sufficient condition for a proposed aggregation. The Abelson condition is both necessary and sufficient, but for many goal functions it is very difficult to verify. 2.7.7 A general puzzle suggested by Question (3): Can Question (3), and similar questions, be answered using the Abelson condition? Consider again the two-person mechanism in Fig. 3, which realizes the goal function F ¼ ðw þ w1 þ x þ x1 þ y þ 1y þ z þ 1zÞ þ ðwxyzÞ: For a general discussion of two-person F-realizing mechanisms it will be useful to start by relabeling the two persons. Let us call the action taker A and the other person B. In the Fig. 3 mechanism, Person B observes three environment variables and has a burden of 3, while Person A, the action taker, also has a burden of 3. (He observes one environment variable and he receives two 41
Precisely stated, our result is as follows:
There exists a finite number of polynomials f1, y , fk in the variables x1, x2, x3, y1 such that on any open subset U of R4 not intersecting the zero sets of f1, y , fk, there do not exist C2 functions G, H1, H2 such that F ðx1 ; x2 ; x3 ; yÞ ¼ GðH 1 ðx1 ; x2 ; x3 Þ; H 2 ðx1 ; x2 ; x3 Þ; yÞ at every point of U.
252
T. Marschak
numbers from B.) Turning to arbitrary two-person mechanisms, including mechanisms wherein some environment variable is observed by more than one person, we shall argue that ðþÞ
F cannot be realized by a two-person mechanism in which the maximum burden is 3 and one person has a burden less than 3:
Suppose a two-person mechanism realizes F and has maximum burden less than or equal to 3. Since the values of all four variables are needed in order to compute F, we can claim: ðþþÞ
the mechanism’s graph must have a path from each
environment variable to A: Now suppose that B observes exactly two variables, say x and y. Then, by (++), A must observe the other two, z and w. A cannot observe both x and y (in addition to z and w) since the maximum burden is less than 3. A cannot observe just one of x, y (in addition to z, w), for if he did, then his burden would exceed 3, since (by (++)) he must also receive at least one number from B. If A’s burden is less than 3, then B must send just one number. So F may be written F (x, y, z, w) ¼ A (B (x, y), z, w), where A and B (in a slight abuse of notation) denote scalar-valued functions computed by persons A and B. Consulting Abelson’s theorem (for the case r ¼ 1), we see that for 2 any fixed values of x, y, say x0, y2 0, the functions F x ðx0 ; y0 ; z; wÞ ¼ 1 1=x0 þ y0 wz; F y x0 ; y0 ; z; w ¼ 1 1=y0 þ x0 wz must be linearly dependent, where the symbol Fj again denotes the partial derivative of F with respect to the variable j. But that is the case if and only if the vectors ð1 x12 ; y0 Þ; ð1 y12 ; x0 Þ 0
0
are linearly dependent in a neighborhood in R2 ; which is, in turn, the case if and only if x0 x10 y0 þ y1 ¼ 0 in a neighborhood in R2 : That, however, is 0 not true. We conclude that F cannot be realized by a two-person mechanism with a minimal burden of 3 in which B observes exactly two variables. On the other hand, we can easily rule out the possibility that B observes exactly one variable. If he does, then (by (++)) A must observe the other three. But then A has reached the maximum burden of 3 and is not able to recieve any variable from B. So B must observe exactly three variables, say x, y, z. Then the only way that (+) could be violated (i.e., the only way to improve on the mechanism of Fig. 3) would be to give A a burden less than 3. By (++), A must observe w and since A needs to know more than w in order to compute F, A must have a burden of at least 2. Since A cannot compute F knowing only two environment variables, he must receive one or more numbers from B. To keep A’s burden below 3, he must receive only a single number from B. If that were so, we could write Fðx; y; z; wÞ ¼ AðBðx; y; zÞ; wÞ,
Ch. 4. Organization Structure
253
where both of the functions A, B are scalar-valued. But this is again ruled out by the Abelson criterion (for r ¼ 1), since the three functions 1 1 F x ðx0 ; y0 ; z0 ; wÞ ¼ 1 2 þ y0 z0 w; F y x0 ; y0 ; z0 ; w ¼ 1 2 x0 y0 1 þ x0 z0 w; F z x0 ; y0 ; z0 ; w ¼ 1 2 þ x0 y0 w z0 are linearly independent on every neighborhood in R: So we have established (+). Looking carefully at the preceding argument, one sees that it generalizes so as to yield the following Proposition. The statement ‘‘s of the n variables in F variables may be smoothly aggregated into ros’’ means that F may be written as a continuously differentiable function whose arguments are r continuously differentiable scalar-valued functions of those s variables, together with the remaining n – s variables. Proposition D. Let F: R4 ! R be continuously differentiable and sensitive to all its variables.42 Suppose that (I) three of F’s variables can be smoothly aggregated into two; (II) no two of F’s variables may be smoothly aggregated into one; and (III) no three of F’s variables may be smoothly aggregated into one. Then:
(IV) there is a two-person F-realizing mechanism which has the graph shown in Fig. 3, is smooth (its sending functions are continuously differentiable), and has burden 3 for both persons, and (V) there is no smooth two-person F-realizing mechanism in which each person has burden at most 3 and one of them has burden less than 3. 2.7.8 Goal functions for which perfect aggregation is possible It is now clear that one faces formidable difficulties in characterizing the efficient speak-once-only mechanisms that realize a given goal function. Is there some general class of goal functions for which the task becomes much easier? One such class consists of the functions for which perfect aggregation (PA) is possible. A goal function F : E 1 E k ! R has the PA property if it behaves like a sum: for any partitioning of the external variables, the variables in each set of the partitioning can be replaced by a single scalar-valued aggregator. If F is a sum, then the aggregator for a given set is the sum of the set’s variables. As before, we confine attention to the case where each environmental variable is a scalar, i.e., each Ej is a subset of the real line. Formally, The function F : E 1 E k ! R has the PA property if for ANY partitioning of {1,y, k}, say into sets T1,y, Ts, srk, there 42
I.e., there is a neighborhood on which all partial derivatives are nonzero.
254
T. Marschak
exist differentiable real-valued functions G, A1,y, As such that for any e in E ¼ E1 y Ek we have FðeÞ ¼ GðA1 ðeT 1 Þ; . . . ; As ðeT s ÞÞ Note that each of the functions A1, y , As must itself have the PA property. Examples of a function with the PA property are: e1+ y +ek e1 e2 y ek–1 ek be1 þ...þek : If our goal function F has the PA property, we have a strong result about the structure of efficient F-realizing mechanisms. It says that we can confine attention to trees if we seek mechanisms that are efficient with respect to number of persons and individual burdens. Proposition E. Suppose F is real-valued and has the PA property. Suppose that L ¼ hG; ð~ rkþ1 ; . . . ~ rn Þi is an F-realizing mechanism and that the graph G, with received-from sets Rk+1, y , Rn, is not a tree. Then there exists an F-realizing mechanism L0 ¼ hG 0 ; ðr~0 kþ1 ; . . . ~ r0n0 Þi, with received-from sets 0 0 Rkþ1 ; . . . ; Rn0 such that (a) G0 is a tree (b) n0 rn (c) for every i A{k+1, y , n0 }, we have #R0i #Ri , where the symbol # means ‘‘number of elements in’’.43 The Leontieff Theorem. It is remarkable that many years ago Leontieff (1947), inspired by the search for useful production functions in economics, and unaware of any implications for the theory of organization, proved a theorem that anticipated the Abelson theorem that we considered above. (Abelson’s theorem can be viewed as a generalization of Leontieff’s.) The theorem says that any function with the PA property is essentially a sum.44 43 The proof has two steps. Step 1 Since F has the PA property, it is clear that given any n-node-directed tree G which meets the requirements of the graph of a speak-once-only mechanism on the k environment variables, one can construct an F-realizing mechanism such that (i) its graph is the tree G, and (ii) exactly one real number is sent over each arc. (In that mechanism some functions rij may be the identity function.) Step 2 Without loss of generality we may suppose that on any arc of G a single real number is sent. (If more than one number is sent, then they can be replaced by an aggregator.) Now convert the given graph G into a tree as follows: Whenever there is more than one directed path between a node i6¼n and the node n, delete all but one of those paths. The result will be a tree, say G0 , with fewer than n nodes. The tree retains the original action-taking node. By Step 1, we can construct a new F-realizing mechanism with the tree G0 as its graph and with exactly one real number sent over each arc. So the new mechanism satisfies conditions (a), (b), and (c). ’ 44 Leontieff did not use our terminology, but his theorem can be restated in the form that follows.
Ch. 4. Organization Structure
255
Proposition F (Leontieff). Let F : E 1 E k ! R be a differentiable function with the PA property. Then there exist differentiable functions f, f1, y ,fk such that for all (e1, y , ek) in E1 ? Ek, FðeÞ ¼ fðf1 ðe1 Þ þ . . . þ fk ðek ÞÞ. So the PA property is not only a very strong one (yielding Proposition E) but it is also more restrictive than one might have supposed. One can develop further propositions about trees that realize PA goal functions, by imposing further conditions on the costs associated with the individual burdens. For example one might attach different costs to the ‘‘observation’’ burden (the number of variables that a node i>k receives from nodes in {1, y , k}), than the other burdens. Then one simple proposition says that if observation cost is linear in the number of observed variables, while the cost associated with the receipt of nonenvironment variables is increasing in the number of those variables, then the only efficient goal-realizing tree—and hence (by Proposition E) the only efficient goal-realizing design—has just one person, who observes all k environment variables. Replacing the PA property by a more general additive-structure property. Once we drop the strong PA requirement, it is natural to begin by studying goal functions that lack the PA property but have an additive structure somewhat more general than the ‘‘generalized sum’’ f(f1(e1)+ ? +fk(ek)) of the Leontieff Theorem. Let us relabel the environment components e1, y , ek so that they become the m real variables x1, y , xm. For 1rrom, we shall say that F : Rm ! R is r – additive on the open set S ¼ S1 ? Sm in Rm ; if there exist CN functions H, H1, y , Hr, V1, y , Vr such that for all (x1, y , xm)AS, we have F ðx1 ; . . . ; xm Þ ¼ HðH 1 ðV 1 ðx1 þ þ V 1 ðxm ÞÞ þ þ H r ðV r ðx1 Þ þ þ V r ðxn ÞÞÞ.
ð4Þ
We shall say that F is weakly r-additive on S if F ¼ HðH 1 ðV 1 ðx1 Þ þ þ V 1 ðxm ÞÞ; ; H r ðV r ðx1 Þ þ þ V r ðxm ÞÞÞ. (5) We shall say that F is minimally r-additive (minimally weakly r-additive) on S if for any positive integer r0 or, F is r-additive on S but not r0 -additive (weakly r-additive on S but not weakly r0 -additive). An economic interpretation might be as follows. The function F gives the organizational action appropriate to the environment (x1, y , xm). That action depends on the aspects of the environment given by the Vi. There are rom aspects. The ith aspect influences the appropriate action through the function Hi. We may
256
T. Marschak
suppose that the goal function F has been parsimoniously written, so that minimality is assured.45 An Example: Suppose there are four real environment variables: w, x, y, z and consider once again our function 1 1 1 1 2 þ þ þ . F ðw; x; y; zÞ ¼ ðw þ x þ y þ zÞ þ ðwxyzÞ þ w x y z (6) This function is 3-additive on any open set in which all four variables are nonzero, since it can be written F ¼ ðw þ x þ y þ zÞ þ expðln w þ ln x þ ln y þ ln zÞ 1 1 1 1 2 . þ þ þ þ w x y z The results we obtained above imply that F is minimally 3-additive. By contrast consider our function F, which we obtain by deleting the exponent 2 in the third term of (6). The function F is 3-additive but it is not minimally 3-additive, since it can also be written in the 2-additive form 1 1 1 1 wþ þxþ þyþ þzþ þ ðwxyzÞ. w x y z 2.7.9 Conjectures about the aggregation properties of goal functions with an additive structure What can be said about efficient speak-once-only mechanisms which realize a goal function that has an additive structure? If certain key conjectures were established, we could develop a substantial theory about such mechanisms. First, let us define F : Rm ! R to be t-aggregative on the open set S Rm ; if for any proper subset W of {1, y , m}, there exist C2 real-valued functions G, A1, y , At, with to#W, such that for all x ¼ (xW, xW) in S we have46 Fðx1 ; . . . ; xm Þ ¼ GðA1 ðxW Þ; . . . ; At ðxW Þ; xW Þ.
45 A possible scenario: The organization operates a production facility in each of m locations. Each location always produces a fixed proportion of the total quantity produced by the organization. That total quantity is the action to be chosen. The appropriate total quantity depends on total production cost, which depends, in turn, on the amounts of each of r centrally procured inputs required for each unit of product in each location. For every location, say location i, the r input requirements are determined by the environmental parameter xi, which describes the current technology in location i. The r input requirements are V1(xi), y ,Vr (xi). Once the case of a scalar-valued additive goal function F is understood, one could turn to the case of a vector-valued additive goal function F. That would, of course, permit more flexible scenarios, including scenarios wherein the organization’s action is a vector, specifying (for example) an output level for each of the m locations. 46 Here, W denotes the complement of W.
257
Ch. 4. Organization Structure
Thus, on the set S, it is enough to know xW and the values of the t aggregators A1, y , At, in order to compute F. However we partition the m variables into the sets W and W, we need no more than t aggregators of the variables in W in order to compute F. The following key conjectures remain open. Conjecture 1. If F is symmetric and r-additive on S and t-aggregative on S, with torom, then F is also t-additive on S. Conjecture 2. (stronger than Conjecture 1). If F is symmetric and t-aggregative on S, with tom – 1, then F is t- additive on S. Weak versions of these conjectures: Put ‘‘weakly’’ in front of ‘‘r-additive’’ and ‘‘t-additive’’ Suppose F is symmetric, and minimally r-additive with ron1. Suppose Conjecture 1 holds. Then we can claim that if we want to aggregate x ¼ (xW, xW) in S, with ro#W, into as few variables as possible, while retaining our ability to compute F, we cannot do better than to use GðA1 ; . . . ; Ar ; xW Þ
¼ H H 1 ðA1 þ
P
V 1 ðx‘ ÞÞ þ þ H r ðAr þ
‘2W
where Ai
X
V i ðx‘ Þ; i ¼ 1; . . . ; r:
P
V r ðx‘ ÞÞ ;
(7)
‘2W
(8)
‘2W
If it were possible to aggregate the variables xW into fewer than r numbers— i.e., if there existed G, A1, y , Aq, with qor, such that F ¼ G(A1(xW), y , Aq (xW), xW)—then Conjecture 1 says that F is q-additive. But that contradicts the assumed minimal r-additivity of F. Note that the converse of Conjecture 2 holds. If F is t-additive, then we can write F in a t-aggregative form, using (7) and (8) (with r ¼ t). Note also that Conjecture 2 is correct for the case m ¼ 3. To see this, call the three environment variables x, y, z. Since m1 ¼ 2, Conjecture 2 takes the following form: If F is symmetric and there exist C2 functions G, A such that Fðx; y; zÞ ¼ GðAðx; yÞ; zÞ then F is 1-additive. But that is just the simplest (three-variable) version of the Leontieff Theorem. A modest proposition about mechanisms with a tree structure. Consider again our function 1 1 1 1 F ¼ wþ þxþ þyþ þzþ þ ðwxyzÞ. w x y z
258
T. Marschak
As we saw in Fig. 3, F can be realized by a two-person mechanism whose graph is a six-node tree. Each person’s burden is 3. There is, however, no F-realizing mechanism whose graph is a tree with all the individual burdens being two or less. Our discussion of the application of Proposition (*) to F showed that while F is 2-aggregative it is not 1-aggregative. (If we choose the set W to contain three variables, then it is not the case that a single aggregator provides the information about those variables that is needed to compute F). Generalizing from the example of F and the Fig. 3 tree, we can obtain a rather modest proposition: Proposition G. Suppose the function F : Rk ! R, where k>2, is 2-aggregative but not 1-aggregative. Then in any twoperson tree which realizes F, each person’s burden is at least 2. Research challenge #6. Find interesting classes of goal functions F for which Conjectures 1 and 2 hold. 2.7.10 Computing a speak-once-only mechanism’s delay A mechanism’s delay is the total elapsed time until the action taker has completed his computation of the organization’s final action. We have considered delay informally, and we have illustrated delay, but we have not yet discussed how one might compute it. To do so we first define delay more carefully. We start by making the following simplifying assumption. (a) The environment variables change their values every G time units. For all the speak-once-only mechanisms we consider, G is sufficiently large that the mechanism’s delay is less than G.
We repeat, in addition, three assumptions already made in our introductory discussion of delay in Section 2.7.1. (b) It takes one time unit for a person to receive or observe one (scalar-valued) variable. (g) A person receives all the message variables that the mechanism specifies, and does all the observing required of him, before he sends messages or computes the final action. (d) It takes zero time for a person to compute a message variable to be sent onward, and for the action taker to compute and put into force the final action.
Thus if a person observes X (scalar-valued) environment variables, receives Y (scalar-valued) message variables from other persons, and computes Z scalar-valued functions of the observed environment variables
Ch. 4. Organization Structure
259
and the received message variables (to be sent forward to other persons), then X+Y time units are required for him to complete these tasks. He is silent until the mechanism has operated for at least X+Y time units. Next, note that for a given speak-once-only mechanism, the sequence of tasks may be subject to choice. Suppose, for example, that in a given three-person mechanism Person 1 observes three environment variables, Person 2 observes three other environment variables, and each sends forward to Person 3 (the action taker) values of two scalarvalued functions. In one possible sequence, Person 1 first observes all his environment variables, then Person 2 observes all of his, and immediately after that (with no further time required), both persons compute and send forward the values of their scalar-valued functions. That would appear to be an inefficient sequence, since time would be saved if 1 and 2 did their observing simultaneously. Nevertheless it is a possible sequence. In general, we may define a protocol p for a given mechanism as a specification of exactly what tasks (observing, sending, receiving, computing) each person performs at each time in a sequence of times (one time unit apart), terminating with the action taker’s final-action computation. A protocol is required to respect the above assumption g. (A formal definition of a protocol can be provided, but it requires an array of new symbols.) It will be useful to use the term entity for a node of the mechanism’s directed graph, whether that node identifies an environment variable or a person. For the protocol p, we may define a time-to-completion function fp on the set of entities. For person p, the number fp (p) is the shortest time after which person p ‘‘does nothing,’’ i.e., he does no further observing, receiving, sending, or computing. If the entity j is an environment variable (i.e., it is a node belonging to the set {1, y , k}), then we define fp (j) to be zero. If the protocol p is used, then the total elapsed time until the mechanism terminates is fp (n) where (as before) n is the action taker. Our defintion of delay is then as follows:
Definition. A speak-once-only mechanism’s delay is the minimum value, over all protocols p, of fp (n). A protocol p for which fp (n) is minimal will be called a minimal protocol for the mechanism. A mechanism’s delay may be computed recursively. To simplify notation, let T denote the function fp, where p is minimal, and write Tp instead of T(p). (Thus Tp is the smallest time until p does nothing further.) We shall use the term ‘‘listens to’’ to cover both observation of an environment variable and receipt of a message variable. Suppose p listens to just one entity, say 1, from whom he receives N1 variables. Then Tp ¼ T1+N1, since
260
T. Marschak
p has to wait T1 units for 1 to finish and then takes N1 units to listen to the N1 variables sent to him by 1. At that point, p is done. Now suppose that p listens to more than two entities. We make the following claim: Claim. There is a minimal protocol in which p listens to each of the persons from whom he receives in the order in which those persons are done with their sending to him. To argue this claim, start by supposing that p listens to two persons, say 1 and 2; 1 sends N1 variables to P and 2 sends N2 variables. Assume that T1rT2. Then there is a minimal protocol in which T p ¼ maxðT 1 þ N 1 ; T 2 Þ þ N 2 . To see this, note first that p must wait T1 time units for 1 to finish. He spends the next N1 time units listening to 1. If T2 has elapsed by then, (i.e., T2rT1+N1), then p spends the next N2 time units listening to 2. If T2 has not yet elapsed (T2>T1+N1), then p must wait for T2 units to elapse and then spends the next N2 units listening to 2. If p listens to three persons, say 1,2,3, with T1rT2rT3, then we obtain, for some minimal protocol: T p ¼ maxðmaxðT 1 þ N 1 ; T 2 Þ þ N 2 ; T 3 Þ þ N 3 . Recalling that Tp ¼ 0 if the node p corresponds to an environment variable, we now see that there is some minimal protocol such that for each person p, the following procedure computes the number Tp for that protocol. (1) Label the entities that p listens to as 1, 2, y , m, where T1rT2r ? rTm. (2) Initialize values u ¼ T1+N1 and k ¼ 1. If k ¼ m, then set Tp ¼ u and stop. (3) Replace u with u ¼ max (u, Tk+1)+Nk+1 and replace k with k+1. (4) If k+1om, repeat step (3). If k+1 ¼ m, then set Tp ¼ u and stop. Note that the recursive scheme just described is indeed realized by some protocol for a given mechanism, since the four steps can clearly be carried out for a person p who observes environment variables but does not receive messages. Since a speak-once-only mechanism has no directed cycles, it follows that the four steps can also be carried out for a person p who receives messages. Since we can, in particular, carry out the four steps for p ¼ n (the action taker), the four steps provide a recursive way of computing the delay; the delay is Tn ¼ fp(n) where p is a minimal protocol. Roughly speaking, the steps insure that each person is ‘‘as busy as possible.’’ Each person begins the work of listening as soon as signals (environment variables or message variables) begin to arrive, and no person
Ch. 4. Organization Structure
261
stays idle while there is work (listening) that he could be doing. (Assumption (g) is crucial for this claim.) Some preliminary computer experiments with the recursive procedure suggest that typically the time required to compute Tn rises rapidly as n and k grow. It seems likely that one can construct improved versions which exploit the properties of narrow classes of goal functions. Research challenge # 7. For interesting classes of goal functions, refine the delay-computation algorithm so that it performs rapidly for mechanisms that realize a function in the class. Examples of the computation of delay (a) In a one-person mechanism, where the action taker observes all environment variables, delay is just the number of those variables. (b) Suppose there are k ¼ 2M environment variables, where M is a positive integer not less than 2. Suppose the mechanism’s graph is a binary tree with a person at each nonleaf node. Then delay is 2 log2 k ¼ 2 M. Similarly if k ¼ tM, where t and M are integers greater than 2, and the mechanism’s graph is a t-ary tree with a person at each nonleaf node, then delay is t logt k ¼ tM. (Since each person listens to t variables, it takes t time units for all the persons at a given tier of the tree to finish, and all persons in the tier finish at the same time; there are M tiers, so delay is tM.) For such a mechanism no protocol can achieve a delay less than tM. In contrast, a oneperson mechanism has a delay of k ¼ 2M as well as a burden of 2M. (c) Consider once again the goal function 1 1 1 1 2 þ þ þ F ðw; x; y; zÞ ¼ ðw þ x þ y þ zÞ þ ðwxyzÞ þ . w x y z Consider the following two nontree mechanisms that realize F. For each of them delay can be checked visually or it can be found using our recursive four-step procedure. The six-person Fig. 4 mechanism has a property of ‘‘U-forms’’ in the economic literature, briefly mentioned in Section 1. Each environment variable is observed by just one person (a specialist). Persons 1 and 2 are the observers. They report to a ‘‘sum’’ specialist (Person 3), a ‘‘product’’ specialist (Person 4), and a ‘‘reciprocal’’ specialist (Person 5). Those three specialists report, in turn, to the action taker (Person 6), who then has all the information needed to compute F. The five-person Fig. 5 mechanism has a property of ‘‘M-forms’’ in the economic literature: each environment variable is separately observed by several persons. Persons 1,2, and 3 all observe w, x, y. For those variables, Person 1 is a ‘‘sum’’ specialist, Person 2 is a ‘‘product’’ specialist, and Person 3 is a ‘‘reciprocal’’ specialist. Person 1 reports directly to Person 5
262 T. Marschak
Fig. 4.
This is a six-person nontree F realizing mechanism with delay 7. It has the ‘‘U-form’’ property: each environment variable is observed by a single person.
Ch. 4. Organization Structure
Fig. 5.
This is a five-person nontree F realizing mechanism with delay 6. It has the ‘‘M-form’’ property: each environment variable is observed by more than one person.
263
264
T. Marschak
(the action taker), but Persons 2 and 3 report to an intermediary (Person 4), who also observes z directly, as does Person 5. We find that the Fig. 4 mechanism uses 6 persons. Five of them have a burden of 2 and one of them (the action taker) has a burden of 3. Delay is 7. the Fig. 5 mechanism uses 5 persons. Each has a burden of 3. Delay47 is 6. The comparison illustrates a general conjecture: Conjecture y. Consider a goal function F which is a sum of three scalarvalued functions of four environment variables and cannot be expressed as a function of fewer than three scalar-valued functions. There does not exist a nontree F-realizing mechanism which dominates all the other nontree F-realizing mechanisms with regard to number of persons, individual burdens, and delay. Research challenge # 8. Apply the recursive delay-computation algorithm to establish Conjecture y and similar conjectures.
3 Models in which the designer is concerned with incentives as well as informational costs A gigantic literature considers the designing of schemes that induce the selfinterested members of an organization to make choices that meet a given organizational goal. One branch of this literature studies contracts. Another studies ‘‘implementation’’; it considers schemes in which each self-interested member chooses an individual message, knowing that an organizational action will be chosen once all the messages are sent, knowing the outcome function that will be used to assign an action to the individual messages, and knowing the impact of each action on his own personal welfare. Such a scheme defines a message-choosing game. If the scheme is properly designed then it ‘‘implements’’ a goal correspondence that has been given to the designer: at an equilibrium of the game, the messages chosen are those for which the assigned action (prescribed by the outcome function) is goal-fulfilling.48 Contracting schemes and goal-implementing mechanisms have informational costs, and it would be of great interest to be able to find the cheap ones. Informational costs are often discussed informally. Indeed some of the classic goal-implementing mechanisms (e.g., those that lead the organization’s self-interested members to make the correct choice as to the quantity of a public good) are striking in their 47 At the end of the first three periods, 1,2, and 3 have each learned w, x, y, while 4 and 5 have learned z. No messages have yet been sent. In the fourth period, 5 receives w+x+y from 1, while 4 receives wxy from 2. In the fifth period, 4 receives 1/w+1/x+1/y from 3. In the sixth period, 5 receives 2 wxyz þ w1 þ x1 þ 1y þ 1z from 4 and is now able to compute F. 48 Two excellent introductory surveys of the implementation literature are: Serrano (2004) and Jackson (2001).
Ch. 4. Organization Structure
265
informational simplicity.49 But it is very rare for a paper to present a goalimplementing mechanism and then to argue its informational minimality in a precise way. The managerial accounting literature studies mechanisms or contracting schemes to be used by a profit-maximizing principal and the selfseeking ‘‘responsibility centers’’ (cost or profit centers) of a firm. The information transmitted by the centers (perhaps by using transfer prices) is far more modest than, for example, that transmitted in a DR mechanism which achieves the same result. But again, informational minimality is not formally shown.50 Occasionally, however, papers in this literature explicitly measure an informational cost, e.g., the cost of reducing the variance in the noise that accompanies a signal sent from the agents to the principal.51 We now turn to some research in which initial steps toward incorporating informational costs as well as incentives are taken. 3.1 The message-space size required for implementation of a goal Suppose, as before, that each person i has a privately observed local environment ei whose possible values comprise the set Ei. In most implementation discussions, ei determines i’s preferences over the possible organizational actions (and may contain other information about the organization’s environment as well). Then an n-person implementation scheme, often called a game form, has two elements: an n-tuple S ¼ (S1, y ,Sn) of individual strategy spaces, and an outcome function h: S-A, where A is the set of possible organizational actions. In particular, a strategy siASi may be a rule that tells i how to behave at each stage of a Tstep message-announcement process, where the organization’s action is a function of the step-T announcements. It tells him what message mi, in a set Mi of possible messages, to announce at step t, given what he knows about the announcements thus far and given his current ei. The organizational action is therefore a function of the current e ¼ (e1, y , en). Given his current ei, and given any strategy choices by the others, person i (who knows the function h) is able to rank any two of his own strategies with regard to his own welfare. For each e, we therefore have a game, denoted Ge, and we may choose a solution concept—say Nash equilibrium—to identify the game’s equilibria. The scheme /S, hS implements a goal correpondence G from E ¼ E1 ? En to A if h (s)AG (e) whenever s ¼ (s1, y , sn) is a Nash equilibrium of the game Ge. 49 Groves and Ledyard (1977) is the classic paper on public-good provision in an organization whose members differ with regard to their private valuation of the good , using a mechanism that appears to be informationally cheap, although no formal claim about its informational minimality is made. By contrast, informational cost is formally studied in a ‘‘nonincentive’’ paper by Sato (1981). That paper finds a lower bound to the message space size for mechanisms used by an economy which seeks an efficient allocation of resources to the production of public goods, if we assume that the agents voluntarily follow the mechanism’s rules. 50 See, for example, three papers by Melumad et al. (1992, 1995, 1997). 51 See, e.g., Ziv (2000).
266
T. Marschak
Now consider the equilibrium strategy profiles of the game Ge. As before, let m denote (m1, y , mn) and let M denote M1 ? Mn. Define the correspondence mi ðei Þ ¼ fm 2 M : for some e 2 E, m is the step-T announcement for ðs; e Þ, s is an equilibrium of Ge ; and ei ¼ ei g. Let h~ be an outcome function from M to A, with the following property: ~ h(m) ¼ a if there is an S such that (1) a ¼ h (s) and (2) for some eAE, s is an equilibrium strategy profile of the game Ge and m is the step-T message for ~ is a (privacy-preserving) mechanism on (e, s). The triple /M, (m1, y , mn), hS E in our previous sense. Moreover it realizes the goal correpondence G. One can ask: among the G-implementing mechanisms (obtained from a G-implementing game form in the manner just described), which ones have a minimal message-space size? Or, less ambitiously, is there a useful lower bound to such a mechanism’s message-space size? The message-space requirements for implementation of a goal correspondence are certainly going to be harsher, in general, than the requirements for realization alone. ‘‘ How much harsher?’’ is a difficult question, and very few papers have addressed it.52 3.2 Models in which the organization’s mechanism is partly designed by its self-interested members, who bear some of the informational costs 3.2.1 A model in which the organization’s decentralized self-interested members choose their search efforts, a ‘‘Decentralization Penalty’’ results, and the effect of improved search technology on that penalty can be traced Consider a three-person organization53 consisting of two Managers, called 1 and 2, and a Headquarters (HQ). There is a changing external environment e ¼ (e1, e2). Manager i, i ¼ 1, 2, observes ei, whose set of possible values is Ei. HQ does no observing. Whenever there is a new value of e ¼ (e1, e2), HQ learns something about it from the managers, though their knowledge of the new e is imperfect. Having received reports about the new e from the managers, HQ chooses an action a. The organization then collects a payoff W (a, e). [Example: The organization makes a product which it markets in two locations. The variable ei is the product’s price in location i next week. The action a has two components a1 and a2, where ai is the amount to be shipped to location i for sale next week. The payoff W (a, e) is the profit earned next week.] Manager i always learns something about a new ei in the same way. He has a finite partitioning of Ei, which he has chosen once and for all out of 52
Three of them are Reichelstein (1984), Reichelstein and Reiter (1988), and Tian (1990). Studied in Marschak (2004).
53
Ch. 4. Organization Structure
267
some set of available partitionings. Whenever ei changes, he conducts a search to find that set of his partitioning which contains the new ei. Let s1, s2 denote the two chosen partitionings. Manager i reports the set he finds, say SiAsi, to HQ. Then HQ chooses an action aˆ (S1, S2) which maximizes the conditional expectation E(W (a, e)|e1AS1, e2AS2), where the symbol e ^ (S1, S2) denote the value of that condiagain denotes expectation. Let O tional expectation when the maximizing action aˆ is used. The highest attainable expected gross performance of the two chosen partitionings, which ^ we simply call gross performance (for brevity), is the expected value of O; where the expectation is taken over all the possible pairs (S1, S2). Our symbol for gross performance is just O. Thus ^ 1 ; S 2 Þ. Oðs1 ; s2 Þ ¼ Eðs1 ;s2 Þ2s1 s2 OðS Even though the technology of search may be advanced, the managers’ search has a cost, since it requires human time and human expertise. We suppose that the cost of searching the partitioning si is measured by y C(si), where y and the values taken by the function C are positive real numbers. When search technology improves, y drops. Consider two definitions of C(si). The first is simply the number of sets in si. The second is the ‘‘Shannon content’’ of si, i.e., X ðprobability of Si Þ ðlog2 ðprobability of S i ÞÞ: si 2si
The number-of-sets measure ignores the fact that some sets occur more frequently than others. But it is an appropriate measure if the searchers who assist the manager have to maintain their ability to distinguish between the sets. That may require substantial training, and the number-of-sets cost measure may be viewed as the opportunity cost of the investment made in such training. As the technology of search improves, the training required to distinguish among a given number of sets becomes less costly. On the other hand, the Shannon content is sensitive to the set probabilities. Using the most elementary of the theorems in the Information Theory which Shannon founded (the noiseless coding theorem),54 one can show that if y is small then the partitioning’s Shannon content approximately equals the average number of steps required when a searcher follows a well-chosen binary tree to find the set in which his current ei lies. Then y times the Shannon content is (approximately) the average amount of dollars paid for searcher time when search is efficient. That drops when search technology improves (i.e., when y drops).55 54
See, for example, Abramson (1963, pp. 72–74). Once the two partitionings s1,s2 have been specified, they define—using our previous terminology— a speak-once-only mechanism with HQ as the action taker. For every e, the action-taker computes the ^ 1 ; S2 Þ when e1AS1,e2AS2. function F (e) which takes the value aðS 55
268
T. Marschak
We now compare a centralized organization with a decentralized one. Here we use the term ‘‘decentralized’’ in a new way, appropriate to the discussion of incentives. In the decentralized organization the managers are self-interested and are free to make choices that were not available to them in the centralized case. In the centralized case, monitoring (or perhaps socialization) insures that each manager strikes the correct balance between his search cost and the gross performance that his search permits. So the chosen partitionings, say s1 ,s2 are first best. They maximize Oðs1 ; s2 Þ y Cðs1 Þ y Cðs2 Þ. In the decentralized case, there is no monitoring and each self-interested Manager i is free to choose his preferred si. He bears the associated cost, namely yC(si). But he is rewarded with a share of the expected gross performance O. So the two decentralized managers play a sharing game in which Manager 1 chooses s1, Manager 2 chooses s2, and each Manager i collects an (average) payoff equal to r Oðs1 ; s2 Þ y Cðsi Þ, where r, each manager’s share, is greater than zero and not more than 12: In the decentralized case, the chosen partitionings comprise a Nash equilibrium of the sharing game. The sharing-game interpretation of decentralization seems very natural if one seeks a workable and plausible model. ‘‘Profit-sharing’’ is, after all, the oldest (and simplest) of the schemes that one sees when one looks at decentralized real-world attempts to reconcile individual incentives with organizational goals. Such schemes will not perform as well as first-best choices would, but they may be the best practical alternative if the only way to ensure first-best choices is to engage in intrusive policing, or to adopt (in a real-world setting) some version of the sophisticated but informationally costly monitoring schemes that theorists have proposed. Now let (sy1, sy2) denote a pair of decentralized partitionings, chosen at an equilibrium of the sharing game. Our central interest is the Decentralization Penalty when y is the level of search technology. The Penalty associated with s1 , s2 ,sy1, sy2 is PðyÞ ¼ ½Oðs1 ; s2 Þ yCðs1 Þ yCðs2 Þ ½Oðsy1 ; sy2 Þ yCðsy1 Þ yCðsy2 Þ. In a shirking equilibrium of the decentralized sharing game, the managers’ total search expenditures are less than the first-best expendures. In a squandering equilibrium the reverse is true. It turns out that under plausible assumptions a search technology improvement (a drop in y) decreases the Decentralization Penalty associated with a squandering equilibrium, but its effect on the penalty associated with a shirking equilibrium is not so clear-cut.
Ch. 4. Organization Structure
269
First consider a ‘‘short-run’’ setting, where a finite collection of possible partitionings is available to each manager, and the technology parameter y changes but its values lie in some interval [R, S], where 0oRoS. The interval is sufficiently small that when y drops within that interval there is no change in the decentralized managers’ partitioning choices or in the firstbest partitionings. For all y in the interval [R, S], the former partitionings are sy1,sy2 and the latter are s1 ,s2 . We can rearrange the expression for the associated Decentralization Penalty to obtain: PðyÞ ¼ Oðs1 ; s2 Þ Oðsy1 ; sy2 Þ þ y½ðCðsy1 Þ þ Cðsy2 ÞÞ ðCðs1 Þ þ Cðs2 ÞÞ. Examining the term in square brackets, we immediately see that if squandering occurs at the decentralized partitionings sy1,sy2, then a drop in y decreases the Decentralization Penalty, but if shirking occurs at the decentralized partitionings, then a drop in y increases the Decentralization Penalty. Now let us turn to a ‘‘long-run,’’ where an infinity of possible partitionings is available and, in addition, the first-best and decentralized (sharinggame equilibrium) partitionings change whenever y changes. Let Di(y)>0 identify Manager i’s first-best partitioning and let Diy (y )>0 identify the decentralized Manager i’s partitioning at an equilibrium of the sharing game. Let y C (D) be the cost of the partitioning D, where C is increasing. Then for a given y, the Decentralization Penalty is PðyÞ ¼ OðD1 ðyÞ; D2 ðyÞÞ yCðD1 ðyÞÞ yCðD2 ðyÞÞ OðDy1 ðyÞ; Dy2 ðyÞÞ yCðDy1 ðyÞÞ yCðDy2 ðyÞÞ . Suppose the functions Di, Diy are differentiable.56 For every y, the first best partitionings Di have to satisfy the first-order condition for the maximization of OyC1yC2, and the decentralized partitionings Diy have to satisfy the first-order condition for D1 to be a best reply (in the sharing game) to D2, and vice versa. When we compute the derivative P0 , while inserting the first-order conditions, we obtain: h i P0 ðyÞ ¼ CðDy1 ðyÞÞ þ CðDy2 ðyÞÞ CðD1 ðyÞÞ þ CðD2 ðyÞÞ 1 y0 y y0 y 0 0 y 1 D1 ðyÞC ðD1 ðyÞÞ þ D2 ðyÞC ðD2 ðyÞÞ . r 56 The differentiability assumption is made for analytic convenience. Typically one would want each of the available partitionings to be identified by a positive integer T. (Thus Manager i’s external variable ei might have the interval [A, B] as its support, with 0oAoB, and T might identify the partitioning in which that interval is divided into T subintervals of equal length.) Under very plausible assumptions about O, the finding that the derivative P0 is negative (when we treat Di,Diy as continuous and differentiable rather than integer-valued) implies that for sufficiently small y, P remains a decreasing function when we restrict the functions Di,Diy to integer values.
270
T. Marschak
Here primes denote derivatives and r, with 0orr1/2, is each manager’s share of the gross performance O in the sharing game which the managers play when they are decentralized. Now suppose we know the following: 8 when search technology improves ðy dropsÞ; > > > > > the decentralized managers > > < spend more on search; i:e:; costlier partitions ð#Þ > > > are chosen at the equilibria > > > > : of the sharing game 0
Then Dyi ðyÞo0; i ¼ 1; 2 for all y>0. Since C is increasing and 1/r1>0, the second of the two terms in square brackets is negative or zero. If there is squandering at the decentralized partitionings, then the first of the two terms in square brackets is positive and hence the entire expression for P0 is positive, i.e., the Decentralization Penalty shrinks when IT improves. If there is shirking, then the sign of P0 is indeterminate unless we make further assumptions. Some of the scarce empirical literature on the effect of IT improvement on organizational structure suggests that as IT improves, the case for ‘‘decentralizing’’ an organization becomes stronger.57 The model of managers who engage in search is a first attempt to see why this might be so. If the IT improvement in question is the explosive growth of efficiently searched databases, and if ‘‘decentralized’’ managers can reasonaby be modeled as players of a sharing game, then it is of considerable interest to find the conditions under which improved search technology indeed implies a drop in the Decentralization Penalty. An explicit formula for the Penalty has been found for certain cases.58 But there are also examples of a W and a probability distribution on the ei in which the decentralized sharing game has a squandering equilibrium.59 The effect of a drop in y on the Decentralized Penalty associated with squandering is strikingly unambiguous, both in the ‘‘short-run’’ setting, and (provided the statement (#) holds) in the long-run setting as well. So it would be valuable to understand the squandering phenomenon much better. For a very wide class 57
See the papers mentioned in the Section 1: Bresnahan et al. (2000, 2002). Suppose, in particular, that the payoff function W has the linear/quadratic structure discussed in Section 2.6, where the techniques of the Theory of Teams were considered. For that W, person-byperson-satisfactoriness is both necessary and sufficient for a team action rule to be best for a given information structure. That allows us to find the best action rules used by HQ once each manager tells HQ the set in which his current local environment lies. Suppose further that each ei is uniformly distributed on a closed interval, that each Manager i divides his interval into Ti subintervals of equal length, and that each choice of Ti defines one of the manager’s possible partitionings. It then turns out that (1) in the unique equilibrium of the sharing game, each manager shirks (chooses a Ti that is lower than the first-best Ti), (2) in the ‘‘long-run’’ setting (where a small change in the technology parameter y leads to new equilibrium values of the Ti as well as new first-best values) the Decentralization Penalty indeed drops when y drops. That is true for the number-of-sets cost measure as well as the Shannon cost measure. 59 In the examples found so far, there is a finite set of possible values for each ei. 58
Ch. 4. Organization Structure
271
of sharing games a sufficient condition for ruling out squandering is complementarity: an increment in Manager 1’s search expenditures improves (or does not damage) Manager 2’s ‘‘marginal productivity,’’ i.e., the increment in O due to an increment in Manager 2’s search expenditures.60 On the other hand, a class of functions O has been found for which the following is true: if O exhibits sufficiently strong ‘‘anticomplementarity’’, then there are squandering equilibria in the sharing game defined by O and the shares ri. (Anticomplementarity means that an increment in Manager 1’s search expenditures damages Manager 2’s marginal productivity.) In an informal way, one can imagine that the much discussed but seldom modeled phenomenon of ‘‘information overload’’ can indeed lead to anticomplementarity and to squandering. As Manager 1 increases his search effort and the level of detail in his report to HQ, he greatly diminishes the value of Manager 2’s report, because Manager 2’s report then becomes somewhat redundant, i.e., its marginal contribution is small. The managers might then be trapped in an equilibrium where Manager 1’s partitioning is very costly, Manager 2’s partitioning is cheap, and the sum of the costs is higher than the cost sum for a first-best partitioning pair. A rough conjecture would be as follows: ^ Conjecture (*). For sets GDE1, HDE2, let aðG; HÞ denote a value of a which maximizes the conditional expected value of the payoff W (a, e1, e2), given that e1AG, e2AH. Given the probability distribution of e ¼ (e1, e2), the payoff function W, and partitionings s1, s2 let us measure the marginal contribution of s2 by
^ 1 ; Q2 Þ; e1 ; e2 Þje1 2 Q1 ; e2 2 Q2 Þ EQ1 2s1 ;Q2 2s2 EðW ðaðQ
^ 1 ; E 2 Þ; e1 ; e2 Þje1 2 Q1 Þ : EQ1 2s1 EðW ðaðQ Then if the marginal contribution of s2 is ‘‘sufficiently small but not too small,’’ (s1,s2) is a squandering equilibrium of the sharing game. Here ‘‘but not too small’’ seems appropriate, since if s2 makes an extremely small marginal contribution, then Manager 2 finds that when 60 Consider n-person sharing games in which each person i chooses a strategy xi, bears the nondecreasing cost ci (xi), and receives a reward Ri (z) when the organization earns the revenue z ¼ A (x1, y , xn), where A is nondecreasing in each of its arguments. Suppose the functions Ri obey a ‘‘nondecreasing residual’’ property: when z increases by any positive amount D, the sum of the rewardsPdoes not rise by more than D. (That is satisfied, for example in the ‘‘balanced-budget’’ case where z ¼ Ri(z) as well as the ‘‘constant-share’’ case Ri(z) ¼ ri z which we have been considering.) Suppose A obeys complementarity: if we look at the increment in A when any player spends more, we find that the increment does not drop when some other player spends more. Then in any equilibrium no player squanders relative to an efficient (first-best) (x1, y , xn), where A minus the cost sum is maximal. Every player spends the efficient amount or less. The argument holds whether the strategy sets are finite or are continua. The argument is provided in Courtney and Marschak (forthcoming).
272
T. Marschak
compared to the one-set partitioning (the cheapest partitioning), his share of s2‘s extra revenue fails to justify its extra cost. Research challenge #9. To better understand the squandering phenomenon, find probability distributions on the local environments, and payoff functions W, such that Condition (#) holds: when search technology improves, the decentralized managers spend more on search at the sharing-game equilibrium. (Then, in the ‘‘long-run’’ setting, a search-technology improvement diminishes the Decentralization Penalty associated with squandering) Conjecture (*) holds. 3.2.2 The effect of search-technology improvement on the Coordination Benefit In addition to studying the Decentralization Penalty, one can conduct a parallel investigation of the effect of improved search technology on the Coordination Benefit. That might be motivated by informal suggestions, in some empirical studies, that improved IT leads to greater ‘‘lateral communication.’’61 To define coordination in our two-manager model, let the organization’s action have two components: a ¼ (a1, a2), where ai is associated with Manager i. Then coordination for the centralized organization (where managers are loyal team members and are not self-interested) means that HQ chooses ai as a function of both Managers’ search results, while no coordination means that each ai is chosen as a function of i’s results only. The Coordination Benefit is the improvement in expected net payoff—i.e., O minus the sum of the search costs—when we move from no coordination to coordination. To define the concept in the decentralized (sharing-game) case, suppose that HQ no longer exists. Instead, the self-interested Manager i chooses ai himself. Coordination now means that i bases his choice on both manager’s search results (after ‘‘lateral communication’’ occurs), and no coordination means that he bases it on his own results only. Two different sharing games are thus defined. The Coordination Benefit is the improvement in net payoff when we move from an equilibrium of the first game to an equilibrium of the second. Classes of probability distributions and payoff fucntions have been found for which it is indeed the case that when search technology improves (y drops), the Coordination Benefit rises, in both the centralized and the decentralized cases.62 61
Again, see Bresnahan et al. (2002). Suppose, once again, that the payoff function W has the linear/quadratic structure and each ei is uniformly distributed on a closed interval. Then the methods of the Theory of Teams again allow us to find explicit formulae for the action rules used by HQ in the centralized/coordinated and centralized/ uncoordinated cases, and the rules used by each manager in the decentralized/coordinated and decentralzied/uncoordinated cases. In all cases one finds (for the number-of-sets cost measure and for the Shannon cost measure) that the Coordination Benefit rises when y drops. 62
Ch. 4. Organization Structure
273
3.2.3 Extending the model so that managers may be decentralized with regard to ‘‘production’’ as well as search; studying the effect of a searchtechnology improvement on the Decentralization Penalty when the ‘‘degree of decentralization’’ is varied Suppose that in our two-manager model the organizational payoff (or gross profit) function W takes the following form: W ¼ V ða; eÞ K 1 ða1 Þ K 2 ða2 Þ. For the action a ¼ (a1, a2), the function Ki is Manager i’s local production cost, while V (a, e), which may reflect externalities between the two managers, is the gross revenue earned by the organization when the environment is e. We modify our model of the decentralized organization. Each manager i now chooses (1) a partioning of Ei (as before) and (2) a rule ai which assigns a value of ai to every possible search result. He receives a share r V of V, but has to pay a share g of his production cost, with 0 ogo1. Thus his net payoff in the game is the expected value of r V minus the expected value of g Ki (when he uses the rule ai) minus the search cost for his chosen partitioning. In the first-best (centralized) situation, by contrast, the partitionings and the rules ai are chosen so as to maximize the expected value of V–K1–K2 minus the total search costs. We can call g the degree of decentralization. As g rises toward one, each manager bears more and more responsibility for his own production costs. We can again study the Decentralization Penalty, i.e., the amount by which the expected value of [V K1K2] ( the total search costs )] in the decentralized case falls short of its first-best value. The Penalty depends on g and on y. We now have a new question. Research challenge #10. When is it the case that a search technology improvement (a drop in y) ‘‘justifies’’ a higher degree of decentralization? When is it the case, in other words, that the value of g which minimizes the Decentralization Penalty for y¯ exceeds the value of g which minimizes ¯ the Penalty for y4y? This is a subtle question. In answering it, a key role is played by another question: For a given level of search technology ( a given y), is the value (to a manager) of another dollar spent on search higher for low g or for high g? What is intriguing about the latter question is that one’s off-the-cuff intuition can go either way. One could very roughly argue that ‘‘when g goes up, each manager ends up with less for every action, so learning a little more about ei becomes less valuable to the manager.’’ But one could just as well argue, in an equally primitive way, that ‘‘when g goes up, it becomes more important for each manager to get his action just right, so learning more about ei becomes more valuable.’’ One could start learning about the matter by studying a one-manager version. To give it a concrete setting, let the organization consist of a
274
T. Marschak
manufacturer and a franchised retailer. The manufacturer can produce and ship the product quantity q to the retailer at a cost C (q). He charges the franchised retailer a fee of g for each unit shipped. We can interpret g as the ‘‘degree of decentralization’’; the higher g is set, the higher the retailer’s contribution to the manufacturing cost. Once the manufacturer chooses g, the retailer is entirely on her own. She sells the received product in a local market. The demand curve she faces is P ¼ e Q, where P is the price, Q the quantity, e>0 and Qre. But e is a random variable. It changes each ‘‘week’’ and its possible values comprise an interval [A, B] with B>A. The retailer has to commit to the quantity she orders for next week’s sales before she knows next week’s demand curve. By spending money on search (market research) she can learn about next week’s e. She chooses a T-interval partitioning of [A, B] from a set of available partitionings and searches to find the interval in which next week’s e lies. Denote the T intervals I1, y , It, y , IT. The search costs the retailer the amount y T. Once the current interval, say It, has been found, the retailer chooses a quantity Tg Q^ ðtÞ so as to maximize the following conditional expectation: pðt; gÞ ¼ E½ðe QÞ Q g Q j e 2 I t . The retailer chooses T so as to maximize X ðprob: of I t Þ pðt; gÞ yT: t2f1;...;Tg
^ yÞ denote the retailer’s chosen T. Then the manufacturer’s exLet Tðg; pected profit depends on both g and y. It is X ^ ^ ðprob: of I t Þ ðg QTðg;yÞ;g ðtÞ CðQTðg;yÞ;g ðtÞÞÞ: ^ t2f1;...;Tðg;yÞg
He chooses the franchise fee g so as to maximize expected profit. We ask: When is it the case that a drop in y leads the manufacturer to raise g? When is it the case that the drop leads the manufacturer to lower g?63 63 Preliminary exercises suggest that both can happen. Going back to the general one-manager problem, consider two examples: Vða; eÞ ¼ ea_ and K (a) ¼ a2; (2) V ¼ J1/ae, K ¼ b In a, where J>0, b>0, a>0. In both cases, the Manager collects (for given a, e) the amount VgK, where 0rgr1. In both cases, let e be uniformly distributed on [A, B] and let the partitioning defined by T consist of T equallength intervals. The partitioning costs the Manager yT. It turns out that if we keep y fixed, then in the first example, raising g leads the manager to choose a lower value of T, while in the second example it leads him to choose a higher value.
Ch. 4. Organization Structure
275
3.3 Networks of self-interested decision-makers, who bear the network’s informational costs There is an abundant literature on the formation of networks, where every node is occupied by a self-interested person, who obtains information from those persons to whom he is linked.64 That literature usually takes as given a ‘‘value function’’ defined on the possible networks. The function expresses the network’s collective performance. Much of the literature then considers alternative ‘‘allocation rules.’’ Such a rule assigns a share of the network’s value to each of its participants. Suppose we specify a rule, and we also specify the cost of forming a link, borne by the two persons who form the link. Then we have a game, in which each person chooses the links to be formed with others, and each person collects his share of value (for the resulting network) minus his link costs. The equilibria of these games are studied, with particular attention paid to the structure of the equilibrium networks. (When is the equilbrium network a tree? When is it a ring?) The literature has developed a great many interesting results about equilibrium networks. In this framework, we could, in principle, study the impact of improved IT. Given a value function and an allocation rule, how does a drop in link costs (due to IT improvement) change the structure of the equilibrium networks? Suppose, however, that we venture in a new direction and we no longer take the value function and the allocation rule as exogenously given. Instead we let each of n persons, say person j, observe some external random variable ej. In addition, person j learns the current value of ek, for all persons k who are neighbors of j, i.e., there is a link between j and k. Finally, we let j be a decision-maker. He responds to the current value of his own and his neigbors’ external environments by taking an action. He then collects a gross revenue Uj, which depends on his action, the others’ actions, and the current value of e ¼ (e1, y , en). Let Vj denote the expected value of Uj. Person j’s net payoff is Vj minus his share of the cost of his links. (In one version we assume that a link’s cost is shared equally between its two users.) In our network formation game, each person chooses his intended neighbors and he also chooses a rule that tells him, what action to choose for each value of his intended neighbors’ and his own current external variables. When all n persons have made these choices, a network emerges. That network has a link between j and some k if and only if both j and k have chosen the link.
64
Two surveys are Jackson (2003a, 2003b).
276
T. Marschak
Research challenge #11. A collection of related questions awaits exploration: Which networks and action-rule n-tuples are stable, i.e., they have the property that if a player changes his intended neighbors or his action rule (while no one else makes any changes) then his expected payoff drops or stays the same? Pn Which networks and action-rule n-tuples are efficient, i.e., i ¼ 1 Vi minus the total link costs is as high as possible? When is a stable network efficient and vice versa? When is a stable network connected (there is a path between any two persons)? When is an efficient network connected? When is a stable network a tree and when is it a ring? When is an efficient network a tree and when is it a ring? For the case of a tree, does its number of tiers shrink as link costs go down—i.e., can we substantiate the classic claim that an improvement in IT indeed leads to the ‘‘flattening of hierarchies’’? When does an inefficient stable network exhibit shirking (total link costs are less than in an efficient network) and when does it exhibit squandering (total link costs exceed those of an efficient network)? Clearly this is an ambitious agenda. The existing literature has made progress on some of these questions, because it suppresses the decisionmaking role of each person and takes network value and its allocation as given. The agenda is even more challenging in the proposed new research path, where each person is explicitly a decision-maker. Nevertheless, progress has been made in the following case.65 Suppose the highest attainable value of the expected-revenue function Vj has the property that it depends only on the number of j’s neighbors. Here is a simple example where this is indeed so. Let each person j have one unit of a product to sell. Each of the n persons is located at a market where the product can be sold. The random variable ei is the product’s current price at location i. Given what he knows about the prices at his neighbors’ locations, person j has to choose the location where he will sell his product. That can be his own location, it can be the location of a neighbor, or it can be the location of a nonneighbor. Now suppose each ei takes just two values: zero and one, each with probability 12; whatever all the other ek (k6¼i) may be. Suppose person i is given a neighbor set, say the set Ni. He wants to choose a selling location so as to maximize the expected value of his revenue. It is clear that he may as well choose his own location if he observes ei ¼ 1 or if no location k in the set Ni has ek ¼ 1. But if he sees ei ¼ 0 and some neighbor k has ek ¼ 1, then he chooses the location k. His highest attainable expected revenue depends only on jN i j; the number of his 65
The results given here are due to Xuanming Su.
Ch. 4. Organization Structure
277
neighbors. It is jN i j 1 hðjN i jÞ ¼ 1 . 2 The function h has a fortunate ‘‘diminishing marginal product property’’: it increases more and more slowly as jN i j increases. When the number of neighbors is all that matters, and each person’s highest attainable expected revenue depends on the number of his neighbors through an increasing function h which (as in the above example) has the diminishing marginal product property, then a fairly complete analysis is possible. Let J be the link cost, and let g (J) denote the largest value of |Ni| for which h (|Ni|+1)h (|Ni|)ZJ. Let half of the cost of each link be paid by each of its users. The results one can establish include the following: P Let us measure a network’s net performance by n1 ¼ 1 h(|Ni|) ( total link costs). If a network is stable and its net performance is maximal among all stable networks, then it is efficient. If n1rg(J), then both stable and efficient networks are connected. If g(J)on1/2og(J/2), then an efficient network is connected but a stable network need not be. The net performance of an efficient network rises as J drops (i.e., IT improves), but more and more slowly. In an inefficient stable network there is shirking, not squandering. One might view the requirement that ‘‘only the number of neighbors matters’’ as very strong. One might guess, in particular that it rules out externalities, wherein one person’s action affects the payoffs collected by others. (There are no externalities in our selling-location example.) But this need not be so, as the following example shows: each person j again observes a random variable ej and again learns the current ek for every neighbor k. In response, he chooses an action aj (a real number). When the action vector a ¼ (a1, y , an) has been chosen, then for a given e ¼ (e1, y ,en), the entire organization obtains a payoff W ða; eÞ ¼
n X i¼1
ei ai
n X n X
qij ai aj ,
i¼1 i¼1
where the matrix ((qij)) is positive definite (which insures that there is a unique W-maximing a). person i’s own revenue is a share of W, namely r W, where 0or 1n: Thus there are externalities between the actions. Let the ei be independently distributed and let each be normally distributed with mean zero. For any given network, consider the possible action rules for each person j. The rule assigns a value of aj to each value of (ej, eN j ), where eN j denotes the external variables observed by j’s neighbors and hence
278
T. Marschak
learned by j as well. Because of the linear/quadratic payoff structure we may use, once again, the methods of the Theory of Teams. It turns out, in spite of the externalities, that in the rule n-tuple which achieves the highest expected value of W attainable by the network, each person’s action depends only on the number of his neighbors. If we now turn to the networkformation game, we find that for a given network, each person’s highest attainable expected payoff (the highest attainable expected value of r W), again depends only on the number of his neighbors, whatever action rules the n1 persons may have chosen. Moreover the function h, which gives that highest attainable expected payoff, has the required diminishing marginal product property.
4 Organizational models in which the primitive is a ‘‘task’’, ‘‘problem’’, ‘‘project’’, or ‘‘item’’ There are formal models of organization which do not explicitly consider the actions the organization takes in response to a changing environment. Instead the model supposes that ‘‘tasks’’ ‘‘problems,’’ ‘‘projects,’’ or ‘‘information items’’ flow in from the outside world, and the organization has to process them. They become the primitives of the model and are not further explained. One seeks to characterize organizations which process the flow efficiently, in some appropriate sense. Just as in the approaches we have discussed, processing costs (or capacities) are part of the model. To illlustrate, we briefly discuss three studies that share this approach but are otherwise rather different.66 In Sah and Stiglitz (1986), the organization has to judge ‘‘projects.’’ A project has a net benefit x, where x is a random variable with known distribution. The organization consists of evaluators. Once he has received a project, an evaluator either chooses to reject it or to pass it on to another evaluator for further evaluation. The total expected benefit of the project portfolio selected by a ‘‘hierarchy’’ is compared to that achieved by a ‘‘polyarchy.’’ In a hierarchy, the evaluators are arranged in two ‘‘bureaus,’’ called 1 and 2. All projects first flow to a member of Bureau 1, who either rejects it or passes it onto Bureau 2, where a final judgment is made. The only projects reviewed by Bureau 2 are those it receives from Bureau 1. In a polyarchy, the organization is divided into two ‘‘firms.’’ Each receives half of the projects. If it accepts the project no further evaluation occurs; if it rejects the project then the other firm performs a final evaluation. The quality of the evaluators can be varied, by changing the conditional distribution of x given the ‘‘reject’’ judgment. Costs could be 66 Other studies, which use similar primitives and pay some attention to informational costs, include the following Keren and Levhari (1983), Beckman (1983), Arenas et al. (2003), and Visser (2000).
Ch. 4. Organization Structure
279
attached to quality improvement but the 1986 paper only sketches how that might be done.67 In Bolton and Dewatripont (1994), the main primitive is a ‘‘cohort of M information items’’ received by the organization from an external source. Each item is a ‘‘type’’ of information about the outside world. All cohorts yield the same ‘‘value’’ R to the organization, once they are ‘‘processed.’’ (‘‘Processing’’ is another primitive, not further explained.) In order for the organization to realize the cohort’s value, at least one agent must receive the entire cohort. (Thus far the problem can be restated as the choice of a speak-once-only mechanism, with an action-taker who needs to know the entire cohort.) While one can study delay (time until at least one agent knows the entire cohort), the paper emphasizes another question. It supposes that economies of scale are achieved when a given agent processes a given type more and more frequently. A network of agents is given and each can be made a specialist in one or more items; he processes only those. For each given network, one seeks an assignment of types to agents so as to minimize ‘‘the total labor time spent per processed cohort.’’ In efficient networks, the labor time for a best assignment is minimal. Some suggestive properties of efficient networks are worked out. In Garicano and Rossi-Hansberg (2005), the agents who form organizations belong to a ‘‘knowledge economy.’’ In each time period each agent receives a problem whose level of difficulty can vary, and solves it if its difficulty is below her level of knowledge. (‘‘Problem,’’ ‘‘level of difficulty,’’ ‘‘solve,’’ and ‘‘level of knowledge’’ are primitives, not further explained.) Each problem is identified by a value of Z, a nonnegative number; a higher Z means the problem is harder. The population of possible problems has a known probability distribution over the possible values of Z. Each agent is endowed with a ‘‘cognitive ability’’ a, a random variable with known distribution. By incurring a cost, an agent can learn to handle all problems of difficulty up to a given level z. The cost is increasing in z and decreasing in a. An agent receives an income, which is increasing in the proportion of problems the agent is able to solve. An agent seeks to maximize income minus learning costs. But high-ability agents can help low-ability agents, and that leads to the formation of organizations. The structure of those organizations in an equilibrium of the economy is studied.68
67 In a further paper (Sah and Stiglitz, 1988), the organization becomes an n-person ‘‘committee.’’ All n persons judge each project and it is accepted if krn members judge it favorably. This time the trade-off between portfolio quality and cost is central. The cost measure is simply n itself. 68 Another model in which ‘‘task’’ is a primitive is developed in Malone (1987) and Malone and Smith (1988). These papers study a number of cost measures for a given organizational structure.
280 5
T. Marschak
Concluding remarks
We have followed a modeling path with four elements: environment, action, goal, and informational cost. The models describe, in detail, the process by which the organization reaches new actions when the environment changes, i.e., they describe the mechanism that the organization chooses. This is not an easy research path. A great deal more work needs to be done before one can judge whether the effort is worthwhile. In the present state of knowledge there is a formidable gulf between the propositions that can be proved and the complexities of organization and information as they appear in the real world. Yet without theory, it is hard to make sense of that reality and to see why casual claims, like those about the impact of IT on organizations, might or might not be true. Perhaps models that omit one or more of our four elements, or forego the detailed description that the concept of mechanism permits, or use entirely different primitives, may prove more useful guides—for the time being—to persons who are able to observe real organizations and assemble new datasets. But surely all four our elements arise in real organizations, and real organizations follow some procedure in choosing new actions. It seems inevitable that future empirical work will eventually try to examine that procedure in detail and will deal, in one way or another, with all four of our elements. Comparing mechanisms with regard to informational cost is particularly tricky. Minimal message-space size provides one fundamental way of judging the complexity of an organizational goal, and it tells us, in a preliminary and abstract way, how expensive one goal is relative to another if the goal is to be met by an organization which is decentralized in the sense that each mmber privately observes some aspect of the environment. But delay, the number of persons, and the individual communication and computation burden that each person faces are all important as well. Modeling of those costs is still in an early stage. We have looked primarily at work conducted by economic theorists. But there are parallel efforts in computer science and in artificial intelligence. Theoretical research that bridges disciplines is finally emerging.69 There are also many parallel efforts by social scientists who are not economists (e.g., persons in the Organizational Behavior field). A review of that literature 69 As noted in Section 2.2.8, the computer science specialty called Communication Complexity deals with minimal privacy-preserving mechanisms (protocols) for an n-person organization whose goal is the computation of an action that depends on n variables, each privately observed by one of the n persons. Both communication-complexity results and economic-theory results are central to the bridge-building paper by Nisan and Segal (2005) on the allocation of many objects among many persons, mentioned in Section 2.2.8. At present it seems fair to say that there is very much more work by theoretical computer scientists influenced by economic ideas, than work by economic theorists influenced by computer science ideas. In particular, computer scientists have explored the merits of market-based approaches to computational problems. Many papers which do so could be cited. A few of them are Deng et al. (2002), Walsh et al. (2003), Walsh and Wellman (2003).
Ch. 4. Organization Structure
281
would doubtless paint a very different picture as to what has been learned or could be learned about the effect of IT advances on organizational structure. Economic theorists are endowed (or perhaps burdened!) with a certain point of view when they approach such a challenging question. That point of view has thoroughly permeated this chapter. References Abelson, H. (1980). Lower bounds on information transfer in distributed computations. Journal of the ACM 27, 384. Abramson, N. (1963). Information Theory and Coding. McGraw-Hill, New York. Aizpura, J., A. Manresa (1995). A decentralized and informationally efficient mechanism realizing fair outcomes in economies with public goods. Economic Design 1, 141. Alon, N., D. Kleitman (1986). Covering a square by small perimeter rectangles. Discrete and Computational Geometry 1, 1. Arenas, A., L. Cabrales, et al. (2003). Optimal information transmission in organizations: search and congestion’’. Working Paper, Universitat Pompeu Fabra. Apostol, T. (1957). Mathematical Analysis. Addison-Wesley, Reading, MA. Beckman, M. (1983). Tinbergen Lectures on Organization Theory. Springer, Heidelberg. Bolton, P., M. Dewatripont (1994). The firm as a communication network. Quarterly Journal of Economics CIX, 809. Bresnahan, T., E. Brynjolfsson, L. Hitt (2000). Technology, organization, and the demand for skilled labor, in: M. Blair, T. Kochan (eds.), The New Relationship: Human Capital in the American Corporation, Brookings Institution, Washington. Bresnahan, T., E. Brynjolfsson, L. Hitt (2002). Information technology, workplace organization, and the demand for skilled labor. Quarterly Journal of Economics CXVII, 339. Calsamiglia, X. (1977). Decentralized resource allocation and increasing returns. Journal of Economic Theory 14, 263. Calsamiglia, X., A. Kirman (1998). A unique informationally efficient decentralized mechanism with fair outcomes. Econometrica 61, 1147. Chander, P. (1983). On the informational size of message spaces for efficient resource allocation processes. Econometrica 51, 919. Courtney, D., T. Marschak (Forthcoming). Shirking and squandering in sharing games, Topics in Theoretical Economics, Berkeley Electronic Press. Creti, A. (2001). Firms’ organization and efficient communication networks. The Manchester School 69, 77. Deng, X., C. Papadimitriou, M. Safra (2002). On the complexity of equilibria. Symposium on the Theory of Computing. Garicano, L., E. Rossi-Hansberg. (2005). Organization and inequality in a knowledge economy. Working paper. Groves, T., J. Ledyard (1977). Optimal allocation of public goods. Econometrica 45, 783. Harris, M., A. Raviv (2002). Organization design. Management Science 48, 852. Hurwicz, L. (1960). Optimality and efficiency in resource allocation, in: K. Arrow, S. Karlin, P. Suppes (eds.), Mathematical Methods in the Social Sciences, Stanford University Press, Stanford, CA. Hurwicz, L. (1986). On informational decentralization and efficiency in resource allocation mechanisms, in: S. Reiter (ed.), Studies in Mathematical Economics, Mathematical Association of America, Washington. Hurwicz, L. (1997). On the dimensional requirements of informationally decentralized Pareto-satisfactory processes, in: K. Arrow, L. Hurwicz (eds.), Studies in Resource Allocation Processes, Cambridge University Press, New York.
282
T. Marschak
Hurwicz, L., T. Marschak (1985). Discrete allocation mechanisms: dimensional requirements for resource allocation mechanisms when desired outcomes are unbounded. Journal of Complexity 1, 264. Hurwicz, L., T. Marschak (2003a). Comparing finite mechanisms. Economic Theory 21, 783. Hurwicz, L., T. Marschak (2003b). Finite allocation mechanisms: approximate Walrasian versus approximate direct revelation. Economic Theory 21, 545. Hurwicz, L., T. Marschak (2004). The informational efficiency of finite price mechanisms, in: C. Aliprantis K.J. Arrow, P. Hommand, F. Kubler, H.-M. Wu, N.C. Yannelis (eds.), Assets, Beliefs, and Equilibria in Economic Dynamics, Springer, Heidelberg. Hurwicz, L., S. Reiter (2006). Designing Economic Mechanisms. Cambridge University Press, New York. Hurwicz, L., H. Weinberger (1990). A necessary condition for decentralization and an application to intertemporal allocation. Journal of Economic Theory 51, 313. Ishikida, T., T. Marschak (1996). Mechanisms that efficiently verify the optimality of a proposed action. Economic Design 2, 33–68. Jackson, M.O. (2001). A crash course in implementation theory. Social Choice and Welfare 18, 655. Jackson, M.O. (2003). On the formation of networks and groups, in: B. Dutta, M.O. Jackson (eds.), Networks and groups: models of strategic formation, Springer, Heidelberg. Jackson, M.O. (2003a). The stability and efficiency of economic and social networks, in: B. Dutta, M. Jackson (eds.), Networks and Groups: Models of Strategic Formation, Springer, Heidelberg. Jackson, M.O. (2003b). A survey of models of network formation: stability and efficiency, in: G. Demange, M. Wooders (eds.), Group Formation in Economics: Networks, Clubs, and Coalitions, Cambridge University Press, New York. Jackson, M.O. (2004). A survey of models of network formation: stability and efficiency, in: G. Demange, M. Wooders (eds.), Group formation in networks, clubs, and coalitions, Cambridge University Press, New York. Jordan, J. (1982). The competitive allocation process is informationally efficient uniquely. Journal of Economic Theory 28, 1. Jordan, J. (1987). The informational requirements of local stability in decentralized allocation mechanisms, in: T. Groves, R. Radner, S. Reiter (eds.), Information, Incentives, and Economic Mechanisms: Essays in Honor of Leonid Hurwicz, University of Minnesota Press, Minneapolis. Jordan, J. (1995). Information flows intrinsic to the stability of economic equilibrium, in: J. Ledyard (ed.), The Economics of Informational Decentralization: Complexity, Efficiency, and Stability, Kluwer, Boston. Jordan, J., D. Xu (1999). On the communication complexity of expected profit maximization. Journal of Economic Theory 86, 185. Kaganovitch, M. (2000). Decentralization of intertemporal economies with discounted maximin criterion. International Economic Review 41, 1021. Karchmer, M. (1989). Communication Complexity: A New Approach to Circuit Depth. MIT Press, Cambridge, MA. Keren, M., D. Levhari (1983). The internal organization of the firm and the shape of average costs. Bell Journal of Economics XL, 474. Kushilevitz, E., N. Nisan (1997). Communication Complexity. Cambridge Universisty Press, New York. Leavitt, H., T. Whistler (1958). Management in the 1980s. Harvard Business Review 36, 6. Leontieff, W. (1947). A note on the interrelation of subsets of independent variables of a continuous function with continuous first derivatives. Bulletin of the American Mathematical Society 53, 343. Lova´sz, L. (1990). Communication complexity: a survey, in: B.H. Korte (ed.), Paths, Flows, and VSLI Layouts, Springer, Berlin. Malone, T. (1987). Modeling coordination in organizations and markets. Management Science 53, 1317. Malone, T., S. Smith (1988). Modeling the performance of organizational structures. Operations Research 36, 421. Manresa, A. (1993). An infinite dimensional extension of the theory of decentralized mechanisms. Mathematical Social Science 26, 157. Marschak, T. (1959). Centralization and decentralization in economic organizations. Econometrica 27, 399.
Ch. 4. Organization Structure
283
Marschak, T. (1972). Computation in organizations: the comparison of price mechanisms and other adjustment processes, in: C.B. McGuire, R. Radner (eds.), Decision and Organization, NorthHolland, Amsterdam. Marschak, T. (1987). Price versus direct revelation: informational judgments for finite mechanisms, in: T. Groves, R. Radner, S. Reiter (eds.), Information, Incentives, and Economic Mechanisms: Essays in Honor of Leonid Hurwicz, University of Minnesota Press, Minneapolis. Marschak, T. (2004). Information technology and the organization of firms. Journal of Economics and Management Strategy 13, 473. Marschak, J., R. Radner (1972). The Economic Theory of Teams. Yale University Press, New Haven. Marschak, T., S. Reichelstein (1995). Communication requirements for individual agents in networks and hierarchies, in: J. Ledyard (ed.), The Economics of Informational Decentralization: Complexity, Efficiency, and Stability, Kluwer, Boston. Marschak, T., S. Reichelstein (1998). Network mechanisms, informational efficiency, and hierarchies. Journal of Economic Theory 44, 106. Melumad, N., D. Mookherjee, S. Reichelstein (1992). A theory of responsibility centers. Journal of Accounting and Economics 15, 445. Melumad, N., D. Mookherjee, S. Reichelstein (1995). Hierarchical decentralization of incentive contracts. RAND Journal of Economics 26, 654. Melumad, N., D. Mookherjee, S. Reichelstein (1997). Contract complexity, incentives, and the value of delegation. Journal of Economics and Management Strategy 6, 257. Mount, K., S. Reiter (1974). The informational size of message spaces. Journal of Economic Theory 8, 161. Mount, K., S. Reiter (1977). Economic environments for which there are Pareto-satisfactory mechanisms. Econometrica 45, 821. Mount, K., S. Reiter (1987). On the existence of a locally stable dynamic process with a statically minimal message space, in: T. Groves, R. Radner, S. Reiter (eds.), Information, incentives, and economic mechanisms, University of Minnesota Press, Minneapolis. Mount, K., S. Reiter (1996). A lower bound on computational complexity given by revelation mechanisms. Economic Theory 7, 237. Mount, K., S. Reiter (1998). A modular network of bounded rationality, in: M. Majumdar (ed.), Organizations with Incomplete Information, Cambridge University Press, New York. Mount, K., S. Reiter (2002). Comunication and Complexity in Economic Behavior and Organizations. Cambridge University Press, New York. Nisan, N., I. Segal (2005). The communication requirements of efficient allocations and supporting Lindahl prices. Journal of Economic Theory 129, 192. Osana, H. (1978). On the informational size of message spaces for resource allocation processes. Journal of Economic Theory 17, 66. Osana, H. (2005). Externalities do not necessarily require larger message spaces for realizing Paretoefficient allocations. Review of Economic Design 19, 227. Radner, R. (1993). The organization of decentralized information processing. Econometrica 62, 1109. Radner, R., T. Van Zandt (1995). Information processing in firms and returns to scale, in: J. Ledyard (ed.), The Economics of Informational Decentralization: Complexity, Efficiency, and Stability, Kluwer Publishing Company, Boston. Reichelstein, S. (1984). Incentive compatibility and informational requirements. Journal of Economic Theory 32, 384. Reichelstein, S., S. Reiter (1988). Game forms with minimal message spaces. Econometrica 53, 661. Sah, K., J. Stiglitz (1986). The architecture of economic systems: hierarchies and polyarchies. American Economic Review LXXVI, 716. Sah, K., J. Stiglitz (1988). Committees, hierarchies, and polyarchies. Economic Journal 98, 451. Sato, F. (1981). On the informational size of message spaces for resource allocation processes in economies with publicg goods. Journal of Economic Theory 24, 48. Saari, D., C. Simon (1978). Effective price mechanisms. Econometrica 46, 1097.
284
T. Marschak
Segal, I (2004). The communication requirements of social choice rules and supporting budget sets. Working Paper. Serrano, R. (2004). The theory of implementation of social choice rules. SIAM Review 46, 377. Stoenescu, T. (2004). Decentralized resource allocation mechanisms in networks. Ph.D. dissertation, Department of EECS-Systems, University of Michigan. Tian, G. (1990). Completely feasible and continuous Nash implementation of the Lindahl correspondence with a message space of minimal dimension. Journal of Economic Theory 51, 443. Tian, G. (2004). A unique informationally efficient allocation mechanism in economies with consumption externalities. International Economic Review 45, 79. Van Zandt, T. (1998). Organizations with an endogenous number of information processing agents, in: M. Majumdar (ed.), Organizations with Incomplete Information, Cambridge University Press, New York. Van Zandt, T. (1999). Real-time decentralized information processing as a model of organizations with boundedly rational agents. Review of Economic Studies 66, 633. Van Zandt, T. (2003a). Real-time hierarchical resource allocation. Discussion Paper. Van Zandt, T. (2003b). Real-time hierarchical resource allocation with quadratic payoffs. Discussion Paper. Visser, B. (2000). Organizational communication structure and performance. Journal of Economic Behavior and Organization 42, 231. Walker, M. (1977). On the informational size of message spaces. Journal of Economic Theory 15, 366. Walsh, W., M. Wellman (2003). Decentralized supply chain formation: a market protocol and competitive equilibrium analysis. Journal of Artifical Intelligence Research 19, 513. Walsh, W., M. Yokoo, K. Hirayama, M. Wellman (2003). On market-inspired approaches to propositional satisfiability. Artificial Intelligence 144, 125. Williams, S. (2007). Communication in Mechanism Design: A Differential Approach. Cambridge University Press, New York. Ziv, A. (2000). Information technology and optimal firm structure. Journal of Accounting Research 38, 297.
Terrence Hendershott, Ed., Handbooks in Information Systems, Vol. 1 Copyright r 2006 by Elsevier B.V.
Chapter 5
Open Source Software: The New Intellectual Property Paradigm
Stephen M. Maurer and Suzanne Scotchmer University of California, Berkeley, CA 94720-7320, USA
Abstract Open source methods for creating software rely on developers who voluntarily reveal code in the expectation that other developers will reciprocate. Open source incentives are distinct from earlier uses of intellectual property, leading to different types of inefficiencies and different biases in R&D investment. The open source style of software development remedies a defect of intellectual property protection, namely, that it does not generally require or encourage disclosure of source code. We review a considerable body of survey evidence and theory that seeks to explain why developers participate in open source collaborations instead of keeping their code proprietary, and evaluates the extent to which open source may improve welfare compared to proprietary development.
1
Introduction
Open source software, which burst on the innovation scene in the mid1990s, is produced in a completely different way than other commercial products. Workers are usually unpaid; management and direction are limited; and legal restrictions on using the product are modest (Lerner and Tirole, 2004). The open source style of development has various features, but it generally involves software developers making their source code available free-of-charge to end users and improvers, usually subject to
285
286
S. M. Maurer and S. Scotchmer
license restrictions such as GPL and BSD.1 Developers often work on code provided by others. The open source movement is a substantial phenomenon. LINUX runs on 29 million machines (The Linux Counter, 2005) and Apache runs on 70 million servers (Netcraft, 2005). Despite this demonstrated success, survey evidence indicates that the nature of open source activities is changing rapidly (Comino et al., 2005). For example, the survey data of Ghosh et al. (2002) show that there is a secular shift from ‘‘hobbyist’’ contributions to ‘‘commercial’’ contributions. It is still unclear to what extent open source will supplant proprietary methods for software development, let alone branch out into other information goods such as pharmaceuticals or geographic data. In this essay, we provide a snapshot of the emerging open source phenomenon, and a discussion of how scholars have tried to make sense of it. There are several natural questions to ask about the phenomenon. Among them, How do open source methods provide sufficient incentive to invest in software, given that users do not pay innovators? What is it about computer software, if anything, that calls for a new invention paradigm? Which other inventive activities share these features? Does the efficacy of open source depend on licenses (e.g., BSD, GPL) and, indirectly, the underlying availability of intellectual property protection? Does the market need a coordination mechanism to collectively choose open source over more traditional ways of exploiting intellectual property, and does it always do so when open source is the better choice? In what circumstances does open source work better than traditional intellectual property incentives or other funding schemes, such as public sponsorship? In Section 2, we lay out the various arguments for why open source works as an incentive scheme, and compare it to more traditional uses of intellectual property. In Section 3, we focus on how open source collaborations are organized. In Section 4, we turn to some of the observable welfare consequences of organizing R&D in this fashion, and in Section 5 we discuss some of the gaps in what is known about open source, and its potential to organize invention in other arenas. 1 The General Purpose License (GPL) is a ‘‘viral’’ license that obligates a further developer of the software to make it available under the same license. In general, there are no restrictions on use or an obligation to pay, but in some versions there is an obligation for attribution. The Berkeley Software Distribution (BSD) license originated through UNIX development at the University of California, Berkeley and is not viral. It requires users to give attribution credit to the University but does not prohibit commercial use or development. For a more complete history, see, for example, Weber (2004).
Ch. 5. Open Source Software: The New Intellectual Property Paradigm 287
2
Incentives for R&D
Not all R&D environments call for the same incentive structure. For example, R&D environments differ in the extent to which ideas for investments are scarce or common knowledge, the extent to which disclosure of progress is inevitable or important, the extent to which innovation proceeds cumulatively through the efforts of many contributors, and the extent to which natural restrictions on entry protect innovators. The open source movement emerged to support an industrial product (software) for which disclosure of code is particularly useful, but not required by intellectual property law. Copyrights for software can be registered without fully revealing the source code, and source code is typically not included in software patents.2 Source code is typically not disclosed in either regime. Of course, code can be released under license, but here the nature of the innovative environment matters. If ‘‘ideas are scarce’’ in the sense that each idea for an improvement occurs to a single, random person (Scotchmer, 2004, Chapter 2), and ideas depend on prior disclosure, then traditional protection through patents and copyrights may cripple inventive activity. This is because rights holders do not know to whom they should license and disclose. In the open source regime, full disclosure is automatic, and thus encourages new ideas and re-use of the code by developers who cannot be identified in advance. The surprise is that this can be done while still preserving incentives. In this section we explore those incentives, but first compare open source with the more traditional way of exploiting intellectual property. Open source communities have been well studied with survey instruments. When asked their motives, survey respondents cite various incentives including: own use benefits, complementarity with proprietary products sold in the market, signaling, education, and social psychological motives such as altruism or simple enjoyment. In terms of the technical problems that contributors seek to address, Ghosh et al. (2002) report that 39.8% are trying to improve the products of other developers. Fewer are trying to realize a good product idea (27%), or trying to solve a problem that could not be solved by proprietary software (29.6%). Among contributors at SourceForge, the top three reasons for participating in open source communities include ‘‘work functionality’’ (33.8%) and ‘‘non-work functionality’’ (29.7%)3 (Lakhani and Wolf, 2005). To control for the possibility that these responses are sensitive to the number and phrasing of the questions, Lakhani and Wolf (2005) use factor 2 See Lemley et al. (2002) at 204-205 (for patents), Samuelson (1984), and U.S. Copyright Office (2002) (for copyrights). 3 The most common response is ‘‘intellectually stimulating’’ (44.9%). Ideological responses (‘‘Beat Proprietary Software’’) are much less frequent (11.9%) (Lakhani et al., 2002; Lakhani and Wolf, 2005).
288
S. M. Maurer and S. Scotchmer
analysis to group the responses into four classes: workers who are primarily motivated by education/intellectual stimulation (‘‘Learning and Fun’’— 29%), by non-work user needs (‘‘Hobbyists’’—27%), by work-related user needs (‘‘Professionals’’—25%), and by feelings of obligation/community (‘‘Community Believers’’—19%). Significantly, the two user needs categories comprise about one-half of all respondents. Not surprisingly, different communities report different incentives. Surveys of the embedded-LINUX community find that most hardware firms release code in order to continue receiving similar donations from others (61.4%), benefit from other participants’ efforts to find and fix bugs (59.9%), to be known as a good player in the open source community (58.9%) and because they hope that others will develop their code further (57.7%). Employees working for software companies report broadly similar motives except that they tend to place slightly more emphasis on marketing (e.g., signaling and reputation). This effect is larger for small, young companies than for older and more established firms (Henkel, 2005b). 2.1 Intellectual property and open source Because open source puts knowledge (software) in the public domain, it would not perform well as an incentive mechanism in the usual innovation environment where the objective is to prevent imitation. To the contrary, the point of putting the knowledge in the public domain is to encourage imitation. As we will see in this section, open source works in environments where the knowledge created (a) is complementary to some other good whose profitability is immune to imitation, such as human capital or another proprietary product, or (b) where the motives to invent are intrinsic and have nothing to do with appropriating value.4 We begin our discussion by comparing open source and the ordinary use of intellectual property in the two environments where open source is mainly used: where innovations are complementary and where innovation is cumulative. Following Scotchmer (2004), we distinguish between having ideas, which are random and costless, and creating innovations, which require investment. Let the idea of firm i be summarized by an indicator of its commercial value and its private cost, (vi,ci). We describe complementarity by the following profit function, for each firm i, where n is the total number of contributions to the open source project, and f is positive, increasing and bounded: vi f ðnÞ ci . 4 For discussion of how economists have treated the design of intellectual property in regard to these issues, see Scotchmer (2004). For a broader set of references that also includes legal scholarship, see Menell and Scotchmer (forthcoming).
Ch. 5. Open Source Software: The New Intellectual Property Paradigm 289
Complementarity produces a type of network effect: contributors prefer to contribute to open source projects with many participants. Even though each developer’s code is freely usable by others, the commercial value vi is assumed to survive. The commercial value must therefore be due to some other proprietary product that embeds the software, and not directly to the software itself. To compare open source with patents, assume that if firms keep their contributions proprietary, they cross-license symmetrically at some price ‘: Then with n contributors, each firm earns licensing revenue (n–1)‘: However, each firm also has licensing obligations in amount (n–1)‘; so there is no net burden due to licensing. GPL leads to the same end without imposing the license fees ‘ and without the burden of negotiating. Notice, however, that participating in such a scheme might not be the best thing from either a private or social point of view if most of the users are not developers so that the open source community confers benefits on non-reciprocating third parties. If such use constitutes the bulk of the social benefits, and if development costs are relatively high, a better incentive scheme would involve royalties. Royalties from third parties may be necessary to cover costs. Now suppose that the software contributions are cumulative. For example, suppose that a set of proprietary products embed a common software product, such as an operating system, which can be improved sequentially by other developers who find bugs or see opportunities for improvement. Suppose further that each improvement increases the quality of the embedded software and that ideas for improvement occur randomly to members of the community to whom the product has been disclosed. In the cumulative context, traditional intellectual property (IP) has the same problem as with complements. If ideas for improvement are scarce, so that a potential improver cannot be identified in advance, then licensing on a one-by-one basis to disclose the prior code will not work very well. But, as with complements, if each member makes the same number of contributions as every other, at least in expectation, then their expected receipts and payouts from licensing will be equal for each contributor, and equivalent to making no payments at all under a GPL scheme. However, the cumulative context is where we see that a viral GPL license may be useful. A GPL system locks the proprietors into a royalty-free exchange of knowledge. Unlike the case of complements, knowledge exchange is not symmetric because later developers can free ride on earlier ones, but not vice versa. Absent GPL, a developer might be tempted to free ride on earlier developers and make his code proprietary so that he can impose royalties on later developers. With GPL, he can only revert to royalties by building his code from the ground up. This may be more costly than using prior code and accepting the GPL. Because the whole point of making code open is to encourage imitation and use, the code itself will not be a profit center for its developer. In the
290
S. M. Maurer and S. Scotchmer
remainder of this section, we discuss why open source works as an incentive system, namely, that the benefits come from own use, other products sold in the market, signaling, and education. We also discuss social psychology motives that claim to transcend traditional economic incentives. Despite broad similarities, these incentives lead to different behaviors and welfare implications. 2.2 Own use User innovation has been documented as far back as Victorian times.5 By the late 20th century it was ubiquitous: examples include everything from machine tools to windsurfing (Lerner and Tirole, 2002a; Harhoff et al., 2003; Von Hippel, 2004). The modern open source movement extends this user innovation model to software (Raymond, 1999). The Apache web server collaboration provides a leading example. Writing code for own use is not entirely about volunteer labor. Eighty six percent of contributors who participate in open source projects for work reasons are paid for at least some of their work. Not surprisingly, paid contributors spend almost twice as much time as volunteers—10.3 hours compared to 5.7 hours (Lakhani and Wolf, 2005). Corporations that employ own use strategies to make or improve products that will be sold to mass market products can afford to bear more cost than individuals. At the same time, own use includes substantial non-commercial activity. Lakhani and Wolf (2005) report that 27% of SourceForge contributors write code for ‘‘non-work needs.’’ Since 10.9% of all SourceForge listings involve games (Comino et al., 2005), ‘‘hobbyist’’ motives are apparently important. Own-use incentives may lead to underprovision of code, since the investing party does not appropriate the benefits conferred on third parties. While reciprocity within the open source community may avoid duplication, it does not solve this basic deficiency in incentives. This is also true in the simple model of complementarity that we started with, where every contributor is concerned with own use. The user will not invest if his ci is larger than his own benefit P vif(n), even if the cost is smaller than the social benefit ½ f ðnÞ f ðn 1Þ j nj . 2.3 Complementary goods and services We have already stressed that open source incentives will not work if the open source software must itself be a profit center. This is because 5 The practice of creating new information for one’s own use is as old as mankind. Scholars have long argued that the apparent perfection of European folklore reflects accumulated interactions (and incremental improvements) of storytellers and audiences. See, e.g., Bettelheim (1976). The high ranking of Homer’s Iliad and Odyssey among Western ‘‘great books’’ is an obvious example.
Ch. 5. Open Source Software: The New Intellectual Property Paradigm 291
imitation is its very lifeblood. Instead, the open source activity must be complementary with something that remains proprietary. West and Gallagher (2004) refer to open source as ‘‘pooled R&D.’’ In particular, companies share code to test software, fix bugs, and to get improvements, feedback, and extensions (Rossi and Bonaccorsi, 2003, 2005), all of which they would otherwise have to do independently with substantial duplicated costs. Contributors can afford to cooperate in this way because the open source software is bundled into different goods and services that are mostly non-rival in the market and allow individual contributors to appropriate benefits. Typical complements include proprietary operating systems; proprietary applications programs; hardware; documentation; distribution through trusted and convenient brand name channels; bundling open source software into convenient, ready-to-use packages; tech support and warranties; custom software services; consulting, education and training; remote services; complete data solutions; making applications more reliable for particular applications or libraries; and organizing fairs and conferences (Raymond, 1999; Ghosh et al., 2002; Hawkins, 2002; Harhoff et al., 2003; O’Mahoney, 2003; Varian and Shapiro, 2003; West, 2003; Dahlander, 2004; West and Gallagher, 2004; Henkel, 2005b). Complements are particularly important for server software, desktop/ client software, enterprise solutions, IT consulting, IT services, and the embedded software used in appliances like DVDs and cell phones (Ghosh et al., 2002). Commercial firms are less likely to participate in open source development where competition among them is significant (von Hippel, 2002; Harhoff et al., 2003). Harhoff et al. (2003) present a model with two users who practice in-house innovation, and are imperfect competitors selling products that are enhanced by the open source product.6 Even though each firm’s rival would benefit from a disclosure of its code, the firm may nevertheless disclose it in the hope that a third-party manufacturer will develop the disclosed product still further. Harhoff et al. find that, if competition, technology spillovers, and the cost of adopting disclosed improvements are high, there is an equilibrium in which neither rival discloses. However, if the cost of adopting the manufacturer’s improvement is less than its benefits, there is also equilibrium where both disclose, provided competition and spillovers are low enough. Henkel (2005a) explores a model in which two firms each need two distinct technologies to manufacture their products. If firms cannot share information, each must invest in both technologies. This raises entry costs and makes monopoly more likely. If they choose open source, there are Nash equilibria where each firm specializes in one technology and obtains
6 Harhoff et al. claim that this symmetric duopoly analysis remains qualitatively correct for moderately large or asymmetric oligopolies.
292
S. M. Maurer and S. Scotchmer
the other through free riding. Henkel finds that firms with similar technology needs disclose even where competition is strong. However, firms may or may not share information where their needs are different. Henkel finds equilibria in which both firms disclose. Each user performs whatever R&D generates the most value for itself and free rides otherwise. In this game, heterogeneous needs suppress the temptation to free ride but still produce useful technology spillovers for the entire industry.7 Models with competition require two strong assumptions. First, each game assumes payoffs in which the parties can earn (and split) non-zero economic profits. Rivals are protected against competition and entry by some unspecified means. The key issue of open source—appropriability is introduced as an assumed parameter. Second, the models assume that parties cannot negotiate licenses with one another. In our view, this makes any comparison with intellectual property suspect, since licensing would also allow the firms to avoid duplication. The authors argue that their licensing assumption is justified by high transactions costs, the legal difficulty of patenting minor (but cumulatively important) innovations, and the alleged weakness of patents and trade secrets (Von Hippel, 2002; Harhoff et al., 2003; Henkel, 2005a).8 Empirical studies of the ‘‘embedded LINUX’’ used in proprietary electronic devices like DVDs and cell phones (Henkel, 2005b) provide a market example in which firms can decide between keeping code proprietary and mutual disclosure. In this case, loopholes in the GPL license give manufacturers the power to keep certain classes of code confidential if they want to.9 Despite this, roughly, half of all industry members (49%) participate in at least limited sharing, and this fraction is growing. In general, firms with strong complements tend to release code more readily. For example, 34.5% of hardware companies release code but only 28.6% of software houses do. More surprisingly, small firms reveal substantially more code than large ones. Henkel (2005b) argues that these firms would prefer to develop code in-house, but lack the resources to do so. Small companies frequently rely on open source communities to fix bugs and improve software (Rossi and Bonaccorsi, 2005).
7 Henkel analogizes this result to a jukebox, in which multiple patrons with heterogeneous tastes produce a stream of music that benefits everyone. 8 We note that evidence showing that most licenses earn modest royalties is equally consistent with the proposition that licensing is efficient. 9 The reason for the loophole is that GPL says that customers are entitled to source code, but confers no broader right on the general public. In the case of embedded LINUX, the customers tend to be a small number of device manufacturers who have good reason to keep their software secret. Other tactics for evading GPL disclosure requirements include releasing drivers as loadable binary modules rather than source code (done at least sometimes by 53.1% of industry respondents); tying revealed software to secret or copyrighted code; releasing code after a delay of up to 18 months (35.7% of firms); and redesigning software architecture so that functions are moved to the (proprietary) application layer (Henkel, 2005b).
Ch. 5. Open Source Software: The New Intellectual Property Paradigm 293
2.4 Signaling Open source software grew out of an academic environment in which researchers write articles to improve their career prospects. There is extensive anecdotal evidence that high-profile open source programmers can trade on their reputations to gain job offers, shares in commercial companies, and possibly access to venture capital (Raymond, 1999; Kogut and Metiu, 2000; Lerner and Tirole, 2002a). Hann et al. (2004) argue that star programmers are an order of magnitude more productive than their peers, so there is much to signal. The study of the Apache collaboration by Roberts et al. (2006) shows that signaling benefits grow stronger as workers rise in rank. A key feature of traditional patent or copyright incentives is that the private value of the right increases with the social value of the contribution. Hence, the prospect of winning an intellectual property right creates a screening mechanism. When a potential inventor has an idea for an innovation, he will compare its cost to a correlate of its social value before deciding to invest. With signaling incentives, the benefits of the R&D investment are still correlated with the social value of the investment, but less so. Instead of investing in the products with most consumer value, contributors will choose projects that showcase their technical virtuosity (Lerner and Tirole, 2002a). Signaling incentives, therefore, explain why open source projects tend to involve server operating systems, programming languages, and other applications aimed at sophisticated users (Schmidt and Schnitzer, 2002). Osterloh (2002) argues on this basis that open source collaborations listen to their smartest users while, for example, Microsoft listens to the dumbest. Similarly, Kollock (1999) argues that, as a consequence of signaling, mass market software tends to be underserved, and that open source software ignores ‘‘duller, more complex, but no less useful public goods.’’ Signaling is also weak for such useful tasks as reporting bugs, submitting comments, suggesting new functionality, preparing documentation, building easy-to-use interfaces, providing technical support, ensuring backwards compatibility, and writing programs for utilitarian tasks like power management or wizards (Schmidt and Schnitzer, 2002; Osterloh and Rota, 2004; Rossi, 2004). Empirically, the sole Apache rank open to bug hunters has no measurable impact on salary (Hann et al., 2004). Signaling is not confined to individuals. Case studies and survey evidence show that computer companies also participate in open source in order to build reputation, although the number of such companies seems to be small (Ghosh et al., 2002; Dahlander, 2004). The Henkel and Tins (2004) study of embedded LINUX finds that 45.4% of manufacturing companies disclose code in order to add to technical reputation, and 32.6% disclose to get on mailing lists as a form of marketing. As in other incentive mechanisms, signaling can create agency problems, such as hiding errors or claiming credit for the work of others. Johnson
294
S. M. Maurer and S. Scotchmer
(2004) argues that open source peer reviewers may collude to hide flaws in each other’s code. The only empirical evidence of agency problems we have found is due to Gandal and Ferschtman (2005), who point out that signaling incentives are likely to be more significant for licenses such as GPL that ban commercialization than for those such as BSD that allow it. They find that SourceForge contributors submit 2.9 times more lines of code to BSD-type licenses than to GPL-type licenses, and interpret this data as evidence that signaling incentives become less important once contributors deliver enough code to obtain formal credit. Signaling incentives can also have an impact on code architecture. Schmidt and Schnitzer (2002) speculate that increased modularity makes individual contributions more visible.10 Assuming that modularity does promote signaling, Weber (2000) argues that open source members may engage in ‘‘strategic forking’’ to become a leader, that is, unnecessarily splitting a collaboration. Dalle and David (2003) similarly hypothesize that programmers gain more reputation by launching new code than by contributing to an existing project; by working on early releases rather than later ones; and by working on frequently called modules (e.g., kernels) instead of applications. Dalle and David point out, somewhat optimistically, that these effects can be beneficial if, for example, working on new modules is socially more valuable than extending existing ones. There is extensive evidence that signaling works. Programmers often receive job offers, stock, and other benefits (Lerner and Tirole, 2002a). Many programmers reportedly believe that being a member of the LINUX community ‘‘commands a $10,000 premium on annual wages.’’ (Kogut and Meitu, 2000). Statistical studies by Hann et al. (2004) confirm that each promotion above the lowest rank boosts Apache programmer salaries by 13.3–29.3%. Similarly, surveys by Bonnacorsi and Rossi (2003, 2005) and Henkel (2005b) confirm that many commercial firms use open source collaborations to find new workers.11 Finally, Lakhani and Wolf (2005) and Lakhani et al. (2002) use factor analysis on their survey to sort respondents into four groups, including ‘‘professionals’’ (25%) who are motivated by signaling (‘‘gaining status’’) as well as solving work needs. This group is only slightly smaller than their largest group (29%), ‘‘learning and fun.’’ It is harder to know how powerful the signaling incentive is. Lakhani and Wolf (2005) compare reported incentives against the number of hours worked each week, and find that signaling (‘‘reputation’’) has only about one-third as much impact as the most powerful predictor, creativity. However, this average figure probably obscures the importance of signaling 10 Baldwin and Clark (2003) disagree, arguing that large numbers of modules dilute the superiority of any one contribution. 11 This pattern is not universal. Henkel and Tins (2004) report that only 11.5% of hardware manufacturers participate in embedded LINUX in order to find potential employees.
Ch. 5. Open Source Software: The New Intellectual Property Paradigm 295
within specific projects. For example, Roberts et al. (2006) find that highstatus Apache volunteers contribute more code than other members. 2.5 Education Due to its emphasis on peer review, open source provides particularly a good vehicle for education (Lakhani and Wolf, 2005). Education incentives are closely aligned with signaling, but focus on objective skills rather than the perceptions of others. For this reason, they avoid the free rider and agency problems referred to above. Education incentives explain why surveys routinely find roughly one-fifth of all open source volunteers are students.12 Education also looms large in how the collaborators report their own motives. In the survey of Lakhani and Wolf (2005), improving skill (41.3%) is the second most common incentive reported by open source volunteers, behind intellectual stimulation (44.9%). Other responses are markedly less important. In the survey of Ghosh et al. (2002), 70.5% of respondents report that they are motivated by learning new skills and 67.2% are motivated by sharing knowledge and skills. These are by far the most commonly cited incentives.13 In the factor analysis of Lakhani and Wolf (2005) and Lakhani et al. (2002), the largest group is ‘‘learning and fun’’ at 29%. 2.6 Achieving network externalities and denying them to others Achieving a favorable market position through network externalities is one of the most important strategic goals for companies in the new economy. Schmidt and Schnitzer (2002) argue that the importance of network effects and switching costs ‘‘is largely independent of whether the software is proprietary or open source.’’ Strategies for solidifying a market position can be loosely grouped into four categories. Achieving common standards. Adopting a single, industry-wide open source standard for software fosters a common pool of skilled workers, reduces costs associated with unnecessary versioning, increases the total number of programmers submitting bug reports and extensions, and avoids transactions cost associated with intellectual property such as ‘‘patent 12 Lakhani and Wolf (2005) report 19.5% of all open source collaborators are students. Hertel et al. (2003) report a 23% figure, while Ghosh et al. (2002) report 21%. 13 Henkel and Tins (2004) report similar results for the embedded LINUX industry. Among developers who work for software companies, the most commonly reported motivations are getting better personal skills (66.7%), recognition (60%), feedback to boost performance in their current job (56.9%), and demonstrating their skills to future employers (41.7%). Results for developers working for hardware companies are broadly similar. Not surprisingly, education and signaling incentives are even more important for contributors who work for universities, non-profits, and hobbyists. The most common survey responses for this group include obtaining feedback to improve personal skills (75.0%), improving technical reputation (53.6%), and demonstrating skills to future employers (50.0%).
296
S. M. Maurer and S. Scotchmer
thickets’’ and ‘‘anticommons’’ effects (Ghosh et al., 2002; Lerner and Tirole, 2004; Rossi, 2004; Rossi and Bonaccorsi, 2005). Such benefits are particularly attractive for competitively supplied code that is likely to earn low profit margins in any case (West and O’Mahoney, 2005). Market penetration. Releasing code in an open source format facilitates customer acceptance by (a) making it impossible for manufacturers to raise prices at a later date, (b) creating a community of developers who will continue to support the code even if the original author abandons it, and (c) releasing information about APIs that dramatically reduces consumer switching costs if companies fail to deliver on their promises (Raymond, 1999; Varian and Shapiro, 2003). Influencing standards. Manufacturers often join open source collaborations in order to steer code in directions that favor their own technology. If first mover advantages are strong, companies may even race to reveal code in order to preempt alternatives and gain a permanent advantage (von Hippel, 2002; Harhoff et al., 2003; Varian and Shapiro, 2003; West and Gallagher, 2004). The effect of these dynamics is ambiguous. For every company that hopes to influence code, others may fear that the open source collaboration will be hijacked so that it no longer supports their needs (Ghosh et al., 2002; Lerner and Tirole, 2002b). Such fears may persuade would-be participants that it is better not to support a project in the first place. Alternatively, companies may decide that joining open source collaborations is the best way to detect and prevent hijacking. Blocking market dominance by others. Finally, companies may support open source as a counterweight to dominant proprietary standards such as Windows (Kogut and Metiu, 2000). Individuals may similarly decide that open source is the best way to prevent large corporations from controlling the tools on which their livelihoods depend. O’Mahoney (2003) says, ‘‘Informants spoke of their contributions as investment in their future tools: they are creating code that they will never have to pay someone to use again.’’ Kogut and Meitu (2000) report that LINUX contributors are frequently motivated by ‘‘a common fear of Microsoft.’’ Is any of this important? Henkel and Tins (2004) report that only 30% of manufacturers in the embedded LINUX industry reveal their code in hopes of making it an industry standard. 2.7 Social psychology So far, we have concentrated on monetizable rewards. But everyday experience confirms that charitable voluntarism can also be a significant force. Experimental economics routinely finds that people contribute more to the provision of public goods than self-interest alone can explain (Kogut and Metiu, 2001). In some cases, voluntarism may also offer quality advantages. For example, blood from volunteer donors contains
Ch. 5. Open Source Software: The New Intellectual Property Paradigm 297
fewer contaminants (e.g., hepatitis) than blood purchased on the open market (Titmuss, 1972). Social psychology is the natural lens to look at non-monetizable incentives. Following Lakhani and Wolf (2005), we distinguish between extrinsic motives (i.e., doing an activity for some separable consequence) and intrinsic motives based on enjoyment or a sense of obligation or community (Osterloh et al., 2003a; Lakhani and Wolf, 2005). Extrinsic motivations include the desire for reputation within the open source community, ‘‘ego boost,’’ ‘‘feelings of personal efficacy,’’ or other similar incentives (Raymond, 1999; Weber, 2000, 2004). By contrast, intrinsic motivations do not require an audience. They include creative pleasure, sometimes characterized by a ‘‘flow state’’ in which the individual loses track of time (Lakhani and Wolf, 2005), the desire to be part of a team (Hann et al., 2004), the ability to express creativity, and experiencing satisfaction and accomplishment (Benkler, 2002; Roberts et al., 2006). Intrinsic motivations also include altruistic incentives, identifying with a particular group, or ideological opposition to proprietary software and software makers14 (Kollock, 1999; Hertel et al., 2003; Osterloh et al., 2003b; Rossi and Bonaccorsi, 2005). In choosing projects, extrinsic motivations may lead volunteers to take account of the social benefits they confer, but intrinsic motivations do not. Volunteers will join projects that place a premium on creativity rather than algorithmic solutions; are neither too easy nor too difficult; are challenging, fun, and simple to learn; and are interesting, fast-moving and even glamorous (Kollock, 1999; Dahlander and Magnusson, 2005). Social psychology incentives may also be unusually strong for groups such as teenagers (Benkler, 2002). This may explain why open source contributors are overwhelmingly young, male, and single (Ghosh et al., 2002, Lakhani and Wolf, 2005). Henkel and Tins (2004) and Hertel et al. (2003) report, respectively, that open source contributors are 98% and 96% male. Commentators claim that social psychology incentives work best when contributors are only asked to contribute small amounts to the public good (Baldwin and Clark, 2003; Osterloh et al., 2003b; Osterloh and Rota, 2004). Nevertheless, even weak incentives can sometimes be important. For example, Baldwin and Clark (2003) argue that the decision to reveal previously written code is a Prisoners Dilemma game. Here, intrinsic motivation 14 The most famous suggestion for obligation/community incentive is due to Raymond (1999), who argued that open source was driven by a post-modern ‘‘gift culture’’ in which social status is determined ‘‘not by what you control but by what you give away.’’ This view represents an elaboration of an older anthropology literature in which gift giving creates the ‘‘compulsion to return a gift,’’ confers status and power on the giver, fosters kinship-like relations in which each person ‘‘takes what they need and gives what they can’’; and encourages gift recipients to praise the giver and feel solidarity with the community (Bergquist and Ljungberg, 2001; Zeitlyn, 2003). One problem with Raymond’s (1999) original formulation argument is that it claimed that gift culture was characteristic of a postscarcity society. As Weber points out, this argument ignores the fact that ‘‘time and brainspace of smart, creative people are not abundant’’ (Weber, 2000, 2004).
298
S. M. Maurer and S. Scotchmer
may provide sufficient reward to cover the small (but non-trivial) cost of communication.15 Similarly, Weber (2000) argues that shared culture and norms help to suppress open source forking. Even weak norms may be amplified if network externalities favor a single, dominant standard. This can happen if the average programmer favors strong economies of scale, dislikes porting to multiple versions, and wants to minimize conflict (Lerner and Tirole, 2002a). The significance of social psychological incentives can also change over the life of a project. Osterloh and Rota (2004) and Franck and Jungwirth (2002) argue that social psychological incentives are most significant for young projects where monetary or reputation incentives are comparatively weak. On the other side, Lerner and Tirole (2002a) argue that visibility of early contributions encourages reputation seekers to join at an early stage. Conversely, social psychology incentives are expected to weaken as the original volunteers lose energy or mature projects become less fast paced and interesting (Lerner and Tirole, 2002b; Dahlander and Magnusson, 2005). Social psychology motives may also be less important for companysponsored projects, which usually start with a large mass of mature code that offers little sense of ownership or creative excitement (West and O’Mahoney, 2005). Finally, ideologically motivated volunteers may decide that the best way to accomplish their goals is to start as many new collaborations as possible. If so, their best strategy may be to leave projects as soon as signaling or monetary incentives kick in (Franck and Jungwirth, 2002). A virtue of enjoyment as an incentive is that free riding does not destroy it, at least in principle (von Hippel, 2002). Open source collaborations based on social psychology incentives, therefore, escape the ‘‘game of chicken’’ dynamics that cause delay under other incentives. Bitzer et al. (2004) explore games in which developers gain utility from a combination of conventional own-use incentives and two social psychology incentives (fun and pleasure from giving gifts). They find that if the value of social psychology incentives exceeds expected costs, open source members start to produce code immediately. The situation is different, however, if volunteers need both types of incentives to cover costs. In this new equilibrium, one member develops code immediately while the others wait. Bitzer et al. also explore an extended model in which agents have a finite life because they expect to change jobs and/or see their human capital become obsolete. In this richer model, whichever agent expects to receive open source payoffs for the longest time realizes that she cannot win a waiting game and starts to write code immediately. Based on this analysis, Bitzer et al. argue that real-world open source development should be swift. They also predict that open 15 Other incentives, including signaling, may also play a role. Alternatively, Baldwin and Clark (2003) argue that open source projects may resemble multistage games without a definite time horizon. This would obviate the problems associated with one-shot Prisoners Dilemma games.
Ch. 5. Open Source Software: The New Intellectual Property Paradigm 299
source members will often be unusually talented and well educated, place unusual emphasis on own use, gift, and play, be patient (have a low discount rate for future benefits) and also young (hence, have long-time horizons). Many observers claim that external incentives can crowd out intrinsic ones, particularly when the new mechanisms are accompanied by monitoring, control, or time pressure (Osterloh et al., 2003b; Osterloh and Rota, 2004). For now, there is little empirical evidence of this. Lakhani and Wolf (2005) report that mixing salary with creativity and political goals does not reduce work effort. Social psychology theorists also predict that intrinsic incentives will decline if there is a widespread perception that third parties are profiting from the community’s efforts. Such perceptions are said to make otherwise willing volunteers feel exploited (Kollock, 1999; Osterloh et al., 2003b). Once again, empirical evidence is limited. Putative cures for crowding out include demonstrating that rules exist and are enforced through social pressure; demonstrated transparency and procedural fairness; encouraging self governance and allowing volunteers to choose their own projects; promoting group identity; providing clearly articulated goals; and making sure that contributions are visible so that members know their contributions are being reciprocated (Kollock, 1999; Kogut and Metiu, 2001; O’Mahoney, 2003; Osterloh et al., 2003b). One problem with these suggestions is that they are more or less identical to the recommendations that one would expect from an analysis of games based on ‘‘own use’’ or ‘‘signaling’’ rewards. It would be interesting to know whether social psychology literature implies any distinctive predictions.16 Survey responses suggest that different social psychology incentives have wildly different strengths. For example, Lakhani and Wolf (2005) and Lakhani et al. (2002) find that intellectual stimulation is the most commonly cited incentive (44.9%) among Sourceforge developers. However, other social psychology incentives appear to be less substantial. These include believing that code should be open (33.1%), feeling an obligation to repay the community (28.6%), and deriving enjoyment from a team enterprise (20.3%). Finally, ideological motives are extremely weak. Only a handful of respondents (11.3%) report that they participate in open source code in order to beat proprietary software and only half of these (5.4%) feel so strongly that they would ‘‘never’’ participate in a closed source project. Ghosh et al. (2002) similarly find that social psychology motivations tend to be reported less often than own use, education, or signaling. They find that the most common social psychology motives include participating in a ‘‘new form of cooperation’’ (37.2%), participating in the open source scene
16 Licenses that block commercialization are said to promote social psychology incentives. See Section 4.
300
S. M. Maurer and S. Scotchmer
(35.5%), believing that software should not be a proprietary good (37.9%), and limiting the power of large software companies (28.9%). The problem with such self-reported responses is that they may convey politically correct opinions more than actual incentives.17 When survey data are combined with information on work effort, intrinsic motives decline sharply in importance. For example, Roberts et al. (2006) find that intrinsic motivations are a statistically insignificant predictor of how much software Apache volunteers contribute. Similarly, Hertel et al. (2003) find that social psychology incentives and identification with the LINUX community are poor predictors of how many hours members will work. On the other hand, Lakhani and Wolf (2005) report that a personal sense of creativity has the largest impact on hours worked per week. They find that this impact is twice as large as enjoyment or receiving a salary and three times larger than reputation incentives.18 Referring again to the cluster analysis of Lakhani and Wolf (2005), the smallest of the four clusters (19%) consists of respondents who are primarily motivated by obligation or community-based intrinsic motivations. Within this group, large majorities report that open source is their most creative experience (61%) and that they lose track of time (i.e., experience ‘‘flow states’’) while programming (73%). Similar majorities either strongly (42%) or somewhat agreed (41%) that the hacker community provided their primary sense of identity.
3
Stability and organizational issues
3.1 Who contributes, and how much? The amount of effort expended by open source volunteers varies widely from individual to individual. Open source projects typically begin with a single programmer making a large investment (Raymond, 1999). Even after additional volunteers join the collaboration, much of the work continues to be done by small minorities. Ghosh et al. (2002) report that 10% of Sourceforge developers create 74% of code. Similarly, Mockus et al. (2002) tell us that 15 core developers provide 83% of all Apache contributions, while Von Krogh et al. (2003) report that about 1% of Freenet’s members 17 The appeal of ‘‘politically correct’’ responses is particularly evident for embedded LINUX. Despite obvious business motivations, Henkel and Tins (2004) report that more than 90% of the developers claim to be motivated by a desire to give code back to the community. This was true whether the developers worked for hardware companies (93%), software companies (91.7%), or non-profits and universities (92.6%). 18 Social psychology motivations seem to have little effect on corporate behavior. A survey of 146 Italian open source companies found that ideological statements had no measurable effect in predicting whether a company would contribute to open source programs. The authors conclude that corporations express such sentiments in order to please developers (Rossi and Bonaccorsi, 2005).
Ch. 5. Open Source Software: The New Intellectual Property Paradigm 301
account for 50% of all developer e-mails.19 This disproportionate reliance on small minorities is particularly evident for new code. Bergquist and Ljungberg (2001) report that almost all new functionality is provided by small groups, while Von Krogh et al. (2003) report that 13% of Freenet developers provide 53% of all new code. In open source tech support, 2% of tech support providers supplied 50% of all answers (Lakhani and Von Hippel, 2000). Small contributors are nevertheless important. This is particularly true for bug reporting.20 In many open source collaborations, these functions are almost entirely performed by volunteers (Bergquist and Ljungberg, 2001). Mockus et al. (2002) estimate that 87% of the Apache members who report bugs submit just one report. Hann et al. (2004) report that many Apache volunteers who report bugs have just one encounter with the project. While less glamorous than creating new code, these activities may be more valuable than creating code in the first place. According to Bessen (2004), testing, debugging, and maintenance account for 82% of software costs.21 Finally, firms face special obstacles in trying to convince open source volunteers to form communities around their products. In particular, firms must persuade volunteers that the code has value as an open source project and is not simply being abandoned because it is technically flawed or losing market share (Lerner and Tirole, 2002b; West and Gallagher, 2004). Perhaps the best way for companies to demonstrate that the code has promise is to make high-profile investments in building an open source collaboration. This can be done by supplying personnel, offering infrastructure like user tool kits, providing awards and other recognition mechanisms for contributors, bundling open source code with company products, and providing coordination functions like recruiting potential contributors, integrating their efforts, building online forums and mailing lists, and developing governance structures (Kogut and Meitu, 2000; West, 2003; West and Gallagher, 2004; West and O’Mahoney, 2005). 3.2 Who pays? The next big question is who pays for open source. Firms participate in open source communities almost as much as individuals do. Roughly half of all open source workers are directly or indirectly supported by 19 The small size of core groups may be enforced by technology. Mockus et al. (2002) argue that cores larger than 12–15 members find it almost impossible to coordinate their actions. Once this point is reached, the number of incompatibilities generated by new software quickly becomes unmanageable. 20 Bug patching represents an intermediate case: the number of people who fix bugs is an order of magnitude larger than those who write new code but an order of magnitude smaller than those who merely report bugs (Kogut and Metiu, 2001; Mockus et al., 2002). 21 Kogut and Metiu (2001) similarly report that maintenance activities account 50–80% of all software costs. See also Raymond (1999).
302
S. M. Maurer and S. Scotchmer
corporations.22 Ghosh et al. (2002) report that 54% of respondents were paid for open source work; Lakhani and Wolf (2005) report that 55% of respondents contributed code during work hours; and Hertel et al. (2003) report that 43% of LINUX kernel developers sometimes or always receive salary for their work. Such data probably understate the importance of corporate support, since other survey data suggest that paid open source contributors work more hours, are older and more educated, and devote more time to communication and coordination activities. In the data of Lakhani and Wolf (2005), paid workers spend roughly twice as much time on open source projects as unpaid ones. Similarly, Roberts et al. (2006) show that salary is an important predictor of effort. Similar arguments can be found in Ghosh et al. (2002) and Kogut and Meitu (2000).
3.3 Why licenses? Most open source communities assume that restrictive licenses like GPL are beneficial or at least unavoidable. However, the need for licenses is not entirely obvious nor, assuming that licenses are needed, is it clear which restrictions are necessary or desirable.23 From a welfare standpoint, the best way to ensure use and re-use of software would be to place it in the public domain without any license at all. This strategy would also be simpler to implement than the elaborate licenses that open source actually uses. This section describes five possible reasons why open source licenses might, after all, be necessary. Intriguingly, most can be satisfied with licenses that are significantly less restrictive than GPL. Symbolism. Dahlander and Magnusson (2005) argue that licenses are sometimes chosen for symbolic reasons. Presumably, the need for symbolism is linked to social psychology incentives that would erode in the presence of private gain, Lakhani and Wolf (2005), or the absence of rules enforcing reciprocity Kollock (1999). By now, there is reason to be skeptical of these explanations. For example, Kogut and Metiu (2001) argued five years ago that the comparatively restrictive Apache license posed a threat to the ‘‘generalized reciprocity that characterizes the community culture.’’ Despite this, Apache has thrived. 22 Approximately, one-third (32%) of the world’s 25 largest software companies engage in significant open source activities. IBM reputedly spent $1 billion on open source projects in 2001 (Ghosh et al., 2002). 23 Like most standards, choice of license exhibits strong network effects. open source collaborations that adopt widespread preexisting licenses face fewer legal impediments to sharing and merging code. They also save the substantial costs associate with writing and learning a new license (Lerner and Tirole, 2002b). The fact that three-fourths (72%) of all SourceForge collaborations use GPL suggests that network effects are substantial. In this view, the dominance of GPL is little more than an historical accident—neither inevitable nor optimal.
Ch. 5. Open Source Software: The New Intellectual Property Paradigm 303
Lerner and Tirole (2002b) present a more nuanced argument based on the hypothesis that social psychology incentives and GPL licenses only become necessary when alternative incentives (e.g., ego gratification, signaling, and own use) are weak. This hypothesis may explain why GPL-style licenses are much less prevalent among projects aimed at developers and system administrators than for games and other applications aimed at end users. Similarly, the symbolic use of licenses may be particularly tempting when open source collaborations with corporate partners place social psychology incentives under threat. Many commentators argue that ideologically motivated licenses and norms pose formidable barriers for most firms (Bonnacorsi and Rossi, 2003; West, 2003; Dahlander, 2004; Dahlander and Magnusson, 2005). That said, the experience of Red Hat and several other companies shows that GPL licenses do not pose insurmountable barriers to business (Osterloh et al., 2002; Weber, 2004). Protecting complementary investments. We stressed above that incentives for participation arise partly through proprietary complements, whether human capital or commercial products. The license may be integral to this, for example, by preventing users from removing an author’s name (Franck and Jungwirth, 2002). Licenses can also shield programmers from potential liability (Gomulkiewicz, 1999; Bonaccorsi and Rossi, 2003). Preventing forking and hijacking. GPL-style blanket prohibitions on commercialization can keep open source collaborations from forking or being diverted in unintended directions (‘‘hijacking’’). However, this problem can also be addressed by alternative and less intrusive measures such as giving a trusted leader the exclusive right to decide which changes and extensions become part of the official code (Bonaccorsi and Rossi, 2003; O’Mahoney, 2003), social pressure (‘‘flaming’’ and ‘‘shunning’’) (Raymond, 1999), trademark (O’Mahoney, 2003), and the use of charisma or astute political skills (Raymond, 1999; Weber, 2000). Furthermore, it is unclear how much danger forking poses in any case. Despite several historical examples (see Lerner and Tirole, 2002a; Varian and Shapiro, 2003), only 11.6% of embedded-LINUX respondents see forking as a threat (Henkel and Tins, 2004). Even if no special steps are taken, Weber (2000) argues that network effects tend to suppress forking in any case.24 LINUX provides an instructive example of how measures short of GPLstyle prohibitions can be used to prevent forking. Although leaders decide which code carries the trademarked LINUX name, users remain free to develop unbranded versions. From the point of view of consumer sovereignty, this may be better than to prohibit forking altogether.
24 Weber argues that forkers necessarily forfeit the benefits of a large community. Thus, they cannot offer followers a built-in audience needed to support signaling incentives or generate bug reports. Community members may also resist forking on the theory that it opens the door to further splintering and balkanization.
304
S. M. Maurer and S. Scotchmer
Ideology. Franck and Jungwirth (2002) argue that licenses reassure activists that their donated labor will not be diverted to commercial ventures that enrich open source leaders.25 More broadly, activists may adopt GPL-style licenses in order to force programmers to choose between writing non-commercial extensions of code and no code at all. Such tactics presumably increase open source software production while reducing the total stock of open plus-proprietary extensions in the market. Stabilizing open source against IP incentives. We argue above that, absent GPL, an opportunistic developer will be tempted to free ride on earlier developers and make his code proprietary so he can impose royalties on later ones. This argument for why GPL may be necessary in the cumulative context is similar to an argument of Hall (2004) and Gambardella and Hall (2005). They point out that a community member who ‘‘cheats’’ by making his code proprietary gets a discrete jump in income, but only reduces the community’s output of publicly available by an infinitesimal amount. This leads to a Prisoner’s Dilemma in which each community member might choose proprietary rights even though they would prefer a collective strategy that preserved open source. The game can be stabilized if actors are required to make decisions within large, organized groups. Norms, deference to lead researchers, and GPL-style licenses all serve this function. The net result is to stabilize open source so that proprietary code is no longer an ‘‘absorbing state.’’ 3.4 Why leadership? Open source communities are often romanticized as collections of atomistic volunteers who self-organize in minimally hierarchical environments. At the same time, open source leaders like Linus Torvalds have cult status. Clearly, self-organization has its limits. In general, open source collaborations rely on leadership to solve a variety of information, agency, and coordination functions. Information problems are particularly important in the early stages of open source collaborations, when would-be contributors must decide whether the proposed project is feasible, interesting, and potentially useful. The easiest way for leaders to demonstrate these qualities is to provide working software, even if it is incomplete and flawed (Lerner and Tirole, 2002a).26 Survey respondents report that the most important leadership functions are providing an initial code base (48.6%), writing code (34.3%), and creating a promise/vision (32.3%) (Lakhani et al., 2002). After an open 25 Franck and Jungwirth (2002) further analogize GPL’s restrictions to the ‘‘non-distribution’’ clauses that prevent traditional charities from diverting surpluses to managers or outsiders. 26 Leadership is especially important where the open source collaboration is built around donated corporate software. Open source volunteers frequently interpret the corporation’s decision to forego intellectual property rights as evidence that the code is worthless (Lerner and Tirole, 2002b).
Ch. 5. Open Source Software: The New Intellectual Property Paradigm 305
source collaboration has been established, information problems change. However, there must still be a procedure to decide which contributions should be included in new releases. Trusted leaders play a key role in this process (Lerner and Tirole, 2002a). Leadership also mitigates agency problems. For example, corporations can commit to keep code in the public domain or to highlight individual contributions by surrendering control to outside leaders (Lerner and Tirole, 2004). Following Max Weber, successful leaders must persuade volunteers that their objectives are congruent and not polluted by ego, commercial incentives, or biases (Weber, 2000). Finally, leadership accomplishes a variety of coordination functions. For example, leaders let volunteers know which projects are worth supporting and, conversely, which constitute forking or abandonment. More generally, someone must make basic architecture choices (e.g., modularity) and coordinate work by volunteers. These functions mitigate the delays and inefficiencies that arise when volunteers interact through decentralized games (Kogut and Meitu, 2000; Lerner and Tirole, 2002a). 3.5 Network effects Proprietary software creates significant network externalities, and it is hard to see why open source software would be any different (Schmidt and Schnitzer, 2002). For example, there is anecdotal evidence that small declines in the popularity of open source projects can snowball into collapse (Lerner and Tirole, 2002a). There are several reasons to believe that open source collaborations should exhibit significant network effects. First, Weber (2000) argues that large open source communities have an intrinsic advantage over small ones, since they are more likely to include ‘‘outliers who have a high level of interest and surplus resources of time and mindspace.’’ These outliers may reflect variance within a single population of contributors or else the presence of distinct subpopulations within a given open source collaboration.27 Second, open source projects need a critical mass of participants. If a competing proprietary system siphons off volunteers and ideas, open source becomes less exciting (Lerner and Tirole, 2002a). Third, some important open source incentives (e.g., ego gratification, signaling) scale with the size of the audience (Lerner and Tirole, 2002a). Finally, bug identification and bug fixing also scale with the number of users. With network externalities, it is natural to expect multiple equilibria and/ or a winner-take-all dynamic (Weber, 2000; Lerner and Tirole, 2002a). Surveys of open source projects are consistent with this intuition. Healy and Schussman (2003) observe, ‘‘It seems clear that for every successful open 27 One frequently overlooked cost factor is competing uses of workers’ time. Hertel et al. (2003) report that ‘‘tolerance for time losses’’ is an important predictor of effort.
306
S. M. Maurer and S. Scotchmer
source project there are thousands of unsuccessful ones.’’28 Indeed, most open source projects are very small scale. Ghosh et al. (2002) find that 29% of projects have just one author, only 45% had more than two, and only 1% had more than 50%.29 Similarly, most open source projects never produce much code.30 Healy and Schussman comment ‘‘Raymond’s image of the bazaar does not capture the fact that the typical project has one developer, no discussion or bug reports, and is not downloaded by anyone.’’ In this environment, agonizing about the choice of license or modular architecture is largely symbolic. Project success may depend on accident as much as quality. An interesting angle explored by Mustonen (2005) is that proprietary software firms can use the open source community to enhance their own networks.
4
Efficiency implications
As pointed out earlier, the open source mechanism does not include a means to appropriate benefits conferred on third parties. We would therefore expect underprovision. Other incentives, not based on appropriability (e.g., signaling and education), not only mitigate this problem but also pose a risk of overprovision. In this section, we discuss some of those issues, and take the opportunity to contrast the inefficiencies of open source against the inefficiencies incurred by the more ordinary use of intellectual property. 4.1 Disclosure of code As we have discussed before, a failing of proprietary software is that proprietors almost never make their source code available to users. Open code enhances welfare in several respects. It is easier to adapt and reuse, and therefore may retain its value rather than becoming obsolete (Raymond, 1999). It facilitates finding, diagnosing, and fixing bugs (Schmidt and Schnitzer, 2002). And it reduces entry costs for firms that supply customization and support services, increasing the likelihood that such services will be competitively supplied (Hawkins, 2002). 28 Provocatively, Madey et al. (2005) find a power law distribution for the number of developers involved in open source projects. The reasons for this distribution are obscure. 29 See also, Comino et al. (2005), reporting that 80% of SourceForge projects have at most two developers. Ninety nine percent of projects have 16 or fewer developers. Healy and Schussman (2003) report that the median number of developers in SourceForge databases is one and that projects in the 95th percentile have only five active developers. Ghosh and Prakash (2000) report that 76% of all projects in their survey have just one author and only 2% have more than five. 30 According to Comino et al. (2005), 80% of SourceForge projects show no activity since registration. Healy and Schussman (2003) find little or no programming activity in more than half of all SourceForge projects.
Ch. 5. Open Source Software: The New Intellectual Property Paradigm 307
But if these advantages are so substantial, why does not the proprietary software industry exploit them for profit? Although formal intellectual property protection does not require disclosure, neither does it prevent it. Given better enforcement (stronger protection), proprietary firms might decide that the protection against copying that comes from keeping the code closed is no longer necessary. From this standpoint, the failings of the proprietary software industry arise from the weakness of intellectual property, not from its strength.31 In fact, proprietary firms already share source code to some extent. Examples include, giving developer toolkits to licensees (Bessen, 2004), sharing source code with selected developers (Microsoft’s ‘‘Shared Source Initiative’’), and promoting code reuse inside Microsoft itself (Lerner and Tirole, 2002a). In yet another twist, Lewis and Talley (2006) point out that developers in the open source community may also withhold their code for strategic reasons, namely, to encourage others to invest time and effort. 4.2 Meeting users’ needs We distinguish here between the incentive to meet the needs of users, and the ability to do so. Obviously, own-use incentives are directed to the user’s needs, even if not to the needs of third parties. However, a programmer’s incentive to signal his proficiency, or to become a better programmer, or to participate in a community of altruistic providers, may not be. Since those incentives are not based on appropriating value from users, there is no need for the innovative activity to track user needs particularly closely. Lakhani and Wolf (2005) tell us that about 58% of all volunteers are IT professionals. Despite their proficiency in writing code, there is no obvious reason they would do mundane tasks useful to third parties like testing for usability (Lakhani and Wolf, 2005), learning unfamiliar programming languages and architectures (Kogut and Meitu, 2000, Von Krogh et al., 2003), deciphering complex commercial code (West and O’Mahoney, 2005), or meeting the needs of highly specialized audiences such as lawyers or accountants (Schmidt and Schnitzer, 2002). Skilled programmers get most benefit from creating tools for other IT professionals. See West (2003), discussing Internet service providers; Johnson (2002) arguing that open source produces more utilities than end-user applications; and Comino et al. (2005) reporting that two-thirds of all SourceForge projects involve software languages, systems, internet code, communications and multimedia tools, or scientific software. However, there is a countervailing argument, namely, that open source communities have closer contact with their users than owners of proprietary software, and therefore have a better ability to meet their needs (von Hippel, 31 This can be overstated. If intellectual property over-rewards proprietary software, then it creates deadweight loss that would better be avoided.
308
S. M. Maurer and S. Scotchmer
2002; Henkel and von Hippel, 2005). Mockus et al. (2002) point out that programmers in large proprietary software projects frequently do not know the domain for which they are writing. User feedback is particularly valuable where consumer needs cannot be reduced to a few simple criteria of merit (Kogut and Metiu, 2000; von Hippel, 2005). Such information is even more valuable if workers are ‘‘lead users’’ who understand needs, risks, and eventual market size before manufacturers do. In this view, open source allows user-developers to play a larger role in developing and extending the products they use (Kogut and Metiu, 2000; Varian and Shapiro, 2003).32 4.3 Deadweight loss and pricing Proprietary prices are generally above competitive levels. The proprietary price reduces consumption of the patented good and causes users to shift their use to less-preferred substitutes. Proprietary pricing can also lead to costly R&D to invent around patents, and may reduce the incentive for second-generation innovators either directly through royalties or indirectly by forcing them to invent around (Henkel and von Hippel, 2005). Open source avoids these problems by dispensing with intellectual property. Of course, that leaves the puzzle of where the developers get their rewards, which has been the subject of most of this essay. 4.4 Training and using programmers Proprietary software firms find it hard to observe the competence of a programmer, and therefore cannot tailor wages to programmers’ marginal productivities (Gaudeul, 2004; Johnson, 2004).33 We, therefore, expect inferior programmers to self-select into private sector jobs that offer a high average wage. Once hired, low-quality programmers also have various incentives to hide errors. If they report bugs, they will likely be asked to fix them34 (Johnson, 2004). If programmers collude in not reporting each other’s failings, firms will find it hard to know whether they have good code (few bugs). This undercuts the value of peer review (Johnson, 2004). Proprietary firms may be forced into suboptimal architectures to control these agency problems. For example, they may substitute top-down supervision for a more stimulating environment in which programmers can 32 Proprietary firms try to replicate these advantages by using in-house developer networks to appraise and debug software (Kogut and Metiu, 2001). Microsoft’s Shared Source Initiative similarly lets select customers view, customize, and patch an open version of Windows (Lerner and Tirole, 2004). 33 Gaudeul (2004) also suggests alternative reasons why wages could be higher than marginal product. These include legal costs associated with enforcing intellectual property, organizational costs associated with setting up and managing a firm, and compensation for restricting the developer’s choice of projects. 34 This result requires the admittedly ‘‘strong assumption’’ that contracts are written in a way that pays programmers a flat wage no matter how many tasks are discovered and assigned ex post (Johnson, 2004).
Ch. 5. Open Source Software: The New Intellectual Property Paradigm 309
devise their own projects and work in parallel on many projects, recognizing that many should fail (Kogut and Metiu, 2000). A management-intensive way of organizing the development effort reduces worker satisfaction as well as worker effectiveness, and firms must offer a wage premium to overcome it. By contrast, open source programmers can join whichever projects fit their personal interests (Mockus et al., 2002; Rossi et al., 2004). Most open source incentives tie rewards to the actor’s own efforts, and therefore avoid the management problems we have identified for proprietary firms.35 Other things equal, we expect talented programmers to self-select away from the private sector and into open source.36 Mockus et al. (2002) and Rossi et al. (2004) take this argument one step further by arguing that talented programmers also self-select into whichever open source project promises to make the best use of their skills. Perhaps the clearest articulation is due to Benkler (2002), who argues that open source selects ‘‘the best person to produce a specific component of a project, all abilities and availabilities to work on the specific module within a specific time frame considered’’ (emphasis original). Such claims are almost certainly overstated. While there are surely selection effects, there is no obvious way to aggregate information in order to match programmers efficiently with projects. The same difficulty arises for patents and prizes (Scotchmer 2004, Chapter 2). We can ask the same question for labor practices as we asked for disclosure of source code: If open source practices are best, why do not proprietary firms emulate them? The answer, at least to some extent, is that they do. First, some proprietary firms deliberately foster reputation incentives by attaching programmers’ names to code. Compared to open source, however, the results are ambiguous. On the one hand, giving credit attracts and elicits more effort from talented workers. On the other, the strategy increases the risk that star employees will be hired away by competitors (von Hippel, 2002; Lerner and Tirole, 2002a). Second, many firms try to create work environments that respect motives like reciprocity, altruism, and being ‘‘part of a team’’ (Lerner and Tirole, 2002a). Finally, some firms have experimented with decentralization. Microsoft’s efforts to have employees challenge each other’s ideas are one step in this direction (Kogut and Meitu, 2000). So far, we have treated programmers’ self-selection decisions as immutable. This is reasonable for incentives based on complementary goods and services, since programmers who join open source communities in order to build their own businesses will presumably stay there. On the other hand, reputation and signaling incentives are meaningless unless programmers 35 This is clearly not true to the extent that corporations pay employees to work on a particular project. Even here, however, open source may have significant advantages to the extent that paid workers also have non-cash incentives and/or must interact with members who have such incentives. 36 We note that this effect is not unique to open source. Proprietary firms frequently reduce agency problems by adopting incentives—notably prizes and patents—that tie worker rewards to measurable results.
310
S. M. Maurer and S. Scotchmer
eventually return to the private sector. In practice, reviewing code is expensive, and firms prefer to use open source hierarchies and ranking systems as a proxy for quality (Lerner and Tirole, 2002). Weber (2000) argues that open source rankings are inherently trustworthy because programmers know that the best way to build reputation is to work with high-quality collaborators and exclude everyone else. 4.5 Free riding If ideas are not scarce, that is, if any good idea is likely to be had and implemented by someone else—it is tempting to let someone else bear the development cost. In this environment, open source programmers who are otherwise willing to write code may wait in hopes that someone else will do the job first (Raymond, 1999). Users of code who do not contribute are free riders. The ability to free ride can reduce welfare. Scholars have used game theory to explore these results. In general, they find equilibria with pure strategies in which some developers invest and others free ride, and other equilibria with mixed strategies in which each developer works with some probability and development sometimes fails.37 Communities that play mixed strategies will deliver code more slowly or less reliably than proprietary software companies do. (Johnson, 2002; Baldwin and Clark, 2003; Bitzer et al., 2004). Patent incentives have the opposite problem. Much of the literature on patent incentives is concerned with racing, given that only the winner will have rights. Depending on the commercial value of the patent, racing may lead to too much investment. (See Scotchmer, 2004, Chapter 4, for an overview.) 4.6 Modularity and the organization of the research effort The degree to which software is modularized affects its suitability for the open source style of development. This suggests that leaders can make open source collaborations stronger by designing more modular architectures. For example, Benkler (2002) claims that modularity (‘‘granularity’’) is the key to making open source projects viable. His conjecture is supported by survey data suggesting that own-use programmers spend relatively few hours on open source, and tend to lose interest as soon as their own narrow needs are addressed (Roberts et al., 2006). Johnson (2002) makes a more systematic investigation of the role of modularity in open source. He considers a game in which each developer 37 Open source members clearly understand this logic. Hertel et al. (2003) find that LINUX kernel members work harder when they believe that their contributions are ‘‘highly important’’ to further progress.
Ch. 5. Open Source Software: The New Intellectual Property Paradigm 311
can choose to create one module based on his own individual ratio of personal benefit to cost. In this model, each developer invests if the added probability of success provided by his own investment, weighted by his personal valuation of the module, outweighs his own personal cost. The probability that at least one member succeeds depends on the number of members who decide to invest. Johnson finds that adding a programmer to the pool, with randomly drawn cost and benefit for each module, typically increases the likelihood that the code will be supplied. Assuming that the ratio of personal benefit to cost is bounded, he finds that the total number of volunteers who write code in any particular period approaches a limit as their numbers increase. Conversely, free riding increases. Johnson shows that the total investment is smaller (produces less software) than the effort needed to maximize social value in expectation. Intuitively, this result follows from the fact that each programmer’s decision to contribute depends only on her own personal benefits, and does not reflect spillovers to other users. Johnson also investigates how variance in (personal) benefit/cost ratios affect the optimality of open source compared to single-source development. Suppose that each of k modules is necessary for completion of the whole. If a single programmer must create the entire project, the code will not be written unless the ratio of total benefit to total cost over all k modules is greater than one for at least one developer. Apply the same test on a module-by-module basis. In this case, the maximum benefit/cost ratio could be greater than one for each module separately even though the maximum ratio of the totals was less than one and vice versa. For this reason, a modularized project can succeed where a non-modularized project fails. Nevertheless, Johnson argues that the success rate of modular projects grows with the number of developers because programmers can self-select into working on the modules for which they are most proficient. The main drawback is that the project may founder if costs are badly distributed among modules. Baldwin and Clark (2003) explore how modularization fares when different programmers can communicate to avoid redundant work. Since workers share costs, a collective effort with adequate communication is always preferable to coding in isolation. Free riding may increase, however, if communications are so slow that programmers cannot monitor which modules have been already written or if the number of developers exceeds the number of modules.38 Even in this environment, increasing the number of modules reduces free riding. Similarly, systems with large potential payoffs will normally attract more workers per module provided that there is enough randomness in outcomes and if costs are small enough.39 Developers may intentionally duplicate each other’s efforts in order to obtain a higher best outcome. 38 This may be a realistic concern for very large projects like LINUX or Apache. The problem is mitigated if most members are paid to participate by small group of employers. 39 See also, Johnson (2002).
312
S. M. Maurer and S. Scotchmer
The degree to which programmers can coordinate their efforts, either to avoid duplication or to reinforce efforts to achieve particularly valuable modules, depends on how isolated they are. Ghosh et al. (2002) report that 17.4% of open source developers have no regular contact with the rest of the community and 67.9% had regular contact with less than five other members. Only 17.3% had regular contact with more than 10 other members. In principle, better and more frequent communication could improve the performance. Corporate support, where it exists, may help fill this gap by supporting the ‘‘professional elite’’ who maintain regular contact with 50 or more developers(Ghosh et al., 2002).
5
Open source and proprietary software
Open source limits the market power of proprietary code by providing competition and the threat of entry (Bessen, 2004; Henkel and von Hippel, 2005). There are two possible scenarios in which this can happen: first, open source and proprietary code can compete head-to-head within a single market. Second, they can occupy separate niche markets. Commentators have explored both scenarios. 5.1 Competition between open source and proprietary software One explanation for why proprietary software can survive in competition with open source products is that users prefer it, either because it is better code or because it is user friendly. Another reason might be that proprietary software becomes entrenched by network effects (Casadesus-Masanell and Ghemawat, 2006). We have already uncovered several reasons why proprietary firms might have a quality advantage. First, open source incentives probably capture less social value than the patent monopoly does, so that certain projects will not be developed in that community (Schmidt and Schnitzer, 2002). Second, if open source relies on signaling incentives, and a programmer receives no benefit from the unglamorous task of making the code user friendly, no one will bother. Third, we have seen that open source contributors may delay writing or improving code in hopes that others will step in instead. Since corporations suffer the opposite coordination problem—they want to be first to market with a good product—we expect them to act faster whenever it is profitable to write good code. There is some evidence that commercial code has a quality edge over open source, at least for some users. Consulting studies report that the up-front costs of purchasing Windows are approximately offset by lower system and administrator costs compared to LINUX. For now, it is unclear whether the cancellation is exact. Varian and Shapiro (2003) suggest that the
Ch. 5. Open Source Software: The New Intellectual Property Paradigm 313
life-cycle costs of owning LINUX may be 10–15% cheaper than Windows, but remark that this difference is not ‘‘striking.’’ We remark, though, that such studies face serious methodological challenges including (a) difficulties in comparing non-comparable software, (b) poor metrics for reliability and quality, (c) sensitivity to IT wages, and (d) limited information about how much additional IT effort is needed to support LINUX compared to Windows (Robert Frances Group, 2002; Cybersource, 2004; Metagroup, 2005). Mustonen (2003) uses a quality differentiation model to show that lowquality open source can sometimes coexist with high-quality proprietary products. The model assumes that both the programmer and user markets are segmented. Mustonen finds that high-quality programmers self-select into the open source community where they are better rewarded for their skills even though the proprietary sector continues to produce superior products by hiring large numbers of low-quality programmers. He also assumes that users with high demand are willing to pay a high price for superior software. Provided that consumers can install software at low cost, Mustonen finds an equilibrium in which consumers who place a low valuation on software always choose open source over proprietary code and the proprietors set prices based on demand from high-valuation consumers. If installation costs are high, Mustonen finds that proprietary software dominates for large markets that value high-quality software. Conversely, proprietary firms tend to abandon markets that are small or which place a low value on quality.40 Kuan (2001) presents a model where proprietary companies offer different versions of their software to high- and low-quality consumers. Absent perfect price discrimination, she finds that low-value consumers always switch to open source. However, the willingness of high-value consumers and consumer/programmers to switch to open source is less clear, depending on how many such individuals exist. In general, Kuan finds that the analysis is ‘‘very undetermined.’’ Casadesus-Masanell and Ghemawat (2006) construct a dynamic model of entrenchment through network effects in which a proprietary product and open source product coexist. Starting from a situation where the proprietary product has a network advantage over the (intrinsically similar) open source product, they ask whether the open source product can supplant it. Although the price of the proprietary product will be higher in each period, the proprietor will choose the path of prices so as to maintain a market advantage through the network advantage. Unless network effects are weak, proprietors never lower prices enough to drive out the 40 Bessen (2004) similarly predicts that open source may crowd out proprietary software in cases where markets are small or only modest effort is needed to develop an initial product. Where markets grow rapidly, open source may use this advantage in small markets to gain a first-mover advantage over proprietary competitors.
314
S. M. Maurer and S. Scotchmer
open source product entirely. Open source and proprietary products therefore coexist forever.
5.2 Market segmentation Even if open source and proprietary code do not compete directly, they may serve different niche markets. Bessen (2004) notes that packaged software like Microsoft Windows accounts for only about 30% of all software spending. The remainder is customized software created by and for individual users, often in open source communities. However, creating customized software products is expensive. Bessen argues that, because it is difficult to write an enforceable contract for customized software in advance, proprietary firms must negotiate a price after the software package is created. But with a single customer, the customer will have a lot of bargaining power (Aghion and Tirole, 1994). Anticipating this, the proprietary firm will be loathe to invest and would-be customers will be forced to assemble the software from open source modules. Bessen argues that users who place a low value on quality or have highly unique needs will choose open source. Users whose needs can be satisfied with simpler applications will continue to use proprietary products. Gaudeul (2004) explores how market segmentation can arise from developers’ profit-maximizing choices about how best to exploit their code. The developer’s first choice is to copyright the software and hire developers to implement it, collecting proprietary profit. This option will not be attractive, though, if wages are too high. The innovator’s second choice is to obtain development efforts at zero cost by choosing a GPL license. However, GPL strategies are only feasible if the project has sufficient value for the open source community to participate. The innovator’s only other choice is to purchase developed software on the open market. This can be done by offering developers a BSD license. Gaudeul argues that GPL is usually worse from a welfare point of view; since, software is developed with lower probability than it would be in a proprietary regime where corporations do all work immediately or a BSD regime in which programmers race for proprietary rights. However, these differences will likely be minor where development costs are small; furthermore, there may be cases in which rising costs reduce the value of proprietary software faster than they undermine GPL incentives. If so, GPL may sometimes generate more social welfare than copyright incentives ‘‘for medium level[s] of c[ost].’’41 41 Gaudeul recognizes that his two-stage model is relatively simple. He adds that more complex models would include wasteful duplication due to racing; GPL developers’ ability to earn income by selling complements; and the possible advantages of GPL in multi-stage games where developers continue to add new ideas over time.
Ch. 5. Open Source Software: The New Intellectual Property Paradigm 315
6
Limitations and extensions
6.1 Limits to open source software Open source incentives do not perform all tasks equally well. For this reason, otherwise useful projects can be limited by open source’s weakness in supplying specific components such as documentation or tech support. Henkel and Tins (2004) report that lack of documentation is the most commonly reported obstacle (57.5%) to embedded LINUX.42 Gambardella and Hall (2005) similarly argue that open source incentives are poorly suited to downstream tasks like assembling knowledge into unified, userfriendly formats. GPL licenses may be counterproductive in this situation if they displace intellectual property rights needed to extract full value from society’s investment in knowledge. Many commentators assume that open source networks are larger than corresponding corporations and, for that reason, draw on more widely scattered ideas and expertise (Kogut and Metiu, 2001; Benkler, 2002; Gambardella and Hall, 2005).43 Wide dissemination is said to be particularly important in fast changing or complex products (Dahlender, 2004). As we have seen, however, most open source collaborations are far smaller than corporations. Arguments based on the superior ability of open source to elicit knowledge should therefore be viewed with caution except, perhaps, in the case of Open Science. Finally, we have seen that open source works best for modularized software. However, it is not clear whether the number of modules is a free parameter. Lerner and Tirole (2002a) speculate that the ability to break projects into modules may be a technical accident peculiar to UNIX that will likely fade as open source moves to other languages. Modularity may also be limited by the ability of a small central group to screen submissions for quality and consistency. 6.2 Beyond software: drug discovery, geographic information systems, and Wikipedia Unlike previous eras, 20th century R&D is specialized in delivering complex systems that required more man-years of effort than a human life span could supply. Examples included airplanes, automobiles, rockets and spacecraft, automobiles, pharmaceuticals, silicon chips, and most recently large computer programs. Prior to open source, all such projects relied on 42 Open source sometimes works surprisingly well despite seemingly implausible incentives. For example, Lakhani and Von Hippel (2000) report that open source volunteers sometimes provide tech support in order to gain information about bugs. Weber (2000) argues that open source collaborations can accomplish even mundane tasks if a handful of members have unusual preferences. 43 This is related to our argument above that disclosure shakes loose the ‘‘scarce ideas’’ that might otherwise remain untapped.
316
S. M. Maurer and S. Scotchmer
organizing large numbers of contract researchers within rigid, top-down hierarchies. From Moscow to Los Angeles, the scene was strikingly similar: hundreds and even thousands of engineers sitting at drafting tables or, more recently, computer terminals. One of the most striking aspects of LINUX is that it is possible to organize at least one form of complex invention— large operating systems—a different way. It is, therefore, tempting to ask whether the open source model can be extended to other information goods. Because of their high information content, the most natural technologies are online reference works (e.g., Wikipedia) drug discovery and geographic information systems (GIS). The Internet hosts innumerable blogs and wikis where volunteers collect and process information for a wider audience. There are also more structured sites that invite members to perform defined tasks like counting Martian craters, ranking the most interesting news stories, and even writing encyclopedia entries (Benkler 2002; Von Hippel, 2005). In many ways, these projects resemble and extend an earlier tradition of large nuclear and particle physics databases that have relied on worldwide networks of volunteer editors since the 1940s (Maurer, 2003). Like those earlier ‘‘big science’’ projects, the quality of at least some online resources seems to be high. Recent peer review tests of the Wikipedia online encyclopedia suggest that its accuracy is comparable to its most prestigious IP-supported counterpart, Encyclopedia Britannica (Giles, 2005). In some ways, blogs and wikis may actually be a more favorable environment than software for open source since each article can be written and enjoyed as a freestanding ‘‘module.’’ The ability of authors to derive value is largely independent of how well their article interacts with other modules or, indeed, whether other articles are written at all. This is very different from open source, in which volunteers cannot derive value unless they are eventually able to combine their parts to create a single, unified product. Drug discovery and GIS are much closer to the software model. Weber (2000, 2004) and Burk (2002) suggest that deciphering and annotating the genome might be organized as an open source project. More recent observers have usually been skeptical. Lerner and Tirole (2004) suggest that many biotechnology tasks cannot be broken up into modules and, in any case, are expensive. Furthermore, there may not be enough ‘‘sophisticated users who can customize the molecules to their own needs’’ (Lerner and Tirole, 2004). Similarly, Kogut and Metiu (2001) assert ‘‘A molecule y is not modular’’ because ‘‘changing atoms drastically alters its pharmaceutical properties.’’44 Maurer (2006) argues that these objections assume that drug discovery is an indivisible black box. In fact, there are approximately one dozen distinct 44 The existence of ‘‘me-too’’ drugs, in which drug companies change existing patented compounds just enough to avoid patent infringement suggests that molecules may be more ‘‘modular’’ than Kogut and Metiu suspect.
Ch. 5. Open Source Software: The New Intellectual Property Paradigm 317
subtasks between basic science and the delivery of a completed drug. Proprietary companies routinely break the black box apart by offering different incentives for each specific substep. Examples include buying and selling drug ideas (‘‘external innovation’’), offering prizes for recalcitrant chemical engineering problems, and outsourcing preclinical and clinical trials to outside entities. There is no obvious reason why open source collaborations could not similarly perform one or more substeps. Scholars have suggested open source collaborations for a wide variety of tasks including basic science, using database (bioinformatics) tools to find the gene sequence ‘‘targets’’ that code for disease, computational design of drugs for specific targets, in vitro chemistry and biology experiments designed to validate proposed targets and drugs, and clinical trials (Benkler, 2002; Maurer et al., 2004; Maurer, 2006; Von Hippel, personal communication). While convincing examples of open source biology do not yet exist,45 it is reasonable to think that incentives based on education, signaling, and ideology should appeal to biologists just as much as they do for computer scientists. The deeper question is whether open source biology can coexist with the powerful incentives offered by conventional patents (Cohen, 2005). One solution is to create open source biology collaborations in fields where patent incentives are weak, for example, tropical disease research (Maurer et al., 2004). More speculatively, much of the risk and expense associated with clinical-phase trials involves documentation costs that pharmaceutical companies incur to convince a skeptical FDA that their data is unbiased. This problem is aggravated by outsourcing, which gives contract researchers obvious incentives to suppress and even falsify data to keep test programs alive. As noted by Titmuss (1972), non-market solutions avoid these problems by relying on volunteers who have nothing to gain by lying. Since most open source incentives similarly suppress agency problems, conventional pharmaceutical companies might decide that funding open source clinical trials was a more cost-effective way to convince FDA that their drugs were effective than conventional contract research. In this case, patent incentives would reinforce open source biology instead of competing with it. Finally, GIS present a second industry where open source methods may be acquiring a foothold. Like biology, the technology is highly computerized and depends on users to notice and correct errors. Some government consortia already style themselves ‘‘open source GIS’’ and invite users to submit reports when mapping data turns out to be inaccurate (National 45 Commentators sometimes argue that bioinformatics software projects and/or science collaborations that adopt open source-like licensing terms deserve the label ‘‘Open Source Biology’’ (Boettinger and Burk, 2004 describing HapMap license; Rai, 2005). The former are indistinguishable from other types of software, while the latter invariably turn out to be grant-supported collaborations that have adopted open source-like licenses. Whether such collaborations should be called ‘‘open source’’ is, of course, a matter of semantics. Suffice to say, they do not seem fundamentally different from traditional big science projects dating back to the 1930s.
318
S. M. Maurer and S. Scotchmer
Research Council, 2004). In theory, proprietary firms could similarly mencourage users to report errors. Open source GIS may, however, have an inherent advantage in fostering social psychology incentives (e.g., altruism) among users. How much further can the open source model be extended? Our discussion suggests several factors. First, where contributors make complementary contributions for their own use, open source projects are more likely to go forward when each contributor’s individual cost is small. This suggests that projects with large, costly, and indivisible modules are disfavored. Examples include projects where volunteers would have to build physical prototypes or conduct physical experiments. Second, we hypothesize that most workers in most industries find incentives like ideology, education, and signaling much weaker than IP rewards. If contributors turn to traditional IP instead of open source, open source collaborations may never acquire the critical mass of workers they need to get started. Instead, we expect open source projects to arise in markets where traditional IP incentives are weak.46 Weaknesses of traditional IP incentives may occur because users are poor (e.g., users of drugs for neglected diseases), because extreme ease of copying undercuts the value of formal IP rights (e.g., software, data), or because licensing is impractical due to market imperfections like high transactions costs or the presence of ‘‘experience goods.’’ Finally, we have seen that public domain sharing of innovation makes sense for cumulative innovation environments in which ‘‘ideas are scarce,’’ so that the owners and would-be improvers of IP cannot readily find each other. In such systems, GPL-style open source stabilizes the public domain by preventing improvers from taking ideas private. The phenomenon will normally be more important for high risk, cutting-edge research (e.g., earlystage drug discovery) that puts a high value on clever new ideas compared to well-understood fields where successful development is more or less assured once funds are invested (e.g., automobiles).
7
Conclusion
Open source comprises not one but an entire suite of incentives. In general, each has separate and distinct welfare implications. Furthermore, the importance of, say, ‘‘signaling’’ or ‘‘own-use’’ incentives vary significantly across and even within projects. While generalizations are difficult, most open source incentives reduce agency problems and deadweight loss compared to patents, and accelerate discovery through automatic disclosure. Against these virtues, open source incentives often lead to an undersupply 46 In principle, policymakers can encourage open source by reducing IP benefits. They should not, however, abolish IP entirely. Open source incentives can certainly supplement intellectual property; because they tend to undersupply goods, however, they should not replace it.
Ch. 5. Open Source Software: The New Intellectual Property Paradigm 319
of goods relative to the patent system. Open source may also be less responsive to certain users, especially when those users are non-programmers. Because of undersupply, open source can only be a partial solution: it is not viable, and cannot operate in every environment where patent incentives do. Where it works, however, it is often superior. Acknowledgments We thank the Toulouse Network on Information Technology for financial support, and Terry Hendershott for thoughtful comments. References Aghion, P., J. Tirole (1994). The management of innovation. Quarterly Journal of Economics 109, 1185. Baldwin, C., K. Clark (2003). Does code architecture mitigate free riding in the open source development model? Mimeo, Harvard University Business School. Benkler, Y. (2002). Coase’s Penguin, or Linux and the nature of the firm. Yale Law Journal 112, 369. Bergquist, M., J. Ljungberg (2001). The power of gifts: organizing social relationships in open source communities. Information Systems Journal 11, 305. Bessen, J. (2004). Open source software: free provision of complex public goods. Mimeo, Boston University Law School. Bettelheim, B. (1976). The uses of enchantment: the meaning and importance of fairy tales. Knopf, New York. Bitzer, J., W. Schrettl, P. Schroder (2004). Intrinsic motivation in software development. Free University of Berlin Discussion Paper 2004/19. Boettinger, S., D. Burk (2004). Open source patenting. Journal of International Biotechnology Law 1, 221. Bonnacorsi, A., C. Rossi (2003). Licensing schemes in the production and distribution of open source software: an empirical investigation. Mimeo, Sant’ Ana School for Advanced Studies Institute for Informatics and Telematics. Bonnacorsi, A., C. Rossi. (2003). Contributing to the common pool of resources in open source software: A comparison between individuals and firms. Mimeo. Sant’ Ana School of Advanced Studies and Institute for Informatics and Telematics, Pisa. Burk, D. (2002). Open source genomics. Boston University Journal of Science and Technology Law 8, 254. Casadesus-Masanell, R., P. Ghemawat (2006). Dynamic mixed duopoly: a model motivated by Linux vs. Windows. Management Science. Cohen, W. (2005). Does open source have legs?, in: R. Hahn (ed.), Intellectual Property Rights in Frontier Industries, AEI-Brookings Press, Washington, DC. Comino, S., F. Manenti, M. Parisi (2005). From planning to mature: on the determinants of open source take off. Mimeo, University of Trento, Department of Economics. Cybersource (2004). Linux vs. Windows: total cost of ownership comparison (self-published). Dahlander, L. (2004). Appropriating returns from open innovation processes: a multiple case study of small firms in open source software. Mimeo, Chalmers University of Technology Deptartment of Industrial Dynamics. Dahlander, L., M.G. Magnusson (2005). Relationships between open source software companies and communities: observations from Nordic firms. Research Policy 34, 481. Dalle, J.-M., P.M. David (2003). The allocation of software development resources in ‘‘open source’’ production mode. SIEPR Discussion Paper No. 02-27.
320
S. M. Maurer and S. Scotchmer
Franck, E., C. Jungwirth (2002). Reconciling investors and donors—the investment structure of open source. University of Zurich Lehrschule fur Unternehmensfuhrung und, politik. Gambardella, A., B. Hall (2005). Proprietary vs. public domain licensing of software and research products. NBER Working Paper 11120. Gandal, N., C. Ferschtman (2005). Open source projects: output per contributor and restrictive licensing. Mimeo, University of Tel Aviv. Gaudeul, A. (2004). Open source software development patterns and license terms. Mimeo, University of Toulouse. Ghosh, R., R. Glott, B. Kriger, G. Robles (2002). Free/Libre and open source software: survey and study. Mimeo, University of Maastricht Institute of Infonomics and Berlecon Research GmbH. Ghosh, R., Prakash, V. (2000). The Orbiten free software survey. First Monday Issue 5 (Number 7). Giles, J. (2005). Internet encyclopedias go head to head. Nature 438, 900. Gomulkiewicz, R.W. (1999). How copyleft uses license rights to succeed in the open source software revolution and the implications for Article 2B. Houston Law Review 36, 179. Hall, B. (2004). Incentives for knowledge production with many producers. Cambridge University ESRC Centre for Business Research Working Paper 292. Hann, I.-H., J. Roberts, S. Slaughter, R. Fielding (2004). An empirical analysis of economic returns to open source participation. Mimeo, Carnegie Mellon University. Harhoff, D., J. Henkel, E. von Hippel (2003). Profiting from voluntary information spillovers: how users benefit by freely revealing their innovations. Research Policy 32, 1753. Hawkins, R.E. (2002). The economics of open source for a competitive firm: Why give it away? Netnomrics 6, 103. Henkel, J. (2005a). The jukebox mode of innovation—a model of commercial open source development. Mimeo, Technische Universitat, Munich. Henkel, J. (2005b). Selective revealing in open innovation processes: the case of embedded Linux. Mimeo, Technische Universitat, Munich. Henkel, J., E. von Hippel (2005). Welfare implications of user innovation. Journal of Technology Transfer 30, 73. Henkel, J., M. Tins (2004). Munich/MIT survey: development of embedded Linux. Mimeo, University of Munich Institute for Innovation Research, Technology Management, and Entrepreneurship. Hertel, G., S. Niedner, S. Herrmann (2003). Motivation of software developers in open source projects: an internet-based survey of contributions to the Linux Kernel. Research Policy 32, 1159. Johnson, J.P. (2002). Open source software: private provision of a public good. Journal of Economics and Management Strategy 24, 637. Johnson, J.P. (2004). Collaboration, peer review and open source software. Mimeo, Cornell University Johnson School of Management. Kogut, B., Metiu, A. (2000). The emergence of e-innovation: insights from open source software development. University of Pennsylvania Wharton School Reginald H. Jones Center Working Paper WP 00-11. Kogut, B., A. Metiu (2001). Open-source software development and distributed innovation. Oxford Review of Economic Policy 17, 248. Kollock, P. (1999). The economics of online cooperation: gifts and public goods in cyberspace, in: M. Smith, P. Kollock (eds.), Communities in Cyberspace, Routledge, London. Kuan, J. (2001). Open source software as consumer integration into production. University of California Haas School of Business, Mimeo. Lakhani, K., E. von Hippel (2000). How open source software works: ‘‘free’’ user-to-user assistance. MIT Sloan School of Management Working Paper No. 4117. Lakhani, K., R. Wolf (2005). Why hackers do what they do: understanding motivation and effort in free/open source software projects, in: J. Feller, B. Fitzgerald, S. Hissam, K. Lakhani (eds.), Perspectives in Free and Open Source Software, MIT, Cambridge. Lakhani, K., R. Wolf, J. Bates, and C. DiBona (2002). The Boston Consulting Group Hacker Survey Release 0.73 (self-published).
Ch. 5. Open Source Software: The New Intellectual Property Paradigm 321 Lemley, M., P. Menell, R. Merges, P. Samuelson (2002). Software and internet law 2nd edition. Aspen Law and Business, New York. Lerner, J., J. Tirole (2002a). Some simple economics of open source. Journal of Industrial Economics 52, 197. Lerner, J., J. Tirole (2002b). The scope of open source licensing. Journal of Law, Economics, and Organization 21, 20. Lerner, J., Tirole, J. (2004). The economics of technology sharing: open source and beyond. NBER Working Paper 10956. Lewis, T.R., E. Talley (2006). Open source innovation. Duke University, Mimeo. Madey, G., V. Freeh, R. Tynan (2005). Understanding open source as a self-organizing process. Mimeo, University of Notre Dame Department of Computer Science and Engineering. Maurer, S. (2003). New institutions for doing science: from databases to open source biology. Mimeo, European Policy for Intellectual Property Conference. Maurer, S. (2006). Choosing the right incentive strategy for R&D in neglected diseases. World Health Organization Bulletin 84, 376. Maurer, S., A. Rai, A. Sali (2004). Finding cures for tropical disease: is open source an answer?. Public Library of Science: Medicine 1, 56. Menell, P., S. Scotchmer (forthcoming). Intellectual property, in: M. Polinsky and S. Shavell (eds.), Handbook of Law and Economics. Elsevier, Amsterdam. Metagroup (2005). File, web, and database server administration (self-published). Mockus, A., R.T. Fielding, J.D. Herbsleb (2002). Two case studies of open source software development: Apache and Mozilla. ACM Transactions on Software Engineering and Methodology 11, 309. Mustonen, M. (2003). Copyleft—the economics of Linux and other open source software. Information Economics and Policy 15, 99. Mustonen, M. (2005). When does a firm support substitute open source programming? Journal of Economics and Management Strategy 14, 121. National Research Council (2004). Licensing Geographic Data and Services. National Academy Press, Washington, DC. Netcraft (2005). Web server survey turns ten (self-published). O’Mahoney, S. (2003). Guarding the commons: how community managed software projects protect their work. Research Policy 32, 1179. Osterloh, M. (2002). Open source software: new rules for the market economy. Mimeo, University of Zurich Institute for Research in Business Administration. Osterloh, M., S. Rota (2004). Trust and community in open source software production. Analyse und Kritik: Zeitschift fur Sozialtheorie 26, 279. Osterloh, M., S. Rota, B. Kuster (2003a). Trust and commerce in open source—a contradiction?, in: O. Petrovic, M. Fallenbock, Ch. Kittle, M. Ksela (eds.), Trust in the Network Economy, Springer, Vienna, p. 129. Osterloh, M., S. Rota, B. Kuster (2003b). Open source software production: climbing on the shoulders of giants. Mimeo, University of Zurich Institute for Research in Business Administration. Rai, A. (2005). Open and collaborative research: a new model for biomedicine, in: R. Hahn (ed.), Intellectual Property Rights in Frontier Industries, AEI-Brookings Press, Washington, DC. Raymond, E.S. (1999). The Cathedral and the Bazaar: Musings on Linux and Open Source by an Accidental Revolutionary. O’Reilly, Sebastopol, CA. Robert Frances Group (2002). Total cost of ownership for Linux in the enterprise (self-published). Roberts, J., I.-H. Hann, S. Slaughter (2006). Understanding the motivations, participation and performance of open source software developers: a longitudinal study of the Apache projects. Management Science. Rossi, M. (2004). Decoding the free/open source (’F/OSS’) puzzle, a survey of theoretical and empirical contributions. Mimeo. Department of Political Economy, Siena. Rossi, M.A. (2004). Decoding the ‘free/open source (F/open source) software puzzle’: a survey of theoretical and empirical contributions. University of Sienna Department of Political Economy Working Paper No. 424.
322
S. M. Maurer and S. Scotchmer
Rossi, C., A. Bonaccorsi (2005). Intrinsic vs. extrinsic incentives in profit-oriented firms supplying open source products and services. First Monday Issue 10(No. 5). Samuelson, P. (1984). CONTU revisited: the case against copyright protection in machine-readable form. Duke Law Review 1984, 663–769. Schmidt, K., M. Schnitzer (2002). Public subsidies for open source? Some economic policy issues of the software market. University of Munich, Mimeo. Scotchmer, S. (2004). Ideas and Incentives. MIT, Cambridge. The Linux Counter (2005). Available at http://www.counter.li.org/estimates.php (self-published). Titmuss, R.M. (1972). The Gift Relationship: From Human Blood to Social Policy. Vintage, New York. Varian, H.R., C. Shapiro (2003). Linux adoption in the public sector: an economic analysis. Mimeo, University of California Haas School of Business. von Hippel, E. (2002). Open source projects as horizontal innovation networks—by and for users. MIT Sloan School of Management Working Paper No. 4366-02. von Hippel, E. (2004). Democratizing innovation. MIT, London. von Krogh, G., S. Spaeth, K. Lakhani (2003). Community, joining, and specialization in open source software innovation: a case study. Research Policy 32, 1217. Weber, S. (2000). The political economy of open source software. Berkeley Roundtable on the International Economy Working Paper 140. Weber, S. (2004). The Success of Open Source. Harvard University Press, Cambridge. West, J. (2003). How open is open enough? Melding proprietary and open source platform strategies. Research Policy 32, 1259. West, J., S. Gallagher (2004). Key challenges of open innovation: lessons from open source software. Mimeo, San Jose State College of Business. West, J., S. O’Mahoney (2005). Contrasting community building in sponsored and community founded open source projects, in: R. H. Sprague (ed.), IEEE Proceedings of the 38th Annual Hawaii International Conference on System Sciences. IEEE Computer Society Press, Los Alamitos, CA. Zeitlyn, D. (2003). Gift economies in the development of open source software: anthropological reflections. Research Policy 32, 1287.
Terrence Hendershott, Ed., Handbooks in Information Systems, Vol. 1 Copyright r 2006 by Elsevier B.V.
Chapter 6
Information, Search, and Price Dispersion
Michael R. Baye Kelley School of Business, Indiana University, Bloomington, IN, USA
John Morgan Haas School of Business and Department of Economics, University of California, Berkeley, CA, USA
Patrick Scholten Department of Economics, Bentley College, Waltham, MA, USA
Abstract We provide a unified treatment of alternative models of information acquisition/transmission that have been advanced to rationalize price dispersion in online and offline markets for homogeneous products. These different frameworks—which include sequential search, fixed sample search, and clearinghouse models—reveal that reductions in (or the elimination of) consumer search costs need not reduce (or eliminate) price dispersion. Our treatment highlights a ‘‘duality’’ between search-theoretic and clearinghouse models of dispersion and shows how auction-theoretic tools may be used to simplify (and even generalize) existing theoretical results. We conclude with an overview of the burgeoning empirical literature. The empirical evidence suggests that price dispersion in both online and offline markets is sizeable, pervasive, and persistent—and does not purely stem from subtle differences in firms’ products or services.
1
Introduction
Simple textbook models of competitive markets for homogeneous products suggest that all-out competition among firms will lead to the so-called 323
324
M.R. Baye et al.
‘‘law of one price.’’ Yet, empirical studies spanning more than four decades (see Table 1a and b) reveal that price dispersion is the rule rather than the exception in many homogeneous product markets. The observation that the prices different firms charge for the same product often differ by 30 percent or more led Hal Varian to suggest that ‘‘the ‘law of one price’ is no law at all’’ (Varian, 1980, p. 651). This chapter provides a unified treatment of several theoretical models that have been developed to explain the price dispersion observed in homogeneous product markets and surveys the burgeoning empirical literature (including the studies summarized in Table 1a and b), which documents ubiquitous price dispersion. A key motivation for this chapter is to dispel the erroneous view that the Internet—through its facilitation of dramatic declines in consumer search costs—will ultimately lead to the ‘‘law of one price.’’ When confronted with evidence of price dispersion, many are quick to point out that even in markets for seemingly homogeneous products, subtle differences among the ‘‘services’’ offered by competing firms might lead them to charge different prices for the same product. Nobel Laureate George Stigler’s initial response to wags making this point was philosophical: ‘‘y [While] a portion of the observed dispersion is presumably attributable to such difference[s] y it would be metaphysical, and fruitless, to assert that all dispersion is due to heterogeneity’’ (Stigler, 1961, p. 215). Thirty-five years later, the literature has amassed considerable support for Stigler’s position. As we shall see in Sections 2 and 3, there is strong theoretical and empirical evidence that much (and in some markets, most) of the observed dispersion stems from information costs—consumers’ costs of acquiring information about firms and/or firms’ costs of transmitting information to consumers. As Fig. 1 reveals, research on information, search, and price dispersion has become increasingly important since the publication of Stigler’s seminal article on the Economics of Information. Until about 1998, most studies focused on environments where consumers incur a positive cost of obtaining each additional price quote. Search costs in these studies consist of consumers’ opportunity cost of time in searching for lower prices (the socalled ‘‘shoe-leather’’ costs), plus other costs associated with obtaining price quotes from competing firms (such as the incremental cost of the postage stamps or phone calls used in acquiring price information from firms). Consumers in these environments weigh the cost of obtaining an additional price quote against the expected benefits of searching an additional firm. As we discuss in Section 2.1, equilibrium price dispersion can arise in these environments under a variety of market conditions and search strategies (including sequential and fixed sample search). While marginal search costs are useful in explaining price dispersion in some markets, in many online markets incremental search costs are very low—and in some cases, zero. For example, price comparison sites and shopbot technologies create environments where consumers may obtain a
Ch. 6. Information, Search, and Price Dispersion
325
Table 1 Measures of price dispersion reported in the literaturea Study
(a) Offline markets Bailey (1998)
Borenstein and Rose (1994) Carlson and Pescatrice (1980) Eckard (2004)
Friberg et al. (2001)
Data period
Product market
Intervals of estimated price dispersion measures
Dispersion measure
1997
1986
Books Books Compact discs Compact discs Software Software U.S. Airline
13.2% 10.4% 17.6% 11.0% 7.1% 8.1% 0.018–0.416
Standard deviation Standard deviation Standard deviation Standard deviation Standard deviation Standard deviation Gini coefficient
1976
Consumer sundries
3.3–41.4%
1901–2001 Baking powder, sugar, salt—1901 Baking powder, sugar, salt—2001 1999 Books
3.1–10.1%
Coefficient of variation Coefficient of variation Coefficient of variation Range
Books Compact discs Compact discs Books (Sweden)
Lach (2002)
Marvel (1976)
Pratt et al. (1979)
Compact discs (Sweden) 1993–1996 Refrigerator (Israel) Chicken, flour, coffee (Israel) 1964–1971 Regular gasoline Regular gasoline Premium gasoline Premium gasoline 1975 Various products and services Various products and services Various products and services
0.0–13.4% $54.00– $122.00 $21.94– $76.20 $20.00– $40.00 $12.91– $23.86 $19.00– $58.00 $21.00– $46.00 4.9% 11.4–19.7% $0.048 $0.015 $0.048 $0.017 4.4–71.4% 11.0– 567.0% 7.2–200.0%
Standard deviation Range Standard deviation Range Range Coefficient of variation Coefficient of variation Range Standard deviation Range Standard deviation Coefficient of variation Range Value of information
326
M.R. Baye et al. Table 1. (Continued )
Study
Data period
Product market
Intervals of estimated price dispersion measures
Dispersion measure
Roberts and Supina (2000)
1963–1987 Wood products
13.8–90.2%
Coefficient of variation Coefficient of variation Coefficient of variation Coefficient of variation Coefficient of variation Coefficient of variation Coefficient of variation Coefficient of variation Coefficient of variation
Scholten and Smith (2002)
Sorensen (2000)
Stigler (1961)
Villas-Boas (1995)
Fabrics
18.8–78.1%
Coffee
14.3–25.1%
Ready-mixed concrete Newsprint
13.2–37.2%
Gasoline
6.2–11.8%
Tinplate steel cans
25.0–31.0%
Pan bread
26.0–49.6%
Corrugated shipping containers 1976–2000 Consumer sundries—1976 Consumer sundries—2000 Consumer sundries—2000 1998 Prescription drugs Prescription drugs 1953
Anthracite coal Anthracite coal 1959 Identical automobiles Identical automobiles 1985–1987 Coffee
4.5–8.2%
21.8–39.6%
3.3–41.4%
$3.46 $1.15 $165.00
Coefficient of variation Coefficient of variation Coefficient of variation Range Coefficient of variation Range Standard deviation Range
$42.00
Standard deviation
21.5%
Coefficient of variation
1.6–42.0% 5.7–28.4% $13.17 22.0%
Ch. 6. Information, Search, and Price Dispersion
327
Table 1. (Continued ) Study
(b) Online markets only Ancarani and Shankar (2004)
Arbatskaya and Baye (2004) Arnold and Saliba (2002)
Data period
Product market
Intervals of estimated price dispersion measures
Dispersion measure
2002
Books (Italy)
h4.26–h4.84
Standard deviation
Books (Italy)
h20.00– h22.88 h2.29–h2.79
Range
1998 2001
Compact discs (Italy) Compact discs (Italy) Mortgage interest rates Textbooks Textbooks
Baye et al. (2003) Baye et al. (2004a)
Baye et al. (2004b)
Baylis and Perloff (2002) Brynjolfsson and Smith (2000)
Textbooks 2000–2001 Consumer electronics 2000–2001 Consumer electronics Consumer electronics 1999–2001 Consumer electronics Consumer electronics 1999 Cameras Scanners 1998–1999 Books
Chevalier and Goolsbee (2003) Clay et al. (2001)
2001
Clay et al. (2003)
1999
Clemons et al. (2002)
Compact discs Books
1999–2000 Books
Standard deviation Range
h11.82– h14.75 >0.25
Range
10.7–52.6%
Range
3.5–10.0%
Coefficient of variation Price gap Range
0.2–12.5% $123.88– $143.15 9.1–9.7% 3.79–5.38%
Coefficient of variation Gap
57.4%
Range
12.5% $342.00
Coefficient of variation Range
$106.00 33.0%
Range Range
25.0% 8.1–2.3%
Range Range
27.7%
Books Books
$7.62 10.0–18.0%
Travel
$8.03–13.40
Coefficient of variation Range Coefficient of variation Range
328
M.R. Baye et al. Table 1. (Continued ) Product market
Data period
Ellison and Ellison (2004) Gatti and Kattuman (2003)
2000–2001 Memory modules
5.9–29.0%
Range
2002
3.0–15.3%
Coefficient of variation
4.3–14.2%
Coefficient of variation
5.6–20.4%
Coefficient of variation
2.2–13.3%
Coefficient of variation
6.6–14.0%
Coefficient of variation
3.5–16.2%
Coefficient of variation Coefficient of variation
Consumer electronics (France) Consumer electronics (Italy) Consumer electronics (Netherlands) Consumer electronics (Spain) Consumer electronics (Sweden) Consumer electronics (UK) Consumer electronics (Denmark) Consumer electronics (France) Consumer electronics (Italy) Consumer electronics (Netherlands) Consumer electronics (Spain) Consumer electronics (Sweden) Consumer electronics (UK)
Intervals of estimated price dispersion measures
Dispersion measure
Study
6.3–20.2%
7.8–47.4%
Range
9.3–27.8%
Range
8.9–54.6%
Range
3.8–32.4%
Range
16.4–50.4%
Range
7.0–54.9%
Range
12.8–42.9%
Range
Ch. 6. Information, Search, and Price Dispersion Table 1. (Continued ) Study
Hong and Shum (2006)
Janssen et al. (2005)
Data period
2002
2004
Product market
Consumer electronics (Denmark) Consumer electronics (France) Consumer electronics (Italy) Consumer electronics (Netherlands) Consumer electronics (Spain) Consumer electronics (Sweden) Consumer electronics (UK) Consumer electronics (Denmark) Books
2000
Dispersion measure
1.6–16.1%
Gap
3.6–13.7%
Gap
8.9–34.6%
Gap
3.7–18.0%
Gap
5.9–15.6%
Gap
2.5–14.5%
Gap
3.6–31.9%
Gap
$8.19–27.05
Range
Books
6.2–8.5%
Keyboards
$6.50– $91.67 8.0–52.0%
Keyboards Pan et al. (2002)
Intervals of estimated price dispersion measures
Books
15.0%
Compact discs
15.4%
DVDs
12.7%
PDAs
11.8%
Software
11.7%
Consumer electronics
9.6%
Coefficient of variation Range Coefficient of variation Coefficient of variation Coefficient of variation Coefficient of variation Coefficient of variation Coefficient of variation Coefficient of variation
329
330
M.R. Baye et al. Table 1. (Continued )
Study
Data period
Pan et al. (2003)
2000–2003 Consumer electronics and Books Books Compact discs DVDs Computers Software Consumer electronics 1999 Books
Smith and Brynjolfson (2001)
Product market
Books a
Intervals of estimated price dispersion measures
Dispersion measure
9.8–11.7%
Coefficient of variation
33.3–48.9% 22.2–51.0% 30.7–43.7% 15.0–34.4% 19.0–35.6% 22.1–45.7%
Range Range Range Range Range Range
28.0–33.0%
Value of information Standard deviation
$6.29– $10.51
Includes studies comparing offline and online price dispersion.
list of the prices that different sellers charge for the same product. Despite the fact that this information is available to consumers in seconds, ultimately at the cost of a single ‘‘mouse click,’’ the overwhelming empirical finding is that even in these environments, price dispersion is pervasive and significant—the law of one price is egregiously violated online. In Section 2.2, we examine an alternative line of theoretical research where marginal search costs are not the key driver for price dispersion. Our theoretical analysis concludes in Section 2.3 with a discussion of alternative behavioral rationales for price dispersion (including bounded rationality on the part of firms and/or consumers). Section 3 provides a more detailed overview of the growing empirical literature. As one might suspect based on the trend in Fig. 1 and the research summarized in Table 1a and b, most empirical studies of price dispersion postdate the Internet and rely on online data. Our view is that this is more an artifact of the relative ease with which data may be collected in online markets—not an indication that price dispersion is more important (or more prevalent) in online than offline markets. For this reason, we have attempted to provide a balanced treatment of the literatures on online and offline price dispersion. As we shall argue, the overwhelming conclusion of both literatures is that price dispersion is not purely an artifact of product heterogeneities.
Ch. 6. Information, Search, and Price Dispersion
331
Fig. 1. Percentage of articles published in the American Economic Review, Journal of Political Economy, and Econometrica on information, search or price dispersion.
2
Theoretical models of price dispersion
This section presents alternative models that have been used to rationalize the price dispersion observed in both offline and online markets. One approach is to assume that it is costly for consumers to gather information about prices. In these ‘‘search-theoretic’’ models, consumers searching for the best price incur a positive cost of obtaining each additional price quote. Representative examples include Stigler (1961), Rothschild (1973), Reinganum (1979), MacMinn (1980), Braverman (1980), Burdett and Judd (1983), Carlson and McAfee (1983), Rob (1985), Stahl (1989, 1996), Dana (1994), McAfee (1995), Janssen and Moraga-Gonza´lez (2004), as well as Janssen et al. (2005). A second approach de-emphasizes the marginal search cost as a source for price dispersion. Instead, consumers access price information by consulting an ‘‘information clearinghouse’’ (e.g., a newspaper or an Internet price comparison site); e.g., Salop and Stiglitz (1977), Shilony (1977), Rosenthal (1980), Varian (1980), Narasimhan (1988), Spulber (1995), Baye and Morgan (2001), and Baye et al. (2004a).1 The distinguishing feature of ‘‘clearinghouse models’’ is that a subset of consumers gain access to a list of prices charged by all firms and purchase at the lowest listed price. In the 1 A third approach de-emphasizes consumer search and mainly focuses on whether price dispersion can arise when consumers ‘‘passively’’ obtain price information directly from firms (as in direct mail advertisements); cf. Butters (1977), Grossman and Shapiro (1984), Stegeman (1991), Robert and Stahl (1993), McAfee (1994), and Stahl (1994). A related marketing literature examines similar issues, ranging from loyalty and price promotion strategies to channel conflicts and the Internet; see Lal and VillasBoas (1998), Lal and Sarvary (1999), Raju et al. (1990), and Rao et al. (1995).
332
M.R. Baye et al.
earliest of these models, equilibrium price dispersion stems from ex ante heterogeneities in consumers or firms. For example, in the Varian and Salop-Stiglitz models, some consumers choose to access the clearinghouse to obtain price information, while others do not. In Shilony et al., some consumers are loyal to a particular firm (and thus will buy from it even if it does not charge the lowest price), while other consumers are ‘‘shoppers’’ and only purchase from the firm charging the lowest price. Spulber (1995) shows that equilibrium price dispersion arises even when all consumers can costlessly access the clearinghouse—provided each firm is privately informed about its marginal cost. Baye and Morgan (2001) offer a clearinghouse model that endogenizes not only the decisions of firms and consumers to utilize the information clearinghouse (in the previous clearinghouse models, firms’ listing decisions are exogenous), but also the fees charged by the owner of the clearinghouse (the ‘‘information gatekeeper’’ to consumers and firms who wish to access or transmit price information. They show that a dispersed price equilibrium exists even in the absence of any ex ante heterogeneities in consumers or firms. In this section, we provide an overview of the key features and ideas underlying these literatures. 2.1 Search-theoretic models of price dispersion We begin with an overview of search-theoretic approaches to equilibrium price dispersion. The early literature stresses the idea that, when consumers search for price information and search is costly, firms will charge different prices in the market. There are two basic sorts of models used: models with fixed sample size search and models where search is sequential. We will discuss each of these in turn. The search models considered in this subsection are all based on the following general environment. A continuum of price-setting firms (with unit measure) compete in a market selling an identical (homogeneous) product. Firms have unlimited capacity to supply this product at a constant marginal cost, m. A continuum of consumers is interested in purchasing the product. Let the mass of consumers in the market be m, so that the number of customers per firm is m. Each consumer has a quasi-linear utility function, u (q)+y, where q is the quantity of the homogeneous product and y is the quantity of some numeraire good whose price is normalized to be unity. This implies that the indirect utility of a consumer who pays a price p per unit of the product and who has an income of M is Vðp; MÞ ¼ uðpÞ þ M where u ( ) is nonincreasing in p. By Roy’s identity, note that the demand for the product of relevance is q (p) –u0 (p). To acquire the product, a consumer must first obtain a price quote from a store offering the product for sale. Suppose that there is a search cost, c, per
Ch. 6. Information, Search, and Price Dispersion
333
price quote.2 If, after obtaining n price quotes, a consumer purchases q (p) units of the product from one of the firms at price p per unit, the consumer’s (indirect) utility is V ¼ uðpÞ þ M cn The analysis that follows focuses on posted price markets where consumers know the distribution of prices but do not know the prices charged by particular stores.3 2.1.1 The Stigler model Stigler (1961) considers the special case of this environment where: 1. Each consumer wishes to purchase KZ1 units of the product; that is, q(p) ¼ –u0 (p) ¼ K; 2. The consumer’s search process is fixed sample search—prior to searching, consumers determine a fixed sample size, n, of firms from whom to obtain price quotes and then buy from the firm offering the lowest price; and 3. The distribution of firms’ prices is given by an exogenous nondegenerate cdf F(p) on ½p; p: Stigler assumes that a consumer chooses a fixed sample size, n, to minimize the expected total cost (expected purchase cost plus search cost) of purchasing K units of the product: E½C ¼ KE½pðnÞ min þ cn where E½pðnÞ min ¼ E½minfp1 ; p2 ; . . . ; pn g; that is, the expected lowest price quote obtained from n draws from F. Since the distribution of the lowest of n n draws is F ðnÞ min ðpÞ ¼ 1 ½1 F ðpÞ ; Z p E½C ¼ K pdF ðnÞ min ðpÞ þ cn "
p
Z
¼ K pþ
#
p n
½1 F ðpÞ dp þ cn p
2 In what follows, we assume that consumers have identical search costs. Axell (1977) offers a model of price dispersion with heterogeneous search costs. 3 This assumption is relaxed in Rothschild (1974), Benabou and Gertner (1993), and Dana (1994), where buyers learn about the distribution of prices as they search, and in Rauh (1997), where buyers’ search strategies depend on only finitely many moments of the distribution of prices. Daughety (1992) offers an alternative search-theoretic model of equilibrium price dispersion that results from informational asymmetries and a lack of price precommitment on the part of firms.
334
M.R. Baye et al.
where the second equality obtains from integration by parts. Notice that the term in square brackets reflects the expected purchase price, which is a decreasing function of the sample size, n. However, since each additional price observation costs c>0 to obtain, an optimizing consumer will choose to search a finite number of times, n ; and thus will generally stop short of obtaining the best price ðpÞ in the market. The distribution of transaction prices is the distribution of the lowest of n draws from F, that is,
ðn Þ ðpÞ ¼ 1 ð1 FðpÞÞn . F min
From this, Stigler concludes that dispersion in both posted prices and transactions prices arises as a consequence of costly search. How do transactions prices and search intensity relate to the quantity of the item being purchased (or equivalently, to the frequency of purchases)?4 Stigler’s model offers sharp predictions in this dimension. Note that the expected benefit to a consumer who increases her sample size from n –1 to n is ðnÞ E½BðnÞ ¼ E½pðn1Þ E½p K, (1) min min which is decreasing in n. Furthermore, the expected benefit from search are greater for products bought in greater quantities or more frequently, that is, equation (1) is increasing in K. Since the cost of the nth search is independent of K while the expected benefit is increasing in K, it immediately follows that the equilibrium search intensity, n ; is increasing in K. That is, consumers obtain more price quotes for products they buy in greater quantities (or more frequently). Despite the fact that the Stigler model assumes each individual inelastically purchases K units of the product, a version of the ‘‘law of demand’’ holds: each firm’s expected demand is a nonincreasing function of its price. To see this, note that a firm charging price p is visited by mn consumers and nn 1 offers the lowest price with probability ð1 F ðpÞÞ Thus, a representative firm’s expected demand when it charges a price of p is n
QðpÞ ¼ mnn Kð1 FðpÞÞn
1
(2)
which is decreasing in p. 4 K may be related to purchase frequency as follows. Suppose prices are ‘‘valid’’ for T periods, and the consumer wishes to buy one unit every trT periods, that is, t represents a consumer’s purchase frequency. Then the total number of units purchased during the T periods is KT/t. Thus, an increase in purchase frequency (t) is formally equivalent to an increase in K in the model above.
Ch. 6. Information, Search, and Price Dispersion
335
The Stigler model implies that both the expected transactions price (Proposition 1) as well as the expected total costs inclusive of search costs (Proposition 2) are lower when prices are more dispersed (in the sense of a mean preserving spread).5 Proposition 1. Suppose that a price distribution G is a mean preserving spread of a price distribution F. Then the expected transactions price of a consumer who obtains n>1 price quotes is strictly lower under price distribution G than under F. ðnÞ Proof. Let D ¼ E F ½pðnÞ min E G ½pmin be the difference in the expected transactions price under F compared to G. We will show that for all n>1, D>0. Using the definition of E½pðnÞ min ;
Z
Z
1
pnð1 F ðpÞÞ
D¼
n1
1
tnð1 GðtÞÞn1 dGðtÞ
dF ðpÞ
1
1
Let u ¼ F(p) and u ¼ G(p), so that du ¼ dF(p), du ¼ dG(p), p ¼ F –1(u), and t ¼ G –1(u). Then Z
1
D¼n
F Z
1
ðuÞð1 uÞ
n1
Z du n
0
1
G 1 ðuÞð1 uÞn1 du
0 1
ðF 1 ðuÞ G 1 ðuÞÞð1 uÞn1 du
¼n 0
Since G is a mean preserving spread of F, there exists a unique interior point u ¼ F(EF[P]) such that F 1 ðun Þ ¼ G 1 ðun Þ: Further, for all uoun ; F 1 ðuÞ G 1 ðuÞ40 and for all u4un ; F 1 ðuÞ G1 ðuÞo0: Thus Z
u
ðF 1 ðuÞ G1 ðuÞÞð1 uÞn1 du
D¼n Z
0 1
þ
ðF
1
1
ðuÞ G ðuÞÞð1 uÞ
n1
du .
u
R1 Rz G is a mean preserving spread of F if (a) 1 ½GðpÞ F ðpÞdp ¼ 0 and ðbÞ 1 ½GðpÞ F ðpÞdp 0; with strict inequality for some z. Note that (a) is equivalent to the fact that the means of F and G are equal. Together, the two conditions imply that F and G cross exactly once (at the mean) on the interior of the support. 5
336
M.R. Baye et al.
Next, notice that (1 – u)n–1 is strictly decreasing in u; hence, Z un ðF 1 ðuÞ G 1 ðuÞÞð1 u Þn1 du D4n 0
Z
1
ðF
þ
1
n1
1
ðuÞ G ðuÞÞð1 u Þ
u n1
Z
¼ nð1 u Þ
du
1
ðF 1 ðuÞ G 1 ðuÞÞdu
0
¼0 where the last equality follows from the fact that F and G have the same mean. ’ Proposition 2. Suppose that an optimizing consumer obtains more than one price quote when prices are distributed according to F, and that price distribution G is a mean preserving spread of F. Then the consumer’s expected total costs under G are strictly less than those under F. Proof. Suppose that, under F, the optimal number of searches is n : Then the consumer’s expected total cost under F is
E½C F ¼ E F ½pðnÞ min K cn
4E G ½pðnÞ min K cn
E½C G , where the strict inequality follows from Proposition 1, and the weak inequality follows from the fact that n searches may not be optimal under the distribution G. ’ At first blush, it might seem surprising that consumers engaged in fixed sample search pay lower average prices and have lower expected total costs in environments where prices are more dispersed. The intuition, however, is clear: In environments where prices are more dispersed, the prospects for price improvement from search are higher because the left tail of the price distribution—the part of the distribution where ‘‘bargains’’ are to be found—becomes thicker as prices become more dispersed. 2.1.2 The Rothschild critique and Diamond’s paradox While Stigler offered the first search-theoretic rationale for price dispersion, the model has been criticized for two reasons. First, as pointed out in Rothschild (1973), the search procedure assumed in Stigler’s model may not be optimal. In fixed sample search, consumers commit to a fixed number, n, of stores to search and then buy at the lowest price at the
Ch. 6. Information, Search, and Price Dispersion
337
conclusion of that search. A clear drawback to such a strategy is that it fails to incorporate new information obtained during search, such as an exceptionally low price from an early search. Indeed, once the best price quote obtained is sufficiently low, the benefit in the form of price improvement drops below the marginal cost of the additional search. As we will see below, sequential search results in an optimal stopping rule such that a consumer searches until she locates a price below some threshold, called the reservation price. Second, the distribution of prices, F, is exogenously specified and is not based on optimizing firm behavior. In fact, in light of equation (2), a representative firm with constant marginal cost of m enjoys expected profits of pðpÞ ¼ ðp mÞQðpÞ. That is, absent any cost heterogeneities, each firm faces exactly the same expected profit function. Why then, would firms not choose the same profit-maximizing price or, more generally, how could the distribution of prices generated by profit-maximizing firms be consistent with the price distribution over which consumers were searching? In short, Rothschild pointed out that it is far from clear that information costs give rise to an equilibrium of price dispersion with optimizing consumers and firms; in Stigler’s model, only one side of one market, the consumers, are acting in an optimizing fashion consistent with equilibrium. For this reason, Rothschild criticized the early literature for its ‘‘partial–partial equilibrium’’ approach. Diamond (1971) advanced this argument even further—he essentially identified conditions under costly search where the unique equilibrium in undominated strategies involves all firms charging the same price—the monopoly price. Diamond’s result may be readily seen in the following special case of our environment where: 1. Consumers have identical downward sloping demand, i.e., u00 ðpÞ ¼ q0 ðpÞo0; 2. Consumers engage in optimal sequential search; 3. A firm acting as a monopoly would optimally charge all consumers the unique monopoly price, p ; and 4. A consumer who is charged the monopoly price earns surplus sufficient to cover the cost of obtaining a single price quote, that is, uðp Þ4c: In this environment, all firms post the monopoly price and consumers visit only one store, purchase at the posted price, p ; and obtain surplus uðp Þ c40: Given the stopping rule of consumers, each firm’s best response is to charge the monopoly price; given that all firms charge p ; it is optimal for each consumer to search only once. To see that this is the
338
M.R. Baye et al.
unique equilibrium in undominated strategies, suppose to the contrary that there is an equilibrium in which some firm posted a price below the monopoly price (clearly, pricing above the monopoly price is a dominated strategy). Let p0 be the lowest such posted price. A firm posting the lowest price could profitably deviate by raising its price to the lower of p or p0 +c. Any consumer visiting that firm would still rationally buy from it since the marginal benefit of an additional search is smaller than c—the marginal cost of an additional search. Thus, such a firm will not lose any customers by this strategy and will raise its earnings on each of these customers. The Diamond paradox is striking: even though there is a continuum of identical firms competing in the model—a textbook condition for perfect competition—in the presence of any search frictions whatsoever the monopoly price is the equilibrium. Rothschild’s criticism of the Stigler model, along with the Diamond paradox, spawned several decades of research into whether costly search could possibly generate equilibrium price dispersion—a situation where consumers are optimally gathering information given a distribution of prices, and where the distribution of prices over which consumers are searching is generated by optimal (profit-maximizing) decisions of firms. 2.1.3 The Reinganum model and optimal sequential search Reinganum (1979) was among the first to show that equilibrium price dispersion can arise in a sequential search setting with optimizing consumers and firms. Reinganum’s result may be seen in the following special case of our environment where: 1. Consumers have identical demands given by u0 (p) ¼ q(p) ¼ Kpe, where e o1 and K>0; 2. Consumers engage in optimal sequential search; 3. Firms have heterogeneous marginal costs described by the atomless distribution G(m) on ½m; m; 4. A consumer who is charged the monopoly price by a firm with the to cover the cost of highest marginal cost, m; earns surplus sufficient obtaining a single price quote, that is, u 1þ m 4c: Reinganum shows that, under these assumptions, there exists a dispersed price equilibrium in which firms optimally set prices and each consumer engages in optimal sequential search. To establish this, we first show how one derives the optimal reservation price in a sequential search setting. Suppose consumers are confronted with a nondegenerate distribution of prices F(p) on ½p; p; that is atomless, except possibly at p: Consumers engage in optimal sequential search with free recall. If, following the nth search, a consumer has already found a best price z min(p1, p2,y,pn), then, by
Ch. 6. Information, Search, and Price Dispersion
339
making an additional search, such a consumer expects to gain benefits of Z z BðzÞ ¼ ðuðpÞ uðzÞÞdFðpÞ p
Z
z
u0 ðpÞFðpÞdp;
¼ p
where the second equality obtains through integration by parts. Using Leibnitz’ rule, we have B0 ðzÞ ¼ u0 ðzÞF ðzÞ ¼ Kz F ðzÞ40.
ð3Þ
Thus, the expected benefits from an additional search are lower when the consumer has already identified a relatively low price. Since search is costly (c>0), consumers must weigh the expected benefits against the cost of an additional search. The expected net benefits of an additional search are hðzÞ BðzÞ c. If the expected benefits from an additional search exceed the additional cost, h (z)>0, it is optimal for the consumer to obtain an additional price quote. If h (z)o0, the consumer is better off purchasing at the price z than obtaining an additional price quote. A consumer’s optimal sequential search strategy may be summarized as follows: Rp Case 1. hðpÞo0 and p uðpÞdF ðpÞoc: Then the consumer’s optimal strategy is to not search. Rp Case 2. hðpÞo0 and p uðpÞdF ðpÞdp c: Then the consumer’s optimal strategy is to search until she obtains a price quote at or below the reservation price, r ¼ p: Case 3. hðpÞ 0: Then the consumer’s optimal strategy is to search until she obtains a price quote at or below the reservation price, r, where r solves Z r hðrÞ ¼ ðuðpÞ uðrÞÞdF ðpÞ c ¼ 0. (4) p
Equation (4) represents a price at which a consumer is exactly indifferent between buying and making an additional search. To see that such a price is uniquely defined by this equation, notice that hðpÞ ¼ co0; hðpÞ 0; and h0 (z) ¼ B0 (z)>0. A consumer who observes a price that exceeds r will optimally ‘‘reject’’ that price in favor of continued search, while a consumer who observes a price below r will optimally ‘‘accept’’ that price and stop searching. Case 1 is clearly not economically interesting as it leads to the absence of any market for the product in the first place. Case 2 arises when the expected utility of purchasing the product exceeds the cost of an initial
340
M.R. Baye et al.
search, but the distribution of prices is sufficiently ‘‘tight’’ relative to search costs to make additional searches suboptimal. Most of the existing search literature, including Reinganum, restricts attention to Case 3, as we shall do hereafter. The reservation price defined in equation (4) has several interesting comparative static properties. Totally differentiating equation (4) with respect to r and c, and using equation (3) reveals that dr 1 1 ¼ ¼ 40. dc qðrÞF ðrÞ Kr F ðrÞ Thus, an increase in search costs leads to a higher reservation price: other things equal, the range of ‘‘acceptable’’ prices is greater for products with higher search costs. Note that, for the special case when q(r) ¼ 1, dr/dc ¼ 1/F(r)>1. In this case, a one unit increase in search costs increases the range of acceptable prices by more than one unit—that is, there is a ‘‘magnification effect’’ of increases in search costs.6 Reinganum avoids Rothschild’s criticism and the ‘‘Diamond paradox’’ by introducing firm cost heterogeneities. Since each firm j differs in its marginal cost, mj, prices will differ across firms even when they price as monopolists. Suppose that a fraction 0rlo1 of firms price above r, and recall that there are m consumers per firm. A representative firm’s expected profit when it prices at pj is m ( if pj r; ðpj mj Þqðpj Þ 1l Epj ¼ 0 if pj 4r: Ignoring for a moment the fact that a firm’s demand is zero if it prices above r, note that profit-maximization implies the first-order condition h i m ¼ 0. ðpj mj Þq0 ðpj Þ þ qðpj Þ 1l Standard manipulation of the first-order condition for profit-maximization implies that firm j ’s (unconstrained) profit-maximizing price is a constant markup over its cost: pj ¼ mj . 1þ Suppose that firms simply ignore the consumer’s reservation price, r, and price at this markup. This would imply that consumers face a distribution ^ of posted prices FðpÞ ¼ Gðpð1 þ Þ=Þ on the interval ½m =ð1 þ Þ; m=ð1 þ Þ: Given this distribution of prices, optimizing consumers would set a 6 In general, there may be either a magnification or an attenuation effect of a one unit increase in the cost of search.
Ch. 6. Information, Search, and Price Dispersion
341
reservation price, r, such that Z r ^ c ¼ 0: hðrÞ ¼ ðuðpÞ uðrÞÞd FðpÞ p
Furthermore, if rom=ð1 þ Þ; firms charging prices in the interval ðr; m=ð1 þ Þ would enjoy no sales. Since the elasticity of demand is constant, firms that would maximize profits by pricing above r in the absence of consumer search find it optimal to set their prices at r when consumers ^ search.7 Thus, the distribution of prices, FðpÞ; is inconsistent with optimizing behavior on the part of firms. In fact, given the reservation price r, optimizing behavior on the part of firms would imply a distribution of prices ( F^ ðpÞ if por FðpÞ ¼ 1 if p ¼ r: To establish that this is, in fact, an equilibrium distribution of prices one must verify that consumers facing this ‘‘truncated’’ distribution of prices have no incentive to change their reservation price. Given this truncated distribution of prices, the net expected benefits of search are Z r hðrÞ ¼ ðuðpÞ uðrÞÞdF ðpÞ c Z
p r
¼ Z
^ ðuðpÞ uðrÞÞd F^ ðpÞ þ ½1 FðrÞ½uðrÞ uðrÞ c
p r
¼
ðuðpÞ uðrÞÞd F^ ðpÞ c ¼ 0,
p
where the last equality follows from the fact that r is the optimal reservation price when consumers face the price distribution F^ : In short, Reinganum’s assumptions of downward sloping demand and cost heterogeneity give rise to an equilibrium of price dispersion with optimizing consumers and firms. Note that downward sloping demand and cost heterogeneities together play a critical role in generating equilibrium price dispersion in this environment. To see that both assumptions are required, suppose first that costs are heterogeneous but that each consumer wished to purchase one unit of the product, valued at u. In this case, given a reservation price of r u; all firms would find it optimal to price at r, and the distribution of prices would be degenerate. Of course, a reservation price of rou is inconsistent with 7 Reinganum assumes that m ¯ m =ð1 þ Þ; which guarantees that firms who would otherwise price above r find it profitable to price at r.
342
M.R. Baye et al.
optimizing behavior on the part of consumers. To see this, suppose that a consumer was unexpectedly presented with a price p0 ¼ r þ d where doc. According to the search strategy, such a consumer is supposed to reject this price and continue searching; however, the benefit from this additional search is less than the cost. Thus, a consumer should optimally accept a price p0 rather than continuing to search. The upshot of this is that the only equilibrium reservation price is r ¼ u: However, these are precisely the conditions given in Case 1; hence, the only equilibrium is where no consumers shop at all.8 If demand were downward sloping but firms had identical marginal costs of m, each firm would have an incentive to set the same price, pn ¼ minfr; m=ð1 þ Þg; given the reservation price. This leads back to Case 2 and one obtains the Diamond paradox: all firms charge the monopoly price, pn ¼ m=ð1 þ Þ: Indeed, in the environment above, a limiting case where the distribution of marginal costs converges to a point is exactly the Diamond model. Finally, we examine how the variance in the distribution of posted (and transactions) prices varies with search costs. Note that, in equilibrium, the variance in prices is given by s2 ¼ E½p2 ðE½pÞ2 !2 Z r Z r ¼ p2 dF ðpÞ pdF ðpÞ p
Z ¼
p r
p2 f^ðpÞdp þ ð1 F^ ðrÞÞr2
p
Z
r
!2 pf^ðpÞdp þ ð1 F^ ðrÞÞr
;
p
where f^ðpÞ is the density of F^ ðpÞ: Hence, ds2 ¼ 2rð1 F^ ðrÞÞ 2 dr
Z
r
! pf^ðpÞdp þ ð1 F^ ðrÞÞr ð1 F^ ðrÞÞ
p
¼ 2½1 F^ ðrÞðr E½pÞ 0 with strict inequality if rom=ð1 þ Þ: Thus, we have: Conclusion 1. In the Reinganum model, a reduction in search costs decreases the variance of equilibrium prices. As we will see below, however, this is not a general property of searchtheoretic models of price dispersion. 8 Carlson and McAfee (1983) show that if one introduces heterogeneities in consumer search costs, a dispersed price equilibrium may exist provided that individual consumers have perfectly inelastic (in contrast to downward sloping) demand.
Ch. 6. Information, Search, and Price Dispersion
343
2.1.4 Remarks on fixed versus sequential search It is useful to highlight some key differences between sequential and fixed sample size search. With sequential search, the number of searches is a random variable from a geometric distribution, and the expected number of searches, given a distribution of prices F(p) and reservation price r, is E½n ¼
1 . FðrÞ
In contrast, with fixed sample size search, consumers commit up front to n searches. Both types of search have advantages and disadvantages, and indeed Morgan and Manning (1985) have shown that both types of search can be optimal in different circumstances. The key advantage of sequential search is that it allows a searcher to economize on information costs— the decision-maker weighs the expected benefits and costs of gathering additional price information after each new price quote is obtained. If an acceptable price is obtained early on, the expected gains from additional searches are small and there is no need to pay the cost of additional searches. The primary advantage of fixed sample size search is that it allows one to gather information quickly. Consider, for instance, a firm that requires raw materials by the end of the week. If it takes a week for a raw materials vendor to provide a price quote, sequential search would permit the firm to obtain a price quote from only a single vendor. In this case, fixed sample size search is optimal—the firm commits to obtain quotes from n vendors, where n is chosen by the firm to minimize expected costs as outlined above in our discussion of the Stigler model. 2.1.5 The MacMinn model In light of the fact that there are instances in which fixed sample size search is optimal, one may wonder whether equilibrium price dispersion can arise in such a setting. MacMinn (1980) provides an affirmative answer to this question. MacMinn’s result may be seen in the following special case of our environment where: 1. Consumers have unit demand with valuation u; 2. Consumers engage in optimal fixed sample size search; and9 3. Firms have privately observed marginal costs described by the atomless distribution G(m) on ½m; m; where mou: At the time, MacMinn derived equilibrium pricing by solving a set of differential equations under the special case where G is uniformly distributed. However, subsequent to his paper, a key finding of auction theory, the
9
MacMinn also provides a version of the model that is valid for optimal sequential search.
344
M.R. Baye et al.
Revenue Equivalence Theorem (Myerson, 1981) was developed.10 Using the revenue equivalence theorem, we can generalize MacMinn’s results to arbitrary cost distributions. To see this, notice that when consumers optimally engage in a fixed sample size search consisting of n firms, each firm effectively competes with n 1 other firms to sell one unit of the product. Of these n firms, the firm posting the lowest price wins the ‘‘auction.’’ Using the revenue equivalence theorem, one can show that the expected revenues to a firm with marginal cost m in any ‘‘auction’’ where the firm charging the lowest price always wins and the firm with the highest marginal cost earns zero surplus is Z m n nn 1 RðmÞ ¼ mð1 GðmÞÞ þ ð1 GðtÞÞn 1 dt. (5) m
In the MacMinn model, expected revenues are simply a firm’s posted price, p(m), multiplied by the nprobability it charges the lowest price, which, in equilibrium, is ð1 GðmÞÞn 1 : Using the fact that RðmÞ ¼ pðmÞð1 nn 1 GðmÞÞ ; substituting into equation (5), and solving for p(m) yields the equilibrium pricing strategy of a firm with marginal cost m when consumers sample n firms n Z m 1 GðtÞ n 1 dt. (6) pðmÞ ¼ m þ 1 GðmÞ m Notice that, after integration by parts, we can rewrite equation (6) to obtain the familiar formula for equilibrium bidding in reverse first-price auctions h n i n ðn 1Þ 1Þ pðmÞ ¼ E mmin jmðn m , (7) min n
1Þ is the lowest of nn 1 draws from the distribution G. where mðn min For the special case where G is uniformly distributed, the equilibrium pricing strategy simplifies to
pðmÞ ¼
nn 1 1 m þ n m. n n n
(8)
Notice that the equilibrium pricing strategy gives rise to a distribution of posted prices, F(p), induced by the distribution of costs, that is, FðpÞ ¼ GðpðmÞÞ. 10 See Klemperer (1999) for a nontechnical survey of auction theory including the revenue equivalence theorem. McAfee and McMillan (1988) establishes an equivalence between search and auctions in a mechanism design context.
Ch. 6. Information, Search, and Price Dispersion
345
For this to be an equilibrium distribution of prices, it must be optimal for consumers to sample n firms. That is, E½Bðn
n
þ1Þ
n
oc E½Bðn Þ
where the expression E½BðnÞ ; as previously defined in equation (1) when K ¼ 1, is the expected benefit from increasing the number of price quotes obtained from n –1 to n. As in the Stigler model, a reduction in search costs increases the optimal sample size n (so that consumers optimally sample more firms). Thus, MacMinn shows that, provided search costs are low enough, a dispersed price equilibrium exists. This not only leads to ex post differences in consumers’ information sets (different consumers sample different firms and so observe different prices), but induces a degree of competition among firms (since they are competing against at least one other firm, whose cost they do not know). As in the Reinganum model, the level of price dispersion depends on the dispersion in firms’ costs. For the special case where costs are uniformly distributed, the variance in equilibrium prices ðs2p Þ is given by n n 1 2 2 2 sp ¼ sm , (9) nn where n is the optimal number of searches by consumers and s2m is the variance in firm’s costs. Two interesting results emerge from the model. First, the variance in prices increases as the variance in firms’ marginal costs increases. This result is intuitive. Somewhat counterintuitively, note that as the sample size increases, the variance in equilibrium prices increases. This implies that, taking into account the interaction between consumers and firms in this fixed sample size search model, dispersion varies inversely with search costs. Conclusion 2. In the MacMinn model, a reduction in search costs increases the variance of equilibrium prices. This conclusion is in contrast to Conclusion 1, where precisely the opposite implication is obtained in the Reinganum sequential search model. This highlights an important feature of search-theoretic models of price dispersion: Depending on the model, a reduction in search costs may be associated with higher or lower levels of price dispersion. In the Reinganum model, a reduction in search costs reduces the reservation price of consumers and thus induces marginal ‘‘high-cost’’ firms to reduce their prices from their monopoly price to the reservation price. Since the monopoly prices of low-cost firms are below the reservation price, their prices remain unchanged; lower search costs thus reduce the range of prices. In the MacMinn model, lower search costs induce consumers to sample more firms before purchasing—in effect, each firm competes with more rivals. As
346
M.R. Baye et al.
a consequence, the optimal amount of ‘‘bid shading’’ (pricing above marginal cost) is reduced, thus increasing the level of price dispersion. 2.1.6 The Burdett and Judd model Burdett and Judd (1983) were the first to show that equilibrium price dispersion can arise in a search-theoretic model with ex ante identical consumers and firms.11 Burdett and Judd’s main result may be seen in the following special case of our environment where: 1. Consumers have unit demand up to a price u; 2. Consumers engage in optimal fixed sample size search;12 3. Each firm has constant marginal cost, m, and would optimally charge all consumers the unique monopoly price, p ¼ u; and 4. A consumer who is charged the monopoly price earns surplus sufficient to cover the cost of obtaining a single price quote.13 In the Burdett and Judd model, an equilibrium consists of a price distribution F( p) (based on optimal pricing decisions by firms) and an 1 optimal search distribution oyn 41 n¼1 ; where oyn 4n¼1 is the distribution of the number of times a consumer searches in the population. Thus, yi is the probability that a consumer searches (or alternatively, the fraction of consumers that search) exactly i firms. If y1 ¼ 1, then all consumers sample only one firm. If y1 ¼ 0, then all consumers sample at least two firms, and so on. Consumers purchase from the firm sampled that offers the lowest price. We begin by studying optimal search on the part of consumers given a price distribution F( p). Recall that the expected benefit to a consumer who increases her sample size from n –1 to n is ðnÞ E½BðnÞ ¼ E½pðn1Þ min E½pmin
as in the Stigler model. Moreover, the expected benefit schedule is strictly decreasing in n. Thus, an optimal number of price quotes, n, satisfies E½Bðnþ1Þ oc E½BðnÞ . First, consider the case where all consumers obtain two or more price quotes, that is, where y1 ¼ 0. In this case, the optimal pricing strategy on 11 Janssen and Moraga-Gonza´lez (2004) provide an oligopolistic version of the Burdett and Judd model. 12 Burdett and Judd also provide a version of the model that is valid under optimal sequential search. 13 These assumptions are satisfied, for example, when 8 if pou; > <1 pu if u p u þ k; 1 qðpÞ ¼ k > :0 if p4u þ k
and k>c/2.
Ch. 6. Information, Search, and Price Dispersion
347
the part of firms is to price at marginal cost (the Bertrand paradox) since each firm is facing pure price competition with at least one other firm and all firms are identical. Of course, if all firms are pricing at marginal cost, then it would be optimal for a consumer to sample only one firm, which contradicts the hypothesis that y1 ¼ 0. Thus, we may conclude that, in any equilibrium y1>0. Next, consider the case where consumers all obtain exactly one price quote. In that case, each firm would optimally charge the monopoly price, p ¼ u: Hence, y1 a1 in any dispersed price equilibrium. From these two arguments it follows that, in any dispersed price equilibrium, y1 2 ð0; 1Þ: In light of the fact that consumers’ expected benefits from search are decreasing in the sample size, it follows that a consumer must be indifferent between obtaining one price quote and obtaining two price quotes. That is, in any dispersed price equilibrium E½Bð1Þ 4E½Bð2Þ ¼ c4E½Bð3Þ 4 4E½BðnÞ . Thus, in any dispersed price equilibrium, y1, y2>0, while yi ¼ 0 for all i>2. Let y1 ¼ y and y2 ¼ 1– y. We are now in a position to characterize an atomless dispersed price equilibrium. First, note that since yA(0, 1), there is a positive probability that a firm faces no competition when it sets its price. Thus, if firm i charges the monopoly price, it earns expected profits of E½pi jpi ¼ u ¼ ðu mÞ my. In contrast, a firm choosing some lower price ‘‘wins’’ when its price is below that of the other firm a consumer has sampled. Thus, if firm i charges a price piru, it earns expected profits of E½pi jpi u ¼ ðpi mÞ mðy þ ð1 yÞð1 F ðpi ÞÞÞ. Thus, for a given distribution of searches, equilibrium price dispersion requires that the distribution of firm prices, F( ), satisfies y þ ð1 yÞð1 F ðpÞÞ ¼
ðu mÞ y ðp mÞ
or FðpÞ ¼ 1
ðu pÞ ðyÞ , ðp mÞ ð1 yÞ
(10)
which is a well-behaved atomless cumulative distribution having support [m+y(u–m), u].
348
M.R. Baye et al.
Finally, it remains to determine an equilibrium value of y. Since each consumer must be indifferent between searching one or two firms, E½Bð2Þ ¼ c. Notice that, when y ¼ 0, or y ¼ 1, E [B(2)] ¼ 0, while E [B(2)]>0 for all yA (0, 1). Burdett and Judd show that E [B(2)] is quasi-concave; thus, when c is sufficiently low, there are generically two dispersed price equilibria—one involving a relatively high fraction of consumers making two searches, the other with a relatively low fraction of consumers.14 To summarize, Burdett and Judd show that equilibrium price dispersion can arise even when all firms and consumers are ex ante identical. In the equilibrium price distribution, all firms charge positive markups. A fraction y of consumers do not comparison shop—they simply search at one store and purchase. The remaining fraction of consumers are ‘‘shoppers’’—these consumers search at two stores and buy from whichever offers the lower price. 2.2 Models with an ‘‘Information Clearinghouse’’ In search-theoretic models, consumers pay an incremental cost for each additional price quote they obtain. These models are relevant, for example, when consumers must visit or phone traditional sellers in order to gather information about prices. They are also relevant in online environments where consumers must search the web sites of individual retailers to gather information about the prices they charge. An alternative class of models is relevant when a third party—an information clearinghouse—provides a subset of consumers with a list of prices charged by different firms in the market. Examples of this environment include newspapers which display prices different stores charge for the same product or service and online price comparison sites. In this section we provide a general treatment of clearinghouse models, and show that these models are surprisingly similar to those that arise under fixed sample size search. One of the key modeling differences is that clearinghouse models tend to be oligopoly models; thus, there is not a continuum of firms in such settings. Where possible, we shall use the same notation as in the previous section; however, for reasons that will become clear when we compare clearinghouse models with the search models presented above, we now let n denote the number of firms in the market. The general treatment that follows relies heavily on Baye and Morgan (2001) and Baye et al. (2004a). Consider the following general environment (which we will specialize to cover a variety of different models). There is a finite number, n>1, of 14 There is a nondispersed price equilibrium where all consumers search once and all firms charge the monopoly price.
Ch. 6. Information, Search, and Price Dispersion
349
price-setting firms competing in a market selling an identical (homogeneous) product. Firms have unlimited capacity to supply this product at a constant marginal cost, m. A continuum of consumers is interested in purchasing the product. This market is served by a price information clearinghouse. Firms must decide what price to charge for the product and whether to list this price at the clearinghouse. Let pi denote the price charged by firm i. It costs a firm an amount f Z0 if it chooses to list its price. All consumers have unit demand with a maximal willingness to pay of u>m.15 Of these, a mass, S>0, of the consumers are price-sensitive ‘‘shoppers.’’ These consumers first consult the clearinghouse and buy at the lowest price listed there provided this price does not exceed u. If no prices are advertised at the clearinghouse or all listed prices exceed u, then a ‘‘shopper’’ visits one of the firms at random and purchases if its price does not exceed u. A mass, LZ0, of consumers per firm purchase from that firm if its price does not exceed u. Otherwise, they do not buy the product at all. It can be shown that if L>0 or f>0, equilibrium price dispersion arises in the general model—provided of course that f is not so large that firms refuse to list prices at the clearinghouse. More precisely, Proposition 3. Let 0 foðn 1Þðu mÞS=n: Then, in a symmetric equilibrium of the general clearinghouse model: 1. Each firm lists its price at the clearinghouse with probability n 1=ðn1Þ n1 f a¼1 . ðu mÞS 2. If a firm lists its price at the clearinghouse, it charges a price drawn from the distribution n 1= ðn1Þ ! f þ ðu pÞL 1 FðpÞ ¼ on ½p0 ; u, 1 n1 a ðp mÞS where p0 ¼ m þ ðu mÞ
L n=ðn 1Þ þ f. LþS LþS
3. If a firm does not list its price at the clearinghouse, it charges a price equal to u. 4. Each firm earns equilibrium expected profits equal to Ep ¼ ðu mÞL þ 15
1 f. n1
Baye and Morgan (2001) consider an environment with downward sloping demand.
350
M.R. Baye et al.
Proof. First, observe that if a firm does not list its price at the clearinghouse, it is a dominant strategy to charge a price of u. Next, notice that aA(0,1] whenever nf o1. ðn 1Þðu mÞS This condition holds, since foðn 1Þðu mÞS=n: Notice that p0>m, provided that L>0 or f>0. In this case, it can be shown that F is a well-defined, atomless cdf on [p0, u]. When L ¼ 0 and f ¼ 0, notice that p0 ¼ m. In this case, the symmetric equilibrium distribution of prices is degenerate, with all firms pricing at marginal cost (the Bertrand paradox outcome). Next, we show that, conditional on listing a price, a firm can do no better than pricing according to F. It is obvious that choosing a price above or below the support of F is dominated by choosing a price in the support of F. A firm choosing a price p in the support of F earns expected profits of ! ! n1 X n1 i n1i i EpðpÞ ¼ ðp mÞ L þ ði Þa ð1 aÞ ð1 F ðpÞÞ S f. i¼0
Using the binomial theorem, we can rewrite this as: EpðpÞ ¼ ðp mÞ L þ ð1 aFðpÞÞn1 S f f ¼ ðu mÞL þ n 1; where we have substituted for F to obtain the second equality. Since a firm’s expected profits are constant on [p0, u], it follows that the mixed pricing strategy, F, is a best response to the other n–1 firms pricing based on F. When f ¼ 0, it is a weakly dominant strategy to list. It remains to show that when f>0 and a 2 ð0; 1Þ; a firm earns the same expected profits regardless of whether it lists its price. But a firm that does not list earns expected profits of S n1 Ep ¼ ðu mÞ L þ ð1 aÞ n f ¼ ðu mÞL þ , n1 which the expected profits earned by listing any price equals p 2 p0 ; u : ’ We are now in a position to examine the many well-known clearinghouse models that emerge as special cases of this general environment.
Ch. 6. Information, Search, and Price Dispersion
351
2.2.1 The Rosenthal model Rosenthal (1980) was among the first to show that equilibrium price dispersion can arise in a clearinghouse environment when some consumers have a preference for a particular firm. Under his interpretation, each firm enjoys a mass L of ‘‘loyal’’ consumers. Rosenthal’s main results may be seen in the following special case of the general clearinghouse model: 1. It is costless for firms to list prices on the clearinghouse: f ¼ 0; and 2. Each firm has a positive mass of loyal consumers: L>0. Since f ¼ 0, it follows from Proposition 3 that a ¼ 1; that is, all of the n firms advertise their prices with probability one. Using this fact and Proposition 3, the equilibrium distribution of prices is ðu pÞ L 1=ðn1Þ FðpÞ ¼ 1 on ½p0 ; u, (11) ðp mÞ S where L . LþS The price dispersion arising in the Rosenthal model stems from exogenous differences in the preferences of consumers. While shoppers view all products as identical and purchase at the lowest listed price, each firm is endowed with a stock of L loyals. The equilibrium price dispersion arises out of the tension created by these two types of consumers. Firms wish to charge u to extract maximal profits from the loyal segment, but if all firms did so a firm could slightly undercut this price and gain all of the shoppers. One might imagine that this ‘‘undercutting’’ argument would lead to the Bertrand outcome. However, once prices get sufficiently low, a firm is better off simply charging u and giving up on attracting shoppers. Thus, the only equilibrium is in mixed strategies—firms randomize their prices, sometimes pricing relatively low to attract shoppers and other times pricing fairly high to maintain margins on loyals. It is interesting to examine the equilibrium transactions prices in the market. Loyal customers expect to pay the average price charged by firms: Z u E½p ¼ pdFðpÞ, p0 ¼ m þ ðu mÞ
p0
while shoppers expect to pay the lowest of n draws from F(p); that is, the expected transaction price paid by shoppers is Z u ðnÞ E½pmin ¼ pdF ðnÞ min ðpÞ, p0
where
F ðnÞ min ðpÞ
is the cdf associated with the lowest of n draws from F.
352
M.R. Baye et al.
How do transactions prices vary with the number of competing firms? Rosenthal’s striking result is that, as the number of competing firms increases, the expected transactions prices paid by all consumers go up. As we shall see below, the result hinges on Rosenthal’s assumption that entry brings more loyals into the market. Indeed, the fraction of shoppers in the market is S/(S+nL) and it may readily be seen that as n becomes large, shoppers account for an increasingly small fraction of the customer base of firms. As a consequence, the incentives to compete for these customers is attenuated and prices rise as a result. The key is to recognize that increases in n change the distribution of prices, and this effect as well as any order statistic effect associated with an increase in n must be taken into account. Formally, notice that the equilibrium distribution of prices, F, is stochastically ordered in n. That is, the distribution of prices when there are n+1 firms competing first-order stochastically dominates the distribution of prices where there are n firms competing. This implies that the transactions prices paid by loyals increase in n. To show that the transactions prices paid by shoppers also increase in n requires a bit more work; however, one can show that the same stochastic ordering obtains for the cdf F ðnÞ min ðpÞ: Finally, it is useful to note the similarity between the Rosenthal version of the clearinghouse model and the search-theoretic model of Burdett and Judd. In Burdett and Judd, even though there is a continuum of firms, each consumer only samples a finite number of firms (one or two). Further, in Burdett and Judd, a fixed fraction of consumers per firm, my, sample only a single firm. In effect, these consumers are ‘‘loyal’’ to the single firm sampled, while the fraction (1–y)m of customers sampling two firms are ‘‘shoppers’’—they choose the lower of the two prices. For this reason, when n ¼ 2 in the Rosenthal model, the equilibrium price distribution given in equation (11) is identical to equation (10) in Burdett and Judd model (modulo relabeling the variables for loyals and shoppers). 2.2.2 The Varian model Varian (1980) was among the first to show that equilibrium price dispersion can arise in a clearinghouse environment when consumers have different ex ante information sets.16 Varian interprets the S consumers as ‘‘informed consumers’’ and the L consumers as ‘‘uninformed’’ consumers. Thus a mass, S, of consumers choose to access the clearinghouse while others, the mass L per firm, do not. Varian’s main result may be seen in the following special case of the general clearinghouse model: 1. It is costless for firms to list prices on the clearinghouse: f ¼ 0; and 2. The total measure of ‘‘uninformed’’ consumers lacking access to the 16 Png and Hirshleifer (1987), as well as Baye and Kovenock (1994), extend the Varian model by allowing firms to also engage in price matching or ‘‘beat or pay’’ advertisements.
Ch. 6. Information, Search, and Price Dispersion
353
clearinghouse is U>0; hence, each firm is visited by L ¼ U/n of these consumers. Again, since f ¼ 0, it follows that a ¼ 1 and hence all n firms advertise their prices at the clearinghouse. Using this fact and setting L ¼ U/n in Proposition 3, the equilibrium distribution of prices is 1=ðn1Þ ðu pÞ Un FðpÞ ¼ 1 on ½p0 ; u; ðp mÞ S where p0 ¼ m þ ðu mÞ
U=n . U=n þ S
The fact that this atomless distribution of prices exists whenever there is an exogenous fraction of consumers who do not utilize the clearinghouse raises the obvious question: Can this equilibrium persist when consumers are making optimal decisions? Varian shows that the answer to this question is yes—provided different consumers have different costs of accessing the clearinghouse. The easiest way to see this is to note that the value of information provided by the clearinghouse is the difference in the expected price paid by those accessing the clearinghouse, E½pðnÞ min ; and those not, E [p], that is; VOI ðnÞ ¼ E½p E½pðnÞ min ,
(12)
where VOI denotes the value of (price) information contained at the clearinghouse. Suppose consumers face a cost of accessing the information provided by the clearinghouse. Note that this cost is essentially a fixed cost of gaining access to the entire list of prices, not a per price cost as in the search-theoretic models considered above. Varian assumes that the cost to type S and L consumers of accessing the clearinghouse is kS and kL, with kSokL. Then provided ksrVOI (n)okL, type S consumers will optimally utilize the clearinghouse, while the type L consumers will not. In short, if different consumers have different costs of accessing the clearinghouse, there exists an equilibrium of price dispersion with optimizing consumers and firms. In such an equilibrium, informed consumers pay lower average prices than uninformed consumers. It is important to emphasize that when one endogenizes consumers’ decisions to become informed in the Varian model, the level of price dispersion is not a monotonic function of consumers’ information costs. When information costs are sufficiently high, no consumers choose to become informed, and all firms charge the ‘‘monopoly price,’’ u. When consumers’ information costs are zero, all consumers choose to become informed, and all firms price at marginal cost in a symmetric equilibrium—the Bertrand paradox. Thus, for sufficiently high or low information costs, there is no
354
M.R. Baye et al.
price dispersion; for moderate information costs, prices are dispersed on the nondegenerate interval ½p0 ; u: A similar result obtains in Stahl (1989), which is related to Varian as follows. Stahl assumes a fraction of consumers have zero search costs and, as a consequence, view all firms’ prices and purchase at the lowest price in the market. These consumers play the role of S in Varian’s model (informed consumers). The remaining fraction of consumers correspond to the L’s in the Varian model, but rather than remaining entirely uninformed, these consumers engage in optimal sequential search in the presence of positive incremental search costs. Stahl shows that when all consumers are shoppers, the identical firms price at marginal cost and there is no price dispersion. When no consumers are shoppers, Diamond’s paradox obtains and all firms charge the monopoly price. As the fraction of shoppers varies from zero to one, the level of dispersion varies continuously—from zero to positive levels, and back down to zero. Conclusion 3. In general, price dispersion is not a monotonic function of consumers’ information costs or the fraction of ‘‘shoppers’’ in the market. How does the number of competing firms affect transactions prices? In the Rosenthal model, we saw that increased ‘‘competition’’ led to higher expected transactions prices for all consumers. In the Varian model, in contrast, the effect of competition on consumer welfare depends on whether or not the consumer chooses to access the clearinghouse. Morgan et al. (forthcoming) show that as n increases, the competitive effect predictably leads to lower average transactions prices being paid by informed consumers. However, the opposite is true for uninformed consumers—as the number of competing firms increases, firms face reduced incentives to cut prices in hopes of attracting the ‘‘shoppers’’ and, as a consequence, the average price charged by a firm, which is also the average price paid by an uninformed consumer, increases. If one views the clearinghouse as representing access to price information on the Internet, then one can interpret the price effect as one consequence of the so-called ‘‘digital divide’’; see Baye et al. (2003). Consumers with Internet access are made better off by sharper online competition while those without such access are made worse off. 2.2.3 The Baye and Morgan model All of the above models assume that it is costless for firms to advertise their prices at the clearinghouse. Baye and Morgan (2001) point out that, in practice, it is generally costly for firms to advertise their prices and for consumers to gain access to the list of prices posted at the clearinghouse. For example, newspapers charge firms fees to advertise their prices and may choose to charge consumers subscription fees to access any posted information. The same is true of many online environments. Moreover, the clearinghouse is itself an economic agent, and presumably has an incentive to endogenously choose advertising and subscription fees to maximize its own
Ch. 6. Information, Search, and Price Dispersion
355
expected profits. Thus, Baye and Morgan examine the existence of dispersed price equilibria in an environment with optimizing consumers, firms, and a monopoly ‘‘gatekeeper’’ who controls access to the clearinghouse. Specifically, Baye and Morgan consider a homogeneous product environment where n identical, but geographically distinct, markets are each served by a (single) local firm. Distance or other transaction costs create barriers sufficient to preclude consumers in one market from buying this product in another market; thus each firm in a local market is a monopolist. Now imagine that an entrepreneur creates a clearinghouse to serve all markets. In the Internet age, one can view the clearinghouse as a virtual marketplace—through its creation, the gatekeeper expands both consumers’ and firms’ opportunities for commerce. Each local firm now has the option to pay the gatekeeper an amount f to post a price on the clearinghouse in order to gain access to geographically disparate consumers. Each consumer now has the option to pay the gatekeeper an amount k to shop at the clearinghouse and thereby purchase from firms outside the local market. The monopoly gatekeeper first sets k and f to maximize its own expected profits. Given these fees, profit-maximizing firms make pricing decisions and determine whether or not to advertise them at the clearinghouse. Similarly, consumers optimally decide whether to pay k to access the clearinghouse. Following this, a consumer can simply click her mouse to research prices at the clearinghouse (if she is a subscriber), visit the local firm, or both. With this information in hand, a consumer decides whether and from whom to purchase the good. Baye and Morgan show that the gatekeeper maximizes its expected profits by setting k sufficiently low that all consumers subscribe, and charging firms strictly positive fees to advertise their prices. Thus, Baye and Morgan’s main results may be seen in the following special case of the general clearinghouse model: 1. The gatekeeper optimally sets positive advertising fees: f>0; and 2. The gatekeeper optimally sets subscription fees sufficiently low such that all consumers access the clearinghouse, that is, L ¼ 0. Under these conditions, using Proposition 3, we obtain the following characterization of equilibrium firm pricing and listing decisions: Each firm lists its price at the clearinghouse with probability n 1=ðn1Þ n1 f a¼1 2 ð0; 1Þ. ðu mÞS When a firm lists at the clearinghouse, it charges a price drawn from the distribution n 1=ðn1Þ ! 1 n1 f FðpÞ ¼ on ½p0 ; v; 1 ðp mÞS a
356
M.R. Baye et al.
where p0 ¼ m þ
n ðn1Þ f
. S When a firm does not list its price, it charges a price equal to u, and each firm earns equilibrium expected profits equal to 1 f. n1 Notice that na represents the aggregate demand by firms for advertising and is a decreasing function of the fee charged by the gatekeeper. Prices advertised at the clearinghouse are dispersed and strictly lower than unadvertised prices (u). Several features of this equilibrium are worth noting. First, equilibrium price dispersion arises with fully optimizing consumers, firms, and endogenous fee-setting decisions on the part of the clearinghouse—despite the fact that there are no consumer or firm heterogeneities and all consumers are ‘‘fully informed’’ in the sense that, in equilibrium, they always purchase from a firm charging the lowest price in the global market. Second, while equilibrium price dispersion in the Varian model is driven by the fact that different consumers have different costs of accessing the clearinghouse, Baye and Morgan show that an optimizing clearinghouse will set its fees sufficiently low that all consumers will rationally access the clearinghouse. Equilibrium price dispersion arises because of the gatekeeper’s incentives to set strictly positive advertising fees. Strikingly, despite the fact that all consumers use the gatekeeper’s site and thus purchase at the lowest global price, firms still earn positive profits in equilibrium. In expectation, these profits are proportional to the cost, f, of accessing the clearinghouse. Ep ¼
Conclusion 4. In the Baye and Morgan model, equilibrium price dispersion persists even when it is costless for all consumers to access the information posted at the gatekeeper’s site. Indeed, price dispersion exists because it is costly for firms to transmit price information (advertise prices) at the gatekeeper’s site. Why does the gatekeeper find it optimal to set low (possibly zero) fees for consumers wishing to access information, but strictly positive fees to firms who wish to transmit price information? Baye and Morgan point out that this result stems from a ‘‘free-rider problem’’ on the consumer side of the market that is not present on the firm side. Recall that the gatekeeper can only extract rents equal to the value of the outside option of firms and consumers. For each side of the market, the outside option consists of the surplus obtainable by not utilizing the clearinghouse. As more consumers access the site, the number of consumers still shopping locally dwindles and the outside option for firms is eroded. In contrast, as more firms utilize the clearinghouse, vigorous price competition among these firms reduces listed
Ch. 6. Information, Search, and Price Dispersion
357
prices and leads to a more valuable outside option to consumers not using the clearinghouse. Thus, to maximize profits, the gatekeeper optimally subsidizes consumers to overcome this ‘‘free-rider problem’’ while capturing rents from the firm side of the market. No analogous ‘‘free-rider problem’’ arises on the firm side; indeed greater consumer participation at the clearinghouse increases the frequency with which firms participate (a increases) and hence permits greater rent extraction from firms. 2.2.4 Models with asymmetric consumers In general, little is known about the general clearinghouse model with asymmetric consumers.17 However, for the special case of two firms, results are available. Narsimhan (1988) analyzes the case where there are two firms, one of whom has more loyal customers than the other, and where the cost of listing at the clearinghouse is zero. He shows that the unique equilibrium involves mixed strategies in which the firm with more loyal customers prices less aggressively than its rival. As a result, the rival (with fewer loyals) earns equilibrium profits that exceed the profits it would earn by exclusively serving its loyal customers at the monopoly price. The firm with more loyals, in contrast, earns equilibrium profits that equal the profits it could earn by exclusively serving its loyal customers at the monopoly price. Interestingly, shoppers are harmed by these asymmetries – they pay, on average, higher prices than would be the case were the loyal customers divided equally between the firms. Further theoretical work on clearinghouse models with consumer asymmetries and positive listing fees would be a useful addition to the literature. 2.5.5 Cost heterogeneities and the Spulber model Spulber (1995) considers a situation where consumers have access to the complete list of prices and buy from the firm offering the lowest price. Of course, in such a setting, if firms were identical one would immediately obtain the Bertrand outcome. To generate price dispersion, Spulber examines the situation where firms have heterogeneous costs and consumers have downward sloping demand. However, the main economic intuition underlying the model may be seen through the following adaptation of our general clearinghouse framework for the unit demand case: 1. All consumers are shoppers: S>0 and L ¼ 0; 2. There is no cost to advertise prices on the clearinghouse: f ¼ 0; and 3. Firms have privately observed marginal costs described by the atomless distribution G (m) on ½m; m: Since there are no costs to advertise prices, all firms list prices on the clearinghouse. Each firm faces competition from n –1 other firms with 17 For specific clearinghouse models, some results are available. For instance, Baye et al. (1992) characterize all equilibria in a version of the Varian model in which firms have asymmetric numbers of consumers.
358
M.R. Baye et al.
random marginal costs. Since the firm charging the lowest price wins the entire market, firms are effectively competing in an auction in which their own costs are private information. For the special case of unit demand, the equilibrium price for a firm is again the familiar expression from a firstprice auction ðn1Þ pðmÞ ¼ E½mðn1Þ min jmmin m,
(13)
mðn1Þ min
where is the lowest of n–1 draws from the distribution G. There are several noteworthy features of this equilibrium. First, equilibrium firm pricing entails positive markups despite the fact that all consumers are ‘‘shoppers’’ and have a complete list of prices. Intuitively, there is a trade-off between lowering one’s price to attract shoppers and the profitability of this price. In equilibrium, this results in a markup which depends on the number of competing firms. As the number of firms grows large, the equilibrium markup becomes small. Second, notice that cost heterogeneity leads to equilibrium price dispersion despite the fact consumers are identical and all consumers are purchasing at the lowest price. It is interesting to compare the Spulber model, which occurs in the clearinghouse framework, with the search-theoretic framework of MacMinn. Notice that when the number of competing firms in Spulber, n, is equal to the optimal fixed sample size for consumers in the MacMinn model, n ; the equilibrium distribution of prices, equations (13) and (7), are identical in the two models. That is, cost heterogeneities are sufficient to generate price dispersion in oligopoly models where all consumers obtain complete price information, as well as in models where a continuum of firms compete but each consumer only obtains price quotes from a finite number n of these firms. 2.3 Bounded rationality models of price dispersion Several recent papers have emphasized that bounded rationality can also lead to price dispersion. The idea is to relax the Nash equilibrium assumption—which requires that each decision-maker in the market is choosing an action (be it a price or a search strategy) that is a best response to given actions of other market participants. Two equilibrium concepts—quantal response equilibrium (McKelvey and Palfrey, 1995) and epsilon equilibrium (Radner, 1980)—are particularly useful because they nest the standard Nash equilibrium concept as a special case. In a quantal response equilibrium (QRE), the likelihood that a particular firm sets a specific price depends on the expected profits arising from that price (see Lopez-Acevedo, 1997). A firm’s price is determined by a stochastic decision rule, but prices leading to higher expected profits are more likely to be charged. Of course, each firm’s expected profits from different pricing decisions depend on the probability distributions of other players’ prices. A QRE requires that all firms hold correct beliefs about the probability distributions
Ch. 6. Information, Search, and Price Dispersion
359
of other players’ actions. The nondegenerate distributions of prices resulting in a QRE may be viewed as shocks to firms’ profit functions. Alternatively, nondegenerate price distributions might stem from decision errors by firms. Such errors may arise from limitations in managers’ cognitive processing abilities or ‘‘bugs’’ in dynamic pricing algorithms used by Internet retailers. In an e-equilibrium, the prices charged by each firm are such that no firm can gain more than e in additional profits by changing its price. Such an equilibrium may arise because of cognitive or motivational constraints on the part of firms. For example, if it is costly to reprogram dynamic pricing algorithms, managers may not be willing to incur these economic or psychic costs when the resulting gain is small (less than e). Recently, Baye and Morgan (2004) applied QRE and e-equilibrium concepts to pricing games and showed that only a little bounded rationality is needed to generate the patterns of price dispersion documented in laboratory experiments as well as observed on Internet price comparison sites. In a similar vein, Rauh (2001) shows that price dispersion can arise when market participants make small but heterogeneous mistakes in their beliefs about the distribution of prices. Ellison (2005) provides a more detailed treatment of recent advances along these lines. 2.4 Concluding remarks: theory Despite a slow start, there are now a variety of models that can be used to rationalize equilibrium price dispersion in online and offline markets. We conclude our theoretical discussion with the following general observations. 1. There is not a ‘‘one-size-fits-all’’ model of equilibrium price dispersion; different models are appropriate for analyzing different market environments. For instance, search-theoretic models are most appropriate for analyzing environments where consumers must visit different stores or firms’ web sites to gather price information. Clearinghouse models are appropriate when consumers are able to access a list of prices (e.g., in a newspaper or at a price comparison site). 2. The distribution of prices is determined by the interaction of all market participants—firms, consumers and, in the case of clearinghouse models, information gatekeepers. As a consequence, the level of price dispersion depends on the structure of the market—the number of sellers, the distribution of costs, consumers’ elasticities of demand, and so on. 3. Reductions in search costs may lead to either more or less price dispersion, depending on the market environment. Furthermore, the elimination of consumer search costs need not eliminate price dispersion. 4. Depending on the market environment, heightened competition (increases in the number of firms) can increase or decrease the level of
360
M.R. Baye et al.
dispersion. Moreover, in some models, heightened competition of this form leads to higher transactions prices paid by all consumers. In other models, the effect of increased competition on the welfare of consumers depends on which side of the ‘‘digital divide’’ a consumer resides. 5. Price dispersion is not purely an artifact of ex ante heterogeneities in firms or consumers. While differences in firms’ costs or base of loyal consumers (stemming from firms’ branding efforts, differential service qualities, or reputations) can contribute to equilibrium price dispersion, such differences are not necessary for equilibrium price dispersion. 6. Thanks to the Internet, information gatekeepers are playing an increasingly important role in the economy. In their attempt to maximize profits and enhance the value of information provided by their sites, information gatekeepers have an incentive to charge fees for their services that induce equilibrium price dispersion. 7. A little bounded rationality goes a long way in explaining price dispersion. 3
Empirical analysis of price dispersion
We now turn to the empirical literature on price dispersion. In Section 3.1, we discuss some of the strengths and weaknesses of commonly used metrics for measuring price dispersion in online and offline markets. Section 3.2 provides an overview of the empirical literature and highlights empirical evidence suggesting that information costs (either on the consumer or firm side of the market) contribute to price dispersion; that is, dispersion is not purely an artifact of subtle product heterogeneities. 3.1 Measuring price dispersion The equilibrium models of price dispersion presented above each imply non-degenerate distributions of prices, F(p), on some interval ½p; p: Given such a distribution, a standard measure of dispersion is the variance in prices. For each model of equilibrium price dispersion, this measure can be directly computed. For instance, in the MacMinn model, if firms have uniformly distributed marginal costs, the variance in prices is n n 1 2 ðm m Þ2 2 sp ¼ . nn 12 Notice that one is then in a position to test comparative static predictions of the model using this measure. In a similar manner, expressions for the variance in prices may be derived from the other models previously presented.
Ch. 6. Information, Search, and Price Dispersion
361
A number of authors use the sample variance to measure price dispersion (e.g., Pratt et al., 1979; Ancarani and Shankar, 2004). The obvious advantage is that it uses all available data. A drawback of this measure is apparent when comparing dispersion across products or over time. For instance, suppose that, during an inflationary period, the marginal costs of all firms in the MacMinn model increased by a factor g>1. In that case, the new variance would simply scale-up the original variance by a factor g2. Thus, this measure of price dispersion would change even though the underlying real economics of the situation are the same after the inflationary period. For this reason, if one wishes to compare levels of price dispersion either across different products or across time, one must standardize the data in some fashion. An alternative is to use the coefficient of variation, CV ¼ sp/E[P] (or its sample analogue), which is homogenous of degree zero in the level of prices. The CV is particularly useful when comparing levels of price dispersion over long periods of time (e.g., Scholten and Smith, 2002; Eckard, 2004) or across different products (e.g., Carlson and Pescatrice, 1980; Sorensen, 2000; Aalto-Seta¨la¨, 2003; Baye, 2004a,b). An added advantage is that, unlike some methods of standardization, the coefficient of variation may preserve the comparative static predictions of the model of interest. For instance, in the MacMinn model, equation (8) implies that the expected price is nn 1 m þ m m E½p ¼ þ n, nn 2 n and thus the coefficient of variation is 1 ðnn 1Þðm mÞ CV ¼ pffiffiffi n . 3 ðn 1Þðm þ mÞ þ 2m One may verify that this statistic is, like the variance, decreasing in search costs, but, unlike the variance, this statistic does not change with a multiplicative shift in firms’ costs. Another widely used measure of price dispersion is the (sample) range; see, for instance, Pratt et al. (1979) and Brynjolfsson and Smith (2000). ðnÞ Letting pðnÞ min and pmax denote, respectively, the lowest and highest of n observed prices drawn from F, then the range is ðnÞ RðnÞ ¼ pðnÞ max pmin .
Given the equilibrium distribution of prices implied by a particular theoretical model, comparative static predictions about changes in the range are possible based on the behavior of the highest and lowest order statistics.
362
M.R. Baye et al.
That is, one can perform comparative static analysis on the expected range18: ðnÞ E½RðnÞ ¼ E½pðnÞ max E½pmin .
Unfortunately, all of the above measures of price dispersion suffer from a potential theoretical defect. Suppose that n>2 firms compete in a classical homogeneous product Bertrand setting. Under standard conditions there will exist a unique symmetric equilibrium where all firms price at marginal cost. But in addition, there are asymmetric equilibria where two firms price at marginal cost and the remaining n –2 firms price strictly above marginal cost. Thus, price dispersion can arise in a classical Bertrand environment. Yet, the apparent price dispersion is arguably not economically relevant because the unique transactions price is marginal cost. To remedy this theoretical defect, Baye et al. (2004a) propose a measure called ‘‘the gap,’’ which they define to be the difference between the two lowest prices in the market. Letting pðnÞ 2 denote the second-lowest price realization from n draws from F, the (sample) gap is defined as19 ðnÞ GðnÞ ¼ pðnÞ 2 pmin .
The classical Bertrand model (as well as textbook models of perfect competition) implies that the gap between the two lowest prices is zero in any equilibrium (symmetric or otherwise). All of the oligopoly models of price dispersion discussed above, in contrast, imply a positive gap. An additional property of the gap is that it gives greater weight to low prices, which, in the absence of quantity data, one might reasonably assume lead to more sales than higher prices. The key disadvantage, shared by the range, is that it relies purely on extreme values of the data. Hence, the range and gap are more sensitive to outliers and other forms of ‘‘noise’’ than measures that use all the available data, such as the sample variance and coefficient of variation. In addition to these measures, the value of information (VOI) defined earlier in equation (12) can also be used as a gauge of dispersion. This measure, which is simply the difference between the average observed price and the lowest observed price, is zero in the absence of any price dispersion but otherwise positive. The principal advantage of this measure of dispersion is that it has a very intuitive interpretation: its value indicates the 18 To facilitate comparisons across different products or over time, it is sometimes useful to normalize the range by dividing it by the minimum or average price; see Baye et al. (2004b) and Brynjolfsson and Smith (2000). 19 As with the range, one can perform comparative static analyses for any of the theoretical models using the expected gap, and it is sometimes useful to normalize the gap by dividing by the lowest price. In this formulation, the gap represents the difference between the two lowest prices expressed as a percentage of the lowest price realization.
Ch. 6. Information, Search, and Price Dispersion
363
amount of money a consumer saves by purchasing at the best price rather than from a randomly selected firm in the market. 3.2 Price dispersion in the field If price dispersion stems from frictions related to the acquisition and transmission of information (as implied by the models in Section 2) rather than subtle differences in firms’ service levels, observed levels of dispersion should systematically depend on ‘‘environmental factors’’ present in the models. For example, in his seminal article on the economics of information, George Stigler advanced the following hypotheses: y dispersion itself is a function of the average amount of search, and this in turn is a function of the nature of the commodity: 1. The larger the fraction of the buyer’s expenditures on the commodity, the greater the savings from search and hence the greater the amount of search. 2. The larger the fraction of repetitive (experienced) buyers in the market, the greater the effective amount of search (with positive correlation of successive prices). 3. The larger the fraction of repetitive sellers, the higher the correlation between successive prices, and hence, the larger the amount of accumulated search. 4. The cost of search will be larger, the larger the geographic size of the market.
Stigler (1961, p. 219)
Stigler’s hypotheses offer a useful guide for understanding the empirical literature on price dispersion. Much of this literature tests Stigler’s hypotheses by examining whether search intensity (proxied by variables that affect the benefits and costs of search) is correlated with levels of price dispersion. As we have seen, however, when one takes Rothschild’s criticism into account, an increase in search intensity can lead to increases or decreases in the level of equilibrium price dispersion, depending on the model. Thus, one challenge for empirical researchers is choosing a model that closely approximates the ‘‘data generating’’ environment. A second challenge is to control for factors outside of the model that might influence levels of dispersion. A third challenge arises because firm optimization is absent in Stigler’s model, but is clearly present in the data. For this reason, a number of empirical studies look beyond Stigler’s hypotheses to test hypotheses derived from specific search-theoretic or clearinghouse models of equilibrium price dispersion. We provide a broad overview of these and related strands of the literature below. 3.2.1 Dispersion and the ‘‘benefits’’ of search The search-theoretic models presented in Section 2 imply that search intensity depends, in part, on the consumer’s demand for a product. In the Stigler model, demand is represented by the parameter, K. The greater is K, the greater the expected benefits of search and hence the greater the search
364
M.R. Baye et al.
intensity. Stigler’s first two hypotheses are based on the notion that the share of an item in a consumer’s overall budget and the frequency with which an item is purchased are good proxies for K. Dispersion for ‘‘cheap’’ versus ‘‘expensive’’ items. Stigler (1961) provides casual evidence in support of his first hypothesis—that dispersion is lower for items that account for a large expenditure share of a searcher’s consumption bundle (‘‘expensive items’’) than those that account for a smaller expenditure share (‘‘cheap items’’). Government coal purchases are a small percentage of the overall government budget, while a household’s expenditures on an automobile comprise (in 1961 as well as today) a much larger percentage of its overall budget. Stigler obtained different sellers’ prices for two homogeneous products—anthracite-grade coal to be sold to the government and an automobile to be sold to a household. Prices for anthracite coal ranged from $15.46 to $18.92, with an average price of $16.90 and a standard deviation of $1.15. Prices for the automobile (based on what Stigler called ‘‘an average amount of higgling’’) ranged from $2,350 to $2,515, with an average price of $2,436 and standard deviation of $42. Stigler’s data thus tend to support his first conjecture: If one calculates the implied coefficient of variation based on Stigler’s figures, the coefficient of variation for coal (which makes up a small percentage of the government’s budget) is 14.7 percent, while that for an automobile (which makes up a large percentage of a household’s budget) is 1.7 percent. Pratt et al. (1979) observe a similar pattern in a cross-section of consumer products sold in Boston in the 1970s. They obtain the following regression result by regressing the sample (log) standard deviation of prices for a given item on the sample (log) mean price for the same item: ln s ¼ 1:517 þ 0:892 ln E½p.
(14)
Straightforward manipulation of equation (14) reveals that a 1 percent increase in the mean price of an item decreases the coefficient of variation by 10.8 percent. Thus, the Pratt, Wise, and Zeckhauser data also suggest that, empirically, the coefficient of variation is lower for more expensive items than cheaper items. However, equation (14) also highlights that the relationship depends crucially on the measure of price dispersion used: if one were to use the standard deviation to measure price dispersion, equation (14) implies that a 1 percent increase in the mean price of a product leads to a 0.892 percent increase in dispersion, as measured by the standard deviation. A number of other authors have reported similar patterns in online and offline markets, both in the US and in Europe for products ranging from consumer sundries and electronic products to gasoline; cf. Marvel (1976), Carlson and Pescatrice (1980), Clay and Tay (2001), Scholten and Smith (2002), Johnson (2002), Gatti and Kattuman (2003), and Aalto-Seta¨la¨ (2003). More recently, Eckard (2004) compares price dispersion for staple products in 1901 and 2001, and reports coefficients of variation in 2001 that
Ch. 6. Information, Search, and Price Dispersion
365
are almost twice those based on data from 1901. Eckard argues that one reason for the increased dispersion is that his sample consists of staple items (such as sugar and baking powder) that accounted for a much larger share of household budgets in 1901 than in 2001. Dispersion and Purchase Frequency. In his second hypothesis, Stigler argues that in markets where there are more repetitive or experienced buyers, the greater is the amount of effective search. Unfortunately, it is difficult to directly test this hypotheses, since in most markets there is no direct (objective) measure of ‘‘buyer experience’’ or ‘‘purchase frequency’’ to use in examining its impact on levels of price dispersion. A number of the studies mentioned above, however, provide casual evidence that purchase frequency impacts the level of price dispersion (cf. Pratt et al. 1979; Carlson and Pescatrice, 1980). Sorensen (2000), however, has provided a very ‘‘clean’’ and elegant test of Stigler’s second hypothesis. His analysis is based on data from the market for prescription drugs. The unique aspect of this market is that purchase frequency—the typical dosage and duration of therapy for a given prescription drug—may be objectively measured. A consumers’ benefit per search is clearly highest for frequently purchased drugs, and, Sorensen argues, this should lead to greater search and lower price dispersion. His empirical analysis identifies a strong inverse relationship between purchase frequency and price dispersion. For example, after controlling for other factors (which together explain about one-third of the variation in prices), Sorensen finds that the price range for a drug that must be purchased monthly is about 30 percent lower than if it were a one-time therapy. Importantly, Sorensen shows that the results are qualitatively similar when alternative measures of price dispersion (such as the standard deviation) are used. 3.2.2 Dispersion and the ‘‘cost’’ of search Researchers studying the empirical relationship between search costs and price dispersion have faced obstacles similar to those of researchers focusing on the benefit side of the search equation. First, the predicted impact of search costs on levels of dispersion depends not only on the model, but also on the metric used for measuring dispersion. Second, search costs are generally unobservable. Some of the more influential papers in the area are ones that have devised innovative methods of dealing with these problems. One important example is Brown and Goolsbee (2002). Their starting point is the Stahl (1989) model of equilibrium price dispersion, which as we noted in Section 2, predicts that price dispersion is initially an increasing function of the fraction of ‘‘shoppers’’ who enjoy zero search costs, but after a threshold, is a decreasing function of the fraction of shoppers. Brown and Goolsbee point out that the Stahl model closely matches the market for term-life insurance during the 1992–1997 period. Consumers who did not have an Internet connection arguably had to search sequentially to obtain price quotes from different insurance agents, while those with
366
M.R. Baye et al.
Internet access could use web sites such as Quickquote.com to ‘‘costlessly’’ identify the company offering the lowest annual premium. In their data, variation in the fraction of ‘‘shoppers’’ (those who research insurance online) stems not only from the general rise in Internet penetration during the 1990s, but more importantly, from variation in the growth rates in Internet usage across different groups of policyholders. Brown and Goolsbee regress the standard deviation in residuals (obtained from a price regression that controls for observable characteristics of people and policy types) on a cubic function of their proxy for the fraction of ‘‘shoppers.’’ Consistent with the prediction of the Stahl model, price dispersion initially rises as the fraction of shoppers increases, but starts to decline once the fraction of consumers researching insurance online exceeds about 5 percent. A similar approach is implicit in a number of papers that have compared levels of dispersion in online versus offline markets (cf. Brynjolfsson and Smith, 2000; Carlton and Chevalier, 2001; Scholten and Smith, 2002; Ancarani and Shankar, 2004). The basic premise is that search costs are lower in online (search entails clicks) versus offline (search entails travel costs) markets.20 In general, since different search models make different predictions about the impact of reductions in search costs on levels of price dispersion, it is not too surprising that the findings of this literature are decidedly mixed; for some products, dispersion is lower in online markets; for other products, dispersion is actually higher online.21 Along these same lines, a number of studies compare average prices in online versus offline markets. The idea is that search costs are lower online, thus affecting not only the range or variance in prices, but also the mean price (and hence the coefficient of variation through both the mean and variance). Scott-Morton et al. (2001) find that prices are lower in online markets for automobiles. Consumers who purchase a car through the Internet referral service Autobytel.com reduce their purchase price by approximately 2.2 percent. A potentially confounding explanation for this price difference is that the consumers who choose to shop online may also be skilled ‘‘higglers,’’ to use Stigler’s phrase, and thus the price difference might purely reflect a difference in the negotiating skills of consumers across the two channels. Interestingly, Zettelmeyer et al. (2004) provide evidence that this is not the case: consumers who purchase automobiles online are not typically those who negotiate well in the traditional channel. 20 The view that online search is either more prevalent or cheaper than offline search is a matter of some debate; see, for instance, Adamic and Huberman (2001) and Johnson et al. (2004). Bakos (1997) was among the first to advance a theoretical argument that when the cost of price information is close to zero, equilibrium price is close to marginal cost. More recently, however, Harrington (2001) has argued that Bakos’ results are flawed. Finally, the Internet itself also offers opportunities for obfuscation (see Ellison and Ellison, 2004) or unobserved lack of inventories (see Arnold and Saliba, 2002) that can raise search and/or transactions costs relative to offline markets. 21 One may speculate that once shipping costs are accounted for, price dispersion online vanishes. This is not the case; cf. Brynjolfsson and Smith (2000), Smith and Brynjolfson (2001), Pan et al. (2002), Ancarani and Shankar (2004), Brynjolfsson et al. (2004), and Dinlersoz and Li (2005).
Ch. 6. Information, Search, and Price Dispersion
367
There are a number of other studies, however, that find equal or higher prices online (cf. Bailey, 1998; Erevelles et al., 2001; Goolsbee, 2001; Clemons et al., 2002; Clay, et al., 2003). Further studies distinguish price levels depending on whether the retailer is a solely online or ‘‘multichannel’’ (cf. Tang and Xing, 2001; Chevalier and Goolsbee, 2003). An alternative approach is to ‘‘recover’’ search costs using structural parameters from a particular model of price dispersion. For example, Hong and Shum (forthcoming) obtain search cost estimates using restrictions imposed by theoretical search models and assuming that observed price dispersion is an equilibrium phenomenon arising from heterogeneous consumer search costs. Their estimation technique is applied to online price data on four economics and statistics textbooks. They obtain search cost estimates ranging from $1.31 to $29.40 for these items. A similar approach can be used in clearinghouse models. Villas-Boas (1995) uses the theoretical density function implied by the Varian (1980) clearinghouse model to obtain estimates of the number of shoppers in the offline coffee and saltine cracker markets. More recently, Baye et al. (2005) used a theoretical clearinghouse model as the basis for estimating the fraction of ‘‘shoppers’’ in an online market for PDAs in the UK. Their results suggest that about 13 percent of the consumers in this market are shoppers. 3.2.3 Dispersion and the number of sellers The oligopoly models presented in Section 2 reveal that equilibrium distributions of prices, and hence levels of dispersion, vary with the number of sellers competing in the market. The direction in which prices move as a consequence of a change in the number of sellers is, however, model specific, as we saw in the Varian and Rosenthal models. Thus, examining the relationship between price dispersion and the number of competing sellers not only provides a test of whether informational factors play a role in generating observed price dispersion, but also in making distinctions among the various theory models. For instance, Baye et al. (2004a) examine the theoretical and empirical relationship between the number of competitors and the level of price dispersion in clearinghouse models. They show that the theoretical relationship between number of competitors and the level of price dispersion in clearinghouse models is, in general, ambiguous, due to competing ‘‘order statistic’’ and ‘‘strategic’’ effects. Through a calibration displayed in Fig. 2, they show that the impact of the number of sellers on price dispersion depends on the variant of the model. As the figure shows, in the Varian model (where firms’ information transmission costs do not drive price dispersion), the expected gap between the two lowest prices is initially increasing in the number of sellers, and then declines. In contrast, in the Baye and Morgan model (where firms’ information transmission costs are the main driver of price dispersion), the expected gap is monotonically decreasing in the number of firms. Based on online data from a popular
368
M.R. Baye et al.
Fig. 2. Theoretical and empirical relationship between price dispersion measured by percentage gap and the number of competing sellers listing prices on a clearinghouse site.
price comparison site for consumer electronics products, and controlling for other factors contributing to price dispersion, they find an inverse relation between the gap and the number of online sellers. This relationship is depicted as the dotted ‘‘observed’’ line in Fig. 2. As the figure reveals, the nonmonotonicity predicted by the Varian model, as well as the relatively flat relationship between the gap and number of firms predicted in the calibrated version of the Rosenthal model, is absent in the data. Specifically, in markets served by between two and four firms, the average gap (as a percentage of the lowest price) is about 14 percent. The average percentage gap falls to about 3 percent in markets with five to ten firms, and is less than 1 percent in markets with more than 10 firms. More broadly, several empirical papers have suggested that the amount of price dispersion observed in the market depends on various measures of the numbers of competitors. Marvel (1976) reports that an increase in the number of competitors (measured by ln(HHI)) reduces the range in the price of gasoline. Barron et al. (2004) study the structural determinants of price dispersion in the retail gasoline industry in four geographic locations, and provide empirical evidence that, controlling for station-level characteristics, an increase in station density decreases both price levels and price dispersion.22 Borenstein and Rose (1994) investigate the relationship 22
See also Png and Reitman (1994).
Ch. 6. Information, Search, and Price Dispersion
369
between dispersion among airfares and the number of competitors or flight density. They find that dispersion in fares increases on routes with lower flight density or more competition. Thus, there is evidence that the number of sellers matters for price dispersion. 3.2.4 Dispersion and price persistence Varian (1980) was the first to distinguish between what he referred to as ‘‘spatial’’ and ‘‘temporal’’ price dispersion. Under spatial price dispersion, different firms charge different prices at any point in time, but a firm’s position in the distribution of prices does not change over time. Absent random cost shocks, spatial price dispersion arises in the Reinganum, MacMinn, and Spulber models. In contrast, with temporal price dispersion, firms charge different prices at each point in time, but their position in the distribution of prices changes over time. Temporal price dispersion arises in the general clearinghouse model (and various special cases) as well as in the Burdett and Judd model. Varian critiques models of spatial price dispersion, arguing that if consumers can learn from experience that some firms persistently offer lower prices than other firms, then models of spatial price dispersion suggest a ‘‘convergence hypothesis’’: price dispersion should diminish over time due to the positive correlation in successive prices (to use Stigler’s terminology) and cumulative search information. This has led to a number of studies that examine whether there is any evidence for the convergence hypothesis and whether the temporal price dispersion predicted by the clearinghouse models is, in fact, present in the data. Using monthly store-level price data from Israel, and after controlling for observed and unobserved product heterogeneities, Lach (2002) finds some evidence of temporal price dispersion. Lach estimates month-to-month transitions among quartiles by firms, that is, the probability that a firm offering a price in a given quartile at the start of the month is still offering a price in the same quartile at the end of the month. His estimates suggest that the probability of remaining in the same quartile is 78 percent for firms selling refrigerators and 71 percent for firms selling flour. These probabilities are somewhat lower for firms selling chicken (51 percent) and coffee (43 percent). When the transition period is extended to six months instead of one month, the probability of remaining in the same quartile is considerably lower—falling to around 30–35 percent. Roberts and Supina (2000) suggest that structural differences in firms’ costs account for a considerable portion of price dispersion in the offline sector—as predicted by a variety of search-theoretic models. Using plant-level US Census data, they find some evidence for price persistence. The evidence is strongest in the tails of the distribution: high-price firms tend to persistently charge high prices, and low-price firms tends to persistently charge low prices. A variety of other studies also suggest that heterogeneities either across firms or across markets impact price dispersion
370
M.R. Baye et al.
in online markets (cf. Smith et al., 1999; Clay et al., 2001; Smith and Brynjolfson, 2001; Chen and Hitt, 2002; Resnick and Zeckhauser, 2002; Brynjolfsson et al., 2004). In all cases, however, even after controlling for various heterogeneities, economically significant levels of price dispersion remain. There is also evidence that online prices exhibit temporal price dispersion. For instance, Baye et al. (2004b) examine turnover of the identity of the low- and high-price firms using a dataset consisting of 36 popular consumer electronics products sold over a 19-month period. They find considerable evidence for month-to-month changes in the identity of the low-price firms, but some evidence of persistence in the identity of high-priced firms. Similarly, Iyer and Pazgal (2003) collect biweekly price data on music CDs, movie videos, and books from five price comparison sites: MySimon, BottomDollar, EvenBetter, Bsilly, and Pricescan during the period April– October 2000 and find empirical results suggesting that no single firm consistently charges the low price. Finally, Baye et al. (2004a) examine the convergence hypothesis of price dispersion using a dataset consisting of over four million daily price observations for over 1,000 consumer electronics products sold on a popular Internet price comparison site over an eight-month period. Even allowing for a nonlinear relationship between observed price dispersion and time, they find no evidence for the convergence hypothesis in this market—the level of price dispersion remained stable over the period. 3.3 Concluding remarks: empirics We conclude with four simple observations. 1. As is evident from the studies highlighted in Table 1, price dispersion is ubiquitous and persistent. Regardless of the particular product (tinplate cans or PDAs), the venue in which they are sold (online or offline, in the US or abroad), or the time period (1901 or 2005), the inescapable conclusion from the empirical literature is a validation of Stigler’s and Varian’s initial observations: Information remains a valuable resource, and the law of one price is still no law at all. 2. Theory is useful for understanding dispersion data, and dispersion data is useful for discriminating among alternative theoretical models. 3. The relationship between price dispersion and economic primitives is often sensitive to the measure of price dispersion used. 4. Despite the widespread adoption of inventions such as the automobile, the telephone, television, and the Internet, price dispersion is still the rule rather than the exception in homogeneous product markets. Reductions in information costs over the past century have neither reduced nor eliminated the levels of price dispersion observed in homogeneous product markets.
Ch. 6. Information, Search, and Price Dispersion
371
Acknowledgments We owe a special debt to Michael Rauh and Felix Va´rdy for providing us with detailed hand-written comments on earlier drafts. We also thank Ville Aalto-Seta¨la¨, Fabio Ancarani, Maria Arbatskaya, Judy Chevalier, Karen Clay, Woody Eckard, Sara Fisher Ellison, Xianjun Geng, Rupert Gatti, Jose Moraga Gonzalez, Joe Harrington, Terry Hendershott, Ganesh Iyer, Maarten Janssen, Ronald Johnson, Ken Judd, Ramayya Krishnan, Sol Lach, Rajiv Lal, Preston McAfee, Xing Pan, Jeff Perloff, Ivan Png, Ram Rao, Jennifer Reinganum, Nancy Rose, Venky Shankar, Jorge Silva-Risso, Michael Smith, Alan Sorensen, Dan Spulber, Mark Stegeman, Beck Taylor, Miguel Villas-Boas, Xiaolin Xing, and Richard Zeckhauser for encouragement, helpful comments, and suggestions. Of course, we are responsible for any shortcomings that remain in this offering. References Aalto-Seta¨la¨, V. (2003). Explaining price dispersion for homogeneous grocery products. Journal of Agricultural and Food Industrial Organization 1, Article 9, 1–14. Adamic, L.A., B.A. Huberman (2001). The web’s hidden order. Communications of the ACM 44(9), 55–59. Ancarani, F., V. Shankar (2004). Price levels and price dispersion within and across multiple retailer types: further evidence and extension. Journal of the Academy of Marketing Science 32(2), 176–187. Arbatskaya, M., M.R. Baye (2004). Are prices ‘sticky’ online? Market structure effects and asymmetric responses to cost shocks in online mortgage markets. International Journal of Industrial Organization 22(10), 1443–1462. Arnold, M.A., C. Saliba (2002). Price dispersion in online markets: the case of college textbooks. Working Paper. Axell, B. (1977). Search market equilibrium. Scandinavian Journal of Economics 79(1), 20–40. Bailey, J. (1998). Electronic commerce: prices and consumer issues for three products: books, compact discs, and software. Organization Economics Co-Operation Development 98(4), 3–22. Bakos, Y. (1997). Reducing buyer search costs: implications for electronic marketplaces. Management Science 43(2), 1676–1692. Barron, J.M., B.A. Taylor, J.R. Umbeck (2004). Number of sellers, average prices, and price dispersion. International Journal of Industrial Organization 22, 1041–1066. Baye, M.R., R. Gatti, P. Kattuman, J. Morgan, (2005). Estimating firm-level demand at a price comparison site: accounting for shoppers and the number of competitors. Working Paper. Baye, M.R., D. Kovenock (1994). How to sell a pickup truck: beat or pay advertisements as facilitating devices. International Journal of Industrial Organization 12(1), 21–33. Baye, M.R., D. Kovenock, C. de Vries (1992). It takes two to tango: equilibria in a model of sales. Games and Economic Behavior 4, 493–510. Baye, M.R., J. Morgan (2001). Information gatekeepers on the internet and the competitiveness of homogeneous product markets. American Economic Review 91(3), 454–474. Baye, M.R., J. Morgan (2004). Price dispersion in the lab and on the internet: theory and evidence. Rand Journal of Economics 35(3), 449–466. Baye, M.R., J. Morgan, P. Scholten (2003). The value of information in an online consumer electronics market. Journal of Public Policy and Marketing 22(1), 17–25. Baye, M.R., J. Morgan, P. Scholten (2004a). Price dispersion in the small and in the large: evidence from an internet price comparison site. Journal of Industrial Economics 52(4), 463–496.
372
M.R. Baye et al.
Baye, M.R., J. Morgan, P. Scholten (2004b). Temporal price dispersion: evidence from an online consumer electronics market. Journal of Interactive Marketing 18(4), 101–115. Baylis, K., J.M. Perloff (2002). Price dispersion on the internet: good firms and bad firms. Review of Industrial Organization 21, 305–324. Benabou, R., R. Gertner (1993). Search with learning from prices: does increased uncertainty lead to higher markups?. Review of Economic Studies 60(202), 69–94. Borenstein, S., N.L. Rose (1994). Competition and price dispersion in the U.S. airline industry. Journal of Political Economy 102(4), 653–683. Braverman, A. (1980). Consumer search and alternative market equilibria. Review of Economic Studies 47, 487–502. Brown, J.R., A. Goolsbee (2002). Does the internet make markets more competitive? Evidence from the life insurance industry. Journal of Political Economy 110(3), 481–507. Brynjolfsson, E., A.A. Dick, M.D. Smith, (2004). Search and product differentiation at an internet shopbot. Working Paper. Brynjolfsson, E., M.D. Smith (2000). Frictionless commerce? A comparison of internet and conventional retailers. Management Science 46(4), 563–585. Burdett, K., K.L. Judd (1983). Equilibrium price dispersion. Econometrica 51, 955–969. Butters, G.R. (1977). Equilibrium distributions of sales and advertising prices. Review of Economic Studies 44, 465–491. Carlson, J.A., R.P. McAfee (1983). Discrete equilibrium price dispersion. Journal of Political Economy 91(3), 480–493. Carlson, J.A., D.R. Pescatrice (1980). Persistent price distributions. Journal of Economics and Business 33, 21–27. Carlton, D., J. Chevalier (2001). Free riding and sales strategies for the internet. Journal of Industrial Economics 49(4), 441–461. Chen, S., L.M. Hitt (2002). Measuring switching costs and the determinants of customer retention in internet-enabled business: a study of the online brokerage industry. Information Systems Research 13(3), 255–274. Chevalier, J., A. Goolsbee (2003). Measuring prices and price competition online: Amazon.com and BarnesandNoble.com. Quantitative Marketing and Economics 1, 203–222. Clay, K., R. Krishnan, E. Wolff (2001). Prices and price dispersion on the web: evidence from the online book industry. Journal of Industrial Economics 49, 521–539. Clay, K., R. Krishnan, E. Wolff, D. Fernandes (2003). Retail strategies on the web: price and non-price competition in the online book industry. Journal of Industrial Economics 50(3), 351–367. Clay, K., C.H. Tay (2001). The cross-country price differentials in the online textbook market. Working Paper. Clemons, E.I., L. Hahn, M. Hitt (2002). Price dispersion and differentiation in online travel: an empirical investigation. Management Science 48, 534–549. Dana, J.D. (1994). Learning in an equilibrium search model. International Economic Review 35, 745–771. Daughety, A. (1992). A model of search and shopping by homogeneous consumers without price precommitment by firms. Journal of Economics and Management Strategy 1(3), 455–473. Diamond, P. (1971). A model of price adjustment. Journal of Economic Theory 3, 156–168. Dinlersoz, E.M., H. Li, (2005). The shipping strategies of internet retailers: evidence from internet book retailing. Working Paper. Eckard, E.W. (2004). The ‘Law of One Price’ in 1901. Economic Inquiry 42(1), 101–110. Ellison, G., (2005). Bounded rationality in industrial organization. MIT Working Paper. Ellison, G., S. Ellison, (2004). Search, obfuscation, and price elasticities on the internet. MIT Working Paper. Erevelles, S., E. Rolland, S. Srinivasan. (2001). Are prices really lower on the internet?: an analysis of the vitamin industry. Working Paper. Friberg, R., M. Ganslandt, M. Sandstro¨m (2001). Pricing strategies in e-commerce: bricks vs. clicks. Working Paper.
Ch. 6. Information, Search, and Price Dispersion
373
Gatti, R., P. Kattuman (2003). Online price dispersion within and between seven European countries. Advances in Applied Microeconomics 12, 107–143. Goolsbee, A. (2001). Competition in the computer industry: online versus retail. Journal of Industrial Economics 49(4), 498–499. Grossman, G.M., C. Shapiro (1984). Informative advertising with differentiated products. Review of Economic Studies 51, 63–84. Harrington, J.E. (2001). Comment on ‘reducing buyer search costs: implications for electronic marketplaces’. Management Science 47(12), 1727–1732. Hong, H., M. Shum (2006). Using price distributions to estimate search costs. Rand Journal of Economics 37(2), 258–276. Iyer, G., A. Pazgal (2003). Internet shopping agents: virtual colocation and competition. Marketing Science 22(1), 85–106. Janssen, M., J.L. Moraga-Gonza´lez (2004). Strategic pricing, consumer search, and the number of firms. Review of Economic Studies 71(4), 1089–1118. Janssen, M., J.L. Moraga-Gonza´lez, M. Wildenbeest (2005). Truly costly sequential search and oligopolistic pricing. International Journal of Industrial Organization 23, 451–466. Johnson, E.J., W. Moe, P. Fader, S. Bellman, J. Lohse (2004). On the depth and dynamics of world wide web shopping behavior. Management Science 50(3), 299–308. Johnson, R.N. (2002). Search costs, lags and prices at the pump. Review of Industrial Organization 20, 33–50. Klemperer, P. (1999). Auction theory: a guide to the literature. Journal of Economic Surveys 13, 227–286. Lach, S. (2002). Existence and persistence of price dispersion: an empirical analysis. Review of Economics and Statistics 84(3), 433–444. Lal, R., M. Sarvary (1999). When and how is the internet likely to decrease price competition. Marketing Science 18(4), 485–503. Lal, R., M. Villas-Boas (1998). Price promotions and trade deals with multiproduct retailers. Management Science 44(7), 935–949. Lopez-Acevedo, G. (1997). Quantal response equilibria for posted offer markets. Estudios Econo´micos 12, 95–131. MacMinn, R.D. (1980). Search and market equilibrium. Journal of Political Economy 88(2), 308–327. Marvel, H.P. (1976). The economics of information and retail gasoline price behavior: an empirical analysis. Journal of Political Economy 84, 1033–1080. McAfee, R.P. (1994). Endogenous availability, cartels, and merger in an equilibrium price dispersion. Journal of Economic Theory 62, 24–47. McAfee, R.P. (1995). Multiproduct equilibrium price dispersion. Journal of Economic Theory 67, 83–105. McAfee, R.P., J. McMillan (1988). ’’Search mechanisms’’. Journal of Economic Theory 44, 99–123. McKelvey, R., T. Palfrey (1995). Quantal response equilibria for normal form games. Games and Economic Behavior 10, 6–38. Morgan, J., H. Orzen, M. Sefton (2006). An experimental study of price dispersion. Games and Economic Behavior 54(1), 134–158. Morgan, P., R. Manning (1985). Optimal search. Econometrica 53(4), 923–944. Myerson, R. (1981). Optimal Auctions. Mathematics of Operations Research 6, 58–63. Narasimhan, C. (1988). Competitive Promotional Strategies. Journal of Business 61(4), 427–449. Pan, X., B.T. Ratchford, V. Shankar (2002). Can price dispersion in online markets by explained by differences in e-tailer service quality?. Journal of the Academy of Marketing Science 30(4), 433–445. Pan, X., V. Shankar, B. Ratchford (2003). The evolution of price dispersion in internet retail markets. Advances in Applied Microeconomics 12, 85–105. Png, I.P.L., D. Hirshleifer (1987). Price discrimination through offers to match price. Journal of Business 60(3), 365–384. Png, I.P.L., D. Reitman (1994). Service time competition. Rand Journal of Economics 25(4), 619–634.
374
M.R. Baye et al.
Pratt, J.W., D.A. Wise, R. Zeckhauser (1979). Price differences in almost competitive markets. Quarterly Journal of Economics 93(2), 189–211. Radner, R. (1980). Collusive behavior in oligopolies with long but finite lives. Journal of Economic Theory 22, 136–156. Raju, J.S., V. Srinivasan, R. Lal (1990). The effects of brand loyalty on competitive price promotional strategies. Management Science 36(3), 376–404. Rao, R.C., R.V. Arjunji, B.P.S. Murthi (1995). Game theory and empirical generalizations concerning competitive promotions. Marketing Science 14(3), G89–G100. Rauh, M.T. (1997). A model of temporary search market equilibrium. Journal of Economic Theory 77(1), 128–153. Rauh, M.T. (2001). Heterogeneous beliefs, price dispersion, and welfare improving price controls. Economic Theory 18(1), 577–603. Reinganum, J.F. (1979). A simple model of equilibrium price dispersion. Journal of Political Economy 87, 851–858. Resnick, P., R. Zeckhauser (2002). Trust among strangers in internet transactions: empirical analysis of eBay’s reputation system. Advances in Applied Microeconomics 11, 127–158. Rob, R. (1985). Equilibrium price distributions. Review of Economic Studies 52(3), 487–504. Robert, J., D.O. Stahl (1993). Informative price advertising in a sequential search model. Econometrica 61, 657–686. Roberts, M.J., D. Supina (2000). Output price and markup dispersion in micro data: the roles of producer heterogeneity and noise. Advances in Applied Microeconomics 9, 1–36. Rosenthal, R.W. (1980). A model in which an increase in the number of sellers leads to a higher price. Econometrica 48(6), 1575–1580. Rothschild, M. (1973). Models of market organization with imperfect information: a survey. Journal of Political Economy 81(6), 1283–1308. Rothschild, M. (1974). Searching for the lowest price when the distribution of prices is unknown. Journal of Political Economy 82(4), 689–711. Salop, S.C., J.E. Stiglitz (1977). Bargains and ripoffs: a model of monopolistically competitive price dispersion. Review of Economic Studies 44, 493–510. Scholten, P., A. Smith (2002). Price dispersion then and now: evidence from retail and e-tail markets. in: M.R. Baye (ed.), The Economics of the Internet and E-commerce. Advances in Applied Microeconomics, 11, 63–88. Scott-Morton, F., F. Zettelmeyer, J. Silva-Risso (2001). Internet car retailing. Journal of Industrial Economics 49(4), 501–519. Shilony, Y. (1977). Mixed pricing in oligopoly. Journal of Economic Theory 14, 373–388. Smith, M.D., J. Bailey, E. Brynjolfsson (1999). Understanding digital markets: reviews and assessments, in: E. Brynjolfsson, B. Kahin (eds.), Understanding the Digital Economy, MIT Press, Cambridge, MA. Smith, M.D., E. Brynjolfsson (2001). Consumer decision-making at an internet shopbot: brand still matters. Journal of Industrial Economics 49(4), 541–558. Sorensen, A. (2000). Equilibrium price dispersion in retail markets for prescription drugs. Journal of Political Economy 108(4), 833–850. Spulber, D. (1995). Bertrand competition when rivals’ costs are unknown. Journal of Industrial Economics 43(1), 1–11. Stahl, D.O. (1989). Oligopolistic pricing with sequential consumer search. American Economic Review 79, 700–712. Stahl, D.O. (1994). Oligopolistic pricing and advertising. Journal of Economic Theory 64, 162–177. Stahl, D.O. (1996). Oligopolistic pricing with heterogeneous consumer search. International Journal of Industrial Organization 14, 243–268. Stegeman, M. (1991). Advertising in competitive markets. American Economic Review 81, 210–223. Stigler, G. (1961). The economics of information. Journal of Political Economy 69(3), 213–225.
Ch. 6. Information, Search, and Price Dispersion
375
Tang, F., X. Xing (2001). Will the growth of multi-channel retailing diminish the pricing efficiency of the web?. Journal of Retailing 77, 319–333. Varian, H.R. (1980). A model of sales. American Economic Review 70, 651–659. Villas-Boas, M. (1995). Models of competitive price promotions: some empirical evidence from the coffee and saltine crackers markets. Journal of Economics and Management Strategy 4(1), 85–107. Zettelmeyer, F., F. Scott-Morton, J. Silva-Risso (2004). Cowboys or cowards: why are internet car prices lower? Working Paper.
This page intentionally left blank
Terrence Hendershott, Ed., Handbooks in Information Systems, Vol. 1 Copyright r 2006 by Elsevier B.V.
Chapter 7
Behavior-Based Price Discrimination and Customer Recognition$
Drew Fudenberg Harvard University
J. Miguel Villas-Boas University of California, Berkeley
Abstract When firms are able to recognize their previous customers, they may be able to use their information about the consumers’ past purchases to offer different prices and/or products to consumers with different purchase histories. This article surveys the literature on this ‘‘behavior-based price discrimination.’’
1
Introduction
When firms have information about consumers’ previous purchases, they may be able to use this information to offer different prices and/or products to consumers with different purchase histories. This sort of ‘‘behaviorbased price discrimination’’ (BBPD) and use of ‘‘customer recognition’’ occurs in several markets, such as long-distance telecommunications, mobile telephone service, magazine or newspaper subscriptions, banking services, credit cards, and labor markets; it may become increasingly prevalent with improvements in information technologies and the spread of e-commerce and digital rights management. This article focuses on models of ‘‘pure’’ BBPD, in which past purchases matter only for their information value, and do not directly alter $
NSF grant SES-04-26199 provided financial support for some of this work.
377
378
D. Fudenberg and J. M. Villas-Boas
consumers’ preferences. We do make some comparisons with switching-cost models, where past purchases do have a direct effect, but we say almost nothing about traditional models of third-degree price discrimination, where firms can base their prices on observable and exogenous characteristics of the consumers. One recurrent theme throughout the article is the possibility that firms may face a commitment problem: although having more information helps the firm extract more surplus with its current prices, consumers may anticipate this possibility, and so alter their initial purchases. Thus, as in the related literatures on bargaining, durable-good monopoly, and dynamic mechanism design,1 the seller may be better off if it can commit to ignore information about the buyer’s past decisions. A second theme is that, as in traditional models of third-degree price discrimination,2 more information may lead to more intense competition between firms. Thus, even if each firm would gain by being the only one to practice BBPD, industry profits can fall when all of the firms practice it. Third, and related, firms would often gain from using long-term contracts when they are able to do so as, for example, in the market for cell-phone services. The last implication is somewhat unfortunate from the analyst’s perspective: The welfare implications of BBPD seem to be ambiguous, and to depend on many aspects of the market structure. Section 2 examines a monopoly supplier of a single, non-durable good. We start with a simple two-period model, and then consider the infinitehorizon models of Hart and Tirole (1988) and Schmidt (1993), where all consumers are infinitely lived, and Villas-Boas (2004), where there are overlapping generations of consumers who live only two periods. We compare this situation with that of a durable-goods monopolist. Then we consider the use of long-term contracts, and relate the resulting outcome again to that in models of durable-goods monopoly. We also discuss the case where the consumer’s preferences vary over time, as in Kennan (2001) and Battaglini (2005). Finally, we consider the situation where the monopolist sells more than one good, which we use as a benchmark when studying BBPD with multiple firms; we also compare this with a monopolist seller of two goods in a model of switching costs. Section 3 studies BBPD with two firms, each still selling a single good. In these models, firms can try to ‘‘poach’’ their rivals’ customers by giving new customers special ‘‘introductory’’ prices. We begin with Fudenberg and Tirole’s (2000) analysis of a two-period model of competition in short-term contracts, and its extension by Chen and Zhang (2004) and Esteves (2004) to other distributions of consumer types, where other insights emerge. Next, we discuss Villas-Boas’ (1999) model of poaching in an infinite-horizon 1 See, for example, Stokey (1981), Bulow (1982), Fudenberg and Tirole (1983), Baron and Besanko (1984), Freixas et al. (1985), and Sobel (1991). 2 See, e.g., Thisse and Vives (1988).
Ch. 7. Behavior-Based Price Discrimination and Customer Recognition 379
model with overlapping generations of consumers, each of whom lives only for two periods, where firms cannot distinguish between new consumers and old ones who bought from their rival. We then return to the two-period setting to present Fudenberg and Tirole’s analysis of competition in simple long-term contracts, meaning that firms offer both a ‘‘spot’’ or one-period price and also a long-term commitment to supply the good in both periods. Finally, we compare the predictions of these models to models of switching costs, where past decisions are directly payoff relevant, and may also provide information, as in Chen (1997) and Taylor (2003). Section 4 discusses models where each firm can produce multiple versions of the same product. We begin with Fudenberg and Tirole (1998), and Ellison and Fudenberg (2000), who study the provision of ‘‘upgrades’’ by a monopolist in a setting of vertical differentiation, where all customers agree that one good is better than the other. We then consider the work of Zhang (2005) on endogenous product lines in a Hotelling-style duopoly model of horizontal differentiation. Finally, we discuss the papers of Levinthal and Purohit (1989), Waldman (1996), and Nahm (2004) on the introduction of a new product in models with anonymous consumers and a frictionless second-hand market. Although these papers do not consider behavior-based pricing, the analysis of the anonymous case is an important benchmark for the effects of behavior-based pricing. Section 5 briefly discusses three related topics: privacy, credit markets, and customized pricing. We discuss the work of Taylor (2004a) and Calzolari and Pavan (2005) on consumer privacy. If consumers are not myopic, they will realize that information revelation can reduce their future surplus; in some cases, this can give firms an incentive to try to protect consumer privacy. In credit markets, lenders may learn about the ability of their borrowers, their customers, to repay loans; this information can then be used by the firms in future loans to those customers. In this case what a firm learns about its previous customers relates to the cost of providing the customer with a given contract, as opposed to the customer’s willingness to pay, which is the focus of most of the work we discuss. Our presentation here is based in large part on Dell’Ariccia et al. (1999), and Dell’Ariccia and Marquez (2004); we also discuss Pagano and Jappelli (1993), and Padilla and Pagano (1997, 2000). Finally, for completeness, we briefly present the case of competition when firms already have information about consumer tastes, starting from the initial work of Thisse and Vives (1988). Section 6 presents concluding remarks. 2
Monopoly
We begin with the case of a monopolist who can base prices to its consumers on their past purchase history. For example, in newspaper or magazine subscriptions, firms with market power may offer different rates
380
D. Fudenberg and J. M. Villas-Boas
depending on the consumers’ past purchase behavior.3 We start by considering a two-period model to illustrate some of the effects that can be present, discussing the role of commitment, and of forward-looking consumers. Then, we consider the case of overlapping generations of consumers (Villas-Boas, 2004), and discuss the case when consumers are long lived (Hart and Tirole, 1988; Schmidt, 1993). We consider the effect of long-term contracts and the relationship to the durable-goods and bargaining literature in that setting (Hart and Tirole). We also discuss the case where the consumer’s preferences vary over time, as in Kennan (2001) and Battaglini (2005), who study short-term and long-term contracts, respectively. Finally, we consider the situation where the monopolist sells more than one good, as in Section 5 of Fudenberg and Tirole (2000), which will be an important benchmark case for the next section on competition, and discuss the differences between purely informational behavior-based price discrimination and price discrimination when previous purchases have a direct impact on consumer preferences as in models of switching costs. 2.1 Two-period model Base model Consider a monopolist that produces a non-durable good at zero-marginal cost in each of two periods. A continuum of consumers with mass normalized to one is in the market in each of the two periods. In each period each consumer can use one unit or zero units of the good; no consumer has any additional gain from using more than one unit in each period. The consumer preferences are fixed across the two periods. The consumers’ valuation for the good is represented by a parameter y distributed in the line segment [0,1] with cumulative distribution function F(y) and density f(y). We assume throughout that p[1F( p)] is strictly quasi-concave in p (which is the condition necessary for the existence of a unique local maximum in the static monopoly case). The assumption on the support of the distribution is without loss of generality relative to any compact interval. Hart and Tirole (1988) and Villas-Boas (2004) consider the case of the two-point distribution. Schmidt (1993) considers the case of any discrete number of types.4 Here, we present the case of a continuum of consumer types, and note differences with the two-type case when they arise. In order to obtain some of the sharper results we will sometimes restrict attention to the uniform distribution, with f(y) ¼ 1,8 y. 3 See, for example, ‘‘Publications are Trying New Techniques to Win over Loyal Readers,’’ The New York Times, January 4, 1999, p. C20. 4 We restrict attention to the case in which the consumers are the only parties with private information. It would also be interesting to investigate what happens when the monopolist has also some private information, and the consumers may learn what price offers they will get in the future from the offers made by the firm in the past. From the literature on ‘‘reputation effects’’ we expect that this could allow the firm to obtain higher profits.
Ch. 7. Behavior-Based Price Discrimination and Customer Recognition 381
Each consumer is endowed with the same y in both periods. This valuation y represents the gross utility the consumer enjoys from using the good in one period. Therefore, the net utility per period of a consumer of type y purchasing the good at price p in one period is yp. The lifetime utility of a consumer is the discounted sum of the net utilities of the two periods the consumer is in the market with discount factor dC with 0rdCo1. In the first period, the monopolist chooses one price a to be charged to all consumers (the monopolist cannot distinguish among them, and all consumers prefer a lower price). In the second period, the monopolist chooses two prices: a price ap to be charged to the previous customers of the firm, and a price an to be charged to the consumers that did not buy in the first period, the new customers. The monopolist wants to maximize the expected discounted value of its profits, using a discount factor dF with 0rdFo1. Except where expressly noted we restrict attention to the case in which dF ¼ dC, and then, the discount factor is denoted by d. Given that there is a continuum of consumers, each of them realizes that his decision does not affect the prices charged by the monopolist in the next period. Then a consumer of type y just entering the market decides to buy in the first period if y a þ dC max½y ap ; 0 dC max½y an ; 0: From this inequality one can then obtain directly that given dCo1, if a type y^ chooses to buy in the first period then all the types y4y^ also choose to buy in the first period. That is, the consumers that buy for the first time in the second period value the product by less than any of the consumers that buy in the first period. In order to compute the type of the marginal consumer it is helpful to consider the pricing decision of the monopolist with respect to its previous customers. Define p arg maxp p½1 FðpÞ; the price that maximizes the profit in one period when the consumers do not have any reason to refrain from buying, that is, they buy if their valuation y is greater than the price charged. This is the monopoly price in the static case, or if the monopolist is not able to recognize its previous customers or price differently to them. Denoting y^ as the type of the marginal consumer in the first period, if ^y4p the monopolist sets ap ¼ y: ^ If, on the other hand yop ^ ; the mon ^ opolist sets ap ¼ p : That is, ap ¼ max½y; p ; the marginal consumer in the first period does not get any surplus in second period. This is the ‘‘ratchet effect’’ of the consumers being hurt (i.e., being charged a higher price) by revealing, even if partially, their types (Freixas et al., 1985). The marginal consumer in the first period is then determined by y^ a ¼ dC max½y^ an ; 0, which results in y^ ¼ a
if
a an
(1)
382
D. Fudenberg and J. M. Villas-Boas
a dC an a y^ ¼ 1 dC
if
a4an .
This expression for y^ shows an important aspect of the market dynamics. If prices are expected to increase, each consumer does not have any reason to behave strategically and buys if his valuation is above the current price. If, on the other hand, prices are expected to decrease, some consumers will behave strategically, not being identified in the first period, and being able to get a better deal in the second period. No customer recognition Consider first as a benchmark the case of no customer recognition, in which the monopolist cannot price discriminate in the second period between the consumers that bought, and did not buy, in the first period. The optimal price charged in each period is then p arg max p½1 FðpÞ; generating a profit in each period equal to p ½1 F ðp Þ: Note that, obviously, there is no price variation through time. For the uniform distribution example we have p ¼ 1=2; a profit per period of 1/4, and a total profit of (1+d)/4. Customer recognition and behavior-based price discrimination Consider now the case in which the monopolist is able to recognize the previous customers, as in Hart and Tirole (1988), Schmidt (1993), and Villas-Boas (2004).5 For example, an internet store may be able to recognize returning customers through cookies installed in their computer, and charge them different prices. In this setting, the monopolist can identify in the second period two different groups of consumers: those who have bought in the first period, and those who have not bought in the first period. In the second period the monopolist can charge two different prices. The price paid by the monopolist’s previous consumers, ap, and the price paid by consumers who have not bought previously, an. Given that the marginal consumer buying the product in the first period is ^ ¼ maxðp ; yÞ ^ and ^ the optimal prices in the second period are a ðyÞ y; p ^ ¼ arg maxa an ½F ðyÞ ^ F ðan Þ. an ðyÞ n ^ The marginal consumer in the first period, y^ ¼ yðaÞ; is determined by ^ ^yðaÞ ¼ ða dC a ðyðaÞÞÞ=ð1 d Þ: In order to obtain the optimal first period C n
5 See also Acquisti and Varian (2005) for results focusing on the role of commitment (see below) and the effect of being able to offer enhanced services. The possibility of enhanced services is also covered in Section 4.
Ch. 7. Behavior-Based Price Discrimination and Customer Recognition 383
price, a ; the monopolist then maximizes 8 9 ^ < maxap ap ½1 F ðmax½ap ; yðaÞÞ = ^ þ dF maxa a½1 F ðyðaÞÞ ^ ^ ^ : þa ðyðaÞÞ½F ; ðyðaÞÞ F ða ðyðaÞÞÞ n
(2)
n
where the first term represent the first-period profit, and the second term represents the second-period profit, both from the consumers who bought in the first period and from the consumers who did not buy in the first ^ ; which is satisfied in equilibrium, period. Under the assumption that y4p the first-order condition that defines the optimal a is then ^ a f ðyÞ ^ y^ 0 þ dF y^ 0 ½1 F ðyÞ ^ f ðyÞ ^ y^ þ f ðyÞa ^ ðyÞ ^ ¼ 0. 1 F ðyÞ n
(3)
Note that for dC ¼ dF ¼ d the marginal consumer buying the product in the first period has a higher valuation than if there were no customer recognition. To see this note that the first derivative of the objective function 0 ^ above evaluated at yðaÞ ¼ p is equal to f ðp Þp ½1 ð1 dÞy^ after substituting for 1 F ðp 0Þ p f ðp0 Þ ¼ 0 and p ð1 dÞ ¼ a dan ðp Þ: Given that 0 y^ ¼ 1=ð1 d þ dan Þ and an 40; that derivative is positive, which means that the monopolist should increase a, which implies a higher valuation of the marginal consumer than p : One can also obtain for dC ¼ dF that the ^ F ðyÞ ^ þ da ½1 Fða Þ; which is strictly bepresent value of profits is y½1 n n low the present value of profits under no customer recognition, as p uniquely maximizes p[1F(p)]. The intuition of this result is that the marginal consumers refrain from buying in their first period in the market because they know that they can get a lower price in the next period. This result of lower profits with customer recognition does not hold if the consumers are myopic, while the monopolist is forward looking (or dF large as compared to dC). ^ ¼ y=2; ^ ^ For the uniform distribution example one can obtain an ðyÞ yðaÞ ¼ 2 2a=ð2 dÞ; and a ¼ ð4 d Þ=ð8 þ 2dÞ: One can also easily check that, as argued above, the present value of profits is lower than in the no customer recognition case for all d. One can also get that 2/(4+d) consumers buy in both periods, while (2+d)/(8+2d) consumers only buy in the second period. As consumers become more strategic (greater d) the number of consumers buying in both periods decreases, as the consumers wait for future deals, and consequently, the number of consumers that only buy in the second period increases. The main ideas from these results can also be obtained with a two-type distribution as presented in the references listed above. The role of commitment A crucial feature in the previous section is that the monopolist could not commit in the first period to its second-period price. This led the consumers to refrain from buying in the first period, because the marginal consumers
384
D. Fudenberg and J. M. Villas-Boas
knew that if they bought in the first period they would get zero surplus in the second period. One could then wonder what would happen if the monopolist were able to commit in the first period to its second-period prices. For example, in the market for cellular phone services firms are sometimes able to commit to prices for some future periods. In this case one can then apply the revelation principle, giving incentives for consumer types to reveal themselves in the first period. That is, we suppose that consumers announce their valuations in the first period, and are then assigned a price and a consumption plan for the two periods, such that consumers announce their valuation truthfully. Without commitment, the firm could change the utility (or consumption) a consumer gets in the second period given what the firm learns in the first period. In a durable-good context, Stokey (1979) shows that when firms can commit to the time path of prices, and dC ¼ dF, the monopolist commits to having the same price in all periods, which ends up being the static monopoly price. Hart and Tirole (1988) show that the same conclusion applies when the firm can engage in behavior-based price discrimination: the optimal policy is to forgo the ability to price discriminate and simply charge the static monopoly price in every period.6 Villas-Boas (2004) shows that the result also applies when there are overlapping generations of consumers. To see this in the model presented here, note that if the monopolist is able to commit to the second-period prices for the consumers who bought in the first period, ap, and who did not buy in the first period, an, the most that it can get is ap ½1 F ðap Þ þ dan ½1 F ðan Þ which is maximized when ap ¼ an ¼ p ; with a first-period price a ¼ p ; no price discrimination. Note also that commitment allows the monopolist to be better off. Note that when the monopolist is more forward-looking than the consumers, dF>dC, the firm may then choose to price discriminate, cutting prices through time. 2.2 Overlapping generations of consumers The two-period model above is able to highlight some of the effects under customer recognition and behavior-based price discrimination, but since it focuses on the effects of the beginning of the market (in the first period) and the end of the market (in the second period), it potentially may not get at some of the effects in an on-going market. Consider then a market where there is an infinitely lived monopolist facing overlapping generations of consumers as in the previous section (Villas-Boas, 2004). Each generation lives for two periods, and in each 6 Acquisti and Varian (2005) derive the same result. The result can also be seen as the same as in Baron and Besanko (1984) who show that in a dynamic single-principal single-agent relationship with constant types over time the optimal long-term contract under full-commitment consists in a sequence of staticoptimal contracts.
Ch. 7. Behavior-Based Price Discrimination and Customer Recognition 385
period there are two generations of consumers in the market (each of mass one for a total mass in the market of two in each period), one in its first period in the market, the other in its second and final period in the market. Assume further that 1 F ðpÞ 2pf ðpÞ ¼ 0 has only one solution in the set of real numbers. This last assumption is not necessary, but simplifies the presentation of the results.7 Note first that if the monopolist is not only able to recognize whether a consumer bought in the past, but also his ‘‘age,’’ all the results that we obtained in the previous section (including the equilibrium prices) apply directly, the monopolist charging three prices in each period: one price for the customers that are just arriving into the market; one price for the consumers who are in their second period in the market and bought the product in the previous period; and finally one price for the consumers who are in their second period in the market and did not buy the product in the previous period. However, in many situations, a firm may not be able to recognize a consumer’s ‘‘age,’’ and therefore have to charge the same price to both consumers that are just entering the market and consumers that have been in the market in the previous period, but did not buy the product. Note also that this has the realistic feature of the monopolist knowing more about the consumers that bought the product in the previous period than about the new customers. In terms of the notation of the previous section, not recognizing the customers’ age means that a ¼ an. In order to concentrate on the dynamic effects of customer recognition we focus the analysis on the Markov perfect equilibria (MPE; Fudenberg and Tirole, 1991, p. 513) of this game, i.e., equilibria in which the actions in each period depend only on the payoff-relevant state variables in that period. In this particular game the payoff-relevant state variable in each period is the stock of previous customers of the monopolist in each period. From the analysis in the previous section, we know that in each period the consumers just arriving in the market who buy the product in that period are the ones with the highest valuation. That is, in a period t, the payoff-relevant state variables can be summarized by the type of the marginal consumer entering the market in period t1 who chooses to buy in period t1, denoted by y^ t : The computation of y^ t is exactly as in the previous section. In what follows, let at be the price charged to new customers in period t, and at be the price charged to previous customers in period t. ^ t Þ the marginal consumer purchasing in period t Denoting as y^ tþ1 ¼ yða ^ given price at, and V ðyt Þ the net present value of the monopolist’s profits from period t onwards if the marginal consumer purchasing in period t1 7 This assumption is implied by the condition 3f ðpÞ þ 2pf 0 ðpÞ40 which is satisfied for distributions close to the uniform or the truncated normal with sufficiently large variance.
386
D. Fudenberg and J. M. Villas-Boas
had valuation y^ t we can write the monopolist’s problem as Vðy^ t Þ ¼ maxat at ½1 Fðmax½at ; y^ t Þ ^ y^ t ÞÞÞ þ max½Fðy^ t Þ F ðaðy^ t ÞÞ; 0 þ max ^ aðy^ t Þ½1 F ðyðað aðyt Þ
^ y^ t ÞÞÞ. þ dV ðyðað
ð4Þ
The function aðy^ t Þ is the price to charge the new customers in period t if the marginal consumer purchasing in period t1 has valuation y^ t : The right-hand side of (4) is composed of three terms. The first term is the profit from repeat buyers. The second term is the profit from first-time buyers ^ y^ t ÞÞÞ; or in their second period which are either new in the market, 1 F ðyðað ^ ^ in the market, max½0; Fðyt Þ F ðaðyt ÞÞ: The set of new buyers who are in their second period in the market has only positive measure if aðy^ t Þoy^ t : The third term represents the net present value of profits from the next period onwards. The MPE is then characterized by the functions V ðy^ t Þ; aðy^ t Þ satisfying (4) ^ t Þ ¼ max½at ; ðat daðyða ^ t ÞÞÞ=ð1 dÞ: Note also that ^ t Þ satisfying yða and yða ^ ^ ^ if aðyt Þ yt then aðyt Þ is a constant (the case of y^ t small) because the maximization in (4) is independent of y^ t : This also means that if for a certain y^ t the optimal aðy^ t Þ y^ t then aðxÞ y^ t ; 8x y^ t : If, on the other hand, aðy^ t Þoy^ t then aðy^ t Þ is increasing in y^ t because the objective function is supermodular in y^ t and aðy^ t Þ: No constant prices in equilibrium We now show that the general prices are not constant through time. Suppose that we are in the steady state, with the monopolist charging the price a¯ to the new customers in every period. Then, because prices are not going to decrease and the marginal consumer gets always zero surplus in the second period, all consumers with valuation above a¯ buy in the current ^ aÞ ¼ a¯ : Then, we also know that aðxÞ ¼ a¯ ; 8x a¯ : Let a^ period, that is, yð¯ ^ 2af ^ ðaÞ ^ ¼ 0; and note that aop ^ be defined by 1 F ðaÞ : ^ a small price cut da from a¯ attracts all consumers with valuation If a¯ 4a; y a¯ da; and the effect on the present value of profits is f1 F ð¯aÞ 2¯af ð¯aÞ þ d min½1 Fð¯aÞ a¯ f ð¯aÞ; 0gda; which is always positive. Then, a¯ 4a^ cannot be an equilibrium. The intuition is that if the candidate constant price is not low enough the monopolist gains from cutting prices in the next period to attract the consumers of the older generation that have a lower valuation for the good. ^ and a deviation where the monopolist chooses in the Consider now a¯ a; current period t, at ¼ d¯a þ ð1 dÞp ; followed by atþ1 ¼ a¯ : That is, in period t the monopolist charges a price above a¯ and in period t+1 returns to the equilibrium price a¯ : Once a¯ is charged, the consumers believe that no lower price is charged in the future, and all the consumers with the valuation above a¯ buy the product. In period t, under the deviation, the
Ch. 7. Behavior-Based Price Discrimination and Customer Recognition 387
marginal consumer buying the product can be computed to be y^ tþ1 ¼ p : The present value of profits from this deviation is then p ½1 F ðp Þð1 þ dÞ þ ½d¯a þ ð1 dÞp ½1 Fðp Þ þ d¯a½1 F ð¯aÞ þ F ðp Þ Fð¯aÞ þ d2 V ð¯aÞ, while the equilibrium present value of profits can be represented by p ½1 F ðp Þð1 þ dÞ þ a¯ ½1 Fð¯aÞð1 þ dÞ þ d2 V ð¯aÞ: The difference between the former and the latter can then be obtained to be ð1 dÞfp ½1 Fðp Þ ¯a½1 F ð¯aÞg, which is greater than zero because p maximizes p[1F(p)]. Then, this deviation is profitable and the monopolist charging always a¯ oa^ cannot also be an equilibrium. That is, if the monopolist charges a sufficiently low price that it does not have the incentive to cut prices in the next period (to attract the consumers of the older generation that have a lower valuation for the good) then it would gain from deviating and charging a high price for one period in order to identify the consumers that value more the good in the incoming generation. This shows that there are going to be price fluctuations in any MPE.8 Let us briefly note that if the analysis is not restricted to MPE one can obtain subgame perfect equilibria in which prices are constant through time (as in Ausubel and Deneckere, 1992) at the level obtained when future price commitments are possible, which is also the case with no customer recognition. In such a case, a deviation by the monopolist is ‘‘punished’’ with the equilibrium path in the MPE. Price cycles in equilibrium Let us now present an equilibrium with price cycles, for the particular case where f(y) ¼ 1, the uniform case. We restrict attention to smooth equilibria—equilibria where in steady state the prices being chosen by the monopolist result from the maximization of a smooth concave function. As noted below, there are equilibria of this type if d is sufficiently small.9 In the steady state, the monopolist alternates between high and low prices for the new customers, denoted by ah and a‘ ; respectively. If in period t the marginal consumer from the previous generation, y^ t ; is high, the monopolist charges a low price in order to attract not only the new generation consumers but also the old generation consumers who did not buy in the previous period. If, on the other hand, in period t the marginal consumer from the previous generation, y^ t ; is low, the monopolist charges a 8 Villas-Boas (2004) shows that this same argument also goes through in a two-type distribution for some parameter values. However, because in a two-type distribution continuous deviations may not be possible, under some parameter values there are equilibria with constant prices through time. 9 When d-0 all the equilibria converge to the equilibrium presented here.
388
D. Fudenberg and J. M. Villas-Boas
high price targeted only at the new generation of consumers. In this case, we can see that V ða‘ Þ ¼ V ‘ is independent of a‘ : One can then obtain that for d small there is an MPE where the behavior of the game settles in steady state into a price cycle alternating between ðat ¼ ð8 d2 Þ=ð16 þ 2dÞ; at ¼ 1=2Þ and ðat ¼ ð6 þ dÞ=ð16 þ 2dÞ; at ¼ ð4 þ dÞ=ð8 þ dÞÞ: The prices for the new customers are always lower than the prices to the previous customers. However, both prices fluctuate in opposite directions: the price for the new customers is high when the price for the previous customers is low, and vice versa. The monopolist charges a high price to the new customers when it had in the previous period a high demand of new customers. Then, it has relatively small demand from new customers of 4/(8+d) (all from the new generation), and a large demand from the previous customers, 1/2. In the next period, the monopolist charges a low price to the new customers attracting all the customers from the new generation that have a valuation higher than the price being charged (with mass (10+d)/(16+2d)), plus the consumers from the previous generation that waited for the low price in this period, with mass (2+d)/(16+2d), for a total demand of new customers of (6+d)/ (8+d). The demand from the previous customers is equal to all the new customers of the previous generation, 4/(8+d). Profits in each of the alternating periods can also be immediately obtained.10 It is also interesting to check the effect of the discount factor on prices, demands, and profits. In the periods in which the monopolist charges a high price to the new customers, an increase in the discount factor decreases that price, the demand from new customers, and therefore profits from new customers. In the periods in which the monopolist charges a low price to the new customers, an increase in the discount factor increases that price, the price to the previous customers, the demand from new customers, and profits from new customers, and decreases the demand from the previous customers. The average per period profit decreases with an increase in the discount factor. An increase in the discount factor makes the customers more willing to wait for price cuts. This means that in periods in which the monopolist charges a high price to new customers, the monopolist has less overall demand, which makes it lower its price, and results in lower profits. Given that the marginal customer buying the product has now a greater valuation, 10 The condition on d being small is important because if d were high, more consumers would wait for the lower prices in the future, which means that there is less advantage for the monopolist to charge a high price. That is, if d were high, after charging supposedly the lowest price (in steady state), (6+d)/ (16+2d), the monopolist would gain from cutting the price even further (and ‘‘surprising’’ some of its previous customers). One can check that if do1/2 there is no such profitable deviation. One can also check that when d is high there is an equilibrium with prices alternating between high and low prices for the new customers, with similar properties to the ones of the equilibrium presented here, and where the low price is such that the monopolist does not want to cut the price even further (for d-1 the low price converges to 1/3).
Ch. 7. Behavior-Based Price Discrimination and Customer Recognition 389
in the next period the profits are greater, and the monopolist chooses to charge a greater price to the new customers. However, if one computes the effect of a higher discount factor on the normalized discounted profit (the constant profit that would yield the same net present value of profits), one finds that profits decrease in the discount factor. This is because with a higher discount factor, consumers are ‘‘more strategic’’, i.e., in the periods in which the monopolist charges a high price more consumers refrain from buying. It is also interesting to compare the equilibrium profits with the case in which the monopolist is not able to recognize its customers from the current period on. One can then obtain, as in the previous section, that the average per period profit without customer recognition is higher than if the monopolist were able to recognize its customers. Comparing the equilibrium profits with the case in which the monopolist is able to recognize both the previous customers and the consumers’ age one obtains that the monopolist is hurt by being able to recognize the consumers’ age in addition to recognizing its previous customers. The result is interesting because it reinforces the idea that the monopolist having more information (in this case the consumers’ age) ends up hurting the monopolist. The intuition is that when the monopolist recognizes the consumers’ age in the market, it offers an even lower price to the consumers that do not buy the product in their first period in the market, which makes consumers refrain even more from buying in the first period. 2.3 Long-lived consumers The longer consumers are in the market, the more information they potentially can give about their preferences through their decisions to buy or not to buy at different prices. This means that the firm’s policy with respect to its previous customers is exponentially more complicated with the number of periods that a consumer has been in the market. Hart and Tirole (1988) consider the perfect Bayesian equilibrium of this case of long-lived consumers with a two-type distribution, {y1,y2} with y1oy2, and only one generation of consumers. They find that in equilibrium, if d>1/2, there is no price discrimination when the horizon tends to infinity, with the monopolist always charging the low price (the valuation of the low type). The intuition for this result is that if a high-valuation consumer y2 were to buy the product at a higher price, he would reveal that he has high valuation and will have zero surplus from that period onwards. If there were a price strictly above the lowest valuation y1 for which the high-valuation consumer would buy the product with positive probability (such that after that price, if there were no purchase, the monopolist would charge a price y1 forever), a high-valuation consumer buying the product would be better off deviating, not buying the product, and getting a low-valuation price from then on. By buying the product the high-valuation consumer would get a
390
D. Fudenberg and J. M. Villas-Boas
surplus of at most y2y1, while if the high-valuation consumer waited for one period (and not be identified as a high valuation consumer) he would get a surplus approaching ½d=ð1 dÞðy2 y1 Þ; which is greater than y2y1 for d>1/2. Schmidt (1993) considers the case with any discrete number of consumer types,11 fy1 ; y2 ; . . . ; yn g with y1 oy2 o oyn ; while restricting attention to the MPE.12 He finds, as in Hart and Tirole, that, for d>1/2, there is no price discrimination when the horizon tends to infinity, with the monopolist always charging the low price y1 (the valuation of the low type). The method of proof used in Schmidt allows us to better understand the relation of this result with the general results on reputation (e.g., Kreps et al., 1982; Fudenberg and Levine, 1989). The proof is similar to the one in Fudenberg and Levine (1989) on the reputation of a long-term player facing a sequence of short-term players. Schmidt first shows that if there is a price strictly above y1 on the equilibrium path, then there is a strictly positive minimum probability of that price being accepted and revealing a consumer type with valuation strictly above y1. He then shows that because types y>y1 can build a reputation for being of type y1, they will do so. That is, the no-discrimination equilibrium can be seen as a response of the monopolist to the consumers’ threat to build a reputation that they have the lowest valuation for the product if the price is above y1. In Fudenberg and Levine’s model, the type that a consumer would like to be seen as is type y1. Given the greater structure of the game considered here (in comparison to the general class of games considered in Fudenberg and Levine), Schmidt is able to extend the results of Fudenberg and Levine to the case of two longrun players, and characterize the equilibrium actions (while Fudenberg and Levine only characterize payoffs). Schmidt looks at a long finite horizon game using backward induction, which is what allows him to show that y1 acts like a Fudenberg-Levine ‘‘commitment type’’ and rejects all prices above y1. Kennan (2001) considers the case in which the consumer types can change randomly through time, but with positive serial correlation. He then finds that we can then have stochastic price cycles because (no) purchases indicate a high- (low-) consumer valuation and are followed by a high (low) price. It is interesting to discuss in this context of long-lived consumers what happens if the firm is allowed to offer long-term contracts, and the relationship of behavior-based price discrimination with the results from the durable-goods and bargaining literatures. 11 He considers that it is the monopolist that is the party that has private information (on her costs). We present here the result in terms of private information of the consumers. 12 The Markov assumption is necessary for the case of any n to guarantee that the continuation payoffs are the same for a price equal or below y1 (with n ¼ 2 this can be shown without the Markov assumption).
Ch. 7. Behavior-Based Price Discrimination and Customer Recognition 391
Long-term contracts Suppose that the firm would be able to offer a contract to a customer committing itself to a sequence of prices for the future to be charged to that consumer. Note that the effect of this possibility is that a consumer would know now that he would not be taken advantage of in the future for revealing his high valuation. Hart and Tirole (1988) consider this situation with the possibility of renegotiation, such that the firm might be able to offer different contracts in the future.13 For example, in the market for cellular phone service firms can offer long-term contracts, and can change which long-term contracts to offer in the future. Hart and Tirole show that in such a setting with two consumer types, the firm might now be able to sell to the high-valuation consumers at a price above the lowest price. The intuition is that with a long-term contract the monopolist has greater ability to price discriminate. It can get the high-valuation consumer to buy the product at an average price per period above the lowest price (low type valuation), because it commits to this average price with a long-term contract. For example, if the monopolist offers a long-term contract at an average per-period price p>y1, the surplus for the high-valuation consumer if he accepts the contract is ðy2 pÞ=ð1 dÞ: If this consumer decides not to buy in this period, the most the consumer is able to get is dðy2 y1 Þ=ð1 dÞ; if the monopolist offers in the next period a contract with an average perperiod price of y1 (the monopolist will never offer a lower average perperiod price). Then, if p ¼ dy1 þ ð1 dÞy2 the high-valuation consumers accept the contract, and the monopolist is able to sell to such consumer at a price strictly above y1. As shown in Hart and Tirole, the equilibrium longterm contract is for the monopolist to offer a contract in a number of initial periods with average per-period price strictly above dy1 þ ð1 dÞy2 ; such that type y2 randomizes between accepting and not accepting the contract, and then, after a certain number of periods, the monopolist offers a contract with average per-period price y1, and both types accept the contract. However, this possibility of selling to the high-valuation consumers with an average per-period price strictly above y1 is not possible without a longterm contract. Without a long-term contract a high-valuation consumer gets zero surplus after revealing his type, and therefore, must be offered a price below the low-type valuation to accept buying.14 But then the lowvaluation consumer would also buy the product, and, therefore, no information would actually be revealed about the type of the customer buying the product. Hart and Tirole then show that, because of this greater ability
13 Laffont and Tirole (1990) consider a two-period version of such contracts with continuous consumption per period in the context of procurement. 14 One can see this as a high-valuation consumer maintaining the reputation that he may have a low valuation. See the discussion above.
392
D. Fudenberg and J. M. Villas-Boas
to price discriminate a firm is better off when it has the ability to offer a long-term contract. It turns out that this effect of long-term contracts does not occur if the consumer lives only for two periods, with the second period being the last period. In the two-period model presented above it turns out that the introduction of long-term contracts does not have any effect, and the equilibrium with long-term contract is exactly the same as the equilibrium without long-term contracts. This is because the zero surplus obtained by the marginal consumer after revealing his type only lasts for one period. Battaglini (2005) considers the case of infinitely lived consumers where the preferences change through time following a Markov process, as in Kennan (2001), but allowing for continuous consumption. A consumer’s per-period utility in period t is ytqp, for q units bought at price p. The monopolist’s cost of selling q units is ð1=2Þq2 : For future reference, note that the efficient quantity to be sold in period t is qe ðyt Þ ¼ yt : The marginal benefit yt in period t is private information of the consumer, can only take ¯ with y4 ¯ y; and evolves through according to a one of two values, fy; yg; Markov process. The transition probabilities between states are in (0,1), and are denoted by Prðytþ1 jyt Þ: Types are assumed to be positively corre¯ yÞ ¯ Prðyj ¯ yÞ and Prðy j yÞ Prðy jyÞ: ¯ At date 0 the lated over time, Prðyj ¯ monopolist has a prior m that the consumer’s type is y and a prior 1m that the consumer’s type is y : Battaglini computes the optimal long-term contract. First, he shows that under commitment the optimal contract always involves the efficient quantity being supplied if in the history of the relationship (including the current period) there has been a period in which the marginal benefit has been equal ¯ That is, with varying types we have the result that a long-term contract to y: supply is at the efficient level in finite time (which is not the case for fixed types). The intuition for this result has to do with the role of the quantity distortions in the contract. Distortions are introduced only to extract more surplus from higher types, and therefore, there is no reason not to offer the highest type the efficient quantity. After any history the rent that must be paid to a high type to reveal himself is independent of the future quantities. That is, the monopolist is the residual claimant on the surplus generated on histories after a high-type report, and therefore the quantities that follow such report are the efficient ones. In addition, Battaglini finds that if the ¯ the quantity history has never had a period where the buyer had type y; distortion vanishes through time as the initial state has less and less information about the current buyer’s type. Battaglini then considers the case in which the contract can be renegotiated, and shows that under general conditions the contract with commitment is renegotiation-proof, and when these conditions fail, the contract is renegotiation-proof after a finite amount of time. Battaglini’s analysis relies heavily on the assumption that there are only two types. As noted in the paper, with n types, then the conditional
Ch. 7. Behavior-Based Price Discrimination and Customer Recognition 393
distribution for each type is represented by a n1 vector, each type has n1 characteristics, and we would need to solve a multidimensional-screening problem. It would be interesting to investigate further this n-type case, even if putting some structure on the conditional distribution for each type. Relationship to durable goods and bargaining The strategic behavior of consumers when firms practice behavior-based price discrimination is related to settings where a monopolist sells a durable-goods, or to settings where two parties bargain, and in which one of the parties has private information. Here, we first briefly discuss some of the forces present in a market where a monopolist sells a durable good, or in a bargaining situation between two parties, in which one party has private information. Then, we relate the durable-goods setting with the behavior-based price discrimination model. (For some discussion of the durable-good monopoly literature see, for example, Chapter 1.5 in Tirole, 1988. For a survey of the bargaining literature see, for example, Chapter 10 in Fudenberg and Tirole, 1991.)15 Durable goods and bargaining. Consider the two-period model above, but suppose now that the monopolist sells a product in the first period that lasts for the two periods. Let A be the price of the durable in the first period. Denoting y^ as the type of the marginal consumer in the first period, the surplus of this consumer is ð1 þ dC Þy^ A when buying in the first period, and is dC ðy^ an Þ if waiting for the second period. The marginal consumer in the first period is then determined by y^ ¼ A dC an
(5)
Given that the marginal consumer buying the product in the first period ^ the optimal price in the second period is a ðyÞ ^ ¼ arg maxa an ½F ðyÞ ^ is y; n n ^ ^ F ðan Þ: Using this we then have that (5) defines y as a function of A, yðAÞ: In order to obtain the optimal first period price, A, the monopolist then maximizes ^ ^ ^ ^ maxA Að1 FðyðAÞÞÞ þ dF an ðyðAÞÞ½F ðyðAÞÞ F ðan ðyðAÞÞÞ
(6)
where the first term represent the first-period profit, and the second term represents the second-period profit. The first order condition that defines the optimal A is then ^ y^ 0 þ dF y^ 0 f ðyÞa ^ ðyÞ ^ ¼ 0. ^ A f ðyÞ 1 F ðyÞ n
(7)
Note that for dC ¼ dF the marginal consumer buying the product in the first period has a higher valuation than if the firms were selling a 15 For early work on the durable-goods monopolist problem see, for example, Stokey (1981) and Bulow (1982).
394
D. Fudenberg and J. M. Villas-Boas
non-durable good. To see this note that the first derivative of the objective 0 ^ function above evaluated at yðAÞ ¼ p is equal to f ðp Þp ½1 y^ after substituting for 1 F ðp Þ p f ðp Þ ¼ 0 and p ¼ A dan ðp Þ: Given that ^y0 ¼ 1=ð1 þ da0 Þ and a0 40; that derivative is positive, which means that n n the monopolist should increase A, which implies a higher valuation of the marginal consumer than p : One can also obtain for dC ¼ dF that the ^ F ðyÞ ^ þ da ½1 Fða Þ; which is strictly present value of profits is y½1 n n below the present value of profits under no customer recognition, as p uniquely maximizes p½1 F ðpÞ: The intuition of this result is that the marginal consumers refrain from buying in their first period in the market because they know that they can get a lower price in the next period. ^ ¼ y=2; ^ ^ For the uniform distribution example one can obtain an ðyÞ yðAÞ ¼ 2 2A=ð2 þ dÞ; and A ¼ ð2 þ dÞ =ð8 þ 2dÞ: One can also get that 2/(4+d) consumers buy in the first period, while (2+d)/(8+2d) consumers buy in the second period. The model above can also represent a bargaining situation where there is a single buyer, and if the buyer does not take the first offer A then he is offered an in the second period. In such a setting one can then obtain that private information of the buyers leads to an inefficient outcome for some consumer types (if rejection occurs in the first period). In a durable-goods setting, if new generations of consumers come into the market in every period, there are incentives for the monopolist to raise its price in order to try to extract more surplus from the consumers who have a high valuation and who have entered the market most recently. This can then generate price cycles in which prices come down to clear the demand from low-valuation consumers, and then go up to better extract the surplus from the consumers with high valuation who just entered the market. This setting is analyzed in Conlisk et al. (1984), and Sobel (1984, 1991). Although having the flavor of the results in Subsection 2.2 for overlapping generations of consumers and behavior-based price discrimination, and as also discussed below, the results are different, as we can have price cycles in the behavior-based price discrimination model, but constant prices in the corresponding durable goods model. In some situations the monopolist may also have some private information regarding its costs, so that the price offers can potentially reveal some information about the monopolist’s costs. Fudenberg and Tirole (1983), with a bargaining set-up, characterize the set of equilibria in two-period games when the monopolist and the buyer each have two potential types (two-sided incomplete information). They show that this additional private information may lead to a continuum of perfect Bayesian equilibria. Ausubel and Deneckere (1992) consider the infinite horizon version of this two-sided incomplete information model, and show that we may have (stationary) equilibria in which prices stay high, and the seller tries to maintain a reputation of having a high cost.
Ch. 7. Behavior-Based Price Discrimination and Customer Recognition 395
Relationship of durable goods to behavior-based price discrimination. When a monopolist is selling a durable good through time, consumers refrain from buying in the initial periods because they foresee that the monopolist may cut its price in future periods. In such a setting consumers may prefer to forsake the benefits of the product if buying earlier, with the lower price if buying later. On the other hand, with customer recognition and behaviorbased price discrimination for a non-durable, consumers refrain from buying in the initial periods because they foresee that the monopolist may cut its price in future periods, and therefore they will be identified as lower valuation consumers and get lower prices for the future. This difference between durable-goods and behavior-based price discrimination for nondurables leads to different consumer surplus effects from purchasing the product, and therefore different market implications. When buying a durable-good the consumer pays a price and gets a benefit of using the product for the duration of the product’s life. Consumers for whom the present value of future benefits is greater than that price may be willing to purchase the product. However, under customer recognition and behavior-based price discrimination for a non-durable good, the marginal consumer buying the product pays the price and gets a benefit in the current period, but then gets zero surplus in all future periods. Therefore, in order for a consumer to be willing to buy, the initial price must be so low, that even consumers with very low valuation may be willing to buy the product. For an infinite horizon with two types, Hart and Tirole (1988) show then that the durable-good case is better for the monopolist than the case of a non-durable with the ability to recognize customers and price discriminate according to past behavior (Hart and Tirole consider this possibility in terms of rental of the durable good). In the long-lived consumers with two consumer types model that they consider, Hart and Tirole also find that the durable-good case is exactly the same as when the monopolist can offer long-term contracts (and different from short-term contracts), as the separation between types can be done exante. In the two-period model considered above the durable-good case results in exactly the same outcome as the long-term contract case, and generates exactly the same outcome as the customer recognition case for a short-term sales of a non-durable, or equivalently with short-term rentals of a durable. This is because, in a two-period model, the consumer surplus effects of purchasing a durable-good are the same as purchasing a nondurable with customer recognition, as the zero surplus of the marginal consumers under customer recognition lasts only for one period. In the case of overlapping generations of consumers, with consumers living for two periods (and without the ability to recognize the customer’s age) selling a durable good may not generate price cycles, as selling the durable good for a consumer that only uses the product for one period requires a much lower price than selling the durable good for a consumer
396
D. Fudenberg and J. M. Villas-Boas
who uses the product for two periods (Villas-Boas, 2004). That is, with overlapping generations of consumers, selling a durable good does not yield the same outcome as selling a non-durable with customer recognition (with or without long-term contracts). Thus, in general, the sale of a durable good is not the same as a sequence of short-term rentals. Although the price falls over time, the price a consumer faces is not based directly on its own past behavior. Loosely speaking, the commitment involved in selling a durable good lets the monopolist commit to not use behavior-based pricing. 2.4 Two-good monopoly In order to serve as an introduction to the next section on competition, and to serve as a benchmark, consider now the case of a monopoly selling two goods, A and B. The presentation here follows closely part of Section 5 of Fudenberg and Tirole (FT, 2000). To focus on the interaction between the two goods we set up preferences such that consumers buy a unit of one of the goods in every period. Indexing the relative preferences for B over A as y, let the valuation per period of a consumer of type y be vy/2 if the consumer buys good A, and v+y/2 if the consumer buys good ¯ where y ¼ yo0; ¯ B, with v ‘‘large’’ and y distributed in ½y; y; with cumulative distribution function F(y), strictly positive density f(y), and F(y) is symmetric about zero and satisfies the monotone hazard rate (MHR) condition that f(y)/[1F(y)] is strictly increasing in y. The parameter v is assumed large, such that the monopolist chooses prices such that all consumers buy one unit of one of the goods in every period. For this reason, we consider that the monopolist’s production will be constant across the pricing regimes , so that the costs of production are a constant that can be ignored. Let a and b be the prices charged in the first period for products A and B, respectively, a and b be the prices charged in the second period for products A and B, respectively, for consumers who bought the same product in the previous period, and a^ and b^ be the prices charged in the second period for products A and B, respectively, for consumers that bought a different product in the previous period. Consider first the case in which long-term contracts are not available. Then the firm will charge a ¼ a ¼ b ¼ b ¼ v, for a present value of profits of (1+d)v. Note that consumers do not switch products. Consider now the case in which long-term contracts are available (with commitment not to renegotiate). Then, in the first period the monopolist can offer four product consumption experiences: product A in both the periods, product A followed by product B, product B followed by product A, and product B in both periods. By symmetry it is enough to analyze the interval ½y; 0: As argued in Fudenberg and Tirole, incentive compatibility
Ch. 7. Behavior-Based Price Discrimination and Customer Recognition 397
^ choose product A in both requires that consumers in an interval ½y; y ^ 0 choose periods (which we call AA), and consumers in the interval ½y; product A followed by product B (which we call AB). In order for the type 0 to be indifferent between buying and not buying, and between the ‘‘switching’’ bundles AB and BA, it must be that the price for each of these bundles is (1+d)v. Indifference for type y^ between bundles AA and AB requires that ^ Thus, it is as if the the price of AA be above the price of AB by dy: monopolist first sold all consumers a ‘‘switching’’ bundle at price ð1 þ dÞv; ^ The and then offered an ‘‘upgrade’’ to AA or BB for a premium of dy: ^ ^ present value of profits is then ð1 þ dÞv 2dyF ðyÞ; where the optimal y^ ^ þ yf ^ ðyÞ ^ ¼ 0: Note that the optimum has some consumers satisfies F ðyÞ switching products across periods. Since consumer preferences are the same in both periods, this switching is inefficient; it is used to extract rents for the privilege of not switching. For the uniform distribution one can obtain y^ ¼ y =2; so that one half of the consumers switch products from the first to the second period. Fudenberg and Tirole also show that the monopolist can do better than the above deterministic menu, with a randomized menu where consumers in ^ y ^ get a (1/2, 1/2) randomization between products A and B. This ½y; allows the monopolist to extract a greater surplus from the consumers that get no ‘‘switching’’. Again, as in the deterministic menu, we have some inefficient (stochastic) switching by some consumers. When we reach the second period, as stated above, ‘‘switching’’ consumers are consuming a product that is not the best for them. This means that there are gains to be made from the monopolist renegotiating the product that is offered to those consumers. This renegotiation may then affect the choices of consumers in the first period (and the monopolist’s offers). It would be interesting to investigate whether we would still have inefficient switching in equilibrium if the monopolist can offer long-term contracts subject to renegotiation. In order to compare also with the next section consider now the case of switching costs, where a monopolist sells two products in each of two successive periods, consumers have the same valuation v for each product per period and incur in a cost s if they change products from the first to the second period. It is clear that in this situation the best the monopolist can do is extract v per period per consumer (with a price equal to v), and there is no switching products from the first to the second period. This can be accomplished either with short or long-term contracts. Consider now in the model above (with heterogeneous consumers), the role of the introduction of switching costs s (suppose s small). The price of the switching bundles can then be at most ð1 þ dÞv ds and indifference for type y^ between bundles AA and AB requires that the price of AA be above ^ Thus, comparing with the no switching costs the price of AB by dðs yÞ: case, the price of the switching bundle is now lower, but the premium to upgrade to the non-switching bundle became now greater. The present
398
D. Fudenberg and J. M. Villas-Boas
^ ðyÞ ^ ds½1 2F ðyÞ; ^ where the optivalue of profits is now ð1 þ dÞv 2dyF ^ ^ ^ ^ mal y satisfies FðyÞ þ ðy sÞf ðyÞ ¼ 0: Note that the optimum has some consumers switching products across periods, but the number of switching consumers is decreasing in the switching cost s. Note that switching also is inefficient, and that the monopolist profit and welfare are decreasing in the switching cost s. For the uniform distribution one can obtain y^ ¼ ðy þsÞ=2; so that less than one half of the consumers switch products from the first to the second period. 3
Competition
Several new issues arise in models of behavior-based price discrimination with multiple firms. Starting with the most obvious, firms can try to ‘‘poach’’ their rivals’ customers by giving them special ‘‘introductory’’ prices.16 This raises the questions of how much switching we should expect to occur, and of its efficiency consequences. At a more theoretical level, we have already seen that in equilibrium a monopolist without commitment power can be made worse off by the ability to condition the price it charges a customer on that customer’s past decisions, because consumers will foresee this condition and adjust their earlier behavior. The same sort of foresight can operate in models with multiple firms, but now its impact on profit is a priori ambiguous, because of the interactions between the customers’ behavior (basically the elasticity of demand) and the equilibrium in the pricing decisions of the firms. Furthermore, while a monopolist with commitment power can always do at least as well when behavior-based discrimination is possible (by committing itself to ignore past behavior in setting prices), a group of oligopolists with commitment power can all be worse off if all of them become able to discriminate based on past customer behavior, as the better information may lead to more intense price competition (see Subsection 5.3 below).17 For this reason, while each firm has a dynamic incentive to adjust its prices so that it learns more about the consumers and can better segment the market, the firm also has an incentive to reduce the information that is obtained by its rivals. The way that these various effects combine to determine equilibrium prices and allocations depends on the nature of preferences and on the form of market competition. The first part of this section considers Fudenberg and Tirole (FT, 2000)’s analysis of a two-period model of competition in short-term contracts, and some variations on the distributions of consumer types studied by Chen and Zhang (2004) and Esteves (2004). The second 16 In 1994, about 20% of all U.S. households changed their provider of long-distance telephone services (Schwartz, 1997). 17 Of course, a single oligopolist with commitment power who is given the ability to condition prices on customer history cannot be made worse off, provided that none of the other firms are allowed to have this ability.
Ch. 7. Behavior-Based Price Discrimination and Customer Recognition 399
part discusses Villas-Boas’ (1999) extension of the two-period model to an infinite horizon with overlapping generations of consumers, each of whom lives only for two periods. We then return to the two-period setting to present Fudenberg and Tirole’s analysis of competition in simple long-term contracts, meaning that firms offer both a ‘‘spot’’ or one-period price and also a long-term commitment to supply the good in both periods. Finally, we compare the predictions of these models of ‘‘pure price discrimination,’’ where the past matters only for the information it provides about preferences, to models of switching costs, where past decisions are directly payoff relevant, and may also provide information, as in Chen (1997) and Taylor (2003). 3.1 Two periods, short-term contracts Following FT, suppose that there are two firms, A and B, who produce non-durable goods A and B, respectively, at constant marginal cost c. There are two periods, 1 and 2; each period a consumer can either consume a unit of good A or a unit of good B or neither, but not both. There is a continuum of consumers, whose preferences are quasi-linear in money and are ¯ where y ¼ yo0: ¯ indexed by y 2 ½y; y; The consumption utility from goods A and B is vy/2 and v+y/2, respectively, so that y measures the consumer’s preference for good B over good A. There is a known distribution F on y, which is assumed to be symmetric about 0. Fudenberg and Tirole assume that F is smooth, with density f, and that F is symmetric and that it satisfies the MHR property that f ðyÞ=½1 F ðyÞ is strictly increasing in y; their sharpest results are for the special case of the uniform distribution. Esteves (2004) considers the case where F has a two-point support;18 Chen ¯ and Zhang (2004) assume that F is concentrated on the three points y; 0; y: Fudenberg and Tirole assume that all agents use a common discount factor d; the other papers suppose that firms use discount factor dF while consumers use the possibly different discount factor dC. With simple short-term contracts, and no commitment power, each firm will offer a single first-period price, which we denote a and b, respectively. In the second period, each firm can offer two prices, one to its own past customers and another price to all others. (We will assume that the reservation value is high enough that all consumers purchase in the first period, so that consumers who didn’t purchase from firm A must have purchased from firm B.19) Note that if firms do not observe the identities of their customers, there is no link between the periods, and the equilibrium 18 Esteves supposes that the two mass points are in the interval [0,1], symmetric about the point 1/2; to map her notation to ours suppose that the mass points are at yA ¼ t(2xA1) and yB ¼ yA, and that v ¼ v0 tð1 xA Þ; where v0 is the reservation value in her notation. 19 Chen and Zhang consider an extension of their model to the case where consumers with y ¼ 0 have lower reservation values; in this case not all consumers purchase in the first period.
400
D. Fudenberg and J. M. Villas-Boas
Fig. 1.
Second-period competition given the first period cut-off.
reduces to two repetitions of the static equilibrium. Our question is how the prices and efficiency of the equilibrium with short-term contracts and customer poaching compare to that of the static benchmark. Under FT’s assumptions, the static one-period problem is well behaved: each firm’s objective function, pi ¼ F ðpj pi Þðpi cÞ; is strictly quasi-concave, so that firms are never willing to randomize, and the game has a unique equilibrium, namely F ð0Þ þ c. f ð0Þ In the case of a uniform distribution, this simplifies to p ¼ c þ ðy¯ yÞ ¯ so that each firm’s profit is y=2: ¯ =2 ¼ c þ y; Moreover, in the uniform case the dynamic equilibrium is also in pure strategies, and can be characterized with first-order conditions. With the discrete supports specified in the other two papers, the static equilibrium is in mixed strategies, which makes the calculations more complex and the intuition more subtle. For this reason we use the FT case for exposition, and try to explain informally the effects of the other distributional assumptions. pA ¼ pB ¼
Analysis of the two-period model under the MHR assumption A standard argument shows that at any pair of first-period prices such that all consumers purchase and both firms have positive sales, there is a cut-off y such that all consumers with types yoy purchase from firm A in the first period.20 Given this cut-off, the second period game is as depicted in Fig. 1: consumers to the left of y lie in ‘‘firm A’s turf’’ and the consumers on the right lie in firm B’s. On firm A’s turf, firm A offers price a, while firm ^ on B’s turf B charges b and A charges a^ : Thus, a consumer B offers price b; 20 To deal with out-of-equilibrium beliefs, we suppose that if first period prices are such that no consumer is expected to buy from firm A, a consumer who unexpectedly does purchase from A is assumed to have type y; and similarly a consumer who unexpectedly purchases from B is assumed to ¯ have type y:
Ch. 7. Behavior-Based Price Discrimination and Customer Recognition 401
^ and otherwise will on firm A’s turf will stick with good A if yoa b; switch to good B. If y is very near the endpoint y; then A’s turf is very small, and consists only of consumers with a strong preference for A, and firm A can charge the monopoly price in this market and not lose any sales to firm B. The paper shows that this occurs when y oy ; where y is the ‘‘isoelastic point’’ where F(y )+f(y )y ¼ 0, so that marginal revenue equals 0. In this case firm A sets a ¼ v y =2 and sells to everyone on its turf, while firm B sets b^ ¼ c: Otherwise, both firms will have positive sales in each market, which implies that the ‘‘poacher’s’’ price in a market must be lower than the incumbent’s. The intuition for this result comes from the fact that on the interior of firm A’s turf, its second-period reaction function reflects a trade-off between losing marginal customers at yA and inframarginal rents on types below yA, and so the reaction function does not depend on the firstperiod cut off y ; while decreasing y decreases B’s sales on A’s turf, and so makes firm B price more aggressively, as shown in Fig. 2, where the curves R are the reaction curves on firm A’s turf when it had the entire first-period market (which is why they intersect on the 45 degree line), and R^ is firm A’s reaction curve on its turf as a function of the first-period cut-off y : The next step is to work backwards and determine the equilibrium of first-period prices. Before presenting the analysis, we can identify some general considerations to keep in mind.
Fig. 2.
Second-period reaction functions as a function of the first-period cut-off.
402
D. Fudenberg and J. M. Villas-Boas
1) If consumers are forward looking (as assumed by FT) they realize that they will be offered a ‘‘poaching’’ offer in the second period. FT show that this can lead to a less elastic first-period demand, and hence higher first-period prices. 2) Firms may distort first-period price to increase second period profit. Specifically, each firm would rather that its opponent have less information about consumer preferences, and is willing to distort firstperiod prices for that purpose. Moreover, this preference is large enough that firms do better when neither of them has any information about consumer’s identities. The impact of this consideration depends on the way that changes in price change what is learned about consumer demand, which in turn depends on the distribution of types. 3) If customers buy from their preferred firm in the first period (as they do in FT) then second-period switching lowers welfare. To explore this second point in more detail, we present a more detailed analysis of second-period competition in the uniform case than is given in FT. Solving for the intersection of the second period reaction curves (corresponding to equation (6) and (7) in FT) shows that yA ¼ ðy þ yÞ=3; yB ¼ bb a ¼ ðy¯ þ y Þ=3: In its home turf, firm A sells to types below yA; this has ¯ y: ¯ On B’s turf A sells to types mass ðy 2 yÞ=3ðy¯ yÞ ¼ ðy þ 2yÞ=6 b ¯ ¯ between y and y ; this has mass ðy 2y Þ=3ðy¯ yÞ ¼ ðy¯ 2y Þ=6y: So the second-period profit of firm A is 2 2 ðy 2 y Þ2 ðy¯ 2y Þ2 5y þ 5y¯ þ ¼ , 18y¯ 18y¯ 18y¯ provided that y 4y ¼ y =2; so that there is poaching in both markets. By symmetry this is also the second-period profit of firm B. Note that the symmetric outcome y ¼ 0 is the global minimum of firm A’s second-period profits; it does better not only with a larger first-period market share but also with2 a smaller one! Specifically, when y ¼ 0 the ¯ ¯ second-period profit is 5y¯ =18y¯ ¼ 52y=18: As y increases to y=2 profit 2 ¯ ¯ ¯ ¯ ¯ ¯ increases to ðy=2 þ 2yÞ =18y ¼ ð25y =4Þ=18y ¼ 25y=72: From this point onwards, there is no competition in firm B’s second-period market. Firm ¯ which converges to the static equilibrium value A’s profit is ðy 2 y Þ2 =18y; ¯ ¯ of y=2 as y goes to y: This shows that both firms do best when neither has first-period information. When y is near the endpoints, firms have less precise information in the larger market, and hence competition there is less intense. Perhaps surprisingly, in the uniform case this second-period consideration has no impact on first-period pricing. This is because the first-period equilibrium will have equal market shares, i.e., y ¼ 0; and because y ¼ 0 leads to the lowest level of second-period profit, there is no first-order effect when it changes. For this reason, the only reason that first-period prices differ from the static equilibrium is that consumer demand differs. In the
Ch. 7. Behavior-Based Price Discrimination and Customer Recognition 403
static case, the cut-off y shifts one-for-one with prices, while in the dy^ ÞÞÞ=ð1 dC Þ; because type y namic setting y ¼ ðb a þ dC ð^aðy Þ bðy must be indifferent between the different plans to switch and get the ‘‘poaching price’’ next period, and so must be indifferent between buying ^ or buying B good A now at price a and then buying B tomorrow at price b; now at price b and then buying A tomorrow at price a^ : In the uniform case, this leads to a less elastic first period demand ðj@y =@ajo1Þ; and hence higher prices; with zero production costs and consumer types distributed on the unit interval, the first-period price is ð1 þ dÞ=3 and the second-period prices (on the equilibrium path) are 2/3 to the firm’s old customers and 1/3 to the customers it is trying to ‘‘poach.’’ This finding for the uniform case leaves open the possibility that for other distributions the second-period-profit effect could have an impact on firstperiod pricing. However, it seems plausible that y ¼ 0 is the global minimum of second-period profits for general symmetric distributions, so that the effect of second-period profit on first-period decisions vanishes, provided that the first-order approach is valid. However, the fact that firms would do better in the second period with a less symmetric first-period outcome suggests a possible non-concavity in the problem. The MHR assumption makes the static optimization problem concave, which implies that the firms’ first-period objective functions are concave for discount factors close to 0 and any distribution that satisfy MHR; FT show that they are also concave under the uniform distribution for all (common) discount factors (that is, df ¼ dc ¼ d). However, concavity does not seem to be implied by the MHR condition, and when it fails there can be mixed-strategy equilibria. To investigate this possibility it may be interesting to abandon the first-order approach altogether, and work with discrete types, as in Esteves and Chen and Zhang. Discrete distributions In Esteves’ model, whenever the difference in price is less than yByA each firm buys from their preferred firm, while if the difference is larger than this all consumers buy from the same firm and no information is revealed, which corresponds to the case y ¼ y¯ in FT. Again as in FT, the second-period profits are symmetric in the information: firms do better when the first-period prices are very different, but as far as second-period prices go they are indifferent between having a large turf or a small one. To simplify the analysis, Esteves assumes that consumers are completely myopic. The first-period equilibrium is in mixed strategies, and she shows that the probability that both firms have positive first-period sales decreases as they become more patient. Moreover, she shows that first period prices tend to fall as the discount factor increases. Chen and Zhang suppose that there are three types. A mass g is loyal to A (they get 0 utility from B, so they buy A whenever the price is below their reservation value v), a mass g is loyal to B, and a mass of 12g who are
404
D. Fudenberg and J. M. Villas-Boas
exactly indifferent. Neither firm can hope to sell to the loyalists of the other, so what each firm wants to do is distinguish its loyalists from the neutrals. Starting from equal first-period prices, a small increase in firm A’s price shifts all of the neutrals to firm B, and results in an asymmetric knowledge about the consumers: firm A knows who its loyalists are, but firm B does not. Thus, in contrast to the previous two papers, the firm with the smaller first-period sales has strictly higher second-period profits. They show that this leads to prices that are, on average, higher than in the static equilibrium, even when consumers are myopic. We should point out some unusual features of the assumed demand distribution. Specifically, second period profits when consumer types are known are exactly the same as in the static model, while in general we may expect that known types could lead to fiercer competition and lower profit. This suggests that competition in the static model is particularly fierce, which observation may help explain why equilibrium profits here are higher than when firms lack information on purchase history. Welfare Finally, we compare the welfare effects of price discrimination in the three models. In FT, the first-period outcome is efficient, so all secondperiod switching lowers welfare. In Esteves, both the static equilibrium and the first-period equilibrium of the two-period price discrimination game are not fully efficient, due to the randomized nature of the equilibrium. Moreover, when the first-period prices reveal the customers’ types, the secondperiod outcome is efficient, and there is no switching, even though firms offer lower second-period prices to their opponents’ customers. This stems from the two-point distribution of demand, and would not extend to a discrete model with more types. Combining these two observations, we see that price discrimination can increase efficiency provided that it doesn’t lower first-period efficiency too much, and she shows that this is indeed the case. In the Chen and Zhang model, efficiency considerations are moot, as the only consumers whose purchases change when price discrimination is allowed are those who are completely neutral. There can, however, be efficiency implications of price discrimination when the reservation value of the neutrals is less than the other players, as price discrimination allows the firms to offer the neutrals a second-period price that is below their reservation value without losing sales to the loyalists. 3.2 Infinite lived firms, overlapping generations of consumers, and shortterm contracts Villas-Boas (1999) extends the FT model to the case of two infinite-lived firms facing overlapping generations of consumers. Each consumer lives for two periods, and each generation has unit mass. Each firm knows the iden-
Ch. 7. Behavior-Based Price Discrimination and Customer Recognition 405
tity of its own past customers, but not those of its opponent, and it does not observe the consumer’s ‘‘age,’’ so it cannot distinguish young consumers from old ones who bought from the opponent last period. The basic setup of the model, and the notation, are the same as in FT, with y uniform on ½1=2; 1=2 and zero production costs. The timing of the game is a bit different, as in each period the firms first simultaneously set the price for new customers, and then set the prices to existing customers after observing the price the competitor charges to new ones. In order to focus on the dynamics of price discrimination, and abstract from (possibly important) repeated game aspects, the paper restricts attention to the state-space or MPE of the game. Given the linear-quadratic nature of the model, there are MPE in which the strategies are piecewise affined in the state variable, and these are the ones considered in the paper.21 As a benchmark case, note that the MPE here would be exactly the outcome in FT if, as in FT, firms can recognize both their own and the opponent’s customers, and all prices are set simultaneously. If firms can recognize both types of old customers, but prices are set sequentially as specified above, the prices will be 1 þ dC ðdF =4Þ to new customers, and the prices will be 3/4 and 1/2 to the firm’ and the competitor’s old customers, as opposed to 2/3 and 1/3 with simultaneous price setting. (Prices are higher with sequential moves because the reaction curves slope up, this is a form of the ‘‘puppy dog effect’’ (Fudenberg and Tirole, 1984).) We now turn to the MPE of the game where firms only recognize their own customers. If the reservation value is high enough that all consumers purchase every period, Villas-Boas shows that the equilibrium is again characterized by cut-offs yt such that each new consumer arriving in period t purchases from firm A if their type yoyt : Thus, the payoff-relevant state in each period is simply the previous period’s cutoff. The easiest part of the model to solve is the prices firms charge to their old customers. Since these consumers will leave the market at the end of the period, neither they nor the firm need to consider future periods in making their decision, and since prices are set after observing the rival’s poaching price, the firm faces a simple static maximization. In contrast, the price set to unrecognized consumers must take into account that some of these are new consumers who will purchase again in the next period, and the demand of new customers must also take the future into account. Neither of these complications is present in the case of complete myopia, dF ¼ dC ¼ 0. Here, the cutoff converges to the steady state with equal-market shares. Except possibly in the first period, the convergence is monotone, and customers sort themselves as in FT: those with strong preference for one 21 The reason to consider piecewise affine strategies instead of affine ones is that there are ‘‘kinks’’ in the value functions corresponding to the states where a firm completely retains all of its clientele; these kinks are roughly analogous to the points y in FT.
406
D. Fudenberg and J. M. Villas-Boas
firm buy from that firm in each period, while those with more intermediate preferences switch. As in FT, prices to the recognized consumers are lower than in the static case. Prices to the unidentified consumers are also lower than the static prices, while in FT the first-period price equals the static price when firms are myopic; this is because the pool of unidentified consumers here contains both new consumers (as in the first period of FT) and old consumers who prefer the other firm. Villas-Boas then considers the case of myopic firms but patient consumers; this differs from the previous analysis where consumers take into account the prices they will be charged next period; it differs from FT because a consumer who buys A in the first period is offered a second-period price for B that is tailored to a mixture of ‘‘A-preferrers’’ (i.e. yoyt ) and new consumers, as opposed to a ‘‘poaching price’’ for A-preferrers alone. This mixed price will in general be less responsive to changes in y than is the poaching price, which makes the marginal new customers more responsive to changes in price. For this reason, the price to new consumers is lower than in FT, and in fact it goes to 0 as dC-1. Finally, Villas-Boas considers the case where dF and dC are both nonzero. As in Esteves and Chen and Zhang, patient firms have an incentive to shift their prices in a way that softens future competition, which here leads to higher prices. In the case dC ¼ dF ¼ d-1, the price charged to new consumers converges to 0 with d, while the price charged to old ones converges to 1/2. Thus firms are worse off than when they could credibly share their information. We discuss the issue of information sharing in Section 5.2 on credit markets. 3.3 Long-term contracts As we remarked in Section 2, long-term contracts are used in a variety of consumer markets. This section considers the impact of competition in simple long-term contracts in the setting of the two-period FT model. Specifically, we suppose that in the first period firms A and B offer to sell their goods this period at spot prices a and b, and that they also offer long-term contracts to supply the goods in both periods for A and B. In the second period, firms know the first-period prices announced by their rival, and they also know from whom each consumer purchased, but do not observe the contracts chosen by their rivals’ customers. If a firm chooses to only sell long-term contracts, it would prevent poaching by its rival; but the fact that a monopolist with commitment power induces switching suggests that the complete lock-in will not be optimal here either. And indeed, Fudenberg and Tirole show that the equilibrium has the form depicted in Fig. 3: consumers who most prefer A buy a long-term contract from A; this is the interval ½y; yA : The next interval ½yA ; yA purchases A in each period on the spot market,
Ch. 7. Behavior-Based Price Discrimination and Customer Recognition 407
Fig. 3.
Equilibrium long- vs. short-term contracts as a function of the consumer type.
interval ½yA ; y buys from A in the first period and B in the second, and so on.22 Thus, as in the case of short-term contracts, there is inefficient switching. A key fact in determining the equilibrium outcome is that when firm A locks in more of its customers with long-term contracts (increases yA ), it becomes more aggressive on its turf in the second period, as cuts in its second-period price a do not reduce revenue from locked-in consumers.23 Since changes in yA do not change firm B’s prices on firm A’s turf, increases in yA lead both firms to set lower prices. Moreover, the monotone hazard rate condition implies that the slopes of the second-period reaction curves are less than 1, so increases in yA move the switchpoint yA to the right, which means fewer agents switch. Hence, if the firms use any long-term contracts at all, there will be less switching than with short-term contacts. Fudenberg and Tirole show that on the path of a symmetric equilibrium, firms do use some long-term contracts, so there is less switching (and more efficiency) than with short-term contracts. The intuition for this is as follows: by locking in some of its customers, a firm can commit itself to more aggressive second-period pricing on its own turf, which induces a lower second-period poaching price from Firm B. The marginal first-period A purchaser plans to switch in the second period, so lowering B’s poaching price lets Firm A charge a higher first-period price, which raises its profit.
22 Because this is a deterministic model, equilibrium prices must satisfy the no-arbitrage condition A ¼ a+da, so that all consumes who plan to purchase from A in both periods are indifferent between purchasing the long-term contract or a sequence of short-term ones. The results reported here rely on the tie-breaking assumption that when the no-arbitrage condition holds, it is the customers who prefer A most choose the long-term contract. Intuitively, there is an option value to the sequence of short-term contracts, and this value is increasing in the probability that the customer decides to purchase B instead of A in the first period. It seems plausible that this option value is higher for consumers with higher values of y, and indeed this tie-breaking rule corresponds to taking the limit of models where the secondperiod valuation is imperfectly correlated with first-period value, and the distributions are ranked by first-order stochastic dominance in the first-period valuation. Some sort of tie-breaking rule is needed in any deterministic model where there are multiple ways of purchasing the same consumption stream. 23 Note that firm A does not directly set yA ; instead, this switchpoint is determined by the condition that equilibrium prices satisfy the no-arbitrage conditions A ¼ a+da and B ¼ b+db.
408
D. Fudenberg and J. M. Villas-Boas
Conversely, a firm always uses some short-term contracts. Indeed, using only short-term contracts dominates using only long-term ones whenever first-period sales exceed the isoelastic point y : To see why, suppose that all customers in the interval ½y; y buy a long-term contract from Firm A, and that y 4y : Now suppose that Firm A deviates and offers only a short-term ^ Þ contract in the first period, where the price a is set so that a ¼ A dbðy ^ þdy ; where bðy Þ is Firm B’s poaching price when none of Firm A’s customers have a short-term contract. This price has been chosen so that a consumer of type y gets exactly the same utility from purchasing A in the first period at price a and then buying B at the poaching price as it received from purchasing the long-term contract from A, and since the change does not affect competition on Firm B’s turf it leads to the same first-period cutoff.24 Moreover, firm A would receive exactly the same payoff as with the long-term contract by offering a second-period price on its turf of a00 ¼ ^ Þ y ; as this price will induce all of its first period customers to purbðy chase from it again. However, when y 4y ; this pricing is more aggressive than is optimal, and firm A does strictly better by raising its second-period price, even though this leads some customers to switch. Fudenberg and Tirole go on to show that the equilibrium they construct remains an equilibrium when more general contracts are allowed, but they do not discuss uniqueness, and it is an open question whether more general contacts can lead to qualitatively different outcomes. Moreover, as with the analysis of short-term contracts, the MHR condition does involve a loss of generality; the effect of long-term contracts with the sorts of distributions studied by Esteves (2004) and Chen and Zhang (2004) is open as well. 3.4 Switching costs To conclude this section, we return to the case of short-term contracts to compare the impact of purely information-based duopoly poaching with price discrimination in the presence of switching costs. These costs are real social costs in, e.g., complementary equipment or in learning how to use the product; as such they differ from ‘‘pecuniary’’ switching costs such as cancellation fees. Before addressing price discrimination, we briefly discuss the forces present in models of switching cost without price discrimination. (For extended surveys of the switching costs literature, see Klemperer, 1995; Farrell and Klemperer, 2004.25) In two-period models such as Beggs (1989) and Klemperer (1987a), all consumers are locked-in in the second period, while 24 It is easy to see that there is an equilibrium with the same cutoff. Fudenberg and Tirole prove that (under their tie-breaking rule) any profile of first-period contracts leads to a unique first-period cutoff. 25 For early papers on switching costs see also, for example, von Weizsacker (1984), Klemperer (1987b), Farrell and Shapiro (1988, 1989). For the case of endogenous switching costs see, for example, Caminal and Matutes (1990) and Kim et al. (2001). For a recent survey on information technology and switching costs see Chen and Hitt (2005).
Ch. 7. Behavior-Based Price Discrimination and Customer Recognition 409
none are in the first. Second-period lock-in leads second-period prices to be higher than without switching costs, while first-period prices are lower, as firms compete for the rents from locked-in customers. Finally, consumers in the first period foresee being locked-in in the second period, and become less price sensitive, which is a force toward higher prices in the first period. To illustrate these forces, we will use a simple two-period model. Each firm sells a fixed and given product in each of the two periods. Each consumer buys at most one unit in each period. Consumers are uniformly distributed along a Hotelling segment, whereas firms are located at the extremes of the segment (as in the previous subsection). Transportation costs are t per unit and production costs are zero. A fraction s of the consumers who buy from Firm i in the first period incur a high-switching cost if buying from Firm j6¼i in the second period (so that they never switch firms in the second period). The parameter s can then be seen as an index of switching costs. The remaining consumers, in fraction 1s, have zeroswitching costs. Given these assumptions, we can start by determining second-period demand for each firm. Let qi1 be the distance to Firm i of a consumer with switching costs that is indifferent in the first period between the two firms (note qi1 ¼ 1 qj1 ). Then Firm i is guaranteed a demand of sqi1 in the second period from the consumers that have switching costs. The total demand in the second period for Firm i is then sqi1 þ ð1 sÞððt þ pj2 pi2 Þ=2tÞ; the unique second-period equilibrium prices are pi2 ðqi1 Þ ¼ t½1 þ sð2qi1 1Þ =3=ð1 sÞ; and the second period equilibrium profit for Firm i as a function of qi1 is pi2 ðqi1 Þ ¼ t=ð2ð1 sÞÞð1 þ ðsð2qi1 1Þ=3ÞÞ2 : This illustrates a first effect of switching costs. Consumers that bought initially from one firm would continue to prefer that firm, and in addition have now a more intense preference due to the switching costs. This would then decrease the demand own-price sensitivity in the second period, which would lead to greater prices and profits in the second period. Working backwards to the first period, consumers without switching costs behave exactly as in the static case, because their decisions do not affect what happens in the second period. Consider now the decisions of the consumers that have switching costs. For the marginal consumer buying product i, denoted by qi1, the total cost of buying product i is pi1 þ tqi1 þ dðpi2 ðqi1 Þ þ tqi1 Þ; while the total cost of buying product j is pj1 þ tð1 qi1 Þ þ dðpj2 ðqi1 Þ þ tð1 qi1 ÞÞ: Indifference between buying product i and j leads then to qi1 ¼ ð1=2Þ þ 3ð1 sÞðpj1 pi1 Þ=ð2t½3ð1 þ d sÞ dsÞ and a total demand in the first period of q¯ i1
3ð1 sÞðpj1 pi1 Þ pj1 pi1 1 ¼ þs . þ ð1 sÞ 2 2t 2t½3ð1 þ d sÞ ds
(8)
If consumers are myopic, or there are no switching costs, this reduces to the static Hotelling demands. Equation (8) illustrates a second effect of
410
D. Fudenberg and J. M. Villas-Boas
competition with switching costs: Switching costs and forward-looking consumers make the first-period demands less price sensitive because the marginal consumers realize that by buying one product they will be lockedin and pay a higher price in the next period. This is a force toward higher equilibrium prices in the first period. The firms set first-period prices to maximize the total value of profits, pi1 q¯ i1 þ dpi2 ðqi1 Þ: This maximization illustrates a third effect of switching costs. In order to get higher profits in the second period, firms charge lower prices in the first period to increase qi1. This is a force toward lower prices and lower profits. In this particular problem this maximization by each firm yields unique first-period equilibrium prices pi1 ¼ pj1 ¼ tð1 þ dÞ=½1 þ dð1 s=3Þ: In general, which effects dominate (for lower or higher profits) will depend on the particular characteristics in the market. In the particular example above, equilibrium profits are higher with switching costs. An example where it goes the other way can be obtained if consumers have small switching costs, change preferences from period to period, and are not too patient. Beggs and Klemperer (1992) look at the impact of large switching costs on the MPE of an infinite horizon duopoly model with uniform pricing. Each period, a fraction u of new consumers enter the market with horizontally differentiated preferences that are fixed over time. Once a consumer purchases from a firm it is unable to purchase from its rival in the future.26 In this model, firms use a single price both to exploit locked-in consumers and to attract new ones, so the effects of switching costs on prices are less obvious; Beggs and Klemperer show that switching costs increase prices in symmetric equilibria of the affine MPE that they consider. In some markets, switching costs can be created endogenously by the competing firms by putting incompatibility features in its products. This possibility may end up making all firms worse off in equilibrium (e.g., Cabral and Villas-Boas, 2005). Nilssen (1992) distinguishes between switching costs that are incurred each time a consumer changes supplier, and ‘‘learning’’ costs that are incurred each time a consumer uses a supplier for the first time. Nilssen argues that a greater relative size of switching to ‘‘learning’’ costs leads to higher prices for the loyal consumers, and lower introductory prices. Turning to our main interest of behavior-based pricing, we focus on the model of Chen (1997), which is a two-period, two-firm model that is very similar to that of Section 3.1, except that all consumers are identical in the first period, and that after making their first-period purchases, each consumer privately observes a switching cost s. As we will see, the main difference with the work discussed above is that second period prices on the two 26 As in Taylor (2003), discussed below, the model abstracts from the determination of initial market shares, and takes these as exogenous.
Ch. 7. Behavior-Based Price Discrimination and Customer Recognition 411
‘‘turfs’’ are independent of the relative sizes of these two markets. We will then discuss Taylor (2003) who extends Chen to oligopoly, multiple periods, and switching costs that are correlated over time, and conclude with a brief mention of some other related work. Following Chen, assume that all consumers have common value v for each of the two goods, and that their switching costs are distributed uniformly on an interval ½0; s¯: In the second period, a consumer will switch from Firm A to Firm B if the difference in prices a b^ is greater than his switching ^ and xGða bÞ ^ for cost, so sales on Firm A’s turf will be xð1 Gða bÞÞ Firms A and B, respectively, where x is the size of Firm A’s turf and G is the cumulative distribution function for the switching costs. Since the size of Firm A’s turf simply has a multiplicative effect on second-period profits, it clearly has no impact on second-period pricing or sales, at least at interior equilibria where both firms have sales on A’s turf.27 Intuitively, the fact that a customer bought from Firm A last period tells us nothing at all about his preferences, except that the customer must now pay the switching cost to use B, so the size of Firm A’s turf has no bearing on second-period competition. This is in contrast to the models of horizontal differentiation we considered earlier, where if firm A has a larger first-period market share it knows that the consumers in B’s turf have a stronger preference for B, and so Firm A is more aggressive on Firm B’s turf as Firm B’s first-period sales decrease. For this reason, we suspect that adding a small amount of horizontal differentiation to the switching cost model would make the second-period prices respond to market shares. With the uniform distribution, each firm charges second-period prices c þ 2¯s=3 and c þ s¯=3 on its own and the rival’s turf respectively, where v is assumed larger than c þ s¯; firms sell to 2/3 of their old consumers and 1/3 of their rivals, so second period profits are ð4¯s=9Þx þ ð1=9Þ¯sð1 xÞ ¼ s¯=3ðx þ 1=3Þ and ð¯s=3Þð1 x þ ð1=3ÞÞ; for Firms A and B, respectively. Because the first period product is completely homogenous, and second-period profit is increasing in market share, the first-period prices will be below cost: at the profile where both firms charged marginal cost, and so have second-period profit of 5¯s=18; either firm would gain by undercutting slightly, capturing the whole market, and having second-period profit 4¯s=9: In fact, Chen shows that the unique subgame perfect equilibrium has first-period prices of c d¯s=3; at this point cutting price would incur a large enough first-period loss to offset the second-period gain. Thus, the conclusion that prices rise over time extends from switching-cost models without targeted pricing to switching-cost models with behavior-based pricing. This prediction is in contrast to that of the FT model of short-term contracts, where prices rise over time.28 27 Chen shows that the equilibrium is interior; Taylor extends this finding to distributions G such that both G and 1-G satisfy the MHR condition. 28 Of course the dynamics of prices are different in stationary infinite-horizon models such as VillasBoas (1999).
412
D. Fudenberg and J. M. Villas-Boas
If firms cannot observe the consumers’ past purchases, then firms with larger first-period sales will price less aggressively in the second period. Chen shows that this would lead to less aggressive first-period pricing, so that, as in FT, firms are made worse off when they can both engage in targeted pricing.29 Moreover, consumers need to forecast first period sales to know second-period prices, and the assumption of homogenous consumers means that the model may have multiple equilibria. As noted above, Taylor extends Chen’s analysis in several ways. To simplify the analysis, he also assumes that consumers are already ‘‘assigned’’ to one of the firms at the start of the first period. For this reason, first-period demand is very different than in Chen’s model, and maintaining the rest of Chen’s set-up, first-period prices are now above marginal cost, and second-period market shares depend on the initial conditions; prices in the second period, being independent of market share, are the same as in Chen. Taylor extends this analysis to multiple periods, finding that prices in the two markets are constant over time until the last period. This is intuitive: only the most recent purchase matters for the evolution of switching costs, so all periods before the last are strategically similar (given the assumption that consumers enter the game already assigned to a firm). More surprisingly, moving from two firms to three makes a substantial qualitative difference: when there are at least three firms, at least three of them offer marginal cost pricing to other firm’s customers. The reason that three is the key number here is that with three firms, there are two firms competing to get customers from each other firm, so that there is Bertrand competition for the switchers. This insight suggests that it would be interesting to study information-based price discrimination in models with three or more firms; this will be complicated by the need to consider a richer specification of preferences, with a two-dimensional taste parameter y. As usual with differentiated products, we would not expect prices to be driven to marginal cost, but new and interesting features could emerge. Finally, Taylor considers a two-period model with two types of consumers, those whose switching costs tend to be low and those whose costs tend to be high. Here, a customer who ‘‘switches’’ in the first period is thought on average to have lower switching costs, so that agents who switch will be offered a lower price by their first-period supplier than agents who buy from that supplier without switching. It would be interesting to extend this analysis to more than two periods. In that case, consumers will be all the more concerned about their ‘‘reputations,’’ and the impact of being known as a low-cost switcher may be ambiguous, as firms may wish to avoid ‘‘recruiting’’ consumers who are likely to soon move on to another brand. 29 Chen analyzes one of the equilibria for the uniform-price model, we do not know whether there are others.
Ch. 7. Behavior-Based Price Discrimination and Customer Recognition 413
In addition to these papers, we should mention the paper by Shaffer and Zhang (2000) that looks at a static game corresponding to the last period of the sort of two-period model studied above, with the additional feature that switching may be more costly in one direction than in the other (see also Shaffer and Zhang, 1995). With symmetric-switching costs, firms always charge a lower price to their rival’s consumers, but this need not be true when switching costs are sufficiently asymmetric. More recently, Dobos (2004) analyzes a model that combines horizontal differentiation in the first period, switching costs in the second, and network externalities in both; he finds that profits are decreasing in the size of the network effect, as this effect leads to more aggressive first-period pricing.30 4
Behavior-based pricing with multiple products, and product design
So far we have been assuming, for the most part, that each firm produces a single good. We now consider cases where each firm may produce multiple versions of the same product. Even in the case where the set of goods is fixed, this leads to interesting forms of behavior-based pricing, such as price discounts for consumers who are upgrading as opposed to new purchasers. In addition, there are the questions of how many different goods a firm will choose to sell, and (assuming it has this choice) what their characteristics will be.31 The literature on behavior-based pricing and multiple goods has studied two rather different sorts of goods and demand structures. Fudenberg and Tirole (1998) and Ellison and Fudenberg (2000) studies ‘‘upgrades’’ in models of vertical differentiation, where all customers agree that one good is better than the other; these models study only the monopoly cases. Thus, these papers are most closely related to the literature we discussed in Section 2. In contrast, Zhang (2005) studies endogenous product lines in a Hotelling style duopoly model of horizontal differentiation that is similar to the model of Fudenberg and Tirole (2000) except for the assumption of quadratic ‘‘transportation costs.’’ We focus on these two sorts of models, and do not discuss the related literature on the monopolist’s profit-maximizing menu of goods and prices in a static model.32 We do, however, discuss the papers of Levinthal and Purohit (1989), Waldman (1996), and Nahm (2004), which study the introduction of a new product in models with anonymous consumers and a frictionless second-hand market. Although behavior-based pricing is not considered in these papers, the 30 His model is an extension of Doganoglu and Grzybowski (2004), who consider the same preferences but without price discrimination. Villanueva et al. (2004) investigates the strategic effects of firms considering the life time value of customers. 31 This latter question can also be asked when each firm is only allowed to produce a single good, but that question does not seem to have been explored in the literature on behavior-based pricing. 32 See Mussa and Rosen (1978) and Deneckere and McAfee (1996) for discussions of the way the monopolist’s desire to extract surplus leads to distortions in the product line.
414
D. Fudenberg and J. M. Villas-Boas
analysis of the anonymous case is an important benchmark for the effects of behavior-based pricing. 4.1 Upgrades and buybacks with an anonymous second-handmarket In this subsection and the next, we discuss the two-period model of Fudenberg and Tirole (1998). We begin with the case of anonymous consumers and a frictionless second-hand market, which corresponds to the market for textbooks, and is also a useful benchmark for evaluating the impact of behavior-based pricing. As noted above, behavior-based pricing is impossible when consumers are anonymous, just as it is in the durablegood models of Section 2. Indeed those models can be viewed as a special case of this one, because whether or not there is a second-hand market that makes no difference given that there is a single perfectly durable good and all consumers enter the market at the beginning and remain until the end. In period 1, the monopolist produces a low-quality version of a durable good; this good is denoted L. In Period 2, the monopolist can produce both L and an improved version H. These goods are produced under constant returns to scale, with cost cL for L and cH ¼ cL+cD for good H, where cDZ0.33 There is a continuum of consumers, indexed by yA[0,1]; a type-y consumer has utility yV+I, where I is her net income, and V ¼ VL or VH ¼ VL+VD, VD>0 depending on whether she consumes L or H. This is a fairly standard demand structure, and it is easy to work with, but involves some loss of generality, as can be seen from the fact that in a static model the monopolist will not offer both goods if their costs are the same.34 Following the paper, we assume that VL>cL and VD>cD. To simplify, we also assume that the distribution of types is uniform; the paper assumes that the distribution has a continuous density that satisfies the monotone hazard rate condition. The firm and the consumers use the common discount factor d. Because the monopolist lacks commitment power, we solve the problem by working backwards from the second period. The solution here depends on the stock x1 of L that is already in the hands of the consumers, but the assumptions of anonymity and a frictionless second-hand market mean that we do not need to worry about which consumers bought the product, and indeed we can suppose that all old units are sold in the second-hand market, with some of them possibly repurchased by their original owners. The form of the utility function implies that there will be three (not necessarily nonempty) segments of consumers in the second period: types in the interval [0, yL] do not consume; types in [yL, yH] consume good L, and types in [yH, 1] consume good H. The market price of good L is then the value pL ¼ yLVL that makes yL indifferent between purchasing L and not purchasing, while 33
The production cost of upgrading an L unit to H is the same as that of making H from scratch. This can be seen by considering equation (9) when x1 ¼ 0.
34
Ch. 7. Behavior-Based Price Discrimination and Customer Recognition 415
the price of H makes yH indifferent between purchasing H or purchasing L, so pH ¼ pL+yHVD. If the mass of consumers consuming good L is greater than the existing stock, i.e., if yHyL>x1, the monopolist is a net seller of L in Period 2; when the reverse inequality holds, the monopolist is engaged in ‘‘buybacks,’’ and when yHyL ¼ x1, the monopolist is inactive on the L market. Each of these regimes can arise for some values of the first-period stock; moreover, each of these regimes can arise for an open set of parameters in the full equilibrium, where x1 is determined by the monopolist’s first-period sales. When yHyL>x1, so there are net sales, the monopolist has secondperiod profit Ynetsales ¼ ðyH yL x1 ÞðyL V L cL Þ 2 þ ð1 yH ÞðyL V L þ yH V D cH Þ ¼ ð1 yL x1 ÞðyL V L cL Þ þ ð1 yH ÞðyH V D cD Þ.
ð9Þ
Thus, it is as if the monopolist faces two separate, unlinked markets in Period 2. All consumers above yL purchase L, with x1 of this coming from the pre-existing supply. Separately, the monopolist supplies the ‘upgrade’ to types above yH; this (fictitious) good has incremental cost cD and sells at price yHVD. Thus, when the net-sales regime prevails, the monopolist sells exactly the same amount of good L as it would if good H did not exist, and sales of the old good follow the standard Coasian path discussed in Section 2. Similarly, price and sales in the upgraded market are not influenced by x1. Thus, the first-order conditions for maximizing (9) are given by the standard formulas: yL V L cL 1 yL x1 ¼ yL V L yL and yH V D cD 1 yH ¼ . yH V D yH When yHyLox1, so there are buy-backs, we suppose that the monopolist has no use for Q repurchased units. Thus, the payoff function in this net sales region is the same as except that cL is replaced by 0. That is, 2 Ybuybacks ¼ ð1 yL x1 ÞðyL V L Þ þ ð1 yH ÞðyH V D cH Þ. 2 Note that once again the ‘‘upgrade market’’ decouples from the market for L. However, the price for L (given x1 and the buy-back regime) is lower than it would have been if H had not been introduced, for now the ‘‘effective cost’’ of L is zero. Thus, while the monopolist’s second-period payoff is continuous at the boundary between net sales and buybacks, it has a kink
416
D. Fudenberg and J. M. Villas-Boas
there, as the effective marginal cost changes from 0 to cL. For this reason, the ‘‘inactive’’ regime is the equilibrium for a range of values of x1. In this regime the constraint yHyL ¼ x1 is binding, and the markets do not decouple. Fudenberg and Tirole show (in Proposition 2) that there are numbers 0 x1 ox¯ 1 o1 such that when xox 1 the solution has net sales, and is exQ net sales actly the solution to maximizing ,2 while ignoring net-sale conQ net the sales straint. For x1 ox1 ox¯ 1 the solution that maximizes has negative Q buybacks2 sales of L, while the solution that maximizes has net sales; here, 2 the second-period equilibrium is at the kink. Finally, for x¯ 1 ox1 the solution has buybacks. Moreover, pL is a continuous and weakly decreasing function of x1, and yL+x1 is continuous and weakly increasing. What we are really interested in is the full equilibrium of the two-period game. Fudenberg and Tirole show that setting a first-period price of p1 leads to sales to all types above the cutoff value y(p1), so that the stock on hand at the start of the second period is x1 ¼ 1y1. The monopolist’s problem is thus to maximize the discounted sum of first- and second-period profits, taking into account the way that first-period sales determine the second-period regime. The following examples show that each regime can arise for some parameter values, and give a flavor of when they might be expected, but stop far short of a characterization of when each regime prevails. First, if cL Q ¼ cH ¼ 0, then there are always buybacks. To see this, note that buybacks in this case simplifies to ð1 yL x1 ÞðyL V L Þ þ ð1 yH ÞðyH V D Þ; 2 so that the optimum in the H market is yH ¼ 1/2, which is the same as the optimum in the L market when x1 ¼ 0.35 Thus, there are buybacks when x1 is close to zero, and as x1 increases, yH is unchanged while yL+x1 increases, so buybacks (which are x1+yLyH) increase as well. Next, net sales occurs whenever cL ¼ 0 and cD is almost as large as VD, so that the new good is sold to only the highest value consumers. This is true for any value of the discount factor, but it is easiest to see for the case d ¼ 0, as here first-period output is the amount sold by a static, zero-cost monopolist, which is 1/2 for the uniform case considered here, while the firstorder condition for yL in the net sales regime simplifies to yL ðx1 Þ ¼ ð1 x1 Þ=2; so that yL ð1=2Þ ¼ 1=4o1=2 and the second-period solution following x1 ¼ 1/2 indeed has net sales. Finally, the equilibrium will have neither sales nor buybacks if cD ¼ 0 and cL ¼ cH is very close to VL. Intuitively, when cD ¼ 0 there will be no production of the old good in Period 2, and because costs are close to VL, there will be very little production of L in the first period, so x1 is small, which makes buybacks less likely.36 35 This equality does not depend on the uniform distribution, but rather on the assumptions that costs are zero and quality enters the demand function multiplicatively. 36 The formal argument uses continuity and monotonicity properties.
Ch. 7. Behavior-Based Price Discrimination and Customer Recognition 417
At this point, we should mention the related work of Levinthal and Purohit (1989) and Lee and Lee (1994) on monopolists with an anonymous second-hand market. Levinthal and Purohit consider a model with costless production, where the second-period market is described by a pair of linear demand curves, and the rental prices of each generation are equally affected by an increase in the output of the new generation.37 In their model, buybacks are only optimal when the firm is sufficiently patient, and otherwise there are net sales.38 Lee and Lee suppose that the monopolist is unable to sell or buy units of the old product in period two. 4.2 Upgrades and buybacks with non-anonymous consumers Fudenberg and Tirole go on to consider two other sorts of information structures: ‘‘identified consumers,’’ where the firms know which consumers purchased at Date 1, and ‘‘semi-anonymous consumers,’’ where consumers can prove that they purchased if they wish to do so, but can also pretend not to have purchased, which constrains the price to new customers to be no lower than the ‘‘upgrade price’’ offered to old ones. Following the paper, we now assume that V L 4dV H ; which implies that any first-period price induces a cut-off y1(p1) such that the consumer of type y purchases when y4y1 ðp1 Þ: This assumption is stronger than one would like, but we are not aware of a weaker condition that guarantees a first-period cutoff, nor of related analyzes that allow for disjoint sets of consumers to purchase in the first period. We also assume that cL ¼ 0, and that cH ¼ 0 as well; the paper does not make this last assumption. We begin with the case of identified consumers. Here, the monopolist faces two distinct second-period markets, patrons, and non-patrons. On the patron’s market the monopolist maximizes ð1 yu Þ yu V D subject to yuZy1, so yu ¼ max ½1=2; y1 ; and pu ¼ yuVD. On the non-patron’s market, the monopolist will sell good H to consumers with values between yH and y1, where yH is chosen to maximize (y1yH)yHVH; the solution to this is yH ¼ y1/2, with price pH ¼ VHy1/2. Comparing the objective functions in the two markets let us identify two competing effects. First, non-patrons of any given type have more to gain from purchasing because they have a lower payoff without a purchase; this ‘‘reservation utility effect’’ pushes the upgrade price to be lower than the price to new consumers. On the other hand, former customers have higher types; this ‘‘ratchet effect’’ means that non-patrons should get lower prices. These effects will help us understand when the identified and semi-anonymous cases coincide. 37 Note that in the Fudenberg and Tirole model, the price of L is yLVL and so depends only on the supply of good L. 38 Their results imply that the inactive region never occurs without pre-commitment. They show that in some cases the monopolist can gain from a first-period commitment not to produce L in the second period, just as it could if good H did not exist.
418
D. Fudenberg and J. M. Villas-Boas
Fudenberg and Tirole show that in equilibrium the monopolist chooses y1>1/2, so that all old patrons upgrade, and there is no ‘‘leapfrogging’’ of lower-value consumers past higher-value ones.39 Moreover in this case the second-period upgrade price is y1VD. In the semi-anonymous case, the payoff functions in the two markets are the same as with identified consumers, but the markets are linked by the customers’ incentive compatibility constraint, which requires that purpH, The calculations above show that this constraint is slack, and the two solutions coincide, if and only if VDrVH/2, or equivalently, if VDrVL, i.e., if the size of the innovation is not too large. The intuition for this is that for large innovations, upgrading is very attractive to high-value types, so the ‘‘ratchet effect’’ dominates the reservation utility effect; this is true for general distributions and not just the uniform. Finally, Fudenberg and Tirole show that with costless production the monopolist’s profits are higher under anonymity than with identified consumers. With costless production, when y1Z1/2 (which is the relevant range) the anonymous-market solution is for customers between y1/2 and 1/2 to consume L, and customers from 1/2 up to consume H; with identified consumers, the monopolist sells H in the second period to all types above y1/2. The commitment solution is to sell H to consumers above 1/2, and nothing at all to the others; the anonymous solution is closer to this outcome, and so yields higher payoffs. The point is that the presence of the second-hand market leads the monopolist to sell less of H in Period 2, which helps alleviate the commitment problem in Period 1. (Note that this finding does not immediately extend to the semi-anonymous case, except for parameters where it coincides with the solution with identified consumers: The no-arbitrage constraint cannot help the monopolist in the second period, for any given first-period outcome, but the constraint could have an impact on first-period play.) 4.3 Endogenous innovation Waldman (1996) and Nahm (2004) analyze endogenous innovation in the anonymous case. Waldman supposes that there are only two types, yL and yH, with yLVLocL. This means that the firm would not sell to the low types in a one-period model, and moreover in the absence of the new good the firm would not produce in Period 2. That is, the assumed demand structure means that the firm would not face the usual Coasian commitment problem. However, the sale price of the low good in Period 1 is decreasing in the probability that the firm will introduce an improved good H in the second period, and Waldman shows that the firm does face a commitment problem with respect to introducing the improved good. 39 Leapfrogging can occur when cH>0, as here the monopolist will not induce all old patrons to upgrade but it will sell H to non-patrons so long as cH is not too high.
Ch. 7. Behavior-Based Price Discrimination and Customer Recognition 419
Nahm points out that this conclusion relies on the assumed demand structure. In a two-type model with yLVL>cL, the price of good L will fall over to yLVL in the second period whether or not the new good is introduced, and the firm does not face a commitment problem with respect to introducing the new good. Nahm goes on to investigate the incentives for introducing the new product in a model of Section 4.1, where in between Period 1 and Period 2 the firm spends resources on R&D, which in turn determines the probability that the high quality good is available in Period 2. As we saw above, in the net-sales case, the second-period price of good L is the same whether or not H is introduced, and investment in R&D only influences payoff in the ‘‘upgrade’’ market. Hence, the monopolist does not face a time-inconsistency problem with respect to R&D, and it chooses the same level of investment that it would chose if it could commit to the choice in Period 1. However, in the inactive and buy-back regimes, the second period price of L is lower if H is introduced than if it is not. Hence, to maximize first-period sales and overall profit, the monopolist would benefit from a commitment that limited its R&D. Ellison and Fudenberg (2000) analyze the semi-anonymous, costless-production case in a model intended to correspond to markets for software. It is very similar to that discussed above, with one good in Period 1 and the possibility of producing an improved version in Period 2; the main difference is that their model includes (positive) network externalities. In their model, consumers incur set-up or training costs each time they adopt or upgrade their software, and differing versions of software are backwards but not forwards compatible, so that users of the newest version of the software enjoy the largest network benefits. In their dynamic model,40 consumers are ex-ante identical, but not all of them are present in the first period. They show that the monopolist suffers from a commitment problem that can lead it to introduce upgrades that are not only welfare-decreasing but also lower its own overall present value. The idea of this result is simple: in the second period the monopolist may prefer to sell upgraded, highervalue software to new consumers, but this forces the old consumers to either incur the costs of learning to use the new version or settle for smaller network benefits due to incompatibility with new consumers. This can lead to a loss of first-period profits that outweighs the second-period gain. As it is common in models of network externalities, consumers’ purchasing decisions have the flavor of a coordination game, and can have multiple equilibria. Ellison and Fudenberg assume that in the second period, new consumers coordinate on the equilibrium that is best for them, and consider two different equilibrium-selection rules for the old consumers 40 The paper also considers a static model with a continuum of types, and shows that even with commitment the monopolist may introduce socially inefficient upgrades to help it price discriminate. That model is less closely related to the themes of this survey.
420
D. Fudenberg and J. M. Villas-Boas
who are deciding whether to upgrade; in either case there is a region of the parameter space where the monopolist introduces the upgrade when the social optimum would be to sell only the old good in both periods. 4.4 Endogenous location choice in duopoly Waldman, Nahm, and Ellison and Fudenberg consider a monopolist whose innovation decision is whether to introduce or research an improved version whose characteristics are fixed. Zhang (2005) considers endogenous location choice in a two-period poaching model. The idea is that the rise of flexible manufacturing makes it cheaper for firms to customize products to various clienteles, and since purchase decisions convey information, firms might want to design one product for its established customers and another for those they are trying to poach from a competitor. The information structure and institutional assumptions are the same as in the short-term contracts section of Fudenberg and Tirole (2000), but the payoff functions are different: Consumers are uniformly distributed on the interval [0,1], while firm locations are endogenous, and transportation cost is quadratic in distance: the utility of for type y of consuming a good at Location a is v tðy aÞ2 ; where the reservation utility is assumed high enough that in equilibrium all consumers purchase. At the start of the first period, the two firms simultaneously choose Locations a and b, respectively, and in the second period, each firm can produce products at two (or more) locations, and offer different prices and goods to consumers whose firstperiod actions were different. In the base model, designing new products is costless.41 If the firms and consumers have the same discount factor, or more generally, if the consumers are sufficiently patient compared to the firms, the equilibrium is for the firms to split the market in the first period, and for each firm to offer two new and distinct models in the second period, with Firm A choosing ao, an and firm B choosing bo, bn, where ‘‘o’’ and ‘‘n’’ are for old and new consumers, respectively. However, as in the poaching models discussed in Section 3, firms do better when they have less first-period information, and if firms are sufficiently patient compared to consumers then the first-period pure-strategy equilibria are asymmetric, with one firm capturing all of the market, so that first-period purchases reveal no information. To understand these results, we explain the outcome in the second-period markets for types that have been revealed to lie in an interval [Z, Z+L], which is the same as in a static model with these types as the single market. It is interesting to note that although introducing varieties is costless, and firms are allowed to introduce as many as they wish, in equilibrium each firm only sells a single product. This fact is closely related to the fact that if 41 The paper speculates briefly about the case where innovation costs are such that firms introduce a single new product in Period 2.
Ch. 7. Behavior-Based Price Discrimination and Customer Recognition 421
each firm can only introduce a single product, they will choose locations outside the support [Z, Z+L] of the distribution of consumer types if such locations are allowed, and at the boundaries of the distribution if it is not. Intuitively, firms face a trade-off between locating near the center of the distribution, which increases profits holding the opponent’s price fixed, and locating toward the edges, which lessens price competition and raises the opponent’s equilibrium price. With quadratic transportation costs and the uniform distribution, the strategic effect dominates until the locations are well outside the support of the distribution of types.42 The fact that the optimal locations for a single product are outside of the support provides an intuition for why introducing a second variety would not be helpful: if the new variety is to provide an increase in efficiency, it must be closer to the opponent’s location, but this would provoke the price competition that the first location was chosen to avoid. Now, consider firms simultaneously choosing locations and prices in two different second-period markets, corresponding to the first-period purchase of the consumers. The previous paragraph explains why each firm will choose a single product for each market; in general, these products will be different, and a better match for the tastes of the market they are designed for. Now we turn to the consumer’s decision in the first period. As in Section 3.1, the first-period decisions of consumers will generate a cut-off rule, so that first-period sales identify two intervals of consumers, corresponding to each firm’s turf. Also as in that model, the consumers who are near the cutoff in the first period switch suppliers in the second period, and increased consumer patience makes first-period demand less elastic. Consumers benefit most when they are identified as being in a small interval, as this leads to intense price competition; the firms’ second-period profit is highest when all consumers purchase from the same firm in the first period, so that the purchases reveal no information. Working backwards to the firm’s first-period decisions, Zhang shows that when consumers and firms are equally patient, and more generally if the consumers are sufficiently patient compared to the firms, the first-period outcome is symmetric, with Firms A and B located at equal distance from the market center, and each taking half the market. In the second period, each firm introduces two new products, one for each segment of the firstperiod market. On the other hand, if firms are patient and consumers are myopic, the firms are able to avoid segmenting the first-period market, and their combined profits increase. 42 Economides (1986) studies the Hotelling location-price game where duopolists each offer one product, with a uniform distribution on types, and transportation costs proportional to ta. He shows that for aA[1.26, 1.67] the firms locate within the distribution of types, while for aA[1.67, 2] they locate at the endpoints. (He constrains them not to locate outside of it.) For aA[1, 1.26] there is no purestrategy equilibrium; see D’Aspremont et al. (1979) for the linear and quadratic cases.
422
D. Fudenberg and J. M. Villas-Boas
Zhang’s results on product design seem to reinforce the idea that customer recognition leads to more intense competition, and lower profits. It would be interesting to understand what happens if we have a longer time horizon (possibly with changing consumer tastes), and what would happen under product choice and monopoly, with customized product advertising (and where this customized advertising could also depend on past behavior). 5
Related topics: privacy, credit markets, and customized pricing
This section briefly discusses the issues of consumer privacy protection, pricing in credit markets, and standard third-degree price discrimination that is based on exogenous characteristics. We focus on the work of Thisse and Vives (1988), Dell’Ariccia et al. (1999), Dell’Ariccia and Marquez (2004), Taylor (2004a), and Calzolari and Pavan (2005) and also discuss Pagano and Jappelli (1993), Padilla and Pagano (1997, 2000), and Taylor (2004b). 5.1 Privacy As we have seen, the efficiency consequences of BBPD are ambiguous, so there is some reason to consider the impact of various regulations and firmbased initiatives that protect consumer privacy.43 One interpretation of consumer privacy is that firms cannot track consumers’ past behavior.44 Consumers that buy early may be recognized as consumers that value the product highly, and then be charged a higher price in subsequent periods. In this sense consumers are hurt by losing their privacy, they are charged higher prices. As discussed above, consumers, if they are aware of this loss of privacy, may be strategic in the earlier periods, and refrain from purchasing the product, not to reveal their high valuation. This may give firms an incentive to commit to privacy protection. Taylor (2004a) uses a variation of the two-period model of Section 2 to focus on the privacy issue. Consumers interact sequentially with each of two firms, and each consumer’s valuations for the products of the two firms are positively correlated, so that, if the second firm is able to observe that a 43 This ambiguity should not be a surprise in view of previous results on related issues. Hirshleifer (1971) noted that the efficiency impact of information acquisition is ambiguous when markets are incomplete. This holds in particular for firms acquiring more information about the characteristics of each consumer. For example, Hermalin and Katz (2006) show that third-degree price discrimination may be better or worse from a social point of view than second-degree price discrimination. Wathieu (2004) argues that information about consumers may lead to inefficiently many products being produced, each at too low a scale. For a recent survey on the economics of privacy see Hui and Png (2005). 44 Upon realizing that Amazon was charging different prices for the same item, possibly based on different purchase histories, some consumers showed concern about shopping there (Customers Balk at Variable DVD Pricing, Computerworld, September 11, 2000, p. 4).
Ch. 7. Behavior-Based Price Discrimination and Customer Recognition 423
consumer bought from the first firm, then the second firm’s beliefs about the valuation of that consumer for its product is higher than if the consumer declined to purchase. Taylor assumes that the second firm is unable to commit to its prices until after consumers interact with the first one. Privacy is the case in which the second firm is not able to observe whether a consumer bought or did not buy in the first period. Without privacy, the first firm can sell the list of its customers, and allow the second firm to price discriminate between the consumers that bought and did not buy from the first firm. If there is no privacy, the first firm sells the customer data to the second, and consumers do not foresee that sale (in the context of Section 2 this is the case when the consumers are myopic), then the first firm has a greater incentive to charge higher prices in order to make the customer data more valuable. If consumers foresee that the first firm is going to sell the customer data to the second firm, then they strategically refrain from buying, which makes the customer data being sold less valuable, and gives incentives for the first firm to lower prices. Firms prefer the no-privacy case when consumers are myopic, but prefer consumer privacy if consumers are able to foresee that under no privacy their purchase information is going to be sold. Taylor shows that welfare can be higher or lower under consumer privacy depending on the demand elasticity. Calzolari and Pavan (2005) consider the case where two principals sequentially contract with a common agent, and where the upstream principal can sell its information to the downstream principal. They assume that the agent’s valuations with the two sellers are perfectly correlated, which is more restrictive than Taylor’s assumption of imperfect correlation, but otherwise their model is more general.45 As in Taylor, the second principal posts its contract after the consumer has already decided whether to accept the contract of the first firm. By selling information to the downstream principal, the upstream principal may get some payment from the downstream principal (possibly due to greater efficiency, or less information rents provided to the agent, in the downstream relationship), or appropriate any rents of the agent in the downstream relationship that are generated by this sale of information. Calzolari and Pavan identify three conditions under which, if the upstream principal can commit not to disclose any information (commitment to privacy) she will choose to do so. The first condition is that the upstream principal is not personally interested in the decisions taken by the downstream principal. In the context of Taylor (2004a) this is just that the profit of the first firm is independent on the decisions taken by the second firm. The second condition is that the agent’s exogenous private information is such that the sign of the single crossing condition is the same for both the upstream and downstream decisions. In the context of Section 45 Ben-Shoham (2005) extends the Calzolari and Pavan analysis to allow for imperfect correlation, and also for imperfect (i.e., noisy or partial) revelation of information from the first principal to the second.
424
D. Fudenberg and J. M. Villas-Boas
2, this condition is just that the valuation of a consumer type is the same across products. In Taylor (2004a) the valuation for the product of the first firm is positively correlated with the valuation for the product of the second firm. Finally, the third condition is that the preferences in the downstream relationship are additively separable in the two contractual decisions. In the context of Section 2, or Taylor (2004a), this is immediately obtained because the second-period profit or utility is independent of whether there was a purchase in the first period. It is interesting to try to informally relate the first condition with the two-period model in Section 2. Denote the first-period profit under disclosure of information as a function of the first-period action a1 as pd1 ða1 Þ; the first-period profit under privacy as a function of the first-period action as pp1 ða1 Þ; the second-period profit under disclosure of information as a function of the first-period action as pd2 ða1 Þ; and the second-period profit under privacy as pp : Note that in the model of Section 2, the second-period profit under privacy is independent of the first-period action.46 In the context of Section 2, the firm chooses its first-period action under disclosure of information to maximize pd1 ða1 Þþpd2 ða1 Þ (where the discount factor was set to one). In Calzolari and Pavan, the upstream principal is able to receive a payment for the disclosure of information from the downstream principal in the amount of pd2 ða1 Þ pp2 : The upstream principal chooses then its action under disclosure of information to maximize pd1 ða1 Þþ½pd2 ða1 Þ pp2 ; which results in the same optimal action as in the model of Section 2. Finally, note that in the model of Section 2 the firm chooses privacy if and only if maxa1 pp1 ða1 Þþpp2 maxa1 pd1 ða1 Þþpd2 ða1 Þ; while in the context of Calzolari and Pavan the upstream principal chooses privacy if and only if maxa1 pp1 ða1 Þ maxa1 pd1 ða1 Þþ½pd2 ða1 Þ pp2 : It is immediate that privacy is chosen in both models in exactly the same conditions (no customer recognition in the model of Section 2). So, even though in Calzolari and Pavan there are two principals, in the case where the upstream principal expropriates the informational rent from the downstream principal, the model corresponds to single-principal models discussed in Section 2. Calzolari and Pavan (2005) also show that under the second condition, if the upstream principal discloses information to the downstream principal, the increase in the rent that has to be given to the agent always offsets any potential benefit from the sale of information, or from a greater rent of the agent in the downstream relationship. This is because, if information is disclosed, the agent becomes more protective of his type and the upstream principal does not have the possibility of using any distortion of the down-
46 Calzolari and Pavan allow for the second-period profit to be additively separable in the two contractual decisions, and therefore to be also a function of the first-period actions even under privacy. This possibility does not affect the argument above.
Ch. 7. Behavior-Based Price Discrimination and Customer Recognition 425
stream relationship contractual variable to help the agent reveal his type. This then implies that when the upstream principal is not personally interested in downstream decisions (the first condition), then there is no advantage in disclosing information and the optimal policy is committing to full privacy. The paper then argues that each of these conditions is necessary for the full privacy result, and that if one of the conditions does not hold it may be optimal for the upstream principal to disclose information to the downstream principal. In such cases, there are also situations in which disclosure of information benefits all three players. Taylor (2004b) considers a market situation in which firms first post prices, and then decide on how much to screen the customers that demand their product. The profit that a firm derives from a customer depends not only on the price charged, but also on the cost of servicing that customer which varies in the population (and that is also not known by the customer). The amount of screening chosen by a firm allows that firm to receive a noisy signal about the cost of servicing a customer. More screening reduces the noise of the signal. In relation to the papers above, this paper can be seen as looking at quantity discrimination, while the papers above looked at price discrimination. Given that the cost of servicing a certain customer depends on the a priori unknown characteristics of the customer, this model matches well the market features of credit markets, discussed below. Consider the case in which the screening device sometimes misses ‘‘bad news’’, that is, good news are always identified appropriately as good news, but bad news are only identified as bad news with some probability less than one. Then, one can obtain that competitive firms screen customers too much. A firm’s incentive to screen customers is given by the difference between the cost of servicing the costly customers and the price it is getting as revenue from those customers, while the social incentive is the difference between the cost of servicing the costly customers and the consumers’ valuation. As in a competitive market the price is below the consumers’ valuation, a firm’s incentive to screen customers is greater than the social incentive. If the screening device is not very good, or the social cost of servicing the costly customers is small, then it may be better not to allow firms to screen (customers have privacy) and for firms to service all customers. If rejected customers stay in the market and apply for the other firms, the situation may become worse, with even more equilibrium screening, so that no screening (privacy) is even better from a welfare point of view. Consumers can improve their situation (of too much screening) by reducing the quantity that they purchase. Another possibility is for firms to offer consumers the option of disclosing their valuation or keeping it private. McAdams (2005) considers this case, in which consumers who do not disclose their valuation pay a ‘‘sticker price,’’ while consumers who allow the firm to learn their valuation pay a fee to get a ‘‘customized price,’’ and where learning a consumer’s valuation
426
D. Fudenberg and J. M. Villas-Boas
is costly to the firms. McAdams shows that there are parameter values such that welfare can increase if the firms are required to offer the same prices to all consumers (and consumers are forbidden to reveal their valuation/give up their privacy). 5.2 Credit markets In credit markets, lenders may learn about the ability of their borrowers, their customers, to repay loans; this information can then be used by the firms in the future loans to those customers. In this case what a firm learns about its previous customers relates to the cost of providing the customer with a given contact, as opposed to the customer’s willingness to pay, which has been the focus of the work we have discussed so far. This feature is also present in other markets, such as labor markets (information about employees), rental markets (information about tenants), insurance markets (information about policy holders), and some forms of service contracts (fussy customers take more time to service). Our presentation here is cast in terms of credit markets because the existing literature has used this type of markets as main motivation. We start by discussing what happens in credit markets when lenders have private information about their own previous borrowers, and then consider the possibility and effects of lenders sharing their information. The presentation is based in large part on Pagano and Jappelli (1993), Padilla and Pagano (1997, 2000), Dell’Ariccia et al. (1999), and Dell’Ariccia and Marquez (2004).47 Some of the discussion is also related to some of the material presented in the privacy section above, in particular, Taylor (2004b). Following Dell’Ariccia et al. (1999), consider a market with two competing Lenders 1 and 2. Borrowers have to borrow $1 to invest in a project that pays R with probability y, and zero with probability 1y. Borrowers are heterogeneous on the success probability y, with cumulative distribution function G(y) (density g(y)) on [0,1]. Furthermore, the borrowers are, independent of y, in one of three groups: either they are ‘‘new’’ borrowers, and so no lender knows about the borrower’s y; or they are ‘‘old’’ borrowers from Lender 1, so that Lender 1 knows y, but this is not known by Lender 2; or they are ‘‘old’’ borrowers from Lender 2, so that Lender 2 knows y, but this is not known by Lender 1. Let l be the proportion of ‘‘new’’ borrowers (1l of ‘‘old’’ borrowers), and let ai be the proportion of ‘‘old’’ borrowers from Lender i. Dell’Ariccia et al. (1999) assume that a lender is not able to distinguish between ‘‘new’’ borrowers and ‘‘old’’ borrowers from the other lender, and that, lenders first simultaneously set interest rates ri for the borrowers for whom they do not know y, and then they set, also simultaneously, the interest rates ri y for the borrowers for 47
For a discussion of informed versus arm’s-length debt see also Rajan (1992).
Ch. 7. Behavior-Based Price Discrimination and Customer Recognition 427
whom they know y.48 The paper focuses on the analysis of this market interaction, which can be seen as the second period of a two-period model.49 Consider first the behavior of a Lender i with respect to its previous ‘‘old’’ borrowers. These borrowers have access to an offer from the other lender at an interest rate rj. In order for Lender i to attract them, it has to offer them at least an interest rate rj. The expected profitability of a borrower of type y is then yrj1. Lender i then wants only to extend credit to the borrowers that will generate positive expected profit, that is, for the borrowers with y 1=rj : Lender i expected profits from its previous ‘‘old’’ borrowers is then Z 1 ðyrj 1ÞgðyÞdy. ai ð1 lÞ 1=rj
Note that these expected profits from the lender’s previous borrowers are independent of the lender’s interest rate to the ‘‘new’’ borrowers. Consider now the profit of a Lender i from the borrowers that borrow from that lender for the first time, given interest rates (ri, rj). Lender i gets an expected profit from the ‘‘new’’ borrowers of lðri EðyÞ 1Þ if riorj, of ð1=2Þlðri EðyÞ 1Þ if ri ¼ rj, and of zero if ri>rj. The expected profits for Lender i of the ‘‘old’’ borrowers of the other lender, due to the poor quality borrowers that are denied credit by the other lender, are aj ð1 lÞ Gð1=ri Þ½ri Eðy=y 1=ri Þ 1: Because of the discontinuity of the expected profits from the ‘‘new’’ borrowers at ri ¼ rj, by standard arguments (for example, related to Varian 1980), one can show that the market equilibrium involves mixed strategies in the interest rates ri and rj. One can also show that the lender with a smaller share of the ‘‘old’’ borrowers, makes zero expected profits from its new customers, while the lender with a greater share makes positive expected profits from this type of customers. This is because the lender with a greater market share of ‘‘old’’ borrowers suffers less asymmetric information, and lends to less poor quality ‘‘old’’ borrowers than the lender with a smaller market share of the ‘‘old’’ borrowers. Dell’Ariccia et al. (1999) go on to show that this equilibrium with two lenders is exactly the same as the equilibrium with a third lender potentially entering the market, as this new lender would prefer to stay out. This is because this potential entrant cannot protect itself from the lower quality ‘‘old’’ borrowers from both firms. As the incumbent smaller market share lender makes zero-expected profits, 48 These two assumptions are as in Villas-Boas (1999), discussed in Section 3. Sharpe (1990), in the context of credit markets, and with borrowers choosing investment levels, makes the assumption that lenders make first the offers to the borrowers that they know, and then, after observing the offer policies (but not the actual offers), make offers to the borrowers that they do not know. 49 The appendix of the paper presents some analysis on the two-period model (without discussing if forward-looking borrowers would play a role), and argues, as in Sharpe (1990), that the first period competition is more intense because of the informational advantages the lenders enjoy in the second period.
428
D. Fudenberg and J. M. Villas-Boas
the new entrant would make negative profits if entering the market (have a positive market share), and prefers to stay out. We have then that the ability to recognize previous customers in credit markets leads to blockaded entry.50 Dell’Ariccia and Marquez (2004) considers a variation of the model above where only one lender has previous ‘‘old’’ borrowers, this informed lender has higher costs of funds than the competitor, and y is uniformly distributed on the segment [0,1]. The paper fully characterizes the mixedstrategy equilibrium, and analyzes how the existence of this informed lender affects the loan portfolio allocation. Greater information asymmetry leads to higher interest rates as the informed lender takes advantage of its information advantage. Furthermore, as the competitor has lower costs of funds, the informed lender concentrates more on its previous borrowers, as competing for the ‘‘new’’ borrowers requires now lower interest rates. This problem of a new firm trying to poach some of the ‘‘old’’ customers of an incumbent firm, and having to be aware of the lemons problem associated with it, is also related to auction problems when one of the bidders is better informed (as in e.g., Engelbrecht-Wiggans et al., 1983), and to competition for auditing business, when the incumbent auditor is better informed about the business risk of a client compared to a rival entrant (e.g., Morgan and Stocken, 1998). One issue that is particularly important in credit markets is what happens if the lenders exchange information about the borrowers. Pagano and Jappelli (1993) investigate this issue with two types of borrower quality, where each lender is in a different ‘‘town,’’ and learns about the credit quality of the borrowers in that town in the previous period. Some of the borrowers change towns from period to period, and there is heterogeneity on the return from the borrowers’ projects if successful. Lenders can price discriminate across three types of borrowers: the safe ‘‘old’’ borrowers, the risky ‘‘old’’ borrowers, and the ‘‘new’’ borrowers. If the interest rate to the ‘‘new’’ borrowers is too high, only the risky ‘‘new’’ borrowers apply for credit. Consider first the case in which lenders are local monopolies in their own towns. In this case, profits are decreasing in the proportion of ‘‘new’’ borrowers as the lenders have less ability to price discriminate between the types of borrowers. If there is information sharing across towns, then lenders can distinguish the types of all borrowers, and profits increase. However, the lending volume increases with information sharing if the safe ‘‘new’’ borrowers were not served in the case without information sharing, and decreases otherwise. Consider now the case of competition where lenders can offer credit to borrowers in neighboring towns, although at a cost disadvantage. ‘‘New’’ 50 Baye et al. (1992) show the existence of a continuity of asymmetric equilibria in the symmetric Varian (1980) model. It would be interesting to investigate the implications of those results for the model above when there are more than two incumbents.
Ch. 7. Behavior-Based Price Discrimination and Customer Recognition 429
borrowers are assumed to come from far away towns. In order to simplify the analysis (to get away from mixed strategy equilibria), Pagano and Jappelli (1993) assume that outside lenders make offers after the offers made by the local lenders. The paper finds that, as above, lenders are able to deter entry if given their informational advantages, and that information sharing leads to lower profits, with the greater threat from the potential entrants. The incentives for lenders to share information depend then on the monopoly effects above for information sharing, and on the competition effects against information sharing. Which effect dominates depends on their relative strength. Another potential important issue in credit markets is the possibility of borrowers exerting effort to increase the probability of success of their project. This issue is addressed in Padilla and Pagano (1997). In this case, borrowers may be concerned about exerting effort and then being taken advantage of by high interest rates from the informed lenders (hold-up problem). Padilla and Pagano suggest that lenders may be able to correct this incentive problem by committing to share their information about the borrowers with other lenders, such that the borrowers can benefit from interest rate competition. In another paper, Padilla and Pagano (2000) consider the case in which lenders cannot take advantage of their information about the borrowers because they compete away ex-ante any gains from future private information. In this case the paper argues that the lenders may still want to commit to share the borrowers default rate with other lenders as an incentive device for the borrowers to exert more effort to increase the probability of the project success. However, if the lenders share the information about the type of the borrower, the incentives to exert effort are lower than if only defaults are shared, and the borrowers exert the same level of effort as if no information were shared. 5.3 Customized pricing In some markets, competing firms may have information about the consumer preferences and price discriminate based on consumer preferences. Competition in such a setting may end up being more intense, if this leads to less differentiation in the competition for each consumer. Thisse and Vives (1988) consider this effect in the Hotelling line with two firms located at the extremes of the segment [0,1]. Suppose that consumers are uniformly distributed on this segment, and that a consumer located at x pays ‘‘transportation costs’’ tx, if buying from the firm located at 0, Firm 0, and ‘‘transportation costs’’ t(1x), if buying from the firm located at 1, Firm 1. If firms do not know the location of the consumers they have to charge a uniform price for all consumers. Let the price charged by Firm 0 be p0, and the price charged by Firm 1 be p1. Then, it is well known that the demand for Firm 0 is D0(p0, p1) ¼ (t+p1p0)/2t, and that the demand for Firm 1 is
430
D. Fudenberg and J. M. Villas-Boas
D1(p0, p1) ¼ 1D0(p0, p1). The equilibrium prices are then p0 ¼ p1 ¼ c+t (assume constant marginal costs c), and the equilibrium profit for each firm is t/2. Consider now that the firms know the location of each consumer. Then, each firm can charge a price per location x, pi(x). The price competition in each location x is like competition with a homogeneous good, where the consumer has different valuations for the product. For xr1/2 (the case of x>1/2 is symmetric) we have in equilibrium p0 ðxÞ ¼ c þ tð1 2xÞ; p1 ðxÞ ¼ c; and the consumers choose Firm 0’s product. The average price received as revenue by a firm is then c+t/2, and each firm has a profit of t/4, one half of the profit when customized prices were not possible. This result points to a general effect that competition with customized prices is more intense than competition without customized prices, if customization leads to less differentiation in the competition for each consumer. That is, competition with customized prices becomes like competition with no differentiation, in which at the equilibrium prices, an infinitesimal small price cut attracts all the demand. Variations of this result can be seen in Borenstein (1985), Holmes (1989), Corts (1998).51 For the case of competition with second-degree price discrimination see, for example, Stole (1995), Villas-Boas and Schmidt-Mohr (1999), Armstrong and Vickers (2001), Desai (2001). For a recent survey of competition with price discrimination, see Stole (2004).52 However, as noted by Armstrong (2005), more information about the consumer preferences may not necessarily lead to less differentiation and lower profits. Armstrong notes that if the additional information is about the ‘‘transportation costs’’ parameter in the traditional Hotelling model, additional information leads to significantly higher prices for the consumers with the higher transport costs; this may lead to higher equilibrium profits. One interesting extension of the variation of the Thisse and Vives model above is the case in which we allow firms to only know the locations of some of the consumers in the line (the firm’s database), and therefore, can only offer customized prices to those consumers. This case is considered in Chen and Iyer (2002). We then have that at each location some consumers are in the database of both firms, some consumers are in the database of only one of the firms, and some consumers are not in any database. The databases can be available from the firms’ internal sources or from external sources such as syndicated vendors of information.53 Chen and Iyer show that firms may choose to have not all consumers in their database as this alleviates price competition. However, it turns out that allowing firms to offer some degree of customized prices leads to higher profits than no customization at all. That is, there is an intermediate level 51
See also Katz (1984) for the case of price discrimination in monopolistic competition. See also Armstrong (2005) for a recent survey on economic models of price discrimination. 53 This can then be seen as a later period of some dynamic interaction where firms learn the complete preferences of some consumers (the consumers in the firm’s database). 52
Ch. 7. Behavior-Based Price Discrimination and Customer Recognition 431
of price customization that leads to higher profits. The intuition for why having limited databases may alleviate price competition is related to Grossman and Shapiro’s (1984), who show, in the context of uniform prices, that decreased advertising costs may reduce profits because it leads firms to increase their advertising. This increased advertising leads to more consumers that can compare prices, which leads to a greater benefit for a firm of cutting prices, and thus to lower equilibrium prices and profits. In Chen and Iyer, larger databases allow firms to do more customized pricing, which we know from Thisse and Vives, may lead to greater price competition. Ulph and Vulkan (2000) consider the incentives for firms to invest in customization capabilities under different transportation cost functions. Ulph and Vulkan (2001) discuss what happens when customization may allow a firm to offer customized products. Iyer et al. (2005) consider the effects of customized advertising (in a model similar to Grossman and Shapiro, 1984, for uniform advertising), and show that customized advertising decreases price competition.54 A related but different form of competition with price discrimination is when firms with capacity constraints advance-sell their products, possibly at a discount. Dana (1998) considers this case, and finds that in equilibrium we may have advance-selling discounts that are bought by consumers with lower valuation for the product, but that have a more certain demand.
6
Conclusion
This paper presents a summary of existing research on the effects of firms being able to recognize their previous customers, and behave differently toward them. The importance of understanding the effects of this market practice has increased in the recent past given the development of information technologies and the Internet (e.g., web–browser cookies) that allow firms to keep, gather, and process more information about their past customers.55 This increase in information has led to the proliferation of customer relationship management practices in most industries. As of now, it seems that many firms collect more information about their customers’ behavior than they are able to process. As firms get better at processing this large amount of information, the effects of customer recognition are going to become more and more important. In fact, the Internet allows also firms 54 See also Stegeman (1991) and Roy (2000). For the case of imperfect targetability see Chen et al. (2001). 55 See Rossi et al. (1996) for a discussion of available databases of purchase histories and their possible use in direct marketing. Pancras and Sudhir (2005) present an empirical application of personalization activities (for example, offering of coupons) in grocery retailing. Lewis (2005) presents an application to subscriber data of a large metropolitan newspaper of the dynamic issues in pricing using the past consumer purchase behavior.
432
D. Fudenberg and J. M. Villas-Boas
to interact more directly with their customers, and better respond to this increase in information. Most of the work until now has been on the firms’ pricing decisions (with the exception of the limited work discussed in Section 4). Firms use consumer behavior to target many other sorts of decisions, including their product offerings and communication policies. As of now we have still very little understanding of how these activities can interact with the ability of firms to recognize customers. This means that research on this problem has so far just uncovered the ‘‘tip of the iceberg,’’ and that there is much work to be done on behavior-based targeting in the future. It would also be interesting to see more empirical work testing for the results presented in this literature.56 Research to date has identified several pricing effects in both monopoly and competition. As discussed in Section 2, in monopoly, we have to account for both behavior of the firm anticipating the future gain of having more information, and the strategic behavior of consumers anticipating what firms will do in the future with their information. As discussed there, we may end up having a ‘‘ratchet effect,’’ as consumers realize that they would be hurt by revealing their information, so that they incur costs (forgo utility) to conceal their preferences. Important factors in how these forces play out include the relative discount factors of the firm and the consumers, the feasibility of the firm offering long-term contracts, the effect of new generations of consumers coming into the market, and the effect of consumer preferences changing (with positive correlation) through time. In markets with multiple firms there is the additional effect of firms poaching each other’s customers with special deals. This generates interesting strategic effects, possibly inefficient switching, and effects on the intensity of competition. In addition to the possibility of firms offering long-term contracts, and the entry of new customers (or customers changing preferences), another effect that can be important in several markets is the presence of switching costs or network externalities. Allowing firms to recognize customers raises the question of what can firms do with such information, and whether consumers should have the right to privacy in their market interactions. Furthermore, in some markets, the characteristics of consumers may affect profits directly and this may have additional effects on the functioning of the market as discussed in Section 5 in the context of credit markets. Finally, the possibility of firms recognizing their past customers interact with several market aspects that have been substantially studied in the past
56 There is already some related empirical work. See, for example, Goldberg (1996) and Guha and Wittink (1996), who show that empirical dealer discounts for new cars are a function of whether it is a first-time purchase and whether there is a trade-in.
Ch. 7. Behavior-Based Price Discrimination and Customer Recognition 433
such as customized pricing, switching costs, durable-goods markets, and bargaining. Acknowledgements We thank Mark Armstrong, Michael Baye, Assaf Ben-Shoham, Yongmin Chen, Terry Hendershott, John Morgan, Jae Nahm, Jean Tirole, and Juanjuan Zhang for insightful comments on an earlier draft of this paper, and for helpful conversations. References Acquisti, A., H.R. Varian (2005). Conditioning prices on purchase history. Marketing Science 24, 367–381. Armstrong, M. (2005). Economics models of price discrimination. Working Paper, University College London. Armstrong, M., J. Vickers (2001). Competitive price discrimination. RAND Journal of Economics 32, 579–605. Ausubel, L., R. Deneckere (1992). Durable goods monopoly with incomplete information. Review of Economic Studies 59, 795–812. Baye, M.R., D. Kovenock, C.G. de Vries (1992). It takes two-to-tango: Equilibria in a Model of Sales. Games and Economic Behavior 4, 493–510. Baron, D., D. Besanko (1984). Regulation and information in a continuing relationship. Information Economics and Policy 1, 447–470. Battaglini, M. (2005). Long-term contracting with Markovian consumers. American Economic Review 95, 637–658. Beggs, A.W. (1989). A note on switching costs and technology choice. Journal of Industrial Economics 37, 437–440. Beggs, A., P. Klemperer (1992). Multi-period competition with switching costs. Econometrica 60, 651–666. Ben-Shoham, A. (2005). Information and order in sequential trade. Working Paper, Harvard University. Borenstein, S. (1985). Price discrimination in free-entry markets. RAND Journal of Economics 16, 380–397. Bulow, J.I. (1982). Durable-goods monopolists. Journal of Political Economy 90, 314–332. Cabral, L., J.M. Villas-Boas (2005). Bertrand supertraps. Management Science 51, 599–613. Calzolari, G., A. Pavan (2005). On the optimality of privacy in sequential contracting. Journal of Economic Theory forthcoming. Caminal, R., C. Matutes (1990). Endogenous switching costs in a Duopoly model. International Journal of Industrial Organization 8, 353–373. Chen, Y. (1997). Paying customers to switch. Journal of Economics and Management Strategy 6, 877–897. Chen, P.-Y., L.M. Hitt (2005). Information Technology and Switching Costs. (this handbook). Chen, Y., G. Iyer (2002). Consumer addressability and customized pricing. Marketing Science 21, 197–208. Chen, Y., C. Narasimhan, Z.J. Zhang (2001). Individual marketing and imperfect targetability. Marketing Science 20, 23–41. Chen, Y., Z.J. Zhang (2004). The benefit of behavior-based Targeted pricing with strategic consumers. Working Paper, New York University and University of Pennsylvania.
434
D. Fudenberg and J. M. Villas-Boas
Conlisk, J., E. Gerstner, J. Sobel (1984). Cyclic pricing by a durable goods monopolist. Quarterly Journal of Economics 99, 489–505. Corts, K.S. (1998). Third-degree price discrimination in oligopoly: all-out competition and strategic commitment. RAND Journal of Economics 29, 306–323. D’Aspremont, C., J. Gabszewicz, J.-F. Thisse (1979). On Hotelling’s stability in competition. Econometrica 17, 1145–1151. Dana, J. (1998). Advance-purchase discounts and price discrimination in competitive markets. Journal of Political Economy 106, 395–422. Dell’Ariccia, G., E. Friedman, R. Marquez (1999). Adverse selection as a barrier to entry in the banking industry. RAND Journal of Economics 30, 515–534. Dell’Ariccia, G., R. Marquez (2004). Information and bank credit allocation. Journal of Financial Economics 72, 185–214. Deneckere, R., P. McAfee (1996). Damaged goods. Journal of Economics and Management Strategy 5, 149–174. Desai, P. (2001). Quality segmentation in spatial markets: when does cannibalization affect product line design?. Marketing Science 20, 265–283. Dobos, G. (2004). Poaching in the presence of switching costs and network externalities. Working Paper, University of Toulouse. Doganoglu, T., L. Grzybowski (2004). Dynamic duopoly competition with switching costs and network externalities. Working Paper, University of Munich. Economides, N. (1986). Minimal and maximal product differentiation in Hotelling’s duopoly. Economics Letters 21, 67–71. Ellison, G., D. Fudenberg (2000). The neo-luddite’s lament: excessive upgrades in the software industry. RAND Journal of Economics 31, 253–272. Engelbrecht-Wiggans, R., P. Milgrom, R. Weber (1983). Competitive bidding and proprietary information. Journal of Mathematical Economics 11, 161–169. Esteves, R.B. (2004). Pricing under customer recognition. Working Paper, Oxford University. Farrell, J., C. Shapiro (1988). Dynamic competition with switching costs. RAND Journal of Economics 19, 123–137. Farrell, J., C. Shapiro (1989). Optimal contracts with lock-in. American Economic Review 79, 51–68. Farrell, J., P. Klemperer (2004). Coordination and lock-in: competition with switching costs and network effects. Handbook of Industrial Organization, forthcoming. Freixas, X., R. Guesnerie, J. Tirole (1985). Planning under incomplete information and the Ratchet effect. Review of Economic Studies 52, 173–191. Fudenberg, D., D.K. Levine (1989). Reputation and equilibrium selection in games with a patient player. Econometrica 57, 759–778. Fudenberg, D., J. Tirole (1983). Sequential bargaining with incomplete information. Review of Economic Studies 50, 221–247. Fudenberg, D., J. Tirole (1984). The fat cat effect, the puppy dog ploy and the lean and hungry look. American Economic Review, Papers and Proceedings 74, 361–368. Fudenberg, D., J. Tirole (1991). Game Theory. The MIT Press, Cambridge, Massachusetts. Fudenberg, D., J. Tirole (1998). Upgrades, trade-ins, and buybacks. RAND Journal of Economics 29, 238–258. Fudenberg, D., J. Tirole (2000). Customer poaching and brand switching. RAND Journal of Economics 31, 634–657. Goldberg, P.K. (1996). Dealer price discrimination in new car purchases: evidence from the consumer expenditure survey. Journal of Political Economy 104, 622–654. Grossman, G.M., C. Shapiro (1984). Informative advertising with differentiated products. Review of Economic Studies 51, 63–81. Guha, R., D. Wittink (1996). The relationship between replacement time, brand loyalty and transaction price for automobiles. Working Paper, Cornell University. Hart, O.D., J. Tirole (1988). Contract renegotiation and coasian dynamics. Review of Economic Studies 55, 509–540.
Ch. 7. Behavior-Based Price Discrimination and Customer Recognition 435 Hermalin, B.E., M.L. Katz (2006). Privacy, property rights & efficiency: the economics of privacy as secrecy. Quantitative Marketing and Economics 4, 209–239. Hirshleifer, J. (1971). The private and social value of information and the reward to inventive activity. American Economic Review 61, 561–574. Holmes, T.J. (1989). The effects of third-degree price discrimination in oligopoly. American Economic Review 79, 244–250. Hui, K.-L., I.P.L. Png (2005). The Economics of Privacy. this handbook. Iyer, G., D. Soberman, J.M. Villas-Boas (2005). The targeting of advertising. Marketing Science 24, 461–476. Katz, M.L. (1984). Price discrimination and monopolistic competition. Econometrica 52, 1453–1472. Kennan, J. (2001). Repeated bargaining with persistent private information. Review of Economic Studies 68, 719–755. Kim, B.D., M. Shi, K. Srinivasan (2001). Reward programs and tacit collusion. Marketing Science 20, 99–120. Klemperer, P. (1987a). The competitiveness of markets with switching costs. RAND Journal of Economics 18, 138–150. Klemperer, P. (1987b). Markets with consumer switching costs. Quarterly Journal of Economics 102, 375–394. Klemperer, P. (1995). Competition when consumers have switching costs: an overview with applications to industrial organization, macroeconomics and international trade. Review of Economic Studies 62, 515–539. Kreps, D., P. Milgrom, J. Roberts, R. Wilson (1982). Rational cooperation in the finitely repeated prisoner’s dilemma. Journal of Economic Theory 27, 245–252. Laffont, J.-J., J. Tirole (1990). Adverse selection and renegotiation in procurement. Review of Economic Studies 57, 597–625. Lee, I.H., J. Lee (1994). Durable goods monopoly under technological innovation. Working Paper, University of Southampton. Levinthal, D.A., D. Purohit (1989). Durable goods and product obsolescence. Marketing Science 8, 35–56. Lewis, M. (2005). Research note: a dynamic programming approach to customer relationship pricing. Management Science 51, 986–994. McAdams, D. (2005). A theory of sticker prices. Working Paper, M.I.T. Morgan, J., P. Stocken (1998). The effects of business risk on audit pricing. Review of Accounting Studies 3, 365–385. Mussa, M., S. Rosen (1978). Monopoly and product quality. Journal of Economic Theory 18, 301–317. Nahm, J. (2004). Durable-goods monopoly with endogenous innovation. Journal of Economics and Management Strategy 13, 303–320. Nilssen, T. (1992). Two kinds of consumer switching costs. RAND Journal of Economics 23, 579–589. Padilla, J., M. Pagano (1997). Endogenous communications among lenders and entrepreneurial incentives. Review of Financial Studies 10, 205–236. Padilla, J., M. Pagano (2000). Sharing default information as a borrower incentive device. European Economic Review 44, 1951–1980. Pagano, M., T. Jappelli (1993). Information sharing in credit markets. Journal of Finance 48, 1693–1718. Pancras, J., K. Sudhir (2005). The personalization services firm: what to sell, whom to sell to and for how much? Working Paper, University of Connecticut. Rajan, R.G. (1992). Insiders and outsiders: the choice between informed and arm’s-length debt. Journal of Finance 47, 1367–1400. Rossi, P.E., R.E. McCulloch, G.M. Allenby (1996). The value of purchase history data in target marketing. Marketing Science 15, 321–340. Roy, S. (2000). Strategic segmentation of a market. International Journal of Industrial Organization 18, 1279–1290. Schmidt, K. (1993). Commitment through incomplete information in a simple repeated bargaining game. Journal of Economic Theory 60, 114–139.
436
D. Fudenberg and J. M. Villas-Boas
Schwartz, M. (1997). Telecommunications reform and the United States: promises and pitfalls, in: P.J.J. Welfens, G. Yarrow (eds.), Telecommunications and Energy in Systemic Transformation, Springer, New York. Shaffer, G., Z.J. Zhang (1995). Competitive coupon targeting. Marketing Science 14, 395–416. Shaffer, G., Z.J. Zhang (2000). Pay to switch or pay to stay: preference-based price discrimination in markets with switching costs. Journal of Economics and Management Strategy 9, 397–424. Sharpe, S. (1990). Asymmetric information, bank lending and implicit contracts: a stylized model of customer relationships. Journal of Finance 45, 1069–1087. Sobel, J. (1984). The timing of sales. Review of Economic Studies 51, 353–368. Sobel, J. (1991). Durable goods monopoly with entry of new consumers. Econometrica 59, 1455–1485. Stegeman, M. (1991). Advertising in competitive markets. American Economic Review 81, 210–223. Stokey, N. (1979). Intertemporal price discrimination. Quarterly Journal of Economics 93, 355–371. Stokey, N. (1981). Rational expectations and durable goods pricing. Bell Journal of Economics 12, 112–128. Stole, L. (1995). Nonlinear pricing and oligopoly. Journal of Economics and Management Strategy 4, 529–563. Stole, L. (2004). Price discrimination in competitive environments. Handbook of Industrial Organization, forthcoming. Taylor, C.R. (2003). Supplier surfing: price-discrimination in markets with repeat purchases. RAND Journal of Economics 34, 223–246. Taylor, C.R. (2004a). Consumer privacy and the market for customer information. RAND Journal of Economics 35, 631–650. Taylor, C.R. (2004b). Privacy and information acquisition in competitive markets. Working Paper, Duke University. Thisse, J.-F., X. Vives (1988). On the strategic choice of spatial price policy. American Economic Review 78, 122–137. Tirole, J. (1988). The Theory of Industrial Organization. The MIT Press, Cambridge, Massachusetts. Ulph, D., N. Vulkan (2000). Electronic commerce and competitive first-degree price discrimination. Technical Report, University College, London. Ulph, D., N. Vulkan (2001). E-commerce, mass customization and price discrimination. Technical Report, University College, London. Varian, H. (1980). A model of sales. American Economic Review 70, 651–659. Villanueva, J., P. Bhardwaj, Y. Chen, S. Balasubramanian (2004). Managing customer relationships: should managers really focus on the long term? Working Paper, University of California, Los Angeles. Villas-Boas, J.M. (1999). Dynamic competition with customer recognition. RAND Journal of Economics 30, 604–631. Villas-Boas, J.M., U. Schmidt-Mohr (1999). Oligopoly with asymmetric information: differentiation in credit markets. RAND Journal of Economics 30, 375–396. Villas-Boas, J.M. (2004). Price cycles in markets with customer recognition. RAND Journal of Economics 35, 486–501. von Weizsa¨cker, C.C. (1984). The cost of substitution. Econometrica 52, 1085–1116. Waldman, M. (1996). Planned obsolescence and the R&D decision. RAND Journal of Economics 27, 583–595. Wathieu, L. (2004). Marketing and the privacy concern. Working Paper, Harvard University. Zhang, J. (2005). Dynamic segmentation and product design. Working Paper, University of California, Berkeley.
Terrence Hendershott, Ed., Handbooks in Information Systems, Vol. 1 Copyright r 2006 by Elsevier B.V.
Chapter 8
Information Technology and Switching Costs
Pei-yu Chen Tepper School of Business, Carnegie Mellon University, 5000 Forbes Ave, Pittsburgh, PA 15213, USA
Lorin M. Hitt University of Pennsylvania, Wharton School, 571 Jon M. Huntsman Hall, Philadelphia, PA 19104, USA
Abstract Switching costs, the real or perceived cost of changing product providers, has become increasingly important in information-intensive businesses. Reduced search and transactions costs have made it possible for customers to more readily evaluate alternative suppliers, increasing switching. At the same time, the high fixed cost structure of customer acquisition and operations in information-intensive businesses increases the impact of customer switching on profitability. Fortunately, the increasing availability of information and better information technology has enabled firms to have greater understanding and control over switching costs, raising the ability of firms to manage switching costs. This paper presents a formal definition of switching costs, surveys the literature on information technology (IT) and switching costs, and discusses various ways firms may use to influence consumer switching costs. A framework for managing switching costs and customer retention is also provided along with a measurement framework for estimating the extent of customer lock-in.
437
438 1
P.-y. Chen and L. M. Hitt
Introduction
Switching costs, the perceived disutility a customer would experience from changing product or service providers, play a large and increasing role in competition and strategy in IT markets and other information-intensive businesses. Many of these markets are characterized by a high fixed cost and a high cost of customer acquisition, but a low marginal cost of product delivery. Profitability in these markets is therefore driven by the size of the customer base and customer retention, which is at least partly determined by switching costs. As noted by Shapiro and Varian (1999) ‘‘You just cannot compete effectively in the information economy unless you know how to identify, measure, and understand switching costs and map strategy accordingly’’. (p. 133)
Switching costs can arise from a number of sources including contractual provisions, product characteristics, and the amount of information customers have about product alternatives. In addition, many informationbased products exhibit network effects, where the value of adopting a technology increases with the number of other users adopting the same technology or the number of complementary products available. These network effects can also create significant barriers from switching to an alternative technology or network (Shapiro and Varian, 1999). To the extent that networks can exist within firms, such as when there are benefits from product compatibility among investments in the same or complementary technologies within the firm (Kat and Shapiro, 1985, 1994), these within-firm network effects can also act as switching costs (e.g., Greenstein, 1993). While all markets can be affected by switching costs, there are several unique issues regarding customer retention in information or IT-based products, which we will collectively refer to as ‘‘information-intensive’’ markets. First, information-intensive markets often have fewer market inefficiencies that often represent the principal obstacle for customer switching. For instance, in markets with high search costs, consumers become ‘‘locked-in’’ because it is not efficient to search for alternatives. This is less likely to be relevant in markets where product information is readily available. Reduced information or transaction costs can generally lower the cost of terminating or initiating a customer relationship, also affecting switching costs. Second, information-intensive products often have significant compatibility issues. On the one hand, interoperability of technology products can be extremely important, especially for complex technologies such as telecommunications gear or for enabling technologies such as operating systems or computer languages. However, the market has recognized this leading to a strong trend toward standardization, allowing users to mixand-match products from different vendors. Thus, standards and the decisions associated with determining standardization strategies are often of special importance in information intensive industries. Finally, technologies
Ch. 8. Information Technology and Switching Costs
439
employed in many information-intensive businesses often enable firms to obtain better information about their customers which is essential to any attempt to manage customer retention. For instance, ‘‘clickstream’’ data can enable the measurement of switching costs (see Section 5), and customer relationship management systems (CRM) can allow firms to tailor product offerings to enhance retention. The literature on switching cost has been most extensively developed in economics and marketing. In economics, the emphasis has been on understanding the wide range of potential competitive implications of marketwide switching costs, principally from a theoretical perspective. For instance, switching costs have been tied to pricing, profitability, entry decisions, the formation of standards, and other issues considered in the economics of industrial organization (see e.g. Klemperer, 1995; Farrell and Klemperer, 2004). Some of the economic literature has made this more specific, such as switching costs due to product compatibility or network externalities (Katz and Shapiro, 1985) that are common in software markets (Bresnahan, 2001). There also have been a few empirical studies examining the market-wide implications of switching costs for credit cards (Ausubel, 1991; Calem and Mester, 1995; Stango, 2002), bank loans (Kim et al., 2003), bank accounts (Shy, 2002), cigarettes (Elzinga and Mills, 1998), breakfast cereal (Shum, 2004), and toll-free numbers (Viard, 2003). However, this literature has principally focused on switching costs that affect all providers in a market similarly (what we term ‘‘exogneous switching costs’’) rather than placing emphasis on switching cost variations that arise from specific actions of individual firms (‘‘endogenous switching costs’’). While not typically labeled switching costs as such, marketing scholars have also worked to understand the implications of switching cost from a customer behavior standpoint, emphasizing issues such as brand loyalty, repeat purchase behavior, and customer retention. Much of the research here treats loyalty as an intrinsic characteristic of consumers and examines the implications of variations across consumers in loyalty. For example, Raju et al. (1990) study how exogenous brand loyalty affects firms’ pricing (or promotional) strategies. Recent research has also examined how loyalty impacts equilibrium prices in the setting where an intermediary (‘‘clearinghouse’’) provides competitive product and price information and firms are faced with a strategic decision on whether or not to target price sensitive ‘‘shoppers’’ or ‘‘switchers’’ (Narasimhan, 1988; Baye, et al., 2004; Baye and Morgan, 2004). Another strand of the literature, mainly empirical, focuses on the identification of loyal customers and switchers by constructing empirical models that can predict customer behavior (see e.g., Grover and Srinivasan, 1987). Research in information systems has mostly followed the economic approach, but tends to be more application focused. For instance, researchers have considered switching cost measurement for online businesses (Chen and Hitt, 2002b), the role of product compatibility in creating switching
440
P.-y. Chen and L. M. Hitt
costs in the market for telecommunications equipment (Forman and Chen, 2005; Chen and Forman, 2006), and the impact of switching costs and network effects on competition between online, traditional, and hybrid firms (Viswanathan, 2005). Since comprehensive reviews of the switching costs literature already exist from both the economics (see Klemperer, 1995; Farrell and Klemperer, 2004) and marketing perspectives (Jacoby and Chestnut, 1978; Seetharam et al., 1999), our emphasis in this review will be on applications to information systems and technology issues. However, since these applications involve a blend of both economic and marketing approaches, we will also present a formal approach integrating these perspectives (drawn heavily from Chen, 2002). The remainder of this review is organized as follows. Section 2 presents a formal definition of switching costs in an economic choice framework, which helps clarify some general points about switching costs. We review some major theoretical and empirical studies of IT and switching cost as well as discuss related issues in Section 3. Section 4 points out the importance of endogenous switching costs in high-tech and information-intensive markets and discusses some instruments for influencing switching costs. We then provide a general framework for managing customer retention and consumer switching costs and a measurement framework for estimating the magnitudes of switching costs in Section 5, followed by general conclusions and a brief discussion on future research issues in Section 6.
2
Switching cost: definition and measurement issues
Switching cost is typically defined as the disutility a customer experiences in switching products or product providers.1 These costs can include the cost to terminate an existing relationship, start a new relationship, and any other costs, either explicit (e.g., contractual penalties) or implicit (e.g., risk aversion and uncertainty about new products) that makes deviating from past product choice more costly than staying with a previous choice. To formally state this definition and explore its limitations we must introduce some notation. For this discussion we assume that consumers’ preferences can be represented by a utility function that depends on the consumer and attributes of products. Let uji to refer to the utility customer i receives from purchasing product j and use the notation Sjki to capture the switching cost of customer i switching from product j to product k. Here and throughout, we will assume that switching costs are small relative to 1 To simply exposition we will refer to ‘‘products’’ as a product purchased from a particular product provider. Unless necessary, we will not make a separate distinction between the product and the product provider or between products and services.
Ch. 8. Information Technology and Switching Costs
441
wealth so we can treat switching cost and utility as additively separable, and will focus on situation where there is unit demand.2 In a specific interval of time in a market, there are always new customers who enter the market for the first time; consumers who have adopted one or more existing products, and consumers who choose to abandon a product. We allow for the existence of costs for first adoption or market exit in our notation by defining a null product (+), which is also in the choice set. Although entrance and exist costs are not typically defined as switching cost in most models, these costs can also behave as switching costs. Following this setup, S+k can be interpreted as the adoption (or entrance) cost of product k, while Sk+ are the exit costs. If a consumer does not purchase a product, we normalize utility to zero (u+i ¼ 0). We are interested in the implications of switching costs for customer behavior, as this is the foundation for all the other competitive implications. The simplest illustration of how switching costs affect choices requires a market with two consumers (iA[A,B]), two goods (jA[X,Y]), and two periods (t ¼ 0,1). Let customer A represent a customer who has previously acquired product X, while customer B has not purchased any product in this market previously (period 0). For each customer, there are three choices in period 1, purchase X, purchase Y or exit the market (represented as purchasing the null product +). Utilities for different choices in period 1 are given by3 Action
Consumer A
Consumer B
Purchase X Purchase Y
uA X
uBX SB+X
A A uA Y S XY S +Y
No purchase
SA X+
uBY SB+Y 0
While this formulation appears somewhat complex, the literature has adopted a convention where initial adoption and abandonment costs (S+ki and Sj+i) respectively are ignored and switching costs as assumed constant across products and consumers (Sijk ¼ S 8i; j; k; j; ka+). While ignoring adoption and abandonment costs is probably reasonable in a wide variety of settings, the assumption of constant switching costs across all choices can be restrictive. In particular, a key component of ‘‘customer retention strategy’’ is to create firm-specific switching costs or to identify customers who have 2 These assumptions are common in switching cost models. With unit demand, we avoid issues of purchase quantity for a single product as well as the possibility that multiple products are purchased. The same insights for the unit demand case generally apply to the more general case, but the notation becomes considerably more cumbersome. 3 The separation of utility and switching cost is reasonably provided there are no significant wealth effects (i.e., consumers disutility of switching does not depend on their existing wealth).
442
P.-y. Chen and L. M. Hitt
unusually high switching costs. For instance, in online brokerage the firm can charge a fee to close an account or transfer assets, offer a subsidy for new customers transferring in from specifically named competitors, offer special programs to target ‘‘buy and hold’’ investors who have large portfolios that are cumbersome to move among brokers.4 Thus, the assumption of constant switching cost rules out strategic choices that enable firms to modify their own switching costs, an issue we will return to later in this review. Nonetheless, imposing these assumptions yields the classic switching cost analysis: Action
Consumer A
Consumer B
Purchase X Purchase Y No purchase
uA X uA YS 0
uBX uBY 0
For additional insights into this framework it is useful to consider product choice behavior in a discrete choice framework (McFadden, 1974). Without introducing extraneous technical details at this point, this formulation argues that the probability of choosing product j is proportional to utility.5 Take consumer A, who is an existing customer of product X. The A utilities she derives from purchasing X, Y and nothing are uA X, uY-S and 0, respectively. If she has not purchased any product before, the utilities she A gets from X, Y and nothing would be uA X , uY and 0. Given these two utility sets, the alternative product (Y) is relatively unattractive to consumer A in period 1 because of their prior purchase of product X, even though the underlying products are the same before and after consumer A adopts X. In another words, customers are more likely to stay with an existing product since it provides greater utility relative to other non-adopted alternatives, even if the alternative product is identical in terms of non-switching cost utility (u). This is why switching cost is typically associated with other informal notions such as ‘‘stickiness’’, ‘‘lock-in’’, ‘‘customer retention’’ or increasing ‘‘repeat purchase’’ propensity. It should be noted that even the more general framework embeds an assumption about customer behavior that can be important: it assumes that customers do not make choices based on perceived future switching costs (i.e., consumers are myopic regarding future switching events). Most insights about switching cost and competition continue to hold if customers have 4 Another example of this behavior is a ‘‘competitive upgrade’’ that is common in software markets where a firm will subsidize customers of other firms products to switch. 5 The simplest case is the logit model where the error term is independently and identically distributed across products and consumers with the ‘‘extreme value’’ distribution (i.e., prob. ð Þ ¼ ee ; where NoeoN). The choice probability of product j out of a larger set of size J is then given by: J P ul uj e : pj ¼ e
l¼1
Ch. 8. Information Technology and Switching Costs
443
rational expectations about future switching behavior (Farrell and Klemperer, 2004) or share a common expectation of the probability of future switching (see, Chen, 2002, Chapter 2). However, it can become important if we are interested in tradeoffs between customer acquisition and retention. High switching cost might favor customer retention but deter initial purchase if consumers are fully rational; there is no such tradeoff if customers are myopic and ignore future switching. Since firms can benefit from encouraging customers to be myopic about future switching, this may explain why contractual penalties for switching are often written ‘‘in small print’’ on new purchase contracts. The general tradeoff between acquisition, retention, and attrition has been discussed in the literature (Chen and Hitt, 2002b), although the specific issue of customers’ perceptions of switching cost has not been fully explored and is an opportunity for future research. This simple framework illustrates why it can be difficult to separate out the effects of product quality (increasing utility) or creating lock-in (increasing switching costs) in empirical data since they can behave in similar ways. Improving the quality of your product increases the utility of your product relative to others. Raising switching costs lowers the utility of other products for your existing customers. Thus, a firm can be successful at retaining customers either because they offer a superior product (at least for a specific set of consumers), or because they have high switching costs. Much of the managerial literature on customer retention uses retention rate as a measure of customer loyalty or consumer switching costs and does not make a distinction between switching cost and quality in driving retention (see. e.g., Reichheld and Schefter, 2000). This is perfectly reasonable if one is trying to predict future customer retention or compare retention across firms; it becomes problematic when one in interested in the drivers of customer retention or how investments in specific practices can improve (or decrease) retention. An excellent product can have poor sales if customers face high switching costs and most customers have already adopted alternatives. Similarly, a poor product can show excellent customer retention if switching costs are high and switching costs may not matter at all during the initial introduction of a completely new product since there are no existing customers. The difficulty in separating out switching cost from other drivers of product choice was highlighted in economic analysis of the Microsoft antitrust case. The key question was whether Microsoft’s dominant position is due to superior quality, switching costs, or anti-competitive behavior (see Liebowitz and Margolis, 1999). While being successful due to quality or switching costs are both legal, the quality explanation is welfare promoting, while the switching cost explanation may not be and might warrant some sort of market intervention if it were acquired through or leveraged by other anti-competitive behavior. We will discuss the distinction between product utility and switching in more detail in Section 5 and provide a framework to distinguish the two effects.
444 3
P.-y. Chen and L. M. Hitt
Switching costs, competition, and firm strategy
3.1 Switching costs and competition Most of the early literature on switching costs considers the impact or the consequences of (exogenous) switching costs. The usual analysis proceeds with exogenously determined (and usually symmetric) switching and proceeds to examine how switching costs affect pricing, entry, and firm profitability under different market conditions. Klemperer (1995) and Farrell and Klemperer (2004) provide extensive surveys on these issues from economic perspective, so we would only briefly summarize the general results pertaining to switching costs and competition in this section. Consumer switching costs provide firms’ market power over their existing customers enabling them to charge a price premium (as high as S) or strategically price to deter switching, even to superior alternatives. This result suggests that it makes sense for firms to invest to acquire new customers, even incurring losses in the process, because they may charge higher prices over their existing customers for repeat purchases (this has been termed ‘‘bargains-then-ripoffs’’). More broadly, in markets with switching costs, firms face tradeoffs between harvesting profits by charging higher prices over their existing customers and investing in market share by charging lower prices to acquire new customers who will be valuable repeat purchasers in the future (termed harvesting vs. investing) (Klemperer, 1995). The general observation is that in otherwise competitive markets with switching costs, firms have additional strategic options that can lead to higher prices and greater profits. Switching costs can also act as a deterrent to entry, or alter the adoption rate of new technologies because they decrease the value of new products in markets where most customers already use an existing product. This result suggests that, when consumers’ switching costs are high, a vendor with initial advantages in a market may be able to parlay its advantage into a larger, lasting one (Katz and Shapiro, 1994). In addition, some authors have noted that some results about single-product competition over many periods with switching costs can carry over to multi-product competition in a single period when there exist consumer shopping costs or cross-product switching costs (Klemperer and Padilla, 1997; Farrell and Klemperer, 2004; Forman and Chen, 2005). These results suggest that a firm with larger product line has a strategic advantage over a firm offering fewer products in the presence of shopping costs and cross-product switching costs. 3.2 Endogenous switching costs An important feature of much of the literature on switching costs is the dual assumptions of symmetry (all firms face the same switching costs) and exogeneity (firms cannot through their own actions affect switching costs),
Ch. 8. Information Technology and Switching Costs
445
which we collectively refer to as ‘‘exogenous switching costs’’. However, the important role switching cost can play in competition strongly suggests that firms can benefit from investments or actions that affect customer switching costs (‘‘endogenous switching costs’’). For instance, a firm will prefer switching costs from but not to its product if possible. The role of endogenous switching costs is gaining increasing interest in the literature. Caminal and Matutes (1990) show that firms may wish to commit to offering repeat-purchase coupons before competing in a twoperiod duopoly in order to soften competition. An incumbent firm will also have an incentive to create contractual switching costs through the use of long-term contracts to lock-in their customers (Fudenberg and Tirole, 2000). In addition, a firm with large installed base may also prefer a technology design that is incompatible with other firms, while a smaller firm will prefer compatibility with the incumbent firm (Katz and Shapiro, 1994). Furthermore, given that a broad product line offers an advantage when there are consumers’ shopping costs or cross-product switching costs, a firm may strategically invest to increase its product breadth or to manipulate cross-product switching costs so that consumers prefer to buy multiple products from the same vendor (Farrell and Klemperer, 2004). Similarly, Baye and Morgan (2004) show that each firm has a unilateral incentive to implement costly loyalty programs, even though it results in lower equilibrium profits for all firms. Demirhan et al. (2004) also show that in the presence of rapid technological advances and declining IT costs, which may provide the late entrant a cost advantage, an early entrant will have stronger incentive to offer products that impose significant switching costs on consumers to mitigate the damage from a decline in IT cost and preserve its first mover advantage. 3.3 Switching costs in information-intensive markets As strategic investments in switching costs become more important, a first step in understanding how firms might be able to incorporate switching costs into competitive strategy is to understand the sources of switching costs in actual markets. Most of the theory literature in economics do not distinguish between different sources of switching costs in their models or just focus on a specific form of switching costs. However, Klemperer (1995) in his classic review of switching costs lists some of the major sources of switching cost identified in economic models including: the need for compatibility with existing equipment, transaction costs of switching suppliers, the cost of learning to use new brands, uncertainty about the quality of untested brands (e.g., search costs), discount coupons and similar devices (e.g., loyalty programs), psychological costs of switching, or non-economic ‘‘brand loyalty’’. Several forms of these switching costs are especially relevant to information products and the digital marketplace and warrant further discussion. We build upon Klemperer’s categorization of switching
446
P.-y. Chen and L. M. Hitt
costs and discuss these switching costs in the context of high-tech and information-intensive markets. Search costs. These are the costs consumers must incur to locate an alternative seller (Stiglitz, 1989). While search costs affect even consumers’ initial purchases, it can act just like consumer switching costs when a consumer has better information on her current supplier, but not other suppliers (Schmalensee, 1982; Moshkin and Shacher, 2000; Villas-Boas, 2004). When product or price information is costly to acquire and consumers have imperfect information about alternative products and prices, it may be optimal for consumer to engage in limited search or not to search at all, remaining with a current supplier. This is particularly problematic for differentiated product markets where the costs of acquiring sufficient information about product alternatives and the cost of consumers processing of this information can be high. Much of the literature on competition and the Internet suggest that the Internet lowers search costs and thus potentially increases switching (Bakos, 1997). However, there is some contrary evidence to this notion. Numerous empirical studies suggest that consumer search costs remain high in electronic markets or at least customers behave as if search costs are high (see the survey by Baye et al., 2005). Transactions costs. These are the costs required to initiate a new relationship, and in some cases to terminate an existing relationship. These costs may include shopping costs, transportation costs, or costs associated with opening a new account or closing an existing account. These costs contribute to S+k (adoption costs) or Sk+ (exit costs) introduced in Section 2. Transactions costs reduce switching even when information is costless. For example, when transaction costs are high, consumers may prefer to have all their transactions done with the same vendor, in order to save on shopping costs. The succes
