CONTENTS SRIHARI GOVINDAN AND ROBERT WILSON: On Forward Induction. . . . . . . . . . . . . . . . . . . . . . .
Public vs. Private Offers in the Market for Lemons . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . RON SIEGEL: All-Pay Contests . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . LANCE FORTNOW AND RAKESH V. VOHRA: The Complexity of Forecast Testing . . . . . . . . . . GARY CHAMBERLAIN AND MARCELO J. MOREIRA: Decision Theory Applied to a Linear Panel Data Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . HIROYUKI KASAHARA AND KATSUMI SHIMOTSU: Nonparametric Identification of Finite Mixture Models of Dynamic Discrete Choices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . LARS PETER HANSEN AND JOSÉ A. SCHEINKMAN: Long-Term Risk: An Operator Approach JONATHAN L. BURKE: Virtual Determinacy in Overlapping Generations Models . . . . . . . . JOHN R. CONLON: Two New Conditions Supporting the First-Order Approach to Multisignal Principal–Agent Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
1
JOHANNES HÖRNER AND NICOLAS VIEILLE:
NOTES AND COMMENTS: BERNARD SINCLAIR-DESGAGNÉ: Ancillary Statistics in Principal–Agent Models . . . . SÍLVIA GONÇALVES AND NOUR MEDDAHI: Bootstrapping Realized Volatility . . . . . . .
29 71 93 107 135 177 235 249 279 283
BIRGIT HEYDENREICH, RUDOLF MÜLLER, MARC UETZ, AND RAKESH V. VOHRA: Char-
acterization of Revenue Equivalence . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Corrigendum to “Existence and Uniqueness of Solutions to the Bellman Equation in the Unbounded Case” . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
307
317
ANNOUNCEMENTS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . FORTHCOMING PAPERS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . REPORT OF THE SECRETARY . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . REPORT OF THE TREASURER . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . REPORT OF THE EDITORS 2007–2008 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ECONOMETRICA REFEREES 2007–2008 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . REPORT OF THE EDITORS OF THE MONOGRAPH SERIES . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . SUBMISSION OF MANUSCRIPTS TO THE ECONOMETRIC SOCIETY MONOGRAPH SERIES . . . . . . . . . .
319 325 327 335 341 347 357 361
JUAN PABLO RINCÓN-ZAPATERO AND CARLOS RODRÍGUEZ-PALMERO:
VOL. 77, NO. 1 — January, 2009
An International Society for the Advancement of Economic Theory in its Relation to Statistics and Mathematics Founded December 29, 1930 Website: www.econometricsociety.org EDITOR STEPHEN MORRIS, Dept. of Economics, Princeton University, Fisher Hall, Prospect Avenue, Princeton, NJ 08544-1021, U.S.A.;
[email protected] MANAGING EDITOR GERI MATTSON, 2002 Holly Neck Road, Baltimore, MD 21221, U.S.A.; mattsonpublishingservices@ comcast.net CO-EDITORS DARON ACEMOGLU, Dept. of Economics, MIT, E52-380B, 50 Memorial Drive, Cambridge, MA 021421347, U.S.A.;
[email protected] STEVEN BERRY, Dept. of Economics, Yale University, 37 Hillhouse Avenue/P.O. Box 8264, New Haven, CT 06520-8264, U.S.A.;
[email protected] WHITNEY K. NEWEY, Dept. of Economics, MIT, E52-262D, 50 Memorial Drive, Cambridge, MA 021421347, U.S.A.;
[email protected] WOLFGANG PESENDORFER, Dept. of Economics, Princeton University, Fisher Hall, Prospect Avenue, Princeton, NJ 08544-1021, U.S.A.;
[email protected] LARRY SAMUELSON, Dept. of Economics, Yale University, New Haven, CT 06520-8281, U.S.A.;
[email protected] HARALD UHLIG, Dept. of Economics, University of Chicago, 1126 East 59th Street, Chicago, IL 60637, U.S.A.;
[email protected] ASSOCIATE EDITORS YACINE AÏT-SAHALIA, Princeton University JOSEPH G. ALTONJI, Yale University JAMES ANDREONI, University of California, San Diego DONALD W. K. ANDREWS, Yale University JUSHAN BAI, New York University MARCO BATTAGLINI, Princeton University PIERPAOLO BATTIGALLI, Università Bocconi DIRK BERGEMANN, Yale University MICHELE BOLDRIN, Washington University in St. Louis VICTOR CHERNOZHUKOV, Massachusetts Institute of Technology J. DARRELL DUFFIE, Stanford University JEFFREY ELY, Northwestern University LARRY G. EPSTEIN, Boston University HALUK ERGIN, Washington University in St. Louis FARUK GUL, Princeton University JINYONG HAHN, University of California, Los Angeles PHILIP A. HAILE, Yale University PHILIPPE JEHIEL, Paris School of Economics YUICHI KITAMURA, Yale University PER KRUSELL, Princeton University and Stockholm University OLIVER LINTON, London School of Economics
BART LIPMAN, Boston University THIERRY MAGNAC, Toulouse School of Economics GEORGE J. MAILATH, University of Pennsylvania DAVID MARTIMORT, IDEI-GREMAQ, Université des Sciences Sociales de Toulouse STEVEN A. MATTHEWS, University of Pennsylvania ROSA L. MATZKIN, University of California, Los Angeles LEE OHANIAN, University of California, Los Angeles WOJCIECH OLSZEWSKI, Northwestern University ERIC RENAULT, University of North Carolina PHILIP J. RENY, University of Chicago JEAN-MARC ROBIN, Université de Paris 1 and University College London SUSANNE M. SCHENNACH, University of Chicago UZI SEGAL, Boston College CHRIS SHANNON, University of California, Berkeley NEIL SHEPHARD, Oxford University MARCIANO SINISCALCHI, Northwestern University JEROEN M. SWINKELS, Washington University in St. Louis ELIE TAMER, Northwestern University IVÁN WERNING, Massachusetts Institute of Technology ASHER WOLINSKY, Northwestern University
EDITORIAL ASSISTANT: MARY BETH BELLANDO, Dept. of Economics, Princeton University, Fisher Hall, Princeton, NJ 08544-1021, U.S.A.;
[email protected] Information on MANUSCRIPT SUBMISSION is provided in the last two pages. Information on MEMBERSHIP, SUBSCRIPTIONS, AND CLAIMS is provided in the inside back cover.
Econometrica, Vol. 77, No. 1 (January, 2009), 1–28
ON FORWARD INDUCTION BY SRIHARI GOVINDAN AND ROBERT WILSON1 A player’s pure strategy is called relevant for an outcome of a game in extensive form with perfect recall if there exists a weakly sequential equilibrium with that outcome for which the strategy is an optimal reply at every information set it does not exclude. The outcome satisfies forward induction if it results from a weakly sequential equilibrium in which players’ beliefs assign positive probability only to relevant strategies at each information set reached by a profile of relevant strategies. We prove that if there are two players and payoffs are generic, then an outcome satisfies forward induction if every game with the same reduced normal form after eliminating redundant pure strategies has a sequential equilibrium with an equivalent outcome. Thus in this case forward induction is implied by decision-theoretic criteria. KEYWORDS: Game theory, equilibrium refinement, forward induction, backward induction.
THIS PAPER HAS TWO PURPOSES. One is to provide a general definition of forward induction for games in extensive form with perfect recall. As a refinement of weakly sequential equilibrium, forward induction restricts the support of a player’s belief at an information set to others’ strategies that are optimal replies to some weakly sequential equilibrium with the same outcome, if there are any that reach that information set. The second purpose is to resolve a conjecture by Hillas and Kohlberg (2002, Sec. 13.6), of which the gist is that an invariant backward induction outcome satisfies forward induction. A backward induction outcome is invariant if every game representing the same strategic situation (i.e., they have the same reduced normal form) has a sequential equilibrium with an equivalent outcome. For a game with two players and generic payoffs, we prove that an invariant backward induction outcome satisfies forward induction. The definitions and theorem are entirely decision-theoretic. None of the technical devices invoked in game theory, such as perturbations of players’ strategies or payoffs, is needed.2 Sections 1 and 2 review the motivations for backward induction and forward induction. Sections 3 and 4 provide general definitions of forward induction and invariance. The formulation and proof of the theorem are in Sections 5 and 6. Section 7 examines an alternative version of forward induction, and 1
This work was funded in part by a grant from the National Science Foundation of the United States. We are grateful for superb insights and suggestions from a referee. 2 We retain Kreps and Wilson’s (1982) definition of sequential equilibrium that consistent beliefs are limits of beliefs induced by sequences of completely mixed strategies converging to equilibrium strategies. However, Kohlberg and Reny (1997, Theorem 3.2) established that this property is implied by elementary assumptions that represent the “fundamental property of noncooperative games, namely that no player’s strategy choice affects any other player’s choice.” © 2009 The Econometric Society
DOI: 10.3982/ECTA6956
2
S. GOVINDAN AND R. WILSON
Section 8 concludes. Appendix A proves a technical lemma and Appendix B describes a definition of forward induction for a game in normal form. 1. INTRODUCTION We consider a finite game in extensive form specified by a game tree and an assignment of players’ payoffs to its terminal nodes. Throughout, we assume perfect recall, so the game tree induces a decision tree for each player. A pure strategy for a player specifies an action at each of his information sets, and a mixed strategy is a distribution over pure strategies. A mixed strategy induces a behavioral strategy that mixes anew according to the conditional distribution among actions at each information set. Kuhn (1953, Theorem 4) established for a game with perfect recall that each behavioral strategy is induced by a mixed strategy, and vice versa, inducing the same distribution on histories of play. 1.1. Backward Induction Economic models formulated as games typically have multiple Nash equilibria. Decision-theoretic criteria are invoked to select among Nash equilibria. For a game in extensive form with perfect recall, the primary criterion is backward induction. Backward induction is invoked to eliminate Nash equilibria that depend on implausible behaviors at information sets excluded by other players’ equilibrium strategies. Thus backward induction requires that a player’s strategy remains optimal after every contingency, even those that do not occur if all players use equilibrium strategies. We assume here that backward induction is implemented by sequential equilibrium as defined by Kreps and Wilson (1982, pp. 872, 882). A sequential equilibrium is a pair of profiles of players’ behavioral strategies and beliefs. Here we define a player’s belief to be a conditional probability system (i.e., satisfying Bayes’ rule where well defined) that at each of his information sets specifies a distribution over pure strategies that do not exclude the information set from being reached. A player’s belief is required to be consistent in that it is a limit of the conditional distributions induced by profiles of completely mixed or equivalent behavioral strategies converging to the profile of equilibrium strategies. Kreps and Wilson’s exposition differs in that for a player’s belief at an information set they used only the induced distribution over nodes in that information set. This restriction cannot be invoked here since the purpose of forward induction is to ensure that the support of a player’s belief at an information set is confined to others’ optimal strategies wherever possible, both before this information set as in their formulation in terms of nodes, and also subsequently in the continuation of the game. Thus we use throughout the more general specification that a player’s belief is over strategies.
ON FORWARD INDUCTION
3
The defining feature of a sequential equilibrium is the requirement that in the event an information set is reached, the player acting there behaves according to a strategy that in the continuation is optimal given his belief about Nature’s and other players’ strategies. A weakly sequential equilibrium as defined by Reny (1992, p. 631) is the same as a sequential equilibrium except that if a player’s strategy excludes an information set from being reached, then his continuation strategy there need not be optimal. Section 3 provides a formal definition of weakly sequential equilibrium, which is then used in our definition of forward induction. 1.2. Forward Induction Kohlberg and Mertens (1986, Sec. 2.3) emphasized that refining Nash equilibrium to sequential equilibrium is not sufficient to ensure that behaviors are justified by plausible beliefs. One of their chief illustrations is their Proposition 6 that a stable set of Nash equilibria contains a stable set of the game obtained by deleting a strategy that is not an optimal reply to any equilibrium in the set. Kohlberg and Mertens labeled this result forward induction, but they and other authors do not define the criterion explicitly. The main idea is the one expressed by Hillas and Kohlberg (2002, Sec. 42.13.6) in their survey article: “Forward induction involves an assumption that players assume, even if they see something unexpected, that the other players chose rationally in the past,” to which one can add “and other players will choose rationally in the future.” This addendum is implicit because rationality presumes that prior actions are part of an optimal strategy. See also Kohlberg (1990) and van Damme (2002). Studies of particular classes of two-player games with generic payoffs reveal aspects of what this idea entails. Outside-option games (as in Example 2.1 below) were addressed by van Damme (1989, p. 485). He proposed as a minimal requirement that forward induction should select a sequential equilibrium in which a player rejects the outside option if in the ensuing subgame there is only one equilibrium whose outcome he prefers to the outside option. Signaling games (as in Example 2.2 below) were addressed by Cho and Kreps (1987, p. 202). They proposed an intuitive criterion that was refined further by Banks and Sobel (1987, Sec. 3) to obtain criteria called divinity and universal divinity. These are obtained from iterative application of criteria called D1 and D2 by Cho and Kreps (1987, p. 205). Briefly, a sequential equilibrium satisfies the intuitive criterion if no type of sender could obtain a payoff higher than his equilibrium payoff were he to choose a nonequilibrium message and the receiver responds with an action that is an optimal reply to a belief that imputes zero probability to Nature’s choice of those types that cannot gain from such a deviation regardless of the receiver’s reply. The D1 criterion requires that after an unexpected message, the receiver’s belief imputes zero probability to a type of sender for which there is another type who prefers this deviation for a larger set of those responses of
4
S. GOVINDAN AND R. WILSON
the receiver that are justified by beliefs concentrated on types who could gain from the deviation and response. See also Cho and Sobel (1990) and surveys by van Damme (2002), Fudenberg and Tirole (1993, Sec. 11), Hillas and Kohlberg (2002), and Kreps and Sobel (1994). Battigalli and Siniscalchi (2002, Sec. 5) derived the intuitive criterion from an epistemic model. They said that a player strongly believes that an event is true if he remains certain of this event after any history that does not contradict this event. They consider a signaling game and a belief-complete space of players’ types; for example, one containing all possible hierarchies of conditional probability systems (beliefs about beliefs) that satisfy a coherency condition. Say that a player expects an outcome if his first-order beliefs are consistent with this outcome, interpreted as a probability distribution on terminal nodes of the game tree. They showed in Proposition 11 that an outcome of a sequential equilibrium satisfies the intuitive criterion under the following assumption about the epistemic model: The sender (1) is rational and (2) expects the outcome and believes that (2a) the receiver is rational and (2b) the receiver expects the outcome and strongly believes that (2b.i) the sender is rational and (2b.ii) the sender expects the outcome and believes the receiver is rational.
The key aspect of this condition is the receiver’s strong belief in the sender’s rationality. This implies that the receiver sustains his belief in the sender’s rationality after any message for which there exists some rational explanation for sending that message. These contributions agree that forward induction should ensure that a player’s belief assigns positive probability only to a restricted set of strategies of other players. In each case, the restricted set comprises strategies that satisfy minimal criteria for rational play. Two prominent contributions apply such criteria iteratively. McLennan (1985, p. 901) and Reny (1992, p. 639) proposed different algorithms for iterative elimination of beliefs that are implausible according to some criterion. McLennan defined the set of justifiable equilibria iteratively by excluding a sequential equilibrium that includes a belief for one player that assigns positive probability at an information set to an action of another player that is not optimal in any sequential equilibrium in the restricted set obtained in the previous iteration. Reny defined the set of explicable equilibria iteratively by excluding a belief that assigns positive probability to a pure strategy that is not a best response to some belief in the restricted set obtained in the previous iteration. Essentially, these procedures apply variants of Pearce’s (1984) iterative procedure for identifying rationalizable strategies to the more restrictive context of sequential equilibria. 1.3. Synopsis In Section 2 we illustrate further the motivation for forward induction via two standard examples from the literature. Our analyses of these examples
ON FORWARD INDUCTION
5
anticipate the theorem in Section 6 by showing that the result usually obtained by “forward induction reasoning” is implied by the decision-theoretic criterion called invariance. Invariance requires that the outcome should be unaffected by whether a mixed strategy is treated as a pure strategy. In Section 3 we propose a general definition of forward induction. Its key component specifies relevant pure strategies, that is, those that satisfy minimal criteria for rational play resulting in any given outcome—the induced probability distribution on terminal nodes of the game tree and thus on possible paths of equilibrium play. Our definition says that a pure strategy is relevant if there is some weakly sequential equilibrium with that outcome for which the strategy prescribes an optimal continuation at every information set the strategy does not exclude.3 We then say that an outcome satisfies forward induction if it results from a weakly sequential equilibrium in which each player’s belief at an information set reached by relevant strategies assigns positive probability only to relevant strategies. In Section 6 we prove for general two-player games with generic payoffs that backward induction and invariance imply forward induction. Thus for such games forward induction is implied by standard decision-theoretic criteria. 2. EXAMPLES In this section we illustrate the main ideas with two standard examples. These examples illustrate how forward induction can reject some sequential equilibria in favor of others. Each example is first addressed informally using the ‘forward induction reasoning’ invoked by prior authors. The literature provides no formal definition of forward induction and we defer the statement of our definition to Section 3, but the main idea is evident from the context. Each example is then analyzed using the decision-theoretic criterion called invariance to obtain the same result. Invariance is defined formally in Section 4 and invoked in Theorem 6.1, but in these examples and the theorem it is sufficient to interpret invariance as requiring only that the outcome resulting from a sequential equilibrium is not affected by adding a redundant pure strategy, that is, a pure strategy whose payoffs for all players are replicated by a mixture of other pure strategies. 2.1. An Outside-Option Game The top panel of Figure 1 displays the extensive and normal forms of a twoplayer game consisting of a subgame with simultaneous moves that is preceded by an outside option initially available to player I. The component of Nash 3 See Govindan and Wilson (2007) for a version that obtains results for the weaker concept of relevant actions, rather than strategies. We are indebted to a referee who provided an example of an irrelevant strategy that uses only relevant actions.
6
S. GOVINDAN AND R. WILSON
FIGURE 1.—Two versions of a game with an outside option.
equilibria in which player I chooses his outside option includes an equilibrium in which player II’s strategy has probability 2/3 of his left column and therefore player I is indifferent about deviating to his top row in the subgame, whereas there is no such equilibrium justifying deviating to the bottom row. Alternatively, player I might anticipate that II will recognize rejection of the outside option as a signal that I intends to choose the top row and therefore II should respond with the left column. To apply forward induction, one excludes from the support of II’s belief the dominated strategy in which I rejects the outside option and chooses the bottom row in the subgame. If this restriction is imposed, then after I rejects the outside option, II is sure that I will play the top row and, therefore, II’s optimal strategy is the left column, and anticipating this, player I rejects the outside option. As in Hillas (1994, Figure 2), one can invoke invariance to obtain this conclusion. The bottom panel of Figure 1 shows the expanded extensive form after adjoining the redundant strategy in which, after tentatively rejecting the outside option, player I randomizes between the outside option and the top row of the subgame with probabilities 3/4 and 1/4. Player II does not observe which strategy of player I led to rejection of the outside option. In the unique sequential equilibrium of this expanded game, player I rejects both the outside option and the redundant strategy, and then chooses the top row of the final subgame. A lesson from this example is that an expanded game has imperfect information in the sense that II has imperfect observability about whether I chose the redundant strategy. This is significant for II because I retains the option to choose the bottom row in the subgame iff he rejected the redundant strategy.
ON FORWARD INDUCTION
7
Even though subgame perfection could suffice in the original and expanded game for this simple example, in general one needs sequential equilibrium to analyze the expanded game—as will be seen later in Figure 4 of Section 7. Addition of redundant strategies can alter equilibrium strategies, but if invariance is satisfied, then the induced probabilities of actions along equilibrium paths of the original game are preserved and thus so too is the predicted outcome. One could explicitly map equilibria of an expanded game into induced behavioral probabilities of actions in the original game, but we omit this complication. 2.2. A Signaling Game The top panel of Figure 2 displays the two-player two-stage signaling game Beer–Quiche studied by Cho and Kreps (1987, Sec. II) and discussed further by Kohlberg and Mertens (1986, Sec. 3.6.B) and Fudenberg and Tirole (1993, Sec. 11.2). Consider sequential equilibria with the outcome QQ-R; that is, both types W and S of player I (the sender) choose Q and player II (the receiver) responds
FIGURE 2.—Two versions of the Beer–Quiche game.
8
S. GOVINDAN AND R. WILSON
to Q with R and to B with a probability of F that is ≥1/2. The equilibria in this component are sustained by player II’s belief after observing B that imputes to I’s type W a greater likelihood of having deviated than to type S. In all these equilibria, B is not an optimal action for type W. But in the equilibrium for which player II assigns equal probabilities to W and S after observing B and mixes equally between F and R, type S is indifferent between Q and B. If II recognizes this as the source of I’s deviation, then he infers after observing B that I’s type is S and therefore chooses R. Alternatively, if player I’s type is S, then he might deviate to B in hopes that this action will credibly signal his type, since his equilibrium payoff is 2 from Q but he obtains 3 from player II’s optimal reply R if the signal is recognized, but type W has no comparable incentive to deviate—this is the “speech” suggested by Cho and Kreps (1987, pp. 180–181) to justify their intuitive criterion. One applies forward induction by excluding from the support of II’s belief after observing B those strategies that take action B when I’s type is W. In fact, the sequential equilibria in which both types of I choose Q do not survive this restriction on II’s belief because II’s optimal response to B is then R, which makes it advantageous for player I’s type S to deviate by choosing B. Thus sequential equilibria with the outcome QQ-R do not satisfy forward induction. This leaves only sequential equilibria with the outcome BB-R in which both types of player I choose B and II chooses R after observing B. As in Example 2.1, one can obtain this same conclusion by invoking invariance. The bottom panel of Figure 2 shows the extensive form after adjoining a mixed action X for type S of player I that produces a randomization between B and Q with probabilities 1/9 and 8/9. Denote by BQ player I’s pure strategy that chooses B if his type is W and chooses Q if his type is S, and similarly for his other pure strategies. The normal form of this expanded game is shown in Table I with all payoffs multiplied by 10 (we intentionally omit the pure strategy BX to keep the analysis simple). Now consider the following extensive form that has the same reduced normal form. Player I initially chooses whether or not to use his pure strategy QQ, and if not then subsequently he chooses among his other pure strategies BB, BQ, QB, and QX. After each of these five pure TABLE I STRATEGIC FORM OF THE BEER–QUICHE GAME WITH THE REDUNDANT STRATEGY QX
W
S
B B Q Q Q
B Q B Q X
B: Q:
F F
F R
R F
R R
9, 1 0, 1 10, 1 1, 1 2, 1
9, 1 18, 10 12, 0 21, 9 20, 8
29, 9 2, 0 28, 10 1, 1 4, 2
29, 9 20, 9 30, 9 21, 9 22, 9
ON FORWARD INDUCTION
9
strategies, the extensive form in the bottom panel ensues, but with I’s action dictated by his prior choice of a pure strategy. That is, nature chooses I’s type to be W or S, the selected pure strategy dictates the subsequent choice of B or Q, and then player II (still having observed only which one of B or Q was chosen) chooses F or R. At player I’s information set where, after rejecting QQ, he chooses among his other pure strategies, a sequential equilibrium requires that he assigns zero probability to BQ because it is strictly dominated by QX in the continuation. At player II’s information set, after observing B, a sequential equilibrium requires that his behavioral strategy is an optimal reply to some consistent belief about those strategies of player I that reach this information set. But every mixture of I’s pure strategies BB, QB, and QX implies that, given his choice of B, the induced conditional probability that his type is S exceeds 9/10. Therefore, player II’s reply to B must be R in every sequential equilibrium of this game. Hence the sequential equilibria with outcome QQ-R are inconsistent with invariance, in agreement with failure to satisfy forward induction. A similar analysis applies to the game in Figure 3, which resembles games considered in studies of signaling via costly educational credentials in labor markets as in Spence (1974) and Kreps (1990, Sec. 17.2). In this game, forward induction rejects the outcome of the pooling equilibrium in which both types of player I move right and II responds to right with up (and to left with probability of up ≥1/2), and accepts the outcome of the separating equilibrium in which only I’s top type moves right. The most common use of forward induction in economic models is to reject pooling equilibria in favor of a separating equilibrium, although often what is actually assumed is a weaker implication of forward induction. The way in which invariance is invoked in Example 2.2 is indicative of the proof of Theorem 6.1 in Section 6.
FIGURE 3.—A signaling game with pooling and separating equilibria.
10
S. GOVINDAN AND R. WILSON
3. DEFINITION OF FORWARD INDUCTION In this section we propose a general definition of forward induction for a game in extensive form with perfect recall. Our definition of forward induction relies on the solution concept called weakly sequential equilibrium by Reny (1992, p. 631). Recall from Section 1.1 that a weakly sequential equilibrium is the same as a sequential equilibrium except that a player’s strategy need not be optimal at information sets it excludes. See Reny (1992, Sec. 3.4) for an expanded justification of weakly sequential equilibrium as the right concept for analysis of forward induction. Our definition differs from Reny’s in that we interpret players’ beliefs as specifying distributions over others’ strategies. Beliefs over strategies typically encode more information than necessary to implement sequential rationality, that is, as in Kreps and Wilson (1982), the conditional distribution over nodes in an information set suffices to verify optimality. However, it is only from a belief specified as a conditional distribution over strategies that one can verify whether a player’s belief recognizes the rationality of others’ strategies. As Examples 2.1 and 2.2 illustrate, the purpose of forward induction as a refinement is to reject outcomes that deter player I’s deviation by the threat of II’s response that is optimal for II only because his belief does not recognize I’s deviation as part of an optimal strategy for some equilibrium with the same outcome. To reject such outcomes, it is sufficient that the support of II’s belief is confined to I’s pure strategies that are optimal replies at information sets they do not exclude. The following definition is the analog of the definitions in Kreps and Wilson (1982) and Reny (1992). DEFINITION 3.1—Weakly Sequential Equilibrium: A weakly sequential equilibrium is a pair (b μ) of profiles of players’ behavioral strategies and beliefs. At each information set hn of player n, his behavioral strategy specifies a distribution bn (·|hn ) over his feasible actions, and his belief specifies a distribution μn (·|hn ) over profiles of Nature’s and other players’ pure strategies that enable hn to be reached. These profiles are required to satisfy the following conditions: (i) Consistency: There exists a sequence {bk } of profiles of completely mixed behavioral strategies converging to b and a sequence {σ k } of completely mixed equivalent normal-form strategies such that for each information set of each player the conditional distribution specified by μ is the limit of the conditional distributions obtained from {σ k }.4 (ii) Weak Sequential Rationality: For each player n and each information set hn that bn does not exclude, each action in the support of bn (·|hn ) is part of a pure strategy that is an optimal reply to μn (·|hn ) in the continuation from hn . 4 In this definition, Nature’s strategy is not perturbed. Also, the belief μn (·|hn ) might entail correlation, as observed by Kreps and Ramey (1987).
ON FORWARD INDUCTION
11
A sequential equilibrium is defined exactly the same except that each player’s actions must be optimal at all his information sets, including those excluded by his equilibrium strategy. We interpret forward induction as a property of an outcome of the game, defined as follows. DEFINITION 3.2—Outcome of an Equilibrium: The outcome of an equilibrium of a game in extensive form is the induced probability distribution over the terminal nodes of the game tree. A key feature in the definition of forward induction is the concept of a relevant strategy. DEFINITION 3.3—Relevant Strategy: A pure strategy of a player is relevant for a given outcome if there is a weakly sequential equilibrium with that outcome for which the strategy at every information set it does not exclude prescribes an optimal continuation given the player’s equilibrium belief there. Thus a relevant strategy is optimal for some expectation about others’ equilibrium play with that outcome and his beliefs at events after their deviations. For instance, in Example 2.2 of a signaling game, the strategy QB of the sender I in which type W chooses Q and type S chooses B is relevant for the outcome QQ-R because it is an optimal reply to the weakly sequential equilibrium with that outcome in which the receiver II responds to B by using F and R with equal probabilities. But the strategies BB and BQ are irrelevant because B is not an optimal reply for I’s type W to any weakly sequential equilibrium with outcome QQ-R. For the standard examples in Section 2 it is sufficient to interpret forward induction as requiring merely that player II’s belief at the information set excluded by I’s equilibrium strategy imputes positive probability only to the node reached by I’s nonequilibrium relevant strategy. For general games, however, a stronger requirement is desirable. We propose a general definition of forward induction that identifies those outcomes resulting from the conjunction of rational play and belief that others’ play is rational, and thus minimally consistent with Battigalli and Siniscalchi’s (2002) epistemic model of strong belief in rationality. Because a relevant strategy is optimal, hence rational, in some weakly sequential equilibrium with the same outcome, the relevant strategies are the minimal set for which one can require the support of one player’s belief to recognize the rationality of other players’ strategies—indeed that is the lesson from the standard examples in Section 2. Our proposed definition of a forward induction outcome therefore requires that the outcome results from a weakly sequential equilibrium in which every player maintains the hypothesis that other players are using relevant strategies throughout the game, as long as that hypothesis is tenable.
12
S. GOVINDAN AND R. WILSON
Thus forward induction is applied only to information sets reached by profiles of relevant strategies: DEFINITION 3.4—Relevant Information Set: An information set is relevant for an outcome if it is not excluded by every profile of strategies that are relevant for that outcome.5 Then we define a forward induction outcome as follows.6 DEFINITION 3.5—Forward Induction: An outcome satisfies forward induction if it results from a weakly sequential equilibrium in which at every information set that is relevant for that outcome the support of the belief of the player acting there is confined to profiles of Nature’s strategies and other players’ strategies that are relevant for that outcome. Section 7 compares this definition with Reny’s alternative interpretation. Applied to the standard examples in Section 2, our definition yields the conclusions obtained from “forward induction reasoning” in the literature. For instance, in Example 2.2 of a signaling game, the outcome QQ-R does not satisfy forward induction because the definition requires that after observing B, player II assigns zero probability to I’s irrelevant strategies BB and BQ, and thus assigns positive probability only to I’s relevant strategy QB that enables II’s information set B to be reached. For signaling games in general, forward induction implies the intuitive criterion, D1, D2 (whose iterative version defines universal divinity), and Cho and Kreps’ (1987, Sec. IV.5) strongest criterion, called never weak best response (NWBR) for signaling games. These implications are verified by showing that a strategy s of the sender is irrelevant if s prescribes that his type t sends a message m that is not sent by any type in some weakly sequential equilibrium with the given outcome, and the pair (t m) satisfies any of these criteria. For instance, the criterion NWBR excludes the strategy s from the receiver’s belief if the continuation strategy m at information set t yields exactly the sender’s type-contingent payoff from the given outcome for some beliefs and optimal responses of the receiver only when some other type t that could send m would get a type-contingent payoff that is higher than from the designated outcome for the same or a larger set of the receiver’s optimal responses. But this condition implies that there is no weakly sequential equilibrium with the same outcome for which m is an optimal action for type t. Were there such an equilibrium, the receiver could use any such response at the off-the-equilibrium-path 5 This differs from Kuhn’s (1953, Definition 6) and Reny’s (1992, p. 631) definition of an information set that is relevant for a pure strategy because the information set is not excluded by that strategy. 6 For readers who prefer normal-form analysis, Definition B.1 in Appendix B proposes a slightly stronger definition applied to the normal form of the game that is equivalent to the definition proposed here when payoffs are generic and that enables an analog of Theorem 6.1.
ON FORWARD INDUCTION
13
information set m, but then type t could obtain a superior payoff by sending m. Thus, m cannot be an optimal continuation by type t in any weakly sequential equilibrium with the given outcome, and therefore s is an irrelevant strategy. For general games in extensive form, forward induction implies a version of NWBR that Fudenberg and Tirole (1993, p. 454) attributed to Kohlberg and Mertens (1986, Proposition 6). A pure strategy that is an inferior reply to every equilibrium with a given outcome chooses an inferior action at some information set that intersects a path of equilibrium play. Such a strategy is irrelevant for that outcome according to Definition 3.3 and, therefore, if the outcome satisfies forward induction according to Definition 3.5, then it results from a weakly sequential equilibrium in which, at every information set that is relevant for that outcome, the support of the belief of the player acting there assigns zero probability to this irrelevant strategy. 4. DEFINITION OF INVARIANCE In this section we define invariance as a property of a solution concept. First we define relations of equivalence between games and between outcomes of equivalent games. Recall that a player’s pure strategy is redundant if its payoffs for all players are replicated by a mixture of his other pure strategies. From the normal form of a game one obtains its reduced normal form by deleting redundant strategies. Thus the reduced normal form is the minimal representation of the essential features of the strategic situation. DEFINITION 4.1—Equivalent Games: Two games are equivalent if their reduced normal forms are the same up to relabeling of strategies. As specified in Definition 3.2, the outcome of an equilibrium of a game in extensive form is the induced probability distribution on terminal nodes and thus on the paths through the tree. Associated with each outcome is a set of profiles of Nature’s and players’ mixed strategies that result in the outcome, and in turn each such profile can be replicated by a profile of mixed strategies in the reduced normal form. Hence we define equivalent outcomes as follows. DEFINITION 4.2 —Equivalent Outcomes of Equivalent Games: Outcomes of two equivalent games are equivalent if they result from the same profile of mixed strategies of their reduced normal form. Trivially, the outcome of any Nash equilibrium is equivalent to the outcome of a Nash equilibrium of any equivalent game. For any solution concept that is a refinement of Nash equilibrium, we define invariance as follows. DEFINITION 4.3—Invariant Outcome: An outcome is invariant for a solution concept if every equivalent game has an equivalent outcome of an equilibrium selected by the solution concept.
14
S. GOVINDAN AND R. WILSON
This definition is used in Section 6 where the solution concept is sequential equilibrium. Existence of invariant outcomes of sequential equilibria for generic games in extensive form with perfect recall is implied by the existence of stable sets as defined by Mertens (1989).7 5. FORMULATION In this section we introduce notation used in the proof of Theorem 6.1 in Section 6. Let Γ be a game in extensive form with perfect recall. For each player n let Hn be the collection of his information sets, and let Sn Bn , and Σn be his sets of pure, behavioral, and mixed strategies. A pure strategy chooses an action at each information set in Hn , a behavioral strategy chooses a distribution over actions at each information set, and a mixed strategy chooses a distribution over pure strategies. We say that a pure strategy enables an information set if the strategy’s prior actions do not exclude the information set from being reached, and similarly for behavioral and mixed strategies. Let P be the outcome of a sequential equilibrium of the game Γ . Say that a pure strategy or an information set is P-relevant if it is relevant for the outcome P. Let Σ(P) and B(P) be the sets of Nash equilibria of Γ represented as profiles of mixed and behavioral strategies, respectively, that result in the outcome P. Also, let BM(P) be the set of weakly sequential equilibria whose outcome is P, where each (b μ) ∈ BM(P) consists of a profile b ∈ B(P) of players’ behaviorial strategies and a profile μ of players’ consistent beliefs. As in Definition 3.1, in each weakly sequential equilibrium (b μ) the belief of a player n at his information set hn ∈ Hn is a probability distribution μn (·|hn ) over Nature’s and other players’ pure strategies that reach hn . A pure strategy sn ∈ Sn for player n is optimal in reply to (b μ) if at each information set of n enabled by sn , the action specified there by sn is optimal given (b μ). Given an outcome P, say that a P-path through the game tree is one that terminates at a node in the support of P. Actions on P-paths are called equilibrium actions. Let Hn (P) be the collection of player n’s information sets that intersect P-paths. Obviously, every equilibrium in B(P) prescribes the same mixture at each information set in Hn (P). Let Sn (P) ⊂ Sn comprise those pure strategies sn of player n such that sn chooses an equilibrium action at every information set in Hn (P) that sn enables. Note that if σ ∈ Σ(P), then the support of σn is contained in Sn (P). Moreover, every strategy in Sn (P) is optimal against every equilibrium in Σ(P). Partition the complement Tn (P) ≡ Sn \Sn (P) into subsets Rn and Qn of n’s pure strategies that are P-relevant and P-irrelevant, respectively. Note that Sn (P) may also contain P-irrelevant strategies. 7 In Govindan and Wilson (2006) we proved that if a solution concept satisfies invariance and a condition called strong backward induction, then it selects sets of equilibria that are stable in the weaker sense defined by Kohlberg and Mertens (1986, Sec. 3.5).
ON FORWARD INDUCTION
15
Define an equivalence relation among player n’s pure strategies as follows. Two strategies are equivalent if they prescribe the same action at each information set in Hn (P). Let En (P) be the set of equivalence classes. Denote a typical element of En (P) by En and let En (sn ) be the equivalence class that contains sn . Let En◦ (P) be the subcollection of equivalence classes that contain strategies in Sn (P). Thus, any strategy that is used in some equilibrium in Σ(P) belongs to some equivalence class in En◦ (P), while any strategy that is in Tn (P) does not. (In Example 2.2, for P = QQ − R and player n = I, Sn (P) = {QQ}, Rn (P) = {QB}, En (P) = {QQ QB BQ BB}, and En◦ (P) = {QQ}.) If Rn is not empty, then for each probability δ ∈ (0 1) let tnδ be a mixed strategy of n of the form [1 − δ]sn∗ + δρn , where sn∗ is a strategy in Sn (P) and ρn is a mixed strategy whose support is Rn . Since sn∗ is a best reply against every equilibrium in Σ(P), tnδ is an approximate best reply against equilibria in Σ(P) when δ is small, a fact we need in the next section. Define a game Gδ in normal form by adding to the normal form G of Γ the redundant pure strategy tnδ for each player n for whom Rn is not empty. In particular, tnδ is added iff there is some information set in Hn (P) where some nonequilibrium action is part of a P-relevant strategy. (In Example 2.1, the redundant strategy tnδ for player I is the one shown in Figure 1 with parameter δ = 1/4; in Example 2.2, it is the strategy QX with parameter δ = 1/9.) Next we define a game Γ δ in extensive form with perfect recall whose normal form is equivalent to Gδ , and thus also Γ δ is equivalent to Γ . A path of play in Γ δ consists of choices by players in an initial stage, followed by a path of play in a copy of Γ . In the subsequent play of Γ , no player is informed about choices made by other players in the initial stage of Γ δ . The rules of Γ δ are the following. If Rn is empty, then in Γ δ player n chooses among all his equivalence classes in En (P) in the initial stage. If Rn is not empty, then in the initial stage he first chooses whether to play an equivalence class in En◦ (P) or not. If he decides to play something in En◦ (P), then he chooses one of these equivalence classes; if he chooses not to, then he proceeds to a second information set where he chooses to play either the redundant pure strategy tnδ or an equivalence class among those not in En◦ (P). After these initial stages for all players, Γ δ evolves the same as Γ does, that is, a copy of Γ follows each sequence of choices in the initial stage. In Γ δ the information sets in Γ are expanded to encompass appropriate copies of Γ to represent that no player ever observes what others chose in the initial stage; thus, the information revealed in Γ δ is exactly the same as in Γ . The information set hn ∈ Hn in Γ has in Γ δ for each En ∈ En (P) an expanded copy hδn (En ) and a copy hδn (tnδ ). Nature makes the choice at the expansions of those information sets in Hn (P) (but not at expansions of those in Hn \Hn (P)) according to the equivalence class chosen in the initial stage or at all expansions of information sets in Hn if tnδ was chosen. That is, if n chooses tnδ at the second information set, then Nature automatically implements the entire strategy; but if he chooses some equivalence class En in En (P), then Nature implements actions prescribed by En at each hδn (En ) when hn ∈ Hn (P)
16
S. GOVINDAN AND R. WILSON
and leaves it to him to choose at those that are not, if and when they occur. (If Γ is the Beer–Quiche game and δ = 1/9, then tnδ = QX and Γ δ is the game in which player I chooses whether to play QQ, and if not then he chooses among BB, BQ, QB, and QX, as described in the text of Example 2.2.) One can interpret player n’s choice of an equivalence class from, say, En◦ (P) as equivalent to making an initial commitment to take a specified equilibrium action at every information set in Hn (P), leaving the actions at n’s other information sets unspecified until those information sets are reached. A pure strategy sn ∈ Sn can be implemented in Γ δ by first choosing En (sn ) in the initial stage and then making the choices prescribed by sn at all hn ∈ Hn \Hn (P), and any strategy in Γ δ that begins by choosing some equivalence class En in the first stage ends up implementing some sn ∈ En . Observe too that the redundant pure strategy tnδ , when available, ends up implementing the mixture given by tnδ . Thus, it is obvious that Gδ is obtained from the normal form of Γ δ by deleting some redundant pure strategies in the latter that are duplicates of other pure strategies. Hence, by a slight abuse of notation, we view Gδ as the normal form of Γ δ . The game Γ δ is now easily seen to be equivalent to Γ . Suppose hn ∈ Hn \Hn (P). If an equivalence class En contains a pure strategy sn that enables hn in Γ , then in Γ δ the corresponding strategy sn —that is, / Hn (P)— choosing En in the initial stage and then making sn ’s choices at all hn ∈ enables hδn (En ). Conversely, if En does not contain such an sn , then there is an information set vn ∈ Hn (P) that precedes hn , is enabled by En , and where En makes a choice different from the one that leads to hn . Thus, in Γ δ , Nature’s choice at vnδ (En ) prevents hδn (En ) from being reached. Therefore, to analyze the game Γ δ , we need to consider only information sets hδn (En ) where En contains a strategy that enables hn in Γ . For simplicity in this section and the next, by an information set hδn (En ) in Γ δ of player n, we mean an hn ∈ Hn \Hn (P) and an En that contains a strategy that enables hn in Γ . Now assume the game has two players. We use m to denote the opponent of player n. Suppose that En and En are two equivalence classes that contain strategies that enable some hn ∈ / Hn (P). The information that n has at hδn (En ) δ and hn (En ) about m’s choices are the same at both information sets. Therefore, a pure strategy of m in Gδ enables one if and only if it enables the other. In particular, in a sequential equilibrium (b˜ δ μ˜ δ ) of Γ δ , player n’s belief at hδn (En ) is independent of En and can thus be denoted μ˜ δn (·|hn ). Likewise, suppose σ˜ mδ is a mixed strategy of m in Gδ that enables some information set hδn (En ) of n. Then σ˜ mδ induces a conditional distribution τ˜ mδ over the pure strategies of m in Gδ that enable hδn (En ). Let σm and τm be the equivalent strategies in G. It is easily checked that τm is the conditional distribution induced by σm over the pure strategies that enable hn , and an action an at hδn (En ) in Γ δ is optimal against τ˜ mδ iff it is optimal against τm in Γ .
ON FORWARD INDUCTION
17
6. STATEMENT AND PROOF THE THEOREM In this section we show for two-player games with generic payoffs that an invariant backward induction outcome satisfies forward induction. The notion of genericity we invoke is the following. Let G be the space of all games generated by assigning payoffs to the terminal nodes of a fixed twoplayer game tree. In Govindan and Wilson (2001) we showed that there exists a closed lower-dimensional subset G0 such that for each game not in G0 there are finitely many outcomes of Nash equilibria. For technical reasons, in Appendix A we construct another closed lower-dimensional subset denoted G1 . Now a game is generic if it is in the complement of both G0 and G1 . With this, the formal statement of our theorem is the following: THEOREM 6.1: An outcome of a two-player game with perfect recall and generic payoffs satisfies forward induction if it is invariant for the solution concept sequential equilibrium. PROOF: Assume that Γ is a two-player game in extensive form with perfect recall and generic payoffs. Assume also that P is an invariant sequential equilibrium outcome of Γ , that is, each game equivalent to Γ has a sequential equilibrium whose outcome is equivalent to P. Because Γ is equivalent to the game Γ δ defined in Section 5, Γ δ has a sequential equilibrium (b˜ δ μ˜ δ ) whose outcome is equivalent to P. Because μ˜ δ is consistent, there exists a sequence {b˜ δε } of profiles of completely mixed behavioral strategies that converges as ε ↓ 0 to b˜ δ and a corresponding equivalent sequence {σ˜ εδ } of profiles of completely mixed strategies in the normal form Gδ that converges to some profile σ˜ δ and such that the belief profile μ˜ δ is the limit of the beliefs derived from the sequence {σ˜ εδ }. Since b˜ δ induces an outcome that is equivalent to P, the strategy σ˜ δ , which is equivalent to b˜ δ , has its support in Sn (P) for each n: indeed, a strategy in Tn (P), or the strategy tnδ when available, chooses a nonequilibrium action at some hn ∈ Hn (P) in Γ that it enables. Therefore, under b˜ δ each player n in the initial stage assigns positive probability only to choices of equivalence classes in En◦ (P). Corresponding to the sequence {σ˜ εδ }, there is an equivalent sequence {σεδ } of profiles of mixed strategies in the normal form of Γ for which there is an equivalent sequence {bδε } of profiles of behavioral strategies in the extensive form of Γ . Let μδε be the profile of beliefs induced by σεδ . Denote selected limit points of these sequences by σ δ , bδ , and μδ . By construction, σ δ ∈ Σ(P), bδ ∈ B(P), and μδ is consistent. It follows from our remarks at the end of the previous section that for each n and hn ∈ / Hn (P), μδn (·|hn ) is equivalent δ to μ˜ n (·|hn ), and an action at hn is optimal in Γ against μδn (·|hn ) iff it is optimal in Γ δ at hδn (En ) against μ˜ δn (·|hn ) for the corresponding copies in Γ δ .
18
S. GOVINDAN AND R. WILSON
Next we argue that (bδ μδ ) is a weakly sequential equilibrium of Γ . Let hn be an information set of player n that bδn enables. We need to show that the choice made by bδn at hn is optimal against μδn (·|hn ). If hn belongs to Hn (P), then μn (·|hn ) is derived from σmδ and obviously bδn chooses optimally at hn . / Hn (P). Let an be an arbitrary action at hn that is chosen Suppose that hn ∈ with positive probability by bδn . Since hn is enabled by bδn there exists a pure strategy sn in the support of σ δ that enables hn and chooses an there. Since σ δ is equivalent to σ˜ δ , in b˜ δ player n with positive probability chooses En (sn ) and then makes the choices prescribed by sn . Sequential rationality of an at hδn (En ) implies its optimality against μ˜ δn (·|hn ). Hence, from the previous paragraph an is optimal against μδn (·|hn ) in Γ . Since an was arbitrary, this shows that (bδ μδ ) is a weakly sequential equilibrium of Γ . For some sequence δ ↓ 0, (σ δ bδ μδ ) converges to some limit point (σ b μ). Clearly, σ ∈ Σ(P), b ∈ B(P), μ is consistent, and (b μ) ∈ BM(P) is a weakly sequential equilibrium of Γ because BM(P) is a closed set. It remains to prove the forward induction property for the belief profile μ. δ } be the sequence For each n for whom Rn is not empty, and each δ, let {τ˜ nε δ δ of conditional distributions over Tn ≡ Tn (P) ∪ {tn } induced by the sequence δ {σ˜ εδ } and let τ˜ nδ be a limit point. The sequence {τ˜ nε } and therefore its limit are determined by choices made after n chooses in the initial stage to avoid equivalence classes in En◦ (P). Therefore, the probability of tnδ is nonzero under τ˜ nδ iff tnδ is chosen with positive probability at n’s second information set in the initial stage, and the probability of sn ∈ Tn (P) is nonzero under τ˜ nδ iff n chooses E(sn ) with positive probability at this stage and then implements the choices of sn with positive probability after this choice. Express τ˜ nδ as a convex combination α˜ δn tnδ + [1 − α˜ δn ]τˆ nδ where the support of τˆ nδ is contained in Tn (P). Then sequential rationality at the initial stage after rejecting equivalence classes in En◦ (P) and at subsequent information sets have the following two implications. First, α˜ δn is nonzero only if tnδ is at least as good a reply as each sn ∈ Tn (P) against bδm . Second, if α˜ δn < 1, then a strategy sn ∈ Tn (P) belongs to the support of τˆ nδ only if it is at least as good a reply against bδm as the other strategies in Tnδ , and for / Hn (P) enabled by sn in Γ , the choice prescribed by sn at hn is optimal each hn ∈ at hδn (En (sn )) given the belief μ˜ δn (·|hn ). If an information set hδm (Em ) of player m is enabled by τ˜ nδ but not by σ˜ nδ , then the beliefs μ˜ δm (·|hm ) are derived from τ˜ nδ . δ The sequence {τ˜ nε } induces a corresponding sequence of equivalent strategies in G that induces a sequence of conditional distributions over Tn (P). Because tnδ = [1 − δ]sn∗ + δρn , the limit of the sequence of strategies in G δ that is equivalent to the sequence {τ˜ nε } is [1 − δ]α˜ δn sn∗ + δα˜ δn ρ + [1 − α˜ δn ]τˆ nδ . Therefore, the limit of the sequence of conditional distributions over Tn (P) is τnδ = αδn ρ + [1 − αδn ]τˆ nδ , where αδn = α˜ δn δ/[α˜ δn δ + (1 − α˜ δn )]. Obviously if an information set hm of player m is enabled by τnδ but not by σnδ , then the beliefs μδm (·|hm ) are those derived from τnδ . Passing to a subsequence if necessary, the limit τn of the sequence τnδ can be expressed as a convex combination τn = αn ρ + [1 − αn ]τˆ n where αn and τˆ n
ON FORWARD INDUCTION
19
are the limits of αδn and τˆ nδ , respectively. As in the previous paragraph, if an information set hm of player m is enabled by τn but not by σn , then the beliefs μm (·|hm ) are those derived from τn . CLAIM 6.2: (i) αn > 0. (ii) If αn < 1, then the support of τˆ n consists of strategies in Rn . In particular, for each sn in its support and each information set hn that sn enables, the choice of sn at hn is optimal given (b μ). PROOF OF CLAIM: We prove (ii) first. Suppose αn < 1. Let sn be a strategy in Tn (P) that is not optimal in reply to (b μ). We show that sn is not in the support of τˆ nδ for all sufficiently small δ, which proves the second statement. Let hn be an information set that sn enables where its action is not optimal. If hn ∈ Hn (P), then every strategy in En (sn ) is suboptimal against bm . Because sn∗ , the strategy that belongs to Sn (P) and to the support of tnδ for all δ, is optimal against bδm for all δ and because b is the limit of bδ , for sufficiently small δ the strategy tnδ does better against bδ than every strategy in the equivalence class En (sn ). At the second information set in the initial stage of Γ δ where n decides among the redundant strategy tnδ and equivalence classes not in En◦ (P), sequential rationality implies that he chooses the equivalence class En (sn ) with zero probability for all small δ. As we remarked above, this implies that for such δ, the probability of sn is zero in τˆ nδ . If hn ∈ / Hn (P), then there exists another strategy sn in the equivalence class En (sn ) that agrees with sn elsewhere but prescribes an optimal continuation at hn . Obviously, for all small δ, sn is a better reply than sn in reply to (bδ μδ ). But then sequential rationality at the copy hδn (En (sn )) of hn in Γ δ for such small δ implies that he would choose according to sn there and not sn . Again, the probability of sn under τˆ nδ is zero for small δ. Thus every strategy in the support of τˆ n is optimal in reply to (b μ) and therefore is a P-relevant strategy. It remains to show that αn = 0. Suppose to the contrary that αn = 0. Let Sˆn be the set of strategies in the support of either σn or τˆ n . Let Hˆ m be the collection of information sets in Hm enabled by strategies in Sˆn . Because (b μ) is a weakly sequential equilibrium, we obtain the following properties for each information set hm in Hˆ m of player m: if hm is enabled by σn , then the action prescribed by bm at hm is optimal against σn ; if hm is enabled by τˆ n and not by σn , then the action prescribed by bm is optimal against τˆ n . Therefore, for each small η > 0 there exists a perturbation Γ (η) of Γ , where only m’s payoffs are perturbed, such that σm is optimal against σn (η) ≡ [1 − η]σn + ητˆ n in Γ (η) and that Γ (η) converges to Γ as η goes to zero. As we argued above, τˆ n is optimal against σm in Γ . Therefore, for all small η, (σm σn (η)) is an equilibrium of Γ (η). Since Γ is generic, it belongs to some component C of the open set G \G1 , where G1 is the set constructed in Appendix A. Since G \G1 has finitely many connected components, C is open in G \G1 and, hence, in G . Therefore, the sequence Γ (η) is in C for all small η. By Lemma A.1 in Appendix A, there
20
S. GOVINDAN AND R. WILSON
exists a strategy τn such that (i) the support of τn equals Sˆn and (ii) σm is a best reply against τn in Γ . Therefore, for all 0 ≤ ε ≤ 1, (σm (1 − ε)σn + ετn ) is an equilibrium of Γ . But because the strategies in the support of τˆ n choose a nonequilibrium action at some hn ∈ Hn (P) that they enable, all these equilibria result in different outcomes. This is impossible because the payoffs in Γ are generic and therefore Γ has only finitely many equilibrium outcomes, as shown Q.E.D. in Govindan and Wilson (2001). Thus αn = 0. Now we prove that P satisfies forward induction by showing that (b μ) induces beliefs that assign positive probability only to P-relevant strategies. Let hn be an information set of n that is enabled by bn . If hn ∈ Hn (P), then obviously n’s belief over the continuation strategies of m is the one derived from / Hn (P), σm , and strategies in the support of σm are obviously P-relevant. If hn ∈ then the only strategies of m that enable hn are those in Tm (P). If there is no strategy in Rm that enables hn , then there is nothing to prove. Otherwise, the subset of strategies in Rm that enable hn is not empty and then the strategy τm , which by the above claim has Rm as its support, enables hn . Therefore, μn (·|hn ) is derived from τm , and in this case, too, the restriction on beliefs imposed by forward induction holds. Thus P satisfies forward induction. Q.E.D. Theorem 6.1 resolves a conjecture by Hillas and Kohlberg (2002, Sec. 13.6). Its remarkable aspect is that backward induction and invariance suffice for forward induction—if there are two players and payoffs are generic. No further assumption about rationality of behavior or plausibility of beliefs is invoked; neither are perturbations of strategies invoked as in studies of perfect equilibria and stable sets of equilibria, and for signaling games there is no reliance on Cho and Kreps’ (1987, p. 181) auxiliary scenario in which the sender makes a “speech” that the other’s intransigent belief ignores the fact that a deviation would be rational provided merely that the receiver recognizes and acts on its implications by excluding irrelevant strategies from the support of his belief. Invariance excludes one particular presentation effect by requiring that the outcome should not depend on whether a mixed strategy is treated as an additional pure strategy. One interpretation of forward induction is that it excludes another presentation effect by requiring that the outcome does not depend on irrelevant strategies. Indeed, van Damme (2002, p. 1555) interpreted forward induction as akin to the axiom called independence of irrelevant alternatives in social choice theory. In the case of a game, the analog of social choice is the outcome (the probability distribution on terminal nodes) and the irrelevant alternatives are players’ irrelevant strategies. Our proof of Theorem 6.1 relies on the assumption that Γ has two players and generic payoffs. Indeed, the conclusion of Claim 6.2 relies on genericity.8 8 That extensions to nongeneric payoffs are problematic can be seen in studies of signaling games where signals are costless to the sender, as in Chen, Kartik, and Sobel (2008).
ON FORWARD INDUCTION
21
Moreover, the proof does not suffice for the Beer–Quiche game in Section 2.2 if the two types of player I are treated as two different players; in particular, the game Γ δ has a sequential equilibrium in which both types choose Q. The intuitive reason why the proof does not apply to an N-player game is that at a player’s information set that does not intersect paths of equilibrium play, his beliefs might need to evaluate the relative likelihood of one opponent choosing a relevant strategy compared to another opponent choosing an irrelevant strategy. Asking for a sequential equilibrium in the game Γ δ does not impose any discipline on such considerations. We surmise that additional decisiontheoretic criteria must be imposed to obtain a version of Theorem 6.1 for a game with more than two players. 7. RENY’S INTERPRETATION OF FORWARD INDUCTION An implication of Theorem 6.1 is that there is no conflict between backward and forward induction if one adopts the decision-theoretic principle of invariance. This conclusion depends on our definitions of relevant strategies and forward induction outcomes; for example, we interpret forward induction as a refinement of weakly sequential equilibrium that ensures the outcome does not depend on one player believing the other is using an irrelevant strategy at a relevant information set. In this section we compare our definitions with the principle alternative, represented by the discussion in Reny (1992, Sec. 4). He invoked “best response motivated inferences” as an instance of “forward induction logic” and concluded from an example that it can conflict with backward induction. Although he does not propose an explicit definition, the main ingredients differ from our formulation as follows. Our definitions are narrow—we interpret forward induction as a property of an outcome of a weakly sequential equilibrium and ask only that the outcome results from one in which the support of a player’s belief at a relevant information set is confined to relevant strategies, which we limit to those strategies that are optimal replies to some weakly sequential equilibrium. Reny’s view applies forward induction reasoning directly to players’ strategies rather than to outcomes, and applies it to more information sets and more strategies. At every information set not excluded by a player’s own strategy, he asks only that the support of the player’s belief is confined to those strategies that reach that information set for which there are some beliefs of the other player that would justify using them. The implications of Reny’s expanded view of forward induction reasoning are illustrated by the motivating example in his Figure 3. The top panel of Figure 4 shows a game in which players I and II alternately choose whether to end the game. Reny argued that this example shows a tension between forward and backward induction. He observed that I’s choice of the pure strategy D strictly dominates Ad. He inferred from this that forward induction should require that if I rejects D, then II must believe that I’s strategy is surely Aa and hence
22
S. GOVINDAN AND R. WILSON
FIGURE 4.—Top panel: Reny’s example of a game between players I and II. Bottom panel: The game modified so that player I can choose the redundant strategy x(δ) after rejecting D.
II’s only optimal reply is Ad. But backward induction requires each player to choose d, and before that D, which contradicts the seeming implication of forward induction that II’s strategy should be Ad. From this Reny concluded that II’s backward induction strategy is “rendered ‘irrational”’ and thus “the inappropriateness, indeed the inapplicability of the usual backward programming argument in the presence of best response motivated inferences” (Reny (1992, p. 637), italics as in original). Our analysis of this example differs in two respects. First, I’s only relevant strategy is to choose D initially, so forward induction according to our definition has no implications for II’s beliefs. This is so because our definitions identify outcomes that result from the conjunction of rational play and beliefs that other players are playing rationally; hence we apply them only to information sets reached by rational play as represented by relevant strategies. In contrast,
ON FORWARD INDUCTION
23
Reny applied forward induction to the belief of a player when the other player’s strategy is an optimal reply to an arbitrary belief. In the top panel of Figure 4, for player II to choose A requires that either II believes I is irrational or II believes that I believes II is irrational. Specifically, for II at her first decision node to believe that I’s strategy is Ad amounts to believing that I is irrational (because D dominates Ad as noted above); for II to believe that I’s strategy is Aa and that I is rational requires II to ascribe to I a belief that II’s strategy is Aa with high probability, which is an irrational strategy for II (because at II’s second decision node the continuation d dominates the continuation a). Our view is that a coherent theory of rational play and beliefs that others are playing rationally (i.e., a theory consistent with strong belief in rationality) is possible only with the more circumscribed definition of relevant strategies that we propose. Note, however, that we admit fewer strategies as relevant but restrict beliefs only at relevant information sets, so our definition of forward induction is neither stronger nor weaker than Reny’s interpretation. The other respect in which our analysis differs is that we invoke invariance. The bottom panel of Figure 4 shows an expanded extensive form in which player I can reject D and then choose between action A or the new pure strategy x(δ) for some probability δ ∈ (1/2 1). (See Govindan and Wilson (2006, Sec. 2.3) for a similar example.) The two information sets indicate that player II cannot know whether I chose A or x(δ) after rejecting D. The branch points indicated by black boxes refer to moves by Nature, that is, Nature takes over and implements the strategy x(δ) using the indicated probabilities (1 − δ δ) and (0 1) at I’s first and second information sets after I chooses x(δ). Note that x(δ) is redundant because it is replicated by the mixed strategy that chooses between D and Aa with probabilities 1 − δ and δ. In the expanded game it is somewhat arbitrary whether one supposes that I can choose x(δ) before or after D. We use the latter because then it is easy to construct the unique Nash equilibrium of the subgame that begins after I rejects D. This is a sequential equilibrium in which I chooses x(δ) with probability 1/[1 + δ] and otherwise chooses A and then d; and, if A occurs, then II chooses D with probability 2δ − 1, and otherwise A and then d. Consistent with Bayes’ rule, II’s strategy is supported by beliefs at her first and second information sets that the conditional probabilities are, respectively, 1/2 and 1 that I chose x(δ). As δ approaches 1/2, II’s strategy in this equilibrium converges to Ad and, as δ approaches 1, to D, which correspond to the two strategies by II that Reny considered. (When δ = 1/2, the Nash equilibria of the subgame require only that I’s probability of x(1/2) is at least 2/3, and II’s belief changes accordingly; when δ = 1, the game is essentially the same as the original game since x(1) is a duplicate of Aa.) Thus we find that II might be indifferent between her backward induction strategy D and her strategy Ad that Reny concluded is implied by best response motivated inferences. Therefore, Reny’s conclusion that II’s backward induction strategy is rendered irrational depends on rejecting either sequential equilibrium or invariance as decision-theoretic principles.
24
S. GOVINDAN AND R. WILSON
8. CONCLUSION Theorem 6.1 offers an explanation of why forward induction is a desirable refinement of sequential equilibrium in two-player games with generic payoffs. If an outcome does not satisfy forward induction—that is, depends on one player believing the other is using an irrelevant strategy—then there is an equivalent game in which this outcome results only from Nash equilibria and not from any sequential equilibrium. Failure of an economic model to predict an outcome that satisfies forward induction could motivate reconsideration of whether the essential features of the strategic situation are well represented by the specific extensive form used in the model, or if one has confidence in the model, then this prediction might be rejected because for the same model there necessarily exists another prediction that does satisfy forward induction. Because the theorem is restricted to games with two players and generic payoffs, it does not establish that our definitions of relevant strategies and forward induction outcomes are surely the right ones for general games. But it suggests that similar definitions might enable “forward induction reasoning” to be justified by decision-theoretic criteria. APPENDIX A: TECHNICAL LEMMA Given an extensive form with two players, for each player n let Sn and Σn be n’s sets of pure and mixed strategies, and let S = S1 × S2 and Σ = Σ1 × Σ2 be the product sets of profiles. Let G be the Euclidean space of games generated by assigning payoffs to the players at the terminal nodes of the given extensive form. LEMMA A.1: There exists a closed, lower-dimensional, semi-algebraic set G1 of G such that G \G1 has finitely many connected components. For each connected component C, the following holds: if for some game Γ ∈ C and profile σ ∈ Σ the set of profiles of pure strategies that are the players’ optimal replies to σ is T = T1 × T2 ⊂ S, then for every game Γ ∈ C there exists a profile σ ∈ Σ with the same support as σ and such that in Γ the set of pure optimal replies to σ is T . PROOF: Let X = G × Σ and let p : X → G be the natural projection. For each pair R = R1 × R2 and T = T1 × T2 of subsets of S, let X(R T ) be the set of (Γ σ) in X such that, for each n, Rn is the support of σn and Tn is the set of n’s pure optimal replies in Γ to the mixed strategy σm of the other player. By the generic local triviality theorem in Bochnak, Coste, and Roy (1998) there exists a closed, lower-dimensional, semi-algebraic subset G1 of G such G \G1 has finitely many connected components. Moreover, for each connected component C of G \G1 there exist (i) a semi-algebraic fiber F , (ii) for each pair (R T ), a subset F(R T ) of F , and (iii) a homeomorphism h : C × F → p−1 (C)
ON FORWARD INDUCTION
25
with the properties that (a) p ◦ h(Γ f ) = Γ for all Γ ∈ C, f ∈ F and (b) h maps C × F(R T ) homeomorphically onto p−1 (C) ∩ X(R T ) for each (R T ). Suppose T is the set of profiles of pure optimal replies to σ in a game Γ ∈ C. Let R be the support of σ. Then (Γ σ) belongs to X(R T ). Therefore, there exists f ∈ F(R T ) such that h(Γ f ) = (Γ σ). For each Γ ∈ C, let σ (f ) be the unique mixed strategy in Σ for which h(Γ f ) = (Γ σ (f )). Then the support of σ (f ) is R and the set of profiles of pure optimal replies in Γ to σ (f ) is T . Q.E.D. APPENDIX B: FORWARD INDUCTION IN THE NORMAL FORM The classical view in game theory is that the normal form of a game is sufficient to capture all strategically significant aspects. Hence the question arises as to whether we can state a comparable version of forward induction for a game in normal form. Here we provide one such definition. The following three components of Definition 3.5 for a game in extensive form need to be rephrased in terms of the normal form: (i) weakly sequential equilibria, (ii) relevant strategies, and (iii) restriction of beliefs to those induced by relevant strategies whenever possible. As will be seen below, if the sequential rationality requirement in the definition of weakly sequential equilibria is strengthened slightly (and only for nongeneric games), then the corresponding definition of forward induction has a normal-form counterpart. Given a game G in normal form, let σ be a profile of players’ mixed strategies and let b be an equivalent profile in behavioral strategies for an extensiveform game Γ with that normal form. Reny (1992, Proposition 1) showed that σ is a normal-form perfect equilibrium of G iff in Γ there exists a sequence bε of completely mixed profiles converging to b such that for each player n and each information set hn that bn does not exclude, the action prescribed by bn at hn is optimal against bε−n for all small ε. Thus the difference between weakly sequential equilibrium and normal-form perfect equilibrium is analogous to that between sequential equilibrium and extensive-form perfect equilibrium: one requires optimality only in reply to the limit, while the other requires optimality in reply to the sequence as well. Reny also showed that weakly sequential equilibria coincide with normal-form perfect equilibria for generic extensiveform games. Therefore, a perfect equilibrium seems to be the right normalform analog of a weakly sequential equilibrium. Suppose Σ∗ is a set of Nash equilibria of G. (To fix ideas, Σ∗ could be the set Σ(P) of equilibria inducing an outcome P in an extensive-form version of the game, but to allow applications to nongeneric games, we allow multiple outcomes.) In the extensive-form case, we said that a strategy was relevant if it was optimal against a pair of profiles of strategies and beliefs inducing the given outcome. But as noted above, if we insist on optimality along the sequence, then the appropriate normal-form definition of a relevant strategy becomes: a strategy is relevant if it is optimal against a sequence of ε-perfect equilibria converging to an equilibrium in Σ∗ .
26
S. GOVINDAN AND R. WILSON
Finally, we turn to belief restrictions. The idea in the extensive-form case is that if an information set hn of player n is reached by a profile of relevant strategies of his opponents, then he assigns zero probability to continuations that are enabled only by profiles that contain an irrelevant strategy for one of the other players. Let R−n (hn ) be the set of profiles of relevant strategies of n’s opponents that reach such an hn . If we use a sequence σ ε of normal-form profiles to generate players’ beliefs and their continuation strategies, then the belief restriction says that n’s belief at hn and the continuation strategies of his opponents should be obtained from the limit of the sequence of conditional distributions over R−n (hn ) induced by the sequence σ ε . That is, the beliefs at all information sets of all players that are reached by relevant strategies can be generated from the sequence of conditional distributions confined to relevant strategies. Because we insist on optimality along the sequence, what we obtain is a perfect equilibrium with a restriction on the form of its representation as a lexicographic probability system, as in Blume, Brandenburger, and Dekel (1991, Propopositions 4, 7). The restriction is that any profile that includes an irrelevant strategy for some player should occur later in the lexicographic sequence than those profiles that include only relevant strategies. This implements the basic requirement that each player believes the other is using a relevant strategy as long as that hypothesis is tenable. Thus, we are led to the following definition: DEFINITION B.1—Normal-Form Forward Induction: A set of Nash equilibria satisfies normal-form forward induction if it contains a perfect equilibrium whose lexicographic representation has all profiles of relevant strategies occurring before all profiles that include irrelevant strategies. In general, this is a stronger requirement than the one in the text, but for a generic two-player game in extensive form with perfect recall it can be shown that the set of weakly sequential equilibria inducing an outcome P satisfies the above definition iff P satisfies forward induction as defined in the text. The reason for this equivalence is similar to the reason that weakly sequential equilibria and normal-form perfect equilibria coincide for generic extensiveform games as established by Reny (1992, Proposition 1). An implication is that the analog of Theorem 6.1 is true with this definition of forward induction, that is, the set of Nash equilibria resulting in an invariant sequential equilibrium outcome of a two-player game in extensive form with perfect recall and generic payoffs satisfies normal-form forward induction. REFERENCES BANKS, J., AND J. SOBEL (1987): “Equilibrium Selection in Signaling Games,” Econometrica, 55, 647–661. [3]
ON FORWARD INDUCTION
27
BATTIGALLI, P., AND M. SINISCALCHI (2002): “Strong Belief and Forward Induction Reasoning,” Journal of Economic Theory, 106, 356–391. [4,11] BLUME, L., A. BRANDENBURGER, AND E. DEKEL (1991): “Lexicographic Probabilities and Equilibrium Refinements,” Econometrica, 59, 81–98. [26] BOCHNAK, J., M. COSTE, AND M.-F. ROY (1998): Real Algebraic Geometry. Berlin: Springer Verlag. [24] CHEN, Y., N. KARTIK, AND J. SOBEL (2008): “Selecting Cheap-Talk Equilibria,” Econometrica, 76, 117–136. [20] CHO, I., AND D. KREPS (1987): “Signalling Games and Stable Equilibria,” Quarterly Journal of Economics, 102, 179–221. [3,7,8,12,20] CHO, I., AND J. SOBEL (1990): “Strategic Stability and Uniqueness in Signaling Games,” Journal of Economic Theory, 50, 381–413. [4] FUDENBERG, D., AND J. TIROLE (1993): Game Theory. Cambridge, MA: MIT Press. [4,7,13] GOVINDAN, S., AND R. WILSON (2001): “Direct Proofs of Generic Finiteness of Nash Equilibrium Outcomes,” Econometrica, 69, 765–769. [17,20] (2006): “Sufficient Conditions for Stable Equilibria,” Theoretical Economics, 1, 167–206. [14,23] (2007): “On Forward Induction,” Research Report 1955, Business School, Stanford, CA. [5] HILLAS, J. (1994): “How Much of Forward Induction Is Implied by Backward Induction and Ordinality,” Mimeo, University of Auckland, New Zealand. [6] HILLAS, J., AND E. KOHLBERG (2002): “Conceptual Foundations of Strategic Equilibrium,” in Handbook of Game Theory, Vol. III, ed. by R. Aumann and S. Hart. New York: Elsevier, 1597–1663. [1,3,4,20] KOHLBERG, E. (1990): “Refinement of Nash Equilibrium: The Main Ideas,” in Game Theory and Applications, ed. by T. Ichiishi, A. Neyman, and Y. Tauman. San Diego: Academic Press. [3] KOHLBERG, E., AND J.-F. MERTENS (1986): “On the Strategic Stability of Equilibria,” Econometrica, 54, 1003–1037. [3,7,13,14] KOHLBERG, E., AND P. RENY (1997): “Independence on Relative Probability Spaces and Consistent Assessments in Game Trees,” Journal of Economic Theory, 75, 280–313. [1] KREPS, D. (1990): A Course in Microeconomic Theory. Princeton, NJ: Princeton University Press. [9] KREPS, D., AND G. RAMEY (1987): “Structural Consistency, Consistency, and Sequential Rationality,” Econometrica, 55, 1331–1348. [10] KREPS, D., AND J. SOBEL (1994): “Signalling,” in Handbook of Game Theory, Vol. II, ed. by R. Aumann and S. Hart. New York: Elsevier, 849–868. [4] KREPS, D., AND R. WILSON (1982): “Sequential Equilibria,” Econometrica, 50, 863–894. [1,2,10] KUHN, H. (1953): “Extensive Games and the Problem of Information,” in Contributions to the Theory of Games, Vol. II, ed. by H. Kuhn and A. Tucker. Princeton, NJ: Princeton University Press, 193–216. Reprinted in H. Kuhn (ed.) (1997): Classics in Game Theory. Princeton, NJ: Princeton University Press. [2,12] MCLENNAN, A. (1985): “Justifiable Beliefs in Sequential Equilibrium,” Econometrica, 53, 889–904. [4] MERTENS, J.-F. (1989): “Stable Equilibria—A Reformulation, Part I: Definition and Basic Properties,” Mathematics of Operations Research, 14, 575–625. [14] PEARCE, D. (1984): “Rationalizable Strategic Behavior and the Problem of Perfection,” Econometrica, 52, 1029–1050. [4] RENY, P. (1992): “Backward Induction, Normal Form Perfection and Explicable Equilibria,” Econometrica, 60, 627–649. [3,4,10,12,21,22,25,26] SPENCE, A. M. (1974): Market Signaling. Cambridge, MA: Harvard University Press. [9] VAN DAMME, E. (1989): “Stable Equilibria and Forward Induction,” Journal of Economic Theory, 48, 476–496. [3]
28
S. GOVINDAN AND R. WILSON
(2002): “Strategic Equilibrium,” in Handbook of Game Theory, Vol. III, ed. by R. Aumann and S. Hart. New York: Elsevier, 1523–1596. [3,4,20]
Dept. Economics, University of Iowa, Iowa City, IA 52242, U.S.A.;
[email protected] and Business School, Stanford University, Stanford, CA 94305-5015, U.S.A.;
[email protected]. Manuscript received February, 2007; final revision received May, 2008.
Econometrica, Vol. 77, No. 1 (January, 2009), 29–69
PUBLIC VS. PRIVATE OFFERS IN THE MARKET FOR LEMONS BY JOHANNES HÖRNER1 AND NICOLAS VIEILLE We study the role of observability in bargaining with correlated values. Short-run buyers sequentially submit offers to one seller. When previous offers are observable, bargaining is likely to end up in an impasse. In contrast, when offers are hidden, agreement is always reached, although with delay. KEYWORDS: Bargaining, delay, impasse, observability, lemons problem.
1. INTRODUCTION WE STUDY THE ROLE of observability in bargaining with correlated values. More precisely, we study how the information available to potential buyers affects the probability of reaching an agreement. Our main result is that if discounting is low and the static incentive constraints preclude first-best efficiency, agreement is always reached when previous offers are kept hidden, while agreement is unlikely to be reached when they are made public. The information and payoff structures are as in Akerlof’s (1970) market for “lemons.” One seller is better informed than the potential buyers about the value of the single unit for sale. It is common knowledge that trade is mutually beneficial. All potential buyers share the same valuation for the unit, which strictly exceeds the seller’s cost. The game is dynamic. The seller bargains sequentially with potential buyers until agreement is reached, if ever, and delay is costly. When it is his turn, a buyer makes a take-it-or-leave-it offer to the seller. That is, the setting is formally a search model: the seller may rationally choose to reject available offers in return for the opportunity to wait for higher prospective offers. This search process is without recall. To take a specific example, consider the sale of a residential property. In most countries, houses are sold through bilateral bargaining. Potential buyers come and go, engaging in private negotiations with the seller, until either an agreement with one of them is reached or the house is withdrawn from the market. Typically, potential buyers know how long the house has been for sale, which provides a rough estimate of the number of rejected offers. However, past offers remain hidden and “only a bad agent would reveal them,” in the words of one broker. Similarly, in most labor markets, employers do not observe the actual offers that the applicant may have previously rejected, but they can infer how long he has been unemployed from the applicant’s vita. In contrast, in other bargaining settings, such as corporate acquisition via tender offer, previous offers are commonly observed. Remarkably, our analysis supports the broker’s point of view. With public offers, the equilibrium outcome is unique. Bargaining typically ends up in an 1
Hörner’s work on this project began when he was affiliated with Northwestern University.
© 2009 The Econometric Society
DOI: 10.3982/ECTA6917
30
J. HÖRNER AND N. VIEILLE
impasse: only the first buyer submits an offer that has any chance of being accepted. If this offer is rejected, no further serious offer is submitted on the equilibrium path. This is rather surprising, since it is common knowledge that no matter how low the quality of the unit may be, it is still worth more to the potential buyers than to the seller. Indeed, the second buyer, for instance, would submit a serious offer if he were the last. So it is precisely the competition from future buyers that deters him from making such an offer. Yet on the equilibrium path, all these buyers also submit losing offers. Why can a buyer not break the deadlock by making an offer above the seller’s lowest cost but below the buyer’s lowest possible valuation? As we show, such an offer necessarily triggers an aggressive offer by the next buyer. Because the seller can wait, this dramatically increases the price that a given type of the seller would find acceptable. Gains from trade between the current buyer and the seller no longer exist once we account for this seller’s outside option, rendering such a deviation futile.2 This result provides an explanation for impasses in bargaining. While standard bargaining models are often able to explain delay, agreement is always reached eventually. Exceptions either rely on behavioral biases (see Babcock and Loewenstein (1997)) or Pareto-inefficient commitments (see Crawford (1982)). Here, it is precisely the inability of the seller not to solicit another offer that discourages potential buyers from submitting serious offers. In contrast, agreement is always reached when offers are private. Because the seller cannot use his rejection of an unusually high offer as a signal to elicit an even higher offer by the following buyer, buyers are not deterred from submitting serious offers. To put it differently, the unique equilibrium outcome with public offers cannot be an equilibrium outcome with private offers. Suppose, per impossibile, that such an equilibrium were to exist. Then consider a deviation in which a potential buyer submits an offer that is both higher than the seller’s lowest possible cost yet lower than the buyer’s lowest possible valuation. Future potential buyers would be unaware of the specific value of this out-of-equilibrium offer. Hence, turning it down would not change their beliefs about the unit’s value. Thus, given that the seller expects to receive losing offers thereafter, he should accept the offer if his cost is low enough. This, in turn, means that the offer is a profitable deviation for the buyer. Our main result may appear surprising in light of one of the “linkage” principles in auction theory, stating that disclosure of additional information increases the seller’s expected revenue. In our dynamic setup, it is important to distinguish between how much information can possibly be revealed given the information structure and the information that is actually revealed in equilibrium. While finer information could be transmitted with public offers than with 2 Academic departments are well aware of this problem when considering making senior offers. As clearly this example cannot fail any of the underlying rationality assumptions, the prevalence of such offers raises an interesting puzzle.
PUBLIC VS. PRIVATE OFFERS
31
private offers, this is not what happens in equilibrium: because all offers but the first one are losing offers, no further information about the seller is ever revealed, so that, somewhat paradoxically, more information is communicated with private offers. As already mentioned, this is a search model. However, unlike most of the search literature, the distribution of offers is not fixed, but endogenously derived. The analysis shows that random offers can, indeed, be part of the equilibrium strategies. In addition, it shows that the offer distribution depends on the information available to the offerers. Therefore, it also suggests that it is not always innocuous to treat the offer distribution as fixed while considering variants of the standard search models. The general setup is described in Section 2. Section 3 solves the case of public offers and addresses issues of robustness. Private offers are considered in Section 4. Related literature, results, and extensions are discussed in Section 5. Proofs are in the Appendix. Extensions to the two-type case are provided in the Supplemental Material (Hörner and Vieille (2009)). 2. THE MODEL We consider a dynamic game between a single seller with one unit for sale and a countably infinite number of potential buyers, or buyers for short. Time is discrete and indexed by n = 1 ∞. At each time or period n, one buyer makes an offer for the unit. Each buyer makes an offer only at one time, and we refer to buyer n as the buyer who makes an offer in period n, provided the seller has accepted no previous offer. After receiving the offer, the seller either accepts or rejects the offer. If the offer is accepted, the game ends. If the offer is rejected, a period elapses and it is another buyer’s turn to submit an offer. The quality q of the good is determined by Nature and is uniformly distributed over the interval [q1 1] for some q1 < 1. The value of q is the seller’s private information, but its distribution is common knowledge. We refer to q as the seller’s type. Given q, the seller’s cost of providing the unit is c(q). The valuation of the unit to buyers is common to all of them and is denoted by v(q). We assume that c(·) is (strictly) positive, (strictly) increasing, and twice differentiable, with bounded derivatives. We also assume that v(·) is positive, increasing, and continuously differentiable. Moreover, we assume that v is positive. We set Mc = max |c |, Mc = sup |c |, Mv = max |v |, M = max{Mc Mc Mv }, and mv = min |v | > 0. Observe that the assumption that q is uniformly distributed is made with little loss of generality, given that few restrictions are imposed on the functions v and c.3 3 In particular, our results are still valid if the distribution of q has a bounded density, bounded away from zero.
32
J. HÖRNER AND N. VIEILLE
We assume that gains from trade are always positive: v(q) − c(q) > 0 for all q ∈ [q1 1]. The seller is impatient, with discount factor δ < 1. We are particularly interested in the case in which δ is sufficiently large. To be specific, we set δ¯ := 1 − mv /3M ¯ and assume throughout that δ > δ. Buyer n submits an offer pn that can take any real value. An outcome of the game is a triple (q n pn ), with the interpretation that the realized type is q and that the seller accepts buyer n’s offer of pn (which implies that he rejected all previous offers). The case n = ∞ corresponds to the outcome in which the seller rejects all offers (set p∞ equal to zero). The seller’s von Neumann– Morgenstern utility function over outcomes is his net surplus δn−1 (pn − c(q)) when n < ∞, and zero otherwise. An alternative formulation that is equivalent to the one above is that the seller incurs no production cost but derives a perperiod gross surplus, or reservation value, of (1 − δ)c(q) from owning the unit. It is immediate to verify that this interpretation yields the same utility function. Buyer n’s utility is v(q) − pn if the outcome is (q n pn ), and zero otherwise (discounting is irrelevant since buyers make only one offer). We define the players’ expected utility over lotteries of outcomes, or payoff for short, in the standard fashion. We consider both the case in which offers are public and the case in which previous offers are private. It is worth pointing out that the results for the case in which offers are public also hold for any information structure (about previous offers) in which each buyer n > 1 observes the offer of buyer n − 1. A history (of offers) hn−1 ∈ H n−1 in case no agreement has been reached at time n is a sequence (p1 pn−1 ) of offers that were submitted by the buyers and rejected by the seller (we set H0 equal to ∅). A behavior strategy for the seller is a sequence {σSn }, where σSn is a probability transition from [q1 1] × H n−1 × R into {0 1}, mapping the realized type q, the history hn−1 , and buyer n’s price pn into a probability of acceptance. In the public case, a strategy for buyer n is a probability transition σBn from H n−1 to R.4 In the private case, a strategy for buyer n is a probability distribution σBn over R. Observe that whether offers are public or private, the seller’s optimal strategy must be of the cutoff type. That is, if σSn (q hn−1 pn ) assigns a positive probability to accepting for some q, then σSn (q hn−1 pn ) assigns probability 1 to it for all q < q. The proof of this skimming property is standard and can be found in, for example, Fudenberg and Tirole (1991, Chapter 10, Lemma 10.1). The infimum over types q accepting a given offer is called the indifferent type (at history (hn−1 pn ) given the strategy profile). Since the specification of the 4 That is, for each hn−1 ∈ H n−1 , σBn (hn−1 ) is a probability distribution over R, and the probability σBn (·)[A] assigned to any Borel set A ⊂ R is a measurable function of hn−1 , and similarly for σSn .
PUBLIC VS. PRIVATE OFFERS
33
action of the seller’s indifferent type does not affect payoffs, we also identify equilibria which only differ in this regard. For definiteness, in all formal statements, we shall follow the convention that a seller’s type that is indifferent accepts the offer. For conciseness, we shall usually omit to specify that some statements only hold “with probability 1.” For instance, we shall say that the seller accepts the offer when he does so with probability 1. We use the perfect Bayesian equilibrium (PBE) concept as defined in Fudenberg and Tirole (1991, Definition 8.2).5 In both the public and the private case, this implies that upon receiving an out-of-equilibrium offer, the continuation strategy of the seller is optimal. In the public case, this also implies that after any history on or off the equilibrium path along which all offers submitted by buyers were smaller than c(1), the belief (over seller’s types) of the remaining buyers is common to all of them and computed on the assumption that the seller’s reasons for rejecting previous offers were rational. Thus, in the public case, after any such history, the belief of buyer n over q is the uniform distribution over some interval [qn 1], where qn may depend on the sequence of earlier offers. In the private case, the only nontrivial information sets that are reached with probability 0 occur in periods such that, along the equilibrium path, the probability is 1 that the seller accepts some earlier offer. The specification of beliefs after such information sets turns out to be irrelevant. Given some (perfect Bayesian) equilibrium, we follow standard terminology in calling a buyer’s offer serious if it is accepted by the seller with positive probability. An offer is losing if it is not serious. Clearly, the specification of losing offers in an equilibrium is, to a large extent, arbitrary. Therefore, statements about uniqueness are understood to be made up to the specification of the losing offers. Finally, an offer is a winning offer if it is accepted with probability 1. We briefly sketch here the static version with one buyer, similar to Wilson (1980). The unique buyer submits a take-it-or-leave-it offer. The game then ends whether the offer is accepted or rejected, with payoffs specified as before (with n = 1). Clearly, the seller accepts any offer p provided p ≥ c(q). Therefore, the buyer offers c(q∗ ), where q∗ ∈ [q1 1] is the maximizer of
q
(v(t) − c(q)) dt q
1
with respect to q. Observe that the corresponding payoff of the buyer must be positive, because the buyer can always submit an offer in the interval (c(q1 ) v(q1 )). 5
Formally speaking, Fudenberg and Tirole defined perfect Bayesian equilibria for finite games of incomplete information only. The suitable generalization of their definition to infinite games is straightforward and is omitted.
34
J. HÖRNER AND N. VIEILLE
3. OBSERVABLE OFFERS Throughout this section, we maintain the assumption that offers are public. We prove that the market breaks down in that case. 3.1. General Statement Our main result, Proposition 1, is paradoxical: there is a unique equilibrium outcome, according to which the first buyer’s offer is rejected with positive probability, and all subsequent offers are rejected with probability 1.6 We first present the equilibrium outcome, then provide some intuition. Given a type q and since v is increasing, there exists at most one value of x ∈ [q1 q) that solves 1 c(q) = q−x
q
v(t) dt x
That is, for this value of x, the expected valuation over types in the interval [x q] is equal to the highest cost in this interval, c(q). We denote by f (q) := x ∈ [q q) this value, as a function of q, if it exists. The value f (q) is well defined whenever asymmetric information is severe enough that the buyer’s expected value does not exceed the highest seller’s cost. This is the case if either the range of existing qualities is large or valuations are close to costs. Define the strictly decreasing sequence qk as follows: q0 = 1 and qk+1 = f (qk ) as long as f (qk ) is defined (that is, as long as f (qk ) ≥ q1 ). Because minq {v(q) − c(q)} > 0, it must be that q − f (q) > κ for some κ > 0 and all q. Hence the sequence (qk ) must be finite, and we denote the last and smallest element of this sequence by qK ≥ q1 . Note that this sequence is always well defined, as q0 = 1 > q1 . The mapping f that is used in the definition of (qk ) plays a key role in the analysis of the static model of adverse selection between one seller and two or more buyers. Indeed, consider the “static” game between n ≥ 2 buyers simultaneously announcing prices and the seller who then either keeps the unit or sells it at the highest price, p. Obviously, the seller only sells the unit if p ≥ c(q) or q ≤ c −1 (p). Therefore, at equilibrium it must be that p ≥ p implies that p ≥ E[v(q)|q ≤ c −1 (p )] (where c −1 (p) = 1 for p ≥ c(1)) with equality for p = p, so that the winner barely breaks even and no higher price yields a positive payoff. That is, q1 = f (c −1 (p)) (unless a winning offer yields a positive payoff).7 Note that any such offer p strictly exceeds the optimal offer that a lone buyer would submit. 6
The probability of rejection can be arbitrarily close to 1, depending on q1 . More generally, in all equilibria, there are at least two buyers whose offers all have the property that higher offers cannot yield a positive payoff. Because f −1 need not be uniquely defined, the equilibrium need not be unique, pure, or symmetric. 7
PUBLIC VS. PRIVATE OFFERS
35
While it is possible that qK = q1 , our result is stated here, for simplicity, for the generic case in which qK > q1 . ¯ There is a unique equilibPROPOSITION 1: Assume that qK > q1 and δ > δ. rium outcome, which is independent of δ. On the equilibrium path, the first buyer submits the offer c(qK ), which the seller accepts if and only if q ≤ qK . If this offer is rejected, all buyers n > 1 submit a losing offer, not exceeding c(qK ). This proposition, proved in the Appendix, both asserts that a bargaining impasse can happen in equilibrium and that it must happen in equilibrium. To shed some light on the first of these assertions, we describe one strategy profile and argue that it is an equilibrium. The seller’s acceptance rule is stationary and continuous. If p ≥ c(q0 ), the seller accepts p, independently of his type. Fix now an offer p and consider the interval [c(qk ) c(qk−1 )), k ≥ 1, that contains p:8 • If p belongs to the interval [c(qk ) (1 − δ)c(qk ) + δc(qk−1 )), a seller with type q accepts the offer if and only if q ≤ qk . • If p belongs to the interval [(1 − δ)c(qk ) + δc(qk−1 ) c(qk−1 )), a seller with type q accepts the offer if and only if p − c(q) ≥ δ(c(qk−1 ) − c(q)). This strategy is optimal given the following buyers’ strategies. Given some stage n and a history up to n, let qn denote the lowest type willing to reject all previous offers, so that buyer n’s belief is uniform over the interval [qn 1]. The offer of buyer n depends both on qn and on the previous offer, pn−1 : • If qn ∈ (qk+1 qk ), for some k, the buyer offers c(qk ). • If qn = qk for some k, the buyer randomizes between c(qk ) and c(qk−1 ), so that, given pn−1 , the seller’s type qk was indeed indifferent in the previous stage between accepting and rejecting pn−1 ; that is, the probability π assigned to the offer c(qk−1 ) solves pn−1 − c(qk ) = δπ(c(qk−1 ) − c(qk )). Let us now provide some intuition on why this is an equilibrium, by focusing on, say, the second buyer. On the equilibrium path, this buyer is supposed to make the losing offer c(qK ). Why can he not do any better? Consider Figure 1 as an illustration. The price P(q) denotes the lowest price that the second buyer must offer so that the seller’s type q is indifferent between accepting or not. If q ≤ qK , offering the price c(qK ) trivially suffices, since this is a losing offer. If q ∈ (qK qK−1 ], then the second buyer must offer the price (1 − δ)c(q) + δc(qK−1 ), since the next price offered will be c(qK−1 ). Similarly, if q ∈ (qK−1 qK−2 ], then the second buyer must offer the price (1 − δ)c(q) + δc(qK−2 ), since the next price offered will be c(qK−2 ), etcetera. There are three features to notice. First, the 8 If p belongs to the interval [c(q1 ) c(qK )), a seller with type q accepts the offer if and only if p − c(q) ≥ δ(c(qK ) − c(q)).
36
J. HÖRNER AND N. VIEILLE
FIGURE 1.
function P is discontinuous at qk for all k ≤ K. Second, its graph goes through the points (qk c(qk )). Finally, as δ → 1, it becomes arbitrarily flat on every subinterval (qk qk−1 ]. Because there are known gains from trade, the cost c(q) falls short of ˜ q˜ ≤ q] whenever q ≤ qK−1 (recall that the conditional expected value E[v(q)| qK = f (qK−1 )). So offers for which the seller’s indifferent type exceeds qK−1 are necessarily unprofitable (they would be unprofitable even if no future buyer ever came). By offering c(qK−1 ), the second buyer just breaks even. Finally, for δ close enough to 1, the function P, being arbitrarily flat, must exceed the con˜ q˜ ≤ q] on the entire interval (qK qK−1 ), so that ditional expected value E[v(q)| offers for which the seller’s indifferent type belongs to this range are unprofitable as well. This exhausts all possibilities. As a result, there are no offers that the buyer could make that would ensure him positive profits, and, in fact, he is indifferent between making a losing offer and the offer c(qK−1 ).9 This is not the only PBE, but other PBEs differ only in irrelevant ways (see the Appendix). As for uniqueness, there is no easy argument. In Section 3.2, we provide a sketch of the proof that relies on the following insights. The first insight is that, provided there is little heterogeneity—that is, if qn is close enough to 1— making a winning offer is optimal (as doing so is optimal in the static case). 9 The discontinuities of the function P are the cause of nonexistence of (strong) Markov equilibrium; if a buyer makes an offer in such an interval, the next buyer must randomize accordingly.
PUBLIC VS. PRIVATE OFFERS
37
Next, if the next offer is independent of the current offer, at least in some range, the indifferent type becomes very sensitive to the current offer and, within that range, only the highest or lowest offers need be considered. These two observations trigger some type of unraveling, with the result that a winning offer is submitted whenever doing so yields a positive payoff, that is, when qn > q1 . The second main insight is that when q1 < q1 , no type beyond q1 can possibly trade at equilibrium. Indeed, when qn < q1 , buyer n does not make an offer with indifferent type strictly above q1 . Hence, full trade would require that some buyer n make an offer that type q1 accepts, eventually followed by a winning offer. Such a buyer could profitably deviate: by making instead an offer with indifferent type slightly below q1 , buyer n would delay the timing of the winning offer. In turn, this implies that the corresponding price is bounded below his “equilibrium” price, while the loss in the probability of acceptance is arbitrarily small. As a consequence, an offer equal to c(q1 ) is accepted by all types up to q1 and is followed by losing offers. Hence, and this is the last insight, it is in a sense as if the highest existing type is q1 , rather than q0 = 1, and the previous arguments can be repeated, first on the interval [q2 q1 ], next on [q3 q2 ], and so on. It is now possible to draw a comparison between the dynamic version with public offers and the static version with one buyer. Observe that, depending on the exact value of q1 , q∗ could be anywhere in the interval (q1 qK−1 ), so that both qK > q∗ and qK ≤ q∗ may occur, where c(q∗ ) is the optimal offer in the static version.10 This means that from the seller’s point of view, the comparison between the dynamic version and the static version with a unique buyer is ambiguous. The probability of sale and the expected revenue could be larger in either format depending on q1 . However, it makes more sense to compare this equilibrium outcome with the equilibria in the static game with multiple buyers. This comparison is immediate, as the offer in the static case must be at least as large as c(qK ). Thus, the seller is better off in the static version, having the different buyers compete simultaneously for the unit, rather than one at a time. The probability of trade is higher in the static version. Only the first buyer is better off in the dynamic version, while all other buyers are indifferent. As an immediate consequence, the bargaining outcome generically fails to be ex ante efficient. That is, there exists an incentive-compatible and individually rational mechanism that yields higher expected gains from trade. Indeed, with a single buyer, consider the mechanism in which the seller must accept or reject the fixed price c(q), where q is the largest root of f (q) = q1 . In the linear case that we introduce in the next section, one has q1 = (1 − α)q∗ while the only constraint on q1 is 1−α q < q1 ≤ qK . For low (resp. high) values of q1 in this interval, the dynamic 1+α K (resp. static) version yields a higher payoff to the seller. 10
38
J. HÖRNER AND N. VIEILLE
As mentioned, it is surprising that trade does not always take place. In a sense, there is infinite delay as soon as the lowest type reaches qk . The most recent contribution to this literature, Deneckere and Liang (2006), considered the case of a single long-run buyer, who had the same discount factor as the seller, rather than a sequence of short-run buyers. They defined a finite sequence (qk ) similar (but not identical) to ours and found that in the unique stationary equilibrium, as the lowest type approaches qk from below, the (single) buyer repeatedly submits offers accepted with small probability, so that delay ensues. As the time period between successive offers vanishes, this delay remains finite and bounded away from zero. In the limit, bursts of trade alternate with long periods of delay. This implies that the equilibrium acceptance function of the seller becomes increasingly similar to the equilibrium acceptance of the seller in Proposition 1. Note, however, that because delay remains finite in their model, the price accepted by type qk exceeds c(qk ), so that the actual values of the two sequences differ. Furthermore, this limit comparison ¯ is only illustrative, since the conclusion of Proposition 1 is valid for every δ > δ. Proposition 1 remains valid in the case of a single long-run buyer, provided the buyer is much less patient than the seller. Hence, when combined, these two results point out that the possibility of trade depends on the relative patience of the buyer relative to the seller, an insight already hinted at by Evans (1989) in the case of binary values. Just as Proposition 1 remains valid with a single buyer who is sufficiently more impatient than the seller, it is also valid if the number of buyers is finite, as long as the probability that each of them is selected to make the offer in each of the countably many periods is sufficiently small. The results of Vincent (1989) and Deneckere and Liang (2006) rely on the screening of types that bargaining over time affords. Because delay is costly for the seller, buyers become more optimistic over time, so that uncertainty is progressively eroded. Our no-trade result points to another familiar force in dynamic games; namely, the absence of commitment. Indeed, if the horizon were finite, the last buyer would necessarily submit a serious offer. However, since Coase’s (1972) original insight, the inability to commit has always been associated with an increase in the probability of trade. To quote Deneckere and Liang (2006, p. 1313), the “absence of commitment power implies that bargaining agreement will eventually be reached.” This is because the traditional point of view emphasizes the inability of the buyer to commit to not making another offer. Instead, the driving force here is the inability of the seller not to solicit another offer. This leads to a collapse in the probability of trade and an increase in the inefficiency. 3.2. Sketch of the Proof We here provide a sketch of why all equilibria result in an impasse. For simplicity, here we let the cost and the valuation functions be given by c(q) = q
PUBLIC VS. PRIVATE OFFERS
39
and v(q) = (1 + α)q, where α ∈ (0 1). With such a parametrization, the average valuation of types in any interval [a b] is (1 + α) a+b , so that the values qk 2 defined in the previous section are given by qk = βk , where β := 1−α ∈ (0 1). 1+α 3 2 For concreteness, we assume here that β < q1 < β . We let an equilibrium be given. Following any history, we let q stand for the currently lowest type. We shall solve the game “backwards.” Starting with high values and moving then to lower values of q, we shall prove that the equilibrium offers are uniquely determined. The first and main insight of the proof is that a gradual resolution of uncertainty ending with full trade cannot be an equilibrium outcome. To be specific, ˜ such that any equilibrium offer is either rejected by all there is a threshold, q, ˜ 1] or accepted by all types in (q ˜ 1]. types in (q We establish this claim in several steps. Note first that no equilibrium offer exceeds c(1) = 1 and that an offer of c(1) = 1 is winning. Indeed, following any history, any offer above v(1) = 1 + α is strictly dominated. Hence, no such offer is ever made at equilibrium. Therefore, following any history, any offer above 1 + δα is accepted by all types, since the discounted benefit from turning it down is at most δα. This implies in turn that any offer which exceeds 1 + δ2 α is accepted by all types, etcetera. Next, we observe that when q is close enough to 1, the equilibrium offer of the current buyer is winning. Indeed, if this buyer were the last one, an offer q ≤ 1 would be accepted by all types in [q q]. The buyer’s expected payoff would then be given by the expected benefit conditional on trade, q+q q−q (1 + α) 2 − q, times the probability of this offer being accepted, 1−q . This payoff is strictly increasing on [q 1], as soon as q ≥ 1 − α. Hence, a winning offer would be submitted. The competition from future buyers only makes the winning offer relatively more attractive, since the offer c(1) = 1 yields the same payoff with or without competition, while any other offer cannot possibly yield more with competition than without. ˜ Suppose that the current buyer considers an Fix a history after which q < q. offer that type q > q˜ would accept. Since the lowest type in the following stage ˜ a winning offer would follow. Hence type q would reject would then exceed q, any offer below p(q) := δ + (1 − δ)q Conversely, any offer above p(q) would be accepted by type q. Since δ is close to 1, it is apparent that p(q) is relatively insensitive to q. In particular, if the buyer were to derive a nonnegative payoff from the offer p(q), any higher offer p(q ) (q > q) would yield an even higher payoff, as (i) it would be accepted with a higher probability and (ii) the expected profit conditional on trade would increase as well. As a consequence, it cannot be optimal to make an offer ac˜ 1]. cepted by only a fraction of types in [q
40
J. HÖRNER AND N. VIEILLE
˜ 11 Whenever q > q, ˜ the equilibThis fact allows us to uniquely pin down q. rium offer is winning. On the other hand and by definition, there are histories following which q < q˜ is arbitrarily close to q˜ and following which the equilib˜ Such an offer is accepted with a rium offer p is rejected by all types beyond q. probability arbitrarily close to 0 and hence yields an expected payoff close to 0. By a limiting argument, it follows that a winning offer must yield an expected ˜ Equivalently, the average valuation of all types in [q ˜ 1] payoff of 0 when q = q. is equal to the winning offer, 1, so that q˜ = β. The second insight of the proof is that no type above q˜ = β can possibly trade on the equilibrium path. We argue by contradiction. If not and in the light of our earlier claims, there must be some history such that q < β, and for which (i) the equilibrium offer of the current buyer is accepted by all types up to β and (ii) a winning offer is eventually submitted with positive probability. This equilibrium offer of the current buyer must exceed c(β); otherwise, seller’s types close to β would reject it. Among such histories, choose one for which the equilibrium offer p of the buyer is highest or close to being highest. We claim that if the current buyer were to offer slightly less, the lower offer p would still be accepted by all types up to β and would, therefore, do better—a contradiction. The logic behind this claim is as follows. If the indifferent type q for the lower offer p were below β, the next offer would at most be accepted by all types up to β, and type q would eventually accept an offer not exceeding p. But, provided p is close enough to p, type q would then strictly prefer to accept the lower offer p immediately. This shows that the indifferent type for p is indeed β. The last insight in the proof is that this scenario can be repeated for q ∈ (β2 β]. Let us elaborate on this. A consequence of the previous argument is that an offer of c(β) is accepted by all types up to β (and all offers after that are losing). As in the first part of the sketch, we claim that the equilibrium offer is c(β) whenever q < β is close enough to β. To see this, we mimic the earlier argument. If q ≥ β(1 − α), if the current buyer were the last, and if he were constrained not to make an offer above c(β), his optimal offer would be c(β). This implies that it is also optimal to offer c(β) given the subsequent buyers. The end of the proof follows similar lines. One first shows that the equilibrium offer is c(β) = β whenever q > β2 and that equilibrium offers are rejected by all types above β2 whenever q < β2 . We next argue that no type above β2 trades on the equilibrium path. In turn, this implies that an offer of c(β2 ) = β2 is accepted by all types up to β2 . Finally, we rely again on the comparison with the fictional scenario in which the current buyer would be last, to claim that c(β2 ) = β2 is the equilibrium offer whenever q < β2 is close enough 11 We formally define q˜ as the supremum over all histories of those values of q for which a winning offer need not immediately follow.
PUBLIC VS. PRIVATE OFFERS
41
to β2 . In the unique equilibrium outcome, all buyers offer c(β2 ) = β2 , and the first offer is accepted if and only if the seller’s type does not exceed β2 . 4. PRIVATE OFFERS 4.1. General Properties Throughout this section, we maintain the assumption that offers are private. As we are unable to construct equilibria in general, we first argue that an equilibrium exists. We will apply a fixed-point argument on the space of buyers’ strategies. Given a profile of buyers’ strategies σB , the buyers’ payoffs are computed using the optimal response of the seller to the profile σB . If no later buyer sets a price exceeding c(1), it is suboptimal for a given buyer to set such a price. Hence, for the purpose of equilibrium existence, we can limit the set of buyers’ mixed (or behavior) strategies to the set M([c(q1 ) c(1)]) of probability distributions over the interval [c(q1 ) c(1)], endowed with the weak- topology. The set of strategy profiles for buyers is thus the countable product M([c(q1 ) c(1)])N . It is compact and metric when endowed with the product topology. Since the random outcome of buyer n’s choice is not known to the seller unless he has rejected the first n − 1 offers, buyer n’s payoff function is not the usual multilinear extension of the payoff induced by pure profiles. Given a period n, we denote by qn (p σB ) the indifferent type when the offer p is submitted in stage n, given the strategy profile σB . It is jointly continuous in p and σB . As a result, the belief of any given buyer (viewed as a probability distribution over [q1 1]) is continuous with respect to σB , in the weak- topology. Hence, the set Bn (σB ) ⊂ M([c(q1 ) c(1)]) of best replies of buyer n to σB is both convex-valued and upper hemicontinuous with respect to σB . The existence of a (Nash) equilibrium follows from Glicksberg’s fixed-point theorem. To any such equilibrium, there corresponds a perfect Bayesian equilibrium, because all buyers’ histories are on the equilibrium path until trade occurs, if ever. While an equilibrium exists, it need not be unique. As an example, for c(q) = q, v(q) = (1 + α)q, α = 1/3, δ = 3/4, and q1 = 4249, it can be shown that the following two (and probably more) equilibria exist. • The first buyer offers p∗1 61 and attracts all types up to β = 1/2 for sure. The second buyer makes a losing offer. Buyer n ≥ 3 mixes between a winning and a losing offer, offering the winning price 1 with probability 3/20. • The first buyer randomizes between a losing offer and the offer p∗1 63 with indifferent type q1∗ 51, assigning a probability μ1 40 to the losing offer. The second buyer mixes between a losing offer and the offer p∗2 72 with indifferent type q2∗ 62, assigning a probability μ2 76 to the losing offer. Buyer n ≥ 3 mixes between a winning and a losing offer, offering the losing price with probability μ 94. We summarize the discussion so far in the following lemma.
42
J. HÖRNER AND N. VIEILLE
LEMMA 1: An equilibrium exists. For some parameters, the equilibrium is not unique. More precisely, the equilibrium is unique if and only if q1 > q1 , as defined in Section 3. The two equilibria described above differ quantitatively in terms of delay, revenue, and payoffs. Nevertheless, there are some qualitative similarities. Both equilibria involve mixed strategies. Also, trade occurs with probability 1. This is no coincidence. PROPOSITION 2: In all equilibria, trade occurs with probability 1. PROOF: Fix some equilibrium. Given q ∈ [q1 1], let Fn (q) denote the unconditional probability that the seller is of type t ≤ q and has rejected all offers submitted by buyers i = 1 n − 1, and denote by F the pointwise limit of the nonincreasing sequence (Fn ). Suppose that F(q) = 0 for some q < 1. In particular, the probability that the seller accepts buyer n’s offer, conditional on having rejected the previous ones, converges to 0 as n increases. Hence, the successive buyers’ payoffs also converge to 0. Choose q such that F(q) > 0 and q ν v(t) − c(q) − (1) dF(t) > 0 2 q 1 where ν := minx∈[q1 1] {v(x)−c(x)}. Note that (F(q)−Fn (q))/Fn (q) is the probability that type q accepts an offer from a buyer after n, conditional on having rejected all previous offers. Since F(q) > 0, this probability converges to 0 and the offer pn (q) with indifferent type q tends to no more than c(q). Thus, pn (q) ≤ c(q) + ν2 for n large and, using (1), buyer n’s payoff is bounded away from 0—a contradiction. Q.E.D. Observe that Proposition 2 holds independently of δ and establishes that offers arbitrarily close to c(1) are eventually submitted. According to Proposition 2, agreement is always reached in finite time. This raises the question of delay. That is, let τ(1) denote the random period in which a winning offer is first submitted. The next proposition places bounds on E[δτ(1) ], the expected delay until agreement. In particular, it implies that τ(1) is finite almost surely (a.s.): a winning offer is submitted in finite time, with probability 1. ¯ There exist constants 0 < c1 < c2 < 1 such PROPOSITION 3: Assume that δ > δ. that, in all equilibria, c1 ≤ E δτ(1) ≤ c2
PUBLIC VS. PRIVATE OFFERS
43
The proof of this and all remaining results can be found in the Appendix. Delay (c2 < 1) should not come as a surprise. Since the seller can wait until the first winning offer is submitted and serious offers until then must yield a nonnegative payoff to the buyers submitting them, delay must make waiting for the winning offer a costly alternative to the seller’s lower types. Slightly less obvious is the second conclusion; namely, that delay does not dissipate all gains from trade (c1 > 0). In the first example of this section, the first buyer enjoys a positive payoff, but all other buyers get a zero payoff. More complicated examples of equilibria can be constructed in which more than one buyer gets a positive payoff. However, in all equilibria, all buyers’ payoffs are small. PROPOSITION 4: There exists a constant M1 > 0 such that, for every δ > δ¯ and every equilibrium, the payoff of any buyer n is at most (1 − δ)2 M1 According to the next proposition, buyers with a positive equilibrium payoff are infrequent in the sense that two such buyers must be sufficiently far apart in the sequence of buyers. PROPOSITION 5: There exists a constant M2 > 0 such that, for every δ ≥ δ¯ and every equilibrium, the following statement holds: If buyer n1 and buyer n2 both get a positive payoff, then |n2 − n1 | ≥
M2 1−δ 4.2. Equilibrium Strategies
As mentioned, we do not provide an explicit characterization of an equilibrium for general parameters. Nevertheless, all equilibria share common features. Given some equilibrium, let Fn (q) denote the (unconditional) probability that the seller’s type t is less than or equal to q and that all offers submitted by buyers i = 1 n − 1 are rejected. Set qn := inf{q : Fn (q) > 0}. Buyer n’s strategy is a probability distribution over offers in [c(q1 ) c(1)]. We denote by Pn its support and by Tn the corresponding (closed) set of indifferent types. That is, if buyer n’s strategy has finite support, q ∈ Tn if it is an equilibrium action for buyer n to submit some pn with indifferent type q. The following proposition complements Proposition 5, as together they imply that the number of buyers with a positive payoff is bounded above, uniformly in the discount factor and the equilibrium. ¯ Given some equilibrium, let N0 := inf{n ∈ PROPOSITION 6: Assume that δ > δ. N ∪ {∞} : 1 ∈ Tn }. There exists a constant M3 > 0 such that, in all equilibria, N0 ≤
44
J. HÖRNER AND N. VIEILLE
M3 /(1 − δ). Further, given some equilibrium, Tn ⊂ {qN 1} for all n ≥ N0 . For all 0 n > N0 , buyer n’s equilibrium payoff is zero. Observe that, by Propositions 4, 5, and 6, the (undiscounted) sum of all buyers’ payoffs is at most (1 − δ)2 M3 M1 /M2 , and therefore vanishes as the seller becomes more patient. Thus, from period N0 on, buyers only make winning or losing offers, and all but the first of these have a payoff of zero. In fact, it follows readily from the proof that the equilibrium payoff of buyer N0 is zero as well, as long as q1 ≤ q1 . If q1 > q1 , it follows from Proposition 6 that in the unique equilibrium outcome, the first buyer offers c(1), which the seller accepts. Indeed, provided that he is called upon to submit an offer, any buyer is guaranteed a positive payoff, because he can always offer c(1). In all other cases, there exist multiple equilibria. In particular, there always exists an equilibrium in which agreement is reached in bounded time, as well as an equilibrium in which agreement is reached in unbounded, yet finite, time. Indeed, given any equilibrium profile σ, the probabilities assigned by buyers n > N0 to the winning offer can be modified so that the modified profile is still an equilibrium with the desired property. While the equilibria obtained in this way are payoff-equivalent, the first example in this section shows that this is not true across all equilibria. For q1 < q1 , the next proposition formalizes the idea that all equilibria are in mixed strategies. PROPOSITION 7: Assume that δ > δ¯ and q1 < q1 . No buyer n ≤ N0 uses a pure strategy except possibly buyer 1. All buyers n ≤ N0 submit a serious offer with positive probability. Indeed, buyer 1 need not use a mixed strategy, as the first example given in this section illustrates. Without further assumptions, it is difficult to establish additional structural properties on equilibrium strategies. However, it can be shown that if v is concave and c is convex over (q1 1), with either v or c being strictly so, then each buyer’s strategy is a distribution with finite support, so that each buyer randomizes over finitely many offers only.12 Propositions 6 and 7 allow us to circumscribe the equilibrium strategies as follows. During a first phase of the game (until period N0 − 1), buyers’ strategies assign positive probability to more than one offer (with the possible exception of the first buyer’s strategy); in particular, they all assign positive probability to serious, but not winning, offers. Some of these buyers may enjoy a small positive payoff, while all others have zero payoff; in fact, the number of those not submitting a losing offer with positive probability is finite as well. In 12
See Hörner and Vieille (2007).
PUBLIC VS. PRIVATE OFFERS
45
a second phase (from period N0 on), all buyers’ payoffs are zero, and they randomize between the winning offer and a losing offer, with relative probabilities that are to a large extent free variables. Thus, as long as offers are rejected, the unit’s expected value increases until N0 and is constant thereafter. It is tempting to investigate the existence of equilibria in which all but the first buyers’ strategies assign positive probability to two offers, as in the second example given at the beginning of this section. Such equilibria need not exist, suggesting that either some buyers’ strategies assign positive probability to more than two offers or that the lower offer in the support of some buyers’ strategies is serious as well. It is possible to construct equilibria of the second kind, but doing so generally is intractable, even numerically. 4.3. Comparison With the Public Case The main difference between the two information structures has been emphasized throughout: trade occurs with probability 1 with private offers, but not generally so with public offers. Other comparisons are less clear-cut. In particular, we have been unable to rank the two structures according to efficiency. However, in all examples that we have been able to solve, including those presented in Section 4, welfare is higher under private offers than under public offers. Further, it can be shown that this result holds whenever there are two types only.13 It is straightforward to show that no equilibrium is second-best efficient. In terms of interim efficiency, the comparison is ambiguous. Considering the second example in Section 4, it is easy to check that very low types prefer the outcome under observable offers, while very high types prefer the outcome under hidden offers. From the buyers’ point of view, since it is possible that q∗ = qK , Samuelson’s (1984) Proposition 1 implies that the outcome with public offers is the preferred one among the outcomes of all bargaining procedures from the first buyer’s perspective. In particular, since eventual agreement in the hidden case implies that serious (but not winning) offers involve prices higher than the cost of the corresponding indifferent type, the first buyer prefers the outcome of the game with public offers to the outcome in the game with private outcome whenever q∗ is sufficiently close to qK . The same argument applies to the aggregate buyers’ payoff. Buyers n ≥ 2 prefer the outcome with hidden offers, although any difference disappears as the discount factor tends to 1 (see Proposition 4). 5. RELATED LITERATURE AND CONCLUDING COMMENTS Our contribution is related to the literatures in three ways. First, several authors have already considered dynamic versions of Akerlof’s model. Second, 13
The proof can be found in the supplemental material (Hörner and Vieille (2009)).
46
J. HÖRNER AND N. VIEILLE
several papers in the bargaining literature considered interdependent values. Third, several papers in the literature on learning addressed the conditions under which learning occurs. In particular, two papers have investigated the difference between public and private offers in the framework of Spence’s signaling model. Janssen and Roy (2002) considered a dynamic, competitive durable good setting with a fixed set of sellers. They proved that trade for all qualities of the good occurs in finite time. The critical difference lies in the market mechanism. In their model, the price in every period must clear the market. That is, by definition, the market price must be at least as large as the good’s expected value to the buyer conditional on trade, with equality if trade occurs with positive probability (this is condition (ii) of their equilibrium definition). This expected value is derived from the equilibrium strategies when such trade occurs with positive probability and it is assumed to be at least as large as the lowest unsold value even when no trade occurs in a given period (this is condition (iv) of their definition). This implies that the price exceeds the valuation to the lowest quality seller, so that trade must occur eventually. Also related works are Taylor (1999), Hendel and Lizzeri (1999), House and Leahy (2004), and Hendel, Lizzeri, and Siniscalchi (2005). In the bargaining literature, Evans (1989), Vincent (1989), and Deneckere and Liang (2006) considered bargaining with interdependent values. Evans (1989) considered a model in which the seller’s unit can have one of two values, and assumed that there is no gain from trade if the value is low. He showed that the bargaining may result in an impasse when the buyer is too impatient relative to the seller. In his Appendix, Vincent (1989) provided another example of equilibrium in which bargaining breaks down. As in Evans, the unit can have one of two values. It follows from Deneckere and Liang (2006) that his example is generically unique. Deneckere and Liang (2006) generalized these findings by considering an environment in which the unit’s quality takes values in an interval. They characterized the (stationary) equilibrium of the game between a buyer and a seller with equal discount factors, in which, as in ours, the uninformed buyer makes all the offers. When the static incentive constraints preclude first-best efficiency, the limiting bargaining outcome involves agreement but delay, and fails to be second-best efficient. Other related contributions include Riley and Zeckhauser (1983), Cramton (1984), and Gul and Sonnenschein (1988). There is a large literature on learning. Some papers have examined the conditions under which full learning occurs as time passes and under rather general conditions (see Aghion, Bolton, Harris, and Jullien (1991)). These models, however, are typically cast as decision problems in which Nature’s response is exogenously specified. Here instead, the information that is being revealed is a function of the seller’s best reply, which is endogenous. Nöldeke and van Damme (1990) and Swinkels (1999) developed an analogous distinction in Spence’s signaling model. Both considered a discrete-time version of the
PUBLIC VS. PRIVATE OFFERS
47
model, in which education is acquired continuously and a sequence of shortrun firms submit offers that the worker can either accept or reject. Nöldeke and van Damme considered the case of public offers, while Swinkels focused mainly on private offers. Nöldeke and van Damme showed that there is a unique equilibrium outcome that satisfies the never-a-weak-best-response requirement and that the equilibrium outcome converges to the Riley outcome as the time interval between consecutive periods shrinks. For private offers, Swinkels proved that the sequential equilibrium outcome is unique and that it involves pooling in the limit. The logic driving these results is similar to ours, at least for public offers. In both papers, firms (buyers) are deterred from submitting mutually beneficial offers because rejecting such an offer sends a strong signal to future firms and elicits wage offers so attractive that only very low types would accept the current offer. There are many conceivable variations to the simple setup considered here and the reader may wonder how far the results extend to more complex environments. For instance, one may wish to model buyers as being unaware of the number of prior offers that have been submitted. One may want to allow for multiple buyers in each period. One could also allow the seller to choose how much information to disclose, to make offers, or to send messages that are more or less committing, such as list prices in the case of real estate. These extensions are further discussed in the working paper (see Hörner and Vieille (2007)). One issue which is not addressed there is the impact of potential liquidity shocks on the seller.14 For instance, assume that the seller is subject to random temporary liquidity shocks, independent across periods, upon which the seller would accept any offer that meets his cost. If these costs are unobserved by the buyers, then beliefs of successive buyers are no longer uniform.15 Finally, it might also have occurred to the reader that the impasse result for the public case cannot possibly hold if the horizon is finite, as the situation of the last buyer reduces then to the static case, for which the unique optimal offer is serious. However, it is not hard to see that, for a fixed horizon T , there exists a discount factor δ¯ T < 1 such that if δ > δ¯ T , all offers but the last one are necessarily losing (provided q1 < q1 ): because the last buyer’s offer is serious, the objective function of the penultimate buyer inherits the convexity property stressed in Section 3.1 that compels him to submit a losing offer, and the result follows by backward induction.16 A crucial assumption throughout the analysis has been that buyers do not receive private signals. Allowing for private information is both economically 14
We thank two referees for raising this issue. As a result, the analysis of this variant seems to be beyond reach. However, in the case where cost and valuations may take only two different values, results are qualitatively similar, with or without liquidity shocks. The proof is available upon request. 16 One could also fix the discount factor δ > δ¯ and let T tend to infinity. The complete analysis of this case is quite intricate and we have not been able to resolve it satisfactorily. 15
48
J. HÖRNER AND N. VIEILLE
relevant and likely to drastically affect the results in the public case. In this case, later buyers do not only learn about the seller’s type through the seller’s earlier actions, but also through the offers that were made by previous buyers, as in the literature on cascades (see Bikhchandani, Hirshleifer, and Welch (1992)). A unit might remain on the market for a longer time period either because the seller is particularly reluctant to give up a good that he knows to be of high quality or because all previous buyers’ signals have been unfavorable. The possibility of the latter event is likely to depress later offers, but we conjecture that this might paradoxically help trade by prompting the seller to accept earlier offers.
APPENDIX A: PROOFS FOR PUBLIC OFFERS A.1. Proof of Proposition 1 We first prove the uniqueness of the PBE outcome: On the equilibrium path of any PBE, all buyers offer c(qK ), which the seller accepts if and only if his type does not exceed qK ; otherwise, the seller rejects all offers. We next discuss the existence issue. We prove the uniqueness of the PBE outcome by induction over K. The proof for K = 0 is, in most respects, identical to the proof of the induction step and we, therefore, provide only the latter. We let a (perfect, Bayesian) equilibrium be given. Recall that qn stands for the lowest type which rejected all offers up to stage n (according to the seller’s strategy). Assume that for some k < K, the following statements hold for each l < k: • For every period n ≥ 1 and after any history hn−1 such that qn ∈ (ql+1 ql ), all subsequent buyers, including buyer n, offer c(ql ). • For every period n ≥ 1 and after any history hn−1 such that qn ≤ ql+1 , none of the subsequent equilibrium offers ever exceeds c(ql ). We will prove that the same conclusion holds for k + 1. The proof is broken into the following four steps: Step 1. Whenever qn < qk , no equilibrium offer of buyer n is accepted by some type q > qk . Step 2. Whenever qn < qk , none of the following equilibrium offers ever exceeds c(qk ). Step 3. If qn < qk is close enough to qk , the unique equilibrium offer of buyer n is c(qk ), which the seller accepts if and only if his type does not exceed qk . Step 4. More generally, if qn ∈ (qk+1 qk ), the unique equilibrium offer of buyer n is c(qk ), which the seller accepts if and only if his type does not exceed qk .
PUBLIC VS. PRIVATE OFFERS
49
Step 1: We let a stage n and a history hn−1 be given, such that q := qn < qk . Fix l < k − 1 and let q ∈ (ql+1 ql ]. We claim that an offer p by buyer n is accepted by type q if and only if p − c(q) ≥ δ(c(ql ) − c(q))
(2)
Indeed, first let p be an offer that would be accepted by type q and denote by q ≥ q the indifferent type of p. If q ≥ ql , then p ≥ c(ql ); hence (2) holds. If, on the other hand, q ∈ [q ql ), the offer following p would be c(ql ) by the induction hypothesis. Since type q is willing to accept p, (2) also holds in that case. Conversely, assume that there is some offer p that satisfies (2) and that would be rejected by type q. Denote by q < q the indifferent type of p. If q > ql+1 , then the offer following p would be c(ql ) and would be accepted by type q by the induction hypothesis. If q ≤ ql+1 , by the induction hypothesis again, none of the equilibrium offers following p would ever exceed c(ql ). In both cases, type q would rather accept p in the first place. For q ∈ (ql+1 ql ], define p(q) by p(q) − c(q) = δ(c(ql ) − c(q)) The indifferent type of the offer p(q) is q. When offering p(q), buyer n’s payoff is therefore q 1 {v(t) − δc(ql ) − (1 − δ)c(q)} dt 1−q q As a function of q, the integral is twice differentiable over the interval (ql+1 ql ], with first and second derivatives given by17 v(q) − δc(ql ) − (1 − δ)c(q) − (1 − δ)c (q)(q − q) and v (q) − (1 − δ) c (q)(q − q) + 2c (q) Since (2Mc + Mc )(1 − δ) < mv , buyer n’s payoff is strictly convex over (ql+1 ql ]. Since buyer n’s payoff is negative for q = ql , the claim follows. Step 2: We argue by contradiction. Assume that, for some stage n and some history hn−1 with qn < qk , there is a positive probability that buyer n makes an equilibrium offer pn which exceeds c(qk ). By Step 1, the indifferent type of any such equilibrium offer does not exceed qk . Let p¯ > c(qk ) be the supremum of 17
Up to the constant 1 − q.
50
J. HÖRNER AND N. VIEILLE
all such offers, where the supremum is taken over all such stages n and histories hn−1 . Define p by the equation p − c(qk ) = δ(p¯ − c(qk )). We claim that, for any stage n and following any history hn−1 such that qn < qk , any offer p ≥ p would be accepted by all types up to qk . Hence, no equilibrium offer would ever ex¯ this will imply the desired contradiction. ceed p. Since p < p, Assume to the contrary that, for some stage n and following some history hn−1 , the indifferent type q of some offer p ≥ p satisfies q < qk . Following the offer p , one has qn+1 = q < qk . Upon declining the offer p , type q would accept the first (if ever) offer whose indifferent type exceeds q. By definition, ¯ By definition of p, type q would strictly this offer cannot possibly exceed p. prefer to accept p in the first place. Step 3: Let a stage n and a history hn−1 ∈ H n−1 be given, with q := qn < qk . Given q ∈ (q qk ], denote by p(q) the infimum of the offers that type q would accept. Obviously, p(q) ≥ c(q). On the other hand, and by Step 2, any offer that is rejected by type q is followed by offers which do not exceed c(qk ); hence p(q) ≤ c(qk ). q q 1 1 {v(t) − p(q)} dt, is at most 1−q {v(t) − Hence, buyer n’s payoff, 1−q q q c(q)} dt, and the difference between the two integrals converges to 0 as q increases to qk . The latter integral, as a function of q, is differentiable, with derivative v(q) − c(q) − c (q)(q − q), which is positive whenever q − q < ν/Mc . q 1 {v(t) − c(q)} dt, is inThus, for q close enough to qk , the upper bound, 1−q q creasing over [q qk ). Hence, for such q, and since an optimal offer is assumed to exist, buyer n’s equilibrium offer is c(qk ), which the seller accepts if and only if his type does not exceed qk . Step 4: Again, we argue by contradiction. We assume that for some stage n and some history hn−1 with qn > qk+1 , buyer n’s strategy assigns a positive probability to serious offers with indifferent type below qk . Among all such n and hn−1 , let q˜ ∈ (qk+1 qk ] be the supremum of qn . By Step 3, q˜ < qk . Consider now any stage n and any history hn−1 such that q := qn satisfies ˜ Much as in Step 1, we claim that an offer p is accepted by type q ∈ (qk+1 q]. ˜ qk ) if and only if q ∈ (q (3)
p − c(q) ≥ δ(c(qk ) − c(q))
Indeed, let p be an offer that would be accepted by type q, and denote by q ≥ q the indifferent type of p. If q ≥ qk , then p ≥ c(qk ); hence (3) holds. On the other hand, if q ∈ [q qk ), then the offer following p would be c(qk ) by the induction hypothesis. Since type q is willing to accept p, (3) also holds in that case.
PUBLIC VS. PRIVATE OFFERS
51
Conversely, assume that there is some offer p that satisfies (3) and that would be rejected by type q. By Step 2, none of the subsequent offers would ever exceed c(qk ). Hence type q would rather accept p in the first place. ˜ qk ), define p(q) as For q ∈ (q p(q) − c(q) = δ(c(qk ) − c(q)) As a function of q, and when offering p(q), buyer n’s payoff is q 1 {v(t) − δc(ql ) − (1 − δ)c(q)} dt 1−q q As in Step 1, the integral is a strictly convex function of q. Therefore, the indifferent type of any equilibrium offer is either equal to qk or lies in the ˜ In the former case, buyer n’s offer is c(qk ) and his payoff is interval [q q]. positive since q˜ > qk+1 . In the latter case, buyer n’s payoff is at most (q˜ − ˜ − c(q)), which is arbitrarily close to 0, provided q is close enough to q. ˜ q)(v(q) ˜ the unique equilibrium offer of buyer As a consequence, for q < q˜ close to q, ˜ n is c(qk ), with indifferent type qk . This contradicts the definition of q. This concludes the proof that all PBE have the same outcome: all buyers offer c(qK ), which the seller accepts if and only if his type does not exceed qK . It is instructive to derive more precise insights into equilibrium strategies. Let a stage n and a history hn−1 be given, with qn ∈ (ql+1 ql ) for some l ≤ K. We already know that the unique equilibrium offer following hn−1 is c(ql ), and all subsequent equilibrium offers are equal to c(ql ). What happens off equilibrium? If buyer n makes an offer p < c(ql ) with independent type q, one has q ∈ (ql+1 ql ); thus the offer p is followed by offers equal to c(ql ). Hence the indifferent type q is the solution of the equation p − c(q) = δ(c(ql ) − c(q)).18 If instead buyer n makes an offer p > c(ql ), say c(ql ) < p ≤ c(ql−1 ), the situation is a bit more subtle. We know that the indifferent type q associated with the offer p is at least equal to ql (for otherwise, all subsequent offers would be equal to c(ql ), and type q would strictly prefer to accept the current offer). If q > ql , then the offer p is followed by offers equal to c(ql−1 ), which type q accepts; hence q is given by p − c(q) = δ(c(ql−1 ) − c(q)) This equation has a solution in (ql ql−1 ] if and only if (1 − δ)c(ql ) + δc(ql−1 ) < p ≤ c(ql−1 ) 18
If it exists. Otherwise, if the offer p is so low that a solution does not exist, the offer is losing.
52
J. HÖRNER AND N. VIEILLE
If instead, p lies in the interval (c(ql ) (1 − δ)c(ql ) + δc(ql−1 )], the indifferent type of the offer p is necessarily equal to ql . But this implies that the offer p is eventually followed by serious offers. Which serious offers? Since qn+1 = ql , the only possible serious equilibrium offer is c(ql−1 ), provided it is accepted by all types up to ql−1 . In other words, indifference of type ql relies on the fact that with positive probability, some subsequent buyer will make an offer equal to c(ql−1 ), followed by offers all equal to c(ql−1 ). These probabilities must be such that the benefit p − c(ql ) derived from the current offer is equal to the discounted benefit derived from declining the offer p. The latter is given by c(ql−1 ) − c(ql ) times the expectation of δτ−n , where τ is the first buyer who makes an offer equal to c(ql−1 ). As a consequence, there is some freedom in specifying the equilibrium behavior of buyers after the history hn−1 and an offer of p. (We may, for instance, assume that only buyer n does randomize, as done in the main body of the paper.) Similar arguments apply to pin down the equilibrium behavior of the buyers after any history hn−1 such that qn = qk for some k ≤ K. In this case, consider the most recent stage m < n with qm < qk , and assume without loss of generality that m = n − 1. If along hn−1 , buyer n − 1 made the offer c(qk ), then buyer n should make the offer c(qk ), as should all subsequent buyers as well. If instead buyer n − 1 made a higher offer, then buyer n (and subsequent buyers) should randomize between c(qk ) and c(qk−1 ) in a way such that the prospect of this randomization made type qk indifferent between accepting and declining the offer of buyer n − 1. The seller’s strategy is then uniquely determined (apart from the acceptance decision of the indifferent type): (i) an offer in the interval [c(qk ) (1 − δ)c(qk ) + δc(qk−1 )] is accepted by all types q ≤ qk and is rejected by all types q > qk , (ii) an offer p in the interval ((1 − δ)c(qk ) + δc(qk−1 ) c(qk−1 )) is accepted by all types q such that p − c(q) ≤ δ(c(qk−1 ) − c(q)) and is rejected otherwise.19 We proved that the equilibrium outcome is at most unique and that all PBE coincide, apart from some off-equilibrium randomizations. To conclude the proof, we briefly sketch the definition of an equilibrium profile. Define first the strategy σS of the seller according to the previous paragraph. Given σS , we define strategies σB of the buyers according to the paragraph before. It is straightforward, but somewhat tedious, to check that the profile σ = (σS σB ) is a PBE of the game. Q.E.D.
19
With the convention qK+1 = q1 and q−1 = q0 = 1.
PUBLIC VS. PRIVATE OFFERS
53
APPENDIX B: PROOFS FOR PRIVATE OFFERS B.1. Preliminaries The remainder of the Appendix is organized as follows. As a preliminary, we set up some additional notation and state a few important facts. We then prove Propositions 3–7, though in a different order. We start with Propositions 4 and 5. We then need to prove Proposition 6—with the exception of the upper bound on N0 . Indeed, it is a logical preliminary to Proposition 7, which we prove next, and its proof is instrumental in the proof of Proposition 3. A strategy of buyer n is a probability distribution σBn over offers. Any profile σB of such distributions induces a probability distribution over sequences of offers, which we denote PσB . Expectation with respect to PσB is denoted by EσB . We denote by p˜ n the random offer of buyer n. If a seller with type q declines the first offer and plans to accept an offer at a (random) time τ > 1, his expected payoff is EσB [δτ−1 (p˜ τ − c(q))]. His optimal continuation payoff is thus supτ>1 EσB [δτ−1 (p˜ τ − c(q))], where the supremum is taken over all stopping times τ > 1, and the offer p1 (q) with indifferent type q is given by p1 (q) − c(q) = sup EσB δτ−1 (p˜ τ − c(q)) τ>1
We assume for concreteness that a seller accepts an offer whenever he is indifferent. Thus, a seller with type q accepts the offer from buyer τ(q) := inf{n ≥ 1 : p˜ k ≥ pk (q)}. Similarly, the offer pn (q) with indifferent type q is given in stage n by (4) pn (q) − c(q) = sup EσB δτ−n (p˜ τ − c(q)) τ>n
The function pn (·) is continuous and increasing. The stopping time τn (q) := inf{k > n : p˜ k ≥ pk (q)} is an optimal stopping time in (4); hence pn (·) also satisfies pn (q) − c(q) = EσB δτn (q)−n p˜ τn (q) − c(q) It follows that (5)
pn (q) − c(q) ≥ δ(pn+1 (q) − c(q))
with equality if and only if buyer n + 1 makes no offer above pn+1 (q): competition between successive buyers prevents pn from being much below pn+1 .20 20 On the other hand, it may happen that pn+1 is much below pn , as is, for example, the case if buyer n + 1 makes high offers with high probability, followed by losing offers.
54
J. HÖRNER AND N. VIEILLE
Using a version of the envelope theorem, the function pn has a left derivative everywhere, given by (6) D− pn (q) = c (q) 1 − EσB δτn (q)−n It also admits a right derivative everywhere, given by + D+ pn (q) = c (q) 1 − EσB δτn (q)−n where τn+ (q) := inf{k > n : p˜ k > pk (q)}. Note that EσB [1 − δτn (q)−n ] is nondecreasing in q and, therefore, pn is convex if the cost function is convex. The function pn plays a crucial role in the proofs. It may be interpreted as an (inverse) offer function faced by buyer n, and (6) provides a direct link between the slope of this offer function at q and the discounted time at which a seller with type q expects to receive an acceptable offer—the earlier the discounted time, the lower the slope of pn . We now comment on the beliefs of the various buyers. Since offers are private, the belief of buyer n need not be a uniform distribution. Recall that Fn (q) is the (unconditional) probability that the seller is of type t ≤ q and rejects offers from buyers 1 through n − 1. Letting 1
Pσ (p˜ k < pk (q)) fn (q) = 1 − q1 k=1 B n−1
denote the (normalized) probability that a seller with type q rejects the first n − 1 offers, one has q Fn (q) = fn (t) dt q
1
Observe that fn is left-continuous and nondecreasing, so that Fn is nondecreasing, convex, and admits a left derivative D− Fn (q) = fn (q). We last introduce qn := max{q ∈ [q1 1] : Fn (q) = 0}, the infimum of the types that reject the first n − 1 offers with probability 1. With these notations at hand, the expected payoff πn (q) of buyer n, when submitting the offer pn (q), is given by q (7) (v(t) − pn (q))fn (t) dt πn (q) := q
n
We denote by vn (q) = 1/(Fn (q))
q q
n
v(t)fn (t) dt the average valuation of types
below q, as seen by buyer n. Then (7) is rewritten as πn (q) = Fn (q)(vn (q) − pn (q))
PUBLIC VS. PRIVATE OFFERS
55
which reads as the probability that the nth offer is accepted, Fn (q), times the conditional payoff, given that trade takes place. The payoff function πn has a left derivative everywhere, equal to (8)
D− πn (q) = (v(q) − pn (q))fn (q) − D− pn (q)Fn (q)
It also has a right derivative, given by (9)
D+ πn (q) = (v(q) − pn (q))fn+ (q) − D+ pn (q)Fn (q)
where fn+ is the right derivative of Fn . We stress that the quantities introduced so far, pn , Fn , fn , qn , πn , and vn all depend on the profile σ under consideration, although the notation does not indicate this. Throughout the Appendix, we let an equilibrium σ ∗ be given, and no confusion should arise. For conciseness, we will refer to the indifferent type associated with an offer p as a type offer. For instance, the statement buyer n submits a type offer q is logically equivalent to the statement buyer n submits an offer with indifferent type q. Since πn is continuous, the equilibrium payoff πn∗ of buyer n is equal to max[qn 1] πn , and one has πn (q) = πn∗ for every q ∈ Tn , where Tn is the support of the random type offer of buyer n.21 Finally, we state a preliminary observation that is used repeatedly below. We consider a buyer, say n + 1, who submits only type offers bounded away from qn+1 —the lowest remaining type. We prove that the previous buyer then makes no serious type offer below the lowest serious type offer of buyer n + 1. LEMMA 2: Assume qn+2 > qn+1 for some n ∈ N. Then buyer n submits no type offer in (qn qn+2 ).22 In particular, qn+1 = qn , buyer n submits a losing offer with positive probability, and πn∗ = 0. ∗ > 0, since πn+1 (qn+1 ) = The inequality qn+2 > qn+1 is satisfied whenever πn+1 0 and πn+1 (q) is therefore arbitrarily close to zero in a neighborhood of qn+1 . Lemma 2 thus implies that there are no two consecutive buyers with a positive equilibrium payoff.
PROOF OF LEMMA 2: By assumption, a seller with type q ∈ (qn qn+2 ) plans to accept buyer n + 1’s offer with probability 1, were he to decline buyer n’s offer. Thus, the seller’s continuation payoff is δEσB∗ [p˜ n+1 − c(q)] and, therefore, 21 22
That is, Tn is the smallest closed set of type offers that is assigned probability 1 by σBn . That is, σB∗n assigns probability 0 to type offers in (qn qn+2 ).
56
J. HÖRNER AND N. VIEILLE
pn (q) = (1 − δ)c(q) + δEσB∗ [p˜ n+1 ]. Since δ > δ, this implies that v(q) − pn (q) is increasing. Set z := inf{q ∈ [qn 1] : v(q) ≥ pn (q)} (with inf ∅ = qn ). Note that D− πn (q) = (v(q) − pn (q))fn (q) − c (q)Fn (q) < 0 on (qn z]. On the other hand, on the interval (z qn+2 ], D− πn is lower semicontinuous since fn is nondecreasing and left-continuous. We now prove that D− πn is increasing. Since fn is nondecreasing, one has (v(q) − pn (q))fn (q) − (v(x) − pn (x))fn (x) xq q−x
lim inf
≥ (v (q) − D− pn (q))fn (q) thus, using (8), D− πn (q) − D− πn (x) ≥ (v (q) − (1 − δ)cn (q))fn (q) xq q−x
lim inf
− (1 − δ)(c (q)Fn (q) + c (q)fn (q)) ≥ v (q) − (1 − δ)(2c (q) + c (q))fn (q) > 0 where the second inequality holds since Fn (q) ≤ fn (q) and the third inequality ¯ holds since δ ≥ δ. Since D− πn is upper semicontinuous, this implies that D− πn is strictly increasing over (z qn+2 ], hence πn is strictly convex over [z qn+2 ]. To summarize, πn is continuous, decreasing over [qn z], and strictly convex over [z qn+2 ]. Therefore, it has no maximum over (qn qn+2 ). This proves the first claim. If buyer n does not submit a losing offer with positive probability, then his lowest type offer is at least qn+2 , which implies qn+1 ≥ qn+2 —a contradiction. In particular, πn∗ = πn (qn ) = 0. This concludes the proof of the lemma. Q.E.D. B.2. Proof of Proposition 4 We here prove that equilibrium payoffs are very small. Proposition 8 below implies Proposition 4. PROPOSITION 8: The equilibrium payoff of each buyer n is at most πn∗ ≤
2 (1 − δ)2 (v(qn+1 ) − c(qn+1 ))2 δmv (1 − q1 )
PUBLIC VS. PRIVATE OFFERS
57
PROOF: Consider a buyer n with a positive equilibrium payoff, πn∗ > 0, so that qn+1 > qn and πn∗ = Fn (qn+1 )(vn (qn+1 ) − pn (qn+1 )) We bound below each of the two terms. ∗ = 0, which implies in turn Note that πn∗ > 0 implies πn+1 (10)
pn+1 (qn+1 ) ≥ v(qn+1 )
(for otherwise buyer n + 1 would get a positive payoff when making a type offer just above qn+1 ). Note also that vn (qn+1 ) < v(qn+1 ) and that πn∗ > 0 implies pn (qn+1 ) < vn (qn+1 ): (11)
pn (qn+1 ) < vn (qn+1 ) < v(qn+1 )
Recall finally (5): (12)
pn (qn+1 ) − c(qn+1 ) ≥ δ(pn+1 (qn+1 ) − c(qn+1 ))
We rely on (10), (11), and (12) to prove first that the expected payoff conditional on trade, vn (qn+1 ) − pn (qn+1 ), is at most of the order 1 − δ. By (11), then (10), then (12), one has 1 vn (qn+1 ) < v(qn+1 ) ≤ pn+1 (qn+1 ) ≤ {pn (qn+1 ) − (1 − δ)c(qn+1 )}; δ hence (13)
vn (qn+1 ) − pn (qn+1 ) ≤
1−δ (v(qn+1 ) − c(qn+1 )) δ
Next, we argue that Fn (qn+1 ) is at most of the order 1 − δ. Substituting (10) into (12) yields δv(qn+1 ) ≤ pn (qn+1 ) − (1 − δ)c(qn+1 ) which then implies, using the first half of (11), (14)
vn (qn+1 ) ≥ δv(qn+1 ) + (1 − δ)c(qn+1 )
The intuition now goes as follows. If the probability Fn (qn+1 ) is nonnegligible, then the computation of vn (qn+1 ) must involve a significant fraction of low types, and vn (qn+1 ) is therefore bounded away from v(qn+1 ), which stands
58
J. HÖRNER AND N. VIEILLE
in contradiction to (14). To verify formally this claim, by compute the highest value for Fn (qn+1 ) which is consistent with (14) and compute the value Ω of the ¯ find auxiliary infinite-dimensional linear problem (P ): given q1 < q ≤ 1 and v, the supremum of q P: f (t) dt q
over the set F of nondecreasing, left-continuous q q functions with values in [0 1/(1 − q1 )] and such that q v(t)f (t) dt ≥ v q f (t) dt. The analysis of (P ) 1 1 is standard. When endowed with the Levy distance, F is compact and the objective of (P ) is continuous; hence there is an optimal solution, f ∗ . Since v(·) − v is strictly increasing, the solution f ∗ must be of the form f ∗ (t) = 1 ∗ × 1/(1 − q1 ) for some q∗ . The location of q∗ is dictated by the constraint t>q q v(t) dt = v(q − q∗ ). Plugging the inequality v(t) ≤ v(q) − mv (q − t) for all q∗ t ∈ [q∗ q] into the constraint yields q − q∗ ≤
2 (v(q) − v) mv
We apply this upper bound with q = qn+1 and v¯ = δv(qn+1 ) + (1 − δ)c(qn+1 ), and get (15)
Ω≤
2(1 − δ) (v(qn+1 ) − c(qn+1 )) mv (1 − q1 )
Collecting (13) and (15) then yields πn∗ ≤
2 (1 − δ)2 (v(qn+1 ) − c(qn+1 ))2 δmv (1 − q1 ) Q.E.D.
as desired. B.3. Proof of Proposition 5
The intuition for the proof is as follows. A seller with type qn +1 < qn +1 ex1 2 pects to receive an acceptable offer at stage n2 at the latest. Thus, the difference n2 − n1 is directly linked to the discounted time at which a seller with type qn +1 expects to trade. In Lemma 3, we first provide a lower bound on D+ pn (q). 1 We then rely on the relation between D+ pn (qn +1 ) and the discounted time at 1 which type qn +1 trades. 1
59
PUBLIC VS. PRIVATE OFFERS
LEMMA 3: For any buyer n and any serious type offer q > qn in Tn , one has D+ pn (q) ≥ mv /2(1 − q1 ). The proof of Lemma 3 uses the following technical inequality. LEMMA 4: Let h : [q1 1] → R+ be nondecreasing. Then, for any [a b] ⊆ [q1 1], one has (16)
h(b)
b
v(t)h(t) dt +
b
a
h(t) dt
mv 2
b
h(t) dt ≤ v(b)h(b) a
a
(with the convention
0 0
= 0).
PROOF: By induction over the cardinality of the range of v, it is easily checked that (16) holds whenever v is a step function. The result then follows using a limit argument. Q.E.D. PROOF OF LEMMA 3: Since πn is maximal at q, one has D+ πn (q) ≤ 0, that is, (17)
(v(q) − pn (q))fn+ (q) − D+ pn (q)Fn (q) ≤ 0
On the other hand, since q ∈ Tn , one has πn (q) = πn∗ which, since q is a serious type offer, implies v(q) ≥ vn (q) ≥ pn (q). Plugging these inequalities into (17), one obtains fn+ (q)(v(q) − vn (q)) − D+ pn (q)Fn (q) ≤ 0 or, equivalently, q v(t)fn+ (t) dt q q
n fn+ (q)v(q) ≤ fn+ (q)
q q
fn+ (t) dt
+ D+ pn (q)
fn+ (t) dt
q
n
n
The result then follows by applying Lemma 4.
Q.E.D.
PROOF OF PROPOSITION 5: Recall from (6) that D+ pn (q) = c (q)(1 − + EσB∗ [δτn (q)−n ]), where τn+ (q) = inf{k > n : p˜ k > pk (q)}. Consider buyer n := n1 + and q := qn +1 > qn . Since q ∈ Tn and by Lemma 3, one has EσB∗ [δτn (q)−n ] ≤ 1 1 1 − mv /2Mc (1 − q1 ). On the other hand, τn+ (q) ≤ n2 PσB∗ -a.s. This implies δn2 −n1 ≤ 1 − mv /2Mc (1 − q1 ) and thus n2 − n 1 ≥
mv (1 − q1 ) 1 × 1−δ mv + 2Mc
60
J. HÖRNER AND N. VIEILLE
Q.E.D.
as desired.
For later use, we note that the very same argument, applied to n = 1 and to any serious offer q ∈ T1 ensures that EσB∗ [δτ(1) ] ≤ 1 − mv /2Mc (1 − q1 ). This yields the upper bound in Proposition 3. B.4. Proof of Proposition 6 We here prove Proposition 9 below. It corresponds to Proposition 6 except for the upper bound on N0 , which is established in the proof of Proposition 3. PROPOSITION 9: There is a stage N0 such that the following statements hold: P1. Tn ⊆ {qN 1} for all n ≥ N0 . 0 P2. max Tn < 1 for all n < N0 . In addition, πn∗ = 0 for all n ≥ N0 . PROOF: Define N := 1 + max{n : max Tn < 1 and Fn (1) ≥ ν/Mc } (where ν = min[q1 1] (v − c)). Since limn Fn (1) = 0, the stage N is well defined and either FN (1) < ν/Mc or max TN = 1. To start with, assume that the latter holds. We prove that P1 and P2 hold with N0 = N. Since 1 ∈ TN0 , one has πN0 (1) = FN0 (1)(vN0 (1) − c(1)) ≥ 0, and thus πn (1) ≥ 0 for all n ≥ N0 . We argue by contradiction and assume that Tn ∩ (qN 1) = ∅ 0 for some n ≥ N0 , and we call n the first such stage. One thus has vn+1 (1) > ∗ vn (1); hence πn+1 (1) > 0 so that πn+1 > 0 and thus qn+2 > qn+1 . This implies that buyer n + 1 makes a winning offer with probability 1. (Otherwise indeed, the equilibrium payoff of buyer n + 2 would also be positive—a contradiction.) Put otherwise, qn+2 = 1. By Lemma 2, buyer n makes no serious type offer in (qn qn+2 ) = (qN 1). This is the desired contradiction. Therefore, 0 Tn ⊆ {qN 1}, for all n ≥ N0 . 0 Assume now that FN (1) < ν/Mc and that q := maxn
n
PUBLIC VS. PRIVATE OFFERS
= π˜ n (q) +
1 1 − qn
61
q
(v(t) − c(q)) dt q
This is the payoff that would accrue to buyer n if he were the last buyer or, alternatively, if all buyers following n would only submit losing offers. Thus, π˜ n (q) ≥ πn (q) with equality if q = 1. The derivative of π˜ n is given by π˜ n (q) = ≥
1 (v(q) − c(q)) − c (q)(Fn (q) − Fn (q)) 1 − q1 ν − Mc Fn (1) > 0 1 − q1
Therefore, π˜ n is increasing over the interval [q 1], so that πn (1) > πn (q) for every q ∈ [q 1): buyer n makes no offer in (q 1). It remains to prove that πn∗ = 0 for all n ≥ N0 . It suffices to prove that ∗ πN0 = 0. Assume to the contrary that πN∗ 0 > 0. Then buyer N0 would make a winning offer with probability 1, for otherwise πN∗ 0 +1 would also be positive. Therefore, by Lemma 2, buyer N0 − 1 would make no serious type offer in (qN −1 qN +1 ) = (qN −1 1). This would imply that buyer N0 − 1’s equilibrium 0 0 0 payoff is also positive—a contradiction. Q.E.D. There may be several (consecutive) stages consistent with P1 and P2. Without further notice, we choose N0 to be the first of those stages. B.5. Proof of Proposition 7 For convenience, we recall the statement of Proposition 7. PROPOSITION 10: All buyers n < N0 submit a serious offer with positive probability. No buyer n < N0 uses a pure strategy, with the possible exception of the first buyer. PROOF: By definition of N0 , buyer N0 − 1 makes a serious, nonwinning offer with positive probability. We start with the first statement. We argue by contradiction and assume that Tn = {qn } for some n < N0 . (In particular, πn∗ = 0.) Let n∗ > n be the first buyer following n who submits a serious offer with positive probability. Since n∗ ≤ N0 − 1, one has qn∗ := max Tn∗ < 1. Because of discounting and using (5) inductively, one has pn (qn∗ ) − c(qn∗ ) = δn∗ −n (pn∗ (qn∗ ) − c(qn∗ )); hence pn (qn∗ ) < pn∗ (qn∗ ).
62
J. HÖRNER AND N. VIEILLE
Since buyers n ≤ k < n∗ only submit losing offers, the distribution of types faced by buyers n and n∗ is the same, and vn (qn∗ ) = vn∗ (qn∗ ). It follows that πn (qn∗ ) > πn∗ (qn∗ ) = πn∗∗ ≥ 0—a contradiction. This concludes the proof of the first statement. Consider now an arbitrary buyer n, where 1 < n < N0 . If buyer n assigns probability 1 to a specific type offer, it must be to a serious type offer qn > qn , and then qn+1 = qn . On the other hand, pn−1 (qn ) − c(qn ) = δ(pn (qn ) − c(qn )); ∗ = 0 and buyer n − 1 makes no type hence pn−1 (qn ) < pn (qn ). By Lemma 2, πn−1 offer in (qn−1 qn ); hence vn−1 (qn ) = vn (qn ). As above, this implies πn−1 (qn ) > 0—a contradiction. Q.E.D. B.6. Equilibrium Delay Recall that τ(1) = inf{n : p˜ n = c(1)} is the first buyer who submits a winning offer. Recall that we proved that EσB∗ [δτ(1) ] ≤ 1 − mv /2Mc (1 − q1 ). We here proceed to provide a lower bound for EσB∗ [δτ(1) ], which is independent of δ and of the equilibrium σ ∗ . Let N0 be given by Proposition 9 and define N1 := inf{n : Fn (1) ≤ ν/Mc }. We proceed in three steps: Step 1. One has N1 ≤ C1 /(1 − δ), where C1 is independent of δ ≥ δ¯ and of the equilibrium σ ∗ . Let q := maxn
0 is independent of δ ≥ δ¯ and of the equilibrium σ ∗ . By Steps 1–3, one thus has EσB∗ [δτ(1) ] ≥ C2 × (δ1/(1−δ) )C0 +C1 , and the result ¯ ¯ follows since δ1/(1−δ) ≥ δ¯ 1/(1−δ) for every δ ≥ δ. Steps 1 and 2 make use of the following technical result, stated without proof, which links the discounted expectation of a random variable (hereafter, r.v.) to its tail distribution. LEMMA 5: Let τ be a random time with integer values and such that (18)
E[δτ−n |τ > n] ≥ a
for some a > 0 and all n ≥ 1. Then P(τ ≥
1 ab(1−δ)
) ≤ b for all b > 0.
Our computation of C1 and C0 involves three parameters. We choose φ ψ ψ−φ ). Next, we set K := 1 + (Mv (1 − such that 0 < φ < ψ < ν and η ∈ (0 2v(1)
63
PUBLIC VS. PRIVATE OFFERS
¯ − c(q1 )). We express C1 and C0 q1 ))/(2(ν − ψ)), ε := 12 ηK , and a := φδ/(c(1) as functions of these constants. We make no attempt to optimize the choice of φ ψ, and η. Step 1. Define C1 := 1 + (ln(Mv /ν)) a1 ε12 . We prove that N1 ≤ C1 /(1 − δ). For a given stage n, we let qn := max{q ∈ [q 1] : vn (q) > c(q) + φ}. Since vn (1) ≤ c(1), one has qn < 1. On the other hand, since vn (qn ) = v(qn ), one also has qn > qn . The proof is organized as follows. In Claim 1 below, we first prove that conditional on the seller having rejected all previous offers, the probability that the seller’s type does not exceed qn is bounded away from zero. That is, from buyer n’s viewpoint, types below qn have a significant probability. CLAIM 1: One has Fn (qn ) ≥ 2εFn (1). Next, we observe that, since vn (qn ) is bounded away from c(qn ), the offer pn (qn ) with independent type qn is also bounded away from c(qn ). Therefore, it is likely that type qn receives acceptable offers shortly after stage n (for otherwise, he would be willing to accept a price close to c(qn )). In Claim 2, we use this insight to prove that conditional on the seller having rejected all previous offers, it is likely that type qn accepts an offer within 1/aε(1 − δ) additional stages. CLAIM 2: One has Fn+Naε (qn ) < εFn (1), where Naε := the proof of Lemma 5.
1 aε(1−δ)
is defined as in
The assertion of Step 1 immediately follows from Claims 1 and 2. Indeed, observe that, for a given q, Fn (1)−Fn (q) is the probability that the seller rejects all offers from buyers 1 2 n − 1 and has a type t in [q 1]. This difference is nonincreasing in n; hence Fn+Naε (1) − Fn+Naε (qn ) ≤ Fn (1) − Fn (qn ) By Claims 1 and 2, this yields Fn+Naε (1) ≤ (1 − ε)Fn (1) and thus also F1+iNaε (1) ≤ (1 − ε)i for all i ≥ 1. In particular, Fn (1) < ν/Mc as soon as n ≥ 1 + C1 /(1 − δ), as desired. PROOF OF CLAIM 1: We introduce an auxiliary sequence of types which is defined by y0 = q1 and yj+1 = max q ∈ [0 1] : E v(t)|t ∈ [yj q] ≥ c(q) + ψ
64
J. HÖRNER AND N. VIEILLE
until yJ = 1. In particular, E[v(t)|t ∈ [yj yj+1 ]] = c(yj+1 ) + ψ for j < J − 1. On the other hand, observe that yj+1 1 E v(t)|t ∈ [yj yj+1 ] = v(t) dt yj+1 − yj yj ≥ v(yj+1 ) −
Mv (yj+1 − yj ) 2
Since v(yj+1 ) − c(yj+1 ) ≥ ν, this implies that yj+1 − yj ≥
2(ν − ψ) Mv
for j < J − 1
hence J ≤ K. For a given stage n, let jn := min{j : Fn (yj ) ≥ ηK−j Fn (1)}. We now check that qn ≥ yjn , which yields Fn (qn ) ≥ Fn (yjn ) ≥ 2εFn (1), as desired. There is nothing to prove if jn = 0; hence assume jn > 0. By definition of jn , one has Fn (yjn −1 ) < ηFn (yjn ): conditional on t ≤ yjn , it is very likely that buyer n faces a type in [yjn −1 yjn ]. Hence23
EFn v(t)|t ≤ yjn − EFn v(t)|t ∈ yjn −1 yjn ≤ 2ηv(1) (19) since |E(X) − E(X1A )| ≤ 2P(A) sup |X| for every bounded r.v. X and every event A. The first expectation in (19) is vn (yjn ), while the second one is at least c(yjn ) + ψ. Therefore, vn (yjn ) ≥ c(yjn ) + φ and thus qn ≥ yjn . Q.E.D. PROOF OF CLAIM 2: For clarity, we abbreviate qn to q. Recall that τn (q) = inf{m > n : p˜ m ≥ pm (q)} denotes the first buyer after n who submits an offer that is acceptable to type q. For any given stage m ≥ n and since pm (q)−c(q) = EσB∗ [δτn (q)−m (p˜ τn (q) − c(q))|τn (q) > m], one has (20)
pm (q) − c(q) ≤ (c(1) − c(q))EσB∗ δτn (q)−m |τn (q) > m
On the other hand, pm (q) = vm (q) ≥ vn (q) if πm∗ = 0, and then pm (q) − c(q) ≥ ∗ φ, while pm (q) − c(q) ≥ δ(pm+1 (q) − c(q)) ≥ δφ if πm∗ > 0, since then πm+1 = 0. Using (20), this implies EσB∗ δτn (q)−m |τn (q) > m ≥
δφ ≥ a c(1) − c(q)
Apply now Lemma 5 to obtain PσB∗ (τ(q) ≥ n + Naε |τ(q) ≥ n) ≤ ε. 23
Denote by EFn the expectation under the belief held by buyer n.
PUBLIC VS. PRIVATE OFFERS
65
Finally, observe that PσB∗ (τ(q) ≥ n + Naε ) ≥ PσB∗ (τ(q) ≥ n + Naε t ∈ [q1 q]) ≥ PσB∗ (τ(t) ≥ n + Naε t ∈ [q1 q]) = Fn+Naε (q) whereas PσB∗ (τ(q) ≥ n) = PσB∗ (τ(q) ≥ n t ∈ [q1 1]) ≤ PσB∗ (τ(1) ≥ n t ∈ [q1 1]) = Fn (1) Therefore, Fn+Naε (q) ≤ PσB∗ (τ(q) ≥ n + Naε |τ(q) ≥ n) ≤ ε Fn (1) Q.E.D.
as desired.
Step 2. Define C0 = 2Mc /((1 − q1 )mv ) × 1/a(1/ε2 ). We prove that N0 − N1 ≤ C0 /(1 − δ). Denote q∗ := maxn vN0 −1 (1) since buyer N0 − 1 makes a serious, nonwinning offer with positive probability. Thus, N1 − N0 is bounded by the time it takes for vn (1) to increase from c(q∗ ) to c(1). Between stages N1 and N0 , and using the proof of Proposition 9, no buyer ever submits a serious type offer above q∗ . Hence, vn (1) increases steadily with time at a speed which is related to the probability with which successive buyers do trade. Lemma 6 below provides a precise estimate of this relationship. LEMMA 6: Let n < m ≤ N0 be any two stages and denote by πnm := (Fn (1) − Fm (1))/(Fn (1)) the probability that the seller accepts an offer from some buyer n n + 1 m − 1, conditional on having declined all previous offers. Then (21)
vm (1) − vn (1) ≥
mv (1 − q1 ) 2
× (1 − q∗ )
πnm Fm (1)
The proof of Lemma 6 is tedious and somewhat lengthy. It is postponed to the end of the section. By Lemma 5, one has, as in Step 1, πnn+Naε ≥ ε. By
66
J. HÖRNER AND N. VIEILLE
Lemma 6, one thus has vn+Naε (1) − vn (1) ≥ ≥
mv (1 − q1 )
× (1 − q∗ )ε 2 mv (1 − q1 ) c(1) − c(q∗ ) Mc
2
ε
Hence, in any block of Naε consecutive stages k < N0 , the average quality increases by at least (mv (1 − q1 )ε)/2Mc times c(1) − c(q∗ ). In particular, it takes no more than 2Mc /(mv (1 − q1 )ε) such blocks to increase from c(q∗ ) to c(1). The result follows. Step 3. In light of the results obtained so far, this last step is straightforward. Observe first that pN0 (qN ) ≥ v(qN ) ≥ c(qN ) + ν, for otherwise buyer N0 0 0 0 would get a positive payoff when submitting a type offer slightly above qN . 0 Since no buyer n ≥ N0 ever submits a serious offer below 1, one has pN0 qN − c qN = c(1) − c qN EσB∗ δτ(1)−N0 0
0
0
which yields EσB∗ δτ(1)−N0 ≥
ν c(1) − c qN 0
This concludes the proof of Lemma 5.
Q.E.D.
PROOF OF LEMMA 6: Fix the distribution fn of types faced by buyer n and the value of πnm . We minimize vm (1) over all distributions of types that buyer m may possibly be facing. It is convenient to parameterize such distributions by g(t), the probability that a seller with type t would reject all offers from buyers n n + 1 m − 1, so that fm (t) = g(t)fn (t). 1 Hence, vm (1) is minimal when q g(t)fn (t)v(t) dt is minimal. The minimum 1 is computed over all nondecreasing functions g, with values in [0 1] and such that (i) g(t) = 1 over [q∗ 1] (since there is no serious offer beyond q∗ ); 1 (ii) q g(t)fn (t) dt = (1 − πnm )Fn (1). 1 Since v is increasing, the minimum is obtained when g is constant over the interval [q q∗ ], that is, g(t) = ω if t < q∗ and g(t) = 1 if t ≥ q∗ . The value of ω is deduced from (ii) and is given by ωFn (q∗ ) = πnm Fn (1). Thus, 1 1 1 1 (22) vm (1) − vn (1) ≥ v(t)g(t)fn (t) dt − v(t)fn (t) dt Fm (1) q1 Fn (1) q1
67
PUBLIC VS. PRIVATE OFFERS
The rest of the proof consists of showing that the right-hand side of (22) is at least equal to the right-hand side in (21). Plugging g into (22), one has vm (1) − vn (1) 1 q∗ 1 v(t)fn (t) dt − ω v(t)fn (t) dt − vn (1) ≥ Fm (1) q1 q 1 1 q∗ 1 1 ωFn (q∗ ) v(t)fn (t) dt − v(t)fn (t) dt = Fm (1) Fn (1) q1 Fn (q∗ ) q1 1 q∗ πnm Fn (q∗ ) v(t)fn (t) dt − Fn (1) v(t)fn (t) dt = Fm (1)Fn (q∗ ) q q 1 1 =
πnm Fm (1)Fn (q∗ ) 1 v(t)fn (t) dt − (1 − q∗ ) × Fn (q∗ )
q∗ q
q∗
v(t)fn (t) dt
1
b a a where the first equality follows from the identity a−a = b−b ( b − b ), the second b−b follows from the value of ω, and the third follows from Fn (1) = Fn (q∗ ) + (1 − q∗ ). We now use the inequality v(t) ≥ v(q∗ ) + mv (t − q∗ ) (t ∈ [q∗ 1]) to bound the first integral and the inequality v(t) ≤ v(q∗ ) + mv (t − q∗ ) (t ∈ [q1 q∗ ]) to bound the second integral. After simplification, this yields
(23)
vm (1) − vn (1) ≥
πnm (1 − q∗ )mv Fm (1)Fn (q∗ ) q∗ 1 − q∗ + × Fn (q∗ ) (q∗ − t)fn (t) dt 2 q 1
Consider finally the right-hand side of (23). For a given value of Fn (q∗ ), the integral is minimized when fn is constant over [q1 q∗ ] and equal to Fn (q∗ )/(q∗ − q1 ). The integral is then equal to 12 (q∗ − q1 )Fn (q∗ ). Substituting into (23), this yields vm (1) − vn (1) ≥ as desired.
1 − q1 πnm (1 − q∗ )mv × Fm (1) 2 Q.E.D.
68
J. HÖRNER AND N. VIEILLE REFERENCES
AGHION, P., P. BOLTON, C. HARRIS, AND B. JULLIEN (1991): “Optimal Learning by Experimentation,” Review of Economic Studies, 58, 621–654. [46] AKERLOF, G. (1970): “The Market for ‘Lemons’: Qualitative Uncertainty and the Market Mechanism,” Quarterly Journal of Economics, 84, 488–500. [29] BABCOCK, L., AND G. LOEWENSTEIN (1997): “Explaining Bargaining Impasse: The Role of SelfServing Biases,” Journal of Economic Perspectives, 11, 109–126. [30] BIKHCHANDANI, S., D. HIRSHLEIFER, AND I. WELCH (1992): “A Theory of Fads, Fashion, Custom, and Cultural Change as Informational Cascades,” Journal of Political Economy, 100, 992–1026. [48] COASE, R. H. (1972): “Durability and Monopoly,” Journal of Law and Economics, 15, 143–149. [38] CRAMTON, P. (1984): “Bargaining With Incomplete Information: An Infinite-Horizon Model With Continuous Uncertainty,” Review of Economic Studies, 51, 573–591. [46] CRAWFORD, V. P. (1982): “A Theory of Disagreement in Bargaining,” Econometrica, 3, 607–638. [30] DENECKERE, R., AND M.-Y. LIANG (2006): “Bargaining With Interdependent Values,” Econometrica, 74, 1309–1364. [38,46] EVANS, R. (1989): “Sequential Bargaining With Correlated Values,” Review of Economic Studies, 56, 499–510. [38,46] FUDENBERG, D., AND J. TIROLE (1991): Game Theory. Cambridge, MA: MIT Press. [32,33] GUL, F., AND H. SONNENSCHEIN (1988): “On Delay in Bargaining With One-Sided Uncertainty,” Econometrica, 56, 601–611. [46] HENDEL, I., AND A. LIZZERI (1999): “Adverse Selection in Durable Goods Markets,” American Economic Review, 89, 1097–1115. [46] HENDEL, I., A. LIZZERI, AND M. SINISCALCHI (2005): “Efficient Sorting in a Dynamic AdverseSelection Model,” Review of Economic Studies, 72, 467–497. [46] HÖRNER, J., AND N. VIEILLE (2007): “Public vs. Private Offers in the Market for Lemons,” Center for Mathematical Studies DP, Northwestern University. [44,47] (2009): “Supplement to ‘Public vs. Private Offers in the Market for Lemons’,” Econometrica Supplement Material, 77, http://www.econometricsociety.org/ecta/Supmat/6917_ extensions.pdf. [31,45] HOUSE, C. L., AND J. V. LEAHY (2004): “An sS Model With Adverse Selection,” Journal of Political Economy, 112, 581–614. [46] JANSSEN, M. C. W., AND S. ROY (2002): “Dynamic Trading in a Durable Good Market With Asymmetric Information,” International Economic Review, 43, 1, 257–282. [46] NÖLDEKE, G., AND E. VAN DAMME (1990): “Signalling in a Dynamic Labour Market,” Review of Economic Studies, 57, 1–23. [46] RILEY, J. G., AND R. ZECKHAUSER (1983): “Optimal Selling Strategies: When to Haggle and When to Hold Firm,” Quarterly Journal of Economics, 76, 267–287. [46] SAMUELSON, W. (1984): “Bargaining Under Asymmetric Information,” Econometrica, 52, 995–1005. [45] SWINKELS, J. (1999): “Educational Signalling With Preemptive Offers,” Review of Economic Studies, 66, 949–970. [46] TAYLOR, C. (1999): “Time-on-the-Market as a Signal for Quality,” Review of Economic Studies, 66, 555–578. [46] VINCENT, D. (1989): “Bargaining With Common Values,” Journal of Economic Theory, 48, 47–62. [38,46] WILSON, C. (1980): “The Nature of Markets With Adverse Selection,” Bell Journal of Economics, 11, 108–130. [33]
Dept. of Economics, Yale University, 30 Hillhouse Avenue, New Haven, CT 06520-8281, U.S.A.; [email protected]
PUBLIC VS. PRIVATE OFFERS
69
and Département Economie et Finance, HEC Paris, 78351 Jouy-en-Jasas, France; [email protected]. Manuscript received January, 2007; final revision received May, 2008.
Econometrica, Vol. 77, No. 1 (January, 2009), 71–92
ALL-PAY CONTESTS BY RON SIEGEL1 This paper studies a class of games, “all-pay contests,” which capture general asymmetries and sunk investments inherent in scenarios such as lobbying, competition for market power, labor-market tournaments, and R&D races. Players compete for one of several identical prizes by choosing a score. Conditional on winning or losing, it is weakly better to do so with a lower score. This formulation allows for differing production technologies, costs of capital, prior investments, attitudes toward risk, and conditional and unconditional investments, among others. I provide a closed-form formula for players’ equilibrium payoffs and analyze player participation. A special case of contests is multiprize, complete-information all-pay auctions. KEYWORDS: All-pay, contests, auctions, rent-seeking, lobbying.
1. INTRODUCTION IN MANY SETTINGS, economic agents compete by making irreversible investments before the outcome of the competition is known. Lobbying activities, research and development races, and competitions for promotions, to name a few, all have this property. This type of competition has been widely studied in the literature. In the classic all-pay auction with complete information (henceforth: all-pay auction), for example, rivals incur a cost of bidding that is the same whether they win or lose, but may differ in their valuation for winning the single prize. The all-pay auction has been used to model rent-seeking and lobbying activities (Hillman and Samet (1987), Hillman and Riley (1989), Baye, Kovenock, and de Vries (1993)), competitions for a monopoly position (Ellingsen (1991)), waiting in line (Clark and Riis (1998)), sales (Varian (1980)), and R&D races (Dasgupta (1986)). Variations of the all-pay auction have been used to model competitions for multiple prizes (Clark and Riis (1998) and Barut and Kovenock (1998)), the effect of lobbying caps (Che and Gale (1998, 2006) and Kaplan and Wettstein (2006)), and R&D races with endogenous prizes (Che and Gale (2003)). While this literature has produced interesting results, the models considered are often restrictive in some or all of the following dimensions: the types of asymmetries across players, the number of prizes, the number of players, and the degree of irreversibility of the investments. 1
I am indebted to Ed Lazear and Ilya Segal for their continuous guidance and encouragement. I thank Jeremy Bulow for many beneficial discussions. I thank Larry Samuelson and four referees for very detailed comments. I thank Julio González-Díaz, Matthew Jackson, Sunil Kumar, Jonathan Levin, Stephen Morris, Marco Ottaviani, Kareen Rozen, Yuval Salant, Andrzej Skrzypacz, Balázs Szentes, Michael Whinston, Robert Wilson, Eyal Winter, and seminar participants at Chicago GSB, CSIO-IDEI, Cornell, Hebrew University, Northwestern, NYU Stern, Penn State, Stanford, Tel-Aviv, Toronto, and Yale for helpful comments and suggestions. © 2009 The Econometric Society
DOI: 10.3982/ECTA7537
72
RON SIEGEL
The goal of this paper is to better understand competitions in which contestants are asymmetrically positioned and make irreversible investments. To this end, I investigate all-pay contests (henceforth: contests). In a contest, each player chooses a costly “score” and the players with the highest scores obtain one prize each (relevant ties can be resolved using any tie-breaking rule). Thus, ex post, each player can be in one of two states: winning or losing. Conditional on winning or losing, a player’s payoff decreases weakly and continuously with his chosen score; choosing a higher score entails a player-specific cost, which may differ across the two states. The primitives of the contest are commonly known. This captures players’ knowledge of the asymmetries among them. Consequently, equilibrium payoffs represent players’ “economic rents,” in contrast to “information rents” that arise in models of competition with private information.2 Contests are defined in Section 2. The generality of players’ cost functions allows for differing production technologies, costs of capital, and prior investments, among others. Moreover, contests allow for nonordered cost functions. These arise when different competitors are disadvantaged relative to others in different regions of the competition (see the example in Section 1.1 below). In addition, state-dependent costs accommodate both sunk and conditional investments, player-specific risk attitudes, and player- and score-dependent valuations for a prize.3 When all investments are unconditional, each player is characterized by his valuation for a prize, which is the payoff difference between the two states, and a weakly increasing, continuous cost function that determines his cost of choosing a score independently of the state. I refer to such contests as separable contests. Separable contests nest many models of competition that assume a deterministic relation between effort and prize allocation.4 For example, the models of Che and Gale (2006) and Kaplan and Wettstein (2006) are two-player, single-prize separable contests.5 Single-prize and multiprize all-pay 2
Such models typically assume ex ante identical players (for example, Moldovanu and Sela (2001, 2006) and Kaplan, Luski, Sela, and Wettstein (2002)). An exception is the work by Parreiras and Rubinchik (2006), who allowed for asymmetry between players and provided a partial characterization of equilibrium. 3 González-Díaz (2007) also allowed for nonlinear state-dependent costs, but accommodated only a single prize and ordered costs. His techniques and results do not generalize to nonordered costs or multiple prizes. 4 Other models assume a probabilistic relation between players’ efforts and prize allocation. The classic examples are Tullock’s (1980) “lottery model,” in which players are symmetric and each player’s probability of winning the single prize is proportional to the player’s share of the total expenditures, and Lazear and Rosen’s (1981) two-player tournaments. Recent contributions to this large, single-prize literature that accommodate a degree of asymmetry include Cornes and Hartley (2005) and Szymanski and Valletti (2005). The analysis of probabilistic models typically focuses on pure-strategy equilibria by using first-order conditions. Functional form assumptions are made “[. . . ] to ensure the existence of pure-strategy equilibria and first-order conditions characterizing these equilibria” (Szymanski and Valletti (2005)). 5 Although Che and Gale (2006) have N ≥ 2 players, they assumed strictly ordered costs so only two players participate in equilibrium.
ALL-PAY CONTESTS
73
auctions are separable contests with linear costs, in which asymmetries among players are captured only by differences in valuations for a prize. Section 3 begins the analysis by identifying an unambiguous ranking of players. Such a ranking is not immediately obvious, because players’ costs may not be ordered. Players are ranked in decreasing order of their reach, where a player’s reach is the highest score he can choose without obtaining a negative payoff if he wins a prize with certainty. The key result of the paper is Theorem 1, which provides a full characterization of players’ expected equilibrium payoffs in a “generic” contest for m prizes. Expected payoffs are determined by reaches, powers, and the threshold. The threshold is the reach of player m + 1. A player’s power equals his payoff from winning a prize with certainty when choosing a score equal to the threshold. Theorem 1 shows that under “generic” conditions—checked using players’ powers—each player’s expected payoff equals the higher of his power and zero. Thus, a generic contest has the same payoffs in all equilibria. Theorem 1 also shows that the number of players who obtain strictly positive expected payoffs equals the number of prizes. The derivation of the payoff characterization does not rely on solving for an equilibrium. The payoff result implies that a player’s expected payoff does not depend on his cost when he loses. For example, a player’s expected payoff in a modified complete-information first-price auction in which he pays a strictly positive fraction of his bid if he loses does not depend on the size of the fraction. When the fraction equals 1 for all players, we have an all-pay auction. Section 3.2 discusses contests that are not generic. In such contests, players’ payoffs in at least one equilibrium are specified by the payoff characterization, but payoffs in other equilibria may be different. Perturbing a contest that is not generic leads to a generic contest. Section 4 examines equilibrium participation. Participation is related to the ordering of players’ costs. Theorem 2 shows that when players’ costs (appropriately normalized) are strictly ordered, at most m + 1 players participate in any equilibrium. This is why precisely m + 1 players participate in an all-pay auction with distinct valuations. In contrast, nonordered costs may lead more than two players to participate even when there is only one prize.6 Section 5 concludes by briefly discussing implications of the analysis for rent dissipation and comparative statics. The Appendix contains an example of a contest with multiple equilibria, the proofs of Corollary 1 and Theorem 2, and a technical lemma. 6 When more than two players participate, general asymmetric costs significantly complicate equilibrium analysis. For this reason, a noncrossing property is the standard assumption in the literature. The few papers that analyze mixed-strategy equilibria with more than two participants make assumptions, in addition to noncrossing, that lead to limited asymmetry among participating players (see Baye, Kovenock, and de Vries (1996), Clark and Riis (1998), and González-Díaz (2007), who showed that the analysis of Baye, Kovenock, and de Vries (1996) generalizes to nonlinear costs).
74
RON SIEGEL
I begin with an example that illustrates the payoff result and some of its implications. 1.1. An Example Three risk-neutral firms compete for one monopoly position allocated by a government official. Each firm chooses how much to invest in lobbying activities, and this investment leads to a score, which can be interpreted as the amount of influence the firm has achieved over the government official. The firm with the highest score obtains the monopoly position, which carries a monetary value of 1, but all firms pay the lobbying costs associated with their respective scores. Firms 1 and 2 have better “lobbying technologies” than firm 3, because they have better lobbyists, are located closer to the government official, or have lower costs of capital. Firm 3 has an initial advantage, due to prior investments or reputation. Figure 1 depicts players’ cost functions, where player i ∈ {1 2 3} corresponds to firm i. Firm 3’s initial advantage is captured by its initial marginal cost, γ ≥ 0, which is low relative to the other firms’ marginal costs. In contrast, firm 3’s cost for high scores is high relative to those of the other two firms. Thus, players’ costs are not ordered. Cost functions are commonly known, and relevant ties are broken randomly. The costs of choosing 1 for players 1, 2, and 3 are K <1, 1, and L > 1, respectively. Consequently, players 2 and 3 would never choose a score higher than 1, since that would cost them more than the value of the prize. Player 1 can therefore guarantee himself a payoff arbitrarily close to 1 − K by choosing a score slightly higher than 1, so 1 − K is a lower bound on his expected payoff in any equilibrium. Player 1 does not, however, choose scores greater or equal to 1 with certainty in equilibrium, since players 2 and 3 would best-reply by choosing scores lower than 1, in which case player 1 would be better off choosing a lower score. In fact, player 1 must employ a mixed strategy in any equilibrium. It may therefore
FIGURE 1.—Players’ costs.
ALL-PAY CONTESTS
75
seem plausible that such a strategy could give player 1 an equilibrium payoff higher than 1 − K. Theorem 1 shows that the expected equilibrium payoff of player 1 is exactly 1 − K. Similarly, players 2 and 3 can guarantee themselves no more than 0; the payoff characterization shows that this is exactly their equilibrium payoff. This implies that aggregate equilibrium expenditures equal K. For low, strictly positive values of γ, all three players must participate (invest) in equilibrium. This is shown in Section 4. Each player contributes to aggregate expenditures and wins the prize with strictly positive probability. This participation behavior results from the nonordered nature of players’ cost functions.7 Now consider a variant of the contest, in which player 3 has 0 marginal cost up to a score whose cost for player 1 is at least 1. This represents a very large initial advantage for player 3. In this case, there is a pure-strategy equilibrium in which player 3 wins with certainty and no player invests. Thus, it may be that no player invests in a contest for a valuable prize. Regardless of the value of γ, precisely one player receives a strictly positive expected payoff. This too follows from the payoff characterization, since there is only one prize. Section 5 discusses the effects of changes in competition structure. For example, the addition of player 3 from Figure 1 to a contest that includes only players 1 and 2 changes neither expected payoffs nor expected aggregate expenditures, but changes individual expenditures for low, strictly positive values of γ, because all three players participate. Thus, the addition of a player may change equilibrium behavior without changing players’ payoffs or aggregate expenditures. In contrast, lowering the prize’s value can lead to a positive payoff for player 3, making him the only player who obtains a positive expected payoff. 2. THE MODEL In a contest, n players compete for m homogeneous prizes, 0 < m < n. The set of players {1 n} is denoted by N. Players compete by each choosing a score simultaneously and independently. Player i chooses a score si ∈ Si = [ai ∞), where ai ≥ 0 is his initial score. Positive initial scores capture starting advantages, or “head starts,” without allowing players to choose lower scores. This may eliminate equilibria involving weakly dominated strategies (see Example 4 in Siegel (2007)). Each of the m players with the highest scores wins one prize. In case of a relevant tie, any procedure may be used to allocate the tie-related prizes among the tied players. 7 Unlike in the all-pay auctions of Baye, Kovenock, and de Vries (1996), participation by more than two players in a contest for one prize does not rely on players’ valuations being identical.
76
RON SIEGEL
Player i has preferences over lotteries whose outcomes are pairs (si Wi ), where si is the player’s score and Wi indicates whether he obtains a prize (Wi = 1) or not (Wi = 0). These preferences are represented by a Bernoulli utility function. Because Wi equals 0 or 1, this function can be written as Wi vi (si ) − (1 − Wi )ci (si ), where vi : Si → R is player i’s valuation for winning and ci : Si → R is player i’s cost of losing. The primitives of the contest are commonly known. Thus, given a profile of scores s = (s1 sn ), si ∈ Si , player i’s payoff is ui (s) = Pi (s)vi (si ) − (1 − Pi (s))ci (si )
×
Sj → [0 1], player i’s probability of winning, satisfies ⎧ 0 if sj > si for m or more players j = i, ⎪ ⎨ 1 if sj < si for N − m or more Pi (s) = players j = i, ⎪ ⎩ any value in [0 1] otherwise, n such that j=1 Pj (s) = m. Note that a player’s probability of winning depends on all players’ scores, but his valuation for winning and cost of losing depend only on his chosen score. I make the following assumptions. where Pi :
j∈N
ASSUMPTION A1: vi and −ci are continuous and nonincreasing. ASSUMPTION A2: vi (ai ) > 0 and limsi →∞ vi (si ) < ci (ai ) = 0. ASSUMPTION A3: ci (si ) > 0 if vi (si ) = 0. Assumption A1 means that, conditional on winning or losing, a lower score is weakly preferable.8 This represents an “all-pay” component. Assumption A2 means that with the initial score, winning is better than losing, so prizes are valuable, but losing with the initial score is preferable to winning with sufficiently high scores. Assumption A3 means that if winning with score si is as good as losing with the initial score, then winning with score si is strictly better than losing with score si . This last condition stresses the all-pay nature of contests. It is not satisfied by complete-information first-price auctions, for example, since a player pays nothing if he loses and is therefore indifferent between losing and winning with a bid that equals his valuation for the prize. But the condition is met when an all-pay element is introduced; for example, when 8 The “nonincreasing” part of Assumption A1 can be relaxed to requiring that (i) vi be nonincreasing where it is nonnegative, that is, if y > x, then vi (x) ≥ 0 and vi (y) ≥ 0 imply vi (x) ≥ vi (y), (ii) vi cross 0 once from above, that is, if y > x, vi (x) < 0 implies vi (y) < 0, and (iii) ci ≥ 0 and if y > x, then ci (x) > 0 implies ci (y) > 0.
ALL-PAY CONTESTS
77
every bidder pays some strictly positive fraction of his bid whether he wins or not, and only the winner pays the balance of his bid. The formulation allows the difference between a player’s valuation for winning and his cost of losing to depend on his chosen score. For example, in a competition for promotions in which a higher score is achieved by investing in managerial skills, such skills are costly to acquire and may increase the value associated with a promotion. Alternatively, it may be that the value of the prize for player i is fixed at Vi but some costs are only borne if the player wins, so his costs are ciW when he wins and ciL when he loses. In this case, vi (si ) = Vi − ciW (si ) and ci = ciL , so ui (s) = Pi (s)(Vi − ciW (si )) − (1 − P(s))ciL (si ). When thinking about the outcome of a contest in monetary terms, contests can capture players’ risk attitudes. In the previous setting, for example, we can let vi (si ) = f (Vi − ciW (si )) and ci = f (ciL ) for some strictly increasing f such that f (0) = 0. One subclass of contests that is of particular interest is separable contests. In a separable contest, every player i’s preferences over lotteries with outcomes (si Wi ) depend only on the marginal distributions of the lotteries.9 This implies that the effect of winning or losing on a player’s Bernoulli utility is additively separable from that of the score. That is, vi (si ) = Vi − ci (si ) and ui (s) = Pi (s)Vi − ci (si ) for Vi = vi (ai ) > 0.10 If we interpret payoffs as money, the value ci (si ) could be thought of as player i’s cost of choosing score si , which does not depend on whether he wins or loses, and Vi could be thought of as player i’s valuation for a prize, which does not depend on his chosen score. All expenditures are unconditional, and players are risk neutral. The example of Section 1.1 depicts a separable contest with three players and nonlinear costs. The lobbying games of Kaplan and Wettstein (2006) and Che and Gale (2006) are two-player separable contests. Separable contests with linear costs are the single- and multi-prize complete-information all-pay auctions (Hillman and Samet (1987), Hillman and Riley (1989), Clark and Riis (1998)).11
9 For example, in a separable contest, player i is indifferent between a lottery in which (ai 1) occurs with probability 12 and (si 0) occurs with probability 12 , and a lottery in which (ai 0) occurs with probability 12 and (si 1) occurs with probability 12 (for any si ≥ ai ), because the marginal distribution of the first coordinate is the same in both lotteries, and the marginal distribution of the second coordinate is the same in both lotteries. 10 Indeed, if player i’s Bernoulli utility is not additively separable, then vi (ai ) + ci (ai ) = vi (si ) + ci (si ) for some si > ai . But then the player is not indifferent between the two lotteries described in the previous footnote, even though their marginal distributions are the same. The converse is also true: if player i’s Bernoulli utility is additively separable, it is immediate that his preferences over lotteries with outcomes (si Wi ) depend only on the marginal distributions of the lotteries. 11 Formally, vi (si ) = Vi − si ci (si ) = si ai = 0, and ties are resolved by randomizing uniformly, where Vi is bidder i’s valuation for a prize.
78
RON SIEGEL
3. PAYOFF CHARACTERIZATION The following concepts are key in analyzing the payoffs of players in equilibrium. DEFINITIONS: (i) Player i’s reach ri is the highest score at which his valuation for winning is 0. That is, ri = max{si ∈ Si |vi (si ) = 0}. Re-index players in (any) decreasing order of their reach, so that r1 ≥ r2 ≥ · · · ≥ rn . (ii) Player m + 1 is the marginal player. (iii) The threshold T of the contest is the reach of the marginal player: T = rm+1 . (iv) Players i’s power wi is his valuation for winning at the threshold. That is, wi = vi (max{ai T }). In particular, the marginal player’s power is 0. In a separable contest, a player’s reach is the highest score he can choose by expending no more than his valuation for a prize. In the example of Section 1.1, players are indexed in decreasing order of their reach, player 2 is the marginal player, and the threshold is 1. Player 1’s power is 1 − K > 0, player 2’s power is 0, and player 3’s power is 1 − L < 0. In an m-prize all-pay auction, player i is the player with the ith highest valuation, and his power equals his valuation less that of the marginal player. Theorem 1 below characterizes players’ equilibrium payoffs in contests that meet the following two conditions. GENERIC CONDITIONS: (i) Power Condition—The marginal player is the only player with power 0. (ii) Cost Condition—The marginal player’s valuation for winning is strictly decreasing at the threshold. That is, for every x ∈ [am+1 T ), vm+1 (x) > vm+1 (T ) = 0.12 I refer to a contest that meets the Generic Conditions as a generic contest. The separable contest in the example of Section 1.1 is generic, because the marginal player’s costs are strictly increasing at the threshold and only he has power 0. An m-prize all-pay auction meets the Cost Condition because costs are strictly increasing. If the Power Condition is met, that is, the valuation of the marginal player is different from those of all other players, the all-pay auction is generic. Contests that do not meet the Generic Conditions can be perturbed slightly to meet them. Perturbing the marginal player’s valuation for winning around the threshold leads to a contest that meets the Cost Condition. Doing the same for all players with power 0 generates a contest 12 In a separable contest, because vm+1 (x) = Vm+1 − cm+1 (x), the cost condition is that for every x ∈ [am+1 T ), cm+1 (x) < cm+1 (T ) = Vm+1 .
ALL-PAY CONTESTS
79
that meets the Power Condition. Note that in a generic contest, players in NW = {1 m} (“winning players”) have strictly positive powers, and players in NL = {m + 1 n} (“losing players”) have nonpositive powers. I now state the main result of the paper. THEOREM 1: In any equilibrium of a generic contest, the expected payoff of every player equals the maximum of his power and 0. An immediate implication of Theorem 1 is that in a generic contest players in NW have strictly positive expected payoffs and players in NL have expected payoffs of 0. Equivalently, a player obtains a strictly positive expected payoff in a generic contest if and only if his reach is strictly higher than the threshold. Players’ equilibrium strategies may be mixed, so players in NW may obtain a prize with probability smaller than 1 and players in NL may obtain a prize with strictly positive probability. It is only expected payoffs that are positive for players in NW and 0 for players in NL . In the example of Section 1.1, NW = {1} and NL = {2 3}. The contest is generic, so player 1’s payoff is 1 − K, and those of players 2 and 3 are 0. In an m-prize generic all-pay auction in which player i’s value is Vi , the payoff of every player i is max{Vi − Vm+1 0}.13 I use the following notation in the proof of Theorem 1. Pi (·), player i’s probability of winning, and ui (·), player i’s utility, are expanded to mixed strategies. A mixed strategy Gi of player i is a cumulative probability distribution that assigns probability 1 to his set of pure strategies Si . When a strategy profile G = (G1 Gn ) is specified, Pi (x) is shorthand for player i’s probability of winning when he chooses x ≥ ai with certainty and all other players play according to G, and similarly for ui (x). For an equilibrium (G1 Gn ), denote by ui = ui (Gi ) player i’s equilibrium payoff. Note that in equilibrium best responses are chosen with probability 1. The phrase “player i beats player j” refers to player i choosing a strictly higher score than player j does. For a set I, denote by |I| the cardinality of I. PROOF OF THEOREM 1: Choose a generic contest and an equilibrium G = (G1 GN ) of the contest. LEAST LEMMA: A player’s expected payoff in G is at least the maximum of his power and 0. PROOF: Every player i can guarantee himself a payoff of 0 by choosing his initial score, ai (recall that vi (ai ) > 0 and ci (ai ) = 0). It therefore suffices to consider players with strictly positive power, all of whom are in NW . In equilibrium, no player chooses scores higher than his reach with a strictly positive 13 Players’ payoffs in a multiprize all-pay auction in which all players have different valuations were first derived by Clark and Riis (1998).
80
RON SIEGEL
probability, since choosing such scores leads to a negative payoff (by Assumptions A1 and A3). So, by choosing max{ai T + ε} for ε > 0, a player i in NW beats all N − m players in NL with certainty. This means that for every player i in NW , ui ≥ vi (max{ai T + ε}) → vi (max{ai T }) = wi ε→0
by continuity of vi .
Q.E.D.
TIE LEMMA: Suppose that in G two or more players have an atom at a score x, that is, choose x with strictly positive probability. Then players who have an atom at x either all win with certainty or all lose with certainty when choosing x. PROOF: Denote by N the set of players who have an atom at x, with |N | ≥ 2. Denote by E the strictly positive-probability event that all players in N choose x. Denote by D ⊆ E the event in which a relevant tie occurs at x, that is, the event in which m prizes are divided among the |N | players in N , with 1 ≤ m < |N |. Suppose D has strictly positive probability. Then, conditional on D, at least one player i in N can strictly increase his probability of winning to 1 by choosing a score slightly higher than x, regardless of the tie-breaking rule. Since i chooses x with strictly positive probability, x ≤ ri so vi (x) > −ci (x) (by Assumptions A1–A3). Thus, by continuity of vi and ci , player i would be strictly better off by choosing a score slightly higher than x. Therefore, D has probability 0. This implies that P(E) = P(E L ) + P(E W ), where E L ⊆ E is the event that at least m players in N\N choose scores strictly higher than x, E W ⊆ E is the event that at most m − |N | players in N\N choose scores strictly higher than x, and P(A) denotes the probability of event A. By independence of players’ strategies, either E L or E W have probability 0; otherwise, D would have strictly positive probability. Suppose that P(E) = P(E L ). Independence of players’ strategies now implies that, without conditioning on E, at least m players in N\N choose scores strictly higher than x with probability 1, so Pi (x) = 0 for every player i in N . Similarly, if P(E) = P(E W ), then Q.E.D. Pi (x) = 1 for every player i in N . Several players may have an atom at the same score in equilibrium. The Tie Lemma only rules out ties in which at least one player wins with a strictly positive probability that is less than 1. That no such ties arise in equilibrium helps establish which players have an expected payoff of 0. ZERO LEMMA: In G, at least n − m players have best responses with which they win with probability 0 or arbitrarily close to 0. These players have an expected payoff of at most 0. PROOF: Denote by J a set of some m + 1 players. Denote by S˜ the union of the best-response sets of the players in J and denote by sinf the infimum
ALL-PAY CONTESTS
81
˜ Consider three cases: (i) two or more players in J have an atom at sinf , of S. (ii) exactly one player in J has an atom at sinf , and (iii) no player in J has an atom at sinf . Case i. Denote by N ⊆ J the set of players in J who have an atom at sinf . It cannot be that Pi (sinf ) = 1 for every player i in N : because any player in J\N chooses scores strictly higher than sinf with probability 1, even if the players in N\J choose scores strictly lower than sinf with probability 1, only m − |(J\N )| = m − (m + 1 − |N |) = |N | − 1 > 0 prizes are divided among the |N | players in N . Thus, the Tie Lemma shows that Pi (sinf ) = 0 for every player i in N . Case ii. Denote by i the only player in J with an atom at sinf . Pi (sinf ) = 0, since all m players in J\{i} choose scores strictly higher than sinf with probability 1. In Cases i and ii, Pi (sinf ) = 0 for some player i in J who has an atom at sinf , so sinf is a best response for this player at which he wins with probability 0. Case iii. By definition of sinf , there exists a player i in J with best responses that approach s . Since 1 ≥ 1 − P (x ) ≥ {xn }∞ inf i n n=1 j∈J\{i} (1 − Gj (xn )), no player in J has an atom at sinf , and G is right-continuous, as n tends to infinity Pi (xn ) approaches 0. Because J was a set of any m + 1 players, at least n − m players have best responses with which they win with probability 0 or arbitrarily close to 0, and therefore have an expected payoff of at most 0 Q.E.D. The Least Lemma, the Tie Lemma, and the Zero Lemma hold regardless of the Generic Conditions. The Least Lemma and the Power Condition show that the m players in NW have strictly positive expected payoffs. Therefore, the Least Lemma and the Zero Lemma imply that under the Power Condition the n − m players in NL obtain expected payoffs of 0. Using this fact, I show that players in NW obtain at most their power. THRESHOLD LEMMA: The players in NW have best responses that approach or exceed the threshold and, therefore, have an expected payoff of at most their power. PROOF: By the Power Condition, players in NL \{m + 1} have strictly negative powers. Their reaches, and therefore the supremum of their best responses, are strictly below the threshold. Consequently, there is some ssup < T such that Gi (x) = 1 for every player i in NL \{m + 1} and every score x > ssup . This implies that every player i in NW chooses scores that approach or exceed the threshold, that is, has Gi (x) < 1 for every x < T . Otherwise, for some s in (ssup T ), Gi (s) = 1 for all but at most m − 1 players in N\{m + 1}. But then the marginal player could win with certainty by choosing a score in (max{am+1 s} T ) (note that am+1 < T ); because of the Cost Condition, this would give him a strictly positive payoff, a contradiction (recall that the marginal player is in NL and therefore has an expected payoff of 0).
82
RON SIEGEL
Take a player i in NW . Because Gi (x) < 1 for every x < T , there exists a sequence {xn }∞ n=1 of best responses for player i that approach some zi ≥ T . Since xn is a best response for player i, who has a strictly positive payoff by the Least Lemma and the Power Condition, vi (xn ) > 0. So, by Assumptions A1 and A2, vi (xn ) > −ci (xn ). By continuity of vi , we have ui = ui (xn ) = Pi (xn )vi (xn ) − (1 − Pi (xn ))ci (xn ) ≤ vi (xn ) → vi (zi ) ≤ vi (T ) = wi
xn →zi
so every player in NW obtains at most his power.
Q.E.D.
The Least Lemma and the Threshold Lemma, which relies on the Generic Conditions, show that players in NW have expected payoffs equal to their power. We have seen that players in NL have expected payoffs of 0. Since NL ∪ NW = N, the expected payoff of every player equals the maximum of his power and 0. Q.E.D. 3.1. Discussion of the Payoff Characterization Equilibrium payoffs in generic contests depend only on players’ valuations for winning at the threshold, even though the equilibria generally depend on players’ valuations for winning and costs of losing at all scores up to the threshold.14 From an applied perspective, only the reach of each player and valuations for winning at a single score—the threshold—need to be computed. In particular, players’ costs of losing do not affect payoffs. This means, for example, that a player’s expected payoff in an all-pay auction does not change if, instead of paying his entire bid, he pays only a strictly positive fraction of his bid in advance and the rest only if he wins (as long as the fraction is specified in advance). The payoff result implies that the number of players who obtain positive expected payoffs equals the number of prizes. That no more than m players obtain positive payoffs for every realization of players’ strategies (and tie-breaking randomization, if necessary) follows from the definition of contests. The payoff result shows that this is also true in expectation. As the example of Section 1.1 illustrates, contests do not, in general, have pure-strategy equilibria.15 Existence of an equilibrium (in pure or mixed strategies) is not immediately obvious, since payoffs are discontinuous in pure strategies, of which there is a continuum. Simon and Zame’s (1990) result shows 14 Equilibria may also include multiple atoms at various scores and multiple gaps in players’ best-response sets (see Example 4 and page 23 of Siegel (2007)). 15 Pure-strategy equilibria arise when players with positive power have head starts sufficiently large to dissuade weaker players from participating. Such equilibria do not arise in all-pay auctions, regardless of the difference in players’ valuations. The payoff result applies to both pureand mixed-strategy equilibria.
ALL-PAY CONTESTS
83
that an equilibrium exists for some tie-breaking rule. The following corollary of their result and the Tie Lemma above, the proof of which is in the Appendix, shows that an equilibrium exists for any tie-breaking rule. COROLLARY 1: Every contest has a Nash equilibrium. The payoff result does not rely on equilibrium uniqueness: Example 3 in the Appendix describes a generic separable contest and two equilibria of the contest. Moreover, these equilibria lead to different allocations of the prize and different aggregate expenditures, so standard revenue equivalence techniques cannot be used to compare players’ payoffs across equilibria. 3.2. Contests That Are Not Generic Although the payoff result does not apply to contests that are not generic, it implies the following statement. COROLLARY 2: Every contest (generic or not) has at least one equilibrium in which every player’s payoff is the maximum of his power and 0. This “upper-hemicontinuity” result can be proved by considering a sequence of generic contests that “approach” the original contest and an equilibrium for each contest in the sequence. Every limit point (in the weak∗ topology) of the resulting sequence of equilibria is an equilibrium of the original contest in which payoffs are given by the payoff result.16 The payoff result also holds for contests in which all players are identical, even though such contests do not meet the Power Condition (all players have power 0). This is because in any equilibrium of any contest, identical players have identical payoffs,17 and the Zero Lemma, which does not require the Generic Conditions, shows that at least one player has payoff 0. Therefore, we can state another corollary: COROLLARY 3: In any equilibrium of a contest in which all players are identical, all players have a payoff of 0. 16 Probability measures on the Borel subsets of the compact metric set [ai ri + ε] are regular (for any ε > 0), so the set of these probability measures is weak∗ compact and has at least one limit point (see, for example, Dunford and Schwartz (1988)). 17 Suppose players i and j = i are identical, and consider an equilibrium G. Player i can choose scores slightly above the supremum of player j’s best responses, beating player j for sure and beating the other players at least as often as player j does with any best response. This implies that player i’s payoff in G is at least as high as that of player j by applying a continuity argument and a reasoning similar to that used in the proof of Theorem 2.
84
RON SIEGEL
When players are not identical and the contest is not generic, the payoff of a player in some equilibrium may be very close to his valuation for winning at his initial score, even if his power is very low. Example 1 below shows this when the Power Condition fails; Example 2 below shows this when the Cost Condition fails. EXAMPLE 1—The Power Condition Fails and the Payoff Result Does Not Hold: Consider the following three-player separable contest for one prize of common value 1, which is a modification of the example from Section 1.1. Players’ costs are ⎧ if 0 ≤ x ≤ h, ⎨ (1 − α)x αh c1 (x) = (x − h) if x > h, ⎩ (1 − α)h + 1 + 1−h ⎧ if 0 ≤ x ≤ h, ⎨ (1 − ε)x εh c2 (x) = (x − h) if x > h, ⎩ (1 − ε)h + 1 + 1−h
γx if 0 ≤ x ≤ h c3 (x) = γh + L(x − h) if x > h for some small α ε in (0 1), small γ ≥ 0, h in (0 1), and L > 0. Regardless of the value of L, the threshold is 1 and the Power Condition is violated (since at least two players have power 0). Costs are strictly increasing at 1 for all players, so the Cost Condition is met. It is straightforward to verify that for any h in (0 1), there exist some β > 0 and M > 0 such that if α ε γ < β and L > M, then (G1 G2 G3 ) is an equilibrium, for ⎧ 0 if x < 0, ⎪ ⎪ ⎪ γ x ⎪ ⎪ −1 if 0 ≤ x ≤ h, ⎨ (1 − ε)h + 1−α h G1 (x) = εh ⎪ ⎪ ⎪ (1 − ε)h + 1 + (x − h) if h < x ≤ 1, ⎪ ⎪ 1−h ⎩ 1 if x > 1, ⎧ 0 if x < 0, ⎪ ⎪ ⎪ if 0 ≤ x ≤ h, ⎨ (1 − α)h αh G2 (x) = (1 − α)h + 1 + (x − h) if h < x ≤ 1, ⎪ ⎪ ⎪ 1−h ⎩ 1 if x > 1,
x/ h if x ≤ h G3 (x) = 1 if x > h The top part of Figure 2 depicts players’ costs. The bottom part depicts players’ atoms and densities in the equilibrium (G1 G2 G3 ).
ALL-PAY CONTESTS
85
FIGURE 2.—Cost functions and the equilibrium (G1 G2 G3 ) of Example 1.
Player 3’s power is 1 − γh − L(1 − h) (his valuation for the prize less his cost of choosing the threshold), so as L increases, player 3’s power becomes arbitrarily low. But as h tends to 1 and ε α and γ tend to 0, for any value of L > M, player 3 wins with near certainty and his payoff (1 − ε)(1 − α)h2 − γh approaches the value of the prize. While the equilibrium of Figure 2 may seem robust, a slight change in player 1’s or 2’s valuation for the prize leads to a generic contest and destroys the equilibrium (since Theorem 1 then applies). EXAMPLE 2—The Cost Condition Fails and the Payoff Result Does Not Hold: When the Cost Condition fails, the marginal player cannot obtain a strictly positive payoff on some interval of scores leading up to the threshold, so competition may stop before the threshold is reached. Consider the following two-player separable contest for one prize of common value 1. Players’ costs are c1 (x) = bx for some b < 1 and ⎧x ⎪ ⎨ c2 (x) = d ⎪ ⎩ 1 2x − 1
if 0 ≤ x < d if d ≤ x ≤ 1 if x > 1
for some d in (0 1). Player 1’s reach is b1 > 1 player 2’s reach is 1, the threshold is 1, and players’ powers are w1 = 1 − b > 0 and w2 = 0, so the Power Condition holds, but the Cost Condition fails, because c2−1 (c2 ( r2 )) = [d 1]. As a result,
86
RON SIEGEL
(G1 G2 ) is an equilibrium in which player 1 has a payoff of 1 − bd > w1 , for ⎧ 0 if x < 0 ⎪ ⎨x if 0 ≤ x ≤ d G1 (x) = ⎪ ⎩d 1 if x > d 0 if x < 0 G2 (x) = 1 − bd + bx if 0 ≤ x ≤ d 1 if x > d As b approaches 1, player 1’s power approaches 0, but for any value of b, as d approaches 0, player 1’s payoff approaches 1, the value of the prize. Note, ˜ 2 ) is an equilibrium in which both players’ payoffs equal however, that (G1 G their powers, for ˜ 2 (x) = G
0 1 − b + bx 1
if x < 0 if 0 ≤ x ≤ 1 if x > 1
4. PARTICIPATION A player participates in an equilibrium of a contest if with strictly positive probability he chooses scores associated with strictly positive costs of losing. Players with strictly negative powers that are disadvantaged everywhere with respect to the marginal player do not participate in any equilibrium. This is the content of Theorem 2, which is proved in the Appendix. THEOREM 2: In a generic contest, if the normalized costs of losing and valuations for winning for the marginal player are, respectively, strictly lower and weakly higher than those of player i > m + 1, that is, ci (x) cm+1 (max{am+1 x}) < vm+1 (am+1 ) vi (ai )
for all x ∈ Si such that ci (x) > 0
vi (x) vm+1 (max{am+1 x}) ≥ vm+1 (am+1 ) vi (ai )
for all x ∈ Si
and
then player i does not participate in any equilibrium. In particular, if these conditions hold for all players in NL \{m + 1}, then in any equilibrium only the m + 1 players in NW ∪ {m + 1} may participate.
ALL-PAY CONTESTS
87
Theorem 2 shows that in a generic all-pay auction only players 1 m + 1 may participate,18 and because players’ cost functions in all-pay auctions are strictly increasing and all initial scores equal 0, players 1 m + 1 do indeed participate.19 This explains why in all-pay auctions with distinct valuations precisely the m + 1 players with the highest valuations place strictly positive bids with strictly positive probability. In contrast, the example of Section 1.1 shows that a player in NL \{m + 1} may participate if he has a local advantage with respect to the marginal player. Indeed, the proof of the Threshold Lemma shows that players 1 and 2 choose scores that approach or exceed the threshold, and so participate in any equilibrium. Suppose player 3 did not participate. Then players 1 and 2 would have to play strategies that make all scores in (0 T ) best responses for both of them.20 For low values of γ > 0, player 3 could then obtain a strictly positive payoff by choosing a low score, contradicting the payoff result. Thus, player 3 must also participate in any equilibrium, even though his expected equilibrium payoff is 0. 5. CONCLUDING REMARKS All-pay contests capture general asymmetries among contestants, and allow for both sunk and conditional investments. This paper has provided a closedform formula for players’ expected payoffs in generic contests and has analyzed players’ participation. The main insight is that reach and power are the right variables to focus on when examining contests. Additional, seemingly complicated questions become simple when this insight in put to use. Consider, for example, the issue of rent dissipation, which is central to the rent-seeking literature. In a separable contest for m prizes of value V , aggregate equilibrium expenditures are simply mV less players’ payoffs. As winning players’ powers approach 0, which happens when their costs at the threshold approach V , rent dissipation is complete. As winning players’ powers approach V , which happens when their costs at the threshold approach 0, no rent is dissipated. The addition of a player to a contest never lowers the threshold and, therefore, makes existing players weakly worse off. If the new player’s reach is below the existing threshold, existing players’ payoffs do not change and the new 18 Baye, Kovenock, and de Vries (1996) showed that more than two players may participate in certain nongeneric, single-prize all-pay auctions. 19 The m players in NW participate because, as shown in the proof of the Threshold Lemma, they choose scores that approach or exceed the threshold. If the marginal player does not choose scores that approach or exceed the threshold, any player in NW can win with certainty and increase his payoff by choosing a score strictly below the threshold, contradicting the Threshold Lemma. 20 Apply Lemma 1 in the Appendix to the contest that includes only players 1 and 2.
88
RON SIEGEL
player has a payoff of 0. The addition of a prize makes player m + 2 the marginal player, and this lowers the threshold and makes existing players better off. In contrast, making prizes more valuable may make players worse off, because it raises the threshold. Further analysis of these issues, which have implications for contest design, is left for future work. APPENDIX EXAMPLE 3: This example depicts a separable contest and two equilibria of the contest, in which different players participate and in which aggregate expenditures differ. Let n = 4 and m = 1. I extend the example of Section 1.1 by adding player 4, without constructing the contest explicitly. I will demonstrate the existence of two equilibria, G and G , such that only players 1, 2, and 3 participate in G, and only players 1, 2, and 4 participate in G . Different player participation implies different allocations, since a player chooses a costly score only if he has a strictly positive probability of winning by doing so. Begin with an equilibrium G = (G1 G2 G3 ) of the example of Section 1.1. When γ is small and positive, all three players participate in G as shown in Section 4. Let t3 = inf{x : G3 (x) = 1}. Add player 4 with V4 = 1 and continuous, increasing costs c4 that are lower than those of player 3 below t3 and equal to them starting from t3 . That is, ∀x ∈ (0 t3 ): c4 (x) ∈ (0 c3 (x)) and ∀x ≥ t3 : c4 (x) = c3 (x). There exist such functions c4 for which an equilibrium is for players 1, 2, and 3 to play G, and for player 4 not to participate. Indeed, assume that player 4 does not participate. Since player 3’s equilibrium payoff is 0 (his power is negative), he obtains at most 0 by choosing any score when players 1 and 2 play G1 and G2 , respectively, and he wins a prize under G if and only if he beats players 1 and 2. By Lemma 1 below, there are no atoms in (0 T ) in G, so ∀x ∈ (0 T ): P3 (x) = G1 (x)G2 (x). Player 4, who considers joining in when the others are playing G, must beat players 1, 2, and 3 to win, that is, ∀x ∈ (0 T ): P4 (x) = G1 (x)G2 (x)G3 (x). Let x ∈ (0 t3 ) (note that t3 < T ), which implies that G3 (x) < 1. If u3 (x) = 0, then P3 (x) > 0 because c3 (x) > 0. Thus, P4 (x) < P3 (x) and P4 (x) − c3 (x) < P3 (x) − c3 (x) = u3 (x) = 0. If u3 (x) < 0, since P4 (x) ≤ P3 (x) again P4 (x) − c3 (x) < 0. Thus, there exist continuous, increasing functions c4 such that ∀x ∈ (0 t3 ): c4 (x) ∈ (0 c3 (x)) and P4 (x) − c4 (x) < 0. For such functions, it is a best response for player 4 not to participate when the other players play G. Since G is an equilibrium of the contest that includes only players 1, 2, and 3, we have an equilibrium. Maintaining the same cost functions, consider now an equilibrium G = (G1 G2 G4 ) of the contest that includes only players 1, 2, and 4. As in the example of Section 1.1, all three players must participate in G . When player 3 is added to the contest and does not participate, this remains an equilibrium. Indeed, player 4’s payoff is zero in G (his power is negative) and at every score, player 3’s costs are weakly higher than those of player 4 whereas his probability of winning is weakly lower than that of player 4.
ALL-PAY CONTESTS
89
If aggregate expenditures under the two equilibria are the same, multiply the valuation and cost of player 4 by some strictly positive d = 1. This does not change the equilibria of the contest, but changes aggregate expenditures in G . PROOF OF COROLLARY 1: Consider a contest C and the restricted contest C , in which every player i chooses scores in Si = [ai K] for K = maxi∈N ri < ∞. Any equilibrium of C is an equilibrium of C, since scores higher than K are strictly dominated by ai for every player i. Thus, it suffices to show that C has an equilibrium. ∗ To do this, consider S ∗ = i∈N Si \{(s1 sn )|∃i = j : si = sj }. That is, S is the set of n-tuples of distinct strategies. Players’ payoffs are bounded and continuous on S ∗ , which is dense in i∈N Si . So, following Simon and Zame (1990, p. 864), there exists some tie-breaking rule, which may be score dependent, such that C has a mixed-strategy equilibrium G when this tie-breaking rule is used. Denote the game in which this tie-breaking rule is used by C˜ and denote player i’s payoff in the equilibrium G by u˜ i . To complete the proof, it suffices to show that G is an equilibrium of C . I do this in two steps. Step 1. A Gi -measure 1 of best responses for player i in C˜ gives player i the same payoff u˜ i in C as it does in C˜ when all other players play according to G. Indeed, choosing a score si at which no player has an atom gives player i the same payoff regardless of the tie-breaking rule. Since the number of atoms in G is countable, it is enough to show that when choosing Gi -atoms in C , player i obtains u˜ i . Consider a Gi -atom si . If only i has an atom at si , then a tie occurs with probability 0 so player i obtains u˜ in C . If there are multiple atoms at si , by the Tie Lemma the tie is never binding regardless of the tie-breaking rule, so again player i obtains u˜ i in C . Step 2. No score gives player i a payoff in C higher than u˜ i . The only scores si to check are those at which player i does not have an atom and another player does. Consider such a score si , and denote player i’s payoff in C when ˜ player choosing si by ui (si ). By choosing scores slightly higher than si in C, i can obtain a payoff of at least ui (si ) − ε for any ε > 0. Thus, ui (si ) ≤ u˜ i . Q.E.D.
×
×
PROOF OF THEOREM 2: Since dividing a player’s Bernoulli utility by vi (ai ) > 0 does not change his strategic behavior, it suffices to prove the result for contests in which vi (ai ) = 1 for every player i. Choose an equilibrium G of such a contest and suppose player i > m + 1 that meets the conditions of the proposition participated in G. Let ti = inf{x : Gi (x) = 1} < T and let t˜i = max{am+1 ti }. Then t˜i < T , Pi (ti ) < 1 (because the m players in NW choose scores that approach or exceed the threshold, as shown in the proof of the Threshold Lemma), and for every δ > 0: Pm+1 (t˜i + δ) ≥ Pi (ti ) since by choosing (t˜i + δ), player m + 1 beats player i for sure and beats the other players
90
RON SIEGEL
at least as often as player i does. Therefore, since for every δ > 0 such that t˜i + δ < rm+1 = T we have vm+1 (t˜i + δ) > 0 ≥ −cm+1 (t˜i + δ) we obtain um+1 ≥ Pm+1 (t˜i + δ)vm+1 (t˜i + δ) − (1 − Pm+1 (t˜i + δ))cm+1 (t˜i + δ) ≥ Pi (ti )vm+1 (t˜i + δ) − (1 − Pi (ti ))cm+1 (t˜i + δ) Now, by definition of participation, ci (ti ) > 0, so ci (ti ) > cm+1 (t˜i ). Since Pi (ti ) < 1 and vm+1 (max{am+1 x}) ≥ vi (x) for all x ∈ Si , by continuity of vm+1 and cm+1 , player m + 1 can choose t˜i + δ for sufficiently small δ > 0 such that Pi (ti )vm+1 (t˜i + δ) − (1 − Pi (ti ))cm+1 (t˜i + δ) > Pi (ti )vi (ti ) − (1 − Pi (ti ))ci (ti ) = ui (ti ) ≥ 0 so um+1 > 0, which contradicts the payoff result because wm+1 = 0.
Q.E.D.
LEMMA 1: In any equilibrium G of a contest with strictly decreasing valuations for winning, strictly increasing costs of losing, and initial scores of 0, (i) G is continuous on (0 T ) and (ii) every score in (0 T ) is a best response for at least two players. PROOF: Suppose that a player i had an atom at x ∈ (0 T ). Because players 1 m + 1 choose scores that approach or exceed the threshold (see the argument in footnote 19), Pj (x) < 1 for every player j and, in particular, Pi (x) < 1. If Pi (x) = 0 then player i would be better off by choosing 0. Therefore, Pi (x) ∈ (0 1). Now consider a player j = i. Because player i has an atom at x and Pi (x) ∈ (0 1), player j would be better off choosing scores slightly above x than choosing scores slightly below x. To see this, suppose player j chose x and all other players played according to G, with some randomization as the tie-breaking rule at x. Then player i would still win with a probability in (0 1) when choosing x, so j would be better off choosing a slightly higher score by reasoning similar to that of the Tie Lemma. Thus, no player j = i chooses scores in some region below x (regardless of the tie-breaking rule) and, by the Tie Lemma, no player j = i has an atom at x. Therefore, player i would be better off by choosing a score slightly below x. This shows that G is continuous on (0 T ). For (ii), note that if x ∈ (0 T ) is not a best response for player i, then continuity of G implies that the same is true for scores in some neighborhood of x. Therefore, if only one player had a best response at x, he could choose scores slightly lower than x and win with the same probability, making him better off. Suppose no player had a best response at x. Then there would be a gap in the union of players’ best response sets. Continuity of G
ALL-PAY CONTESTS
91
on (0 T ) implies that the top of the gap cannot be below the threshold, and because valuations for winning are strictly decreasing and Gi (T ) = 1 for every player i in NL , players in NW do not have best responses above the threshold, so Gi (T ) = 1 for every player i. Now every player i in {1 m + 1} has Gi (x) < 1 because he chooses scores that approach or exceed the threshold. So the top of the gap must be the threshold, and players {1 m + 1} must each have an atom there, contradicting the Tie Lemma. This shows that x is a best response for at least two players. Q.E.D. REFERENCES BARUT, Y., AND D. KOVENOCK (1998): “The Symmetric Multiple Prize All-Pay Auction With Complete Information,” European Journal of Political Economy, 14, 627–644. [71] BAYE, M. R., D. KOVENOCK, AND C. G. DE VRIES (1993): “Rigging the Lobbying Process: An Application of All-Pay Auctions,” American Economic Review, 83, 289–294. [71] (1996): “The All-Pay Auction With Complete-Information,” Economic Theory, 8, 291–305. [73,75,87] CHE, Y.-K., AND I. GALE (1998): “Caps on Political Lobbying,” American Economic Review, 88, 643–651. [71] (2003): “Optimal Design of Research Contests,” American Economic Review, 93, 646–671. [71] (2006): “Caps on Political Lobbying: Reply,” American Economic Review, 96, 1355–1360. [71,72,77] CLARK, D. J., AND C. RIIS (1998): “Competition Over More Than One Prize,” American Economic Review, 88, 276–289. [71,73,77,79] CORNES, R., AND R. HARTLEY (2005): “Asymmetric Contests With General Technologies,” Economic Theory, 26, 923–946. [72] DASGUPTA, P. (1986): “The Theory of Technological Competition,” in New Developments in the Analysis of Market Structure, ed. by J. E. Stiglitz and G. F. Mathewson. Cambridge, MA: MIT Press, 519–547. [71] DUNFORD, N., AND J. T. SCHWARTZ (1988): Linear Operators Part I: General Theory. New York: Wiley. [83] ELLINGSEN, T. (1991): “Strategic Buyers and the Social Cost of Monopoly,” American Economic Review, 81, 648–657. [71] GONZÁLEZ-DÍAZ, J. (2007): “A Unifying Model for First-Price Winner-Takes-All Contests,” Mimeo. [72,73] HILLMAN, A. L., AND J. G. RILEY (1989): “Politically Contestable Rents and Transfers,” Economics and Politics, 1, 17–39. [71,77] HILLMAN, A. L., AND D. SAMET (1987): “Dissipation of Contestable Rents by Small Numbers of Contenders,” Public Choice, 54, 63–82. [71,77] KAPLAN, T. R., AND D. WETTSTEIN (2006): “Caps on Political Lobbying: Comment,” American Economic Review, 96, 1351–1354. [71,72,77] KAPLAN, T. R., I. LUSKI, A. SELA, AND D. WETTSTEIN (2002): “All-Pay Auctions With Variable Rewards,” Journal of Industrial Economics, 50, 417–430. [72] LAZEAR, E. P., AND S. ROSEN (1981): “Rank-Order Tournaments as Optimum Labor Contracts,” Journal of Political Economy, 89, 841–864. [72] MOLDOVANU, B., AND A. SELA (2001): “The Optimal Allocation of Prizes in Contests,” American Economic Review, 91, 542–558. [72] (2006): “Contest Architecture,” Journal of Economic Theory, 126, 70–96. [72] PARREIRAS, S., AND A. RUBINCHIK (2006): “Contests With Many Heterogeneous Agents,” Mimeo. [72]
92
RON SIEGEL
SIEGEL, R. (2007): “All-Pay Contests,” Ph.D. Thesis, Stanford Graduate School of Business. [75, 82] SIMON, L. K., AND W. R. ZAME (1990): “Discontinuous Games and Endogenous Sharing Rules,” Econometrica, 58, 861–872. [82,89] SZYMANSKI, S., AND T. M. VALLETTI (2005): “Incentive Effects of Second Prizes,” European Journal of Political Economy, 21, 467–481. [72] TULLOCK, G. (1980): “Efficient Rent Seeking,” in Toward a Theory of the Rent Seeking Society, ed. by J. M. Buchanan, R. D. Tollison, and G. Tullock. College Station, TX: Texas A&M University Press, 269–282. [72] VARIAN, H. (1980): “A Model of Sales,” American Economic Review, 70, 651–658. [71]
Dept. of Economics, Northwestern University, 2001 Sheridan Road, Evanston, IL 60208, U.S.A.; [email protected]. Manuscript received November, 2007; final revision received July, 2008.
Econometrica, Vol. 77, No. 1 (January, 2009), 93–105
THE COMPLEXITY OF FORECAST TESTING BY LANCE FORTNOW AND RAKESH V. VOHRA1 Consider a weather forecaster predicting a probability of rain for the next day. We consider tests that, given a finite sequence of forecast predictions and outcomes, will either pass or fail the forecaster. Sandroni showed that any test which passes a forecaster who knows the distribution of nature can also be probabilistically passed by a forecaster with no knowledge of future events. We look at the computational complexity of such forecasters and exhibit a linear-time test and distribution of nature such that any forecaster without knowledge of the future who can fool the test must be able to solve computationally difficult problems. Thus, unlike Sandroni’s work, a computationally efficient forecaster cannot always fool this test independently of nature. KEYWORDS: Forecast testing, prediction, bounded rationality.
1. INTRODUCTION SUPPOSE ONE IS ASKED to forecast the probability of rain on successive days. Sans knowledge of the distribution that governs the change in weather, how should one measure the accuracy of the forecast? One criterion for judging the effectiveness of a probability forecast, called calibration, has been an object of interest. Dawid (1982) offered the following intuitive definition of calibration: Suppose that, in a long (conceptually infinite) sequence of weather forecasts, we look at all those days for which the forecast probability of precipitation was, say, close to some given value ω and (assuming these form an infinite sequence) determine the long run proportion p of such days on which the forecast event (rain) in fact occurred. The plot of p against ω is termed the forecaster’s empirical calibration curve. If the curve is the diagonal p = ω, the forecaster may be termed (empirically) well calibrated.
Foster and Vohra (1993) exhibited a randomized forecasting algorithm that with high probability will be calibrated on all sequences of wet–dry days. Thus, a forecaster with no meteorological knowledge would be indistinguishable from one who knew the distribution that governs the change in weather.2 This has inspired the search for tests of probability forecasts that can distinguish between a forecaster who knows the underlying distribution of the process being forecast and one who simply “games” the test. A test takes as input a forecasting algorithm and a sequence of outcomes, and after some period accepts the forecast (PASS) or rejects it (FAIL). Sandroni (2003) proposed two properties that such a test should have. The first is that the test should declare PASS/FAIL after a finite number of periods. This seems unavoidable for a practical test. Second, suppose the forecast is indeed 1 Research supported in part by NSF grant ITR IIS-0121678. We thank A. Sandroni and W. Olszewski and the referees for useful comments. 2 Lehrer (2001), Sandroni, Smorodinsky, and Vohra (2003), and Vovk and Shafer (2005) gave generalizations of this result.
© 2009 The Econometric Society
DOI: 10.3982/ECTA7163
94
L. FORTNOW AND R.V. VOHRA
correct, that is, accurately gives the probability of nature in each round. Then the test should declare PASS with high probability. We call this second condition passing the truth. Call a test that satisfies these two conditions a good test. A test based on calibration is an example of a good test. A forecaster with no knowledge of the underlying distribution who can pass a good test with high probability on all sequences of data is said to have ignorantly passed the test. For every good test, Sandroni (2003) showed that there exists a randomized forecasting algorithm that will ignorantly pass the test. Therefore, no good test can distinguish between a forecaster who knows the underlying distribution of the process being forecast from one who simply games the test. Since this randomized forecast can pass the test for all distributions, it must be independent of the underlying (if any) distribution, being forecasted. Hence, in some sense these forecasts provide no information at all about the process being forecasted. Dekel and Feinberg (2006) as well as Olszewski and Sandroni (2007) got around the impossibility of result of Sandroni (2003) by relaxing the first property of a good test; for example, allowing the test to declare PASS/FAIL at infinity, allowing the test to declare FAIL in a finite number of periods but PASS at infinity, or relaxing the condition that the test always passes the truth. These tests can often be made efficient in the sense that they can run in time linear in the length of the current sequence, but the number of forecasts before a bad forecaster is failed could be extremely large as a function of the forecaster. Olszewski and Sandroni (2008) have noted that the tests considered by Dekel and Feinberg (2006) and Olszewski and Sandroni (2007) rely on counterfactual information. Specifically, the test can use the predictions the forecast would have made along sequences that did not materialize because the test has access to the forecasting algorithm itself. As noted by Olszewski and Sandroni (2008), this is at variance with practice. For this reason they considered tests that are not permitted to make use of counterfactual predictions on the part of the forecaster, but relaxed the condition that the test must decide in finite time. Formally, two different forecasting algorithms that produce the same forecast on a realization must be treated in the same way. If such tests pass the truth with high probability, they show that for each such test, there is a forecasting algorithm that can ignorantly pass the test. It is natural to ask if a test, using a proper scoring rule3 like log-loss, can circumvent these difficulties. Here one penalizes the forecaster log p if the forecaster predicts a probability p of rain and it rains, and a penalty of log(1 − p) if it does not rain. The lowest possible score that can be obtained is the longrun average entropy of the distribution. One could imagine the test passing the forecaster if the log-loss matches the entropy. However, such a test would need 3 Assuming the forecaster is compensated on the basis of the scores associated with the rule, a proper scoring rule gives the forecaster the incentive to reveal his/her true beliefs. See Good (1952).
THE COMPLEXITY OF FORECAST TESTING
95
to know the entropy of the distribution. As noted in the Introduction, we are concerned with tests which operate without any prior knowledge of the distribution. Proper scoring rules are good methods to compare two forecasters, but are not useful for testing the validity of a single forecaster against an unknown distribution of nature. 1.1. Computationally Bounded Forecasters This paper will examine the consequences of imposing computational limits on both the forecaster and the test. We measure the complexity as a function of the length of the history so far. Most practical tests have a complexity that is polynomial in the length of the history, so it seems reasonable to restrict attention to good tests that have a complexity that is polynomial in the length of the history. Restricting the test in this way should make it easier to be ignorantly passed. It seems natural to conjecture that for every polynomial-time good test, there exists a polynomial-time randomized forecasting algorithm that will ignorantly pass the test. Remarkably, this is not the case. We exhibit a good linear-time test that would require the forecaster to factor numbers under a specific distribution or fail the test. The existence of an efficient (i.e., probabilistic polynomialtime) algorithm for factoring composite numbers is considered unlikely. Indeed, many commercially available cryptographic schemes are based on just this premise. This result suggests that the “ignorant” forecaster of Sandroni (2003) must have a complexity at least exponential in n. Hence, the ignorant forecaster must be significantly more complex than the test. In particular, its complexity may depend on the complexity of nature’s distribution. To prove this result, we interpret the observed sequence of 0–1’s as encoding a number followed by a list of its possible factors. A sequence that correctly encodes a list of factors is called correct. The test fails any forecaster who does not assign high probability to these correct sequences when they are realized. Consider now the distribution that puts most of its weight on correct sequences. If the forecaster can ignorantly pass the test, he/she must be able to identify sequences that correspond to correct answers to our computational question. The factoring proof does not generalize to all NP problems, because we need a unique witness so as to guarantee that the test always passes the truth. Witness reduction techniques as in Valiant and Vazirani (1986) do not appear to help. Our second result strengthens the previous one by exhibiting a good test that requires the forecaster to solve PSPACE-hard problems4 by building on the structure of specific interactive proof systems. While the latter result is strictly stronger than the factoring result, we present both since the factoring proof 4 See Section 5 of this paper for a definition. This class contains the more well known class of NP-hard problems.
96
L. FORTNOW AND R.V. VOHRA
is much simpler and illustrates some of the techniques used in the PSPACE proof. In both cases the tests are deterministic. Furthermore, they use only the realized outcomes and forecasts to render judgement. We can modify the factoring proof to use efficiently sampleable distributions of nature. Whether we can do the same for the more general PSPACE result remains open. In addition we also consider the possibility that the test may have more computational power than the forecaster. If we restrict ourselves to forecasters using time O(t(n)), there is a test T using time O(nO(1) t(n)) with the following properties. • For all distributions of nature μ, T will pass, with high probability, a forecaster forecasting μ. • For some distribution τ of nature, for every forecaster F running in time O(t(n)), T will fail F with high probability. If one takes a highly noncomputable distribution τ, a forecaster would not be able to forecast τ well, but T could not test this in general if T is also required to always pass the truth. In a nutshell, no forecasting algorithm can ignorantly pass a good test that is more complex than itself. In Section 2 we give formal definitions of forecasters and tests. Section 3 shows that the forecaster needs to be nearly as powerful as the test. Section 4 gives a simple test where a successful ignorant forecasters must be able to factor. In Section 5 we sketch a more complicated test that checks if the forecaster successfully acts like a prover in an interactive proof system. This section will also provide a brief description of an interactive proof system and its implications. 2. DEFINITIONS Let N be the set of natural numbers. Let S = {0 1} be the state space.5 An element of S is called an outcome. Let S n , for n ∈ N, be the n-Cartesian product of S. An n-sequence of outcomes is denoted s = (s1 s2 sn ) ∈ S n , where si denotes the state realized in period i. Given s ∈ S n and r < n, let sr = (s1 s2 sr ) ∈ S r be the prefix of length r of s. An element of [0 1] is called a forecast of the event 1. A forecast made at period r refers to outcomes that will be observed in period r + 1. Let Δ∗ be the set of probability distributions over [0 1]. A forecasting algorithm is a function F:
n−1 (S r × [0 1]r ) → Δ∗ r=0
At the end of each stage r < n, an r-history (sr f0 f1 fr−1 ) ∈ S r × [0 1]r is observed. Here fj ∈ [0 1] is the forecast made by F in period j. Let 5
The results can easily be extended to more than two states.
THE COMPLEXITY OF FORECAST TESTING
97
f r = (f0 fr ). Based on this r-history, the forecaster must decide which forecast fr ∈ [0 1] to make in period r. The forecaster is allowed to randomize, so fr ∈ [0 1] can be selected (possibly) at random, using a probability distribution in Δ∗ . An n-outcome sequence s ∈ S n and a forecasting algorithm F determine a probability measure F¯ s on [0 1]n , where, conditional on (sr f r−1 ), the probabilities of forecasts next period are given by F(sr f r−1 ). The vector of realized forecasts associated with F on a sequence s will be denoted f (s). Denote the unknown data generating process by P. Given P and sr ∈ [0 1]r , let Psr ∈ [0 1] be the probability that sr+1 = 1 conditional on sr . Given P, let F P (s) ∈ [0 1]n be the forecast sequence such that frP (s) = Psr . A test is a function T : S n × [0 1]n → {0 1}. After a history of n forecasts and outcomes are observed, a test must either accept (PASS) or reject (FAIL) the forecast. When the test returns a 0, the test is said to FAIL the forecast based on the outcome sequence. When the test returns a 1, the test is said to PASS the forecast based on the outcome sequence. A test is said to pass the truth with probability 1 − ε if Pr s : T (s F P (s)) = 1 ≥ 1 − ε P
for all P. A test T can be ignorantly passed by a forecasting algorithm F with probability 1 − ε if for all P, Pr s : T (s f (s)) = 1 ≥ 1 − ε P
Equivalently, for all s ∈ S n , Pr T (s f (s)) = 1 ≥ 1 − ε where the probability is with respect to the forecaster’s randomness. Therefore, F can ignorantly pass T if on any sequence of outcomes, the realized forecast sequence will not be failed with probability at least 1−ε (under the distribution induced by the forecasting scheme). A test T is said to fail the forecasting algorithm F on the distribution Q with probability 1 − ε if
Pr s : Pr T (s f (s)) = 1 ≥ 1 − ε ≤ ε Q
F¯ s
THEOREM 1—Sandroni’s Theorem: Suppose test T passes the truth with probability 1 − ε. Then there is a forecasting algorithm F that can ignorantly pass T with probability 1 − ε.
98
L. FORTNOW AND R.V. VOHRA
3. TESTS MORE COMPLEX THAN THE FORECAST We show that no forecasting algorithm can ignorantly pass a test with probability 1 − ε that is more complex than itself. The basic idea of the proof can be found in Dawid (1985). For notational simplicity our exposition will be limited to deterministic forecasts. The extension to randomized forecasts is straightforward and we outline it at the end of the section. Let t(·) be a time-constructible function and let p(·) be a polynomial.6 The results here and later in the paper hold only for sufficiently large n, that is, n > n0 , where n0 can depend on the forecaster and ε. For small n, a forecaster could be hard-wired with enough information to fool any test that passes the truth. LEMMA 1: For every deterministic forecaster F using time t(n) there is a distribution P, polynomial p, and test T , using time p(n)t(n), so that for all ε > 0 and n sufficiently large: (i) T passes the truth with probability at least 1 − ε. (ii) T fails F on P with probability 1 − ε. PROOF: Let n be sufficiently large and let a sequence s∗ = (s1∗ s2∗ s3∗ sn∗ ) of outcomes be such that the probability F assigns to seeing sj∗ given (s1∗ ∗ sj−1 ) and f j−2 is at most 1/2 for all j ≤ n. To describe the test, let G be any forecasting algorithm and let (s g(s)) be an n-history. • If s = s∗ , then T (s g(s)) = 1. • If s = s∗ and the probability that G assigns to s∗ is less than ε, then T (s g(s)) = 1. • If s = s∗ and the probability that G assigns to s∗ is at most ε, then T (s g(s)) = 0. Observe that the truth is failed if and only if s∗ is realized and the probability assigned by the truth to this event is less than ε. Since this can only happen with probability less than ε, it follows that the test passes the truth with probability 1 − ε. Now we exhibit a distribution P such that PrP ({s : T (s f ) = 1}) = 0 Let P be the distribution that puts measure 1 on s∗ . The probability that F assigns to s = s∗ is less than (1/2)n ≤ ε. Hence, the test fails F on P with probability 1 − ε. Q.E.D. LEMMA 2: Suppose m deterministic forecasting algorithms, F1 F2 Fm , that each use time t(n). Then there is a distribution P, polynomial p, and test T that uses mp(n)t(n) time so that for all ε > 0 and n sufficiently large: 6 A time-constructible function t(n) is one which can be constructed from n by a Turing machine in time order t(n). Formally, there exists a Turing machine M which, given a string of n 1’s, returns the binary representation of t(n) in O(t(n)) steps. All natural functions are time constructible.
THE COMPLEXITY OF FORECAST TESTING
99
(i) T passes the truth with probability 1 − ε. (ii) T fails all F1 F2 Fm on P with probability 1 − ε. PROOF: Consider the sequence of integers 1 1 2 1 2 3 1 2 3 4 1 2 3 4 5 Let h(j) be the jth element of the sequence. Let n be sufficiently large and let a sequence s∗ = (s1∗ s2∗ s3∗ sn∗ ) of outcomes be such j−2 ∗ ) and fh(j) is that the probability Fh(j) assigns to seeing sj∗ given (s1∗ sj−1 j−2 ∗ fh(j) (sj∗ |s1∗ sj−1 fh(j) ) ≤ 1/2 for all j ≤ n. If h(j) > m, let sj∗ = 0. To describe the test, let G be any forecasting algorithm and let (s g(s)) be an n-history. • If s = s∗ , then T (s g(s)) = 1. • If s = s∗ and the probability that G assigns to s∗ is at least ε, then T (s g(s)) = 1. • If s = s∗ and the probability that G assigns to s∗ is at most ε, then T (s g(s)) = 0. Observe that the truth is failed if and only if s∗ is realized and the probability assigned by the truth to this event is less than ε. Since this can only happen with probability less than ε, it follows that the test passes the truth with probability 1 − ε. Let P be the distribution that puts measure 1 on s∗ . Choose any Fk . Consider the subsequence {sj∗ }h(j)=k of s∗ . The probability that Fk assigns to this subsequence is at most (1/2)|{j : h(j)=k}| < ε. Hence, the failure condition in the part third of the test holds. Q.E.D. THEOREM 2: For any t(n) there is a polynomial p and a test T of complexity p(n)t(n) with the following properties. (i) For all n sufficiently large the test passes the truth with probability 1 − ε. (ii) There is a distribution P on infinite 0–1 sequences such that each forecaster F of complexity t(n) fails the test on P with probability 1 − ε for all n ≥ nF , where nF depends on the forecaster F . (iii) P is independent of the forecaster and n. PROOF: Let F1 F2 be an enumeration of all t(n)-computable forecasts. Note that the sequence s∗ defined in Lemma 2 does not depend on n. Let P be the distribution that puts all its weight on the sequence s∗ . For each n the test we require coincides with the test as defined in Lemma 2. This test will pass the truth with high probability for every n. For any t(n) computable test Fk , by Lemma 2, for sufficiently large n, the test will fail Fk on P with probability 1 − ε. Q.E.D. To extend the results to randomized forecasts it suffices to show how the proof of Lemma 1 can be modified. Recall that the sequence s∗ was chosen so that the forecasted probability assigned by the forecast to sj∗ (given the previous history) is less than 1/2. For a randomized forecast, we choose sj∗ so as to
100
L. FORTNOW AND R.V. VOHRA
maximize the probability of choosing a forecast of less than 1/2 of observing event sj∗ . 4. FORECASTS THAT FACTOR In this section we describe a test that always passes the truth and any forecaster who could ignorantly pass this test must be able to factor any number. We prove our results for deterministic forecasters, but one can extend, in a manner similar to Section 3, the results to randomized forecasters. Given an integer k and a 0–1 sequence s, we can interpret the sequence as encoding an arbitrary tuple of numbers. The first number in the tuple we shall call the prefix of the sequence. The remaining numbers (the suffix) will be interpreted as a potential factorization of the prefix. A sequence that encodes the tuple (m π1 e1 π2 e2 πk ek ) is said to encode the unique factorization of m if (i) π1 π2 πk are primes, (ii) π1 > π2 > · · · > πk > 1, and e e e (iii) m = π1 1 π2 2 · · · πkk . To describe the test, let G be any forecasting algorithm and let (s g(s)) be an n-history. • Let S ∗ be the set of all sequences that encode the factorization of a number. • If s ∈ / S ∗ , then T (s g(s)) = 1. • If s ∈ S ∗ , then the prefix of s corresponds to some number m. Determine the probability, p, that G assigns to s conditional on the prefix being m. • If p ≥ ε, then T (s g(s)) = 1; otherwise T (s g(s)) = 0. Call this the factoring test. It can be implemented in polynomial time since we have efficient algorithms for multiplication and primality testing (see Agrawal, Kayal, and Saxena (2004)). We can make a linear-time test by appropriately padding the sequence. THEOREM 3: For any ε > 0, for sufficiently large m, the factoring test passes the truth with probability 1 − ε and any forecasting algorithm that can ignorantly pass the test with probability 1 − ε can be used to factor m. PROOF: Consider any distribution of nature P and the forecaster F P . Let pm be the probability that P assigns a sequence whose prefix encodes m. Let qm be the probability that a sequence encodes the factorization of m conditional on its prefix encoding m. The factoring test will fail F P if for some m a sequence s ∈ S ∗ with prefix m is realized and qm < ε. The probability of this happening is pm qm < pm ε = ε m : qm <ε
m
Hence, the factoring test will pass the truth with high probability. Now, fix an m and consider the distribution that puts its full weight on the sequence that encodes the unique factorization of m. If the forecaster passes the
THE COMPLEXITY OF FORECAST TESTING
101
test for this distribution, the conditional probability that suffix of the sequence reveals the prime factors of m is at least ε. Use the forecasted probabilities of the suffix as a distribution to generate a sequence. With probability at least ε, we generate the factors of m. If we repeat the process O(1/ε) times, we determine the factors with high probability. Q.E.D. 4.1. Efficiently Sampleable Distributions Theorem 3 makes use of the fact that the test must be passed for all distributions and the distribution given in the proof is one where nature must know how to factor. We can easily modify the proof to nicer distributions, particularly polynomial-time sampleable distributions. A distribution μ is polynomial-time sampleable (see Ben-David, Chor, Goldreich, and Luby (1992)) if there is a polynomial-time computable function f and a polynomial q such that h maps strings of length q(n) to strings of length n, and μ(x) = Prr (h(r) = x), where r is chosen uniformly from strings of length q(|x|). Specifically, we sample the distribution μ by randomly choosing two large primes p and q and output the sequence corresponding to the tuple (pq p q). Using exactly the same test as in Theorem 3, we get the following result. THEOREM 4: For any ε > 0, for sufficiently large n, if we use a distribution that creates factors of length n, the factoring test passes the truth with probability at least 1 − ε, but any forecast that passes the test with distribution μ described above will be able to factor numbers on average by those generated by μ projected on the first coordinate (the value pq). Most cryptographic work based on factoring assumes that factoring numbers even on average is very difficult. This proof can be generalized in a straightforward way to invert any cryptographic one-way function with unique inverses. 5. PSPACE HARDNESS It is natural to suspect that the proof of Theorem 3 should generalize to all NP problems. The obstacle to such a generalization is the absence of unique witnesses. This means we cannot create a test that will pass the truth for all distributions of nature and still force the forecaster to put heavy weight on a witness. Witness reducing techniques as in Valiant and Vazirani (1986) do not appear to help. In this section we take a different approach to create a linear-time test that can be ignorantly passed by a forecaster only if he/she can solve NP-hard and even PSPACE-hard problems by using the theory of interactive proof systems. Once again we assume deterministic forecasters, though the results extend to probabilistic forecasters as well. In complexity theory the class PSPACE is the set of decision problems that can be solved by a deterministic or nondeterministic Turing machine using a
102
L. FORTNOW AND R.V. VOHRA
polynomial amount of memory and unlimited time. They contain the more well known class of NP problems. A logical characterization of PSPACE is that it is the set of problems expressible in second-order logic. A major result of complexity theory is that PSPACE can be characterized as all the languages recognizable by a particular interactive proof system. The hardest problems within the class PSPACE are called PSPACE-complete. An interactive proof system is an abstraction that models computation as an exchange of messages between a prover (P) and a verifier (V). P would like to convince V that a given x belongs to a given set L.7 P has unlimited computational resources, while V’s computational resources are bounded. Messages are passed, in rounds, between P and V until V reaches a conclusion about the correctness of the statement x ∈ L. The parties take turns sending a message to the other party. A strategy for P (or V) is a function that specifies in each round what message is to be sent as a function of the history of messages exchanged in prior rounds. A pair of strategies, sP for P and sV for V, is called a protocol. Membership in L admits an interactive proof system if there is a protocol (sP sV ) with the following two properties: Completeness: If x ∈ L, and both P and V follow the protocol (sP sV ), then V will be convinced that x ∈ L. Soundness: Suppose x ∈ / L. If V follows sV but P is allowed to deviate from sP , V can be convinced that x ∈ L with some small probability. Shamir (1992), building on the work of Lund, Fortnow, Karloff, and Nisan (1992), exhibited an interactive proof system for any PSPACE language L. The proof system has the following properties for an input x of length n. • Let F be a field GF(q) for a prime q exponential in n. • The protocol alternates between the prover (P) giving a polynomial of degree m over F and the verifier (V) choosing a random element of F . Both m and the number of rounds r of the protocol are bounded by a polynomial in n. • V takes all of the messages and decides whether to accept or reject with a deterministic polynomial-time algorithm. • In each round for V there are at most m bad choices from the possible q elements of F that depend only on the previous messages. The other choices we call good. It can be computationally difficult to decide whether a choice is good or bad, but since m q the verifier will pick good choices with very high probability. • If x ∈ / L, then for any strategy of P, V will reject if all of its choices are good. • If x ∈ L, there is a strategy for P that will cause V to accept always. Moreover, if all of V’s choices are good, the messages of P that cause V to accept are unique. 7
More formally, that a given string x belongs to a certain language L.
THE COMPLEXITY OF FORECAST TESTING
103
Fix a PSPACE-complete language L. Consider a sequence of outcomes interpreted as the tuple (x ρ1 v1 ρr vr ), where the ρi ’s are P’s messages and the vi ’s are V’s messages in an interactive proof system for showing x ∈ L. We call the tuple accepting if the verifier would accept if the protocol played out with these messages. Let pi be the probabilities that the realized forecaster predicts for each ρi and let bi be the probabilities for each vi . Given a sequence interpreted as such a tuple and the realized forecasts, our test will declare FAIL if all of the following occur: (i) (x ρ1 v1 ρr vr ) is an accepting tuple. (ii) Each bi is at most √1q . (iii) The product of the ρi ’s is at most n1 . Otherwise the test declares PASS. The test runs in polynomial time and we can pad the sequence so that the test can run in linear time. Call this the PSPACE test. THEOREM 5: For any ε > 0, for sufficiently large n, the PSPACE test passes the truth on inputs of length n with probability at least 1 − ε. A forecaster who can ignorantly pass the PSPACE test with probability at least 1 − ε can solve all inputs of PSPACE problems of length at least n. PROOF: Consider any distribution of nature P and the associated forecaster F P . Let pi ’s and bi ’s be as described above for F P . Let dx be the probability that P puts on choosing a sequence starting with x. By the properties of the proof system, we have that the probability that the test fails is bounded by px (ux + wx ) x
where ux is the probability that a sequence beginning with x fails the test when all the elements are good and wx is the probability that a sequence beginning with x fails the test when some element is bad. The probability wx is bounded by the probability that some element is bad, which is bounded by i
mr mbi ≤ √ = o(1) q
If the vi ’s are good, then there is at most one choice of the ρi ’s that leads to an accepting tuple. So ux ≤
i
pi ≤
1 = o(1) n
104
L. FORTNOW AND R.V. VOHRA
So the probability that the test fails is x
px (ux + wx ) = o(1)
px = o(1)
x
For any x in L we define the following distribution. Fix a strategy for the prover in the interactive proof system for L that guarantees acceptance for all verifier’s messages. For each ρi play that strategy; for each vi pick an element from F uniformly. Output (x ρ1 v1 ρr vr ). Suppose we have a forecaster who passes the test on this distribution with high probability. The first condition of the test is always true. Fix an i. There are q possible vi of which the forecaster can only put weight greater than √1q on √ √ q of those possibilities. So with probability at least 1 − r q/q = 1 − o(1/n), all of the bi ’s will be less than √1q and the second condition of the test will be satisfied. By a similar argument, with high probability all of the vi ’s are good. Since the forecaster passes the test with high probability, then with high probability the product of the pi ’s must be at least 1/n. So if we use the forecaster as a distribution to generate the prover’s responses, the verifier will accept with probability close to 1/n. If we repeat the process O(n) times at least once, the verifier will accept with high probability, whereas if x were not in L, repeating the protocol a polynomial number of times will rarely cause the verifier to accept, no matter what the prover’s strategy. Thus the forecaster gives us a probabilistic algorithm for determining whether x is in L. Q.E.D. 6. CONCLUSION Sandroni (2003) showed that any reasonable forecast tester can be passed by an ignorant forecaster. However his proof requires the forecaster to run in exponential time. We show this exponential blowup is necessary by exhibiting efficient forecast testers that any forecaster ignorant of nature requires to solve PSPACE-complete problems, generally believed to require exponential time. We showed that for efficiently sampleable distributions of nature, an ignorant forecaster would still be required to factor random numbers. It remains open whether one can prove a similar result for NP-hard or PSPACE-hard problems. In the future we would also like to consider what happens when nature is drawn from some smaller set of simpler distributions (see, for example, AlNajjar, Sandroni, Smorodinsky, and Weinstein (2008)). Note that this may change results in two directions: the forecaster need only succeed on these distributions of nature, but also the tester need only pass the truth on that set of distributions as well.
THE COMPLEXITY OF FORECAST TESTING
105
REFERENCES AGRAWAL, M., N. KAYAL, AND N. SAXENA (2004): “PRIMES Is in P,” Annals of Mathematics, 160, 781–793. [100] AL -NAJJAR, N., A. SANDRONI, R. SMORODINSKY, AND J. WEINSTEIN (2008): “Testing Theories With Learnable and Predictive Representations,” Manuscript, Northwestern University. [104] BABAI, L., L. FORTNOW, AND C. LUND (1991): “Nondeterministic Exponential Time Has TwoProver Interactive Protocols,” Computational Complexity, 1, 3–40. BEN-DAVID, S., B. CHOR, O. GOLDREICH, AND M. LUBY (1992): “On the Theory of Average Case Complexity,” Journal of Computer and System Sciences, 44, 193–219. [101] DAWID, A. P. (1982): “The Well Calibrated Bayesian,” Journal of the American Statistical Association, 77, 605–613. [93] (1985): “The Impossibility of Inductive Inference,” Journal of the American Statistical Association, 80, 340–341. [98] DEKEL, E., AND Y. FEINBERG (2006): “Non-Bayesian Testing of a Stochastic Prediction,” Review of Economic Studies, 73, 893–906. [94] FOSTER, D. P., AND R. V. VOHRA (1993): “Asymptotic Calibration,” Biometrika, 85, 379–390. [93] GOOD, I. J. (1952): “Rational Decisions,” Journal of the Royal Statistical Society, Series B, 14, 107–114. [94] LEHRER, E. (2001): “Any Inspection Rule Is Manipulable,” Econometrica, 69, 1333–1347. [93] LUND, C., L. FORTNOW, H. KARLOFF, AND N. NISAN (1992): “Algebraic Methods for Interactive Proof Systems,” Journal of the Association for Computing Machinery, 39, 859–868. [102] OLSZEWSKI, W., AND A. SANDRONI (2009): “A Non-Manipulable Test,” Annals of Statistics (forthcoming). [94] (2008): “Manipulability of Future-Independent Tests,” Econometrica, 76, 1437–1480. [94] SANDRONI, A. (2003): “The Reproducible Properties of Correct Forecasts,” International Journal of Game Theory, 32, 151–159. [93-95,104] SANDRONI, A., R. SMORODINSKY, AND R. VOHRA (2003): “Calibration With Many Checking Rules,” Mathematics of Operations Research, 28, 141–153. [93] SHAMIR, A. (1992): “IP = PSPACE,” Journal of the Association for Computing Machinery, 39, 869–877. [102] VALIANT, L., AND V. VAZIRANI (1986): “NP Is as Easy as Detecting Unique Solutions,” Theoretical Computer Science, 47, 85–93. [95,101] VOVK, V., AND G. SHAFER (2005): “Good Sequential Probability Forecasting Is Always Possible,” Journal of the Royal Statistical Society, Series B, 67, 747–763. [93]
Dept. of Electrical Engineering and Computer Science, Northwestern University, Evanston, IL 60208, U.S.A. and Dept. of Managerial Economics and Decision Sciences, Kellogg Graduate School of Management, Northwestern University, Evanston, IL 60208, U.S.A. Manuscript received May, 2007; final revision received June, 2008.
Econometrica, Vol. 77, No. 1 (January, 2009), 107–133
DECISION THEORY APPLIED TO A LINEAR PANEL DATA MODEL BY GARY CHAMBERLAIN AND MARCELO J. MOREIRA1 This paper applies some general concepts in decision theory to a linear panel data model. A simple version of the model is an autoregression with a separate intercept for each unit in the cross section, with errors that are independent and identically distributed with a normal distribution. There is a parameter of interest γ and a nuisance parameter τ, a N × K matrix, where N is the cross-section sample size. The focus is on dealing with the incidental parameters problem created by a potentially high-dimension nuisance parameter. We adopt a “fixed-effects” approach that seeks to protect against any sequence of incidental parameters. We transform τ to (δ ρ ω), where δ is a J × K matrix of coefficients from the least-squares projection of τ on a N × J matrix x of strictly exogenous variables, ρ is a K × K symmetric, positive semidefinite matrix obtained from the residual sums of squares and cross-products in the projection of τ on x, and ω is a (N − J) × K matrix whose columns are orthogonal and have unit length. The model is invariant under the actions of a group on the sample space and the parameter space, and we find a maximal invariant statistic. The distribution of the maximal invariant statistic does not depend upon ω. There is a unique invariant distribution for ω. We use this invariant distribution as a prior distribution to obtain an integrated likelihood function. It depends upon the observation only through the maximal invariant statistic. We use the maximal invariant statistic to construct a marginal likelihood function, so we can eliminate ω by integration with respect to the invariant prior distribution or by working with the marginal likelihood function. The two approaches coincide. Decision rules based on the invariant distribution for ω have a minimax property. Given a loss function that does not depend upon ω and given a prior distribution for (γ δ ρ), we show how to minimize the average—with respect to the prior distribution for (γ δ ρ)—of the maximum risk, where the maximum is with respect to ω. There is a family of prior distributions for (δ ρ) that leads to a simple closed form for the integrated likelihood function. This integrated likelihood function coincides with the likelihood function for a normal, correlated random-effects model. Under random sampling, the corresponding quasi maximum likelihood estimator is consistent for γ as N → ∞, with a standard limiting distribution. The limit results do not require normality or homoskedasticity (conditional on x) assumptions. KEYWORDS: Autoregression, fixed effects, incidental parameters, invariance, minimax, correlated random effects.
1. INTRODUCTION THIS PAPER APPLIES some general concepts in decision theory to a linear panel data model. An example of the model is an autoregression with a separate intercept for each unit in the cross section, with errors that are independent and identically distributed with a normal distribution. There is a parameter of interest γ and a nuisance parameter τ, an N × K matrix, where N is the crosssection sample size. The focus is on dealing with the incidental parameters problem created by a potentially high-dimension nuisance parameter. 1 We thank two referees and a co-editor for helpful comments. Financial support was provided by the National Science Foundation (SES-0819761).
© 2009 The Econometric Society
DOI: 10.3982/ECTA6869
108
G. CHAMBERLAIN AND M. J. MOREIRA
In our general model, the observation is the realized value of an N × M matrix Y of random variables. We shall be conditioning on the value of an N × J matrix x, which is observed and has rank J. Our model specifies a conditional distribution for Y given x, as a function of the parameter of interest γ and the nuisance parameter τ: (1)
d
Y |x = xa(γ) + τb(γ) + W c(γ)
where τ is N × K, W is N × p, and J + K ≤ N, J + M ≤ N, and M ≤ p. The components of W , conditional on x, are independently and identically distributed N (0 1), which we shall denote by
L(W ) = N (0 IN ⊗ Ip ) The functions a, b, and c are given. (For a random matrix V , the notation L(V ) = N (μ Λ) indicates that the vector formed by joining the rows of V has a multivariate normal distribution with covariance matrix Λ and mean vector formed by joining the rows of the matrix μ.) All distributions throughout the paper are conditional on x. A simple version of our model arises from the reduced form of the autoregression Yit = ψYit−1 + αi + Uit
(i = 1 N; t = 1 T¯ )
where the Uit are independent and identically distributed N (0 σ 2 ). We observe the realized value of the random variable Yit for i = 1 N and t = 1 T¯ . We do not observe Yi0 . The reduced form is Yi1 = ψYi0 + αi + Ui1 Yit = ψt Yi0 + (1 + ψ + · · · + ψt−1 )αi + Uit + ψUit−1 + · · · + ψt−1 Ui1 (t = 2 T¯ ) Conditional on Yi0 = yi0 , we can write this as (2)
Y = τb(γ) + W c(γ)
where γ = (ψ σ), ⎛Y ⎛y Y1T¯ ⎞ 10 11 ⎠ ⎝ ⎝ Y= (3) τ= YN1 YN T¯ yN0 ⎛W W1T¯ ⎞ 11 ⎠ W = ⎝ WN1 WN T¯
α1 ⎞ ⎠ αN
109
DECISION THEORY APPLIED TO PANEL DATA
(4)
¯
ψ ψ2 ψT ¯ 1 (1 + ψ) (1 + ψ + · · · + ψT −1 ) ⎛ 1 ψ ψT¯ −1 ⎞ ¯ ⎜ 0 1 ψT −2 ⎟ ⎜ c(γ) = σ ⎝ ⎟ ⎠ 0 0 1
b(γ) =
and the Wit are independent and identically distributed N (0 1). The observation is the realized value of Y . The parameters are γ and τ. We shall focus on inference for γ, and treat the initial conditions and individual effects in τ as nuisance parameters. We shall try to deal with the large number of incidental parameters in τ that arise when N is large. We shall adopt a “fixed-effects” approach that seeks to protect against any sequence of incidental parameters in τ. There are recent discussions of incidental parameters and panel data in Lancaster (2000, 2002) and Arellano (2003). An alternative analysis could be based on the distribution of (Yi2 YiT¯ ) conditional on the observed value Yi1 = yi1 . This can fit into our framework by removing the first column of Y and including yi1 in the ith row of x. We prefer to work with the full distribution of the observed Y to avoid possible loss of information from conditioning. Now consider a second-order autoregression with time-varying coefficients on the individual effect (a factor model) and time-varying variances for the innovations: (t = 1 T¯ )
Yit = ψ1 Yit−1 + ψ2 Yit−2 + αi ζt + Uit
where Yi0 = yi0 and Yi−1 = yi−1 are not observed, and the Uit are mutually independent with Uit ∼ N (0 σt2 ). With Y and W defined as above, we can write this as ˜ ˜ Y d(ψ) = τb(ψ ζ) + W c(σ) where
⎛1
−ψ1 1 0
−ψ2 −ψ1 1
0 0 0
⎞
⎟ ⎜0 ⎟ ⎜ ⎜ ⎟ y10 y1−1 α1 ⎜0 ⎟ ⎜ ⎟ ⎠ d(ψ) = ⎜ τ = ⎝ ⎟ ⎜ ⎟ ⎜ yN0 yN−1 αN 0 0 −ψ2 ⎟ ⎜0 ⎟ ⎝0 0 0 −ψ1 ⎠ 0 0 0 1 ⎞ ⎛ ψ1 ψ2 0 0 ˜ ˜ = diag(σ1 σT¯ ) b(ψ ζ) = ⎝ ψ2 0 0 0 ⎠ c(σ) ζ1 ζ2 ζ3 ζT¯ ⎛
⎞
110
G. CHAMBERLAIN AND M. J. MOREIRA
We can impose a normalization such as model is
T¯ t=1
ζt2 = 1. The reduced form of the
Y = τb(γ) + W c(γ) with γ = (ψ ζ σ) and ˜ b(γ) = b(ψ ζ)d(ψ)−1
−1 ˜ c(γ) = c(σ)d(ψ)
We can include strictly exogenous variables xit , Yit = xit ξ + ψ1 Yit−1 + ψ2 Yit−2 + αi ζt + Uit where xit and ξ are L × 1 matrices, ⎞ ⎛ x11 · · · x1T¯ ⎜ ⎟ x = ⎝ ⎠ xN1 · · · xN T¯ and conditional on x, the Uit are mutually independent with Uit ∼ N (0 σt2 ). The reduced form of this model is Y = xa(γ) + τb(γ) + W c(γ) ˜ with γ = (ξ ψ ζ σ), a(ξ) = IT¯ ⊗ ξ, and −1 ˜ a(γ) = a(ξ)d(ψ)
˜ b(γ) = b(ψ ζ)d(ψ)−1
−1 ˜ c(γ) = c(σ)d(ψ)
Note that if ψ1 or ψ2 is not equal to zero, then the reduced form has a distributed lag: the conditional expectation of Yit given x depends upon xi1 xit . An alternative model has Yit = xit ξ + αi ζt + Uit where, conditional on x, the vector (Ui1 UiT¯ ) is independent and identically distributed with a multivariate normal distribution: iid
(Ui1 UiT¯ ) ∼ N (0 Λ(χ)) The function Λ is given, and specifies the variances and serial correlations of the errors Uit as a function of the parameter vector χ with fixed dimension. We can write this as ˜ ˜ ˜ Y = xa(ξ) + τb(ψ ζ) + W c(χ) 2 ˜ ˜ where c(χ) is the symmetric square root of Λ(χ): c(χ) = Λ(χ).
DECISION THEORY APPLIED TO PANEL DATA
111
In our general model, the observation is the realized value of an N × M matrix Y of random variables. For example, in a vector autoregression involving the variables Y (1) Y (k) , the ith row of Y could be
Yi1(1) Yi1(k) Yi(1) Yi(k) T¯ T¯
so that M = kT¯ . We only consider linear, complete systems whose reduced forms match equation (1). See Arellano’s (2003, p. 144) discussion of incomplete systems with unspecified feedback processes. The next section shows that the model is invariant under the actions of a group. This group is isomorphic to O(N − J), the group of orthogonal matrices with N − J rows and columns. This isomorphism suggests a canonical form for the model, based on a one-to-one transformation, which simplifies the subsequent analysis. We transform τ to (δ ρ ω), where δ is a J × K matrix of coefficients from the least-squares projection of τ on x, ρ is a K × K symmetric, positive semidefinite matrix obtained from the residual sums of squares and cross-products in the projection of τ on x, and ω is an (N − J) × K matrix whose columns are orthogonal and have unit length. Only ω has a dimension that increases with N. Section 3 finds a maximal invariant statistic. The distribution of the maximal invariant statistic does not depend upon ω. Section 4 obtains the unique, invariant distribution for ω. We use this invariant distribution as a prior distribution to obtain an integrated likelihood function. It depends upon the observation only through the maximal invariant statistic. We use the maximal invariant statistic to construct a marginal likelihood function, so we can eliminate ω by integration with respect to the invariant prior distribution or by working with the marginal likelihood function. The two approaches coincide. Section 5 shows that decision rules based on the invariant distribution for ω have a minimax property. Given a loss function that does not depend upon ω and given a prior distribution for (γ δ ρ), we show how to minimize the average—with respect to the prior distribution for (γ δ ρ)—of the maximum risk, where the maximum is with respect to ω. Section 6 shows that there is a family of prior distributions for (δ ρ) that leads to a simple closed form for the integrated likelihood function. This integrated likelihood function coincides with the likelihood function for a normal, correlated random-effects model. Section 7 develops the example of a simple autoregression and relates our results to the literature. Under random sampling, the quasi maximum likelihood (ML) estimator for the correlated random-effects model is consistent for γ as N → ∞, with a standard limiting distribution. The limit results do not require normality or homoskedasticity (conditional on x) assumptions.
112
G. CHAMBERLAIN AND M. J. MOREIRA
2. MODEL INVARIANCE AND CANONICAL FORM This section shows that the model is invariant under the actions of a group. This group invariance implies a maximal invariant statistic and an invariant prior distribution. Their derivation is simplified by working with a canonical form for the model, based on a one-to-one transformation. We shall briefly describe invariance in the original form of the model and then provide more detail in the canonical form, where most of the subsequent analysis takes place. Let O(N) denote the group of N × N orthogonal matrices (gg = g g = IN ). ˜ is the subgroup of O(N) that preserves x: The group G ˜ = {g˜ ∈ O(N) : gx ˜ = x} G ˜ on the sample space maps y to gy. ˜ Note that The action of G d
˜ |x = xa(γ) + (gτ)b(γ) ˜ gY + W c(γ) ˜ on the parameter space maps ˜ ) = L(W )). The action of G (because L(gW ˜ on the sample ˜ so the model is invariant under the actions of G (γ τ) to (γ gτ), space and the parameter space. ˜ is in fact isoThe canonical form follows from recognizing that the group G morphic to O(N − J). To see this, use the polar decomposition of x to obtain s x=q 0 where q ∈ O(N) and s is the unique symmetric, positive semidefinite square root of x x: s = (x x)1/2 , with ss = x x. (See Golub and Van Loan (1996, p. 149).) The J ×J matrix s is positive definite because x has full column rank J. ˜ = x is equivalent to Then gx s s ˜ (q gq) = 0 0 Let ˜ = q gq
h11 h21
h12 h22
Then we have h11 s = s
⇒
h11 = IJ
h21 s = 0
⇒
h21 = 0
DECISION THEORY APPLIED TO PANEL DATA
113
˜ ∈ O(N) implies that and so q gq IJ 0 ˜ = (5) q gq 0 g ˜ → G by where g ∈ O(N − J). Define G = O(N − J) and define the map ι : G using (5) to map g˜ to g. It is straightforward to check that this map is bijective. In addition, q (g˜ 1 g˜ 2 )q = (q g˜ 1 q)(q g˜ 2 q) IJ 0 IJ 0 IJ = = 0 g1 0 g2 0
0 g 1 g2
so that ι is a group homomorphism: ι(g˜ 1 g˜ 2 ) = g1 g2 = ι(g˜ 1 )ι(g˜ 2 ) ˜ and G are isomorphic. Since ι is a bijective homomorphism, the groups G ˜ Because of this isomorphism, the action of G on the sample space implies an action of G on the sample space: IJ 0 z1 z1 ˜ = q(q gq)q ˜ =q y =q gy z2 gz2 0 g where z = q y. This applies the orthogonal transformation q to y, multiplies the last N − J rows by the orthogonal matrix g, and then uses q to transform back. So we can simplify notation by working with the one-to-one transformation Z ≡ q Y , with d s Z|x = a(γ) + τb(γ) ˜ + W c(γ) 0 and τ˜ = q τ. Let Z = RN×M denote the sample space and partition a point z ∈ Z as z1 z= z2 where z1 is J × M and z2 is (N − J) × M. Then the action of G on the sample space is given by IJ 0 z1 z= m1 : G × Z → Z m1 (g z) = gz2 0 g
114
G. CHAMBERLAIN AND M. J. MOREIRA
We shall abbreviate m1 (g z) = g · z. This defines a group action because for all g1 g2 ∈ G and z ∈ Z , we have e · z = z and (g1 g2 ) · z = g1 · (g2 · z), where e = IN−J is the identity element in G. Partition τ˜ as τ˜ 1 τ˜ = τ˜ 2 where τ˜ 1 is J × K and τ˜ 2 is (N − J) × K. Note that τ˜ 1 d s g · Z|x = (6) b(γ) + W c(γ) a(γ) + 0 gτ˜ 2 (because L(g · W ) = L(W )). It is convenient to define δ = s−1 τ˜ 1 and write (6) as d s 0 b(γ) + W c(γ) π(γ δ) + g · Z|x = 0 gτ˜ 2 where π(γ δ) = a(γ) + δb(γ) Note that the least-squares projection coefficient of τ on x is (x x)−1 x τ = s−1 τ˜ 1 = δ so δ captures a linear relationship between the individual effects τ and x. Let FKN−J denote the set of (N − J) × K matrices whose columns are orthogonal and have unit length:
FKN−J = d ∈ R(N−J)×K : d d = IK (FKN−J is the Stiefel manifold of ordered sets of K orthonormal vectors in RN−J ; see Bishop and Crittenden (1964, p. 137).) The matrix τ˜ 2 has polar decomposition τ˜ 2 = ωρ
ω ∈ FKN−J
ρ = (τ˜ 2 τ˜ 2 )1/2
where ρ is the unique, symmetric positive semidefinite square root of τ˜ 2 τ˜ 2 . Partition q = ( q1 q2 ), where q1 is N × J and q2 is N × (N − J). Note that x = q1 s implies that x(x x)−1 x = q1 q1 and qq = IN implies that q2 q2 = IN − x(x x)−1 x ρ2 = τ˜ 2 τ˜ 2 = τ q2 q2 τ = τ (IN − x(x x)−1 x )τ
DECISION THEORY APPLIED TO PANEL DATA
115
so ρ2 is formed from the residual sums of squares and cross-products in the least-squares projection of τ on x. Then we can write the model in (1) as d s 0 Z|x = (7) π(γ δ) + ρb(γ) + W c(γ) 0 ω
L(W ) = N (0 IN ⊗ Ip ) Let θ = (β ω) denote the parameter, with β = (γ δ ρ). The parameter space is Θ = Θ1 × Θ2
with
Θ2 = FKN−J
(and Θ1 is a subset of some Euclidean space). We shall let Pθ denote the distribution of Z (conditional on x) when the parameter takes on the value θ: L(Z) = Pθ . The action of the group G on the parameter space is given by m2 : G × Θ → Θ
m2 (g θ) = m2 (g (β ω)) = (β gω)
We shall abbreviate m2 (g θ) = g · θ. This defines a group action because for all g1 g2 ∈ G and θ ∈ Θ, we have e · θ = θ and (g1 g2 ) · θ = g1 · (g2 · θ). Then d s 0 π(γ δ) + ρb(γ) + W c(γ) g · Z|x = 0 gω
L(W ) = N (0 IN ⊗ Ip ) and so
L(Z) = Pθ
implies L(g · Z) = Pg·θ
and the model is invariant under the actions of G on the sample space and the parameter space. 3. MAXIMAL INVARIANT STATISTIC A statistic S is a (measurable) function defined on the sample space. S is invariant if S(g · z) = S(z) for all g ∈ G and z ∈ Z . Let PθS denote the distribution of S(Z) when L(Z) = Pθ . If S is an invariant statistic, then for all g ∈ G and θ ∈ Θ, S PθS = L(S(Z)) = L(S(g · Z)) = Pg·θ
The orbit of a point θ ∈ Θ under the action of G is the set {g · θ : g ∈ G}. Note that for any ω1 ω2 ∈ Θ2 , there exists a g ∈ G such that gω1 = ω2 , and hence
116
G. CHAMBERLAIN AND M. J. MOREIRA
for any β ∈ Θ1 , the points (β ω) are in the same orbit for all ω ∈ Θ2 (The action of G on Θ2 , defined by m(g ω) = gω, is transitive.) So the distribution of an invariant statistic does not depend upon ω. Let T (z) = (T1 (z) T2 (z)) = (z1 z2 z2 ) Then T (g · (z1 z2 )) = T (z1 gz2 ) = (z1 z2 g gz2 ) = (z1 z2 z2 ) and so T is an invariant statistic. We shall show that T is a maximal invariant statistic: if S is an invariant statistic, then for any z z˜ ∈ Z , T (z) = T (z˜ ) implies that S(z) = S(˜z). This result is a consequence of the following proposition: PROPOSITION 1: If T (z) = t = (t1 t2 ), then there exists a gz ∈ G such that z = gz · r(t), where ⎛ ⎞ t1 r(t) = ⎝ t21/2 ⎠ ∈ Z 0 PROOF: The matrix z2 can be decomposed as 1/2 (z2 z2 ) where h ∈ O(N − J) z2 = h 0 Set gz = h. Then gz−1 · z =
IJ 0
0 gz−1
z1 z2
⎛
t1
⎞
= ⎝ t21/2 ⎠ = r(t) 0
Q.E.D.
COROLLARY: T is a maximal invariant statistic. ˜ = t, then PROOF: Suppose that S is an invariant statistic. If T (z) = T (z) Proposition 1 implies that ˜ = r(t) gz−1 · z = gz−1 ˜ ·z
with
gz gz˜ ∈ G
Hence z and z˜ are in the same orbit, g · z = z˜
for g = gz˜ gz−1 ∈ G
so S(z) = S(g · z) = S(˜z )
Q.E.D.
The orbit of a point z ∈ Z under the action of G is the set {g · z : g ∈ G}. The maximal invariant T indexes the orbits in the sample space: if T (z1 ) = T (z2 ) =
DECISION THEORY APPLIED TO PANEL DATA
117
t, then z1 and z2 are in the orbit of r(t). The set {r(T (z)) : z ∈ Z } contains one point from each orbit. It is a measurable cross section; see Eaton (1989, p. 58). In the parameter space, for any point β ∈ Θ1 , the points (β ω) are in the same orbit for all ω ∈ Θ2 , so we can fix some point ω0 ∈ Θ2 and then the set {(β ω0 ) : β ∈ Θ1 } contains one point from each orbit in the parameter space. It is a measurable cross section in the parameter space. 4. INVARIANT PRIOR DISTRIBUTION Since G is a compact group, there is a unique invariant distribution μ on G that is Haar measure normalized so that μ(G) = 1. Let U denote a random variable taking on values in G. The invariance property is that
L(U) = μ implies L(gU) = L(Ug) = μ for all g ∈ G We shall refer to the invariant distribution μ as the uniform distribution on G. This invariant distribution on G implies a unique invariant distribution λ on the compact set Θ2 = FKN−J ; see Eaton (1989, Example 2.10, p. 27). This distribution can be obtained from μ by fixing some point ω0 ∈ Θ2 and setting λ = L(Uω0 ), where L(U) = μ. The distribution λ does not depend upon the point ω0 , since if ω1 is some other point in Θ2 , with ω1 = gω0 for some g ∈ G, then
L(Uω1 ) = L(U(gω0 )) = L((Ug)ω0 ) = L(Uω0 ) = λ Let V be a random variable taking on values in Θ2 . Then the invariance property of λ is that
L(V ) = λ implies L(gV ) = L(g(Uω0 )) = L((gU)ω0 ) = L(Uω0 ) = λ for all g ∈ G. We shall refer to the invariant distribution λ as the uniform distribution on Θ2 . Define Ω(γ) = c(γ) c(γ) and assume that c(γ) has full column rank for all β = (γ δ ρ) ∈ Θ1 . Let f (z|β ω) denote the likelihood function, f (z|β ω) = (2π)−NM/2 det(Ω(γ))−N/2 1 −1 × exp − trace[Ω(γ) k(z β ω) k(z β ω)] 2
118
G. CHAMBERLAIN AND M. J. MOREIRA
where s 0 k(z β ω) = z − π(γ δ) − ρb(γ) 0 ω We can use the uniform distribution on Θ2 as a prior distribution to obtain an integrated likelihood function: f (z|β ω)λ(dω) fλ (z|β) = Θ2
The next proposition shows that this integrated likelihood function depends upon z only through the maximal invariant T (z). PROPOSITION 2: For all z ∈ Z and β ∈ Θ1 , fλ (z|β) = fλ (r(T (z))|β) PROOF: Note that for any g ∈ G, IJ 0 s 0 −1 z− k(g · z β ω) = π(γ δ) − ρb(γ) 0 gω 0 g−1 0 IJ k(z β gω) = 0 g−1 and so, for all z ∈ Z and (β ω) ∈ Θ, f (g−1 · z|β ω) = f (z|β gω) (See Eaton (1989, p. 44) for a general discussion of this point.) As in Proposition 1, z = gz · r(T (z)), so f (z|β ω)λ(dω) = f gz · r(T (z))|β ω λ(dω) Θ2
Θ2
f r(T (z))|β gz−1 ω λ(dω)
=
Θ2
f r(T (z))|β ω λ(dω)
= Θ2
Q.E.D.
We can use the maximal invariant statistic T to construct a marginal likelihood function, based on a density for the distribution of T . The next proposition uses Proposition 2 to show that this marginal likelihood function can be obtained from the integrated likelihood function. Let PβT denote the distribution of T (Z) when L(Z) = P(βω) ; the value of ω does not matter since T is
DECISION THEORY APPLIED TO PANEL DATA
119
an invariant statistic. Let ζ denote Lebesgue measure on RN × RM and let ν = ζT −1 denote the measure ν(A) = ζ(T −1 (A)) for (measurable) sets A in a Euclidean space containing T (Z ). Define f T (t|β) = fλ (r(t)|β)
for t ∈ T (Z ) β ∈ Θ1
Proposition 3 shows that f T (t|β) provides a density function for PβT : T Pβ (A) = f T (t|β)ν(dt) A
PROPOSITION 3: fλ (r(t)|β) is a density for PβT with respect to the measure ν. PROOF: For all ω ∈ Θ2 , PβT (A) = P(βω) (T −1 (A)) and so
P(βω) (T −1 (A))λ(dω)
P (A) = T β
Θ2
= Θ2
=
T −1 (A)
T −1 (A)
=
T −1 (A)
f (z|β ω)ζ(dz) λ(dω) f (z|β ω)λ(dω) ζ(dz)
Θ2
fλ r(T (z))|β ζ(dz)
fλ (r(t)|β)ζT −1 (dt)
= A
Q.E.D.
We can eliminate the parameter ω by integration with respect to the invariant prior distribution to obtain the integrated likelihood function fλ (z|β). Alternatively, we can eliminate ω by working with the marginal likelihood function f T (t|β), based on the maximal invariant statistic T . Propositions 2 and 3 show that these likelihood functions coincide: fλ (z|β) = fλ r(T (z))|β = f T (T (z)|β) Having eliminated ω, we can ask whether γ is identified in these likelihood functions. This will depend on the particular specifications for a(γ), b(γ), and
120
G. CHAMBERLAIN AND M. J. MOREIRA
c(γ), and one can examine the following moment conditions based on T (Z): E[(x x)−1 x Y ] = s−1 E(Z1 ) = π(γ δ) E(Y Y ) = E(Z1 Z1 + Z2 Z2 ) = π(γ δ) x xπ(γ δ) + b(γ) ρ2 b(γ) + Nc(γ) c(γ) 5. OPTIMALITY Using the likelihood function of an invariant statistic has the advantage of eliminating dependence on the parameter ω. The concern is that, even using the maximal invariant statistic, we are not using all of the data. This concern can be addressed in our case, since the marginal likelihood function based on T coincides with the integrated likelihood function when we use the invariant prior distribution for ω. Suppose the loss function does not depend upon ω, L : Θ1 × A → R where A is the action space. The corresponding risk function is R((β ω) d) = L(β d(z))f (z|β ω)ζ(dz) Z
where d : Z → A is in the set D of feasible decision rules; D is unrestricted except for regularity conditions. Let η be some prior distribution on Θ1 and consider the average risk with respect to the prior distribution η × λ on Θ: R((β ω) d)λ(dω)η(dβ) R∗ (η × λ d) = Θ1
Θ2
Θ1
Z
=
L(β d(z))fλ (z|β)ζ(dz)η(dβ)
So, choosing d to minimize this average risk function can be based on the integrated likelihood function. Under regularity conditions, we have the standard result that the optimal d is obtained by minimizing posterior expected loss, L(β a)fλ (z|β)η(dβ) d(z) = arg min a∈A
Θ1
= arg min a∈A
L(β a)f T (T (z)|β)η(dβ) Θ1
so we can obtain an optimal decision rule using the marginal likelihood function. This optimal decision rule is a function of the maximal invariant statistic— it depends upon z only through T (z)—but this was not imposed as a constraint
DECISION THEORY APPLIED TO PANEL DATA
121
on D in the optimization. See Eaton (1989, Chapter 6) for a general discussion of invariant decision rules. Suppose that dη×λ minimizes average risk, dη×λ = arg min R∗ (η × λ d) d∈D
and depends upon z only through T (z), dη×λ (z) = d˜η×λ (T (z)) The next proposition establishes a minimax property for this decision rule. The argument is based on Chamberlain (2007, Theorem 6.1). PROPOSITION 4: dη×λ solves the following problem, which combines the average risk and maximum risk criteria: dη×λ = arg min sup R((β ω) d) η(dβ) d∈D
Θ1 ω∈Θ2
PROOF: Let L(Z) = P(βω) . Then R((β ω) dη×λ ) = E L β d˜η×λ (T (Z)) which does not depend upon ω since T is an invariant statistic. So, we can fix ˜ a point ω0 ∈ Θ2 , define R(β dη×λ ) = R((β ω0 ) dη×λ ), and then, for all β ∈ Θ1 ˜ and ω ∈ Θ2 , we have R((β ω) dη×λ ) = R(β dη×λ ). For any d ∈ D , sup R((β ω) d) η(dβ) Θ1 ω∈Θ2
R((β ω) d)λ(dω) η(dβ)
≥ Θ1
Θ2
Θ1
Θ2
≥ = Θ1
=
R((β ω) dη×λ )λ(dω) η(dβ)
˜ R(β dη×λ )η(dβ)
sup R((β ω) dη×λ ) η(dβ)
Θ1 ω∈Θ2
Q.E.D.
The use of minimax here does not eliminate the choice of a prior distribution: the average risk criteria on the parameter space Θ1 for β requires that we specify a prior distribution η. But we can replace the choice of a prior distribution on the parameter space FKN−J for ω by the maximum risk criterion. The
122
G. CHAMBERLAIN AND M. J. MOREIRA
solution to the minimax problem calls for a particular, least favorable, distribution on FKN−J : the uniform distribution λ. This minimax treatment of the incidental parameters can be obtained using the marginal likelihood function f T (·|β) based on the maximal invariant statistic. Recall from Section 2 that δ = (x x)−1 x τ
ρ2 = τ (IN − x(x x)−1 x )τ
Define the set-valued function B(· ·) by
B(δ ρ) = τ ∈ RN×K : (x x)−1 x τ = δ τ (IN − x(x x)−1 x )τ = ρ2 The minimax result in Proposition 4 can be related to the original parametrization by replacing the sup over ω in Θ2 by the sup over τ in B(δ ρ). 6. A CLOSED FORM INTEGRATED LIKELIHOOD AND CORRELATED RANDOM EFFECTS
Our finite sample optimality result uses a prior distribution η for β, where β = (γ δ ρ). This section develops a family of prior distributions for (δ ρ) that leads to a simple, explicit form for the integrated likelihood. The basic idea is that the uniform distribution λ on FKN−J can be combined with a central Wishart distribution to obtain a multivariate normal distribution. We start with a family of prior distributions for ρ that is indexed by a parameter Φ, which is a K × K symmetric, positive semidefinite matrix. Let
L(Q) = N (0 IN−J ⊗ Φ) Let κΦ = L (Q Q)1/2 be the prior distribution for ρ with parameter Φ. Then the corresponding integrated likelihood function is fλκ (z|γ δ Φ) = fλ (z|(γ δ ρ))κΦ (dρ) =
f (z|(γ δ ρ) ω)λ(dω)κΦ (dρ)
Suppose that V is independent of Q Q, with L(V ) = λ. Then L(Q) = L V (Q Q)1/2 ; see Eaton (1989, Example 4.4, p. 61). So the distribution for τ˜ 2 = ωρ implied by λ × κΦ is N (0 IN−J ⊗ Φ). This implies that the log of the integrated likelihood
DECISION THEORY APPLIED TO PANEL DATA
123
function is log[fλκ (z|γ δ Φ)] J NM log(2π) − log det(Ω(γ)) 2 2 N −J − log det[b(γ) Φb(γ) + Ω(γ)] 2 1 − trace Ω(γ)−1 (z1 − sπ(γ δ)) (z1 − sπ(γ δ)) 2 1 − trace [b(γ) Φb(γ) + Ω(γ)]−1 z2 z2 2
=−
Fix a value for the parameter in (7): β∗ = (γ ∗ δ∗ ρ∗ ) ∈ Θ1
ω∗ ∈ Θ2
Let L(Z) = P(β∗ ω∗ ) and define l(γ δ Φ) = E log[fλκ (Z|γ δ Φ)] Note that this expectation does not depend upon the normality assumption for W in (7); only the first and second moments of Z are used, and so l(γ δ Φ) depends upon L(W ) only through its first and second moments. Evaluating E(Z1 ), E(Z1 Z1 ), and E(Z2 Z2 ) gives l(γ δ Φ) J NM log(2π) − log det(Ω(γ)) 2 2 N −J − log det[b(γ) Φb(γ) + Ω(γ)] 2 1 − trace Ω(γ)−1 (π(γ ∗ δ∗ ) − π(γ δ)) x x 2 × (π(γ ∗ δ∗ ) − π(γ δ)) + JΩ(γ)−1 Ω(γ ∗ )
=−
−
1 trace [b(γ) Φb(γ) + Ω(γ)]−1 2
× [b(γ ∗ ) ρ∗ 2 b(γ ∗ ) + (N − J)Ω(γ ∗ )] The maximum of l(γ δ Φ) is attained at γ = γ∗
δ = δ∗
Φ = ρ∗ 2 /(N − J)
124
G. CHAMBERLAIN AND M. J. MOREIRA
This result is useful for obtaining asymptotic properties of the estimator that maximizes the integrated (quasi) log-likelihood function. To make connections between a correlated random-effects model and our fixed-effects approach, it is convenient to introduce a prior distribution for δ, in addition to the prior distribution for ρ that was chosen to obtain a closed form for the integrated likelihood. The family of prior distributions for (δ ρ) is indexed by the parameter (ι Φ), where ι is a J × K matrix and Φ is a K × K symmetric, positive semidefinite matrix. Let Q1 = N (0 IN ⊗ Φ) L Q2 where Q1 is J × K and Q2 is (N − J) × K. Let κιΦ = L ι + s−1 Q1 (Q2 Q2 )1/2 be the prior distribution for (δ ρ). The distribution for ωρ implied by λ × κιΦ is N (0 IN−J ⊗ Φ) (as above) and the distribution for (sπ(γ δ) ωρ) is
N (sπ(γ ι) IJ ⊗ b(γ) Φb(γ)) × N (0 IN−J ⊗ Φ) The corresponding integrated likelihood function is ¯ fλκ (z|γ ι Φ) = fλ (z|(γ δ ρ))κιΦ (dδ dρ) Evaluating the log of this integrated likelihood function gives (8)
log[f¯λκ (z|γ ι Φ)] N NM log(2π) − log det[b(γ) Φb(γ) + Ω(γ)] 2 2 1 − trace [b(γ) Φb(γ) + Ω(γ)]−1 2 × (z1 − sπ(γ ι)) (z1 − sπ(γ ι)) + z2 z2
=−
As above, fix a value (β∗ ω∗ ) for the parameter, let L(Z) = P(β∗ ω∗ ) , and define ¯ ι Φ) = E log[f¯λκ (Z|γ ι Φ)] l(γ As before, this expectation does not depend upon the normality assumption for W in (7); only the first and second moments of Z are used. Evaluating E(Z1 ), E(Z1 Z1 ), and E(Z2 Z2 ) gives (9)
¯ ι Φ) = − NM log(2π) − N log det[b(γ) Φb(γ) + Ω(γ)] l(γ 2 2
DECISION THEORY APPLIED TO PANEL DATA
125
1 trace [b(γ) Φb(γ) + Ω(γ)]−1 2 × (π(γ ∗ δ∗ ) − π(γ ι)) x x(π(γ ∗ δ∗ ) − π(γ ι)) + b(γ ∗ ) ρ∗ 2 b(γ ∗ ) + NΩ(γ ∗ ) −
¯ ι Φ) is attained at The maximum of l(γ γ = γ∗
ι = δ∗
Φ = ρ∗ 2 /N
Consider the following correlated random-effects specification for the incidental parameters: (10)
d
τ|x = N (xι IN ⊗ Φ)
Combining this with the model in (1), the implied distribution for the observation is d Y |x = N xπ(γ ι) IN ⊗ [b(γ) Φb(γ) + Ω(γ)] (11) and the log-likelihood function is (12)
log[f re (y|γ ι Φ)] NM N log(2π) − log det[b(γ) Φb(γ) + Ω(γ)] 2 2 1 − trace [b(γ) Φb(γ) + Ω(γ)]−1 2 × (y − xπ(γ ι)) (y − xπ(γ ι))
=−
We shall refer to this as the normal, correlated random-effects model. As in Section 2, let s x=q z = q y 0 where q is an N × N orthogonal matrix; then we can write the log-likelihood function as (13)
log[f re (qz|γ ι Φ)] NM N log(2π) − log det[b(γ) Φb(γ) + Ω(γ)] 2 2 1 − trace [b(γ) Φb(γ) + Ω(γ)]−1 2
=−
126
G. CHAMBERLAIN AND M. J. MOREIRA
× (z1 − sπ(γ ι)) (z1 − sπ(γ ι)) + z2 z2 = log[f¯λκ (z|γ ι Φ)] So, the log of the normal, correlated random-effects likelihood function coincides with the log of the integrated likelihood function in equation (8). This connection with a correlated random-effects model helps to relate our results to the literature on panel data. 7. CONNECTIONS WITH THE LITERATURE A simple version of our model arises from the reduced form of the autoregression (14)
(i = 1 N; t = 1 T¯ )
Yit = ψYit−1 +αi +Uit
where the Uit are independent and identically distributed N (0 σ 2 ). We observe the realized value of the random variable Yit for i = 1 N and t = 1 T¯ . We do not observe Yi0 . This specification implies a likelihood function for {Yi1 YiT¯ }Ni=1 conditional on {yi0 αi }Ni=1 . Our framework allows for conditioning on time-varying covariates xit , but in this simple version we shall just use x = 1N , where 1N denotes an N × 1 matrix of ones. To obtain our canonical form, use an N√× N orthogonal matrix q whose first column is proportional to 1N : q = ( 1N / N q2 ) so that √ N x=q 0 Note that qq = IN implies that q2 q2 = IN − 1N 1N /N. Then our transformation of the N × T¯ matrix Y is √ N Y¯ Z = q Y = where q2 Y N N Y¯ = Yi1 /N · · · YiT¯ /N i=1
i=1
Our transformation of the parameters uses √ N τ¯ τ˜ = q τ = τ˜ 2 with τ˜ 2 = q2 τ and (15)
¯ (τ − 1N τ) ¯ τ˜ 2 τ˜ 2 = τ q2 q2 τ = (τ − 1N τ)
127
DECISION THEORY APPLIED TO PANEL DATA
⎛
⎞ N ¯ (y − y )(α − α) ¯ i0 0 i ⎜ ⎟ ⎜ ⎟ i=1 i=1 ⎜ ⎟ =⎜ N ⎟ N ⎝ ⎠ 2 (yi0 − y¯0 )(αi − α) ¯ (αi − α) ¯ N (yi0 − y¯0 )2
i=1
Then
δ = τ¯ =
N i=1
i=1
yi0 /N
N
αi /N
ρ = (τ˜ 2 τ˜ 2 )1/2
i=1
The maximal invariant statistic T (Z) = (T1 (Z) T2 (Z)) has √ T1 (Z) = N Y¯ T2 (Z) = (Y − 1N Y¯ ) (Y − 1N Y¯ ) The distribution of this statistic depends only upon (ψ σ δ ρ), which has dimension 7; the distribution does not depend upon ω, which has dimension 2N − 5. Let γ = (ψ σ). The distribution of the maximal invariant statistic has density f T (·|γ δ ρ), which is based on a normal distribution for Y¯ and an independent noncentral Wishart distribution for (Y − 1N Y¯ ) (Y − 1N Y¯ ). Our optimality result requires a prior distribution for (ψ σ δ ρ). The dimension reduction shows up in not requiring a prior distribution for ω—that is, where the minimax result is used. There is a particular family of prior distributions for (δ ρ) that connects to the literature on random-effects models. The family is indexed by the parameter (ι Φ), where ι is 1 × 2 and Φ is a 2 × 2 symmetric, positive semidefinite matrix. Let Q1 = N (0 IN ⊗ Φ) L Q2 where Q1 is 1 × 2 and Q2 is (N − 1) × 2. The prior for (δ ρ) is κιΦ = L ι + N −1/2 Q1 (Q2 Q2 )1/2 Combining f T (T (z)|γ δ ρ) with this family of prior distributions for (δ ρ) gives the integrated likelihood f¯λκ (z|γ ι Φ), as in equation (8) in Section 6. Now consider the following normal random-effects model: the specification in (14) plus (16)
iid
(yi0 αi ) ∼ N ((ι1 ι2 ) Φ)
(i = 1 N)
The likelihood function is f re (y|γ ι Φ), as in equation (12) in Section 6. Using the transformation z = q y, our result in (13) shows that f re (qz|γ ι Φ) = f¯λκ (z|γ ι Φ)
128
G. CHAMBERLAIN AND M. J. MOREIRA
A prior distribution for (ψ σ ι Φ), of dimension 7, is needed to obtain minimum average risk in finite samples. We do not have a specific recommendation for this prior. Our point is that the normal random-effects likelihood function can be obtained from the likelihood function for the maximal invariant, in which the dimension of the parameter space has already been reduced to a number (7) that does not depend upon N. In this sense, the incidental parameter problem has been dealt with in the original fixed-effects model in (14), which conditions on {yi0 αi }Ni=1 , without relying on the specification of a random-effects distribution in (16). Our paper has a finite sample perspective, but in connecting with the literature, we shall consider limits as N → ∞ in the context of our general model. In equation (9) of Section 6, we fix a “true value” θ∗ = (γ ∗ δ∗ ρ∗ ω∗ ) for the parameter in equation (7), and evaluate ¯ ι Φ) = E log[f¯λκ (Z|γ ι Φ)] l(γ with Z distributed according to Pθ∗ . This expectation does not depend upon the ¯ ι Φ) is attained normality assumption for W in (7), and the maximum of l(γ ∗ ∗ ∗2 at γ = γ , ι = δ , and Φ = ρ /N. If there is a unique maximizing value for γ, it should be feasible to go from here to a consistency result for γ ∗ (as N → ∞), without the normality assumption for W . In fact, if we add an assumption of random sampling over the cross-section dimension i, then the asymptotics are straightforward and well known. Let Y(i) , x(i) , and τ(i) denote the ith rows of Y , x, and τ. Assume that (i = 1 N) Y(i) x(i) τ(i) are independent and identically distributed from a joint distribution F , and let EF denote expectation with respect to this distribution. We shall assume that the (unconditional) second moments of (Y(i) x(i) ) correspond to the normal, correlated random-effects model, but we shall not make normality or homoskedasticity (conditional on x) assumptions in obtaining the limit distribution of the estimator. We shall refer to this semiparametric model simply as the correlated random-effects model. Assume that EF x(i) Y(i) − x(i) π(γ ∗ ι∗ ) = 0 so that x(i) π(γ ∗ ι∗ ) is the minimum mean-square-error linear predictor of Y(i) given x(i) . Define ε(i) = Y(i) − x(i) π(γ ∗ ι∗ ) and assume that EF ε(i) ε(i) = b(γ ∗ ) Φ∗ b(γ ∗ ) + Ω(γ ∗ )
DECISION THEORY APPLIED TO PANEL DATA
129
Let υ denote the column vector formed from γ, ι, and the lower triangle of Φ, and let 1 h Y(i) x(i) υ = − log det[b(γ) Φb(γ) + Ω(γ)] 2 1 − trace [b(γ) Φb(γ) + Ω(γ)]−1 2 × Y(i) − x(i) π(γ ι) Y(i) − x(i) π(γ ι) Then it is straightforward to show that max EF h Y(1) x(1) υ υ
is attained at υ∗ , which is formed from the distinct elements of (γ ∗ ι∗ Φ∗ ). The quasi ML estimator is υˆ N = arg max υ
N 1 h Y(i) x(i) υ N i=1
Standard method-of-moments arguments, as in Hansen (1982), MaCurdy (1982), and White (1982), provide regularity conditions under which υˆ N has a limiting normal distribution as N → ∞ (with J, K, and M fixed): √ d N(υˆ N − υ∗ ) → N (0 Λ∗ ) where
∂2 h Y(1) x(1) υ∗ −1 Λ = EF ∂υ ∂υ ∂h Y(1) x(1) υ∗ ∂h Y(1) x(1) υ∗ × EF ∂υ ∂υ ∂2 h Y(1) x(1) υ∗ −1 × EF ∂υ ∂υ
∗
Since f¯λκ (q y|γ ι Φ) equals f re (y|γ ι Φ), this limit distribution result applies to a quasi maximum likelihood estimator based on the integrated likelihood f¯λκ from Section 6. The quasi maximum likelihood estimator is asymptotically equivalent to a minimum distance estimator that imposes the restrictions on the second moments. An optimal minimum distance estimator uses a weight matrix based on the covariance matrix of the sample second moments. The minimum distance
130
G. CHAMBERLAIN AND M. J. MOREIRA
estimator that corresponds to quasi ML uses a weight matrix that would be optimal under normality but not in general. See Chamberlain (1984, Section 4.4) and Arellano (2003, Sections 5.4.3 and 7.4.2). Returning to the example in equation (14), we can use the reduced form from equations (2)–(4) in Section 1 to calculate moment conditions based on T (Z), conditional on {yi0 αi }Ni=1 . This gives ¯ E(Y¯ ) = N −1/2 E(Z1 ) = τb(γ) E(Y Y ) = E(Z1 Z1 + Z2 Z2 ) = b(γ) (N τ¯ τ¯ + τ˜ 2 τ˜ 2 )b(γ) + Nc(γ) c(γ) where γ = (ψ σ), τ˜ τ˜ = ρ2 is displayed in (15), and b(γ) and c(γ) are displayed in (4). These moments can be used to examine the identification of ψ and σ, treating τ¯ and τ˜ 2 τ˜ 2 as unrestricted. Now add the normal random-effects specification in (16), and calculate moment conditions without conditioning on {yi0 αi }Ni=1 : E(Y¯ ) = ιb(γ) E(Y Y ) = b(γ) (Nι ι + NΦ)b(γ) + Nc(γ) c(γ) Working with these moments leads to the same identification analysis for γ, because ι and Φ are unrestricted. Bhargava and Sargan (1983) considered maximum likelihood estimation in a model with lagged dependent variables and strictly exogenous variables. They used a normal, correlated random-effects model for the initial conditions. Their model is discussed by Arellano (2003, Sections 7.4.1 and 7.4.2). Arellano (2003, Section 7.4.3) considered a normal, correlated random-effects specification for the individual effects in the Bhargava–Sargan model. Chamberlain (1980, pp. 234–235) and Blundell and Smith (1991) considered maximum likelihood estimation, conditional on the first observation, in normal, correlated random-effects models. Alvarez and Arellano (2003, Section 3.5) obtained limiting results for inference in these models as N and T¯ tend to infinity.2 Alvarez and Arellano (2004) considered quasi maximum likelihood estimators in correlated random-effects models, with a stress on allowing for time-series heteroskedasticity. Lancaster (2002) dealt with incidental parameters by first reparametrizing so that the information matrix is block diagonal, with the common parameters in one block and the incidental parameters in the other. In his application to a nonstationary dynamic regression model (Lancaster (2002, p. 653)), the parameter space for the reparametrized incidental parameters is RN . Then he 2 Hahn and Kuersteiner (2002) considered maximum likelihood estimation in fixed-effects models, and obtained bias corrections as N and T¯ tend to infinity.
DECISION THEORY APPLIED TO PANEL DATA
131
formed an integrated likelihood function, integrating with respect to Lebesgue measure on RN . He showed that maximizing this integrated likelihood function provides a consistent estimator of the common parameters. Note that the information matrix block diagonality would be preserved by a smooth bijective transformation of the incidental parameters, so the use of Lebesgue measure does not by itself provide a unique prior measure. Our approach is similar in that it uses an integrated likelihood function. The prior measure, however, is different. Our reparametrization is motivated by the invariance of the model under the actions of the orthogonal group, and this determines a unique invariant distribution for ω on the compact space FKN−J . This distribution is least favorable in our minimax optimality result. Another difference is that the use of Lebesgue measure on RN for a prior measure does not correspond to the normal, correlated random-effects model. It amounts to specifying that the (reparametrized) individual effects have very large variances, instead of treating the individual effects as draws from a distribution whose variance is a parameter to be estimated. Sims (2000) used a likelihood perspective in his analysis of dynamic panel data models. He dealt with incidental parameters by treating the individual effects and initial conditions as draws from a bivariate normal distribution (Sims (2000, p. 454)). Our approach has a different starting point, since our model treats the individual effects and initial conditions as parameters (fixed effects). But our minimax optimality argument calls for a particular least favorable distribution for ω. We have seen that this unique distribution can be combined with a particular family of prior distributions for (δ ρ) to obtain a normal, correlated random-effects model, which corresponds to Sims’s specification. 8. CONCLUSION We started with a fixed-effects model. After reparametrizing, only the parameter ω has dimension depending on the cross-section sample size N. The model is invariant under the actions of the orthogonal group, and we obtained a maximal invariant statistic, T , whose distribution does not depend upon ω. So we can solve the incidental parameters problem by working with a marginal likelihood, based on the sampling distribution of T . This approach has a finite sample, minimax optimality. The argument is based on expressing the marginal likelihood as an integrated likelihood for a particular prior distribution for ω. The prior distribution is the unique, invariant distribution under the group action on that part of the parameter space. In addition to ω, the nuisance parameter consists of (δ ρ), whose dimension does not depend upon N. A convenient way to implement our approach is to use a particular family of prior distributions for (δ ρ), indexed by the parameter (ι Φ). This leads to an integrated likelihood function with a closed form expression. It is a function of (γ ι Φ), where γ is the original parameter of interest, which is not affected by the reparametrization. It turns out that
132
G. CHAMBERLAIN AND M. J. MOREIRA
this integrated likelihood function coincides with the likelihood function for a normal, correlated random-effects model. So our finite sample optimality arguments take us from the initial fixedeffects model to a normal, correlated random-effects model. The normal distribution for the effects is not part of our model in equations (1) and (7); the model only specifies a normal distribution for the errors. The normal distribution for the effects arises from two sources: the unique uniform distribution for ω on the compact manifold FKN−J , whose dimension depends upon N, and the convenient choice of prior distribution for (δ ρ), whose dimension does not depend upon N. The first source is motivated by our invariance and minimax arguments. The second source lacks this motivation, but since the dimension of (δ ρ) does not depend upon N, the particular choice made here may not be so important when N is large. In fact, using the integrated likelihood function as a quasi likelihood, the large N asymptotics of the quasi ML estimator are covered by standard arguments, under random sampling. These large N arguments do not require the assumption of normal errors in (1) and (7). So one way to view our finite sample results is that, starting with a fixedeffects model, they provide motivation for a normal, correlated random-effects model. At that point, robustness concerns can lead to dropping the normality assumption. Our quasi ML estimator can still provide the basis for large N inference, but it would not be (semiparametric) efficient, so one may prefer to use a different weighting scheme for the moment restrictions implied by the correlated random-effects model. This leads to standard optimal minimum distance and generalized method of moments estimators for the correlated random-effects model. REFERENCES ALVAREZ, J., AND M. ARELLANO (2003): “The Time Series and Cross-Section Asymptotics of Dynamic Panel Data Estimators,” Econometrica, 71, 1121–1159. [130] (2004): “Robust Likelihood Estimation of Dynamic Panel Data Models,” Unpublished Manuscript, CEMFI. [130] ARELLANO, M. (2003): Panel Data Econometrics. Oxford, U.K.: Oxford University Press. [109, 111,130] BHARGAVA, A., AND J. D. SARGAN (1983): “Estimating Dynamic Random Effects Models From Panel Data Covering Short Time Periods,” Econometrica, 51, 1635–1659. [130] BISHOP, R., AND R. CRITTENDEN (1964): Geometry of Manifolds. New York: Academic Press. Reprinted with corrections, 2001. Providence, RI: AMS Chelsea Publishing. [114] BLUNDELL, R., AND R. SMITH (1991): “Initial Conditions and Efficient Estimation in Dynamic Panel Data Models: An Application to Company Investment Behaviour,” Annales d’Économie et de Statistique, 20/21, 109–123. [130] CHAMBERLAIN, G. (1980): “Analysis of Covariance With Qualitative Data,” The Review of Economic Studies, 47, 225–238. [130] (1984): “Panel Data,” in Handbook of Econometrics, Volume II, ed. by Z. Griliches and M. Intriligator. Amsterdam: North-Holland, 1247–1318. [130] (2007): “Decision Theory Applied to an Instrumental Variables Model,” Econometrica, 75, 609–652. [121]
DECISION THEORY APPLIED TO PANEL DATA
133
EATON, M. (1989): Group Invariance Applications in Statistics, Regional Conference Series in Probability and Statistics, Vol. 1. Hayward, CA: Institute of Mathematical Statistics. [117,118, 121,122] GOLUB, G., AND C. VAN LOAN (1996): Matrix Computations (Third Ed.). Baltimore, MD: The Johns Hopkins University Press. [112] HAHN, J., AND G. KUERSTEINER (2002): “Asymptotically Unbiased Inference for a Dynamic Panel Model With Fixed Effects When Both n and T Are Large,” Econometrica, 70, 1639–1657. [130] HANSEN, L. P. (1982): “Large Sample Properties of Generalized Method of Moments Estimators,” Econometrica, 50, 1029–1054. [129] LANCASTER, T. (2000): “The Incidental Parameter Problem Since 1948,” Journal of Econometrics, 95, 391–413. [109] (2002): “Orthogonal Parameters and Panel Data,” Review of Economic Studies, 69, 647–666. [109,130] MACURDY, T. (1982): “The Use of Time Series Processes to Model the Error Structure of Earnings in a Longitudinal Data Analysis,” Journal of Econometrics, 18, 83–114. [129] SIMS, C. (2000): “Using a Likelihood Perspective to Sharpen Econometric Discourse: Three Examples,” Journal of Econometrics, 95, 443–462. [131] WHITE, H. (1982): “Maximum Likelihood Estimation of Misspecified Models,” Econometrica, 50, 1–25. [129]
Dept. of Economics, Harvard University, Cambridge, MA 02138, U.S.A.; [email protected] and Dept. of Economics, Harvard University, Cambridge, MA 02138, U.S.A. and FGV/EPGE, Rio de Janeiro, RJ 22250, Brazil; [email protected]. Manuscript received December, 2006; final revision received November, 2007.
Econometrica, Vol. 77, No. 1 (January, 2009), 135–175
NONPARAMETRIC IDENTIFICATION OF FINITE MIXTURE MODELS OF DYNAMIC DISCRETE CHOICES BY HIROYUKI KASAHARA AND KATSUMI SHIMOTSU1 In dynamic discrete choice analysis, controlling for unobserved heterogeneity is an important issue, and finite mixture models provide flexible ways to account for it. This paper studies nonparametric identifiability of type probabilities and type-specific component distributions in finite mixture models of dynamic discrete choices. We derive sufficient conditions for nonparametric identification for various finite mixture models of dynamic discrete choices used in applied work under different assumptions on the Markov property, stationarity, and type-invariance in the transition process. Three elements emerge as the important determinants of identification: the time-dimension of panel data, the number of values the covariates can take, and the heterogeneity of the response of different types to changes in the covariates. For example, in a simple case where the transition function is type-invariant, a time-dimension of T = 3 is sufficient for identification, provided that the number of values the covariates can take is no smaller than the number of types and that the changes in the covariates induce sufficiently heterogeneous variations in the choice probabilities across types. Identification is achieved even when state dependence is present if a model is stationary first-order Markovian and the panel has a moderate time-dimension (T ≥ 6). KEYWORDS: Dynamic discrete choice models, finite mixture, nonparametric identification, panel data, unobserved heterogeneity.
1. INTRODUCTION IN DYNAMIC DISCRETE CHOICE ANALYSIS, controlling for unobserved heterogeneity is an important issue. Finite mixture models, which are commonly used in empirical analyses, provide flexible ways to account for it. To date, however, the conditions under which finite mixture dynamic discrete choice models are nonparametrically identified are not well understood. This paper studies nonparametric identifiability of finite mixture models of dynamic discrete choices when a researcher has access to panel data. Finite mixtures have been used in numerous applications, especially in estimating dynamic models. In empirical industrial organization, Crawford and Shum (2005) used finite mixtures to control for patient-level unobserved heterogeneity in estimating a dynamic matching model of pharmaceutical demand. Gowrisankaran, Mitchell, and Moro (2005) estimated a dynamic model of voter behavior with finite mixtures. In labor economics, finite mixtures are a popular choice for controlling for unobserved person-specific effects when 1 The authors are grateful to the co-editor and three anonymous referees whose comments greatly improved the paper. The authors thank Victor Aguirregabiria, David Byrne, Seung Hyun Hong, Hidehiko Ichimura, Thierry Magnac, and the seminar participants at Hitotsubashi University, University of Tokyo, University of Toronto, New York Camp Econometrics II, and 2006 JEA Spring Meeting for helpful comments. The financial support from SSHRC is gratefully acknowledged.
© 2009 The Econometric Society
DOI: 10.3982/ECTA6763
136
H. KASAHARA AND K. SHIMOTSU
dynamic discrete choice models are estimated (e.g., Keane and Wolpin (1997), Cameron and Heckman (1998)). Heckman and Singer (1984) used finite mixtures to approximate more general mixture models in the context of duration models with unobserved heterogeneity. In most applications of finite mixture models, the components of the mixture distribution are assumed to belong to a parametric family. The nonparametric maximum likelihood estimator (NPMLE) of Heckman and Singer (1984) treats the distribution of unobservables nonparametrically but assumes parametric component distributions. Most existing theoretical work on identification of finite mixture models either treats component distributions parametrically or uses training data that are from known component distributions (e.g., Titterington, Smith, and Makov (1985), Rao (1992)). As Hall and Zhou (2003) stated, “very little is known of the potential for consistent nonparametric inference in mixtures without training data.” This paper studies nonparametric identifiability of type probabilities and type-specific component distributions in finite mixture dynamic discrete choice models. Specifically, we assess the identifiability of type probabilities and typespecific component distributions when no parametric assumption is imposed on them. Our point of departure is the work of Hall and Zhou (2003), who proved nonparametric identifiability of two-type mixture models with independent marginals: (1)
F(y) = π
T
Ft1 (yt ) + (1 − π)
t=1
T
Ft2 (yt )
t=1
where F(y) is the distribution function of a T -dimensional variable Y , and j Ft (yt ) is the distribution function of the tth element of Y conditional on type j. Hall and Zhou showed that the type probability π and the type-specific compoj nents Ft are nonparametrically identifiable from F(y) and its marginals when T ≥ 3, while they are not when T = 2. The intuition behind their result is as follows. Integrating out different elements of y from (1) gives lower-dimensional submodels, (2)
F yi1 yi2 yil = π
l s=1
l F yis + (1 − π) Fi2s yis 1 is
s=1
where 1 ≤ l ≤ T , 1 ≤ i1 < · · · < il ≤ T , and F(yi1 yi2 yil ) is the l-variate marginal distribution of F(y). Each lower-dimensional submodel implies a difj ferent restriction on the unknown elements, that is, π and the Ft ’s. F and T its marginals imply 2 − 1 restrictions, while there are 2T + 1 unknown elements. When T = 3, the number of restrictions is the same as the number of unknowns, and one can solve these restrictions to uniquely determine π and j the Ft ’s.
MODELS OF DYNAMIC DISCRETE CHOICES
137
While Hall and Zhou’s analysis provides the insight that lower-dimensional submodels (2) provide important restrictions for identification, it has limited applicability to the finite mixture models of dynamic discrete choices in economic applications. First, it is difficult to generalize their analysis to three or more types.2 Second, their model (1) does not have any covariates, while most empirical models in economics involve covariates. Third, the assumption that elements of y are independent in (1) is not realistic in dynamic discrete choice models. This paper provides sufficient conditions for nonparametric identification for various finite mixture models of dynamic discrete choices used in applied work. Three elements emerge as the important determinants of identification: the time-dimension of panel data, the number of the values the covariates can take, and the heterogeneity of the response of different types to changes in the covariates. For example, in a simple case where the transition function is type-invariant, a time-dimension of T = 3 is sufficient for identification, provided that the number of values the covariates can take is no smaller than the number of types and that the changes in the covariates induce sufficiently heterogeneous variations in the choice probabilities across types. The key insight is that, in models with covariates, different sequences of covariates imply different identifying restrictions in the lower-dimensional submodels; in fact, if d is the number of support points of the covariates and T is the time-dimension, then the number of restrictions becomes on the order of d T . As a result, the presence of covariates provides a powerful source of identification in panel data even with a moderate time-dimension T . We study a variety of finite mixture dynamic discrete choice models under different assumptions on the Markov property, stationarity, and typeinvariance in the transition process. Under a type-invariant transition function and conditional independence, we analyze the nonstationary case that conditional choice probabilities change over time because time-specific aggregate shocks are present or agents are finitely lived. We also examine the case where state dependence is present (for instance, when the lagged choice affects the current choice and/or the transition function of state variables is different across types), and show that identification is possible when a model is stationary first-order Markovian and the panel has a moderate time-dimension T ≥ 6. This result is important since distinguishing unobserved heterogeneity and state dependence often motivates the use of finite mixture models in empirical studies. On the other hand, our approach has a limitation in that it does not simultaneously allow for both state dependence and nonstationarity. 2
When the number of types, M, is more than three, Hall, Neeman, Pakyari, and Elmore (2005) showed that for any number of types, M, there exists TM such that type probabilities and typespecific component distributions are nonparametrically identifiable when T ≥ TM , and that TM is no larger than (1 + o(1))6M ln(M) as M increases. However, such a TM is too large for typical panel data sets.
138
H. KASAHARA AND K. SHIMOTSU
We also study nonparametric identifiability of the number of types, M. Under the assumptions on the Markov property, stationarity, and type-invariance used in this paper, we show that the lower bound of M is identifiable and, furthermore, M itself is identified if the changes in covariates provide sufficient variation in the choice probabilities across types. Nonparametric identification and estimation of finite mixture dynamic discrete choice models are relevant and useful in practical applications for, at least, the following reasons. First, choosing a parametric family for the component distributions is often difficult because of a lack of guidance from economic theory; nonparametric estimation provides a flexible way to reveal the structure hidden in the data. Furthermore, even when theory offers guidance, comparing parametric and nonparametric estimates allows us to examine the validity of the restrictions imposed by the underlying theoretical model. Second, analyzing nonparametric identification helps us understand the identification of parametric or semiparametric finite mixture models of dynamic discrete choices. Understanding identification is not a simple task for finite mixture models even with parametric component distributions, and formal identification analysis is rarely provided in empirical applications. Once type probabilities and component distributions are nonparametrically identified, the identification analysis of parametric finite mixture models often becomes transparent as it is reduced to the analysis of models without unobserved heterogeneity. As we demonstrate through examples, our nonparametric identification results can be applied to check the identifiability of some parametric finite mixture models. Third, the identification results of this paper will open the door to applying semiparametric estimators for structural dynamic models to models with unobserved heterogeneity. Recently, by building on the seminal work by Hotz and Miller (1993), computationally attractive semiparametric estimators for structural dynamic models have been developed (Aguirregabiria and Mira (2002), Kasahara and Shimotsu (2008a)), and a number of papers in empirical industrial organization have proposed two-/multistep estimators for dynamic games (e.g., Bajari, Benkard, and Levin (2007), Pakes, Ostrovsky, and Berry (2007), Pesendorfer and Schmidt-Dengler (2008), Bajari and Hong (2006), and Aguirregabiria and Mira (2007)). To date, however, few of these semiparametric estimators have been extended to accommodate unobserved heterogeneity. This is because these estimators often require an initial nonparametric consistent estimate of type-specific component distributions, but it has not been known whether one can obtain a consistent nonparametric estimate in finite mixture models.3 The identification results of this paper provide an apparatus 3 It is believed that it is not possible to obtain a consistent estimate of choice probabilities. For instance, Aguirregabiria and Mira (2007) proposed a pseudo maximum likelihood estimation algorithm for models with unobserved heterogeneity, but stated that (p. 15) “for [models with unobservable market characteristics] it is not possible to obtain consistent nonparametric
MODELS OF DYNAMIC DISCRETE CHOICES
139
that enables researchers to apply these semiparametric estimators to the models with unobserved heterogeneity. This is important since it is often crucial to control for unobserved heterogeneity in dynamic models (see Aguirregabiria and Mira (2007)). In a closely related paper, Kitamura (2004) examined nonparametric identifiability of finite mixture models with covariates. Our paper shares his insight that the variation in covariates may provide a source of identification; however, the setting as well as the issues we consider are different from Kitamura’s. We study discrete choice models in a dynamic setting with panel data, while Kitamura considered regression models with continuous dependent variables with cross-sectional data. We address various issues specific to dynamic discrete choice models, including identification in the presence of state dependence and type-dependent transition probabilities for endogenous explanatory variables. Our work provides yet another angle for analysis that relates current and previous work on dynamic discrete choice models. Honoré and Tamer (2006) studied identification of dynamic discrete choice models, including the initial conditions problem, and suggested methods to calculate the identified sets. Rust (1994), Magnac and Thesmar (2002), and Aguirregabiria (2006) studied the identification of structural dynamic discrete choice models. Our analysis is also related to an extensive literature on identification of duration models (e.g., Elbers and Ridder (1982), Heckman and Singer (1984), Ridder (1990), and Van den Berg (2001)). The rest of the paper is organized as follows. Section 2 discusses our approach to identification and provides the identification results using a simple “baseline” model. Section 3 extends the identification analysis of Section 2, and studies a variety of finite mixture dynamic discrete choice models. Section 4 concludes. The proofs are collected in the Appendix. 2. NONPARAMETRIC IDENTIFICATION OF FINITE MIXTURE MODELS OF DYNAMIC DISCRETE CHOICES
Every period, each individual makes a choice at from the discrete and finite set A, conditioning on (xt xt−1 at−1 ) ∈ X × X × A, where xt is observable individual characteristics that may change over time and the lagged choice at−1 is included as one of the conditioning variables. Each individual belongs to one of M types, and his/her type attribute is unknown. The probability of belonging to type m is π m , where the π m ’s are positive and sum to 1. Throughout this paper, we impose a first-order Markov property on the conditional choice probability of at and denote type m’s conditional choice probability by P m (at |xt xt−1 at−1 ). The initial distribution of (x1 a1 ) and the transition probability function of xt are also different across types. For each type estimates of [choice probabilities].” Furthermore, Geweke and Keane (2001, p. 3490) wrote that “the [Hotz and Miller] methods cannot accommodate unobserved state variables.”
140
H. KASAHARA AND K. SHIMOTSU
m, we denote them by p∗m (x1 a1 ) and ftm (xt |{xτ aτ }t−1 τ=1 ), respectively. With a slight abuse of notation, we let p∗m (x1 a1 ) and ftm (xt |{xτ aτ }t−1 τ=1 ) denote the density of the continuously distributed elements of xt and the probability mass function of the discretely distributed elements of xt . Suppose we have a panel data set with time-dimension equal to T . Each individual observation, wi = {ait xit }Tt=1 , is drawn randomly from an M-term mixture distribution, (3)
P({at xt }Tt=1 ) =
M
π m p∗m (x1 a1 )
m=1
=
M
T
m t−1 ftm (xt |{xτ aτ }t−1 τ=1 )Pt (at |xt {xτ aτ }τ=1 )
t=2
π m p∗m (x1 a1 )
m=1
T
m ftm (xt |{xτ aτ }t−1 τ=1 )Pt (at |xt xt−1 at−1 )
t=2
where the first equality presents a general mixture model, while the second equality imposes the Markovian assumption on the conditional choice probam bilities, Ptm (at |xt {xτ aτ }t−1 τ=1 ) = Pt (at |xt xt−1 at−1 ). This is the key identifying assumption of this paper. The left-hand side of (3) is the distribution function of the observable data, while the right-hand side of the second equality contains the objects we would like the data to inform us about. REMARK 1: In models where at and xt follow a stationary first-order Markov process, it is sometimes assumed that the choice of the distribution of the initial observation, p∗m (x1 a1 ), is the stationary distribution that satisfies the fixed point constraint p∗m (x1 a1 ) = (4) P m (a1 |x1 x a )f m (x1 |x a )p∗m (x a ) x ∈X a ∈A
when all the components of x have finite support. When x is continuously distributed, we replace the summation over x with integration. Our identification result does not rely on the stationarity assumption of the initial conditions. The model (3) includes the following examples as special cases. EXAMPLE 1—Dynamic Discrete Choice Model With Heterogeneous Coefficients: Denote a parameter vector specific to type m’s individual by θm = (βm ρm ) . Consider a dynamic binary choice model for individual i who belongs to type m: (5)
m P m (ait = 1|xit {xiτ aiτ }t−1 τ=1 ) = P (ait = 1|xit ait−1 )
= Φ(xit βm + ρm ait−1 )
141
MODELS OF DYNAMIC DISCRETE CHOICES
where the first equality imposes the Markovian assumption and the second follows from the parametric restriction with Φ(·) denoting the standard normal cumulative distribution function (c.d.f.). The distribution of xit conditional on (xit−1 ait−1 ) is specific to the value of θm . Since the evolution of (xit ait ) in the presample period is not independent of random coefficient θm , the initial distribution of (xi1 ai1 ) depends on the value of θm (cf. Heckman (1981)). Browning and Carro (2007) estimated a continuous mixture version of (5) for the purchase of milk using a Danish consumer “long” panel (T ≥ 100), and provided evidence for heterogeneity in coefficients. Their study illustrates that allowing for such heterogeneity can make a significant difference for outcomes of interest such as the marginal dynamic effect. In practice however, researchers quite often only have access to a short panel. The results of this paper are therefore useful to understand the extent to which unobserved heterogeneity in coefficients is identified in such a situation. Our identification results are not applicable, however, to a parametric dynamic discrete choice model with serially correlated idiosyncratic shocks; for example, ait = 1(xit βm + ρm ait−1 + εit ), where εit is serially correlated. EXAMPLE 2—Structural Dynamic Discrete Choice Models: ∞ Type m’s agent maximizes the expected discounted sum of utilities, E[ j=0 βj {u(xt+j at+j ; θm ) + εt+j (at+j )}|at xt ; θm ], where xt is an observable state variable and εt (at ) is a state variable that are known to the agent but not to the researcher. The Bellman equation for this dynamic optimization problem is m m V (x) = max u(x a; θ ) + ε(a) + β (6) V (x )f (x |x a; θ ) a∈A
x ∈X
× g(dε|x) where g(ε|x) is the joint distribution of ε = {ε(j) : j ∈ A} and f (x |x a; θm ) is a type-specific transition function. The conditional choice probability is
Pθm (a|x) = 1 a = arg max u(x j; θm ) + ε(j) (7) j∈A
+β
Vθm (x )f (x |x j; θ ) m
x ∈X
× g(dε|x) where Vθm is the fixed point of (6). Let Ptm (at |xt xt−1 at−1 ) = Pθm (at |xt ) m and ftm (xt |{xτ aτ }t−1 τ=1 ) = f (xt |xt−1 at−1 ; θ ) in (3). The initial distribution of (x1 a1 ) is given by the stationary distribution (4). Then the likelihood function for {at xt }Tt=1 is given by (3) with (4).
142
H. KASAHARA AND K. SHIMOTSU
We study the nonparametric identifiability of the type probabilities, the initial distribution, the type-specific conditional choice probabilities, and the type-specific transition function in equation (3), which we denote by θ = {π m p∗m (·) {Ptm (·|·) ftm (·|·)}Tt=2 }M m=1 . Following the standard definition of nonparametric identifiability, θ is said to be nonparametrically identified (or identifiable) if it is uniquely determined by the distribution function P({at xt }Tt=1 ), without making any parametric assumption about the elements of θ. Because the order of the component distributions can be changed, θ is identified only up to a permutation of the components. If no two of the π’s are identical, we may uniquely determine the components by assuming π 1 < π 2 < · · · < π M . 2.1. Our Approach and Identification of the Baseline Model The finite mixture models studied by Hall and Zhou (2003) have no covariates as discussed in the Introduction. In this subsection, we show that the presence of covariates in our model creates a powerful source of identification. First, we impose the following simplifying assumptions on the general model (3) and analyze the nonparametric identifiability of the resulting “baseline model.” Analyzing the baseline model helps elucidate the basic idea of our approach and clarifies the logic behind our main results. In the subsequent sections, we relax Assumption 1 in various ways and study how it affects the identifiability of the resulting models. ASSUMPTION 1: (a) The choice probability of at does not depend on time. (b) The choice probability of at is independent of the lagged variable (xt−1 at−1 ) t−1 t conditional on xt . (c) ftm (xt |{xτ aτ }t−1 τ=1 ) > 0 for all (xt {xτ aτ }τ=1 ) ∈ X × t−1 A and for all m. (d) The transition function is common across types; t−1 ftm (xt |{xτ aτ }t−1 τ=1 ) = ft (xt |{xτ aτ }τ=1 ) for all m. (e) The transition function is t−1 stationary; ft (xt |{xτ aτ }τ=1 ) = f (xt |xt−1 at−1 ) for all m. Under Assumptions 1(a) and (b), the choice probabilities are written as Ptm (at |xt xt−1 at−1 ) = P m (at |xt ), where at−1 is not one of the elements of xt . Under Assumption 1(b), the lagged variable (xt−1 at−1 ) affects the current choice at only through its effect on xt via ftm (xt |{xτ aτ }t−1 τ=1 ). Assumption 1(c) implies that, starting from any combinations of the past state and action, any state x ∈ X is reached in the next period with positive probability. With Assumption 1 imposed, the baseline model is (8)
P({at xt }Tt=1 ) =
M m=1
π m p∗m (x1 a1 )
T
f (xt |xt−1 at−1 )P m (at |xt )
t=2
Since f (xt |xt−1 at−1 ) is nonparametrically identified directly from the observed data (cf. Rust (1987)), we may assume f (xt |xt−1 at−1 ) is known without affecting the other parts of the argument. Divide P({at xt }Tt=1 ) by the transition
143
MODELS OF DYNAMIC DISCRETE CHOICES
functions and define (9)
˜ t xt }T ) = P({a t=1
P({at xt }Tt=1 ) T f (xt |xt−1 at−1 ) t=2
=
M
π m p∗m (x1 a1 )
m=1
T
P m (at |xt )
t=2
which can be computed from the observed data. Assumption 1 guarantees that ˜ t xt }T ) is well defined for any possible sequence of {at xt }T ∈ (A×X)T . P({a t=1 t=1 Let I = {i1 il } be a subset of the time indices, so that I ⊆ {1 T }, where 1 ≤ l ≤ T and 1 ≤ i1 < · · · < il ≤ T . Integrating out different elements ˜ t xt }T ), which we call from (9) gives the l-variate marginal version of P({a t=1 lower-dimensional submodels (10)
M l
π m p∗m x1 a1 P m (ais |xis ) P˜ ais xis is ∈I = m=1
when {1} ∈ I
s=2
and (11)
M l
P˜ ais xis is ∈I = πm P m ais |xis m=1
when
{1} ∈ / I
s=1
In model (9), a powerful source of identification is provided by the difference in each type’s response patterns to the variation of the covariate (x1 xT ). The key insight is that for each different value of (x1 xT ), (10) and (11) imply different restrictions on the type probabilities and conditional choice probabilities. Let |X| denote the number of elements in X. The variation of (x1 xT ) generates different versions of (10) and (11), providing restrictions whose number is on the order of |X|T , while the number of the parameters {π m p∗m (a x) P m (a|x) : (a x) ∈ A × X}M m=1 is on the order of |X|. This identification approach is much more effective than one without covariates, in particular, when T is small.4 In what follows, we assume that the support of the state variables is discrete and known. This is assumed for the sake of clarity: our identification results are easier to understand in the context of a discrete state space, although they hold more generally. We also focus on the case where A = {0 1} to simplify notation. It is straightforward to extend our analysis to the case with a multinomial For example, when T = 3 and A = {0 1}, (10) and (11) imply at least tions while there are 3M|X| − 1 parameters. 4
|X|+2 3
different restric-
144
H. KASAHARA AND K. SHIMOTSU
choice of a, but with heavier notation. Note also that Chandra (1977) shows that a multivariate finite mixture model is identified if all the marginal models are identified. It is convenient to collect notation first. Define, for ξ ∈ X, (12)
∗m λ∗m and ξ = p ((a1 x1 ) = (1 ξ))
m λm ξ = P (a = 1|x = ξ)
Let ξj , j = 1 M − 1, be elements of X. Let k be an element of X. Define a matrix of type-specific distribution functions and type probabilities as ⎤ ⎡ 1 λ1ξ1 · · · λ1ξM−1 ⎢ ⎥ (13) L = ⎣ ⎦ (M×M) M M 1 λξ1 · · · λξM−1 ∗M Dk = diag(λ∗1 k λk )
V = diag(π 1 π M )
The elements of L, Dk , and V are parameters of the underlying mixture models to be identified. Now we collect notation for matrices of observables. Fix at = 1 for all t in ˜ t xt }3t=1 ), and define the resulting function as P({a (14)
˜ Fx∗1 x2 x3 = P({1 xt }3t=1 ) =
M
m m π m λ∗m x1 λx2 λx3
m=1
and λm where λ∗m x x are defined in (12). Next, integrate out (a1 x1 ) from 3 ˜ P({at xt }t=1 ), fix a2 = a3 = 1, and define the resulting function as (15)
˜ Fx2 x3 = P({1 xt }3t=2 ) =
M
m π m λm x2 λx3
m=1
Similarly, define the following “marginals” by integrating out other elements ˜ t xt }3t=1 ) and setting at = 1: from P({a (16)
˜ Fx∗1 x2 = P({1 xt }2t=1 ) =
M
m π m λ∗m x1 λx2
m=1
˜ Fx∗1 x3 = P({1 x1 1 x3 }) =
M m=1
˜ x1 }) = Fx∗1 = P({1
M m=1
π m λ∗m x1
m π m λ∗m x1 λx3
MODELS OF DYNAMIC DISCRETE CHOICES
˜ x2 }) = Fx2 = P({1
M
145
π m λm x2
m=1
˜ x3 }) = Fx3 = P({1
M
π m λm x3
m=1
Note that F·∗ involves (a1 x1 ) while F· does not contain (a1 x1 ). In fact, Fx∗1 x2 = Fx∗1 x3 if x2 = x3 because P m (a|x) does not depend on t, but we keep separate notation for the two because later we analyze the case where the choice probability depends on t. Evaluate Fx∗1 x2 x3 Fx2 x3 , and their marginals at x1 = k, x2 = ξ1 ξM−1 , and x3 = ξ1 ξM−1 , and arrange them into two M × M matrices: ⎡ ⎤ 1 Fξ1 ··· FξM−1 ⎢ Fξ Fξ1 ξ1 ··· Fξ1 ξM−1 ⎥ 1 ⎢ ⎥ P =⎢ (17) ⎥ ⎣ ⎦ FξM−1 FξM−1 ξ1 · · · ⎡ ∗ Fk∗ Fkξ 1 ∗ ∗ ⎢ Fkξ F kξ1 ξ1 ⎢ 1 Pk = ⎢ ⎣ ∗ ∗ FkξM−1 FkξM−1 ξ1
FξM−1 ξM−1 ··· ··· ···
⎤ ∗ Fkξ M−1 ∗ ⎥ Fkξ 1 ξM−1 ⎥ ⎥ ⎦ ∗ Fkξ M−1 ξM−1
The following proposition and corollary provide simple and intuitive sufficient conditions for identification under Assumption 1. Proposition 1 extends the idea of the proof of nonparametric identifiability of finite mixture models from Anderson (1954) and Gibson (1955) to models with covariates.5 Proposition 1 gives a sufficient condition for identification in terms of the rank of the matrix L and the type-specific choice probabilities evaluated at k. In practice, however, it may be difficult to check this rank condition because the elements of L are functions of the component distributions. Corollary 1 provides a sufficient condition in terms of the observable quantities P and Pk . The proofs are constructive. PROPOSITION 1: Suppose that Assumption 1 holds and assume T ≥ 3. Suppose further that there exist some {ξ1 ξM−1 } such that L is nonsingular and that ∗n ∗m there exists k ∈ X such that λ∗m k > 0 for all m and λk = λk for any m = n. Then 5 Anderson (1954) and Gibson (1955) analyzed nonparametric identification of finite mixture models similar to (9) but without covariates and derived a sufficient condition for nonparametric identifiability under the assumption T ≥ 2M − 1. Madansky (1960) extended their analysis to obtain a sufficient condition under the assumption 2(T −1)/2 ≥ M. When T is small, the number of identifiable types by their method is quite limited.
146
H. KASAHARA AND K. SHIMOTSU
3 3 m M ˜ {π m {λ∗m ξ λξ }ξ∈X }m=1 is uniquely determined from {P({at xt }t=1 ) : {at xt }t=1 ∈ 3 (A × X) }.
COROLLARY 1: Suppose that Assumption 1 holds, and assume T ≥ 3. Suppose further that there exist some {ξ1 ξM−1 } and k ∈ X such that P is of full rank and that all the eigenvalues of P −1 Pk take distinct values. Then 3 3 m M ˜ {π m {λ∗m ξ λξ }ξ∈X }m=1 is uniquely determined from {P({at xt }t=1 ) : {at xt }t=1 ∈ (A × X)3 }. REMARK 2: (i) The condition of Proposition 1 implies that all columns in L must be linearly independent. Since each column of L represents the conditional choice probability of different types for a given value of x, the changes in x must induce sufficiently heterogeneous variations in the conditional choice probabilities across types. In other words, the covariate must be relevant, and different types must respond to its changes differently. (ii) When λ∗m k = 0 for some m, its identification fails, because we never ∗n observe (x1 a1 ) for such type. The condition that λ∗m k = λk for some k ∈ X is satisfied if the initial distributions are different across different types. If either of these conditions is violated, then the initial distribution cannot be used as a source of identification and, as a result, the requirement on T becomes T ≥ 4 instead of T ≥ 3. (iii) One needs to find only one set of M − 1 points to construct a nonsingular L. The identification of choice probabilities at all other points in X follows without any further requirement. (iv) When X has |X| < ∞ support points, the number of identifiable types is at most |X|+1. When x is continuously distributed, we may potentially identify as many types as we wish. (v) By partitioning X into M − 1 disjoint subsets (Ξ1 Ξ2 ΞM−1 ), we may characterize a sufficient condition in terms of the conditional choice probabilities given a subset Ξj of X rather than an element ξj of X. (vi) We may check the conditions of Corollary 1 empirically by computing the sample counterpart of P and Pk for various {ξ1 ξM−1 }’s and/or for various partitions Ξj ’s. The latter procedure is especially useful when x is continuously distributed. The foundation for our identification method lies in the following relationship between the observables, P and Pk , and the parameters L, Dk , and V , which we call the factorization equations: (18)
P = L V L
Pk = L Dk V L
M Note that the (1 1)th element of P = L V L is 1 = m=1 π m . These two equations determine L Dk , and V uniquely. The first equation of (18) alone does
MODELS OF DYNAMIC DISCRETE CHOICES
147
not give a unique decomposition of P in terms of L and V , because this equation provides M(M + 1)/2 restrictions due to the symmetry of P, while there are M 2 − M + M = M 2 unknowns in L and V . Indeed, when M = 2, there are three restrictions and four unknowns, and L and V are just not identified. To shed further light on our identification method, we provide a sketch of how we constructively identify L, Dk , and V from P and Pk . Suppose P is invertible or, equivalently, L is invertible. As is apparent from equation (18), Pk is similar to P except that Pk contains an extra diagonal matrix Dk . Since P −1 Pk = L−1 Dk L, the eigenvalues of P −1 Pk identify the elements of Dk . Furthermore, multiplying both sides of P −1 Pk = L−1 Dk L by L−1 , we have (P −1 Pk )L−1 = L−1 Dk , suggesting that the columns of L−1 are identified with the eigenvectors of P −1 Pk . Finally, once L is identified, V is identified since V = (L )−1 PL−1 . By applying the above algorithm to a sample analogue of P and Pk , we may m M construct an estimator for {π m {λ∗m ξ λξ }ξ∈X }m=1 , which will have the same rate of convergence as the estimates of P and Pk . Alternatively, once identification is established, we may use various nonparametric estimation procedures, such as a series-based mixture likelihood estimator. Magnac and Thesmar (2002, Proposition 6) studied a finite mixture dynamic discrete choice model similar to our baseline model and showed that their model is not nonparametrically identified. They assumed that the transition probability is common across types and that the initial distribution is independent of the types. Hence, we may express their model in terms of our notation as (19)
P({at xt }Tt=1 ) =
M m=1
π m f (x1 )P m (a1 |x1 )
T
f (xt |xt−1 at−1 )P m (at |xt )
t=2
Setting p∗m (x1 a1 ) = f (x1 )P m (a1 |x1 ) gives our baseline model (3). Our results differ from those of Magnac and Thesmar in two ways: the length of the periods considered and the variation of xt . First, Magnac and Thesmar considered a two-period model, whereas our identifiability result requires at least three periods.6 Second, Magnac and Thesmar restricted the variation of xt by assuming that there are only two states, one of which is absorbing. In terms of our notation, this restriction is the same as assuming xt ∈ {0 1}, and xt = 1 with probability 1 if xt−1 = 1. This reduces the possible variation of the sequences of xt substantially, making identification difficult because only T + 1 different sequences of xt are observable. For example, when T = 3, the only possible sequences of (x1 x2 x3 ) are (1 1 1), (0 1 1), (0 0 1), and (0 0 0). 6 The possibility of identifying many types using the variation of |X| under T = 2 is currently under investigation. A related study on nonidentifiability of multivariate mixtures by Kasahara and Shimotsu (2008b) suggests, however, that T ≥ 3 is necessary for identification even when |X| ≥ 2.
148
H. KASAHARA AND K. SHIMOTSU
If we assume T ≥ 3 and that there are more than two nonabsorbing states, then we can apply Proposition 1 and Corollary 1 to (19). Alternately, identifying the single nonabsorbing state model is still possible if T ≥ 2M − 1 by applying Remark 3 below with x¯ = 0, although Remark 3 uses the stationarity of P m (a|x). For the sake of brevity, in the subsequent analysis we provide sufficient conditions only in terms of the rank of the matrix of the type-specific component distributions (e.g., L). In each of the following propositions, sufficient conditions in terms of the distribution function of the observed data can easily be deduced from the conditions in terms of the type-specific component distributions. The identification method of Proposition 1 uses a set of restrictions implied by the joint distribution of only (a1 x1 a2 x2 a3 x3 ). When the variation of (x1 x2 xT ) for T ≥ 5 is available, we may adopt the approach of Madansky (1960) to use the information contained in all xt ’s. Define u = (T − 1)/2, and write the functions corresponding to (14) and (15) as (20)
˜ Fx∗1 ···xT = P({1 xt }Tt=1 ) =
M
m m π m λ∗m x1 λx2 · · · λxT
m=1
=
M
m m m m π m λ∗m x1 λx2 · · · λxu+1 λxu+2 · · · λxT
m=1
and (21)
Fx2 ···xT
˜ = P({1 xt }Tt=2 ) =
M
m m m π m λm x2 · · · λxu+1 λxu+2 · · · λxT
m=1
Equations (20) and (21) have the same form as (14) and (15) if we view m m m u λm x2 · · · λxu+1 and λxu+2 · · · λxT as marginal distributions with |X| support points. Consequently, we can construct factorization equations similar to (18), in which the elements of a matrix corresponding to the matrix L are based on m m m λm x2 · · · λxu+1 and λxu+2 · · · λxT and their subsets. This extends the maximum number of identifiable types from on the order of |X| to on the order of |X|(T −1)/2 . Despite being more complex than Proposition 1, the following proposition is useful when T is large, making it possible to identify a large number of types even if |X| is small. For notational simplicity, we assume |X| is finite and X = {1 2 |X|}. PROPOSITION 2: Suppose that Assumption 1 holds. Assume T ≥ 5 is odd and define u = (T − 1)/2. Suppose X = {1 2 |X|} and define ⎤ ⎡ 1 ⎡ ⎤ λ1 · · · λ1|X| 1 ⎢ ⎥ Λ0 = ⎣ ⎦ Λ1 = ⎣ ⎦ M M 1 λ ··· λ 1
|X|
MODELS OF DYNAMIC DISCRETE CHOICES
149
For l = 2 u, define Λl to be a matrix, each column of which is formed by choosing l columns (unordered, with replacement) from the columns of Λ1 and taking their Hadamard product. There are |X|+l−1 ways to choose such columns; l |X|+l−1 thus the dimension of Λl is M × . For example, Λ2 and Λ3 take the form l ⎤ ⎡ 1 1 λ1 λ1 · · · λ11 λ1|X| λ12 λ12 · · · λ12 λ1|X| · · · λ1|X| λ1|X| ⎥ ⎢ Λ2 = ⎣ ⎦ M M M M M M M M M M λ1 λ1 · · · λ1 λ|X| λ2 λ2 · · · λ2 λ|X| · · · λ|X| λ|X| ⎡ 1 1 1 λ1 λ1 λ1 · · · λ11 λ11 λ1|X| ⎢ Λ3 = ⎣ M M M M M λM λ λ · · · λ λ 1 1 1 1 1 λ|X| λ12 λ11 λ12 · · · λ12 λ11 λ1|X| M M M M M M λ2 λ1 λ2 · · · λ2 λ1 λ|X| u |X|+l−1 ) matrix Λ as Define an M × ( l=0 l
⎤ · · · λ1|X| λ1|X| λ1|X| ⎥ ⎦ M M · · · λM |X| λ|X| λ|X|
Λ = [Λ0 Λ1 Λ2 Λu ] u Suppose (a) l=0 |X|+l−1 ≥ M, (b) we can construct a nonsingular M ×M matrix l L by setting its first column as Λ0 and choosing other M − 1 columns from the columns of Λ other than Λ0 , and (c) there exists k ∈ X such that λ∗m k > 0 for all m ∗n m ∗m m |X| M and λ∗m = λ for any m = n. Then {π {λ λ } } is uniquely determined k k j j j=1 m=1 T T T ˜ from {P({at xt }t=1 ) : {at xt }t=1 ∈ (A × X) }. REMARK 3: In a special case where there are no covariates and |X| = 1, the matrix Λ becomes ⎤ ⎡ 1 λ11 (λ11 )2 · · · (λ11 )u ⎢ ⎥ Λ = ⎣ ⎦ 1 λM 1
2 (λM 1 )
u · · · (λM 1 )
and the sufficient condition of Proposition 2 reduces to (a) T ≥ 2M − 1, n ∗n ∗m ∗m (b) λm 1 = λ1 for any m = n, and (c) λ1 > 0 and λ1 = λ1 for any m = n. Not surprisingly, the condition T ≥ 2M − 1 coincides with the sufficient condition of nonparametric identification of finite mixtures of binomial distributions (Blischke (1964)). This set of sufficient condition also applies to the case where the covariates have no time variation (x1 = · · · = xT ), such as race and/or sex. In this case, because of the stationarity, the time-series variation of at substitutes for the variation of xt .
150
H. KASAHARA AND K. SHIMOTSU
Houde and Imai (2006) studied nonparametric identification of finite mix¯ ture dynamic discrete choice models by fixing the value of the covariate x (to x, for instance) and derived a sufficient condition for T . They also considered a model with terminating state. If the conditional choice probabilities of different types are heterogeneous and the column vectors (λ1x λM x ) for x = 1 |X| are linearly independent, the rank condition of this proposition is likely to be satisfied, since the Hadamard products of these column vectors are unlikely to be linearly dependent, unless by chance. Since the construction of the matrices in Proposition 2 is rather complex, we provide a simple example with T = 5 to illustrate its connection to the L matrix in Proposition 1. EXAMPLE 3—An Example for Proposition2: Suppose that T = 5 and X = 2 {1 2}. In this case, we can identify M = l=0 1+l = 6 types. Consider a matrix l ⎡
λ11
λ12
λ11 λ11
λ11 λ12
1 λM 1
λM 2
M λM 1 λ1
M λM 1 λ2
1 ⎢
L =Λ=⎣
⎤ λ12 λ12 ⎥ ⎦ M M λ2 λ2
Then the factorization equations that correspond to (18) are given by (22)
P = (L ) V L
Pk = (L ) Dk V L
∗M where V = diag(π 1 π M ) and Dk = diag(λ∗1 k λk ), as defined in (13).
We can verify that the elements of P and Pk can be constructed from the distribution function of the observed data. For instance,
⎡
1 ⎢ F1 ⎢ ⎢F P = ⎢ 2 ⎢ F11 ⎣F 12 F22
F1 F11 F21 F111 F121 F221
F2 F12 F22 F112 F122 F222
F11 F111 F211 F1111 F1211 F2211
F12 F112 F212 F1112 F1212 F2212
⎤ F22 F122 ⎥ ⎥ F222 ⎥ ⎥ F1122 ⎥ F1222 ⎦ F2222
M M M m m m m m m m where Fi = m=1 π m λm i , Fij = m=1 π λi λj , Fijk = m=1 π λi λj λk , and M m m m Fijkl = m=1 π m λm i λj λk λl for i j k l ∈ {1 2} are identifiable from the population. Once the factorization equations (22) are constructed, we may apply the argument following Corollary 1 to determine L , V , and Dk uniquely from P and Pk .
MODELS OF DYNAMIC DISCRETE CHOICES
151
2.2. Identification of the Number of Types So far, we have assumed that the number of mixture components M is known. How to choose M is an important practical issue because economic theory usually does not provide much guidance. We now show that it is possible to nonparametrically identify the number of types from panel data with two periods.7 Assume T ≥ 2 and X = {1 |X|}. Define a (|X| + 1) × (|X| + 1) matrix which is analogous to P in (17) but uses the first two periods and all the support points of X: ⎡ 1 ∗ ⎢ F1 ∗ ⎢ P = ⎣ ∗ F|X|
F1 ∗ F11 ∗ F|X|1
··· ··· ···
F|X| ⎤ ∗ F1|X| ⎥ ⎥ ⎦ ∗ F|X||X|
M m ∗m ˜ where, as defined in (16), Fi∗ = P({(a 1 x1 ) = (1 i)}) = m=1 π λi , Fi = M m m ∗ ˜ ˜ P({(a 2 x2 ) = (1 i)}) = m=1 π λi , and Fij = P({(a1 x1 a2 x2 ) = (1 i 1 M m ∗m m ∗ j)}) = m=1 π λi λj . The matrix P contains information on how different types react differently to the changes in covariates for all possible x’s. The following proposition shows that we may nonparametrically identify the number of types from P ∗ under Assumption 1. PROPOSITION 3: Suppose that Assumption 1 holds. Assume T ≥ 2 and X = {1 |X|}. Then M ≥ rank(P ∗ ). Furthermore, in addition to Assumption 1, suppose that the two matrices L∗1 and L∗2 defined below both have rank M: ⎤ ⎡ 1 λ∗1 · · · λ∗1 1 |X| ⎢ ⎥ L∗1 = ⎣ ⎦ (M×(|X|+1)) ∗M ∗M · · · λ|X| 1 λ1 ⎤ ⎡ 1 1 λ1 · · · λ1|X| ⎢ ⎥ L∗2 = ⎣ ⎦ (M×(|X|+1))
1
λM 1
···
λM |X|
Then M = rank(P ∗ ). REMARK 4: (i) The rank condition of L∗1 and L∗2 is not empirically testable from the observed data. The rank of P ∗ , which is observable, gives the lower bound of the number of types. 7
We thank the co-editor and a referee for suggesting that we investigate this problem.
152
H. KASAHARA AND K. SHIMOTSU
(ii) Surprisingly, two periods of panel data, rather than three periods, may suffice for identifying the number of types. (iii) The rank condition on L∗1 implies that no row of L∗1 can be expressed as a linear combination of the other rows of L∗1 . The same applies to the rank condition on L∗2 . Since the mth row of L∗1 or L∗2 completely summarizes type m’s conditional choice probability within each period, this condition requires that the changes in x provide sufficient variation in the choice probabilities across types and that no type is “redundant” in one-dimensional submodels. (iv) The rank condition on L∗2 is equivalent to the rank condition on L in Proposition 1 when X = {1 |X|}. In other words, rank(L∗2 ) = M if and only if rank(L) = M. (v) We may partition X into disjoint subsets and compute P ∗ with respect to subsets of X rather than elements of X. (vi) When T ≥ 4, we use an approach similar to Proposition 2 to con may u u |X|+l−1 struct a ( l=0 |X|+l−1 × ) matrix P ∗ (similar to P in Example 3 l=0 l l but using (x1 x2 ) and (x3 x4 )) and increase the number of identifiable M to the order of |X|(T −1)/2 . 3. EXTENSIONS OF THE BASELINE MODEL In this section, we relax Assumption 1 of the baseline model in various ways to accommodate real-world applications. In the following subsections, we relax Assumption 1(a) and (e) (stationarity), Assumption 1(b) and (d) (typeinvariant transition), and Assumption 1(c) (unrestricted transition) in turn and analyze nonparametric identifiability of resulting models. In all cases, identification is achieved by constructing a version of the factorization equation similar to (18), specific to the model under consideration, and then applying an argument that follows the one presented in (18). The differences arise solely from the ways in which the factorization equations are constructed across the various models. 3.1. Time-Dependent Conditional Choice Probabilities The baseline model (8) assumes that conditional choice probabilities and the transition function do not change over periods. However, the agent’s decision rules may change over periods in some models, such as a model with timespecific aggregate shocks or a model of finitely lived individuals. In this subsection, we keep the assumption of the common transition function, but relax Assumption 1(a) and (e) to extend our analysis to models with time-dependent choice probabilities. When Assumption 1(a) and (e) (stationarity) are relaxed but Assumption 1(b) and (d) (conditional independence and type-invariant transition) are maintained, the choice probabilities and the transition function are written as
MODELS OF DYNAMIC DISCRETE CHOICES
153
t−1 Ptm (at |xt xt−1 at−1 ) = Ptm (at |xt ) and ftm (xt |{xτ aτ }t−1 τ=1 ) = ft (xt |{xτ aτ }τ=1 ), respectively, where at−1 is not an element of xt . Equation (9) then becomes
(23)
˜ t xt }T ) = P({a t=1
P({at xt }Tt=1 ) T
ft (xt |{xτ aτ }t−1 τ=1 )
t=2
=
M
π m p∗m (x1 a1 )
m=1
T
Ptm (at |xt )
t=2
The next proposition states a sufficient condition for nonparametric identification of the mixture model (23). In the baseline model (8), the sufficient condition is summarized to the invertibility of a matrix consisting of the conditional choice probabilities. In the time-dependent case, this matrix of conditional choice probabilities becomes time-dependent, and hence its invertibility needs to hold for each period. We consider the case of A = {0 1}. Define, for ξ ∈ X, ∗m λ∗m ξ = p ((a1 x1 ) = (1 ξ)) m λm tξ = Pt (at = 1|xt = ξ)
and
t = 2 T
PROPOSITION 4: Suppose that Assumptions 1(b)–(d) hold and assume T ≥ 3. For t = 2 T − 1, let ξjt , j = 1 M − 1, be elements of X and define ⎡
···
1 λ1tξt 1 ⎢ ⎢ Lt = ⎣ (M×M) M 1 λtξt
···
1
⎤ λ1tξt M−1 ⎥ ⎥ ⎦ λM tξt M−1
Suppose there exist {ξ ξ } such that Lt is nonsingular for t = 2 T and ∗n there exists k ∈ X such that λ > 0 for all m and λ∗m k = λk for any m = n. Then m T M T ˜ {π m {λ∗m ξ {λtξ }t=2 }ξ∈X }m=1 is uniquely determined from {P({at xt }t=1 ) : {at T T xt }t=1 ∈ (A × X) }. t 1
t M−1 ∗m k
When the choice probabilities are time-dependent, the factorization equations (that correspond to (18)) are also time-dependent: Pt = Lt V Lt+1
and
Ptk = Lt Dk V Lt+1
for
t = 2 T − 1
where V and Dk are defined as before. The elements of Pt and Ptk are the ˜ t xt }T ) in equation (23) with at = 1 for all t. Specifically, “marginals” of P({a t=1
154
H. KASAHARA AND K. SHIMOTSU
Pt and Ptk are defined as ⎡
···
Fξt+1 t+1
1
⎤
Fξt+1 t+1
⎥ ⎢ t ⎢ Fξt ⎥ Fξtt+1 ··· Fξtt+1 t ξt+1 t ξt+1 ⎢ 1 1 1 1 M−1 ⎥ Pt = ⎢ ⎥ ⎢ ⎥ ⎣ ⎦ tt+1 tt+1 t Fξt Fξt ξt+1 · · · Fξt ξt+1 M−1 M−1 1 M−1 M−1 ⎡ F∗ ∗t+1 ∗t+1 Fkξ ··· Fkξ t+1 t+1 k 1 M−1 ⎢ ∗t ∗tt+1 ∗tt+1 ⎢ Fkξt Fkξ ··· Fkξ t t+1 t t+1 ⎢ 1 1 ξ1 1 ξM−1 Ptk = ⎢ ⎢ ⎣ 1
∗t Fkξ t
M−1
∗tt+1 Fkξ t
t+1 M−1 ξ1
M−1
···
∗tt+1 Fkξ t
⎤ ⎥ ⎥ ⎥ ⎥ ⎥ ⎦
t+1 M−1 ξM−1
M tt+1 t m m ˜ where, similar to (15), Fξt t = P({(a t xt ) = (1 ξ )}) = m=1 π λtξt , Fξt ξt+1 = M m ∗ t t+1 ˜ ˜ P({(a )}) = m=1 π m λm t xt at+1 xt+1 ) = (1 ξ 1 ξ tξt λt+1ξt+1 , Fk = P({(a1 M ∗t m ∗m t ˜ x1 ) = (1 k)}) = m=1 π λk , Fkξt = P({(a 1 x1 at xt ) = (1 k 1 ξ )}) = M ∗tt+1 m ∗m m t ˜ m=1 π λk λtξt , and Fkξt ξt+1 = P({(a1 x1 at xt at+1 xt+1 ) = (1 k 1 ξ 1 M m m ξt+1 )}) = m=1 π m λ∗m k λtξt λt+1ξt+1 . Since Pt and Ptk are identifiable from the data, we may construct V , Dk , Lt , and Lt+1 from Pt and Ptk for t = 2 T − 1 by applying an argument that follows the one presented in (18) to each period. The following proposition corresponds to Proposition 2 and relaxes the identification condition of Proposition 4 when T ≥ 5 by utilizing all the marginals ˜ t xt }T ). The proof is omitted because it is similar to that of Propoof P({a t=1 sition 2. The difference from Proposition 2 is (i) the conditions are stated in m terms of λm tξ instead of λξ because of time-dependence and (ii) the number of restrictions implied by the submodels, analogously defined to (10) and (11) but with time subscripts, is larger because the order of the choices becomes relevant. As a result, the condition on |X| of Proposition 5 is weaker than that of Proposition 2. PROPOSITION 5: Suppose Assumption 1(b)–(d) hold. Assume T ≥ 5 is odd and define u = (T − 1)/2. Suppose X = {1 |X|} and further define ⎤ ⎡ 1 ⎡ ⎤ λ21 · · · λ12|X| 1 ⎢ ⎥ Λ¯ 0 = ⎣ ⎦ Λ¯ 11 = ⎣ ⎦ M M 1 ··· λ λ 21
2|X|
For l = 2 u, define Λ¯ 1l to be a matrix whose elements consist of the l-variate m m product of the form λm 2j2 λ3j3 · · · λljl+1 , covering all possible l ordered combinations
MODELS OF DYNAMIC DISCRETE CHOICES
155
(with replacement) of (j2 j3 jl+1 ) from (1 |X|). For example, ⎡ 1 1 λ21 λ31 · · · λ121 λ13|X| ⎢ Λ¯ 12 = ⎣ M M M λ · · · λ λ λM 21 31 21 3|X| λ122 λ131 M M λ22 λ31
· · · λ122 λ13|X| M M · · · λ22 λ3|X|
λ12|X| λ131 M λ2|X| λM 31
⎤ · · · λ12|X| λ13|X| ⎥ ⎦ M · · · λM 2|X| λ3|X|
Similarly, for l = 1 u, define Λ¯ 2l to be a matrix whose elements consist of m m the l-variate product of the form λm u+1j2 λu+2j3 · · · λu+ljl+1 , covering all possible l ordered combinations (with replacement) of (j2 j3 jl+1 ) from (1 |X|). Let Λ¯ 1 = [Λ¯ 0 Λ¯ 11 Λ¯ 12 Λ¯ 1u ]
and
Λ¯ 2 = [Λ¯ 0 Λ¯ 21 Λ¯ 22 Λ¯ 2u ]
Define L¯ 1 to be an M × M matrix whose first column is Λ¯ 0 and whose other M − 1 columns are from the columns of Λ¯ 1 other than Λ¯ 0 . Define L¯ 2 to be an M × M matrix whose first is Λ¯ 0 and whose other columns are from Λ¯ 2 . column u Suppose (a) l=0 |X|l ≥ M, (b) L¯ 1 and L¯ 2 are nonsingular, and (c) there ∗n ∗m exists k ∈ X such that λ∗m k > 0 for all m and λk = λk for any m = n. |X| M m T T ˜ Then {π m {λ∗m j {λtj }t=2 }j=1 }m=1 is uniquely determined from {P({at xt }t=1 ) : {at T T xt }t=1 ∈ (A × X) }. 3.2. Lagged Dependent Variable and Type-Specific Transition Functions In empirical applications, including the lagged choice in explanatory variables for the current choice is a popular way to specify dynamic discrete choice models. Furthermore, we may encounter a case where the transition pattern of state variables is heterogeneous across individuals, even after controlling for other observables. In such cases, the transition function of both at and xt becomes type-dependent. In this subsection, we relax Assumption 1(b) and (d) of the baseline model (8) to accommodate type-specific transition functions as well as the dependence of current choice on lagged variables. In place of Assumption 1(b) and (d), we impose stationarity and a first-order Markov property on the transition process of xt . Assumption 2(a) and (c) are identical to Assumption 1(a) and (c). ASSUMPTION 2: (a) The choice probability of at does not depend on time. (b) xt follows a stationary first-order Markov process; ftm (xt |{xτ aτ }t−1 τ=1 ) = f m (xt |xt−1 at−1 ) for all t and m. (c) f m (x |x a) > 0 for all (x x a) ∈ X × X × A and for all m.
156
H. KASAHARA AND K. SHIMOTSU
Under Assumption 2, the model is (24)
P({at xt }Tt=1 ) =
M
π m p∗m (x1 a1 )
m=1
T
f m (xt |xt−1 at−1 )P m (at |xt xt−1 at−1 )
t=2
and the transition process of (at xt ) becomes a stationary first-order Markov process. Define st = (at xt ), q∗m (s1 ) = p∗m (x1 a1 ), and Qm (st |st−1 ) = f m (xt | xt−1 at−1 )P m (at |xt xt−1 at−1 ), and rewrite the model (24) as (25)
P({st }Tt=1 ) =
M
π m q∗m (s1 )
m=1
T
Qm (st |st−1 )
t=2
Unlike the transformed baseline model (9), st appears both in Qm (st |s t−1 ) and Qm (st+1 |s t ), and creates the dependence between these terms. Consequently, the variation of st affects P({st }Tt=1 ) via both Qm (st |s t−1 ) and Qm (st+1 |s t ). This dependence makes it difficult to construct factorization equations that correspond to (18), which is the key to obtaining identification. We solve this dependence problem by using the Markov property of st . The idea is that if st follows a first-order Markov process, looking at every other period breaks the dependence of st across periods. Specifically, consider the sequence (st−1 st st+1 ) for various values of st , while fixing the values of st−1 and st+1 . Once st−1 and st+1 are fixed, the variation of st does not affect the state variables in other periods because of the Markovian structure of Qm (st |st−1 ). As a result, we can use this variation to distinguish different types. Let s¯ ∈ S = A × X be a fixed value of s and define (26)
πs¯m = π m q∗m (¯s)
m λm s|s)Qm (s|¯s) s¯ (s) = Q (¯
m λ∗m s) s¯ (sT ) = Q (sT |¯
Assume T is even and consider P({st }Tt=1 ) with st = s¯ for odd t: T −2 M ∗m (27) πs¯m λm P({st }Tt=1 |st = s¯ for t odd) = s¯ (st ) λs¯ (sT ) m=1
t=24
This conditional mixture model shares the property of independent marginals with (9). Consequently, we can construct factorization equations similar to (18) and, hence, can identify the components of the mixture model (27) for each s¯ ∈ S. Assume T = 6. Let ξj , j = 1 M − 1, be elements of S and let k ∈ S. Define ⎡ ⎤ 1 λ1s¯ (ξ1 ) · · · λ1s¯ (ξM−1 ) ⎢ ⎥ Ls¯ = ⎣ ⎦ (M×M)
1 λM s¯ (ξ1 )
···
λM s¯ (ξM−1 )
MODELS OF DYNAMIC DISCRETE CHOICES
Vs¯ = diag(πs¯1 πs¯M )
157
∗M Dk|¯s = diag(λ∗1 s¯ (k) λs¯ (k))
Then, from (27), the factorization equations that correspond to (18) are (28)
Ps¯ = Ls¯ Vs¯ Ls¯
Ps¯k = Ls¯ Dk|¯s Vs¯ Ls¯
where the elements of Ps¯ and Ps¯k are various marginals of the left-hand side of (27) and are identifiable from the data. Then we can construct Vs¯ , Dk|¯s , and Ls¯ uniquely from Ps¯ and Ps¯k by applying the argument following Corollary 1. The following proposition establishes a sufficient condition for nonparametric identification of model (27). Because of the temporal dependence in st , the requirement on T becomes T ≥ 6 instead of T ≥ 3. PROPOSITION 6: Suppose Assumption 2 holds and assume T ≥ 6. Suppose that q∗m (¯s) > 0 for all m, there exist some {ξ1 ξM−1 } such that Ls¯ is nonsin∗m ∗n gular, and there exists k ∈ S such that λ∗m s¯ (k) > 0 for all m and λs¯ (k) = λs¯ (k) m m ∗m M for any m = n. Then {πs¯ {λs¯ (s) λs¯ (s)}s∈S }m=1 is uniquely determined from {P({st }Tt=1 ) : {st }Tt=1 ∈ S T }. REMARK 5: (i) The assumption of stationarity and a first-order Markov property is crucial. When st follows a second-order Markov process (e.g., P m (at |{xτ aτ }t−1 τ=1 ) = P m (at |xt−1 at−1 xt−2 at−2 )), the requirement on T becomes T ≥ 9 instead of T ≥ 6 because we need to look at every two other periods so as to obtain the “independent” variation of st across periods. (ii) The transition functions and conditional choice probabilities are idenm tified from λ∗m s) = λ∗m s¯ (s) as follows. Recall Q (s|¯ s¯ (s) by definition (see (26)) m m m ¯ a)P ¯ ¯ a) ¯ with (a ¯ x) ¯ = s¯ . Summing Qm (s|¯s) over (a|x x and Q (s|¯s) = f (x|x ¯ a), ¯ and we then identify the conditional choice probabilia ∈ A gives f m (x|x ¯ a) ¯ = Qm (s|¯s)/f m (x|x ¯ a). ¯ ties by P m (a|x x (iii) If |S| M and the transition pattern of s is sufficiently heterogeneous across different types, the sufficient conditions in Proposition 6 are likely to hold for all s¯ ∈ S, and we may therefore identify the primitive parameters π m , p∗m (a x), f m (x |x a), and P m (a |x x a). Specifically, repeating Proposition 6 for all s¯ ∈ S, we obtain π m q∗m (s) = π m p∗m (a x) for all (a x) ∈ A × X. Then m m π is determined by π = (ax)∈A×X π m p∗m (a x) and we identify p∗m (a x) = (π m p∗m (a x))/π m . EXAMPLE 4 —An Example for Proposition 6: Consider a case in which T = 6, A = {0 1}, and X = {0 1}. Then M = |S| + 1 = 5 types can be identified. Fix s1 = s3 = s5 = s¯ ∈ A × X. Then Ls¯ is given by ⎡ ⎤ 1 λ1s¯ (0 0) λ1s¯ (0 1) λ1s¯ (1 0) λ1s¯ (1 1) ⎢ ⎥ Ls¯ = ⎣ ⎦ (M×M)
1
λM λM λM λM s¯ (0 0) s¯ (0 1) s¯ (1 0) s¯ (1 1)
158
H. KASAHARA AND K. SHIMOTSU
For example, the (5 5)th elements of Ps¯ and Ps¯k in (28) are given by P({s1 = 5 2 s3 = s5 = s¯ s2 = s4 = (1 1)}) = m=1 πs¯m (λm s¯ (1 1)) and P({s1 = s3 = s5 = 5 2 ∗m s¯ s2 = s4 = (1 1) s6 = k}) = m=1 πs¯m (λm s¯ (1 1)) λs¯ (k), respectively. When T ≥ 8, we can relax the condition |S| ≥ M − 1 of Proposition 6 by m s), λm applying the approach of Proposition 2. Define λ∗m s¯1 (s1 ) = s¯ (s) = Q (s|¯ m Qm (¯s|s1 )Qm (s1 |¯s), and λs¯2 (s1 s2 ) = Qm (¯s|s1 )Qm (s1 |s2 )Qm (s2 |¯s), and similarly m define λm s¯l (s1 sl ) for l ≥ 3 as an (l + 1)-variate product of Q (s |s)’s of l m m m l the form Q (¯s|s1 ) · · · Q (sl−1 |sl )Q (sl |¯s) for {st }t=1 ∈ S . PROPOSITION 7: Suppose Assumption 2 holds. Assume T ≥ 8 and is even, and define u = (T − 4)/2. Suppose that S = {1 2 |S|} and further define ⎡ 1 ⎤ ⎡ ⎤ λs¯1 (1) · · · λ1s¯1 (|S|) 1 ⎢ ⎥ Λ˜ 0 = ⎣ ⎦ Λ˜ 1 = ⎣ ⎦ M M 1 λs¯1 (1) · · · λs¯1 (|S|) For l = 2 u, define Λ˜ l to be a matrix whose elements consists of λm s¯l (s1 sl ), covering all possible unordered combinations (with replacement) of (s1 sl ) from S l . For example, ⎡ 1 λs¯2 (1 1) · · · λ1s¯2 (1 |S|) ⎢ =⎣ Λ˜ 2 |S|+1 (M×( 2 )) M λM (1 1) · · · λ (1 |S|) s¯2 s¯2 ⎤ λ1s¯2 (2 2) · · · λ1s¯2 (2 |S|) · · · λ1s¯2 (|S| |S|) ⎥ ⎦ · · · λM · · · λM λM s¯2 (2 2) s¯2 (2 |S|) s¯2 (|S| |S|) u |S|+l−1 Define an M × l=0 matrix Λ˜ as Λ˜ = [Λ˜ 0 Λ˜ 1 Λ˜ 2 Λ˜ u ] and define L s¯ l to be a M × M matrix consisting of M columns from Λ˜ but with the first column unchanged. u Suppose (a) l=0 |S|+l−1 ≥ M, (b) q∗m (¯s) > 0 for all m, (c) L s¯ is nonsingular, l ∗m ∗n and (d) there exists k ∈ S such that λ∗m s¯ (k) > 0 for all m and λs¯ (k) = λs¯ (k) |S| l u ∗m M for any m = n. Then {πs¯m {λm s¯l (s1 sl ) : (s1 sl ) ∈ S }l=1 {λs¯ (s)}s=1 }m=1 is T T T uniquely determined from {P({st }t=1 ) : {st }t=1 ∈ (A × X) }. The identification of the primitive parameters π m , p∗m (a x), f m (x |x a), P (a|x) follows from Remark 5(ii) and (iii). m
EXAMPLE 5 —An Example for Proposition 7: Browning and Carro (2007, Section 4) considered a stationary first-order Markov chain model of ait ∈
MODELS OF DYNAMIC DISCRETE CHOICES
159
{0 1} without covariates and showed that their model is not nonparametrically identified when T = 3 and M = 9. In our notation, Browning and Carro’s model is written as P(a1 aT ) =
M
π m p∗m (a1 )
m=1
T
P m (at |at−1 )
t=2
Note s = a because there are no covariates. If T = 8, we can identify M = u that 2+l−1 = 6 types, provided that, for s¯ = {0 1}, p∗m (¯s) > 0 for all m, L s¯ is l=0 l nonsingular, and P m (1|¯s) = P n (1|¯s) for any m = n. Here, L s¯ is given by ⎡ ⎤ 1 λ1s¯1 (0) λ1s¯1 (1) λ1s¯2 (0 0) λ1s¯2 (0 1) λ1s¯2 (1 1) ⎢ ⎥ L s¯ = ⎣ ⎦ λM λM λM λM 1 λM s¯1 (0) s¯1 (1) s¯2 (0 0) s¯2 (0 1) s¯2 (1 1) and the elements of L 0 are given by, for example, m m λm 01 (0) = P (0|0)P (0|0)
m m m λm 02 (1 1) = P (0|1)P (1|1)P (1|0)
The factorization equations that correspond to (18) are Ps¯ = (L s¯ ) Vs¯ L s¯ and Ps¯ k = (L s¯ ) Dk|¯s Vs¯ L s¯ , where V˜s¯ and Dk|¯s are defined as before. Ps¯ and Ps¯ k are identifiable from the data, and we can construct V˜s¯ , Dk|¯s , and L s¯ from these factorization equations. Similarly, if T = 10 12 14, then the maximum number of identifiable types by Proposition 7 is 10 15 21, respectively. The following example demonstrates that nonparametric identification of component distributions may help us understand the identification of parametric finite mixture models of dynamic discrete choices. EXAMPLE 6 —Identification of Models With Heterogeneous Coefficients: Consider the model of Example 1. For an individual who belongs to type m, P m (at = 1|xt at−1 ) = Φ(xt βm + ρm at−1 ) and the initial observation, (a1 x1 ), is randomly drawn from p∗m (a1 x1 ) while the transition function of xt is given by f m (xt |xt−1 at−1 ). If the conditions in Proposition 6 including T ≥ 6, |S| ≥ M − 1, and the rank of Ls¯ are satisfied, then p∗m (a1 x1 ), f m (xt |xt−1 at−1 ), and P m (at = 1|xt at−1 ) are identified for all m. Once P m (at = 1|xt at−1 ) is identified, taking an inverse mapping gives xt βm + ρm at−1 = Φ−1 (P m (at = 1|xt at−1 )). Evaluating this at all the points in A × X gives a system of |A||X| linear equations with dim(βm ) + 1 unknown parameters (βm ρm ), and solving this system for (βm ρm ) identifies (βm ρm ). m m For instance, consider a model P m (at = 1|xt at−1 ) = Φ(βm 0 + β1 xt + ρ at−1 ) m with A = X = {0 1}. If P (at = 1|x a) is identified for all (a x) ∈ A × X, then
160
H. KASAHARA AND K. SHIMOTSU
m m the type-specific coefficient (βm 0 β1 ρ ) is identified as the unique solution to the linear system ⎤ ⎡ −1 m ⎡ ⎤ Φ (P (at = 1|0 0)) 1 0 0 ⎡ βm ⎤ 0 ⎢ 1 1 0 ⎥ ⎣ m ⎦ ⎢ Φ−1 (P m (at = 1|0 1)) ⎥ ⎣ 1 0 1 ⎦ β1 = ⎣ Φ−1 (P m (a = 1|1 0)) ⎦ t ρm Φ−1 (P m (a = 1|1 1)) 1 1 1 t
The next example shows that the degree of underidentification in structural dynamic models with unobserved heterogeneity can be reduced to that in models without unobserved heterogeneity. Furthermore, a researcher can now apply various two-step estimators for structural models developed by Hotz and Miller (1993) (and others listed in the Introduction) to models with unobserved heterogeneity since, with our identification results, one can obtain an initial nonparametric consistent estimate of type-specific component distributions. Kasahara and Shimotsu (2008a) provided an example of such an application.8 EXAMPLE 7 —Dynamic Discrete Games (Aguirregabiria and Mira (2007, Section 3.5)): Consider the model of dynamic discrete games with unobserved market characteristics studied by Aguirregabiria and Mira (2007, Section 3.5). There are N ex ante identical “global” firms competing in H local markets. There are M market types and each market’s type is common knowledge to all firms, but unknown to a researcher. In market h ∈{1 2 H}, firm i maximizes the expected discounted sum of profits ∞ E[ s=t βs−t {Πi (xhs ahs ahs−1 ; θh ) + εhis (ahis )}|xht aht aht−1 ; θh ], where xht is a state variable that is common knowledge for all firms, θh ∈ {1 2 M} is the type attribute of market h, ahs = (ah1s ahNs ) is the vector of firms’ decisions, and εhit (ahit ) is a state variable that is private information to firm i. The profit function may depend on, for example, the past entry/exit decision of firms. The researcher observes xht and aht , but neither θh nor εhit . There is no interaction across different markets. Let a−1 denote the vector of firms’ decision in the preceding period. Assume that the εi ’s are independent from x and independent and identically distributed (i.i.d.) across firms. Let σ ∗ (θh ) = {σi∗ (x a−1 εi ; θh ) : i = 1 N} denote a set of strategy functions in a stationary Markov perfect equilibrium (MPE). Then, theequilibrium conditional choice probabilities are given ∗ by Piσ (ai |x a−1 ; θh ) = 1{ai = σi∗ (x a−1 εi ; θh )}g(εi ) dεi , where g(εi ) is the density function for ε = {ε(a) : a ∈ A}. A MPE induces a transition function ∗ of x, which we denote by f σ (xt |xt−1 a−1 ; θh ). 8 Kasahara and Shimotsu (2008a) showed that in structural discrete Markov decision models with unobserved heterogeneity, it is possible to obtain an estimator that is higher-order equivalent to the maximum likelihood estimator (MLE) by iterating the nested pseudo-likelihood (NPL) algorithm of Aguirregabiria and Mira (2002) sufficiently many, but finite times.
MODELS OF DYNAMIC DISCRETE CHOICES
161
Suppose that panel data {{aht xht }Tt=1 }H h=1 are available. As in Aguirregabiria and Mira (2007), consider the case where H → ∞ with N and T fixed. The initial distribution of (a x) differs across market types and is given by p∗m (a x). N σ ∗ Let P m (aht |xht aht−1 ) = and f m (xht |xht−1 i=1 Pi (ahit |xht aht−1 ; m) σ∗ aht−1 ) = f (xht |xht−1 aht−1 ; m). Then the likelihood function for market h becomes a mixture across different unobserved market types, P({aht xht }Tt=1 ) =
M m=1
m
∗m
π p (ah1 xh1 )
T
P m (aht |xht aht−1 )f m (xht |xht−1 aht−1 )
t=2
for which Propositions 6 and 7 are applicable. EXAMPLE 8—An Empirical Example of Aguirregabiria and Mira (2007, Section 5): Aguirregabiria and Mira (2007, Section 5) considered an empirical model of entry and exit in local retail markets based on the model of Example 7. Each market is indexed by h. The firms’ profits depend on the logarithm of the market size, xht , and their current and past entry/exit decisions, aht and aht−1 . The profit of a nonactive firm is zero. When active, firm i’s profit is Πio (xht aht aht−1 ) + ωh + εhit (ahit ), where the function Πio is common across all the markets and εhit is i.i.d. across markets. The parameter ωh captures the unobserved market characteristics and has a discrete distribution with 21 points of support. The logarithm of market size follows an exogenous firstorder Markov process. Their panel covers 6 years at annual frequency (i.e., T = 6). This satisfies the requirement for T in Proposition 6. Given the nonlinear nature of the dynamic games models, the rank condition on the Ls¯ matrix in Proposition 6 is likely to be satisfied. In their specification however, the transition function for xht , fh (xht |xht−1 ), is market-specific, so that the number of types is equal to the number of markets. Consequently, Proposition 6 does not apply to this case, and the type-specific conditional choice probabilities may not be nonparametrically identified.9 If we limit the number of types for transition functions, then we may apply Proposition 6 to identify the type-specific conditional choice probabilities and the type-specific transition probabilities. For the mth type market, the joint conditional choice probabilities across all firms are P m (aht |xht aht−1 ) = 9 Aguirregabiria and Mira estimated the market-specific transition function using 14 years of data on market size from other data sources. Given the relatively long time length, the identification of market-specific transition functions may come from the time variation of each market. When the transition function is market-specific, however, the conditional choice probabilities also become market-specific, leading to the incidental parameter problem. Even if the market-specific transition function is known, nonparametrically identifying the market-specific conditional choice probabilities is not possible given a short panel.
162
H. KASAHARA AND K. SHIMOTSU
N
Pi (ahit |xht aht−1 ; θm ), where aht = (ah1t ahNt ) . The market size is discretized with 10 support points (i.e., |X| = 10) and A = {0 1}N . Consequently, the size of the state space of sht = (aht xht ) for this model is |A||X| = 2N × 10, and we may identify up to M = 2N × 10 + 1 types. For instance, their Table VI reports that 63.5 percent of markets have no less than five potential firms (i.e., N ≥ 5) in the restaurant industry. Hence, M = 25 × 10 + 1 = 321 market types can be identified for these markets.10 i=1
The following proposition extends Proposition 3 for identification of M under Assumption 2. Because of the state dependence, the required panel length becomes T = 5. We omit the proof because it is essentially the same as that of Proposition 3. PROPOSITION 8: Suppose that Assumption 2 holds. Assume T ≥ 5 and S = {1 |S|}. Fix s1 = s3 = s5 = s¯ ∈ S. Define, for s s ∈ S, Ps¯ (s) = P(s2 = s s1 = s3 = s¯) Ps¯ (s s ) = P (s2 s4 ) = (s s ) s1 = s3 = s5 = s¯ and define a (|S| + 1) × (|S| + 1) matrix ⎡ 1 P (1) ··· s¯
⎢ Ps¯ (1) Ps¯∗ = ⎢ ⎣ Ps¯ (|S|)
Ps¯ (1 1) P˜s¯ (|S| 1)
··· ···
Ps¯ (|S|) Ps¯ (1 |S|)
⎤ ⎥ ⎥ ⎦
Ps¯ (|S| |S|)
Suppose q∗m (¯s) > 0 for all m. Then M ≥ rank(Ps¯∗ ). Furthermore, if the matrix L∗s¯ defined below has rank M, then M = rank(Ps¯∗ ): ⎡ ⎤ 1 λ1s¯ (1) · · · λ1s¯ (|S|) ⎢ ⎥ L∗s¯ = ⎣ ⎦ (M×(|S|+1))
1 λM s¯ (1)
···
λM s¯ (|S|)
In some applications, the model has two types of covariates, zt and xt , where the transition function of xt depends on types, while the transition function of zt is common across types. In such a case, we may use the variation of zt as a main source of identification and relax the requirement on T in Proposition 6. We impose an assumption analogous to Assumption 2, as well as the conditional independence assumption on the transition function of (x z ): 10 Even if the type-specific conditional choice probabilities are nonparametrically identified, it is generally not possible to nonparametrically identify the primitive objects such as the discount factor β, profit functions Πi , and the distribution of shocks, εiht , in structural dynamic models (Rust (1994), Magnac and Thesmar (2002)).
MODELS OF DYNAMIC DISCRETE CHOICES
163
ASSUMPTION 3: (a) The choice probability of at does not depend on time and is independent of zt−1 . (b) The transition function of (xt zt ) conditional m on {xτ zτ aτ }t−1 τ=1 takes the form g(zt |xt−1 zt−1 at−1 )f (xt |xt−1 at−1 ) for all t. m (c) f (x |x a) > 0 for all (x x a) ∈ X × X × A and g(z |x z a) > 0 for all (z x z a) ∈ Z × X × Z × A and for m = 1 M. Under Assumption 3, consider a model P({at xt zt }Tt=1 ) =
M
π m p∗m (x1 z1 a1 )
m=1
T
g(zt |xt−1 zt−1 at−1 )f m (xt |xt−1 at−1 )
t=2
× P m (at |xt xt−1 at−1 zt ) Assuming g(zt |xt−1 zt−1 at−1 ) is known and defining st = (at xt ), q˜ ∗m (s1 z1 ) = p∗m (x1 z1 a1 ), and Q˜ m (st |st−1 zt ) = f m (xt |xt−1 at−1 )P m (at |xt xt−1 at−1 zt ), we can write this equation as (29)
˜ t zt }T ) = P({s t=1
P({at xt zt }Tt=1 ) T
g(zt |xt−1 zt−1 at−1 )
t=2
=
M
π m q˜ ∗m (s1 z1 )
m=1
T
Q˜ m (st |st−1 zt )
t=2
We fix the value of {st }Tt=1 and use the “independent” variation of zt to identify unobserved types. The next proposition provides a sufficient condition for nonparametric identification of the model (29). Define, for s¯ ∈ S and h ξ ∈ Z, π˜ s¯mh = π m q˜ ∗m (¯s h)
˜ m s|¯s ξ) λ˜ m s¯ (ξ) = Q (¯
PROPOSITION 9: Suppose that Assumption 3 holds and assume T ≥ 4. Define ⎡ ⎤ 1 λ˜ 1s¯ (ξ1 ) · · · λ˜ 1s¯ (ξM−1 ) ⎢ ⎥ L¯ s¯ = ⎣ ⎦ (M×M)
1
λ˜ M s¯ (ξ1 )
···
λ˜ M s¯ (ξM−1 )
Suppose that q˜ ∗m (¯s h) > 0 for all m, there exist some {ξ1 ξM−1 } such that L¯ s¯ is nonsingular, and there exist (r k) ∈ S × Z such that Q˜ m (r|¯s k) > 0 for all m and Q˜ m (r|¯s k) = Q˜ n (r|¯s k) for any m = n. Then {π˜ s¯mh {λm s¯ (ξ)}ξ∈Z , m M T ˜ t zt } ) : {st zt }T ∈ {Q˜ (s|¯s ξ)}(sξ)∈S×Z }m=1 is uniquely determined from {P({s t=1 t=1 T (S × Z) }.
164
H. KASAHARA AND K. SHIMOTSU
We may identify the primitive parameters π m , p∗m (a x), f m (x |x a), P (a|x) using an argument analogous to those of Remark 5(ii) and (iii). The requirement of T = 4 in Proposition 9 is weaker than that of T = 6 in Proposition 6 because the variation of zt , rather than (xt at ), is used as a main source of identification. When T > 4, we may apply the argument of Proposition 2 to relax the sufficient condition for identification in Proposition 9, but we do not pursue it here; Proposition 2 provides a similar result. m
3.3. Limited Transition Pattern This section analyzes the identification condition of the baseline model when Assumption 1(c) is relaxed. In some applications, the transition pattern of x is limited, as not all x ∈ X are reachable with a positive probability. In such instances, the set of sequences {at xt }Tt=1 that can be realized with a positive probability also becomes limited and the number of restrictions from a set of the submodels falls, making identification more difficult. EXAMPLE 9—Bus Engine Replacement Model (Rust (1987)): Suppose a ∈ {0 1} is the replacement decision for a bus engine, where a = 1 corresponds to replacing a bus engine. Let x denote the mileage of a bus engine with X = {1 2 }. The transition function of xt is ⎧ θf1 for xt+1 = (1 − at )xt + at , ⎪ ⎪ ⎨ for xt+1 = (1 − at )xt + at + 1, θf2 f (xt+1 |xt at ; θ) = ⎪ − θ for xt+1 = (1 − at )xt + at + 2, 1 − θ f1 f2 ⎪ ⎩ 0 otherwise, and not all x ∈ X can be realized from (x a). Henceforth, we assume the transition function of x is stationary and takes the form f (x |x a) to simplify the exposition. If f (x |x a) = 0 for some (x x a) and not all x ∈ X can be reached from (a x), then some values of ˜ t xt }T ) in (9) and its lower{at xt } are never realized. For such values, P({a t=1 dimensional submodels in (10) and (11) are not well defined. Hence, their restrictions cannot be used for identification. Thus, we fix the values of (a1 x1 ) and (aτ xτ ), and focus on the values of (at xt ) that are realizable between (a1 x1 ) and (aτ xτ ). The difference in response patterns between (a1 x1 ) and (aτ xτ ) provides a source of identification. To fix the idea, assume T = 4, and fix at = 0 for all t, x1 = h, and xτ = k. Of course, it is possible to choose different sequences of {at }Tt=1 . Let Bh and C h be subsets of X, of which elements are realizable between (a1 x1 ) and (aτ xτ ). We use the variations of x within Bh and C h as a source of identification. Define, for h ξ ∈ X, (30)
π˜ hm = π m p∗m (a1 = 0 x1 = h) and
m λ˜ m ξ = P (a = 0|x = ξ)
MODELS OF DYNAMIC DISCRETE CHOICES
165
˜ k = diag(λ˜ 1 λ˜ M ). We identify V˜h , D ˜ k, and V˜h = diag(π˜ h1 π˜ hM ) and D k k m and λ˜ ξ ’s from the factorization equations corresponding to (18): (31)
P h = L˜ b V˜h L˜ c
and
˜ k V˜h L˜ c Pkh = L˜ b D
where L˜ b and L˜ c are defined analogously to L in (13), but using λ˜ m ξ , and with ξ ∈ Bh and ξ ∈ C h , respectively. As we discuss below, we choose Bh , C h , and k so that P h and Pkh are identifiable from the data. Each equation of these factorization equations (31) represents a submodel in (9) and (10) for a sequence of at ’s and xt ’s that belongs to one of the sets (32)
A1 = {x1 = h (x2 x3 ) ∈ Bh × C h x4 = k; at = 0 for all t} A2 = {x1 = h x2 ∈ Bh x3 = k; at = 0 for all t} A3 = {x1 = h x2 ∈ C h x3 = k; at = 0 for all t} A4 = {x1 = h x2 = k; at = 0 for all t}
For instance, a submodel for a sequence q1 ∈ A1 in (32) is ˜ 1) = P(q =
P(q1 ) f (k|x3 0)f (x3 |x2 0)f (x2 |h 0) M
π m p∗m (h 0)P m (0|x2 )P m (0|x3 )P m (0|k)
m=1
for (x2 x3 ) ∈ Bh × C h ˜ k V˜h L˜ c in (31). which represents one of the equations of Pkh = L˜ b D For all the submodels implied by (31) to provide identifying restrictions, all the sequences of xt ’s in A1 –A4 in (32) must have positive probability; otherwise, some elements of P h and Pkh in (31) cannot be constructed from the data, and our identification strategy fails. This requires that all the points in Bh must be reachable from h, while all the points in C h must be reachable from h and all the points in Bh . Finally, k must be reachable from h and all the points in Bh and C h . EXAMPLE 9 —Continued: In Example 9, assume the initial distribution p∗m (x a) is defined as the type-specific stationary distribution. Set at = 0 for t = 1 4 and x1 = h. Choose Bh = {h h + 1} and C h = {h + 1 h + 2}, and k = h + 2. For this choice of Bh , C h , and k, the corresponding transition probabilities are nonzero, and we may construct all the elements of P h and Pkh in (31) from the observables. For each h ∈ X, these submodels provide 4 + 3 + 1 = 8 restrictions for identification.
166
H. KASAHARA AND K. SHIMOTSU
We now state the restrictions on Bh and C h formally. First we develop useful notation. For a singleton {x} ⊂ X, let Γ (a {x}) = {x ∈ X : f (x |x a) > 0} denote a set of x ∈ X that can be reached from (a x) in the next period with a positive probability. For a subset W ⊆ X, define Γ (a W ) as the intersection of Γ (a {x})’s across all x’s in W : Γ (a W ) = x∈W Γ (a {x}). We summarize the assumptions of this subsection including the restrictions on Bh and C h : ASSUMPTION 4: (a) The choice probability of at does not depend on time. (b) The choice probability of at is independent of the lagged variable (xt−1 at−1 ) conditional on xt . (c) P m (a|x) > 0 for all (a x) ∈ A × X and m = 1 M. h h (d) ftm (xt |{xτ aτ }t−1 τ=1 ) = f (xt |xt−1 at−1 ) for all m. (e) h k ∈ X, B and C sat∗m isfy p (a1 = 0 x1 = h) > 0 for all m, and Bh ⊆ Γ (0 {h})
C h ⊆ Γ (0 Bh ) ∩ Γ (0 {h})
{k} ⊆ Γ (0 C h ) ∩ Γ (0 Bh ) ∩ Γ (0 {h}) Assumption 4(a) and (b) are identical to Assumption 1(a) and (b). Assumption 4(c) is necessary for the submodels to be well defined. Assumption 4(d) strengthens Assumption 1(d) by imposing stationarity and a first-order Markov property. It may be relaxed, but doing so would add substantial notational complexity. Assumption 4(e) guarantees that all the sequences we consider in the subsets in (32) have nonzero probability. Note that the choice of C h is affected by how Bh is chosen. If Assumption 1(c) holds, it is possible to set Bh = C h = X. The next proposition provides a sufficient condition for identification under Assumption 4. PROPOSITION 10: Suppose that Assumption 4 holds T = 4, and |Bh |, |C h | ≥ b c } and {ξ1c ξM−1 } be elements of Bh and C h , respecM − 1. Let {ξ1b ξM−1 m m tively. Define π˜ h and λ˜ ξ as in (30), and define ⎤ ⎡ 1 λ˜ 1ξb λ˜ 1ξb · · · λ˜ 1ξb 1 2 M−1 ⎢ ⎥ ⎥ L˜ b = ⎢ ⎦ ⎣ (M×M) 1 λ˜ M · · · λ˜ M λ˜ M b ξ1b ξ2b ξM−1 ⎡ 1 λ˜ 1 λ˜ 1 · · · λ˜ 1 ⎤ L˜ c (M×M)
⎢ = ⎣
ξ1c
˜ 1 λM ξc
1
ξ2c
˜λMc ξ
2
c ξM−1
···
⎥ ⎦
λ˜ M ξc
M−1
b } and {ξ1c Suppose that L˜ b and L˜ c are nonsingular for some {ξ1b ξM−1 c ˜m ˜n }, and that λ˜ m ˜ hm λ˜ m ξM−1 k > 0 for all m and λk = λk for any m = n. Then {π ξ :ξ ∈ h h M T T ˜ t xt } ) : {at xt } ∈ (A × X)T }. B ∪C } is uniquely determined from {P({a m=1
t=1
t=1
MODELS OF DYNAMIC DISCRETE CHOICES
167
Assuming that all the values of x can be realized in the initial period, we may repeat the above argument for all possible values of x1 to identify λ˜ m ξ for any ξ ∈ h∈X Bh . Furthermore, we can repeat the argument for different sequences of {at }4t=1 to increase the identifiable elements of P m (a|x). For instance, by choosing Bh = Γ (a {h}), λ˜ m l is identified for all l ∈ X if the union of Γ (a {h}) across different (a h) ∈ A × X includes all the elements of X so that X = Γ (a {h}). This is a weak condition and is satisfied if X is an ergodic (ah)∈A×X set. However, setting Bh = Γ (a {h}) may lead to a small number of identifiable types. EXAMPLE 9—Continued: Setting at = 0 for t = 1 4, we have Γ (0 {h}) = {h h + 1 h + 2} for any h ∈ X. To satisfy Assumption 4(e), choose Bh = {h h + 1}, C h = {h + 1 h + 2}, and k = h + 2. If the other assumptions of Proposition 10 are satisfied, we can identify M = 3 types. From the factoriza˜ k , L˜ b , and L˜ c , and idention equations (31), we can uniquely determine V˜h , D m ∗m m tify {π p (0 x) P (0|x) : x = h h + 1 h + 2}m=123 . Repeating for all h ∈ X, we identify P m (a|x) for all (a x) ∈ A × X. We then identify p∗m (x a) using P m (a|x), f (x |x a), and the fixed point constraint, while π m is determined as π m p∗m (0 x)/p∗m (0 x). The sufficient condition of Proposition 10 does not allow one to identify many types when the size of Bh or C h is small. It is possible to identify more types when we can find a subset D of X that is reachable from itself, namely D ⊆ Γ (0 D). For example, if the transition pattern is such that Γ (0 {x}) = {x − 2 x − 1 x x + 1 x + 2} for some x ∈ X, then the set {x − 1 x x + 1} serves as D. In such cases, we can apply the logic of Proposition 2 to identify many types if T ≥ 5. ASSUMPTION 5: (a) Assumptions 4(a)–(d) hold. (b) A subset D of X satisfies D ⊆ Γ (0 D). m ∗m Set D = {d1 d|D| }, and define λ∗m d = p ((a x) = (1 d)) and λd = P (a = 1|x = d) for d ∈ D. Under Assumption 5, replacing X with D and simply repeating the proof of Proposition 2 gives the following proposition: m
PROPOSITION 11: Suppose Assumption 5 holds. Assume T ≥ 5 is odd and define u = (T − 1)/2. Define Λr , r = 0 u, analogously to Proposition 2 except u |D|+l−1 ∗m m m (X λ∗m λ ) is replaced with (D λ λ ). Define an M × ( ) matrix dj dj ξj ξj l=0 l Λ Λ ]. Λ as Λ = [Λ0 Λ 1 2 u u Suppose (a) l=0 |D|+l−1 ≥ M, (b) we can construct a nonsingular M × M l matrix L by setting its first column as Λ0 and choosing the other M − 1 columns from the columns of Λ but Λ0 , and (c) there exists dk ∈ D such that λ∗m dk > 0 ∗n ∗m ∗m m |D| M m for all m and λdk = λdk for any m = n. Then {π {λdj λdj }j=1 }m=1 is uniquely ˜ t xt }T ) : {at xt }T ∈ (A × X)T }. determined from {P({a t=1
t=1
168
H. KASAHARA AND K. SHIMOTSU
4 if |D| = 3 and T = 5, the number of identifiable types becomes 2For3example, + + 2 = 10. Identifying more types is also possible when the model has 0 1 an additional covariate zt whose transition pattern is not limited and there is a ¯ > 0 for some sequence of at . Then, for state x¯ such that P(x1 = · · · = xT = x) ¯ we can use the variation of zt and apply Proposition 9. This increases x = x, the number of identifiable types to |Z| + 1. 4. CONCLUDING REMARK This paper studies dynamic discrete choice models with unobserved heterogeneity that is represented in the form of finite mixtures. It provides sufficient conditions under which such models are identified without parametric distributional assumptions. While we emphasize that the variation in the covariate and in time provides important identifying information, our identification approach does require assumptions on the Markov property, stationarity, and type-invariance in transition processes. To clarify our identification results, consider a general nonstationary finite mixture model of dynamic discrete choices: (33)
P({at xt }Tt=1 ) =
M
π m p∗m (x1 a1 )
m=1
T
m t−1 ftm (xt |{xτ aτ }t−1 τ=1 )Pt (at |xt {xτ aτ }τ=1 )
t=2
Such a general mixture model (33) cannot be nonparametrically identified without imposing further restrictions.11 One possible nonparametric restriction is a first-order Markovian assumption on (xt at ), that yields a less general nonstationary model: (34)
P({at xt }Tt=1 ) =
M m=1
m
∗m
π p (x1 a1 )
T
ftm (xt |xt−1 at−1 )Ptm (at |xt xt−1 at−1 )
t=2
We do not know whether this model is nonparametrically identified without additional assumptions. Section 3.1 shows that the identification of the nonstaThe model (33) is equivalent to a mixture model P({at xt }Tt=1 ) = because it is always to possible to decompose 11
P m ({at xt }Tt=1 ) = p∗m (x1 a1 )
T
M m=1
π m P m ({at xt }Tt=1 ),
t−1 m ftm (xt |{xτ aτ }t−1 τ=1 )Pt (at |xt {xτ aτ }τ=1 )
t=2
The number of restrictions implied by P({at xt }Tt=1 ) is (|A||X|)T − 1, while the number of un T m m T knowns in M m=1 π P ({at xt }t=1 ) is M − 1 + M((|A||X|) − 1).
MODELS OF DYNAMIC DISCRETE CHOICES
169
tionary model (34) is possible under the assumptions of type-invariant transition processes and conditional independence of discrete choices. In Section 3.2, we provide identification results under the stationarity assumption on the transition function and choice probabilities in (34). Relaxing these identifying assumptions as well as investigating identifiability, or perhaps nonidentifiability, of finite mixture model (34) is an important future research area. Estimation and inference on the number of components (types), M, is an important topic because of the lack of guidance from economic theory. It is known that the likelihood ratio statistic has a nonstandard limiting distribution when applied to testing the number of components of a mixture model (see, for example, Liu and Shao (2003)). Leroux (1992) considered a maximumpenalized-likelihood estimator for the number of components, which includes the Akaike information criterion and Bayesian information criterion as a special case. McLachlan and Peel (2000, Chapter 6) surveyed the methods for determining the number of components in parametric mixture models. To our best knowledge, all of these existing methods assume that the component distributions belong to a parametric family. Developing a method for testing and selecting the number of components without imposing any parametric assumption warrants further research. A statistical test of the number of components may be possible by testing the rank of matrix P ∗ in Proposition 3. When the covariate has a large number of support points, we may test the number of components by testing a version of matrix P in (17) across different partitions of X. In Kasahara and Shimotsu (2008b), we pursued this idea, and proposed a selection procedure for the number of components by sequentially testing the rank of matrices. APPENDIX: PROOFS PROOF OF PROPOSITION 1 AND COROLLARY 1: Define V = diag(π 1 ∗M π ) and Dk = diag(λ∗1 k λk ) as in (13). Define P and Pk as in (17). Then P and Pk are expressed as (see (14)–(16)) M
P = L V L
Pk = L V Dk L
We now uniquely determine L, V , and Dk from P and Pk constructively. Since L is nonsingular, we can construct a matrix Ak = P −1 Pk = L−1 Dk L. Because Ak L−1 = L−1 Dk , the eigenvalues of Ak determine the diagonal elements of Dk while the right eigenvectors of Ak determine the columns of L−1 up to multiplicative constants; denote the right eigenvectors of Ak by L−1 K, where K is some diagonal matrix. Now we can determine V K from the first row of PL−1 K because PL−1 K = L V K and the first row of L is a vector of ones. Then L is determined uniquely by L = (PL−1 K)(V K)−1 = (L V K)(V K)−1 .
170
H. KASAHARA AND K. SHIMOTSU
Having obtained L , we may determine V from the first column of (L )−1 P because (L )−1 P = V L and the first column of L is a vector of ones. Therefore, M−1 M we identify {π m {λm ξj }j=1 }m=1 as the elements of V and L. Once V and L are determined, we can uniquely determine Dζ = diag(λ∗1 ζ ∗M λζ ) for any ζ ∈ X by constructing Pζ in the same way as Pk and using the relationship Dζ = (L V )−1 Pζ L−1 . Furthermore, for arbitrary ζ ξj ∈ X, evaluate Fx2 x3 , Fx2 , and Fx3 defined in (15) and (16) at (x2 x3 ) = (ζ ξj ), and define ⎤ ⎡ 1 λ1ζ
1 Fξ1 FξM−1 ⎥ ⎢ ζ ζ ⎦ (35) P = L =⎣ (M×2) (2×M) Fζ Fζξ1 FζξM−1 M 1 λζ Since P ζ = (Lζ ) V L, we can uniquely determine (Lζ ) = P ζ (V L)−1 . Therefore, M m M {λ∗m ζ }m=1 and {λζ }m=1 are identified for any ζ ∈ X. This completes the proof of Proposition 1, and Corollary 1 follows immediately. Q.E.D. PROOF OF PROPOSITION 2: The proof is similar to the proof of Proposition 1. Let T = (τ2 τp ), 2 ≤ p ≤ T , be a subset of {2 T }. Let X (T ) be a subset of {xt }Tt=2 with t ∈ T . For example, if T = {2 4 6}, then X (T ) = ˜ t xt }T ), integrating out (at xt ) if t ∈ {x2 x4 x6 }. Starting from P({a / T , and t=1 evaluating it at (a1 x1 ) = (1 k) and at = 1 for t ∈ T gives a “marginal” M ∗ m ∗m m ˜ Fk X (T ) = P({a1 x1 } = {1 k} {1 xt }τ∈T ) = m=1 π λk t∈T λxt . For example, M ∗ m ∗m m m m if T = {2 4 6}, then Fk X (T ) = m=1 π λk λx2 λx4 λx6 . Integrating out (a1 x1 ) ˜ xt }τ∈T ) = additionally and proceeding in a similar way gives FX (T ) = P({1 M m m t∈T λxt . m=1 π ∗M Define V = diag(π 1 π M ) and Dk = diag(λ∗1 k λk ) as in (13). Define
P = (L ) V L and Pk = (L ) V Dk L . Then the elements of P take the form M m m |T | . m=1 π t∈T λxt and can be expressed as FX (T ) for some T and {xt }t∈T ∈ X
∗ Similarly, the elements of Pk can be expressed as FkX (T ) . For instance, if u = 3, T = 7, and both Λ and L are M × M, then P is given by ⎡
1 ⎢ F1 ⎢ ⎢ ⎢ F ⎢ |X| ⎢ F 11 ⎢ ⎢ ⎢ ⎢ ⎢ F|X||X| ⎢ ⎢ F111 ⎢ ⎣ F|X||X||X|
F1
· · · F|X|
F11
···
F|X|11
F|X||X|
F111
···
F|X||X||X|
F|X||X||X||X|
F111|X||X||X|
F|X||X||X|11
···
F|X||X||X||X||X||X|
⎤ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎥ ⎦
MODELS OF DYNAMIC DISCRETE CHOICES
171
where the (i j)th element of P is Fσ , where σ consists of the combined subscripts of the (i 1)th and (1 j)th element of P . For example, the (|X| + 1 2)th ∗ and element of P is F|X|1 (= F1|X| ). Pk is given by replacing Fσ in P with Fkσ ∗ setting the (1 1)th element to Fk . Consequently, P and Pk can be computed from the distribution function of the observed data. By repeating the argument of the proof of Proposition 1, we determine L , V , and Dk uniquely from P and Pk first, and then Dζ = ∗M ζ
ζ ζ diag(λ∗1 ζ λζ ) and L for any ζ ∈ X from P , Pζ , L , and P , where L and P ζ are defined in (35). Q.E.D. PROOF OF PROPOSITION 3: Let V = diag(π 1 π M ). Then P ∗ = (L∗1 ) × V L∗2 . It follows that rank(P ∗ ) ≤ min{rank(L∗1 ) rank(L∗2 ) rank(V )}. Since rank(V ) = M, it follows that M ≥ rank(P ∗ ), where the inequality becomes strict when rank(L∗1 ) or rank(L∗2 ) is smaller than M. When rank(L∗1 ) = rank(L∗2 ) = M, multiplying both sides of P ∗ = (L∗1 ) V L∗2 from the right by (L∗2 ) (L∗2 (L∗2 ) )−1 gives P ∗ (L∗2 ) (L∗2 (L∗2 ) )−1 = (L∗1 ) V . There are M linearly independent columns in (L∗1 ) V , because (L∗1 ) has M linearly independent columns while V is a diagonal matrix with strictly positive elements. Therefore, rank(P ∗ (L∗2 ) (L∗2 (L∗2 ) )−1 ) = M. It follows that rank(P ∗ ) = M because M ≤ min{rank(P ∗ ) rank(L∗2 ) rank(L∗2 (L∗2 ) )−1 }, and rank(L∗2 ) = M imQ.E.D. ply rank(P ∗ ) ≥ M. PROOF OF PROPOSITION 4: The proof is similar to the proof of Proposition 1. Define Pt and Ptk analogously to P and Pk but with λx2 and λx3 replaced with λtxt and λt+1xt+1 in the definition of F· and F·∗ . Define V and Dk as before. Then Pt and Ptk are expressed as Pt = Lt V Lt+1 and Ptk = Lt V Dk Lt+1 . Since Lt and Lt+1 are nonsingular, we have Ak = Pt−1 Ptk = L−1 t+1 Dk Lt+1 . Be−1 cause Ak L−1 = L D , the eigenvalues of A determine the diagonal elek k t+1 t+1 ments of Dk while the right eigenvectors of Ak determine the columns of L−1 t+1 up to multiplicative constants; denote the right eigenvectors of Ak by L−1 t+1 K, where K is some diagonal matrix. Now we can determine V K from the first row −1 of Pt L−1 t+1 K because Pt Lt+1 K = Lt V K and the first row of Lt is a vector of ones. Then Lt is determined uniquely by Lt = (Lt V K)(V K)−1 . Having obtained Lt , we may determine V and Lt+1 from V Lt+1 = (Lt )−1 P because the first column of V Lt+1 equals the diagonal of V and Lt+1 = V −1 (V Lt+1 ). Therefore, we de λm }M−1 }M as elements of V , Lt , and Lt+1 . Once V , Lt termine {π m {λm tξt t+1ξt+1 j=1 m=1 j
j
∗M and Lt+1 are determined, we can uniquely determine Dζ = diag(λ∗1 ζ λζ ) for any ζ ∈ X by constructing Ptζ in the same way as Ptk and using the relationship Dζ = (Lt V )−1 Ptζ (Lt+1 )−1 . Furthermore, for arbitrary ζ ∈ X, define ⎤ ⎡ 1 λ1tζ ⎢ ⎥ Lζt = ⎣ ⎦ (M×2)
1
λM tζ
172
H. KASAHARA AND K. SHIMOTSU
Then Ptζ = (Lζt ) V Lt+1 is a function of the distribution function of the observable data, and we can uniquely determine (Lζt ) for 2 ≤ t ≤ T − 1 as Ptζ (V Lt+1 )−1 . For t = T , we can use the fact that (LT −1 ) V LζT is also a function of the distribution function of the observable data and proceed in the m M−1 same manner. Therefore, we can determine {λ∗m ζ λtζ }j=1 for any ζ ∈ X and 2 ≤ t ≤ T. Q.E.D. PROOF OF PROPOSITION 6: Without loss of generality, set T = 6. Integrating out st ’s backward from P({st }6t=1 ) and fixing s1 = s3 = s5 = s¯ gives the “marginals” F˜s∗2 s4 s6 =
M
m ∗m πs¯m λm s¯ (s2 )λs¯ (s4 )λs¯ (s6 )
F˜s∗2 s6 =
m=1
F˜s∗6 =
M
M
∗m πs¯m λm s¯ (s2 )λs¯ (s6 )
m=1
πs¯m λ∗m s¯ (s6 )
F˜s2 s4 =
m=1
F˜s2 =
M
M
m πs¯m λm s¯ (s2 )λs¯ (s4 )
m=1
πs¯m λm s¯ (s2 )
F˜ =
m=1
M
πs¯m
m=1
As in the proof of Proposition 1, evaluate these F˜·’s at s2 = ξ1 ξM−1 , s4 = ξ1 ξM−1 , and s6 = r, and arrange them into two M × M matrices: ⎡ ˜ ⎤ F F˜ξ1 ··· F˜ξM−1 ⎢ F˜ξ ··· F˜ξ1 ξM−1 ⎥ F˜ξ1 ξ1 ⎢ ⎥ 1 Ps¯ = ⎢ ⎥ ⎣ ⎦ F˜ξM−1 F˜ξM−1 ξ1 · · · F˜ξM−1 ξM−1 ⎡ ˜∗ ⎤ Fk ··· F˜ξ∗M−1 k F˜ξ∗1 k ⎢ ˜∗ ⎥ ⎢ Fξ1 k ··· F˜ξ∗1 ξM−1 k ⎥ F˜ξ∗1 ξ1 k ⎢ ⎥ Ps¯k = ⎢ ⎥ ⎣ ⎦ ∗ ∗ ∗ ˜ ˜ ˜ F ··· F F ξM−1 k
ξM−1 ξ1 k
ξM−1 ξM−1 k
∗M Define Vs¯ = diag(πs¯1 πs¯M ) and Dk|¯s = diag(λ∗1 s¯ (k) λs¯ (k)). Then Ps¯ and Ps¯k are expressed as Ps¯ = Ls¯ Vs¯ Ls¯ and Ps¯k = Ls¯ Vs¯ Dk|¯s Ls¯ . Repeating the argument of the proof of Proposition 1 shows that Ls¯ , Ls¯ , Vs¯ , and Dk|¯s are uniquely determined from Ps¯ and Ps¯k , and that Ds|¯s and λm s¯ (s) can be uniquely determined for any s ∈ S and m = 1 M. Q.E.D.
PROOF OF PROPOSITION 7: Define Vs¯ = diag(πs¯1 πs¯M ) and Dk|¯s = ∗M diag(λ∗1 s¯ (k) λs¯ (k)). Applying the argument of the proof of Proposition 6
MODELS OF DYNAMIC DISCRETE CHOICES
173
with Ls¯ replaced by L s¯ , we can identify L s¯ , Vs¯ , and Dk|¯s , and then Ds|¯s and λm s¯ (s) for any s ∈ S and m = 1 M. The stated result immediately follows. Q.E.D. PROOF OF PROPOSITION 9: The proof uses the logic of the proof of Proposition 6. Consider a sequence {st zt }4t=1 with (s1 s2 s3 s4 ) = (¯s s¯ s¯ r) and (z1 z4 ) = (h k). Summarize the value of s4 and z4 into ζ = (r k). For M ˜ m s k) and F˜ h = ˜m (z2 z3 ) ∈ Z 2 , define F˜zh∗2 z3 ζ = m=1 π˜ s¯mh λ˜ m s¯ (z2 )λs¯ (z3 )Q (r|¯ z2 z3 M M m ˜m m ˜m m h∗ m ˜ ˜ ˜ ˜ s¯h λs¯ (z2 )λs¯ (z3 ). Define Fz2 ζ = m=1 π˜ s¯h λs¯ (z2 )Q (r|¯s k), and define m=1 π F˜ζh∗ , F˜zh2 , and F˜ h analogously to the proof of Proposition 6. As in the proof of Proposition 6, arrange these marginals into two matrices ∗ replaced P¯ h and P¯ζh . P¯ h and P¯ζh are the same as Ps¯ and Ps¯k , but F˜· and F˜·k h h∗ ˜ ˜ with F· and F·ζ and subscripts are elements of Z instead of S. Define V˜s¯h = ˜ ζ|¯s = diag(Q˜ 1 (r|¯s k) Q˜ M (r|¯s k)). It then follows diag(π˜ s¯1h π˜ s¯Mh ) and D h h ˜ ζ|¯s L¯ s¯ . By repeating the argument of the that P¯ = L¯ s¯ V˜s¯ L¯ s¯ and P¯ζh = L¯ s¯ V˜s¯h D ˜ ζ|¯s from P¯ h proof of Proposition 1, we can uniquely determine L¯ s¯ , V˜s¯h , and D h ˜ (sz)|¯s for any (s z) ∈ S × Z. and P¯ζ , and, having determined L¯ s¯ , determine D Q.E.D. PROOF OF PROPOSITION 10: For (x2 x3 ) ∈ Bh × C h and xc ∈ Bh ∪ C h , deM M M ˜ m ˜ m h∗ ˜ m h∗ fine Fxh∗2 x3 k = m=1 π˜ hm λ˜ m ˜ hm λ˜ m ˜ hm λ˜ m k, x2 λx3 λk , Fxc k = xc λk , Fk = m=1 π m=1 π M M M m˜m ˜m m˜m m h h h Fx2 x3 = m=1 π˜ h λx2 λx3 , Fxc = m=1 π˜ h λxc , and F = m=1 π˜ h . They can be constructed from sequentially integrating out P({at xt }4t=1 ) backward and then dividing them by a product of f (xt |xt−1 0). Note that Assumption 4(b) guarantees f (xt |xt−1 0) > 0 for all xt and xt−1 in the subsets of X considered. As in the proof of Proposition 1, arrange these marginals into two matrices ∗ are replaced P h and Pkh . P h and Pkh are the same as P and Pk but F· and Fk· h∗ ˜ k = diag(λ˜ 1 λ˜ M ). By . Define V˜h = diag(π˜ h1 π˜ hM ) and D with F·h and F·k k k applying the argument in the proof of Proposition 4, we may show that L˜ b , L˜ c , ˜ k are uniquely determined from P({a ˜ t xt }4 ) and its marginals, and V˜h , and D t=1 m M ˜ then show that {λξ }m=1 is determined for ξ ∈ Bh ∪ C h . Q.E.D. REFERENCES AGUIRREGABIRIA, V. (2006): “Another Look at the Identification of Dynamic Discrete Decision Processes,” Unpublished Manuscript, University of Toronto. [139] AGUIRREGABIRIA, V., AND P. MIRA (2002): “Swapping the Nested Fixed Point Algorithm: A Class of Estimators for Discrete Markov Decision Models,” Econometrica, 70, 1519–1543. [138,160] (2007): “Sequential Estimation of Dynamic Discrete Games,” Econometrica, 75, 1–54. [138,139,160,161]
174
H. KASAHARA AND K. SHIMOTSU
ANDERSON, T. W. (1954): “On Estimation of Parameters in Latent Structure Analysis,” Psychometrika, 19, 1–10. [145] BAJARI, P., AND H. HONG (2006): “Semiparametric Estimation of a Dynamic Game of Incomplete Information,” Technical Working Paper 320, NBER. [138] BAJARI, P., C. L. BENKARD, AND J. LEVIN (2007): “Estimating Dynamic Models of Imperfect Competition,” Econometrica, 75, 1331–1370. [138] BLISCHKE, W. R. (1964): “Estimating the Parameters of Mixtures of Binomial Distributions,” Journal of the American Statistical Association, 59, 510–528. [149] BROWNING, M., AND J. CARRO (2007): “Heterogeneity and Microeconometrics Modelling,” in Advances in Economics and Econometrics, Theory and Applications: Ninth World Congress of the Econometric Society, Vol. 3, ed. by R. Blundell, W. Newey, and T. Persson. Cambridge, U.K.: Cambridge University Press, 47–74. [141,158] CAMERON, S. V., AND J. J. HECKMAN (1998): “Life Cycle Schooling and Dynamic Selection Bias: Models and Evidence for Five Cohorts of American Males,” Journal of Political Economy, 106, 262–333. [136] CHANDRA, S. (1977): “On the Mixtures of Probability Distributions,” Scandinavian Journal of Statistics, 4, 105–112. [144] CRAWFORD, G. S., AND M. SHUM (2005): “Uncertainty and Learning in Pharmaceutical Demand,” Econometrica, 73, 1137–1173. [135] ELBERS, C., AND G. RIDDER (1982): “True and Spurious Duration Dependence: The Identifiability of the Proportional Hazard Model,” Review of Economic Studies, 49, 403–409. [139] GEWEKE, J., AND M. KEANE (2001): “Computationally Intensive Methods for Integration in Econometrics,” in Handbook of Econometrics, Vol. 5, ed. by J. Heckman and E. Leamer. Amsterdam: North-Holland. [139] GIBSON, W. A. (1955): “An Extension of Anderson’s Solution for the Latent Structure Equations,” Psychometrika, 20, 69–73. [145] GOWRISANKARAN, G., M. F. MITCHELL, AND A. MORO (2005): “Why Do Incumbent Senators Win? Evidence From a Dynamic Selection Model,” Unpublished Manuscript, Washington University in St. Louis. [135] HALL, P., AND X.-H. ZHOU (2003): “Nonparametric Estimation of Component Distributions in a Multivariate Mixture,” Annals of Statistics, 31, 201–224. [136,142] HALL, P., A. NEEMAN, R. PAKYARI, AND R. ELMORE (2005): “Nonparametric Inference in Multivariate Mixtures,” Biometrika, 92, 667–678. [137] HECKMAN, J. J. (1981): “The Incidental Parameters Problem and the Problem of Initial Conditions in Estimating a Discrete Time-Discrete Data Stochastic Process,” in Structural Analysis of Discrete Panel Data With Econometric Applications, ed. by C. F. Manski and D. McFadden. Cambridge, MA: MIT Press, 179–195. [141] HECKMAN, J. J., AND B. SINGER (1984): “A Method of Minimizing the Impact of Distributional Assumptions in Econometric Models for Duration Data,” Econometrica, 52, 271–320. [136,139] HONORÉ, B. E., AND E. TAMER (2006): “Bounds on Parameters in Panel Dynamic Discrete Choice Models,” Econometrica, 73, 611–629. [139] HOTZ, J., AND R. A. MILLER (1993): “Conditional Choice Probabilities and the Estimation of Dynamic Models,” Review of Economic Studies, 60, 497–529. [138,160] HOUDE, J.-F., AND S. IMAI (2006): “Identification and 2-Step Estimation of DDC Models With Unobserved Heterogeneity,” Unpublished Manuscript, Queen’s University. [149,150] KASAHARA, H., AND K. SHIMOTSU (2008a): “Pseudo-Likelihood Estimation and Bootstrap Inference for Structural Discrete Markov Decision Models,” Journal of Econometrics, 146, 92–106. [138,160] (2008b): “Nonparametric Identification and Estimation of Multivariate Mixtures,” Unpublished Manuscript, Queen’s University. [147,169] KEANE, M. P., AND K. I. WOLPIN (1997): “The Career Decisions of Young Men,” Journal of Political Economy, 105, 473–522. [136]
MODELS OF DYNAMIC DISCRETE CHOICES
175
KITAMURA, Y. (2004): “Nonparametric Identifiability of Finite Mixtures,” Unpublished Manuscript, Yale University. [139] LEROUX, B. G. (1992): “Consistent Estimation of a Mixing Distribution,” Annals of Statistics, 20, 1350–1360. [169] LIU, X., AND Y. SHAO (2003): “Asymptotics for Likelihood Ratio Tests Under Loss of Identifiability,” Annals of Statistics, 31, 807–832. [169] MADANSKY, A. (1960): “Determinantal Methods in Latent Class Analysis,” Psychometrika, 25, 183–198. [145,148] MAGNAC, T., AND D. THESMAR (2002): “Identifying Dynamic Discrete Decision Processes,” Econometrica, 70, 801–816. [139,147,162] MCLACHLAN, G. J., AND D. PEEL (2000): Finite Mixture Models. New York: Wiley. [169] PAKES, A., M. OSTROVSKY, AND S. BERRY (2007): “Simple Estimators for the Parameters of Discrete Dynamic Games (With Entry/Exit Examples),” RAND Journal of Economics, 38, 373–399. [138] PESENDORFER, M., AND P. SCHMIDT-DENGLER (2008): “Asymptotic Least Squares Estimators for Dynamic Games,” Review of Economic Studies, 75, 901–928. [138] RAO, P. (1992): Identifiability in Stochastic Models. San Diego: Academic Press. [136] RIDDER, G. (1990): “The Non-Parametric Identification of Generalized Accelerated FailureTime Models,” Review of Economic Studies, 57, 167–181. [139] RUST, J. (1987): “Optimal Replacement of GMC Bus Engines: An Empirical Model of Harold Zurcher,” Econometrica, 55, 999–1033. [142,164] (1994): “Estimation of Dynamic Structural Models, Problems and Prospects: Discrete Decision Processes,” in Advances in Econometrics: Sixth World Congress of the Econometric Society, ed. by C. Sims. Cambridge, U.K.: Cambridge University Press, 119–170. [139,162] TITTERINGTON, D. M., A. F. M. SMITH, AND U. E. MAKOV (1985): Statistical Analysis of Finite Mixture Distributions. New York: Wiley. [136] VAN DEN BERG, G. J. (2001): “Duration Models: Specification, Identification and Multiple Durations,” in Handbook of Econometrics, Vol. 5, ed. by J. Heckman and E. Leamer. Amsterdam: North-Holland. [139]
Dept. of Economics, University of Western Ontario, London, Ontario, N6A 5C2 Canada; [email protected] and Dept. of Economics, Queen’s University, 94 University Avenue, Kingston, Ontario, K7L 3N6 Canada; [email protected]. Manuscript received October, 2006; final revision received April, 2008.
Econometrica, Vol. 77, No. 1 (January, 2009), 177–234
LONG-TERM RISK: AN OPERATOR APPROACH BY LARS PETER HANSEN AND JOSÉ A. SCHEINKMAN1 We create an analytical structure that reveals the long-run risk-return relationship for nonlinear continuous-time Markov environments. We do so by studying an eigenvalue problem associated with a positive eigenfunction for a conveniently chosen family of valuation operators. The members of this family are indexed by the elapsed time between payoff and valuation dates, and they are necessarily related via a mathematical structure called a semigroup. We represent the semigroup using a positive process with three components: an exponential term constructed from the eigenvalue, a martingale, and a transient eigenfunction term. The eigenvalue encodes the risk adjustment, the martingale alters the probability measure to capture long-run approximation, and the eigenfunction gives the long-run dependence on the Markov state. We discuss sufficient conditions for the existence and uniqueness of the relevant eigenvalue and eigenfunction. By showing how changes in the stochastic growth components of cash flows induce changes in the corresponding eigenvalues and eigenfunctions, we reveal a long-run riskreturn trade-off. KEYWORDS: Risk-return trade-off, long run, semigroups, Perron–Frobenius theory, martingales.
1. INTRODUCTION WE STUDY LONG-RUN NOTIONS of a risk-return relationship that feature the pricing of exposure to uncertainty in the growth rates of cash flows. In financial economics, risk-return trade-offs show how expected rates of return over small intervals are altered in response to changes in the exposure to the underlying shocks that impinge on the economy. In continuous-time modeling, the length of the interval is driven to zero to deduce a limiting local relationship. In contrast to the local analysis, we focus on what happens as the length of time between valuation and payoff becomes large. In a dynamic setting, asset payoffs depend on both the future state and on the date when the payoff will be realized. Risk-averse investors require compensation for their risk exposure, giving rise to risk premia in equilibrium pricing. There is a term structure of risk premia to consider. There are many recent developments in asset pricing theory that feature the intertemporal composition of risk. The risk premia for different investment horizons are necessarily related, just as long-term interest rates reflect a compounding of short-term 1
Comments from Jaroslav Boroviˇcka, Rene Carmona, Vasco Carvalho, Junghoon Lee, Angelo Melino, Eric Renault, Chris Rogers, Mike Stutzer, Grace Tsiang, and Yong Wang were very helpful in preparing this paper. We also benefited from valuable feedback from the co-editor, Larry Samuelson, and the referees of this paper. This material is based on work supported by the National Science Foundation under award numbers SES-05-19372, SES-03-50770, and SES-0718407. Portions of this work were done while José Scheinkman was a Fellow of the John Simon Guggenheim Memorial Foundation. © 2009 The Econometric Society
DOI: 10.3982/ECTA6761
178
L. P. HANSEN AND J. A. SCHEINKMAN
rates. The risk counterpart to this compounding is most transparent in lognormal environments with linear state dynamics and constant volatility (e.g., see Hansen, Heaton, and Li (2008)). Our aim, however, is to support the analysis of a more general class of models that allow for nonlinearity in the state dynamics. Risk premia depend on both risk exposure and the price of that exposure. The methods we develop in this paper are useful for representing the exposure of cash flows and the price of that exposure in long horizons. While we are interested in the entire term structure of risk prices, the aim of this paper is to establish limiting behavior as investment horizons become large. There are two reasons for such an investigation. First, it provides a complementary alternative to the local pricing that is familiar from the literature on asset pricing. Comparison of the two allows us to understand the (average) slope of the term structure of risk prices. Second, economics is arguably a more reliable source of restrictions over longer time horizons. Thus it is advantageous to have tools that allow us to characterize how equilibrium risk prices are predicted to behave in the long run and how the prices depend on ingredients of the underlying model of the economy. The continuous-time local analysis familiar in financial economics is facilitated by the use of stochastic differential equations driven by a vector Brownian motion and jumps. An equilibrium valuation model gives the prices of the instantaneous exposure of payoffs to these risks. Values over alternative horizons can be inferred by integrating appropriately these local prices. Such computations are nontrivial when there are nonlinearities in the evolution of state variables or valuations. This leads us to adopt an alternative approach based on an operator formulation of asset pricing. As in previous research, we model asset valuation using operators that assign prices today to payoffs on future dates. Since these operators are defined for each payoff date, we build an indexed family of such pricing operators. This family is referred to as a semigroup because of the manner in which the operators are related to one another.2 We show how to modify valuation operators in a straightforward way to accommodate stochastic cash-flow growth. It is the evolution of such operators as the payoff date changes that interests us. Long-run counterparts to risk-return trade-offs are reflected in how the limiting behavior of the family of operators changes as we alter the stochastic growth of the cash flows. We study the evolution of the family of valuation operators in a continuoustime framework, although important aspects of our analysis are directly applicable to discrete-time specifications. Our analysis is made tractable by assuming the existence of a Markov state that summarizes the information in the economy pertinent for valuation. The operators we use apply to functions of this Markov state and can be represented as Mt ψ(x) = E[Mt ψ(Xt )|X0 = x] 2 See Garman (1984) for an initial contribution featuring the use of semigroups in modeling asset prices.
LONG-TERM RISK
179
for some process M appropriately restricted to ensure intertemporal consistency and to guarantee that the Markov structure applies to pricing. The principal restriction we impose is that M has a “multiplicative” property, resulting in a family of operators indexed by t that is a semigroup. A central mathematical result that we establish and exploit is a multiplicative factorization, (1)
ˆ t φ(X0 ) Mt = exp(ρt)M φ(Xt )
ˆ is a martingale whose logarithm has stationary increments, and φ is where M a positive function.3 While this decomposition is typically not unique, we show that there is at most one such decomposition that is of value to study longterm approximation. Intuitively, we may think of the scalar ρ as a deterministic growth rate and of the ratio of positive random variables (φ(X0 ))/(φ(Xt )) as a transitory contribution. We construct this representation using a function φ which is a principal (that is positive) eigenfunction of every operator Mt in the semigroup, that is, (2)
Mt φ(x) = exp(ρt)φ(x)
ˆ to change the probability measure In our analysis, we use the martingale M prior to our study of approximation. The principal eigenfunction φ used in constructing this change characterizes the limiting state dependence of the semigroup. We use the multiplicative factorization (1) to study two alternative long-run counterparts to risk-return trade-offs. It allows us to isolate enduring components to cash-flows or prices and to explore how these components are valued. For instance, cash flows with different stochastic growth components are valued differently. One notion of a long-run risk-return trade-off characterizes how the asymptotic rate of return required as compensation depends on the long-run cash-flow risk exposure. A second notion cumulates returns that are valued in accordance to a local risk-return trade-off. A corresponding long-run trade-off gives the asymptotic growth rates of alternative cumulative returns over long time horizons as a function of the risk exposures used to construct the local returns. Previous papers have explored particular characterizations of the term structure of risk pricing of cash flows. (For instance, see Hansen, Heaton, and Li 3
Alvarez and Jermann (2005) proposed the use of such a decomposition to analyze the longrun behavior of stochastic discount factors and cited an early version of our paper for proposing the link between this decomposition and principal eigenvalues and functions. We develop this connection, formally, and establish existence and uniqueness results for a general class of multiplicative functionals.
180
L. P. HANSEN AND J. A. SCHEINKMAN
(2008) and Lettau and Wachter (2007).) In this regard, local pricing characterizes one end of this term structure and our analysis characterizes the other end. Hansen, Heaton, and Li (2008) characterized the long-run cash-flow risk prices for discrete-time log-normal environments. Their characterization shares our goal of pricing the exposure to stochastic growth risk, but to obtain analytical results, they excluded potential nonlinearity and temporal variation in volatility. Hansen, Heaton, and Li (2008) examined the extent to which the long-run cash-flow risk prices from a family of recursive utility models can account for the value heterogeneity observed in equity portfolios with alternative ratios of book value to market value. Our paper shows how to produce such pricing characterizations for more general nonlinear Markov environments. There is a corresponding equation to (2) that holds locally, obtained essentially by differentiating with respect to t and evaluating the derivative at t = 0. More generally, this time derivative gives rise to the generator of the semigroup. By working with the generator, we exploit some of the well known local characterizations of continuous-time Markov models from stochastic calculus to provide a solution to equation (2). While continuous-time models achieve simplicity by characterizing behavior over small time increments, operator methods have promise for enhancing our understanding of the connection between short-run and long-run behavior. The remainder of the paper is organized as follows. In Sections 3 and 4 we develop some of the mathematical preliminaries pertinent for our analysis. Specifically, in Section 3 we describe the underlying Markov process and introduce the reader to the concepts of additive and multiplicative functionals. Both type of functionals are crucial ingredients to what follows. In Sections 3.3, 3.4, and 3.5 we consider three alternative multiplicative functionals that are pertinent in intertemporal asset pricing. In Section 3.3 we use a multiplicative functional to model a stochastic discount factor process, in Section 3.4 we introduce valuation functionals that are used to represent returns over intervals of any horizon, and in Section 3.5 we use multiplicative functionals to represent the stochastic growth components to cash flows. In Section 4 we study semigroups. Semigroups are used to evaluate contingent claims written on the Markov state indexed by the elapsed time between trading date and the payoff date. In Section 5 we define an extended generator associated with a multiplicative functional, and in Section 6 we introduce principal eigenvalues and functions and use these objects to establish a representation of the form (1). We present approximation results that justify formally the long-run role of a principal eigenfunction and eigenvalue, and show that there is at most one appropriate eigenfunction in Section 7. Applications to financial economics are given in Section 8. Among other things, we derive long-term counterparts to risk-return trade-offs by making the valuation horizon arbitrarily large. Finally, in Section 9 we discuss sufficient conditions for the existence of the principal eigenvectors needed to support our analysis.
LONG-TERM RISK
181
2. STOCHASTIC DISCOUNT FACTORS AND PRICING Consider a continuous-time Markov process {Xt : t ≥ 0} and the (completed) filtration Ft generated by its histories. A strictly positive stochastic discount factor process {St : t ≥ 0} is an adapted (St is Ft measurable) positive process that is used to discount payoffs. The date zero price of a claim to the payoff Πt at t is E[St Πt |F0 ] For each date t, this representation follows from the representation of positive linear functionals on appropriately constructed payoff spaces. The stochastic discount factor process is positive with probability 1 and satisfies S0 = 1 because of the presumed absence of arbitrage. Let τ ≤ t be an intermediate trading date between date zero and date t. The time t + u payoff could be purchased at date zero or alternatively it could be purchased at date τ with a prior date zero purchase of a claim to the date τ purchase price. The law of one price guarantees that these two ways to acquire the payoff Πt must have the same initial cost. The same must be true if we scale the Πt by a bounded random variable available to the investor at date τ. This argument allows us to infer the date τ prices from the date zero prices. Specifically, for τ ≤ t, St (3) E Πt Fτ Sτ is the price at time τ of a claim to the payoff Πt at t. Thus once we have a date zero representation of prices at alternative investment horizons via a stochastic discount factor process, we may use that same process to represent prices at subsequent dates by forming the appropriate ratios of the date zero stochastic discount factors. This representation imposes temporal consistency in valuation enforced by the possibility of trading at intermediate dates.4 We add a Markov restriction to this well known depiction of asset prices. Expression (3) can then be used to define a pricing operator St . In particular, if ψ(Xt ) is a random payoff at t that depends only on the current Markov state, its time zero price is (4)
St ψ(x) = E[St ψ(Xt )|x0 = x]
expressed as a function of the initial Markov state. The fact that the price depends only on the current state restricts the stochastic discount factor process. While St can depend on the Markov process between dates 0 and t, we do not 4 This temporal consistency property follows formally from a “consistency axiom” in Rogers (1998).
182
L. P. HANSEN AND J. A. SCHEINKMAN
allow it to depend on previous history of the Markov state; otherwise, this history would be reflected in the date zero prices. As first remarked by Garman (1984), the temporal consistency in valuation insures the that family of such linear pricing operators {St : t ≥ 0} satisfies a semigroup property: S0 = I and St+u ψ(x) = St Su ψ(x). In our asset pricing setting, the semigroup property is thus an iterated value property that connects pricing over different time horizons.5 Consider again trading at the intermediate date τ. Then Markov valuation between dates τ and t can be depicted using the operator St−τ or it can be depicted using (3). As a consequence, St /Sτ should depend only on the Markov process between dates t and τ. Moreover, (5)
St = St−τ (θτ ) Sτ
where θτ is a shift operator that moves the time subscript of the Markov process used in the construction of St−τ forward by τ time units. Property (5) is a restriction on functionals built from an underlying Markov process, and we will call functionals that satisfy property (5) and have initial value 1 multiplicative functionals. Later we will give convenient representations of such functionals. Our approach is motivated by this multiplicative property of the stochastic discount factor and uses the connection between this multiplicative property and the semigroup property of the family of pricing operators. We will also use this multiplicative property to study the valuation of payoffs with stochastic growth components. To accommodate these other processes, we set up a more general framework in the next couple of sections. 3. MARKOV AND RELATED PROCESSES We first describe the underlying Markov process, and then build other convenient processes from this underlying process. These additional processes are used to represent stochastic discounting, stochastic growth, and returns over long horizons. 3.1. Baseline Process Let {Xt : t ≥ 0} be a continuous-time, strong Markov process defined on a probability space {Ω F Pr} with values on a state space D0 ⊂ Rn The sample paths of X are continuous from the right and with left limits, and we will sometimes also assume that this process is stationary and ergodic. Let Ft be completion of the sigma algebra generated by {Xu : 0 ≤ u ≤ t}. 5 Garman (1984) allowed for non-Markov environments. In this case, the family of operators forms an “evolution semigroup.” We adopt a Markov formulation of the law of one price for tractability.
LONG-TERM RISK
183
One simple example is the following: EXAMPLE 3.1—Finite-State Markov Chain: Consider a finite-state Markov chain with states xj for j = 1 2 N. The local evolution of this chain is governed by an N × N intensity matrix U. An intensity matrix encodes all of the transition probabilities. The matrix exp(tU) is the matrix of transition probabilities over an interval t. Since each row of a transition matrix sums to unity, each row of U must sum to zero. The diagonal entries are nonpositive and represent minus the intensity of jumping from the current state to a new one. The remaining row entries, appropriately scaled, represent the conditional probabilities of jumping to the respective states. When treating infinite-state spaces, we restrict the Markov process X to be a semimartingale. As a consequence, we can extract a continuous component X c and what remains is a pure jump process X j . The evolution of the jump component is given by j dXt = yζ(dy dt) Rn
where ζ = ζ(· ·; ω) is a random counting measure. That is, for each ω, ζ(b [0 t]; ω) gives the total number of jumps in [0 t] with a size in the Borel set b in the realization ω. In general, the associated Markov stochastic process X may have an infinite number of small jumps in any time interval. In what follows we will assume that this process has a finite number of jumps over a finite time interval. This rules out most Lévy processes, but greatly simplifies the notation. In this case, there is a finite measure η(dy|x) dt that is the compensator of the random measure ζ. It is the (unique) predictable random measure, such that for each predictable stochastic function f (x t; ω), the process t t f (y s; ω)ζ(dy ds; ω) − f (y s; ω)η[dy|Xs− (ω)] ds 0
Rn
0
Rn
is a martingale. The measure η encodes both a jump intensity and a distribution of the jump size given that a jump occurs. The jump intensity is the implied conditional measure of the entire state space D0 , and the jump distribution is the conditional measure divided by the jump intensity. We presume that the continuous sample path component satisfies the stochastic evolution dXtc = ξ(Xt− ) dt + Γ (Xt− ) dBt where B is a multivariate Ft -Brownian motion and Γ (x) Γ (x) is nonsingular. Given the rank condition, the Brownian increment can be deduced from the sample path of the state vector via dBt = [Γ (Xt− ) Γ (Xt− )]−1 Γ (Xt− ) [dXtc − ξ(Xt− ) dt]
184
L. P. HANSEN AND J. A. SCHEINKMAN
EXAMPLE 3.2—Markov Diffusion: In what follows we will often use the following example. Suppose the Markov process X has two components, X f and X o , where X f is a Feller square root process and is positive, and X o is an Ornstein–Uhlenbeck process and ranges over the real line, f f f f dXt = ξf (x¯ f − Xt ) dt + Xt σf dBt dXto = ξo (x¯ o − Xto ) dt + σo dBto f with ξi > 0, x¯ i > 0 for i = f o, and 2ξf x¯ f ≥ (σf )2 , where B = BBo is a bivariate standard Brownian motion. The parameter restrictions guarantee that there is a stationary distribution associated with X f with support contained in R+ .6 For convenience, we make the two processes independent. We use X o to model predictability in growth rates and X f to model predictability in volatility. We normalize σo to be positive and σf to be negative. We specify σf to be negative f so that a positive increment to Bt reduces volatility. 3.2. Multiplicative Functionals A functional is a stochastic process constructed from the original Markov process: DEFINITION 3.1: A functional is a real-valued process {Mt : t ≥ 0} that is adapted (Mt is Ft measurable for all t). We will assume that Mt has a version with sample paths that are continuous from the right with left limits. Recall that θ denotes the shift operator. Using this notation, write Mu (θt ) for the corresponding function of the X process shifted forward t time periods. Since Mu is constructed from the Markov process X between dates zero and u, Mu (θt ) depends only on the process between dates t and date t + u. DEFINITION 3.2: The functional M is multiplicative if M0 = 1 and Mt+u = Mu (θt )Mt . Products of multiplicative functionals are multiplicative functionals. We are particularly interested in strictly positive multiplicative functionals. In this case, one may define a new functional A = log(M) that will satisfy an additive property. It turns out that it is more convenient to parameterize M using its logarithm A. The functional A will satisfy the following definition: 6 We could accommodate the case where Bf or Bo is each multidimensional by considering a filtration {Ft } larger than the one generated by X. In effect, we would enlarge the state space in ways that were inconsequential to the computations that interest us. However, for simplicity we have assumed throughout this paper that {Ft } is the filtration generated by X.
LONG-TERM RISK
185
DEFINITION 3.3: A functional A is additive if A0 = 0 and At+u = Au (θt ) + At for each nonnegative t and u.7 Exponentials of additive functionals are strictly positive multiplicative functionals. While the joint process {(Xt At ) : t ≥ 0} is Markov, by construction the additive functional does not Granger cause the original Markov process. Instead it is constructed from that process. No additional information about the future values of X are revealed by current and past values of A. When X is restricted to be stationary, an additive functional can be nonstationary but it has stationary increments. The following are examples of additive functionals: EXAMPLE 3.3: Given any continuous function ψ, At = ψ(Xt ) − ψ(X0 ). EXAMPLE 3.4: Let β be a Borel measurable function on D0 and construct t At = β(Xu ) du 0
where
t 0
β(Xu ) du < ∞ with probability 1 for each t.
EXAMPLE 3.5: Form t γ(Xu− ) dBu At = 0
where
t 0
|γ(Xu )|2 du is finite with probability 1 for each t.
EXAMPLE 3.6: Form
κ(Xu Xu− ) At = 0≤u≤t
where κ : D0 × D0 → R, κ(x x) = 0. Sums of additive functionals are additive functionals. We may thus use Examples 3.4, 3.5, and 3.6 as building blocks in a parameterization of additive functionals. This parameterization uses a triple (β γ κ) that satisfies the following situations: t (a) β : D0 → R and 0 β(Xu ) du < ∞ for every positive t. t (b) γ : D0 → Rm and 0 |γ(Xu )|2 du < ∞ for every positive t. 7 Notice that we do not restrict additive functionals to have bounded variation as, for example, Revuz and Yor (1994).
186
L. P. HANSEN AND J. A. SCHEINKMAN
(c) κ : D0 × D0 → R, κ(x x) = 0 for all x ∈ D0 , exp(κ(y x))η(dy|x) < ∞ for all x ∈ D0 . Form t t
β(Xu ) du + γ(Xu− ) dBu + κ(Xu Xu− ) At = 0
0
0≤u≤t
t
=
β(Xu ) du 0
+
t
γ(Xu− ) [Γ (Xu− ) Γ (Xu− )]−1 Γ (Xu− ) [dXuc − ξ(Xu− ) du]
0
+
κ(Xu Xu− )
0≤u≤t
This additive functional is a semimartingale. While we will use extensively these parameterizations of an additive functional, they are not exhaustive as the following example illustrates. EXAMPLE 3.7: Suppose that {Xt : t ≥ 0} is a standard scalar Brownian motion, suppose b is a Borel set in R, and define the occupation time of b up to time t as t At = 1{Xu ∈b} du 0
At is an additive functional. As a consequence, the local time at a point r, defined as 1 t Lt = lim 1{Xu ∈(r−r+)} du ↓0 2 0 is also an additive functional. Since the logarithm of a strictly positive multiplicative process is an additive process, we will consider parameterized versions of strictly positive multiplicative processes by parameterizing the corresponding additive process. For instance, if M = exp(A) when A is parameterized by (β γ κ), we will say that the multiplicative process M is parameterized by (β γ κ). Notice that Ito’s lemma guarantees that dMt |γ(Xt− )|2 dt + γ(Xt− ) dBt = β(Xt− ) + Mt− 2 + exp[κ(Xt Xt− )] − 1
187
LONG-TERM RISK
The multiplicative process {Mt : t ≥ 0} of this form is a local martingale if and only if |γ|2 β+ (6) + exp[κ(y ·)] − 1 η(dy|·) = 0 2 For the remainder of this section we describe three types of multiplicative functionals that we use in our subsequent analysis. 3.3. Stochastic Discount Factors In this section we write down two parameterized examples of multiplicative stochastic discount factors that we will use to illustrate our results. EXAMPLE 3.8—Breeden Model: Using the Markov process given in Example 3.2, we consider a special case of Breeden’s (1979) consumption-based asset pricing model. Suppose that equilibrium consumption evolves according to f f dct = Xto dt + Xt ϑf dBt + ϑo dBto (7) where ct is the logarithm of consumption Ct . Given our previous sign convention that σo > 0, when ϑo > 0 a positive shock dBto is unambiguously good news because it gives a favorable movement for consumption and its growth f rate. Similarly, since σf < 0, when ϑf > 0 a positive shock dBt is unambiguously good news because it reduces volatility while increasing consumption. Suppose also that investors’ preferences are given by
∞
E
exp(−bt) 0
Ct 1−a − 1 dt 1−a
for a and b strictly positive. The implied stochastic discount factor is St = exp(Ast ), where t t t f s o f Xs ds − bt − a ϑo dBso Xs ϑf dBs − a At = −a 0
0
0
EXAMPLE 3.9—Kreps–Porteus Model: When investors have time-separable logarithmic utility and perfect foresight, the continuation value process W ∗ for the consumption process satisfies the differential equation (8)
dWt ∗ = b(Wt ∗ − ct ) dt
where b is the subjective rate of time discount. This equation is solved forward with an appropriate “terminal” condition. In constructing this differential
188
L. P. HANSEN AND J. A. SCHEINKMAN
equation, we have scaled the logarithm of consumption by b for convenience. Let Wt =
1 exp[(1 − a)Wt ∗ ] 1−a
for a > 1 and notice that Wt is an increasing transformation of Wt ∗ . Thus for the purposes of representing preferences, W can be used as an ordinally equivalent continuation value process. The process W satisfies the differential equation
dWt 1 = b(1 − a)Wt (9) log[(1 − a)Wt ] − ct dt 1−a = bWt (a − 1)ct + log[(1 − a)Wt ] Next suppose that investors do not have perfect foresight. We may now think of the right-hand sides of (8) and (9) as defining the drift or local means of the continuation values. As we know from Kreps and Porteus (1978) and Duffie and Epstein (1992), the resulting preferences cease to be ordinally equivalent. The first gives the recursive equation for continuation values that are expectations of the discounted logarithmic utility. Instead we use the counterpart to the second differential equation: lim ↓0
E(Wt+ − Wt |Ft ) = bWt (a − 1)ct + log[(1 − a)Wt ]
where a > 1. The resulting preferences can be viewed as a special case of the continuous-time version of the preferences suggested by Kreps and Porteus (1978) and of the stochastic differential utility model of Duffie and Epstein (1992) and Schroder and Skiadas (1999). If we were to take the continuation value process W as a starting point in a stochastic environment and transform back to the utility index W ∗ using Wt ∗ =
1 log[(1 − a)Wt ] 1−a
the resulting drift would include a contribution of the local variance as an application of Ito’s lemma. For these preferences the intertemporal composition of risk matters. Bansal and Yaron (2004) have used this feature of preferences in conjunction with predictable components in consumption and consumption volatility as a device to amplify risk premia. This particular utility recursion we use imposes a unitary elasticity of intertemporal substitution as in the original preference specification with logarithmic utility. The parameter a alters risk prices as we will illustrate.8 8 Epstein and Zin (1989) used a more general discrete-time version of these preferences as a way to distinguish risk aversion from intertemporal substitution.
189
LONG-TERM RISK
Suppose again that consumption evolves according to equation (7). Conjecture a continuation value process of the form Wt =
1 f exp[(1 − a)(wf Xt + wo Xto + ct + w¯ )] 1−a
The coefficients satisfy −ξf wf +
(1 − a)σf2 2
(wf )2 + (1 − a)ϑf σf wf +
(1 − a)ϑ2f 2
= bwf
−ξo wo + 1 = bwo (1 − a)σo2 (wo )2 2 (1 − a)ϑ2o = bw¯ + (1 − a)ϑo σo wo + 2
ξf x¯ f wf + ξo x¯ o wo +
The equation for wf is quadratic and there are potentially two solutions. The solution that interests us is wf = (a − 1)σf ϑf + b + ξf − [(a − 1)σf ϑf + b + ξf ]2 − (a − 1)2 σf2 ϑ2f /((1 − a)σf2 ) See Appendix A. Furthermore, w0 > 0 and, as we show in Appendix A, wf < 0. The stochastic discount factor is the product of two multiplicative functionals. One has the same form as the Breeden model with a logarithmic instantaneous utility function. It is the exponential of t t t f B o f At = − Xs ds − bt − Xs ϑf dBs − ϑo dBso 0
0
0
The other functional is a martingale constructed from the forward-looking continuation value process. It is the exponential of t t f Xs (ϑf + wf σf ) dBsf + (1 − a) (ϑo + wo σo ) dBso Awt = (1 − a) 0
−
(1 − a) 2
2
0
t
Xsf (ϑf + wf σf )2 ds − 0
(1 − a)2 (ϑo + wo σo )2 t 2
We next consider a variety of ways in which multiplicative functionals can be used to build models of asset prices and to characterize the resulting implications.
190
L. P. HANSEN AND J. A. SCHEINKMAN
3.4. Valuation Functionals and Returns We use a special class of multiplicative functionals called valuation functionals to characterize local pricing. The result of this analysis will be the Markov version of a local risk-return trade-off. A valuation functional is constructed to have the following property. If the future value of the process is the payout, the current value is the price of that payout. For instance, a valuation process could be the result of continually reinvesting dividends in a primitive asset. Equivalently, it can be constructed by continually compounding realized returns to an investment. To characterize local pricing, we use valuation processes that are multiplicative functionals. Recall that the product of two multiplicative functionals is a multiplicative functional. The following definition is motivated by the connection between the absence of arbitrage and the martingale properties of properly normalized prices. DEFINITION 3.4: A valuation functional {Vt : t ≥ 0} is a multiplicative functional such that the product functional {Vt St : t ≥ 0} is a martingale. Provided that V is strictly positive, the associated gross returns over any horizon u can be calculated by forming ratios: Rtt+u =
Vt+u Vt−
This increment in the value functional scaled by the current (pre-jump) value gives an instantaneous net return. The martingale property of the product V S gives a local pricing restriction for returns. To deduce a convenient and familiar risk-return relation, consider the multiplicative functional M = V S, where V is parameterized by (βv γv κv ) and {St : t ≥ 0} is parameterized by (βs γs κs ). In particular, the implied net return evolution is dVt |γv (Xt− )|2 dt + γv (Xt− ) dBt = βv (Xt− ) + Vt− 2 + exp[κv (Xt Xt− )] − 1 Thus the expected net rate of return is |γv |2 + exp[κv (y ·)] − 1 η(dy|·) εv = βv + 2 Since both V and S are exponentials of additive processes, their product is the exponential of an additive process and is parameterized by β = βv + βs
LONG-TERM RISK
191
γ = γv + γ s κ = κ v + κs PROPOSITION 3.1: A valuation functional parameterized by (βv γv κv ) satisfies the pricing restriction |γv + γs |2 βv + βs = − − exp[κv (y ·) + κs (y ·)] − 1 η(dy|·) (10) 2 PROOF: The proof follows from the definition of a valuation functional and the martingale restriction (6). Q.E.D. This restriction is local and determines the instantaneous risk-return relation. The parameters (γv κv ) determine the Brownian and jump risk exposure. The following corollary gives the required local mean rate of return: COROLLARY 3.1: The required mean rate of return for the risk exposure (γv κv ) is |γs |2 εv = −βs − γv · γs − 2 − exp[κv (y ·) + κs (y ·)] − exp[κv (y ·)] η(dy ·) The vector −γs contains the factor risk prices for the Brownian motion components. The function κs is used to price exposure to jump risk. Then εv is the required expected rate of return expressed as a function of the risk exposure. This local relation is familiar from the extensive literature on continuous-time asset pricing.9 In the case of Brownian motion risk, the local risk price vector of the exposure to risk is given by −γs . A valuation functional is typically constructed from the values of a selffinancing strategy. Not every self-financing strategy results in a valuation which can be written as a multiplicative functional, but the class of (multiplicative) valuation functionals is sufficiently rich to extract the implied local risk prices. For this reason, we restrict ourselves in this paper to (multiplicative) valuation functionals. EXAMPLE 3.10—Breeden Example (Continued): Consider again the Markov diffusion Example 3.2 with the stochastic discount factor given in Exam9
Shaliastovich and Tauchen (2005) presented a structural model of asset prices in discrete time with a Lévy component to the risk exposure. The continuous-time counterpart would include Markov processes with an infinite number of jumps expected in any finite time interval.
192
L. P. HANSEN AND J. A. SCHEINKMAN
ple 3.8. This is a Markov version of Breeden’s model. The local risk price for exposure to the vector of Brownian motion increments is −γs =
√ a xf ϑf aϑo
and the instantaneous risk-free rate is b + axo −
a2 (xf (ϑf )2 + (ϑo )2 )
2
Consider a family of valuation processes parameterized by (β γ), where √ γ(x) = ( xf γf γo ). To satisfy the martingale restriction, we must have 1 β(x) = b + axo − [xf (γf − aϑf )2 + (γo − aϑo )2 ] 2 EXAMPLE 3.11 —Kreps–Porteus Model (Continued): Consider again the Markov diffusion Example 3.2 with the stochastic discount factor given in Example 3.9. The local risk price for exposure to the vector of Brownian motion increments is √ √ a xf ϑf + (a − 1) xf wf σf −γs = aϑo + (a − 1)wo σo and the instantaneous risk-free rate is 1 [xf (ϑf )2 + (ϑo )2 ] − (a − 1)xf ϑf (ϑf + wf σf ) 2 − (a − 1)ϑo (ϑo + wo σo )
b + xo −
In particular, the local risk prices are larger than for their counterparts in the Breeden (1979) model when ϑo and ϑf are both positive.10 As we have seen, alternative valuation functionals reflect alternative risk exposures. The examples we just discussed show how the required expected rate of return (βv ) for a given local risk exposures (γv κv ) depends on the underlying economic model and the associated parameter values. The methods we will describe allow us to characterize the behavior of expectations of valuation functionals over long horizons. 10 When ϑf is positive, Kleshchelski and Vincent (2007) showed that the real term structure will be often downward sloping.
LONG-TERM RISK
193
3.5. Growth Functionals In our analysis of valuation, we will investigate the pricing of cash flows as we extend the payoff horizon. To investigate the value implications for cash flows that grow stochastically, we introduce a reference growth process {Gt : t ≥ 0} that is a positive multiplicative functional. Consider a cash flow that can be represented as (11)
Dt = Gt ψ(Xt )D0
for some initial condition D0 , where G is a multiplicative functional. The initial condition is introduced to offset the restriction that multiplicative functionals are normalized to be unity at date zero. Heuristically, we may think of ψ(X) as the stationary component of the cash flow and of G as the growth component.11 As we will illustrate, however, the covariance between components sometimes makes this interpretation problematic. The fact that the product of multiplicative functionals is a multiplicative functional implies that the product of a stochastic discount factor functional and a growth functional is itself multiplicative. This property facilitates the construction of valuation operators designed for cash-flow processes that grow stochastically over time. In Section 2 we emphasized the connection between the multiplicative property of stochastic discount factors and the semigroup property of pricing operators. In the next section we discuss how multiplicative functionals give rise to semigroups. This development lays the groundwork for considering a variety of ways in which multiplicative functionals and their implied semigroups can be used to characterize the implications of asset pricing models over long horizons. 4. MULTIPLICATIVE FUNCTIONALS AND SEMIGROUPS Given a multiplicative functional M, our aim is to establish properties of the family of operators (12)
Mt ψ(x) = E[Mt ψ(Xt )|X0 = x] 4.1. Semigroups
Let L be a Banach space with norm · and let {Tt : t ≥ 0} be a family of operators on L. The operators in these family are linked according to the following property: 11 One can easily write down securities with a payout that cannot be represented as in equation (11), but we are interested in deriving properties of the pricing of securities with a payout as in (11) to evaluate alternative models and parameter configurations.
194
L. P. HANSEN AND J. A. SCHEINKMAN
DEFINITION 4.1: A family of linear operators {Tt : t ≥ 0} is a one-parameter semigroup if T0 = I and Tt+s = Tt Ts for all s t ≥ 0. One possibility is that these operators are conditional expectations operators, in which case this link typically follows from the law of iterated expectations restricted to Markov processes. We will also use such families of operators to study valuation and pricing. As we argued in Section 2, from a pricing perspective, the semigroup property follows from the Markov version of the law of iterated values, which holds when there is frictionless trading at intermediate dates. We will often impose further restrictions on semigroups such as follows: DEFINITION 4.2: The semigroup {Tt : t ≥ 0} is positive if for any t ≥ 0, Tt ψ ≥ 0 whenever ψ ≥ 0. 4.2. Multiplicative Semigroup The semigroups that interest us are constructed from multiplicative functionals. PROPOSITION 4.1: Let M be a multiplicative functional such that for each ψ ∈ L, E[Mt ψ(Xt )|X0 = x] ∈ L. Then Mt ψ(x) = E[Mt ψ(Xt )|X0 = x] is a semigroup in L. PROOF: For ψ ∈ L, M0 ψ = ψ and Mt+u ψ(x) = E E[Mt+u ψ(Xt+u )|Ft ]X0 = x = E E Mt Mu (θt )ψ(Xu (θt ))|Ft X0 = x = E Mt E Mu (θt )ψ(Xu (θt ))|X0 (θt ) X0 = x = E[Mt Mu ψ(Xt )|X0 = x] = Mt Mu ψ(x) which establishes the semigroup property.
Q.E.D.
In what follows we will refer to semigroups constructed from multiplicative functionals as in this proposition as multiplicative semigroups. If the multiplicative process is a stochastic discount factor, we will refer to the corresponding multiplicative semigroup as the pricing semigroup. Other semigroups also interest us.
LONG-TERM RISK
195
4.3. Valuation Semigroups Associated with a valuation functional V is a semigroup {Vt : t ≥ 0}. For any such valuational functional, we will derive the asymptotic growth rates of the implied cumulative return over long time horizons. The limiting growth rate expressed as a function of the risk exposures (γv κv ) gives one version of longterm risk-return trade-off. While measurement of long-horizon returns in loglinear environments has commanded much attention, operator methods can accommodate volatility movements as well. (See Bansal, Dittmar, and Kiku (2008) for a recent addition to this literature.) Our characterization of the long-run expected rate of return is motivated by our aim to quantify a risk-return relation. In contrast, Stutzer (2003) used the conditional expectation of a valuation functional raised to a negative power to develop a large deviation criterion for portfolio evaluation over large horizons. He also related this formulation to the familiar power utility model applied to terminal wealth appropriately scaled. Since a multiplicative functional raised to a negative power remains multiplicative, the limits we characterize are also germane to his analysis. In what follows we will suggest another way to represent a long-term risk return trade-off. 4.4. Semigroups Induced by Cash-Flow Growth We study cash flows with a common growth component using the semigroup Qt ψ(x) = E[Gt St ψ(Xt )|X0 = x] instead of the pricing semigroup {St } constructed previously. The date zero price assigned to Dt is D0 Qt ψ(X0 ). More generally, the date τ price assigned to Dt+τ is D0 Gτ Qt ψ(Xτ ). Thus the date τ price to (current period) payout ratio is D0 Gτ Qt ψ(Xτ ) Qt ψ(Xτ ) = Dτ ψ(Xτ ) provided that ψ(Xτ ) is different from zero. For a security such as an equity with a perpetual process of cash payouts or dividends, the price–dividend ratio is the integral of all such terms for t ≥ 0. Our subsequent analysis will characterize the limiting contribution to this value. The rate of decay of Qt ψ(Xτ ) as t increases will give a measure of the duration of the cash flow as it contributes to the value of the asset. This semigroup assigns values to cash flows with common growth component G but alternative transient contributions ψ. To study how valuation is altered when we change stochastic growth, we will be led to alter the semigroup.
196
L. P. HANSEN AND J. A. SCHEINKMAN
When the growth process is degenerate and equal to unity, the semigroup is identical to the one constructed previously in Section 2. This semigroup is useful in studying the valuation of stationary cash flows including discount bonds and the term structure of interest rates. It supports local pricing and generalizations of the analyses in Backus and Zin (1994) and Alvarez and Jermann (2005) that use fixed income securities to make inferences about economic fundamentals. This semigroup offers a convenient benchmark for the study of long-term risk just as a risk-free rate offers a convenient benchmark in local pricing. The decomposition (11) used in this semigroup construction is not unique. For instance, let ϕ be a strictly positive function of the Markov state. Then ϕ(Xt ) ψ(Xt ) [D0 ϕ(X0 )] Dt = Gt ψ(Xt )D0 = Gt ϕ(X0 ) ϕ(Xt ) Since (ψ(Xt ))/(ϕ(Xt )) is a transient component, we can produce (infinitely) many such decompositions. For decomposition (11) to be unique, we must thus restrict the growth component. ˆ t , where G ˆ is a marA convenient restriction is to require that Gt = exp(δt)G tingale. With this choice, by construction G has a constant conditional growth ˆ from a large rate δ. Later we show how to extract martingale components, G’s, class of multiplicative functionals G. In this way we will establish the existence of such a decomposition. Even with this restriction, the decomposition will not necessarily be unique, but we will justify a particular choice. We investigate long-term risk by changing the reference growth functionals. These functionals capture the long-term risk exposure of the cash flow. Our approach extends the analysis of Hansen, Heaton, and Li (2008) beyond loglinear environments. As we will demonstrate, the valuation of cash flows with common reference growth functionals will be approximated by a single dominant component when the valuation horizon becomes long. Thus the contributions to value that come many periods into the future will be approximated by a single pricing factor that incorporates an adjustment for risk. Changing the reference growth functional alters the long-term risk exposure with a corresponding adjustment in valuation. Each reference growth functional will be associated with a distinct semigroup. We will characterize long-term risk formally by studying the limiting behavior of the corresponding semigroup. As we have just seen, semigroups used for valuing growth claims are constructed by forming products of two multiplicative functionals: a stochastic discount factor functional and a growth functional. Pricing stationary claims and constructing cumulative returns lead to the construction of alternative multiplicative functionals. Table I gives a summary of the alternative multiplicative functionals and semigroups. For this reason, we will study the behavior of a general multiplicative semigroup: Mt ψ(x) = E[Mt ψ(Xt )|X0 = x]
LONG-TERM RISK
197
TABLE I ALTERNATIVE SEMIGROUPS AND MULTIPLICATIVE FUNCTIONALS Object
Stochastic discount factor Cumulated return Stochastic growth Valuation with stochastic growth
Multiplicative Functional
Semigroup
S V G Q = GS
{St } {Vt } {Gt } {Qt }
for some strictly positive multiplicative functional M. The next three sections establish some basic representation and approximation results for multiplicative semigroups that are needed for our subsequent economic analysis of long-term risk. An important vehicle in this study is the extended generator associated with a multiplicative process. This generator is a local (in time) construct. We develop its properties in Section 5. In Section 6 we show how to use a principal eigenfunction of this extended generator to construct our basic multiplicative decomposition (1). As we demonstrate in Section 7, an appropriately chosen eigenfunction and its associated eigenvalue dictate the long-term behavior of a multiplicative semigroup and the corresponding multiplicative functional. After establishing these basic results, we turn to the featured application in our paper: How do we characterize the longterm risk-return trade-off? 5. GENERATORS In this section we define a notion of an extended generator associated with a multiplicative functional. The definition parallels the definition of an extended (infinitesimal) generator associated with Markov processes as in, for example, Revuz and Yor (1994). Our extended generator associates to each function ψ a function χ such that Mt χ(Xt ) is the “expected time derivative” of Mt ψ(Xt ). DEFINITION 5.1: A Borel function ψ belongs to the domain of the extended generator A of the multiplicative functional M if there exists a Borel function t χ such that Nt = Mt ψ(Xt ) − ψ(X0 ) − 0 Ms χ(Xs ) ds is a local martingale with respect to the filtration {Ft : t ≥ 0}. In this case, the extended generator assigns the function χ to ψ and we write χ = Aψ. For strictly positive multiplicative processes M, the extended generator is (up to sets of measure zero) single valued and linear. In the remainder of the paper, if the context is clear, we often refer to the extended generator simply as the generator. Our first example deals with Markov chains:
198
L. P. HANSEN AND J. A. SCHEINKMAN
EXAMPLE 5.1—Markov Chain Generator: Recall the finite-state Markov chain Example 3.1 with intensity matrix U. Let uij denote entry (i j) of this matrix. Consider a multiplicative functional that is the product of two components. The first component decays at rate βi when the Markov state is xi . The second component only changes when the Markov process jumps from state i to state j, in which case the multiplicative functional is scaled by exp[κ(xj xi )]. From this construction we can deduce the generator A for the multiplicative semigroup depicted as a matrix with entry (i j): uii − βi if i = j, aij = uij exp[κ(xj xi )] if i = j. This formula uses the fact that in computing the generator we are scaling probabilities by the potential proportional changes in the multiplicative functional. The matrix A is not necessarily an intensity matrix. The row sums are not necessarily zero. The reason for this is that the multiplicative functional can include pure discount effects or pure growth effects. These effects can be present even when the βi ’s are zero, since it is typically the case that
uij exp[κ(xj xi )] = −uii j =i
The unit function is a trivial example of a multiplicative functional. In this case the extended generator is exactly what is called in the literature the extended generator of the Markov process X. When X is parameterized by (η ξ Γ ), Ito’s formula shows that the generator has the representation
∂2 φ(x) ∂φ(x) 1 (13) + trace Σ(x) Aφ(x) = ξ(x) · ∂x 2 ∂x ∂x + [φ(y) − φ(x)]η(dy|x) where Σ = Γ Γ provided φ is C 2 and the integral in (13) is finite. Recall our earlier parameterization of an additive functional A in terms of the triple (β γ κ). The process M = exp(A) is a multiplicative functional. We now display how to go from the extended generator of the Markov process X, that is, the generator associated with M ≡ 1, to the extended generator of the multiplicative functional M. The formulas below use the parameterization for the multiplicative process to transform the generator of the Markov process into the generator of the multiplicative semigroup and are consequences of Ito’s lemma: (a) Jump measure: exp[κ(y x)]η(dy|x). (b) First derivative term: ξ(x) + Γ (x)γ(x). (c) Second derivative term: Σ(x).
LONG-TERM RISK
199
(d) Level term: β(x) + |γ(x)|2 /2 + (exp[κ(y x)] − 1)η(dy x). The Markov chain example that we discussed above can be seen as a special case where γ ξ and Γ are all null. There are a variety of direct applications of this analysis. In the case of the stochastic discount factor introduced in Section 3.3, the generator encodes the local prices reflected in the local risk-return trade-off of Proposition 3.1. The level term that arises gives the instantaneous version of a risk-free rate. In the absence of jump risk, the increment to the drift gives the factor risk prices. The function κ shows us how to value jump risk in small increments in time. In a further application, Anderson, Hansen, and Sargent (2003) used this decomposition to characterize the relation among four alternative semigroups, each of which is associated with an alternative multiplicative process. Anderson, Hansen, and Sargent (2003) featured models of robust decision making. In addition to the generator for the original Markov process, a second generator depicts the worst-case Markov process used to support the robust equilibrium. There is a third generator of an equilibrium pricing semigroup, and a fourth generator of a semigroup used to measure the statistical discrepancy between the original model and the worst-case Markov model. 6. PRINCIPAL EIGENFUNCTIONS AND MARTINGALES As stated in the Introduction, we use a decomposition of the multiplicative functional to study long-run behavior. We construct this decomposition using an appropriate eigenfunction of the generator associated to the multiplicative functional. DEFINITION 6.1: A Borel function φ is an eigenfunction of the extended generator A with eigenvalue ρ if Aφ = ρφ. Intuitively, if φ is an eigenfunction, the “expected time derivative” of Mt φ(Xt ) is ρMt φ(Xt ). Hence the expected time derivative of exp(−ρ × t)Mt φ(Xt ) is zero. The next proposition formalizes this intuition. PROPOSITION 6.1: Suppose that φ is an eigenfunction of the extended generator associated with the eigenvalue ρ. Then exp(−ρt)Mt φ(Xt ) is a local martingale. t PROOF: Nt = Mt ψ(Xt ) − ψ(X0 ) − ρ 0 Ms ψ(Xs ) ds is a local martingale that is continuous from the right with left limits and thus is a semimartingale (Protter (2005, Chap. 3, Corollary to Theorem 26)); hence Yt = Mt φ(Xt ) is also a
200
L. P. HANSEN AND J. A. SCHEINKMAN
semimartingale. Since dNt = dYt − ρYt− dt, integration by parts yields t t exp(−ρt)Yt − Y0 = − ρ exp(−ρs)Ys ds + exp(−ρs) dYs =
0
0
t
exp(−ρs) dNs 0
Q.E.D.
It is the strictly positive eigenfunctions that interest us. DEFINITION 6.2: A principal eigenfunction of the extended generator is an eigenfunction that is strictly positive. COROLLARY 6.1: Suppose that φ is a principal eigenfunction with eigenvalue ρ for the extended generator of the multiplicative functional M. Then this multiplicative functional can be decomposed as ˆ t φ(X0 ) Mt = exp(ρt)M φ(Xt ) ˆ t = exp(−ρt)Mt (φ(Xt ))/(φ(X0 )) is a local martingale and a multiwhere M plicative functional. ˆ be the local martingale from Corollary 6.1. Since M ˆ is bounded from Let M below, the local martingale is necessarily a supermartingale and thus for t ≥ u, ˆ u ˆ t |Fu ) ≤ M E(M We are primarily interested in the case in which this local martingale is actually a martingale: ˆ is a martingale. ASSUMPTION 6.1: The local martingale M For Assumption 6.1 to hold, it suffices that the local martingale N introduced in the proof of Proposition 6.1 is a martingale. In Appendix C we give primitive conditions that imply Assumption 6.1. ˆ to find a new probability on the When Assumption 6.1 holds, we may use M sigma algebra generated by Ft for each t. Later, we will use this new probability to establish approximation results that hold for long horizons. We did not restrict φ to belong to the Banach space L where the semigroup ˆ is a supermartingale, Mt φ := exp(ρt)φ(x) × {Mt : t ≥ 0} was defined. Since M ˆ t |X0 = x] is always well defined. In addition, the semigroup {M ˆ t : t ≥ 0} is E[M well defined at least on the Banach space of bounded Borel measurable functions. Moreover:
LONG-TERM RISK
201
PROPOSITION 6.2: If φ is a principal eigenfunction with eigenvalue ρ for the extended generator of the multiplicative functional M, then for each t ≥ 0, exp(ρt)φ ≥ Mt φ. If, in addition, Assumption 6.1 holds then, for each t ≥ 0, (14)
Mt φ = exp(ρt)φ
Conversely, if φ is strictly positive, Mt φ is well defined for t ≥ 0, and (14) holds, ˆ is a martingale. then M PROOF: ˆ t |X0 = x] = 1 ≥ E[M
exp(−ρt) E[Mt φ(Xt )|X0 = x] φ(x)
ˆ is a martingale. Conversely, using (14) and the multiwith equality when M plicative property of M one obtains, E exp(−ρt)Mt φ(Xt )|Fs = exp(−ρt)Ms E Mt−s (θs )φ(Xt )|Xs = exp(−ρs)Ms φ(Xs )
Q.E.D.
Proposition 6.2 guarantees that under Assumption 6.1 a principal eigenfunction of the extended generator also solves the principal eigenvalue problem given by (14). Conversely, a strictly positive solution to the principal eigenvalue ˆ problem (14) yields a decomposition as in Corollary 6.1, where the process M is actually a martingale. In light of the decomposition given by Corollary 6.1, when the local marˆ is a martingale, we will sometimes refer to ρ as the growth rate tingale M ˆ as its martingale component, and of the multiplicative functional M, to M to (φ(X0 ))/(φ(Xt )) as its transient or stationary component. This decomposition is typically not unique, however. As we have defined them, there may be multiple principal eigenfunctions even after a normalization. Each of these principal eigenfunctions implies a distinct decomposition, provided that we establish that the associated local martingales are martingales. Since the martingale and the stationary components are correlated, it can happen that ˆ t (φ(X0 ))/(φ(Xt ))|X0 = x] diverges exponentially, challenging the interE[M pretation that ρ is the asymptotic growth rate of the semigroup. We take up this issue in the next section. REMARK 6.1: There are well known martingale decompositions of additive functionals with stationary increments used to deduce the central limit approximation and to characterize the role of permanent shocks in time series. The nonlinear, continuous-time Markov version of such a decomposition is At = ωt + mt − υ(Xt ) + υ(X0 )
202
L. P. HANSEN AND J. A. SCHEINKMAN
where {mt : t ≥ 0} is a martingale with stationary increments (see Bhattacharya (1982) and Hansen and Scheinkman (1995)). Exponentiating this decomposition yields another decomposition similar to the one in Corollary 6.1 except that the exponential of a martingale is not a martingale. When the martingale increments are constant functions of Brownian increments, then exponential adjustment has simple consequences.12 In particular, the exponential adjustment is offset by changing ω. With state dependent volatility in the martingale approximation, however, there is no longer a direct link between the additive and the multiplicative decompositions. In this case, the multiplicative decomposition of Corollary 6.1 is the one that is valuable for our purposes. EXAMPLE 6.1—Markov Chain Example: Recall that for a finite-state space, we can represent the Markov process in terms of a matrix U that serves as its generator. Previously we constructed the corresponding generator A of the multiplicative semigroup. For this example, the generator is a matrix. A principal eigenvector is found by finding an eigenvector of A with strictly positive entries. Standard Perron–Frobenius theory implies that if the chain is irreducible, since the multiplicative functional is strictly positive, there is such an eigenvector which is unique up to scale. While there is uniqueness in the case of an irreducible finite-state chain, there can be multiple solutions in more general settings. EXAMPLE 6.2—Markov Diffusion Example (Continued): Consider a multiplicative process M = exp(A), where (15)
¯ + At = βt
t
t
t
βf Xsf ds + 0
0
t
f
βo Xso ds +
Xs γf dBsf + 0
γo dBso 0
where X f and X o are given in Example 3.2. Guess an eigenfunction of the form exp(cf xf + co xo ). The corresponding eigenvalue equation is γf2
γo2 2 2 + cf [ξf (x¯ f − xf ) + xf γf σf ] + co [ξo (x¯ o − xo ) + γo σo ]
ρ = β¯ + βf xf + βo xo +
+ (cf )2 xf
12
σf2 2
+ (co )2
xf +
σo2 2
This is the case studied by Hansen, Heaton, and Li (2008).
203
LONG-TERM RISK
This generates two equations: one that equates the coefficients of xf to zero and another that equates the coefficients of xo to zero: 0 = βf +
γf2
+ cf (γf σf − ξf ) + (cf )2
2 0 = βo − co ξo
σf2 2
The solution to the first equation is (ξf − γf σf ) ± (ξf − γf σf )2 − σf2 (2βf + γf2 ) (16) cf = (σf )2 provided that (ξf − γf σf )2 − σf2 (2βf + γf2 ) ≥ 0 The solution to the second equation is (17)
co =
βo ξo
The resulting eigenvalue is ρ = β¯ +
σ2 γo2 + cf ξf x¯ f + co (ξo x¯ o + γo σo ) + (co )2 o 2 2
Write f
o ˆ t = exp(−ρt)Mt exp(cf Xt + co Xt ) M f exp(cf X0 + co X0o )
ˆ is multiplicative, we can express it as M ˆ t = exp(Aˆ t ), where Since M t t f Aˆ t = Xs (γf + cf σf ) dBsf + (γo + co σo ) dBso 0
(γf + cf σf )2 − 2
0
t 0
(γo + co σo )2 t Xsf ds − 2
ˆ is always a martingale. Hence we may use this Also, it can be shown that M martingale to define a new probability measure. Using this new measure entails appending an extra drift to the law of motion of X. The resulting distorted or twisted drift for X f is ξf (x¯ f − xf ) + xf σf (γf + cf σf )
204
L. P. HANSEN AND J. A. SCHEINKMAN
and the drift for X o is ξo (x¯ o − xo ) + σo (γo + co σo ) Later we will argue that only one of these solutions interests us. We will select a solution for cf so that the implied distorted process for X f remains stationary. Notice that ξf (x¯ f − xf ) + xf σf (γf + cf σf ) = ξf x¯ f ± xf (ξf − γf σf )2 − σf2 (2βf + γf2 ) For mean reversion to exist, we require that the coefficient on xf be negative. REMARK 6.2: At the cost of an increase in notational complexity, we could add an “affine” jump component as in Duffie, Pan, and Singleton (2000). Suppose that the state variable X o , instead of being an Ornstein–Uhlenbeck process, satisfies dXto = ξo (x¯ o − Xto ) dt + σo dBto + dZt where Z is a pure jump process whose jumps have a fixed probability distribution ν on R and arrive with intensity 1 xf + 2 with 1 ≥ 0, 2 ≥ 0. Suppose that the additive functional A has an additional jump term modeled using κ(y x) = κ(y ¯ o − xo ) for y = x and exp[κ(z)] ¯ dν(z) < ∞. The generator A has now an extra term given by ( 1 xf + 2 ) [φ(xf xo + z) − φ(xf xo )] exp[κ(z)] ¯ dν(z) Hence when φ(x) = exp(cf xf + co xo ), the extra term reduces to ¯ dν(z) ( 1 xf + 2 ) exp(cf xf + co xo ) [exp(co z) − 1] exp[κ(z)] As before we must have co =
βo ξo
and hence cf must solve σf2 + cf (γf σf − ξf ) + (cf )2 2 2 βo + 1 exp z − 1 exp[κ(z)] ¯ dν(z) ξo
0 = βf +
γf2
LONG-TERM RISK
205
The resulting eigenvalue is σ2 γ2 ρ = β¯ + o + cf ξf x¯ f + co (ξo x¯ o + γo σo ) + (co )2 o 2 2 βo + 2 exp z − 1 exp[κ(z)] ¯ dν(z) ξo 7. LONG-RUN DOMINANCE In this section we establish approximation results for semigroups that apply over long time horizons. The limiting result we justify is ψ (18) d ς ˆ lim exp(−ρt)Mt ψ = φ t→∞ φ where the limit is expressed in terms of principal eigenvalue ρ and eigenfunction φ for a collection of functions ψ and a measure ςˆ that we will characterize. We have illustrated that there may be multiple principle eigenfunctions. We show that at most one of these principle eigenfunctions is the one germane for establishing this limiting behavior. In light of (18), the eigenvalue ρ governs the growth (or decay) of the semigroup. When we rescale the semigroup to eliminate this growth (decay), the limiting state dependence is proportional to the dominant eigenfunction φ (which is itself only determined up to a scale factor) for alternative functions ψ. The precise characterization of this limiting behavior of the semigroup provides the fundamental inputs to our characterization of valuation over long time horizons. It provides us with a measure of longterm growth rates is asset payoffs and of long-term decay rates in the values assigned to these payoffs. It gives us ways to formalize long-term risk-return trade-offs for nonlinear Markov models as we will show in the next section. Prior to our more general investigation, we first illustrate the results in the case of a Markov chain. 7.1. Markov Chain Consider the finite-state Markov chain example with intensity matrix U. In this section we will study the long-run behavior of the semigroup by solving the eigenvalue problem Aφ = ρφ for an eigenvector φ with strictly positive entries and a real eigenvalue ρ. This solution exists whenever the chain is irreducible and the multiplicative functional is strictly positive. Given this solution, then Mt φ = exp(tA)φ = exp(ρt)φ
206
L. P. HANSEN AND J. A. SCHEINKMAN
The beauty of Perron–Frobenius theory is that ρ is the eigenvalue that dominates in the long run. Its real part is strictly larger than the real parts of all of the other eigenvalues. This property dictates its dominant role. To see this, suppose for simplicity that the matrix A has distinct eigenvalues, A = TDT−1 where T is a matrix with eigenvectors in each column and D is a diagonal matrix of eigenvalues. Then exp(tA) = T exp(tD)T−1 Let the first entry of D be ρ and let the first column of T be φ. Scaling by exp(−ρt) and taking limits, lim exp(−ρt) exp(tA)ψ = (φ∗ · ψ)φ
t→∞
where φ∗ is the first row of T −1 . Thus ρ determines the long-run growth rate of the semigroup. After adjusting for this growth, the semigroup has an approximate one factor structure in the long run. Provided that φ∗ · ψ is not zero, exp(−ρt)Mt ψ is asymptotically proportional to the dominant eigenvector φ. 7.2. General Analysis To establish this dominance more generally, we use the martingale construction as in the decomposition of Corollary 6.1 to build an alternative family of distorted Markov transition operators and apply known results about Markov operators to this alternative family. ˆ denote the exIn what follows we will maintain Assumption 6.1 and let A ˆ ˆ astended generator of the martingale M. We will also call the semigroup M ˆ the principal eigenfunction semigroup. This semigroup is well sociated with M defined at least on the space L∞ , and it maps constant functions into constant functions. Consistent with the applications that interest us, we consider only multiplicative functionals that are strictly positive. ASSUMPTION 7.1: The multiplicative functional M is strictly positive with probability 1. ˆ can be used to define a new meaAs we mentioned earlier, the martingale M sure on sets f ∈ Ft for any t. We are interested in the case where we may initialize the process X such that, under the new probability measure, the process X is stationary. The next assumption guarantees that this is possible:
LONG-TERM RISK
207
ASSUMPTION 7.2: There exists a probability measure ςˆ such that ˆ d ςˆ = 0 Aψ ˆ for all ψ in the L∞ domain of the generator A. ˆ for the expectation operator and the probability measure We write Eˆ and Pr obtained when we use ςˆ as the distribution of the initial state X0 and the marˆ to distort the transition probabilities, that is, for each event f ∈ Ft , tingale M ˆ t 1f |X0 = x] d ς(x) ˆ ˆ Pr(f ) = E[M ˆ For each x, the probability Pr(·|X 0 = x) is absolutely continuous with respect to probability measure implied by Pr conditioned on X0 = x when restricted to Ft for each t ≥ 0. Assumption 7.2 guarantees that ςˆ is a stationary distribution for the distorted Markov process. (For example, see Proposition 9.2 of Ethier and Kurtz (1986).) Furthermore, ψ(Xt ) (19) E[Mt ψ(Xt )|X0 = x] = exp(ρt)φ(x)Eˆ X0 = x φ(Xt ) If we treat exp(−ρt)φ(Xt ) as a numeraire, equation (19) is reminiscent of the familiar risk-neutral pricing in finance. Note, however, that the numeraire depends on the eigenvalue–eigenfunction pair, and equation (19) applies even when the multiplicative process does not define a price.13 Let Δˆ > 0 and consider the discrete time Markov process obtained by samˆ for j = 0 1 This discrete process is often referred pling the process at Δj to as a skeleton. In what follows we assume that the resulting discrete time process is irreducible. ASSUMPTION 7.3: There exists a Δˆ > 0 such that the discretely sampled process {XΔjˆ : j = 0 1 } is irreducible. That is, for any Borel set Λ of the state space D0 with ς(Λ) ˆ > 0, ∞
1{X ∈Λ} X0 = x > 0 Eˆ ˆ Δj
j=0
for all x ∈ D0 . 13 The idea of using an appropriately chosen eigenfunction of an operator to construct and analyze a twisted probability measure is also featured in the work of Kontoyiannis and Meyn (2003).
208
L. P. HANSEN AND J. A. SCHEINKMAN
Under Assumption 7.1 it is equivalent to assume that this irreducibility restriction holds under the original probability measure.14 We establish approximation results by imposing a form of stochastic stability under the distorted probability measure. We assume that the distorted Markov process satisfies the next assumption: ˆ ASSUMPTION 7.4: The process X is Harris recurrent under the measure Pr. That is, for any Borel set Λ of the state space D0 with positive ςˆ measure, ∞ ˆ 1{Xt ∈Λ} = ∞X0 = x = 1 Pr 0
for all x ∈ D0 . Among other things, this assumption guarantees that the stationary distribution ςˆ is unique. Under these assumptions, we characterize the role of the principal eigenvalue and function on the long-run behavior of the semigroup. The proof is given in Appendix B. ˆ satisfies Assumptions 6.1 and 7.1–7.4, and PROPOSITION 7.1: Suppose that M let Δ > 0. (a) For any ψ for which (|ψ|/φ) d ςˆ < ∞, ψ lim exp(−ρΔj)MΔj ψ = φ d ςˆ j→∞ φ for almost all (ς) ˆ x. (b) For any ψ for which ψ/φ is bounded, ψ d ςˆ lim exp(−ρt)Mt ψ = φ t→∞ φ for x ∈ D0 . The approximation implied by Proposition 7.1, among other things, gives a formal sense in which ρ is a long-run growth rate. It also provides more precise information, namely that after eliminating the deterministic growth, application of the semigroup to ψ is approximately proportional to φ, where the ˆ Subsequently, we will consider other versions of this scale coefficient is φψ d ς. approximation. We will also impose additional regularity conditions that will guarantee convergence without having to sample the Markov process. 14 Irreducibility and Harris recurrence are defined relative to a measure. This claim uses the ςˆ measure when verifying irreducibility for the original probability measure. Since irreducibility depends only on the probability distribution conditioned on X0 , it does not require that the X process be stationary under the original measure.
LONG-TERM RISK
209
7.2.1. Uniqueness As we mentioned earlier, there may exist more than one principal eigenfunction of the extended generator even after a scale normalization is imposed. To be of interest to us, a principal eigenfunction must generate a twisted probaˆ must be a martingale. As we showed in Example 6.2, bility measure, that is, M this requirement is not enough to guarantee uniqueness—there may exist more ˆ t is a martingale. than one principal eigenfunction for which the implied M However, in that example, only one of the two solutions we exhibited implies a Markov evolution for X that is stochastically stable. The other solution will also result in a Markov process, but it fails to be stationary. Recall that the two candidate drift distortions are ξf x¯ f ± xf (ξf − γf σf )2 − σf2 (2βf + γf2 ) Only when we select the solution associated with the negative root do we obtain a process that has a stationary density. This approach to uniqueness works much more generally. The next proposition establishes that stochastic stability requirements will typically eliminate the multiplicity of principal eigenvectors that generate appropriate twisted probabilities. More generally, it states that the eigenvalue of interest to us is always the smallest one. PROPOSITION 7.2: Assume that Assumption 7.1 is satisfied and that there exists a sampling interval Δ such that {XΔj : j = 0 1 } is irreducible. Suppose φ is a principal eigenfunction of the extended generator A of a multiplicative process M ˆ t : t ≥ 0} satisfies Assumptions 6.1, 7.2 with for which the associated process {M a stationary distribution ς, ˆ 7.3, and 7.4. Then the associated eigenvalue ρ is the smallest eigenvalue associated with a principal eigenfunction. Furthermore, if φ˜ is another positive eigenfunction associated with ρ, then φ˜ is proportional to φ (ςˆ almost surely). The proof can be found in Appendix B. This proposition guarantees that once we find a positive eigenfunction that generates a martingale that satisfies the required stochastic stability restriction, then we have found the only eigenfunction of interest (up to a constant scale factor). For instance, in Example 6.2 we only examined candidate eigenfunctions of a particular functional form, but found one that satisfies the assumptions of Proposition 7.2. Hence there exist no other eigenfunctions that satisfy these assumptions. 7.2.2. Lp Approximation When there exists a stationary distribution, it follows from Nelson (1958) ˆ t : t ≥ 0} can be extended to Lˆ p for any p ≥ 1 constructed that the semigroup {M
210
L. P. HANSEN AND J. A. SCHEINKMAN
using the measure d ς. ˆ The semigroup is a weak contraction. That is, for any t ≥ 0, ˆ t ψ p ≤ ψ p M where · p is the Lˆ p norm. PROPOSITION 7.3: Under Assumption 7.2, for p ≥ 1,
|Mt ψ|
p
provided that
1/p
1/p 1 1 p d ςˆ d ςˆ ≤ exp(ρt) |ψ| p φ φp
|ψ|p (1/φp ) d ςˆ < ∞.
PROOF: This follows from the weak contraction property established by Nelson (1958) together with the observation that
1 ψ ˆ exp(−ρt) Mt ψ = M t Q.E.D. φ φ REMARK 7.1: This proposition establishes an approximation in an Lp space ˆ Notice that φ itself is constructed using the transformed measure (1/φp ) d ς. always in this space. In particular, we may view the semigroup {Mt : t ≥ 0} as operating on this space. Proposition 7.3 shows that when the distorted Markov process constructed using the eigenfunction is stationary, ρ can be interpreted as an asymptotic growth rate of the multiplicative semigroup. The eigenfunction is used to characterize the space of functions over which the bound applies. We now produce a more refined approximation. ˆp Let Z pdenote the set of Borel measurable functions ψ such that ψ d ςˆ = 0 and |ψ| d ςˆ < ∞. Make the following supposition: ASSUMPTION 7.5: For any t > 0, sup ψ∈Zˆ p : ψ ≤1
ˆ t ψ p < 1 M
In the case of p = 2, Hansen and Scheinkman (1995) gave sufficient conditions for Assumption 7.5 to be satisfied.15 15 Assumption 7.5 for p = 2 is equivalent to requiring that the distorted Markov process be rho-mixing.
LONG-TERM RISK
211
7.4: Under Assumptions 7.2 and 7.5, for any ψ such that PψROPOSITION | φ |p d ςˆ < ∞, p 1/p exp(−ρt)Mt ψ − φ ψ d ςˆ 1 d ςˆ ≤ c exp(−ηt) φ φp for some rate η > 0 and positive constant c. 7.2.3. Lyapunov Functions Meyn and Tweedie (1993a) established, under an additional mild continuity condition, sufficient conditions for the assumptions in this section using a “Lyapunov function” method. In this subsection we will make the following assumption: ASSUMPTION 7.6: The process X is a Feller process under the probability meaˆ 16 sure associated with M. We use Lyapunov functions that are restricted to be norm-like. DEFINITION 7.1: A continuous function V is called norm-like if the set {x : V (x) ≤ r} is prec