Springer Series in Statistics Advisors: P. Bickel, P.J. Diggle, S.E. Feinberg, U. Gather, I. Olkin, S. Zeger
For furth...
13 downloads
465 Views
3MB Size
Report
This content was uploaded by our users and we assume good faith they have the permission to share this book. If you own the copyright to this book and it is wrongfully on our website, we offer a simple DMCA procedure to remove your content from our site. Start by pressing the button below!
Report copyright / DMCA form
Springer Series in Statistics Advisors: P. Bickel, P.J. Diggle, S.E. Feinberg, U. Gather, I. Olkin, S. Zeger
For further volumes: http://www.springer.com/series/692
Satoshi Aoki • Hisayuki Hara • Akimichi Takemura
Markov Bases in Algebraic Statistics
123
Satoshi Aoki Department of Mathematics and Computer Science Graduate School of Science and Engineering Kagoshima University Kagoshima, Japan
Hisayuki Hara Faculty of Economics Niigata University Niigata, Japan
Akimichi Takemura Department of Mathematical Informatics Graduate School of Information Science and Technology University of Tokyo Tokyo, Japan
ISSN 0172-7397 ISBN 978-1-4614-3718-5 ISBN 978-1-4614-3719-2 (eBook) DOI 10.1007/978-1-4614-3719-2 Springer New York Heidelberg Dordrecht London Library of Congress Control Number: 2012938710 © Springer Science+Business Media New York 2012 This work is subject to copyright. All rights are reserved by the Publisher, whether the whole or part of the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations, recitation, broadcasting, reproduction on microfilms or in any other physical way, and transmission or information storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology now known or hereafter developed. Exempted from this legal reservation are brief excerpts in connection with reviews or scholarly analysis or material supplied specifically for the purpose of being entered and executed on a computer system, for exclusive use by the purchaser of the work. Duplication of this publication or parts thereof is permitted only under the provisions of the Copyright Law of the Publisher’s location, in its current version, and permission for use must always be obtained from Springer. Permissions for use may be obtained through RightsLink at the Copyright Clearance Center. Violations are liable to prosecution under the respective Copyright Law. The use of general descriptive names, registered names, trademarks, service marks, etc. in this publication does not imply, even in the absence of a specific statement, that such names are exempt from the relevant protective laws and regulations and therefore free for general use. While the advice and information in this book are believed to be true and accurate at the date of publication, neither the authors nor the editors nor the publisher can accept any legal responsibility for any errors or omissions that may be made. The publisher makes no warranty, express or implied, with respect to the material contained herein. Printed on acid-free paper Springer is part of Springer Science+Business Media (www.springer.com)
Preface
Algebraic statistics is a rapidly developing field, where ideas from statistics and algebra meet and stimulate new research directions. Statistics has been relying on classical asymptotic theory as a basis for statistical inferences. This classical basis is still very useful. However, when the validity of asymptotic theory is in doubt, for example, when the sample size is small, statisticians rely more and more on various computational methods. Similarly, algebra has long been considered as the purest field of mathematics, far apart from practical computations. However, due mainly to the development of Gr¨obner basis technology, algebra is now becoming a field where computations for practical applications are feasible. It is an interesting trend, because historically algebra was invented to speed up various calculations. These two trends meet in the field of algebraic statistics. Algebraic algorithms are now very useful and essential for some practical statistical computations such as Markov chain Monte Carlo tests for discrete exponential families, which is the main topic of this book. On the other hand algebraic structures and computational needs of statistical models provide new challenging problems to algebraists. Some algebraic structures are naturally motivated from statistical modeling, but not necessarily from pure mathematical considerations. Algebraic statistics has two origins. One origin is the work by Pistone and Wynn in 1996 on the use of Gr¨obner bases for studying confounding relations in factorial designs of experiments. Another origin is the work by Diaconis and Sturmfels in 1998 on the use of Gr¨obner bases for constructing a connected Markov chain for performing conditional tests of a discrete exponential family. These two works opened up the whole new field of algebraic statistics. In this book we take up the second topic. We give a detailed treatment of results following the seminal work of Diaconis and Sturmfels. We also briefly consider the first topic in Chap. 15 of this book. As a general reference to the first origin of algebraic statistics we mention Pistone et al. [118]. For the second origin we mention Drton et al. [55], Pachter and Sturmfels [116], and our review paper [15]. For Japanese people the following two books are very useful: Hibi [86], and JST CREST Hibi team [93]. The Markov bases
v
vi
Preface
database (http://markov-bases.de/) provides very useful online material for studying Markov bases. Algebraic statistics gave us some exciting opportunities for research and collaboration. In particular we enjoyed working with Takayuki Hibi and Hidefumi Ohsugi, who are the leading researchers on Gr¨obner bases in Japan. Since 2008 Takayuki Hibi has a project, “Harmony of Gr¨obner Bases and the Modern Industrial Society,” in the mathematics program of the Japan Science and Technology Agency. Algebraic statistics offers a rare ground where algebraists and statisticians can talk about the same problems, albeit often with different terminologies. This book is intended for statisticians with minimal backgrounds in algebra. As we ourselves learned algebraic notions through working on statistical problems, we hope that this book with many practical statistical problems is useful for statisticians to start working on algebraic statistics. In preparing this book we very much benefited from comments of Takayuki Hibi, Hidehiko Kamiya, Kei Kobayashi, Satoshi Kuriki, Mitsunori Ogawa, Hidefumi Ohsugi, Toshio Sakata, Tomonari Sei, Kentaro Tanaka, and Ruriko Yoshida. Finally we acknowledge great editorial help from John Kimmel. Kagoshima, Japan Niigata, Japan Tokyo, Japan
Satoshi Aoki Hisayuki Hara Akimichi Takemura
Contents
Part I 1
2
3
Introduction and Some Relevant Preliminary Material
Exact Tests for Contingency Tables and Discrete Exponential Families . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 1.1 Independence Model of 2 × 2 Two-Way Contingency Tables.. . . . . 1.2 2 × 2 Contingency Table Models as Discrete Exponential Family . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 1.3 Independence Model of General Two-Way Contingency Tables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 1.4 Conditional Independence Model of Three-Way Contingency Tables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 1.4.1 Normalizing Constant of Hypergeometric Distribution for the Conditional Independence Model.. . . 1.5 Notation of Hierarchical Models for m-Way Contingency Tables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . Markov Chain Monte Carlo Methods over Discrete Sample Space.. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 2.1 Constructing a Connected Markov Chain over a Conditional Sample Space: Markov Basis . . . .. . . . . . . . . . . . . . . . . . . . 2.2 Adjusting Transition Probabilities by Metropolis–Hastings Algorithm . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . Toric Ideals and Their Gr¨obner Bases . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 3.1 Polynomial Ring . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 3.2 Term Order and Gr¨obner Basis . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 3.3 Buchberger’s Algorithm . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 3.4 Elimination Theory.. . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 3.5 Toric Ideals . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . .
3 3 8 10 14 18 19 23 23 27 33 33 35 38 39 39
vii
viii
Contents
Part II
Properties of Markov Bases
4
Definition of Markov Bases and Other Bases. . . . . . . .. . . . . . . . . . . . . . . . . . . . 4.1 Discrete Exponential Family.. . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 4.2 Definition of Markov Basis . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 4.3 Properties of Moves and the Lattice Basis . . . . . .. . . . . . . . . . . . . . . . . . . . 4.4 The Fundamental Theorem of Markov Basis . . .. . . . . . . . . . . . . . . . . . . . 4.5 Gr¨obner Basis from the Viewpoint of Markov Basis . . . . . . . . . . . . . . . 4.6 Graver Basis, Lawrence Lifting, and Logistic Regression . . . . . . . . .
47 47 50 51 54 59 60
5
Structure of Minimal Markov Bases . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 5.1 Accessibility by a Set of Moves . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 5.2 Structure of Minimal Markov Basis and Indispensable Moves . . . . 5.3 Minimum Fiber Markov Basis . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 5.4 Examples of Minimal Markov Bases . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 5.4.1 One-Way Contingency Tables . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 5.4.2 Independence Model of Two-Way Contingency Tables .. . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 5.4.3 The Unique Minimal Markov Basis for the Lawrence Lifting . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 5.5 Indispensable Monomials .. . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . .
65 65 66 71 72 72
Method of Distance Reduction . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 6.1 Distance Reducing Markov Bases . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 6.2 Examples of Distance-Reducing Proofs.. . . . . . . .. . . . . . . . . . . . . . . . . . . . 6.2.1 The Complete Independence Model of Three-Way Contingency Tables. . . . .. . . . . . . . . . . . . . . . . . . . 6.2.2 Hardy–Weinberg Model .. . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 6.3 Graver Basis and 1-Norm Reducing Markov Bases . . . . . . . . . . . . . . . . 6.4 Some Results on Minimality of 1-Norm Reducing Markov Bases . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . .
79 79 81
6
7
73 75
81 83 85 86
Symmetry of Markov Bases . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 91 7.1 Motivations for Invariance of Markov Bases . . .. . . . . . . . . . . . . . . . . . . . 91 7.2 Examples of Invariant Markov Bases . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 92 7.3 Action of Symmetric Group on the Set of Cells . . . . . . . . . . . . . . . . . . . . 93 7.4 Symmetry of a Toric Model and the Largest Group of Invariance .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 96 7.5 The Largest Group of Invariance for the Independence Model of Two-Way Tables . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 98 7.6 Characterizations of a Minimal Invariant Markov Basis . . . . . . . . . . . 100
Part III 8
73
Markov Bases for Specific Models
Decomposable Models of Contingency Tables . . . . . . .. . . . . . . . . . . . . . . . . . . . 109 8.1 Chordal Graphs and Decomposable Models .. . .. . . . . . . . . . . . . . . . . . . . 109 8.2 Markov Bases for Decomposable Models . . . . . .. . . . . . . . . . . . . . . . . . . . 111
Contents
8.3 8.4 8.5 8.6 9
ix
Structure of Degree 2 Fibers .. . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . Minimal Markov Bases for Decomposable Models . . . . . . . . . . . . . . . . Minimal Invariant Markov Bases . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . The Relation Between Minimal and Minimal Invariant Markov Bases . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . .
Markov Basis for No-Three-Factor Interaction Models and Some Other Hierarchical Models . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 9.1 No-Three-Factor Interaction Models for 3 × 3 × K Contingency Tables . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 9.2 Unique Minimal Markov Basis for 3 × 3 × 3 Tables . . . . . . . . . . . . . . . 9.3 Unique Minimal Markov Basis for 3 × 3 × 4 Tables . . . . . . . . . . . . . . . 9.4 Unique Minimal Markov Basis for 3 × 3 × 5 and 3 × 3 × K Tables for K > 5 . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 9.5 Indispensable Moves for Larger Tables . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 9.6 Reducible Models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 9.7 Markov Basis for Reducible Models . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 9.8 Markov Complexity and Graver Complexity . . .. . . . . . . . . . . . . . . . . . . . 9.9 Markov Width for Some Hierarchical Models . .. . . . . . . . . . . . . . . . . . . .
113 115 119 127 129 129 130 139 142 145 149 150 153 156
10 Two-Way Tables with Structural Zeros and Fixed Subtable Sums . . . 10.1 Markov Bases for Two-Way Tables with Structural Zeros . . . . . . . . . 10.1.1 Quasi-Independence Model in Two-Way Incomplete Contingency Tables. . . . . . . .. . . . . . . . . . . . . . . . . . . . 10.1.2 Unique Minimal Markov Basis for Two-Way Quasi-Independence Model . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 10.1.3 Enumerating Elements of the Minimal Markov Basis . . . . 10.1.4 Numerical Example of a Quasi-Independence Model . . . . 10.2 Markov Bases for Subtable Sum Problem . . . . . .. . . . . . . . . . . . . . . . . . . . 10.2.1 Introduction of Subtable Sum Problem .. . . . . . . . . . . . . . . . . . . 10.2.2 Markov Bases Consisting of Basic Moves . . . . . . . . . . . . . . . . 10.2.3 Markov Bases for Common Diagonal Effect Models .. . . . 10.2.4 Numerical Examples of Common Diagonal Effect Models .. . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . .
159 159
11 Regular Factorial Designs with Discrete Response Variables . . . . . . . . . 11.1 Conditional Tests for Designed Experiments with Discrete Observations . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 11.1.1 Conditional Tests for Log-Linear Models of Poisson Observations .. . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 11.1.2 Models and Aliasing Relations . . . . . . . .. . . . . . . . . . . . . . . . . . . . 11.1.3 Conditional Tests for Logistic Models of Binomial Observations . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 11.1.4 Example: Wave-Soldering Data . . . . . . . .. . . . . . . . . . . . . . . . . . . .
181
159 161 164 167 168 168 169 172 176
181 181 184 191 193
x
Contents
11.2 Markov Bases and Corresponding Models for Contingency Tables . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 11.2.1 Rewriting Observations as Frequencies of a Contingency Table . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 11.2.2 Models for the Two-Level Regular Fractional Factorial Designs with 16 Runs . . . . . . . .. . . . . . . . . . . . . . . . . . . . 11.2.3 Three-Level Regular Fractional Factorial Designs and 3s−k Continent Tables . . . .. . . . . . . . . . . . . . . . . . . . 12 Groupwise Selection Models . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 12.1 Examples of Groupwise Selections. . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 12.1.1 The Case of National Center Test in Japan .. . . . . . . . . . . . . . . 12.1.2 The Case of Hardy–Weinberg Models for Allele Frequency Data . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 12.2 Conditional Tests for Groupwise Selection Models . . . . . . . . . . . . . . . . 12.2.1 Models for NCT Data . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 12.2.2 Models for Allele Frequency Data . . . . .. . . . . . . . . . . . . . . . . . . . 12.3 Gr¨obner Basis for Segre–Veronese Configuration .. . . . . . . . . . . . . . . . . 12.4 Sampling from the Gr¨obner Basis for the Segre–Veronese Configuration . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 12.5 Numerical Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 12.5.1 The Analysis of NCT Data . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 12.5.2 The Analysis of Allele Frequency Data . . . . . . . . . . . . . . . . . . . 13 The Set of Moves Connecting Specific Fibers . . . . . . .. . . . . . . . . . . . . . . . . . . . 13.1 Discrete Logistic Regression Model with One Covariate . . . . . . . . . . 13.2 Discrete Logistic Regression Model with More than One Covariate .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 13.3 Numerical Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 13.3.1 Exact Tests of Logistic Regression Model . . . . . . . . . . . . . . . . 13.4 Connecting Zero-One Tables with Graver Basis . . . . . . . . . . . . . . . . . . . . 13.5 Rasch Model .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 13.6 Many-Facet Rasch Model .. . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 13.7 Latin Squares and Zero-One Tables for No-Three-Factor Interaction Models . . . . . . . .. . . . . . . . . . . . . . . . . . . . Part IV
194 194 200 203 209 209 209 212 213 214 215 217 219 219 219 221 229 229 231 238 238 240 241 242 245
Some Other Topics of Algebraic Statistics
14 Disclosure Limitation Problem and Markov Basis .. . . . . . . . . . . . . . . . . . . . 14.1 Swapping with Some Marginals Fixed . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 14.2 E-Swapping.. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 14.3 Equivalence of Degree-Two Square-Free Move of Markov Bases and Swapping of Two Records .. . . . . . . . . . . . . . . . . . 14.4 Swappability Between Two Records . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 14.5 Searching for Another Record for Swapping . . .. . . . . . . . . . . . . . . . . . . .
251 251 252 253 254 257
Contents
xi
15 Gr¨obner Basis Techniques for Design of Experiments . . . . . . . . . . . . . . . . . 15.1 Design Ideals . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 15.2 Identifiability of Polynomial Models and the Quotient with Respect to the Design Ideal . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 15.3 Regular Two-Level Designs . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 15.4 Indicator Functions .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . .
261 261
16 Running Markov Chain Without Markov Bases. . . .. . . . . . . . . . . . . . . . . . . . 16.1 Performing Conditional Tests When a Markov Basis Is Not Available . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 16.2 Sampling Contingency Tables with a Lattice Basis . . . . . . . . . . . . . . . . 16.3 A Lattice Basis for Higher Lawrence Configuration . . . . . . . . . . . . . . . 16.4 Numerical Experiments . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 16.4.1 No-Three-Factor Interaction Model . . .. . . . . . . . . . . . . . . . . . . . 16.4.2 Discrete Logistic Regression Model . . .. . . . . . . . . . . . . . . . . . . .
275
262 267 269
275 275 277 278 278 282
References .. .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 287 Index . . . . . . . . .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . 295
Part I
Introduction and Some Relevant Preliminary Material
In Part I of this book we give introductory material on performing exact tests using Markov basis and a short survey on Gr¨obner basis. In Chap. 1, using the example of Fisher’s exact test for the independence model in two-way contingency tables, we give an introduction to exact tests. We also discuss conditional independence model for three-way contingency tables. In Chap. 2 we discuss basic notions of Markov chain and Markov bases. In particular we explain the Metropolis-Hastings procedure for adjusting transition probabilities to achieve a desired stationary distribution. Chapter 3 is a brief summary of results in the theory of Gr¨obner basis. In this chapter we collect relevant facts on ideals in polynomial rings and their Gr¨obner bases, which are often needed for discussion of Markov bases. In this book, R, Q, Z, N = {0, 1, . . . } stand for the set of reals, rationals, integers and nonnegative integers, respectively. For a positive integer n, we denote the set of n-dimensional vectors of elements from R, Q, Z, N, by Rn , Qn , Zn , Nn , respectively.
Chapter 1
Exact Tests for Contingency Tables and Discrete Exponential Families
1.1 Independence Model of 2 × 2 Two-Way Contingency Tables The theory of exact tests for discrete exponential families is best explained by Fisher’s exact test of homogeneity of two binomial populations and the independence model of 2 × 2 contingency tables. We begin with the test of homogeneity of two binomial populations. An excellent introduction to contingency tables is given in [59]. We also refer to Agresti [3] as a survey paper of the exact methods. Fisher’s exact test can be applied to three different sampling schemes: (i) test of homogeneity of two binomial populations, (ii) test of independence in multinomial sampling for 2 × 2 tables, (iii) the main effect model for logarithms of mean parameters of independent Poisson random variables in 2 × 2 tables. We discuss these three sampling schemes in this order. With this example we confirm that the same Markov basis can be used for different sampling schemes. Let X be distributed according to a binomial distribution Bin(n1 , p1 ), where n1 is the number of trials and p1 is the success probability. Let Y be distributed according to the binomial distribution Bin(n2 , p2 ). Suppose that X and Y are independent. We can display X and Y in the following 2 × 2 contingency table: X n1 − X n1 Y n2 − Y n2 t n−t n where t = X + Y and n = n1 + n2 . The hypothesis of homogeneity of two binomial populations is specified as H : p1 = p2 .
S. Aoki et al., Markov Bases in Algebraic Statistics, Springer Series in Statistics 199, DOI 10.1007/978-1-4614-3719-2 1, © Springer Science+Business Media New York 2012
3
4
1 Exact Tests for Contingency Tables and Discrete Exponential Families
The joint probability function of X and Y is written as p(x, y) =
n1 x n2 y p1 (1 − p1)n1 −x p2 (1 − p2)n2 −y . x y
Note that here we are using the conventional notational distinction between random variables X,Y in capital letters and their values x, y in lower-case letters. However, for the rest of this book for notational simplicity we do not necessarily stick to this convention. Under the null hypothesis H, the joint probability is written as n1 n2 x+y p(x, y) = p1 (1 − p1)n−(x+y) . x y
(1.1)
This joint probability depends on (x, y) through t = x + y. Therefore from the factorization theorem for sufficient statistics (see Sect. 2.6 of Lehmann and Romano [98]), T = X + Y is a sufficient statistic under the null hypothesis H. Given T = t, the conditional distribution of X does not depend on the value of p1 = p2 . Hence by using X as the test statistic, we obtain a testing procedure, whose level does not depend on the value of p1 = p2 ; that is, we obtain a similar test (Sect. 4.3 of [98]). Under H the distribution of T = X + Y is the binomial distribution Bin(n, p1 ). Therefore the conditional distribution of X given T = t is calculated as n n P(X = x | T = t) = =
1 2 pt (1 − p1)n−t xn +nt−x t 1 1 2 p1 (1 − p1)n−t t
n n 1
=
x
2
nt−x t
n1 !n2 !t!(n − t)! . n!x!(n1 − x)!(t − x)!(n2 − t + x)!
(1.2)
This is a hypergeometric distribution. Indeed the conditional distribution does not depend on the value of p1 = p2 . The null hypothesis H is rejected if the value of X is too large or too small. Because the distribution of X is not symmetric when n1 = n2 , the rejection region is usually determined by unbiasedness consideration. For optimality of similar unbiased test see Sect. 4.4 of [98]. This testing procedure is called Fisher’s exact test. It is an exact test because the significance level is computed from the hypergeometric distribution. It is also called a conditional test because we use the conditional null distribution given T = t. In contrast, the usual large-sample test is based on the large-sample normal approximation to the following “z-statistic”: z=
pˆ1 − pˆ 2 pˆ1 (1− pˆ 1 ) n1
+
pˆ2 (1− pˆ 2 ) n2
,
pˆ1 =
X Y , pˆ2 = . n1 n2
1.1 Independence Model of 2 × 2 Two-Way Contingency Tables Table 1.1 Cross-classification of belief in afterlife by gender
5
Gender
Belief in Afterlife Yes No or Undecided
Females Males
509 398
116 104
The test based on z is an unconditional test. However, when the sample size is small, it is desirable to use the exact test (Haberman [68]). In the case of homogeneity of two binomial populations, we saw that X +Y (total number of successes) is a sufficient statistic. We could also take n − X − Y (total number of failures) or even the pair (X +Y, n − X −Y ) as a sufficient statistic. Note that the pair contains redundancy, but it is still a sufficient statistic, because fixing (x + y, n − x − y) is equivalent to fixing x + y. Furthermore we could also include n1 and n2 into the sufficient statistic, although these values are fixed in the case of homogeneity of two binomial populations. Indeed T = (X + Y, n − X − Y, n1 , n2 ) is a sufficient statistic, because given the value of the vector T the conditional distribution of X is the hypergeometric distribution in (1.2) and it does not depend on p1 = p2 . Next we discuss the multinomial sampling scheme. Let xi j , i = 1, 2, j = 1, 2, be frequencies of four cells of a 2 × 2 contingency table. The row sums and the column sums (i.e., the marginal frequencies) are denoted as xi+ , x+ j , i, j = 1, 2. The total sample size is n = x11 + x12 + x21 + x22. The data are displayed as follows. x11 x12 x1+ x21 x22 x2+ x+1 x+2 n
(1.3)
At this point we mention some customary terminology of contingency tables. We look at the frequencies in (1.3) as the frequencies of a two-dimensional random variable Y = (Y1 ,Y2 ), such that both Y1 and Y2 take the values 1 or 2. For example, in Table 1.1 taken from Chap. 2 of [5], Y1 is the gender and Y2 is the belief in afterlife. The values taken by a variable are often called levels of the variable. For example, in Table 1.1 two levels of the variable “gender” are “female” and “male”. In this terminology xi j is the joint frequency such that Y1 takes the level i and Y2 takes the level j. The row and the column of the contingency table are sometimes called axes of the table. Then Y1 is the random variable for the first axis and Y2 is the random variable for the second axis. Let pi j ≥ 0,
i = 1, 2,
j = 1, 2,
2
∑
pi j = 1
i, j=1
be the probabilities of the cells. In a single multinomial trial, we observe one of the four cells according to the probabilities. With n independent and identical
6
1 Exact Tests for Contingency Tables and Discrete Exponential Families
multinomial trials, the joint probability function of X = (X11 , X12 , X21 , X22 ) is given as n p(xx) = (1.4) px11 px12 px21 px22 . x11 , x12 , x21 , x22 11 12 21 22 As in this example, we use the boldface letter x for the vector of frequencies and call x the frequency vector. When necessary, we make the notational distinction between column vector and row vector. For example, x is meant as a column vector when we write x = (x11 , x12 , x21 , x22 ) . We use for denoting the transpose of a vector or a matrix in this book. Let pi+ = pi1 + pi2 , i = 1, 2, denote the marginal probability of the first variable of the contingency table and similarly let p+ j = p1 j + p2 j , j = 1, 2, denote the marginal probability of the second variable. The hypothesis of independence H in the multinomial sampling scheme is specified as follows: H : pi j = pi+ p+ j ,
i = 1, 2,
j = 1, 2.
(1.5)
On the other hand, if there is no restriction on the probability vector p = (p11 , p12 , p21 , p22 ), except that the elements of p are nonnegative and sum to one, we call the model saturated. Write ri = pi+ and c j = p+ j . Then pi j = ri c j under H. Note that in (1.5), 2
1 = ∑ pi+ = i=1
2
∑ p+ j .
j=1
However, when we write ri = pi+ and c j = p+ j , we can remove the restriction 1 = r1 + r2 = c1 + c2 and only assume that ri and c j are nonnegative such that the total probability is 1: 1=
2
∑
ri c j = (r1 + r2 )(c1 + c2 ).
i, j=1
Furthermore we can incorporate the total probability into the normalizing constant and write the probability as pi j =
1 ri c j , (r1 + r2 )(c1 + c2 )
i, j = 1, 2,
(1.6)
where we only assume that ri and c j are nonnegative without any further restrictions. In this example of 2 × 2 tables, the normalizing constant is obvious and the above discussion may be pedantic. However, for more general models of contingency tables, it is best to consider the joint probability in the form of (1.6). Under H, with the normalization 1 = (r1 + r2 )(c1 + c2 ), the joint probability function p(xx) is written as
1.1 Independence Model of 2 × 2 Two-Way Contingency Tables
7
n p(xx) = (r1 c1 )x11 (r1 c2 )x12 (r2 c1 )x21 (r2 c2 )x22 x11 , x12 , x21 , x22 n x x x x r 1+ r 2+ c +1 c +2 = x11 , x12 , x21 , x22 1 2 1 2 n x x x x = p 1+ p 2+ p +1 p +2 . x11 , x12 , x21 , x22 1+ 2+ +1 +2
(1.7)
Hence the sufficient statistic under H is given as T = (x1+ , x2+ , x+1 , x+2 ). Given T , in the case of the 2 × 2 table, there is only one degree of freedom in x . Namely, if x11 is given, then the other values x12 , x21 , x22 are automatically determined as x12 = x1+ − x11,
x21 = x+1 − x11 ,
x22 = n − x1+ − x+1 + x11 .
As mentioned above, let us consider (i, j) as the pair of levels of two random variables Y1 and Y2 . Under the null hypothesis H of independence in (1.5), Y1 and Y2 are independent. Suppose that we observe n independent realizations (y11 , y12 ), . . . , (yn1 , yn2 ) of (Y1 ,Y2 ). Then xi+ is the number of times that Y1 takes the value i. Hence x1+ is distributed according to the binomial distribution Bin(n, p1+ ). Similarly x+1 is distributed according to the binomial distribution Bin(n, p+1 ). Furthermore they are independent. Therefore the joint distribution of x1+ and x+1 is written as n n x1+ x2+ x x p(x1+ , x+1 ) = (1.8) p p p +1 p +2 . x1+ 1+ 2+ x+1 +1 +2 From (1.7) and (1.8) it follows that the conditional distribution of X11 given the sufficient statistic is computed as follows. x1+ x2+ x+1 x+2 n x11 ,x12 ,x21 ,x22 p1+ p2+ p+1 p+2 n x1+ x2+ n x+1 x+2 x1+ p1+ p2+ x+1 p+1 p+2
p(x11 | x1+ , x2+ , x+1 , x+2 ) =
=
n x11 ,x12 ,x21 ,x22 n n x1+ x+1
=
x1+ ! x2+ ! x+1 ! x+2 ! . n! x11 ! x12 ! x21 ! x22 !
(1.9)
This is again a hypergeometric distribution. Equation (1.9) is clearly the same as (1.2) if we write the row sums and the column sums as n1 = x1+ , n2 = x2+ , t = x+1 , n − t = x+2 . Therefore Fisher’s exact test is the same in this multinomial sampling scheme as in the case of testing the homogeneity of two binomial populations. Note that in this scheme n is fixed and x2+ = n − x1+ and x+2 = n − x+1 can be omitted from the sufficient statistic T = (x1+ , x2+ , x+1 , x+2 ). However, as in the first scheme we can allow the redundancy in the sufficient statistic.
8
1 Exact Tests for Contingency Tables and Discrete Exponential Families
Finally we consider the sampling scheme of Poisson random variables. Let Xi j , i, j = 1, 2, be independently distributed according to the Poisson distribution with mean λi j . The joint probability of X is written as p(xx) =
x
2
λi ji j
i, j=1
xi j !
∏
e− λ i j .
Consider the null hypothesis H that λi j can be factored as H : λi j = ri c j ,
i, j = 1, 2,
where ri , c j are nonnegative. Again by writing down the joint probability under the null hypothesis H, we can easily check that a sufficient statistic under H is given by T = (x1+ , x2+ , x+1 , x+2 ), where now the redundancy is only in x+2 = x1+ + x2+ − x+1 . Instead of writing out the joint probability, we use the following property of independent Poisson random variables for verifying that T is a sufficient statistic under H. Let n = X11 + X12 + X21 + X22 . Then n is distributed as the Poisson random variable with mean μ = ∑2i, j=1 λi j . Under H, μ = (r1 + r2 )(c1 + c2 ). Given n, the conditional distribution of (X11 , X12 , X21 , X22 ) is the multinomial distribution with cell probabilities pi j = λi j /μ . Under H, the cell probability is written as pi j =
1 ri c j , (r1 + r2 )(c1 + c2 )
i, j = 1, 2,
which is the same as (1.6). From this fact we see that T = (x1+ , x2+ , x+1 , x+2 ) is a sufficient statistic under H. Given T , the conditional distribution of x is the same as the multinomial case; that is, X11 follows the hypergeometric distribution in (1.9). We now note the relation between the cell frequencies and the sufficient statistic. The column vector of cell frequencies x = (x11 , x12 , x21 , x22 ) and the column vector of the sufficient statistic (x1+ , x2+ , x+1 , x+2 ) are related as follows: ⎛ ⎞ ⎛ ⎞⎛ ⎞ 1100 x1+ x11 ⎜x2+ ⎟ ⎜0 0 1 1⎟ ⎜x12 ⎟ ⎜ ⎟=⎜ ⎟⎜ ⎟ (1.10) ⎝x+1 ⎠ ⎝1 0 1 0⎠ ⎝x21 ⎠ . x+2 x22 0101 We write this as t = Axx and call the matrix A the configuration for the above three models.
1.2 2 × 2 Contingency Table Models as Discrete Exponential Family In the previous section we explained three sampling schemes for 2 × 2 contingency tables and pointed out that they share the same sufficient statistic when redundancies are allowed. In this section we present the standard formulation of the sampling
1.2 2 × 2 Contingency Table Models as Discrete Exponential Family
9
schemes as discrete exponential family models. We confirm that the sufficient statistics under the null hypothesis correspond to nuisance parameters. Hence fixing the sufficient statistic has the effect of eliminating the nuisance parameters and the resulting conditional test is a similar test. Here we only consider the multinomial scheme of the previous section, because the other cases can be treated in a similar manner. A family of joint probability functions p(xx) = p(xx; θ ), θ ∈ Θ , is said to form an exponential family (see Sect. 2.7 of [98]) if p(xx, θ ) is written in the following form.
p(xx; θ ) = h(xx)exp
k
∑ Tj (xx)φ j (θ ) − ψ (θ )
.
(1.11)
j=1
By the factorization theorem (Sect. 2.6 of [98]), T = (T1 (xx), . . . , Tk (xx)) is a sufficient statistic of this family. Note that p(xx; θ ) and ψ (θ ) depend on θ only through φ = (φ1 , . . . , φk ) and we can write ψ (φ ) instead of ψ (θ ). In Chap. 4 we simply denote φ j (θ ) itself as θ j . Let pi j , i, j = 1, 2, denote the cell probabilities in the multinomial sampling of a 2 × 2 contingency table. Now consider the following transformation:
φ1 = log
p12 , p22
φ2 = log
p21 , p22
λ = log
p11 p22 . p12 p21
(1.12)
In the region where the elements of the probability vector p = (p11 , p12 , p21 , p22 ) are positive, the transformation is one-to-one and the inverse transformation is written as p11 = p12 =
eφ1 , 1 + eφ1 + eφ2 + eφ1 +φ2 +λ
eφ2 , + eφ1 +φ2 +λ 1 = . 1 + eφ1 + eφ2 + eφ1 +φ2 +λ
p21 = p22
eφ1 +φ2 +λ , + eφ1 +φ2 +λ
1 + eφ1 + eφ2
1 + eφ1 + eφ2
(1.13)
Substituting this into (1.4) we can write the joint probability function of x as n p(xx) = exp (x11 + x12 )φ1 + (x11 + x21)φ2 + x11 λ x11 , x12 , x21 , x22 −n log(1 + eφ1 + eφ2 + eφ1 +φ2 +λ ) . (1.14) This is written in the form (1.11) and hence the family of p(xx) forms an exponential family. By putting r1 = eφ1 , r2 = 1, c1 = eφ2 , c2 = 1 we see that the null hypothesis of the independence (1.5) is equivalently written as H : λ = 0.
10
1 Exact Tests for Contingency Tables and Discrete Exponential Families
Note that λ is the parameter of interest for the null hypothesis and φ1 , φ2 are the nuisance parameters under the null hypothesis. Under the null hypothesis, λ = 0 is no longer a parameter of the family of distributions and the distributions under the null hypothesis are parametrized by the nuisance parameters φ1 , φ2 . In (1.14) the sufficient statistic corresponding to (φ1 , φ2 ) is x1+ = x11 + x12 ,
x+1 = x11 + x21.
In (1.11) and (1.14) we considered the joint probability of the frequency vector. In fact, when we consider a single observation n = 1, then the cell probabilities are already in the exponential family form. Write log p = (log p11 , log p12 , log p21 , log p22 ),
ψ (φ1 , φ2 ) = log(1 + eφ1 + eφ2 + eφ1 +φ2 ). Taking the logarithms of pi j in (1.13) with λ = 0, in a matrix form we can write ⎛
1 ⎜0 log p = (φ1 , 0, φ2 , 0) ⎜ ⎝1 0
⎞ 100 0 1 1⎟ ⎟ − ψ (φ1 , φ2 ) × (1, 1, 1, 1). 0 1 0⎠ 101
(1.15)
Note that the matrix on the right-hand side is the configuration A appearing in the right-hand side of (1.10).
1.3 Independence Model of General Two-Way Contingency Tables Generalizing the discussion of the previous section we now consider the independence model of general I × J two-way contingency tables. The discussion on three sampling schemes is entirely the same as in the case of 2 × 2 tables. Therefore we only discuss the multinomial sampling. Let pi j , i = 1, . . . , I, j = 1, . . . , J, denote the cell probabilities of an I × J contingency table. Let pi+ and p+ j denote the marginal probabilities. The null hypothesis of independence is written as H : pi j = pi+ p+ j ,
i = 1, . . . , I,
j = 1, . . . , J.
We can also write pi j = ri c j without requiring that ri s and c j s correspond to probabilities. Let xi j denote the frequency of the cell (i, j). A sufficient statistic
1.3 Independence Model of General Two-Way Contingency Tables
11
T under the null hypothesis H is the set of the row sums xi+ , i = 1, . . . , I and the column sums x+ j , j = 1, . . . , J. Let n denote the total sample size. Under the null hypothesis the joint probability of x = {xi j } is written as
n p(xx) = x11 , . . . , xIJ =
n x11 , . . . , xIJ
I
J
∏ ∏ (pi+ p+ j )xi j i=1 j=1 I
J
∏ pi+i+ ∏ p++jj . x
i=1
x
j=1
Also, under the null hypothesis, as in the case of 2 × 2 tables, the vector of row sums {xi+ } and the vector of column sums {x+ j } are independently distributed according to multinomial distributions: n x xI+ p({xi+ }) = , p 1+ · · · pI+ x1+ , . . . , xI+ 1+ n x x+J . p({x+ j }) = p +1 · · · p+J x+1 , . . . , x+J +1 From this fact, the conditional distribution of x = {xi j } given the sufficient statistic t is written as n p({xi j }) x ,...,x = n 11 IJ n p(xx | T = t ) = p({xi+ })p({x+ j }) x ,...,x x ,...,x 1+
=
∏Ii=1 xi+ !
∏Jj=1 x+ j !
n! ∏i, j xi j !
I+
+1
+J
.
(1.16)
This distribution is often called the multivariate hypergeometric distribution. However in this book we show many variations of distributions of this type and we often refer to them simply as hypergeometric distributions. Given the row sums and the column sums, the degrees of freedom in the frequency vector x is (I − 1) × (J − 1) because the elements of the last row and the last column are determined uniquely from the other elements. This degrees of freedom is also the dimension of the parameter of interest when the joint probability distribution is written in the exponential family form. More precisely let piJ , pIJ pI j φ2 j = log , pIJ pi j pIJ λi j = log , piJ pI j
φ1i = log
i = 1, . . . , I − 1, j = 1, . . . , J − 1, i = 1, . . . , I − 1, j = 1, . . . , J − 1.
(1.17)
12
1 Exact Tests for Contingency Tables and Discrete Exponential Families
Then the null hypothesis is written as H : λi j = 0,
i = 1, . . . , I − 1,
j = 1, . . . , J − 1.
One consequence of the multidimensionality of the parameter of interest is that there is no unique best choice for a test statistic, even under the requirement of similarity and unbiasedness. Let xi+ x+ j mˆ i j = n pˆi j = n denote the “expected frequency” of the cell (i, j), where pˆi j is the maximum likelihood estimate (MLE) of pi j . For testing the null hypothesis of independence, popular test statistics are Pearson’s chi-square test (xi j − mˆ i j )2 ≥ cα ⇒ reject H mˆ i j
χ 2 (xx) = ∑ ∑ i
j
and the (twice log) likelihood ratio test G2 (xx) = 2 ∑ ∑ xi j log i
j
xi j ≥ cα ⇒ reject H, mˆ i j
where cα is the critical value for the respective test statistic. G2 (xx) is actually twice the logarithm of the likelihood ratio. In the usual asymptotic theory, cα is approximated by the upper α -quantile of the chi-square distribution with (I − 1)(J − 1) degrees of freedom. In this book we denote the chi-square distribution with m degrees of freedom by χm2 . These two statistics are “omnibus test statistics” in the sense that all possible alternative hypotheses are roughly equally treated. When some specific deviations from the null hypothesis are expected, then a more suitable test statistic, which is sensitive against the deviation, can be used. For performing a test of H, once a test statistic is chosen, it only remains to evaluate its null distribution. As in the previous section, in this book we consider exact tests; that is, we are interested in the distribution of a test statistic under the hypergeometric distribution (1.16). At this point we investigate the conditional sample space; that is, the set of contingency tables given the sufficient statistic for I × J case. As in the 2 × 2 case, the relation between the sufficient statistic and the frequency vector is written in a matrix form. Let t = (x1+ , . . . , xI+ , x+1 , . . . , x+J ) denote the (column) vector of the sufficient statistic and let x = (x11 , x12 , . . . , x1J , x21 , . . . , xIJ ) denote the frequency vector. Then t = Axx,
(1.18)
where the configuration A is an (I + J) × IJ matrix consisting of 0s and 1s as in (1.10).
1.3 Independence Model of General Two-Way Contingency Tables
13
An explicit form of A can be given using the Kronecker product notation. For two matrices, C = {ci j } : m1 × n1 and D : m2 × n2, their Kronecker product C ⊗ D is an m1 m2 × n1 n2 matrix of the following block form ⎛
⎞ c11 D . . . c1n1 D ⎜ .. ⎟ . C ⊗ D = ⎝ ... . ⎠ cm1 1 D . . . cm1 n1 D
(1.19)
Let 1 n = (1, . . . , 1) denote the n-dimensional vector consisting of 1s and let Em denote an m × m identity matrix. Then A in (1.18) is written as EI ⊗ 1J A= . 1 I ⊗ EJ Alternatively let e j,n = (0, . . . , 0, 1, 0, . . . , 0) ∈ Rn denote the jth standard basis vector of Rn . When the dimension n is clear from the context, we simply write the standard basis vector as e j instead of e j,n . Then the columns of A are of the form
e i,I , e j,J
i = 1, . . . , I,
j = 1, . . . , J.
(1.20)
We sometimes denote the stacked vector in (1.20) as e i,I ⊕ e j,J =
e i,I . e j,J
(1.21)
It is easily checked that the rank of A is rank A = I + J − 1. Hence the dimension of the kernel of A is given as dim ker A = IJ − (I + J − 1) = (I − 1)(J − 1). As mentioned above, this dimension corresponds to the fact that, if we ignore the requirement of nonnegativity, we can choose the elements of the first I − 1 rows and the first J − 1 columns freely. With the additional requirement of nonnegativity, the conditional sample space given the sufficient statistic is defined as Ft = {xx ∈ ZIJ | x ≥ 0 ,tt = Axx},
(1.22)
where x ≥ 0 means that the elements of x are nonnegative. We call Ft the fiber of t (or also call it the t-fiber). The hypergeometric distribution in (1.16) is a probability
14
1 Exact Tests for Contingency Tables and Discrete Exponential Families
distribution over the fiber Ft . When a test statistic φ (xx) is given, we want to evaluate the distribution of φ (xx ), where x is distributed according to the hypergeometric distribution over Ft . Suppose that φ is chosen such that a larger value of φ indicates more deviation from the null hypothesis, as in Pearson’s chi-square statistic or the likelihood ratio statistic. Then testing can be conveniently performed via p-value. Let x o denote the observed contingency table. The p-value of x o is defined as p = P(φ (xx) ≥ φ (xxo ) | H) = x ∈Ft
∑
,φ (xx)≥φ (xxo )
p(xx | t = Axxo , H),
(1.23)
which is the probability under the hypergeometric distribution of observing the value φ (xx) which is larger than or equal to φ (xxo ). Given the level of significance α , we reject H if p ≤ α . There are three methods to evaluate the p-value in (1.23). 1. By enumerating Ft , t = Axxo , and performing the sum in (1.23) for all x ∈ Ft such that φ (xx) ≥ φ (xxo ). 2. Directly sampling x from the hypergeometric distribution and approximating (1.23) by Monte Carlo simulation. 3. By sampling x by a Markov chain whose stationary distribution is the hypergeometric distribution, that is, by a Markov chain Monte Carlo method. Clearly the enumeration is the best if it is feasible. However, when the row sums and the column sums become large, the size of the fiber Ft becomes large and the enumeration becomes infeasible. In the case of the independence model of this section, direct sampling of a frequency vector from the hypergeometric distribution is easy to carry out. In more complicated models treated later in the book, though, direct sampling is not easy. On the other hand, there exists a general theory of constructing a Markov chain having the hypergeometric distribution as the stationary distribution. Hence the subject of this book is the Markov chain sampling from the fiber Ft . In the next chapter, again employing the independence model of I × J contingency tables, we discuss how to perform Markov chain sampling from the fiber Ft .
1.4 Conditional Independence Model of Three-Way Contingency Tables In this section we discuss the conditional independence model for three-way contingency tables. It is a relatively simple model in the sense that for each level of the conditioning variable, the problem reduces to the case of an independence model of two-way contingency tables for the other variables. However, it is a convenient model for introducing a notation for general m-way contingency tables in the next section.
1.4 Conditional Independence Model of Three-Way Contingency Tables
15
Consider an I1 × I2 × I3 three-way contingency table x . We denote each cell of the table by a multi-index i = (i1 , i2 , i3 ). For a positive integer J write [J] = {1, . . . , J}. The set of the cells is the following direct product I = {ii = (i1 , i2 , i3 ) | i1 ∈ [I1 ], i2 ∈ [I2 ], i3 ∈ [I3 ]} = [I1 ] × [I2] × [I3]. With this notation the three-way contingency table, or the frequency vector, is denoted as x = {x(ii) | i ∈ I }. Note that this notation is somewhat heavy and in fact for three-way tables we prefer to use subscripts i, j, k. The merit of this notation is that it can be used for general m-way tables. For a subset D ⊂ {1, 2, 3} of the variables, let iD denote the set of indices in D. For example, i {1,2} = (i1 , i2 ). Note that i D corresponds to the D-marginal cell of the contingency table. The set of D-marginal cells is denoted by ID =
∏ [Ik ].
(1.24)
k∈D
For example I{1,2} = {(i1 , i2 ) | i1 ∈ [I1 ], i2 ∈ [I2 ]}. The D-marginal frequencies of x are written as xD (iiD ) =
∑
i DC ∈IDC
x(iiD , iDC ),
(1.25)
where DC denotes the complement of D. Note that in x(iiD , i DC ), for notational simplicity, the indices in ID are collected to the left. Also we are writing x(iiD , i DC ) instead of x((iiD , i DC )). In the two-way case xi+ = x{1} (i) = ∑ xi j . j
For a probability distribution {p(ii), i ∈ I }, we denote the D-marginal probability as pD (iiD ). Note that in xD (iiD ) and pD (iiD ), the subset D is indicated twice. If there is no notational confusion we alternatively write x(iiD ), xD (ii), p(iiD ) or
pD (ii)
(1.26)
for simplicity. We call a D-marginal probability distribution saturated if there is no restriction on the probability vector {pD (iiD ), i D ∈ ID }.
16
1 Exact Tests for Contingency Tables and Discrete Exponential Families
Let Y1 ,Y2 ,Y3 be random variables corresponding to the three axes of the contingency table. We consider the model that Y1 and Y3 are conditionally independent given the level i2 of Y2 . The relevant conditional probabilities are written as p(i1 , i3 | i2 ) =
p(ii) , p{2} (i2 )
p(i1 | i2 ) =
p{1,2} (i1 , i2 ) , p{2} (i2 )
p(i3 | i2 ) =
p{2,3}(i2 , i3 ) . p{2} (i2 )
In the following we omit subscripts to p and write, for example, p(i1 , i2 ) instead of p{1,2} (i1 , i2 ). Similarly we write x(i1 , i2 ) instead of x{1,2} (i1 , i2 ). The null hypothesis of conditional independence is written as H:
p(ii) p(i1 , i2 ) p(i2 , i3 ) = × , p(i2 ) p(i2 ) p(i2 )
∀ii ∈ I ,
(1.27)
1 p(i1 , i2 )p(i2 , i3 ), p(i2 )
∀ii ∈ I .
(1.28)
or equivalently as H : p(ii) =
Here we are assuming p(i2 ) > 0. In the case p(i2 ) = 0 for a particular level i2 , we have p(ii) = p(i1 , i2 ) = p(i2 , i3 ) = 0 for indices containing this level i2 of Y2 . Hence in this case we understand (1.28) as 0 = 0 × 0/0. Let
α (i1 , i2 ) =
p(i1 , i2 ) , p(i2 )
β (i2 , i3 ) = p(i2 , i3 ).
Then the conditional independence model is written as H : p(ii) = α (i1 , i2 )β (i2 , i3 ).
(1.29)
Note that there is some indeterminacy in specifying α and β . For example we can include the factor 1/p(i2 ) into β (i2 , i3 ) instead of into α (i1 , i2 ). We can show that (1.27), (1.28), and (1.29) are in fact equivalent. Suppose that p(ii) = p(i1 , i2 , i3 ) can be written as p(ii) = α (i1 , i2 )β (i2 , i3 ). Then
p (i2 ) =
∑ p (i1 , i2 , i3 ) = ∑ α (i1 , i2 ) β (i2 , i3 ) = ∑ α (i1 , i2 )
i1 ,i3
i1 ,i3
i1
p(i1 , i2 ) = ∑ p(i1 , i2 , i3 ) = α (i1 , i2 ) ∑ β (i2 , i3 ), i3
p(i2 , i3 ) = ∑ p(i1 , i2 , i3 ) = i1
i3
∑ α (i1 , i2 ) i1
β (i2 , i3 ).
∑ β (i2 , i3 ) i3
,
1.4 Conditional Independence Model of Three-Way Contingency Tables
17
Therefore
α (i1 , i2 )β (i2 , i3 )(∑i1 α (i1 , i2 ))(∑i3 β (i2 , i3 )) p(i1 , i2 )p(i2 , i3 ) = p(i2 ) (∑i1 α (i1 , i2 ))(∑i3 β (i2 , i3 )) = α (i1 , i2 )β (i2 , i3 ) = p(ii) and hence (1.28) holds. This shows that the null hypothesis of conditional independence can be written in any one of (1.27), (1.28), and (1.29). Now suppose that we observe a contingency table x of sample size n from the conditional independence model. The joint probability function is written as p(xx) = =
n! ∏ (α (i1 , i2 )β (i2 , i3 ))x(ii) ∏i ∈I x(ii)! i∈I n! ∏ α (i1 , i2 )x(i1 ,i2 ) ∏ β (i2 , i3 )x(i2 ,i3 ) . ∏i ∈I x(ii)! i{1,2} ∈I{1,2} i {2,3} ∈I{2,3}
(1.30)
Hence a sufficient statistic T is the set of {1, 2}-marginals and {2, 3}-marginals of x : T = ({x(ii{1,2} ) | i {1,2} ∈ I{1,2} }, {x(ii{2,3} ) | i {2,3} ∈ I{2,3} }). In this case the marginal distribution of T is not immediately clear and hence the conditional probability of x given T = t is also not immediately clear. However, without worrying about the marginal distribution of T at this point, we can proceed as follows. Let A be the configuration relating the frequency vector to the sufficient statistic: t = Axx. Define Ft = {xx ≥ 0 | t = Axx} as in (1.22). The terms containing the parameters α , β on the right-hand side of (1.30) are fixed by the sufficient statistic, therefore these terms do not appear in the conditional distribution of x given t . It follows that the conditional distribution of x given t is written as p(xx | t ) = c ×
1 ∏i∈I x(ii)!
,
c=
∑
x∈Ft
1 ∏i ∈I x(ii )!
−1 .
(1.31)
As in the previous examples, an exact test of the null hypothesis H of conditional independence can be performed if either we can enumerate the elements of Ft or if we can sample from this distribution. Note that we often call (1.31) the hypergeometric distribution over Ft . In general, the normalizing constant c cannot be written explicitly. The Markov chain sampling discussed in the next chapter can be performed without knowing the explicit form of the normalizing constant. This is one of the major advantages of Markov chain Monte Carlo methods.
18
1 Exact Tests for Contingency Tables and Discrete Exponential Families
It turns out that for the conditional independence model the marginal distribution of the sufficient statistic T and the normalizing constant c can be written down explicitly. This is a special case of the result of Sundberg [140] for decomposable models, which is studied in Chap. 8. In the following section, we explain the marginal distribution of T . The following section can be skipped, because the normalizing constant c is not needed for performing Markov chain Monte Carlo methods.
1.4.1 Normalizing Constant of Hypergeometric Distribution for the Conditional Independence Model For illustration let us explicitly write out the configuration for relating the frequency vector to the sufficient statistic for the case of 2 × 2 × 2 tables. We order the elements of T according to the level of Y2 . Then t = Axx is written as ⎛ ⎞ ⎛ x{1,2} (1, 1) 110 ⎜x{1,2} (2, 1)⎟ ⎜0 0 1 ⎜ ⎟ ⎜ ⎜x ⎟ ⎜ ⎜ {2,3} (1, 1)⎟ ⎜1 0 1 ⎜ ⎟ ⎜ ⎜x{2,3} (1, 2)⎟ ⎜0 1 0 ⎜ ⎟=⎜ ⎜x{1,2} (1, 2)⎟ ⎜ ⎜ ⎟ ⎜ ⎜x{1,2} (2, 2)⎟ ⎜ ⎜ ⎟ ⎜ ⎝x{2,3} (2, 1)⎠ ⎝
0 1 0 1
0
x{2,3} (2, 2)
⎞⎛
⎞ x(1, 1, 1) ⎟ ⎜x(1, 1, 2)⎟ ⎟⎜ ⎟ ⎟ ⎜x(2, 1, 1)⎟ ⎟⎜ ⎟ ⎟⎜ ⎟ ⎟ ⎜x(2, 1, 2)⎟ ⎟⎜ ⎟, 1 1 0 0 ⎟ ⎜x(1, 2, 1)⎟ ⎟⎜ ⎟ ⎜ ⎟ 0 0 1 1⎟ ⎟ ⎜x(1, 2, 2)⎟ 1 0 1 0 ⎠ ⎝x(2, 2, 1)⎠ 0101 x(2, 2, 2)
0
(1.32)
where the big 0 is the 4 × 4 zero matrix. Note that the 8 × 8 matrix on the right-hand side is a block diagonal with identical blocks. Furthermore, the diagonal block is the same as on the right-hand side of (1.10). Partition x on the right-hand side of (1.32) into two 4-dimensional subvectors x1 , x2 . We call each xi2 , i2 = 1, 2, the slice of the contingency table x by fixing the level i2 of the second variable. Similarly we partition t on the left-hand side of (1.32) into two 4-dimensional subvectors t 1 ,tt 2 . Then clearly x ∈ Ft
⇔
x1 ∈ Ft 1
and x2 ∈ Ft 2 ,
(1.33)
where Ft 1 and Ft 2 are fibers in (1.22) for the independence model of 2 × 2 contingency tables. We have thus far looked at the 2 × 2 × 2 case. However, it is clear that a similar result holds for the general I1 × I2 × I3 case. Namely, when we sort the cells according to the levels of Y2 , then the configuration is in a block diagonal form with identical blocks, which correspond to the configuration of the independence model for I1 × I3 contingency tables.
1.5 Notation of Hierarchical Models for m-Way Contingency Tables
19
Also from (1.16) it follows that for I × J contingency tables we have
∑
x∈Ft
n! 1 = I . ∏i, j xi j ! ∏i=1 xi+ ! ∏Jj=1 x+ j !
Combining this with (1.33) and by summing for each slice separately, we have the following expression of c−1 in (1.31) for the conditional independence model. I2 1 x(i2 )! = ∏ I I3 1 c i2 =1 ∏i =1 x(i1 , i2 )! ∏i =1 x(i2 , i3 )! 1
=
3
∏i2 ∈I{2} x(i2 )! ∏(i1 ,i2 )∈I{1,2} x(i1 , i2 )! ∏(i2 ,i3 )∈I{2,3} x(i2 , i3 )!
.
If we apply this sum to (1.30), we see that the joint probability distribution of T is given as p(T ) =
n! ∏i2 ∈I{2} x(i2 )! ∏(i1 ,i2 )∈I{1,2} x(i1 , i2 )! ∏(i2 ,i3 )∈I{2,3} x(i2 , i3 )! ×
∏
i {1,2} ∈I{1,2}
α (i1 , i2 )x(i1 ,i2 )
∏
i {2,3} ∈I{2,3}
β (i2 , i3 )x(i2 ,i3 ) .
Then the conditional probability in (1.31) is explicitly written as p(xx | t ) =
∏(i1 ,i2 )∈I{1,2} x(i1 , i2 )! ∏(i2 ,i3 )∈I{2,3} x(i2 , i3 )! ∏i2 ∈I{2} x(i2 )! ∏i∈I x(ii)!
.
(1.34)
1.5 Notation of Hierarchical Models for m-Way Contingency Tables In this section we introduce notation for general m-way contingency tables and hierarchical models for these tables. The notation introduced here is used extensively later in this book, such as Chaps. 8 and 9. Readers may skip this section and check the notation when it is needed later in the book. In the previous section we considered three-way contingency tables. We generalize the notation to m-way tables. The set of the cells for an m-way table is the direct product I = {ii = (i1 , . . . , im ) | i1 ∈ [I1 ], . . . , im ∈ [Im ]} = [I1 ] × · · · × [Im ].
20
1 Exact Tests for Contingency Tables and Discrete Exponential Families
An m-way contingency table, or the frequency vector, is denoted by x = {x(ii) | i ∈ I }. We denote the set of m variables as Δ = [m] = {1, . . . , m}. The notation for marginal cells, marginal frequencies, and marginal probabilities was already given in (1.24), (1.25), and (1.26). Consider the logarithm of the probability function of the conditional independence model in (1.29): log p(ii) = log α (i1 , i2 ) + log β (i2 , i3 ).
(1.35)
Here log p(ii) is written as a sum of two functions, one of which depends only on (i1 , i2 ) and the other depends only on (i2 , i3 ). Generalizing this formulation we now define a hierarchical model for m-way contingency tables. Let D1 , . . . , Dr be subsets of Δ , such that there is no inclusion relation between Di and D j , 1 ≤ i = j ≤ m. Denote D = {D1 , . . . , Dr }. A hierarchical model with the generating class D is defined as follows. log p(ii) =
∑
μD (iiD ),
(1.36)
D∈D
where μD is a function depending only on the marginal cell i D . For a general hierarchical model let K = K (D) = {D | D ⊂ Di for some Di ∈ D} denote the set of subsets of D1 , . . . , Dr . Note that K has the following property, A∈K , B⊂A ⇒ B∈K . A family of subsets of Δ satisfying this property is called a simplicial complex ([96]). Note that D1 , . . . , Dr are maximal elements of K with respect to set inclusion. Maximal elements of a simplicial complex K are called facets of K . From a statistical viewpoint, the facets correspond to maximal interaction terms in the hierarchical model. In a hierarchical model, when an interaction term is present in the model, then all smaller interaction terms and the main effects included in the interaction term are also present in the model. This is a natural assumption, because, for example, a two-variable interaction is usually interpreted only in the presence of main effects of the variables. A sufficient statistic for a hierarchical model is given by the set of marginal frequencies for D1 , . . . , Dr : T = {xD (iiD ) | i D ∈ ID , D ∈ D}. Finally we discuss indeterminacy of μD s in (1.36). As an example consider again the conditional independence model in (1.35). There is some indeterminacy on the right-hand side of (1.35). One way of resolving this indeterminacy is to use the
1.5 Notation of Hierarchical Models for m-Way Contingency Tables
21
standard ANOVA (analysis of variance) decomposition (e.g., Scheff´e [132]). In (1.35) we can write log p(ii) = μ0 + μ1(i1 ) + μ2(i2 ) + μ3 (i3 ) + μ12(i1 , i2 ) + μ23(i2 , i3 ),
(1.37)
where we require Ij
0=
0=
∑
i j =1
μ j (i j ), j = 1, 2, 3,
I2
I3
i2 =1
i3 =1
0=
I1
∑
i1 =1
μ12 (i1 , i2 ) =
I2
∑ μ12 (i1 , i2 ), ∀i1 , i2 ,
i2 =1
∑ μ23 (i2 , i3 ) = ∑ μ23 (i2 , i3 ), ∀i2 , i3 .
Under these requirements the right-hand side of (1.37) is unique. Similarly, for a general hierarchical model we can uniquely express log p(ii) as log p(ii) =
∑
μD (iiD ),
(1.38)
D∈K
where for every D = { j1 , . . . , jl } ∈ K we require I jh
∑
i jh =1
μD (i j1 , . . . , i jl ) = 0,
h = 1, . . . , l.
Another popular method for avoiding the indeterminacy is to treat a particular level, for example, the last level I j , j = 1, . . . , m, as the “base level” and require for every D ∈ K , D = 0, / that
μD (i j1 , . . . , i jl ) = 0,
if i jh = I jh for some h = 1, . . . , l.
(1.39)
In the case of a complete independence model of two-way tables, this corresponds to the lattice basis in (2.5).
Chapter 2
Markov Chain Monte Carlo Methods over Discrete Sample Space
2.1 Constructing a Connected Markov Chain over a Conditional Sample Space: Markov Basis In the previous chapter we discussed exact tests for some simple models of contingency tables. As we discussed at the end of Sect. 1.3, the Markov chain Monte Carlo method is general and useful when the cardinality of conditional sample space (fiber) is large. We first consider connectivity of a Markov chain, without fully specifying the transition probabilities. Consider the independence model of general two-way contingency tables in Sect. 1.3. The fiber is the set of I × J contingency tables with fixed row sums and column sums: Ft = {xx ≥ 0 | xi+ , i ∈ [I], x+ j , j ∈ [J] are fixed according to t }.
(2.1)
Let A be the configuration in (1.18). The kernel of A is denoted by ker A. The set of integer vectors in ker A is called the integer kernel of A and is denoted by kerZ A = {zz | Azz = 0, z ∈ Zη },
η = IJ.
An element of kerZ A is called a move for the configuration A. If x and y belong to the same fiber Ft , then y − x is a move, because A(yy − x ) = Ayy − Axx = t − t = 0.
(2.2)
Now consider the following integer matrix z = z (i1 , i2 ; j1 , j2 ) = {zi j }, ⎧ ⎨ +1, (i, j) = (i1 , j1 ), (i2 , j2 ), zi j = −1, (i, j) = (i1 , j2 ), (i2 , j1 ), ⎩ 0, otherwise. S. Aoki et al., Markov Bases in Algebraic Statistics, Springer Series in Statistics 199, DOI 10.1007/978-1-4614-3719-2 2, © Springer Science+Business Media New York 2012
(2.3)
23
24
2 Markov Chain Monte Carlo Methods over Discrete Sample Space
The nonzero elements of z (i1 , i2 ; j1 , j2 ) are depicted as j1 j2 i1 +1 −1 . i2 −1 +1
(2.4)
Adding z (i1 , i2 ; j1 , j2 ) to a contingency table x does not alter the row sums and the column sums. Hence z (i1 , i2 ; j1 , j2 ) is a move for A in (1.18); that is, z (i1 , i2 ; j1 , j2 ) ∈ kerZ A. We call a move of the form (2.4) a basic move for the independence model of two-way contingency tables. Because of the elements −1 in z (i1 , i2 ; j1 , j2 ), x + z contains a negative element if xi2 j1 = 0 or xi1 j2 = 0. If both of these elements are positive, then x + z is in Ft if x ∈ Ft . We have “moved” from x to x + z in Ft . This is why we call z (i1 , i2 ; j1 , j2 ) a move. The following is an example of adding a move for the case of I = J = 3, i1 = j1 = 1, i2 = j2 = 2. 2 2 1 5
1 0 2 3
14 1 −1 0 30 2 4 −1 1 0 11 + = 03 0 0 0 12 3 53
1 2 0 3
4 4 . 3
Suppose that we always use the last row I and the last column J in the move and let i2 = I and j2 = J. Then {zz(i1 , I; j1 , J) | 1 ≤ i1 ≤ I − 1, 1 ≤ j1 ≤ J − 1} forms a basis of kerZ A. More precisely the set forms a lattice basis of kerZ A in the sense that every z ∈ kerZ A is uniquely written as an integer combination of z (i1 , I; j1 , J)s. In fact the elements of the last row and the last column of z = {zi j } ∈ kerZ A are uniquely determined from the other elements. Hence z ∈ kerZ A can be uniquely written as z=
I−1 J−1
∑ ∑ zi1 j1 × z(i1 , I; j1 , J),
(2.5)
i1 =1 j1 =1
because both sides have the same elements in the first I − 1 rows and the first J − 1 columns. This is related to use of the last level as the base level discussed at the end of Chap. 1. Note that the lattice basis is very simple for the independence model of I × J tables. However, for the fiber in (2.1) we are requiring nonnegativeness of the frequency vectors. As an example consider the following two elements of the fiber for I = J = 3 with 1 = x1+ = x2+ = x+1 = x+2 , 0 = x3+ = x+3 . 1 0 0 1
0 1 0 1
0 0 0 0
1 1 , 0 2
0 1 0 1
1 0 0 1
0 0 0 0
1 1 . 0 2
2.1 Constructing a Connected Markov Chain over a Conditional Sample. . .
25
We see that we cannot add or subtract any of z (i1 , 3; j1 , 3) to/from these tables without making some cell frequency negative. However, obviously these two tables are connected by the following move: 1 −1 0 −1 1 0 . 0 0 0 This example suggests that we can move around a fiber if we can use all moves of the form (2.3). Let B ⊂ kerZ A be a finite set of moves for a configuration A. B is called a Markov basis if for all fibers Ft and for all elements x , y ∈ Ft , x = y , there exist K > 0, z 1 , . . . , z K ∈ B and ε1 , . . . , εK ∈ {−1, 1}, such that K
y = x + ∑ εk z k , k=1
L
x + ∑ εk z k ∈ Ft ,
L = 1, . . . , K − 1.
(2.6)
k=1
The first condition says that by adding or subtracting elements of B, we can move from x to y . The second condition says that on the way from x to y we never encounter a negative frequency. Therefore if a Markov basis B is given, then we can move all over any fiber by adding or subtracting moves from B. Thus connectivity of every fiber is guaranteed by a Markov basis. We define Markov basis again in Chap. 4 for a general configuration A. In this introductory explanation, we give a proof that a Markov basis for the I × J independence model of two-way contingency tables is given by the set of moves z (i1 , i2 ; j1 , j2 ). We state this as a theorem. Theorem 2.1. Let B = {zz(i1 , i2 ; j1 , j2 ) | 1 ≤ i1 < i2 ≤ I, 1 ≤ j1 < j2 ≤ J}. B forms a Markov basis for the I × J independence model of two-way contingency tables. The following proof is a typical “distance reducing argument,” that is frequently used in later chapters of this book. Proof. We argue by contradiction. Suppose that B is not a Markov basis. Then there exists a fiber Ft and two elements x , y ∈ Ft of the fiber, such that we cannot move from x to y by the moves of B as in (2.6). Let Nx = {yy ∈ Ft | we cannot move from x to y by moves of B}. Then Nx is not empty by assumption. For z = {zi j } ∈ kerZ A, let |zz| = ∑Ii=1 ∑Jj=1 |zi j | denote its 1-norm. In Sect. 4.3 we define deg z as |zz|/2.
26
2 Markov Chain Monte Carlo Methods over Discrete Sample Space
Define y ∗ = arg min |xx − y|. y ∈Nx
(2.7)
y ∗ is one of the closest elements of Ft that cannot be reached from x by B: |xx − y ∗ | = min |xx − y|. y ∈Nx
Now let w = x − y ∗ and consider the signs of elements of w . Because w contains a positive element, let wi1 j1 > 0. Then because w is a move, there exist j2 = j1 with wi1 j2 < 0 and i2 = i1 with wi2 j1 < 0. Hence for y ∗ = {y∗i j } we have y∗i1 j2 > 0, y∗i2 j1 > 0. Then y ∗ + z (i1 , i2 ; j1 , j2 ) ∈ Ft . y ∗ cannot be reached from x by B, therefore y ∗ + z (i1 , i2 ; j1 , j2 ) cannot be reached from x by B either and y ∗ + z (i1 , i2 ; j1 , j2 ) ∈ Nx . Now we check the value of |xx − (yy∗ + z (i1 , i2 ; j1 , j2 ))|. • If wi2 j2 > 0, then |xx − (yy∗ + z (i1 , i2 ; j1 , j2 ))| = |xx − y ∗ | − 4 , • If wi2 j2 ≤ 0, then |xx − (yy∗ + z (i1 , i2 ; j1 , j2 ))| = |xx − y ∗ | − 2 . Therefore for both cases, |xx − (yy∗ + z (i1 , i2 ; j1 , j2 ))| < |xx − y ∗ |. However, this contradicts the minimality in (2.7) of y ∗ . By this theorem, we can construct a connected Markov chain over any fiber. We choose i1 , i2 ∈ [I] and j1 , j2 ∈ [J] randomly. We add or subtract z (i1 , i2 ; j1 , j2 ) to/from the current state x and move to y = x + z (i1 , i2 ; j1 , j2 ) as long as there is no negative frequency in y . In the case where y contains a negative element, we choose another set of indices i1 , i2 ∈ [I] and j1 , j2 ∈ [J] and continue. Then connectivity of every fiber is guaranteed by Theorem 2.1. Note that in the above explanation we are not precisely specifying the probability distribution of choosing an element z (i1 , i2 ; j1 , j2 ). Also, when we say “add or subtract,” we are not exactly saying which to choose. In fact, we should choose the sign of a move z (i1 , i2 ; j1 , j2 ) (i.e., whether we add it or subtract it) with probability 1/2. This is related to the Markov chain symmetry for the Metropolis–Hastings algorithm in the next section. Other than the choice of the sign of a move, the distribution for choosing a move can be arbitrary. In this section we considered the independence model of two-way contingency tables. We now briefly mention the conditional independence model of three-way contingency tables. As we saw in the previous section, the conditional independence model of three-way contingency tables can be treated as the two-way independence model given each level of the conditioning variable. Therefore a Markov basis for the conditional independence model of three-way contingency tables is given as a union of Markov bases for two-way cases in each slice of the contingency table given the level of the conditioning variable. The two-way independence model and the conditional independence model of three-way contingency tables are actually
2.2 Adjusting Transition Probabilities by Metropolis–Hastings Algorithm
27
simple examples. Markov bases for more complicated models of contingency tables are in fact difficult and each model needs separate consideration. One notable exception is the decomposable model studied in Chap. 8. On the other hand, there exists a general algorithm to compute a Markov basis in the form of the Gr¨obner basis for any configuration. So is the problem of obtaining a Markov basis already solved by a general algorithm? The answer is yes and no, depending on the viewpoint. The existence of a general algorithm means that the answer is yes from a certain theoretical viewpoint. On the other hand, for practical purposes, the computation of the Gr¨obner basis for a complicated model is often infeasible in a practical amount of time and in this sense the answer is no. Therefore, both theoretical investigations of Markov bases for specific models and the further general improvements in the algorithms for Gr¨obner basis computation are very much needed at present.
2.2 Adjusting Transition Probabilities by Metropolis–Hastings Algorithm In this section we explain how to construct a Markov chain that has a specified distribution as the stationary distribution. A good reference on important facts on Markov chains is H¨aggstr¨om [69]. Consider a Markov chain over a finite sample space F . Suppose that the elements of F are given as F = {xx1 , . . . , xs }.
(2.8)
Let {Zt , t = 0, 1, 2, . . .}, Zt ∈ F , be a Markov chain over F with the transition probability Q = (qi j ): qi j = P(Zt+1 = x j | Zt = xi ),
1 ≤ i, j ≤ s.
A Markov chain is called symmetric if Q is a symmetric matrix (qi j = q ji ). Let
π = (π1 , . . . , πs ) denote the initial probability distribution of Z0 (by standard notation, we consider π as a row vector). π is called a stationary distribution if
π = π Q. π is the eigenvector from the left of Q with the eigenvalue 1. It is known that the stationary distribution exists uniquely under the assumption that the Markov chain is irreducible and aperiodic. We only consider Markov chains satisfying these conditions. Under these conditions, starting from an arbitrary state Z0 = x i , the distribution of Zt for large t is close to the stationary distribution π . Therefore if we can construct a Markov chain with the “target”
28
2 Markov Chain Monte Carlo Methods over Discrete Sample Space
stationary distribution π , then by running a Markov chain and discarding a large number t of initial steps (called burn-in steps), we can consider Zt+1 , Zt+2 , . . . as observations from the stationary distribution π . For our problem, the target distribution π is already given as the hypergeometric distribution over the fiber in (1.31). We want to construct a Markov chain over Ft just for the purpose of sampling from the hypergeometric distribution. For this purpose the Metropolis–Hastings algorithm is very useful. By the algorithm, once we can construct an arbitrary irreducible (i.e., connected) chain over Ft , we can easily modify the stationary distribution to the given target distribution π . Theorem 2.2 (Metropolis–Hastings algorithm). Let π be a probability distribution on F . Let R = (ri j ) be the transition probability matrix of an irreducible, aperiodic, and symmetric Markov chain over F . Define Q = (qi j ) by πj qij = rij min 1, , i = j, πi qii = 1 − ∑ qi j .
(2.9)
j =i
Then Q satisfies π = π Q. This result is a special case of Hastings [82] and the symmetry assumption on R can be removed relatively easily. In this book we only consider symmetric R and the simple statement of the above theorem is sufficient for our purposes. Proof (Theorem 2.2). It suffices to show that the above Q is “reversible” in the following sense.
πi qi j = π j q ji .
(2.10)
In fact, under the reversibility s
πi = πi ∑ qi j = j=1
s
∑ π j q ji
j=1
and we have π = π Q. Now (2.10) clearly holds for i = j. Also for i =j πj πi qij = πi rij min 1, = rij min(πi , π j ) ; πi hence (2.10) holds if ri j = r ji .
Equation (2.10) is often called the detailed balance or detailed balance equation. An important advantage of the Markov chain Monte Carlo method is that it does not need the explicit evaluation of the normalizing constant of the stationary distribution π . We only need to know π up to a multiplicative constant. In fact in (2.9) the stationary distribution only appears in the form of ratios of its elements πi /π j and the normalizing constant is canceled. Another important point in (2.9) is how the transition probability ri j is modified. It is modified by min(1, π j /πi ), which does not depend on how ri j is specified.
2.2 Adjusting Transition Probabilities by Metropolis–Hastings Algorithm
29
In fact (2.9) can be understood as follows. ri j is the proposal transition probability. Suppose that we are at state i and we propose to move to j with the conditional probability ri j by some random mechanism. Then after the proposal, we actually move to j with probability min(1, π j /πi ) (or stay at i with probability 1 − min(1, π j /πi )). We can do this even without knowing the value of ri j , as long as it is symmetric. This fact is relevant in the application of the Markov basis, because when a Markov basis element is chosen “randomly,” the probability distribution of choosing an element can be arbitrary, as long as there is a positive probability of choosing every element. Irrespective of the distribution, the Metropolis–Hastings algorithm yields a Markov chain whose stationary distribution is π . By Theorem 2.2 we only need to construct one Markov chain, which is irreducible, aperiodic, and symmetric. By the Metropolis–Hastings algorithm, we can then modify the transition probability to achieve the desired stationary distribution π . In the previous section we obtained a Markov basis for two-way tables. Once a Markov basis is obtained for some model, it is easy to construct an irreducible and symmetric Markov chain over FAxxo , where x o is the observed frequency vector and FAxxo is the fiber containing x o . For example, at each step of the Markov chain, randomly choose an element z ∈ B of the Markov basis and the sign ε ∈ {−1, +1}. If x + ε z ∈ Ft then we move to x + ε z . If x + ε z ∈ Ft we stay at x . Then the resulting Markov chain is irreducible and symmetric. It is important to note that this holds irrespective of the distribution of choosing an element from B, as long as each element of B is chosen with positive probability. On the other hand, the sign of ε should be chosen with probability 1/2. We can then apply the Metropolis–Hastings algorithm of Theorem 2.2 to this Markov chain. The resulting algorithm is given as follows. Algorithm 2.1 Input: Observed frequency vector x o , Markov basis B, number of steps N, configuration A, the null distribution f (·), test statistic T (·), t = Axxo . Output: Estimate of the p-value. Variables: obs, count, sig, x , x next . Step 1: obs = T (xxo ), x = x o , count = 0, sig = 0. Step 2: Choose z ∈ B randomly. Choose ε ∈ {−1, +1} with probability 12 . Step 3: If x + ε z ∈ Ft then x next = x and go to Step 5. If x + ε z ∈ Ft then let u be a uniform random number between 0 and 1. x +ε z ) f (xx+ε z ) Step 4: If u ≤ f (xf (x x) then let x next = x + ε z and go to Step 5. If u > f (xx ) then let x next = x and go to Step 5. Step 5: If T (xxnext ) ≥ obs then let sig = sig + 1 . Step 6: x = x next , count = count + 1 . Step 7: If count < N then go to Step 2. Step 8: The estimate of p-value is sig/N . We should mention one important point concerning the counting of steps. There are two cases where we stay at the same state x next = x . One case is that we reject a move z because x + ε z ∈ Ft in Step 3. Another case is that the proposed state is
30
2 Markov Chain Monte Carlo Methods over Discrete Sample Space
Fig. 2.1 The fiber F2
0
+z
1
-z
Fig. 2.2 Transition probabilities ignoring rejections in Step 3
0
1 0.5
+z
2
x1
-z 1
0.5
2
x1
1
rejected because of u > f (xx + ε z )/ f (xx ) in Step 4. In both cases, we evaluate the value of the test statistic T (xxnext ) = T (xx) and the counter count is increased. For unbiased estimation of the p-value, we need to include both cases in evaluation of T and the counting of the steps. In Step 3, if x is close to the boundary of Ft , then it may be the case that x + ε z ∈ Ft with high probability. In this case we might be tempted to choose z depending on x such that the probability of x + ε z ∈ Ft is higher. This is an interesting topic for investigation, although it is not trivial to guarantee the symmetry ri j = r ji if we choose a move depending on the state. The above point can be illustrated by the following very simple example. Consider a configuration A = (1, 1), which is a 1 × 2 matrix. Let t = Axx, x = (x1 , x2 )
and consider the fiber with t = 2: F2 = {(x1 , x2 ) | x1 + x2 = 2, x1 , x2 ∈ N},
N = {0, 1, 2, . . .}.
Then z = (1, −1) is a move, which obviously connects F2 . The fiber is depicted as in Fig. 2.1, where the states are labeled by the values of x1 . Note that z cannot be subtracted from (0, 2) and z cannot be added to (2, 0), because these operations produce −1. Therefore if we are at (0, 2) we can only add z . Similarly if we are at (2, 0) we can only subtract z . Now suppose that we want to sample from the uniform distribution over F2 . Then in the Metropolis–Hastings algorithm, min(1, π j /πi ) ≡ 1. Therefore we stay at the same state only because of Step 3 of Algorithm 1. If we ignore the rejections in Step 3 for this example, the transition probabilities of the chain are depicted in Fig. 2.2. The stationary distribution of this chain is given by (π (0, 2), π (1, 1), π (2, 0)) =
1 1 1 , , , 4 2 4
which is not uniform. On the other hand if we count the rejections in Step 3, then the Markov chain has self-loops and the transition probabilities of the chain are depicted in Fig. 2.3. For this chain the stationary distribution is the uniform distribution, which was our target.
2.2 Adjusting Transition Probabilities by Metropolis–Hastings Algorithm Fig. 2.3 Transition probabilities taking rejections in Step 3 into account
0.5
0
0.5
1
0.5
31 0.5
2
0.5 x1
0.5
Algorithm 2.1 is a very simple algorithm and various improvements are possible. For example, grouping several steps of Algorithm 2.1 in one step makes the convergence to the stationary distribution faster. This can be achieved as follows. Algorithm 2.2 Modify Steps 2, 3, 4 in Algorithm 2.1 as follows. Step 2: Step 3: Step 4:
Choose z ∈ B randomly. Let I = {n | x + nzz ∈ Ft }. Choose x next from {xx + nzz | n ∈ I} according to the probability pn =
f (xx + nzz) . ∑ f (xx + nzz)
n∈I
Note that both in Algorithms 2.1 and 2.2, the target distribution f (·) appears in the form of the ratio. Hence we do not need to compute the normalizing constant for f (·). Often the computation of the normalizing constant is difficult, therefore this is an important advantage of the Markov chain Monte Carlo method.
Chapter 3
Toric Ideals and Their Gr¨obner Bases
Readers can skip this chapter and come back to individual results when they are referenced in later chapters. There are many good textbooks on computational algebra and Gr¨obner basis. This chapter is based on a great deal Chapter 1 of [93] by Takayuki Hibi, although for individual results we cite Cox et al. [42] as a reference. Sturmfels [139] gives more specific results relevant for algebraic statistics and toric ideals. In presenting results on polynomial rings, the difference of standard notation for statistics and algebra is annoying. For example, in statistics n usually denotes the sample size, whereas in the notation for polynomial rings n usually stands for the number of indeterminates, which corresponds to the total number of cells |I |. In this book x(ii) stands for the frequency of the cell i , whereas in the polynomial ring, x1 , . . . , xn usually denote indeterminates. In this chapter we use a mixture of these different notations to make the correspondences easier to understand.
3.1 Polynomial Ring Let Q, R, C denote the fields of rational numbers, real numbers and complex numbers, respectively. Let k stand for any of these fields. We denote the indeterminates by u1 , . . . , uη , where η = |I | is the total number of the cells. A monomial in u1 , . . . , uη is a product of powers of us (with the coefficient 1 ∈ k). For k = Q and η = 3, u21 u2 u33
(3.1)
is an example of a monomial in u1 , u2 , u3 . We write this as ux where x = (2, 1, 3) and u = (u1 , u2 , u3 ). Note that the elements of x are used as powers, rather than as indeterminates. This notation is used because frequencies in a contingency table correspond to the powers of a monomial. Each indeterminate ui stands for a cell S. Aoki et al., Markov Bases in Algebraic Statistics, Springer Series in Statistics 199, DOI 10.1007/978-1-4614-3719-2 3, © Springer Science+Business Media New York 2012
33
34
3 Toric Ideals and Their Gr¨obner Bases
i in a contingency table. Hence the monomial u21 u2 u33 in (3.1) corresponds to the following 1 × 3 contingency table, 2
1
3
.
A monomial is called square-free if the power for each ui is at most one. A polynomial is a finite sum of monomials multiplied by coefficients in k. Again for k = Q and η = 3, an example of a polynomial is 3 2 3 1 u u2 u + u1 u32 u23 . 2 1 3 3 A polynomial with more than two terms does not correspond to a contingency table. In Chap. 1 we denoted the vector of frequencies as x = (x(1), . . . , x(η )) ∈ Nη . Accordingly we also often denote ui as u(ii). Below we use these two indexing notations interchangeably. Then a monomial is written as ux =
∏ u(ii)x(ii) .
i ∈I
A polynomial f is written as f=
∑
x ∈Nη
cx u x ,
where the sum is finite; that is cx ∈ k is zero except for a finite number of x . The set of polynomials in u with coefficients from k is written as k[uu] = k[u1 , . . . , uη ]. k[uu] is called the polynomial ring in u1 , . . . , uη over k. It is called a ring because the operations of addition f + g and multiplication f g of polynomials are defined for k[uu ]. Let M denote the set of monomials in k[uu]. Let v = u x and w = u y be two monomials in M. Then w divides v if y ≤ x : x(ii) ≤ y(ii)
for all i ∈ I .
We write w|v if w divides v. Let M ⊂ M be a subset of monomials. v ∈ M is called a minimal element of M if w ∈ M, w|v implies v = w. We present Dickson’s lemma (Sect. 2.4 of [42]) in the following form. Lemma 3.1 (Dickson’s Lemma). Let M ⊂ M be a nonempty set of monomials. The set of minimal elements of M is finite.
3.2 Term Order and Gr¨obner Basis
35
Another important notion on the polynomial ring is the notion of an ideal. A subset I ⊂ k[uu] is called an ideal of k[uu] if • f , g ∈ I ⇒ f + g ∈ I. • f ∈ I, g ∈ k[uu ] ⇒ f g ∈ I. Let { fλ | λ ∈ Λ } ⊂ k[uu] be a nonempty set of polynomials. Let I denote the set of polynomials of the form
∑ gλ f λ ,
λ ∈Λ
gλ ∈ k[uu ], ∀λ ∈ Λ ,
where the sum is finite, that is, gλ = 0 except for a finite number of λ . Clearly this I is an ideal. This I is called the ideal generated by { fλ | λ ∈ Λ } and is denoted by I = { fλ | λ ∈ Λ }. { fλ | λ ∈ Λ } is called a set or a system of generators of I. In particular if { fλ | λ ∈ Λ } = { f1 , . . . , fs } is a finite set, then I = { fλ | λ ∈ Λ } is simply denoted as I = f1 , . . . , fs . An ideal I is called a monomial ideal if it is generated by a subset M ⊂ M of monomials. By Dickson’s lemma, a monomial ideal I = M is generated by the (finite) set of minimal monomials v1 , . . . , vs ∈ M: I = v1 , . . . , vs .
(3.2)
Note that the set of minimal monomials of M is unique.
3.2 Term Order and Gr¨obner Basis Term order (term ordering, monomial ordering) is a total order ≺ on the set M of monomials, such that • 1 ≺ v for 1
= v. • v ≺ w implies vt ≺ wt for every t ∈ M. An example of a term order is pure lexicographic term order ≺lex , where v = u x is ordered by the lexicographic order of the exponents x = (x1 , . . . , xη ). In the lexicographic order we order the indeterminates as u1 u2 · · · uη .
36
3 Toric Ideals and Their Gr¨obner Bases
Another example is the graded lexicographic term order ≺grlex , where monomials v = u x are first compared by the total degree |xx|1 = ∑i ∈I x(ii) and then (in the case of the same degree) by the lexicographic order of x . The third example is the graded reverse lexicographic term order ≺grevlex , where monomials are first compared by the total degree and then (in the case of the same degree) u x grevlex u y if the last nonzero element of x − y is negative. Let f = c1 v1 + · · · + cs vs ∈ k[uu ] be a nonzero polynomial, where 0
= c j ∈ k, v j ∈ M, j = 1, . . . , s. Given a term order ≺, we can take v1 such that v1 v j , j = 2, . . . , s. v1 is called the initial monomial (leading monomial, leading term) of f and written as in≺ ( f ). For example, with the pure lexicographic term order, the initial monomial of f = (3/2)u21u2 u33 + (1/3)u1u32 u23 is given by 3 1 in≺ ( f ) = in≺ ( u21 u2 u33 + u1 u32 u23 ) = u21 u2 u33 . 2 3 For an ideal I
= {0}, its initial ideal in≺ (I) is defined as in≺ (I) = {in≺ ( f ) | 0
= f ∈ I}, which is the monomial ideal generated by initial monomials of f ∈ I, f
= 0. By (3.2) there exist f1 , . . . , fs ∈ I such that in≺ (I) = in≺ ( f1 ), . . . , in≺ ( fs ). Based on this fact a Gr¨obner basis is defined as follows. Definition 3.1. Fix a term order ≺. A finite subset G = { f1 , . . . , fs } of nonzero elements of an ideal I is a Gr¨obner basis of I with respect to the term order if in≺ (I) = in≺ ( f1 ), . . . , in≺ ( fs ). More informally, { f1 , . . . , fs } is a Gr¨obner basis of I if the initial monomial of any f ∈ I is divisible by the initial monomial of some f j , j = 1, . . . , s. From this definition it is not immediately clear that G is indeed a set of generators of I. However, again based on Dickson’s lemma, it can be shown that G is indeed a set of generators of I (Sect. 2.5 of [42]). Proposition 3.1. I is generated by any Gr¨obner basis G of I. The following “Hilbert basis theorem” (Theorem 4 in Sect. 2.5 of [42]) is an immediate consequence of this proposition. Corollary 3.1 (Hilbert basis theorem). Every ideal I of the polynomial ring k[uu ] has a finite set of generators. If in≺ ( f1 ), . . . , in≺ ( fs ) are minimal monomials of {in≺ ( f ) | 0
= f ∈ I}, then G = { f1 , . . . , fs } is a minimal Gr¨obner basis, in the sense that any proper subset of G is no longer a Gr¨obner basis. Given a minimal Gr¨obner basis G = {in≺ ( f1 ), . . . , in≺ ( fs )}, no initial monomial in≺ ( fi ) is divisible by any one of the other initial monomials in≺ ( f j ), j
= i. However, other (noninitial) monomials appearing in fi may be
3.2 Term Order and Gr¨obner Basis
37
divisible by some in≺ ( f j ), j
= i. By replacing noninitial monomials appearing in fi with the remainders by f j , j
= i, we arrive at the reduced Gr¨obner basis. More formally, the reduced Gr¨obner basis is defined as follows. Definition 3.2. A Gr¨obner basis G is reduced if (i) for each f ∈ G, the coefficient of in≺ ( f ) is one, and (ii) for each f ∈ G, no monomial appearing in f lies in {in≺ (g) | g ∈ G, g
= f }. It is known that the reduced Gr¨obner basis is unique for any given term order ≺. Given a term order ≺ and an ideal I, a monomial v is called a standard monomial if v
∈ in≺ (I). Every polynomial f ∈ k[uu] is a unique finite linear combination of monomials with coefficients from k, thus k[uu] is an infinite-dimensional vector space over k, where the monomials are the basis vectors. Similarly I can be regarded as a vector space over k. Regarding them as vector spaces means that we only consider addition of polynomials and ignore multiplication of polynomials. Then we can regard the “quotient ring” k[uu]/I as a complementary linear subspace of I in k[uu]. Concerning this quotient vector space k[uu ]/I, the following theorem holds (cf. Sect. 5.3 of [42], Chapter 1 of [139]). Theorem 3.1. The set of standard monomials forms a basis of the vector space k[uu]/I over k. An ideal I is zero-dimensional if the vector space k[uu]/I over k is finitedimensional (Appendix D of [42]). By Theorem 3.1, I is zero-dimensional if and only if the number of standard monomials is finite. Theorem 3.1 for the case of a zero-dimensional ideal is fundamental for a Gr¨obner basis approach to design of experiments in Chap. 15. We now consider dividing a polynomial by a set of polynomials. For univariate polynomials f (u) and g(u), the division of f by g can be performed by repeatedly eliminating the leading term (i.e., the initial monomial) of f (u) by the leading term of g. The resulting expression is f (u) = q(u)g(u) + r(u), where q(u) is the quotient and r(u) is the remainder with degr < deg g. A generalization of this division to more than one indeterminate is given as follows. Fix a term order ≺. Let f
= 0 be the dividend and let g1 , . . . , gs
= 0 be the divisors. Then f can be written as follows. f = q1 g1 + · · · + qs gs + r,
q1 , . . . , qs ∈ k[uu ],
(3.3)
where (i) if r
= 0, then every monomial appearing in r is divisible by none of in≺ (g1 ), . . . , in≺ (gs ), and (ii) in≺ ( f ) in≺ (qi gi ) for i = 1, . . . , s. r is the remainder of this division. Actually there is an algorithm called the division algorithm (Chapter 1 of [42]) that yields (3.3). In general, a remainder r is not unique and the output of the division algorithm depends on the order of g1 , . . . , gs . However,
38
3 Toric Ideals and Their Gr¨obner Bases
if G = {g1 , . . . , gs } is a Gr¨obner basis for an ideal I, then the remainder r is uniquely determined. Also in this case we have r = 0 ⇔ f ∈ I.
(3.4)
3.3 Buchberger’s Algorithm The importance of the theory of Gr¨obner basis lies in the fact that there is an algorithm to compute the Gr¨obner basis. For two monomials v = u x , w = u y , the least common multiple of them is defined as lcm(v, w) = u max(xx,yy) , where max denotes the elementwise maximum. For nonzero polynomials f , g ∈ k[uu ], let c f and cg denote coefficients of their initial monomials. The S-polynomial of f and g is defined as S( f , g) =
lcm(in≺ ( f ), in≺ (g)) lcm(in≺ ( f ), in≺ (g)) f− g. c f · in≺ ( f ) cg · in≺ (g)
(3.5)
The right-hand side looks somewhat complicated, but the purpose of the operation on the right-hand side is to cancel the initial monomials of f and g. For example, with the pure lexicographic term order, 1 u1 u2 u3 1 S u1 u2 − 2u3u4 , u1 u3 − u2 u5 = 1 u1 u2 − 2u3u4 2 2 2 u1 u2 −
u1 u2 u3 (u1 u3 − u2u5 ) u1 u3
= −4u23 u4 + u22u5 .
(3.6)
Fix a term order ≺. The following theorem is called Buchberger’s criterion (Theorem 6, Sect. 2.5 of [42]). Theorem 3.2 (Buchberger’s Criterion). Let G = {g1 , . . . , gs } be a set of generators of an ideal I
= {0}. G is a Gr¨obner basis of I if and only if for all pairs i
= j, the remainder on division of S(gi , g j ) by G (listed in some order) is zero. Based on this criterion, the following simple idea can be implemented as an algorithm, called Buchberger’s algorithm to compute a Gr¨obner basis of I = g1 , . . . , gs : As long as G = {g1 , . . . , gs } is not a Gr¨obner basis of I, keep adding to G a remainder of some S(gi , g j ) by G. Note that Buchberger’s algorithm can be used when a finite set of generators of I is known. In contrast, toric ideals, which are important for algebraic statistics, are defined implicitly and the problem is to obtain a set of generators for the ideals. In this case Buchberger’s algorithm cannot be used directly. However, it is also fundamental to toric ideals via the elimination theory described in the next section.
3.5 Toric Ideals
39
3.4 Elimination Theory In the polynomial ring k[uu], u = (u1 , . . . , uη ), consider the set of polynomials involving only uζ +1 , . . . , uη , where 1 ≤ ζ < η . Let k[uζ +1 , . . . , uη ] denote the set of these polynomials. For an ideal I of k[uu], it is easy to check that I ∩ k[uζ +1 , . . . , uη ] is an ideal of k[uζ +1 , . . . , uη ]. I ∩ k[uζ +1 , . . . , uη ] is called an elimination ideal because the indeterminates u1 , . . . , uζ are eliminated. The main use of the elimination ideal is for solving a set of polynomial equations. For a Markov basis, its use is to give a general algorithm for computing a Gr¨obner basis of a toric ideal (see Proposition 3.2 below). When a Gr¨obner basis G of I with respect to the lexicographic order u1 · · · uη is given, it is straightforward to obtain a Gr¨obner basis of the elimination ideal (Sect. 3.1 of [42]). Theorem 3.3 (The Elimination Theorem). Let G be a Gr¨obner basis of I with respect to the lexicographic order u1 · · · uη . Then G ∩ k[uζ +1 , . . . , uη ] is a Gr¨obner basis of the elimination ideal I ∩ k[uζ +1 , . . . , uη ].
3.5 Toric Ideals So far we summarized relevant facts on a Gr¨obner basis of a general ideal I of k[uu]. For the theory of Markov basis we only need to consider a special kind of ideal, called a toric ideal. In this section we give more detailed explanations than in previous sections, because toric ideals are not covered in [42]. For defining a toric ideal we start with a ν × η integer matrix A. We call A a configuration. In this book we assume that the row vector (1, 1, . . . , 1) is in the real vector space spanned by the rows of A; that is, there exists a ν -dimensional real vector θ such that θ A = (1, 1, . . . , 1). (3.7) This assumption is called homogeneity of the toric ideal (Lemma 4.14 of [139]). Under the assumption of homogeneity, for z ∈ kerZ A 0 = θ Azz = (1, 1, . . . , 1)zz =
∑ z(ii).
(3.8)
i ∈I
In algebraic statistics, the rows of A are indexed by sufficient statistics and the columns of A are indexed by the cells i of the sample space I . Hence let us denote the elements of A as a j (ii), j = 1, . . . , ν , i = 1, . . . , η . Let a (ii) = (a1 (ii), . . . , aν (ii)) denote the i th column of A. We often consider A also as a set of its column vectors {aa(ii), i = 1, . . . , η }. Note that under the homogeneity, each column a (ii) is a nonzero vector, because otherwise θ A has 0 in the i th position.
40
3 Toric Ideals and Their Gr¨obner Bases
A difference of two monomials (with coefficient 1 and −1) f = w − v,
v = ux ,
w = uy
is called a binomial. For a monomial v = u x or the frequency vector x , the support is defined as the set of (indices of) indeterminates with positive powers: supp(uu x ) = supp(xx) = {ii | x(ii) > 0}.
(3.9)
A binomial w − v is called square-free if both w and v are square-free monomials. Consider two nonnegative integer vectors x , y ∈ Nη such that Axx = Ayy. Then z = y − x belongs to kerZ A. Now we give the first definition of toric ideal IA ⊂ k[uu]. Definition 3.3. The toric ideal IA = {uuy − ux | Axx = Ayy, x , y ∈ Nη } is the ideal generated by binomials u y − ux such that y − x ∈ kerZ A. For example, for the configuration A in (1.18) for the independence model of I × J two-way contingency tables, IA is the ideal generated by u x − uy , where x , y share the common row sums and column sums. So far we allowed negative elements in A. We now argue that without loss of generality we can assume that the elements of A are nonnegative under the assumption of homogeneity. Let a ∈ Nν be a column vector whose elements are large enough. We add a (1, 1, . . . , 1) to A. Let A˜ = A + a (1, 1, . . . , 1) be the resulting matrix whose elements are nonnegative. Note that θ A˜ is written as
θ A˜ = (1, 1, . . . , 1) + θ a (1, 1, . . . , 1) = (1 + θ a )(1, 1, . . . , 1). By appropriately choosing a we can make 0
= (1 + θ a ). Hence A˜ also satisfies the assumption of homogeneity. Now by (3.8) we have kerZ A = kerZ A˜ and IA = IA˜ . Therefore we can assume that the elements of A are nonnegative. We are now going to present another definition of a toric ideal. Introduce indeterminates q1 , . . . , qν corresponding to the rows of A. Let q = {q1 , . . . , qν } and let k[qq ] denote the polynomial ring in q over k. Consider a map πA from k[uu] to k[qq ] such that each indeterminate u(ii) is mapped to a monomial in k[qq ] as a (ii ) a (ii )
a (ii )
πA : u(ii) → q a(ii) = q11 q22 . . . qνν , where a (ii) is the i th column of A. For a polynomial f ∈ k[uu], πA ( f ) is obtained by substituting q a(ii) into the indeterminate u(ii), i ∈ I . Then, for a monomial u x = ∏i ∈I u(ii)x(ii) ,
3.5 Toric Ideals
41
πA (uux ) = πA ( ∏ u(ii)x(ii ) ) = i ∈I
=
ν
∑
∏ q j i∈I
x(ii )a j (ii )
∏ πA(u(ii))x(ii) = ∏ qa(ii)x(ii)
i ∈I
i ∈I
= q Axx
(3.10)
j=1
and for a polynomial f = ∑x cx u x ∈ k[uu ]
πA ( f ) = ∑ cx πA (uux ) = ∑ cx q Axx ∈ k[qq]. x
x
We see that πA is a homomorphism from k[uu] to k[qq ]. Let us illustrate πA for the case of the independence model of two-way tables under the multinomial sampling scheme. Let (i, j) denote the cell of a two-way table and consider the probability pi j of the cell as an indeterminate (instead of u(ii)). Under the independence model pi j = ri c j . We can understand this as “substituting ri c j into pi j ” and consider πA : pi j → ri c j . For I = J = 2, consider the following contingency table: x=
12 . 10
(3.11)
The probability, without the hypothesis of independence, of this contingency table x is written as 4 p(xx) = p x = 12p11 p212 p21 . 1, 2, 1, 0 Furthermore under the hypothesis H of independence, by substituting ci r j into pi j , the probability of x is given by
πA (12p11 p212 p21 ) = 12πA(p11 )πA (p212 )πA (p21 ) = 12(r1 c1 )(r1 c2 )2 (r2 c1 ) = 12r13 r21 c21 c22 . By (3.10) the exponents of ri and c j on the right-hand side correspond to the marginal frequencies of x . Indeed, for example, r13 on the right-hand side shows that the first row sum of x is x1+ = 3. Now the second definition of IA is given as the kernel of this πA . Definition 3.4.
IA = { f ∈ k[uu] | πA ( f ) = 0}.
(3.12)
It can be shown that Definitions 3.3 and 3.4 are equivalent. See Lemma 4.1 of [139]. By (3.10) it is easily seen that Ayy = Axx ⇔ πA (uuy − ux ) = 0.
(3.13)
42
3 Toric Ideals and Their Gr¨obner Bases
Hence the ideal in Definition 3.3 is clearly a subset of the ideal in Definition 3.4. We need some extra argument to show that they are the same. Again consider an example of 2 × 2 table. Let x be as in (3.11) and let y=
21 , 01
which has the same marginal frequencies as x . Then
πA (uuy − ux ) = πA (uuy ) − πA(uux ) = (r1 c1 )2 (r1 c2 )(r2 c2 ) − (r1 c1 )(r1 c2 )2 (r2 c1 ) = r13 r2 c21 c22 − r13 r2 c21 c22 = 0. Hence u y − ux ∈ IA . A Gr¨obner basis of IA can be obtained by the elimination theory of the previous section and Definition 3.4. Let k[qq , u ] = k[q1 , . . . , qν , u1 , . . . uη ] be the polynomial ring in q1 , . . . , qν , u1 , . . . uη over k. Consider an ideal JA = {u(ii) − q a(ii) , i ∈ I } of k[qq, u ]. Then IA is characterized as an elimination ideal of JA . Combining this fact with Buchberger’s algorithm and the elimination theory, a Gr¨obner basis of IA can be computed as follows (Algorithm 4.5 of [139]). Proposition 3.2. The toric ideal IA is written as IA = JA ∩ k[uu]. Let ≺ be the pure lexicographic term order such that q1 · · · qν u1 · · · uη . By Buchberger’s algorithm compute a Gr¨obner basis G of JA . Then G ∩ k[uu] is a Gr¨obner basis of IA . In this algorithm, as long as q j ui for all 1 ≤ j ≤ ν , 1 ≤ i ≤ η , the orders within q and u can be different from the lexicographic term order. As mentioned above, in Definitions 3.3 and 3.4 no finite set of generators of IA is given and Proposition 3.2 gives a general algorithm for obtaining a set of generators of IA in the form of a Gr¨obner basis. However, it is also of interest to consider Buchberger’s algorithm for a toric ideal, when a finite set of generators is known. Consider the operation of forming an Spolynomial in (3.5) and (3.6). In (3.6), for illustrative purpose, we had coefficients 1 2 and 2. Suppose we compute the S-polynomial of two binomials. For example, S(u1u2 − u3 u4 , u1 u3 − u2u5 ) =
u1 u2 u3 u1 u2 u3 (u1 u2 − u3u4 ) − (u1 u3 − u2u5 ) u1 u2 u1 u3
= −u23 u4 + u22u5 .
(3.14)
3.5 Toric Ideals
43
We note that the right-hand side is a binomial. It is clear that this holds for two arbitrary binomials and the following lemma holds. Lemma 3.2. The S-polynomial of two binomials is a binomial. This lemma is important for proving the fundamental theorem of Markov basis in Sect. 4.4. Furthermore, from this lemma the following result can be shown (Corollary 4.4 of [139]). Proposition 3.3. For any term order, the reduced Gr¨obner basis of a toric ideal IA consists of binomials. This is consistent with Proposition 3.2 and Definition 3.3, because the reduced Gr¨obner basis G of JA consists of binomials and hence G ∩ k[uu ] consists of binomials as well. Also by Definition 3.2 it is obvious that G ∩ k[uu] is reduced if G is reduced.
Part II
Properties of Markov Bases
In Part II of this book, we define Markov bases more precisely and develop a general theory of Markov bases. In Chap. 4 we define Markov bases for discrete exponential family models and discuss other relevant bases, such as lattice bases and the Graver basis. In Chap. 5 we consider minimality of Markov bases and establish basic structures of minimal Markov bases. We define the notion of indispensable moves and establish a condition for the existence of the unique minimal Markov basis. In Chap. 6 we give a formal presentation of the distance reduction argument, which is often very useful for proving that a given set of moves forms a Markov basis. Finally in Chap. 7 we define the notion of invariance of Markov bases using the notion of action of groups on the set of cells.
Chapter 4
Definition of Markov Bases and Other Bases
4.1 Discrete Exponential Family As in Chap. 1 let I denote a finite sample space. Because of many applications to contingency table models, we call an element i ∈ I a cell. We consider a family {p(ii; θ )}, θ = (θ1 , . . . , θν ) of distributions over I of the form log p(ii; θ ) =
ν
∑ θ j a j (ii) − ψ (θ ),
(4.1)
j=1
where exp(−ψ (θ )) is the normalizing constant of the exponential family. Note that θ j was denoted as φ j (θ ) in (1.11). Equation (4.1) corresponds to the multinomial sampling scheme of Chap. 1. This model is often called a log affine model of probability distributions over I . Let η = |I | and let A = {a j (ii)},
j = 1, . . . , ν , i ∈ I
denote a ν × η matrix with the ( j, i ) element a j (ii). A is the configuration introduced in the previous chapter. In many regression settings, A corresponds to the design matrix of regressors. Then it is natural to call A a design matrix. Actually the transpose A of A is called the design matrix in regression settings. In our setting a configuration A has more columns than rows. In a regression setting the design matrix usually has more rows than columns. It is also useful to look at the rows a j = (a j (ii), i ∈ I ) of the configuration A. Except for the (negative logarithm of the) normalizing constant ψ , (4.1) implies that the logarithm of the probability vector {p(ii; θ ), i ∈ I } lies in the linear space spanned by the rows of A. In this sense we call the linear space spanned by the rows of A the model space and denote it by rowspan(A). In this book, we assume homogeneity (3.7); that is, the constant row vector (1, 1, . . . , 1) is in rowspan(A). In many contingency table models we often allow linear dependence among the rows of A for symmetry of describing the model. For example, in the two-way S. Aoki et al., Markov Bases in Algebraic Statistics, Springer Series in Statistics 199, DOI 10.1007/978-1-4614-3719-2 4, © Springer Science+Business Media New York 2012
47
48
4 Definition of Markov Bases and Other Bases
independence model we usually take ν = I + J for the independence model of I × J two-way tables, although the degrees of the model (the dimension of the model) is I + J − 1. Therefore ν is not necessarily the dimension of rowspan(A). Let q j = eθ j , j = 1, . . . , ν . Taking the exponential, (4.1) is written in the monomial form ν
a (ii )
p(ii; θ ) = e−ψ (θ ) × ∏ q j j
= e−ψ (θ ) q a(ii) .
(4.2)
j=1
The exponential family notation in (4.1) is more traditional in statistics. The monomial form (4.2) is often called a “toric model” in algebraic statistics. In the exponential form it is assumed that p(ii; θ ) > 0 for all i. However in (4.2) p(ii; θ ) = 0 is allowed. Suppose that we obtain n observations from the distribution (4.1) under the multinomial sampling and let x(ii), i ∈ I denote the frequencies of the cells. Then the joint probability function of the frequency vector x = {x(ii)}i ∈I is written as n! ∏ p(ii; θ )x(ii) ∏i ∈I x(ii)! i ∈I ν n! exp ∑ θ j ∑ a j (ii)x(ii ) − nψ (θ ) . = ∏i ∈I x(ii)! j=1 i ∈I
p(xx) =
(4.3)
Then a sufficient statistic of the model is given by t j = ∑i ∈I a j (ii)x(ii), j = 1, . . . , ν . We write this relation as t = Axx,
(4.4)
where t = (t1 , . . . ,tν ) is the ν -dimensional column vector of t j s and x is the η -dimensional column vector of frequencies. Then (4.3) is written as p(xx) =
n! exp(θ Axx − nψ (θ )). ∏i ∈I x(ii)!
(4.5)
We denote the set of frequency vectors as X = Nη and the set of frequency vectors with the common value of the sufficient statistic by Ft = {xx ∈ X | Axx = t } and call it a t -fiber. We denote the set of possible values of the sufficient statistic as T = TA = {tt | t = Axx, x ∈ Nη }.
(4.6)
TA is often referred to as a semigroup generated by A. In the notation of Sect. 3.5, let q = {q1 , . . . , qν } be indeterminates corresponding to the rows of A and let k[qq ] denote the polynomial ring in q . The image πA (k[uu]), which is a subring of k[qq], is called the semigroup ring associated with the configuration A.
4.1 Discrete Exponential Family
49
Under the assumption of homogeneity, let θ be a row vector such that θ A = (1, 1, . . . , 1). Then for two frequency vectors x , x˜ in the same fiber we have 0 = A˜x − Axx
⇒
˜ i) − ∑ x(ii). 0 = θ A˜x − θ Axx = ∑ x(i i
i
Therefore the sample size n of frequency vectors in the same fiber is common. We call this sample size n the degree of x as well as the degree of t = Axx: n = deg x = ∑ x(ii) and
degtt = deg x
(for any x such that t = Axx).
i
In this book we sometimes write |xx| = deg x and |tt | = degtt , although degtt is not the 1-norm of the vector t . Also note that each fiber is finite, because Ft ⊂ {xx ∈ Nη | deg x = degtt } and the right-hand side is a finite set. Given the values of the sufficient statistic t , the conditional distribution of x does not depend on the parameters θ = (θ1 , . . . , θν ). Note that the marginal probability function of t is written as ν n! exp ∑ θ j t j − nψ (θ1, . . . , θν ) . p(tt ) = ∑ i j=1 x ∈Ft ∏i ∈I x(i )! Therefore the conditional distribution of x given t is written as p(xx | t ) = c ×
1 ∏i ∈I x(ii)!
,
x ∈ Ft ,
(4.7)
where c is the normalizing constant. We call (4.7) the hypergeometric distribution over the fiber Ft . If we can sample from the hypergeometric distribution, we can perform the conditional tests of the fit of the model (4.3). The Markov basis allows us to construct a Markov chain over the fiber Ft for this purpose. The normalizing constant −1 1 c = ct = ∑ i x ∈Ft ∏i ∈I x(i )! cannot be expressed in a closed form except for special cases, such as the decomposable models for contingency tables. In this respect, the Markov chain Monte Carlo method is especially useful, because a Markov chain can be constructed without knowing the normalizing constant.
50
4 Definition of Markov Bases and Other Bases
4.2 Definition of Markov Basis A Markov basis is a set of “moves” for constructing a Markov chain over any fiber. Let z ∈ Zη denote an η -dimensional column vector of integers. z is called a move if Azz = 0; that is, z belongs to the integer kernel kerZ A = ker A ∩ Zη of A. If Axx = t and z is a move, then A(xx + z ) = Axx = t . Therefore by adding z to x we remain in the same fiber as long as x + z does not contain a negative element. If x + z contains a negative element, then we have to choose another move z to add to x . Suppose that we have a set of moves B, then by adding moves from B to the current frequency vector we can “move around” a fiber. Our purpose is to find a finite set of moves B = {zz1 , . . . , z L }, such that we can move all over the fiber. For the Markov basis we also require that z 1 , . . . , z L allow us to move all over every fiber t , namely, for every possible value of the sufficient statistic t . Note that −zz is a move if z is a move. When x + z contains a negative element, we might try x − z instead. So we can also subtract a move from x . For convenience we often ignore the sign of z and think of ±zz as a move. Suppose that we are given a finite set of moves B = {zz1 , . . . , z L }. We consider an undirected graph G = Gt ,B whose vertices are the elements of a fiber Ft . We draw an (undirected) edge between x and y if there exists z ∈ B such that y = x + z or y = x − z . Being able to move all over Ft corresponds to the connectedness of Gt ,B . Therefore we are led to the following definition of a Markov basis. Definition 4.1. A finite set B = {zz1 , . . . , z L } of moves is called a Markov basis if Gt ,B is connected for every t ∈ T . In this definition we require the finiteness of B. This causes no difficulty because the existence of a Markov basis is guaranteed by Hilbert’s basis theorem (see Corollary 3.1 of Chap. 3). Note that the definition of a Markov basis in Definition 4.1 is equivalent to the earlier definition given in (2.6). How about uniqueness in the definition of a Markov basis? Except for some special cases, Markov bases are not unique. First, if B is a Markov basis, then B ∪ {zz} is a Markov basis for every move z . Therefore when we ask the question of uniqueness, we naturally should consider Markov bases that are minimal in the sense of set inclusion. Even with the requirement of minimality, Markov bases are not unique in general. This fact leads to various notions and classes of Markov bases. Another point in the definition of a Markov basis is that it is common for every fiber Ft , ∀tt ∈ T . Given a particular data set x ∈ FAxx , the set of moves connecting FAxx alone may be smaller and need not be a Markov basis. However, at present there is no general methodology for obtaining a set of moves connecting a particular fiber. Therefore we use Definition 4.1, except for Chap. 13.
4.3 Properties of Moves and the Lattice Basis
51
At this point we discuss the basic relation between the model space rowspan(A) and the kernel of A. It is a standard fact in linear algebra that rowspan(A)⊥ = ker A,
rowspan(A) = (ker A)⊥ ,
(4.8)
where ⊥ denotes the orthogonal complement. The first equality can be seen as follows. For w ∈ Rη , w=0 w ∈ ker A ⇔ Aw w = 0, ∀θ ∈ Rν ⇔ θ Aw ⇔ w ∈ rowspan(A)⊥ .
(4.9)
The second equality holds because (L⊥ )⊥ = L for any subspace L of Rη . It is useful to remember that the model space of A and the kernel of A are equivalent in the sense of (4.8). In studying a statistical model, we can use either rowspan(A) or ker A.
4.3 Properties of Moves and the Lattice Basis In this section we summarize basic properties of moves. For a move z ∈ kerZ A, we distinguish its positive elements and negative elements. Collect the positive elements of z into its positive part z + ∈ Nη as z+ (ii) = max(0, z(ii)),
i ∈I.
Similarly define the negative part z − of z by z− (ii) = − min(0, z(ii)), i ∈ I . Then z is written as the difference of its positive part and negative part z = z + − z− . Note that Azz = 0 means that Azz+ = Azz− ; that is, the positive part and the negative part of a move belong to the same fiber. In view of this fact we sometimes say that z is a move belonging to the fiber Ft , where t = Azz+ . We define the degree of z (degzz) by the degree of z + (or z − ). Note that |zz| = 2 degzz, where |zz| is the 1-norm of z . For z ∈ Zη , the support of z is defined to be the set of cells where z is nonzero: supp(zz) = {ii | z(ii ) = 0}. Note that supports of z + and z − are disjoint: / supp(zz+ ) ∩ supp(zz− ) = 0.
(4.10)
52
4 Definition of Markov Bases and Other Bases
Now let x and y be two frequency vectors of the same fiber with disjoint supports (supp(xx) ∩ supp(yy) = 0). / Then z = y−x is a move with z + = y and z − = x . Therefore a move is in a one-to-one relation to an ordered pair of two frequency vectors in the same fiber with disjoint supports. Note that z = y − x for x , y ∈ Ft is always a move. However, if supports of x and y have a nonempty intersection, then x is larger than z − in some cell i . Similarly y is larger than z + in some cell i . In fact we can write z + = y − min(xx, y ),
z − = x − min(xx, y ),
(4.11)
where min(xx, y ) is the elementwise minimum of x and y . In the definition of Markov basis, we are concerned whether adding a move z to a frequency vector x produces a negative cell. Write x + z = (xx − z − ) + z + . Here we are subtracting the frequencies z − from x and then adding the frequencies z + in different cells. Therefore x + z contains a negative cell if and only if x − z − contains a negative cell. In other words, x + z does not contain a negative cell if and only if x ≥ z−, (4.12) where the inequality is elementwise. When (4.12) holds, we say that z can be added to x . When z can be added to x or can be subtracted from x , we say that z is applicable to x . By (4.11), in the notation of monomials and binomials of the previous chapter, x + ε z = y , ε = ±1, if and only if +
−
u y = u x + ε umin(xx,yy) (uuz − u z ). +
(4.13)
−
A move z is called square-free if u z − u z is a square-free binomial; that is, if the elements of z are −1,0, or 1. Markov bases are difficult exactly because we are worried about producing negative elements. Suppose that we just ignore the nonnegativeness of elements of frequency vector. Then the notion of a basis is simple. Note that kerZ A as a subset of Zη is closed under integer multiplication and addition: z 1 , z 2 ∈ kerZ A ⇒ azz1 + bzz2 ∈ kerZ A,
∀a, b ∈ Z.
A subset of Zη with this property is called an integer lattice. Let d = dim kerZ A = η − rankA denote the dimension of linear space spanned by the elements of kerZ A in Rη . It is a standard fact [134] that an integer lattice contains a lattice basis {zz1 , . . . , z d }, such that every z ∈ kerZ A is a unique integer combination of z 1 , . . . , z d .
4.3 Properties of Moves and the Lattice Basis
53
Given A, it is fairly easy to obtain a lattice basis of kerZ A using the Hermite normal form of A. In statistical applications, the configuration A often has redundant rows as in the case of two-way contingency tables in Sect. 1.3. However, ker A and hence kerZ A are defined by linearly independent rows of A. Therefore consider A : ν × η with linearly independent rows; rank A = ν . Then there exists an integer matrix U with detU = ±1 (called a unimodular matrix) such that AU = (B, 0), where B is a ν × ν upper triangular matrix with positive elements. Then the columns ν + 1, . . ., η of U give a lattice basis of kerZ A. See Sect. 4.1 of [134]. In the case of the independence model of I × J two-way contingency tables, as elements of the lattice basis we can take the moves +1 −1 −1 +1 where the lower-right +1 is in the (I, J) cell. However, as we saw in Sect. 2.1, these moves do not form a Markov basis. On the other hand, it is easy to see that a Markov basis B always contains a lattice basis. Given any z = 0 ∈ kerZ A, we can move from z − to z + by a sequence of moves from B, namely we can write z + = z − + ε1 z i1 + · · · + εK z iK ,
ε j = ±1,
z i j ∈ B,
j = 1, . . . , K.
Then z = z + − z − is written as an integer combination of elements of B. So far we have considered the integer lattice kerZ A. We now look at the integer lattice L generated by the columns of A: L = ZA = {Azz | z ∈ Zη }. Also let cone(A) = R≥0 A = { ∑ ci a (ii) | ci ≥ 0, i ∈ I } i ∈I
denote the cone generated by the column of A. Then the semigroup TA in (4.6) is clearly a subset of L ∩ cone(A): TA ⊂ L ∩ cone(A).
(4.14)
We introduce some terminology concerning the semigroup TA . If the equality holds in (4.14) then the semigroup TA is called normal. L ∩ cone(A) is called the saturation of TA (Definition 7.24 of [105]). When TA is not normal, the elements of L ∩ cone(A) \ TA are called holes of the saturation of TA [85, 145, 146]. If L ∩ cone(A) \ TA is a finite set, then the semigroup is called very ample [113]. Although in this book we do not go into details on normality of semigroups, the notion of normality is important for Proposition 5.4 and in discussing non-squarefree indispensable moves in Sect. 9.5.
54
4 Definition of Markov Bases and Other Bases
4.4 The Fundamental Theorem of Markov Basis In this section we explain relations between moves and binomials of a toric ideal. Then we prove the fundamental theorem of Markov bases, which states that a Markov basis is a set of generators of a toric ideal. For readability we repeat some material from Chap. 3. Consider the monomial form of the model (4.2): ν
a (ii )
p(ii) ∝ ∏ q j j . j=1
Let us regard p(ii), i ∈ I , as “symbols” or indeterminates rather than probabilities. Also let us regard q j , j = 1, . . . , ν , as indeterminates. Then the above model a (ii )
assigns to each indeterminate p(ii) a monomial ∏νj=1 q j j in q j s. We formalize this consideration as follows. Let k be a field, such as the field R of real numbers. Let k[pp ] = k[p(ii), i ∈ I ] be the polynomial ring in p = {p(ii), i ∈ I }. Similarly define k[qq] = k[q1 , . . . , qν ] to be the polynomial ring in q1 , . . . , qν . As in Sect. 3.5, define a homomorphism πA : k[pp] → k[qq ] by ν
a (ii)
πA (p(ii)) = ∏ q j j . j=1
πA for a general polynomial of k[pp] is defined by a homomorphism, that is, by a (ii ) substituting ∏νj=1 q j j into each p(ii). The toric ideal IA is defined as the kernel of πA : IA = ker πA = { f ∈ k[pp ] | πA ( f ) = 0}. These notions have already been illustrated in Chap. 1 and Sect. 3.5 with the example of two-way tables. Let x be a frequency vector. As in Chap. 3, x is identified with the monomial px =
∏ p(ii)x(ii) ,
i ∈I
which corresponds to the joint probability of x except for the multinomial coeffi+ − + − cient. A move z = z + − z − corresponds to a binomial v = p z − pz . p z − p z is in IA if and only if z = z + − z − is a move (see (3.13)). Now the fundamental theorem of Markov bases established by [50] states that a Markov basis corresponds to a system of generators of IA . Theorem 4.1 ([50]). A finite set of moves B is a Markov basis for A if and only if + − the set of binomials {pp z − p z | z ∈ B} generates the toric ideal IA . The “only if” (necessity) part is easy to prove. However, the proof of the “if” part (sufficiency) is somewhat hard. Because this theorem is of basic importance for
4.4 The Fundamental Theorem of Markov Basis
55
the whole theory of Markov bases, we give a careful proof. For a finite set of moves B = {zz1 , . . . , z L } for a configuration A, we write the set of corresponding binomials + − as FB = {ppzi − p z i , i = 1, . . . , L}. Proof. We first show the necessity: if B is a Markov basis for A then FB generates IA . Binomials generate (cf. Definition 3.3) the toric ideal IA , therefore we only need to show that any binomial f = p y − p x , Axx = Ayy, belongs to the ideal FB . Inasmuch as B is a Markov basis, x and y (∈ FAxx ) are mutually accessible by B. Hence S
s
y = x + ∑ ε j zi j ,
x + ∑ ε j z i j ∈ FAxx ,
j=1
1 ≤ s ≤ S,
j=1
for some S > 0, ε j ∈ {−1, 1}, z i j ∈ B, j = 1, . . . , S. Write x s = x + ∑sj=1 ε j z i j , 0 ≤ s ≤ S, with x 0 = x and x S = y . Then by (4.13), with the notation for indeterminates p instead of u , it follows that s
p x s = p x + ∑ ε j p min(xx j−1 ,xx j ) (pp
z+ i j
z−
− p i j ),
s = 1, . . . , S.
j=1
Hence p y − px =
S
z+
z−
∑ ε j p min(xx j−1 ,xx j ) (pp i j − p i j ) ∈ FB .
(4.15)
j=1
This proves the necessity part. Next we prove the “if” part (sufficiency). We want to show that if IA is generated by FB , then every x , y ( = x ) ∈ FAxx is mutually accessible by B. By the assumption p y − p x can be written as a finite sum L
+
−
p y − p x = ∑ fi (pp )(ppzi − p z i ),
(4.16)
i=1
where fi (pp) ∈ k[pp], i = 1, . . . , L, are polynomials in p . Expand fi (pp ) into monomials. Then allowing repetitions, we can write p y − p x as a finite sum p y − p x = ∑ al p h l (pp
z+ i l
z−
− p il ),
al ∈ k.
(4.17)
l
In Lemma 4.1 below we show that we can choose al as integers. Given this fact, z+
z−
z+
z−
instead of al p h l (pp il − p il ), we can write |al | times the binomial p hl (pp il − p il ) and use εl = ±1. Then by allowing further repetitions, we can write p y − p x as p y − px =
S
z+
z−
∑ ε j ph j (pp i j − p i j ),
ε j = ±1.
(4.18)
j=1
Note that (4.18) is already similar to (4.15). The difference between them is that in (4.15) the order of the terms from x to y on the right-hand is already given. On the
56
4 Definition of Markov Bases and Other Bases
other hand the sum on the right-hand side of (4.18) is not ordered for moving from x to y . We need to find a suitable path from x to y on the right-hand side of (4.18). The path can be found as follows. Expand (4.18) into 2S terms. Then at least one term has to be equal to p x . Namely for some j we have p x = ε j p h j p
z+ i j
Then by (4.13) we can move from x to x 1 such that p x 1 = p x − ε j p h j p z+ ij
z−
or p x = −ε j p h j p i j . z+ i j
or p x 1 =
px + ε j p h j p . In either case we can now write z+ i
p y − p x1 = ∑ εl p hl (pp
l
l =j
z−
− p il ),
where the sum on the right-hand side is a sum of S − 1 terms. Now we can employ induction on S and find the steps x 2 , . . . , x S to move from x to y = x S . This proves the sufficiency. In the above proof of sufficiency, the proof of the integerness of coefficients al in (4.17) is left to the following lemma. Lemma 4.1. If FB generates IA , then each binomial p y − p x ∈ IA can be written as a finite sum p y − p x = ∑ al phl (pp
z+ i l
z−
− p il ),
(4.19)
l
where the al s are integers. We give two different proofs of this lemma. The first proof is based on a Gr¨obner basis. The second proof is longer, but only uses linear algebra. Proof. Denote a Gr¨obner basis of IA by {g1 , . . . , gL }, which is obtained by Buchberger’s algorithm from the set of generators FB of IA . Because p y − p x ∈ IA = g1 , . . . , gL , the binomial p y − p x is written as L
p y − p x = ∑ f i gi ,
fi ∈ k[pp ].
i=1
By expanding fi into monomials, p y − p x is further written as p y − p x = ∑ al p v il gil , l
where a j ∈ k. Note that the sum on the right-hand side corresponds to a division by a Gr¨obner basis and each step of the division is an operation of eliminating the leading term by a monomial with coefficient ±1. Hence, if we allow repetitions, we can indeed assume a j = ±1. On the other hand, because g1 , . . . , gL ∈ IA = FB each element of the Gr¨obner basis is written as g j = ∑ d p w (pp
z+ i
z−
− p i ).
4.4 The Fundamental Theorem of Markov Basis
57
The right-hand side corresponds to obtaining g j by the Buchberger algorithm from the set of generators. In Lemma 3.2 we saw that the S-polynomial of two binomials is again a binomial. This implies that we can also assume d = ±1. Therefore, the binomial p y − p x can be written as (4.17) where al are integers. This completes our first proof. We now give an alternative proof. By the assumption, (4.19) holds with al ∈ k. At this point al s are not necessarily integers and we want to show that we can always replace al by integers. Note that there are only a finite number of monomials appearing in (4.19). Choose a sufficiently large D such that the degrees of all monomials in (4.19) are less than or equal to D. Let MD denote the set of monomials of degrees less than or equal to D. Let M = |MD | denote the cardinality of MD . Then MD is a basis of the M-dimensional vector space VD of polynomials of degree less than or equal to D. With respect to this basis, each binomial is represented as a column vector with two nonzero elements which are 1 and −1 and other elements are zeros. Let N = ND denote the number of binomials whose degrees are less than or equal to D. Let C be an M × N matrix whose columns correspond to binomials of degree less than or equal to D. Then the right-hand side of (4.19) is written as b = Caa , where b corresponds to p y − p x . Now by Lemma 4.2 below, there exists an M × M permutation matrix P and an N × N unimodular matrix U such that C˜ = PCU is of the form (4.20) below. Then b = Caa can be equivalently written as Pbb = C˜ a˜ ,
a˜ = U −1 a .
˜ it is clear that a˜ can be chosen to be an integer vector. Then From the form of C, a = U a˜ is an integer vector. This finishes our second proof. Lemma 4.2. Let C be an M × N matrix, such that each column c of C has two nonzero elements which are 1 and −1; namely, c is written as c = e i − e j , i = j, where e i is the ith standard basis vector with 1 in the ith position. Then by the following three elementary operations (i) sign change of columns, (ii) addition (or subtraction) of a column to (or from) another column, and (iii) permutation of rows, C can be transformed to the following block diagonal form. ⎛ Bd1 ⎜ 0 ⎜ ⎜ .. ⎜ . ⎜ ⎝ 0 0
0 Bd2 .. .
... ... .. .
0 0 .. .
⎞ 0 0⎟ ⎟ .. ⎟ , .⎟ ⎟ 0⎠
0 . . . BdK 0 ... 0 0
⎛
0 ... ⎜ 1 ... ⎜ ⎜ .. . . where Bd = ⎜ . . ⎜ ⎝ 0 0 ... −1 −1 . . . 1 0 .. .
0 0 .. .
⎞
⎟ ⎟ ⎟ ⎟ : d × (d − 1). (4.20) ⎟ 1⎠ −1
58
4 Definition of Markov Bases and Other Bases
Proof. We give a proof based on the induction on the number of columns. Let M be fixed. If N = 1, the result is trivial. Suppose that the result holds up to N and consider adding a new column c . We can assume that the first N columns have already been transformed to the form in (4.20). Let the new column be denoted as c = e i − e j . There are four cases to consider. Case 1. If neither of i, j belongs to (the rows of the) blocks Bd1 , . . . , BdK , then c = B2 forms a new block. Case 2. If both of i, j belong to a block, say Bd1 , then c can be transformed to 0. This is obvious if c is equal to some column of Bd1 . Otherwise c is the difference of the ith and jth columns of Bd1 . Case 3. Suppose that i belongs to, say, Bd1 and j does not belong to any block. Let j = d1 + 1 without loss of generality. Subtracting the ith column from c , we obtain (0, . . . , 0, 1, −1) . d1 −1
Adding this to other columns of Bd1 we obtain a new block of the form Bd1 +1 . Case 4. Suppose that i and j belong to different blocks, say Bd1 and Bd2 . If i = d1 and j = d2 , then adding c to the columns of Bd1 we obtain a new block of the form Bd1 +d2 . Otherwise, if i < d1 , subtract the ith column from c . Similarly if j < d2 , then subtract the jth column from c . Then c is transformed to e d1 − ed2 and this reduces to the former case. This lemma shows that the diagonal block of the Smith normal form of C is the identity matrix and every elementary divisor of C is 1 (cf. Sect. 4.4 of [134]). Remark 4.1. We can consider columns of C in Lemma 4.2 as edges of a graph. Consider a graph G with M vertices. For c = e i − e j , draw an edge between i and j. Then it can be easily seen that the blocks in (4.20) correspond to connected components of G . Remark 4.2. Two proofs of Lemma 4.2 look different but they are essentially the same. The second proof is based on the relations among vectors with two nonzero elements which are 1 and −1, such as ⎛
⎞ ⎛ ⎞ ⎛ ⎞ 1 0 1 ⎝−1⎠ + ⎝ 1 ⎠ = ⎝ 0 ⎠ , 0 −1 −1 where the second element is canceled. This in fact corresponds to forming an S-polynomial of two binomials.
4.5 Gr¨obner Basis from the Viewpoint of Markov Basis
59
4.5 Gr¨obner Basis from the Viewpoint of Markov Basis In the previous section we discussed the Gr¨obner basis from an algebraic viewpoint. Here we discuss it from the viewpoint of the Markov basis. In Chap. 3 we have already summarized relevant facts on the Gr¨obner basis. Here we discuss the Gr¨obner basis from a viewpoint close to our definition of the Markov basis. In the case of the Markov basis we tend to ignore the sign of a move z . In the Gr¨obner basis it is important to keep track of the sign of the move. As in Chap. 3 let a term order ≺ be given. Let G = {g1 , . . . , gL } ⊂ k[pp ] be a Gr¨obner basis with respect to ≺, such that g1 , . . . , gL are binomials. Then g1 , . . . , gL correspond to moves. By Proposition 3.3 the reduced Gr¨obner basis consists of binomials. Write +
−
gl = p z l − p z l ,
+
p zl = in≺ (gl ),
l = 1, . . . , L.
As before we identify the monomial p x with the frequency vector x . Because the term order is the total order, every fiber Ft (which is finite as remarked in Sect. 4.1) has the unique minimum element xt∗ . For any other x = xt∗ of ∗ ∗ x x x x x the fiber, p − p t ∈ IA . Also p p t . Therefore p is divisible by some in≺ (gl ); + − x that is, x ≥ z + l . Dividing p by gl corresponds to moving from x to x − (zzl − z l ), xt∗ which is a smaller element than x in Ft . On the other hand, p is divisible by none ∗ of in≺ (gl ), l = 1, . . . , L, because otherwise p xt would not be the minimum element of Ft . By the definition of the standard monomial (cf. Sect. 3.2) we have now shown the following fact. ∗
Lemma 4.3. Given a term order ≺, {ppx t | t ∈ T } is the set of standard monomials of IA . Now let B be any finite set of moves. For a given term order we always choose a sign of a move z = z + − z − ∈ B by +
−
pz pz . In a fiber Ft , we draw a directed edge from y to x if there exists z ∈ B such that y − x = z. Then each fiber Ft becomes a directed graph G¯t ,B . As discussed in the previous section, if y − x = z then x = z − + min(xx, y),
y = z+ + min(xx, y).
Hence p y p x by the second property of the term order. Therefore by subtracting z ∈ B from y , we always move to a smaller element of the fiber and there exists no directed loop in G¯t ,B ; that is, G¯t ,B is a directed acyclic graph (DAG). We now have the following proposition.
60
4 Definition of Markov Bases and Other Bases
Proposition 4.1. A finite set of moves B is a Gr¨obner basis if and only if from every element of every G¯t ,B there exists a directed path to the minimum element xt∗ of the fiber. Proof. If B is a Gr¨obner basis, then by Lemma 4.3 from every element of every G¯t ,B there exists a directed path to the minimum element xt∗ . Conversely, suppose that from every state of every G¯t ,B there exists a directed +
path to the minimum element xt∗ . Then every x ∈ Ft is divisible by some p z , z ∈ B. Hence B is a Gr¨obner basis.
4.6 Graver Basis, Lawrence Lifting, and Logistic Regression Finally we discuss the Graver basis and the Lawrence lifting. Consider a sum of two moves z = z 1 + z 2 . We say that there is no cancellation of signs in this sum if + supp(zz+ ) = supp(zz+ 1 ) ∪ supp(zz2 ),
− supp(zz− ) = supp(zz− 1 ) ∪ supp(zz2 ).
In this case we also say that z is a conformal sum of z 1 and z 2 . Similarly we say that there is no cancellation of signs in the sum of m moves z = z 1 + · · · + z m (or z is a conformal sum of m moves) if + supp(zz+ ) = supp(zz+ 1 ) ∪ · · · ∪ supp(zzm ),
− supp(zz− ) = supp(zz− 1 ) ∪ · · · ∪ supp(zz m ).
We also say that z 1 + · · · + z m is a conformal decomposition of z . We call a move z conformally primitive if it cannot be written as a sum of two nonzero moves z = z 1 + z 2 with no cancellation of signs. For clarity we say “conformally” primitive, but usually a conformally primitive move is simply called a primitive move. A binomial corresponding to a conformally primitive move is called a primitive binomial. Definition 4.2. The Graver basis is the set of conformally primitive moves. In Sect. 5.4.3 we show that the Graver basis is finite. We first see that the Graver basis is a Markov basis. For x , y in the same fiber let z = y − x . If a move z is not conformally primitive, then we can recursively decompose z into a conformal sum of moves. This implies that z can be written as a conformal sum z = z 1 + · · · + zm ,
(4.21)
where z 1 , . . . , z m are (not necessarily distinct) nonzero elements of the Graver basis. Then we can move from x to y by the above sequence of conformally primitive moves. This shows that the Graver basis is a Markov basis. Also note that because of no cancellation of signs, whenever z is applicable to some x , z can be replaced by z 1 , . . . , z m in arbitrary order without causing negative elements on the way. As an important example we consider the Graver basis for the independence model of I × J contingency tables.
4.6 Graver Basis, Lawrence Lifting, and Logistic Regression
61
Definition 4.3. For 2 ≤ r ≤ min(I, J), let i1 , . . . , ir be distinct row indices and let j1 , . . . , jr be distinct column indices. Denote i [r] = (i1 , . . . , ir ), j [r] = (i1 , . . . , ir ). A loop of degree r z r (ii[r] ; j [r] ) = {zi j },
1 ≤ i1 , . . . , ir ≤ I, 1 ≤ j1 , . . . , jr ≤ J,
(4.22)
is a move such that zi1 j1 = zi2 j2 = · · · = zir−1 jr−1 = zir jr = 1, zi1 j2 = zi2 j3 = · · · = zir−1 jr = zir j1 = −1, and all the other elements are zeros. There is at most one +1 and −1 in each row and each column of a degree r loop. An example of a loop of degree r = 3 is depicted as follows. 1 −1 0 0 1 −1 . −1 0 1 Then we have the following proposition. Proposition 4.2. Loops of degree 2 ≤ r ≤ min(I, J) form the Graver basis for the independence model of I × J contingency tables. Proof. Consider any nonzero move z . Let z (i1 , j1 ) > 0. Because the i1 -row sum of z is zero, we can find j2 such that z (i1 , j2 ) < 0. The j2 -column sum of z is zero, therefore we can now find i2 such that z (i2 , j2 ) > 0. Visiting cells in this way, we come back to a cell that was already visited. Among such “cycles,” consider a shortest one. Then the shortest one is a loop, namely, the row indices and the column indices are distinct among themselves. Taking away this loop, we have a move of smaller degree. If we apply this procedure recursively, we can express z as a conformal sum of loops. On the other hand, it is obvious that each loop cannot be written as a conformal sum of other nonzero moves. This proves the proposition. Let A be a configuration. The Lawrence lifting Λ (A) of A is the configuration ((2ν + η ) × 2η matrix) ⎞ ⎛ A 0 Λ (A) = ⎝ 0 A ⎠ , (4.23) Eη Eη where Eη is the η × η identity matrix. Note that (0 A) = (A A) − (A 0) = A(Eη Eη ) − (A 0).
62
4 Definition of Markov Bases and Other Bases
Therefore the second block (0 A) in (4.23) is redundant for defining ker Λ (A). Hence instead of Λ (A) we can also use
Λ˜ (A) =
A 0 . Eη Eη
(4.24)
However it is more convenient to retain the second block in Λ (A) for explanation. From (4.23), an element of ker Λ (A) is of the form z , −zz
where z ∈ ker A. Clearly z = z 1 + z 2 is a conformal sum of two moves for A if and only if z z1 z2 = + −zz −zz1 −zz2 is a conformal sum of two moves for Λ (A). By this observation we have the following proposition. Proposition 4.3. Let {zz1 , . . . , z L } be the Graver basis for A. The Graver basis of Λ (A) is given by zL z1 ,..., . (4.25) −zz1 −zzL From the viewpoint of statistical models, the Lawrence lifting corresponds to a logistic regression. See Christensen [37] for a detailed treatment of logistic regression. Consider the model (4.1). We make two copies I and I of the sample space I and consider a corresponding pair of cells (ii , i ). Call i a “success” of the cell i and i a “failure” of the cell i . Consider a Bernoulli random variable Yi ∈ {0, 1}, such that P(Yi = 1) = pi =
exp(∑νj=1 θ j a j (ii)) . 1 + exp(∑νj=1 θ j a j (ii))
(4.26)
We let Yi = 1 correspond to an observation in i and Yi = 0 correspond to an observation in i . For each i , observe ni independent Bernoulli random variables with the success probability (4.26). Let x(ii ) be the frequency of the cell i (i.e., the number of successes for the cell i ) and let x(ii ) = ni − x(ii ) be the number of failures. Under the logistic regression model, x(ii ) has the binomial distribution Bin(ni , pi ). We now consider the sufficient statistic for the logistic regression model. It is easily seen that a sufficient statistic for this logistic regression model, when ni s are
4.6 Graver Basis, Lawrence Lifting, and Logistic Regression
63
fixed and regarded as parameters, is a sufficient statistic for the original A computed from the number of successes x(ii ), i ∈ I . When ni s are fixed, we can alternatively compute a sufficient statistic for the original A from the number of failures x(ii ), i ∈ I . As in the discussion on three sampling schemes for 2 × 2 contingency tables in Sect. 1.1, we can also regard ni s as a part of a sufficient statistic. If we vary ni s and regard them as arbitrary nonnegative integers, then the configuration of the logistic regression is given by the Lawrence lifting (4.24). As we show in Sect. 5.4.3 the unique Markov basis of Λ (A) coincides with the Graver basis of Λ (A) and the latter is essentially the same as the Graver basis of A by Proposition 4.3. Hence if we allow arbitrary nonnegative ni s, then we need the Graver basis of A. However, it seems that many elements of the Graver basis of A are needed in order to guarantee the connectivity when some ni s are zeros. When all ni s are positive and fixed, connectivity of a particular fiber may be guaranteed by a proper subset of the Graver basis. This problem is discussed mainly in Chap. 13. In this chapter we discussed a Markov basis, a lattice basis, a Gr¨obner basis, and the Graver basis. We here summarize implications among them. By definition of these bases, the inclusion relations between them are given as follows. a lattice basis ⊂ a minimal Markov basis ⊂ a reduced Gr¨obner basis ⊂ the Graver basis.
Chapter 5
Structure of Minimal Markov Bases
5.1 Accessibility by a Set of Moves Let B = {zz1 , . . . , z L } be a finite set of moves, which may not be a Markov basis. Let x , y ( = x ) ∈ Ft . We say that y is accessible from x by B and denote this by x∼y
(mod B),
if there exists a sequence of moves z i1 , . . . , z iL from B and ε j = ±1, j = 1, . . . , L, such that y = x + ∑Lj=1 ε j z i j and h
x + ∑ ε j z i j ∈ Ft ,
h = 1, . . . , L − 1;
(5.1)
j=1
that is, we can move from x to y by moves from B without causing negative elements on the way. Obviously the notion of accessibility is symmetric and transitive: x ∼y x1 ∼ x2,
⇒
y∼x x2 ∼ x3
(mod B), ⇒
x1 ∼ x3
(mod B).
Allowing moves to be 0 also yields reflexivity. Therefore accessibility by B is an equivalence relation and each fiber Ft is partitioned into disjoint equivalence classes by moves of B. We call these equivalence classes B-equivalence classes of Ft . Because the notion of accessibility is symmetric, we also say that x and y are mutually accessible by B if x ∼ y (mod B). Let x and y be elements from two different B-equivalence classes of Ft . We say that a move z = x − y connects these two equivalence classes.
S. Aoki et al., Markov Bases in Algebraic Statistics, Springer Series in Statistics 199, DOI 10.1007/978-1-4614-3719-2 5, © Springer Science+Business Media New York 2012
65
66
5 Structure of Minimal Markov Bases
In this chapter the set of moves with degree less than or equal to n, Bn = {zz | degzz ≤ n},
(5.2)
is of particular importance. Consider Bn−1 -equivalence classes of a fiber Ft with n = degtt . Let Kt denote the number of equivalence classes and partition Ft as Ft = Ft ,1 ∪ · · · ∪ Ft ,Kt .
(5.3)
We also define Bt = {zz ∈ kerZ A | t = Azz+ = Azz− } = {zz ∈ kerZ A | z + , z − ∈ Ft }.
(5.4)
We call z ∈ Bt a move belonging to Ft . The equivalence classes Ft ,1 , . . . , Ft ,Kt in (5.3) can be understood as follows. Let x , y ∈ Ft . Suppose that there exists z with deg z ≤ n − 1, such that y = x + z = (xx − z − ) + z + . Then x − z − is not a zero vector and supp(xx − z − ) = 0. / Because supp(xx − z − ) is contained in both supp(xx) and supp(yy), we have supp(xx) ∩ supp(yy) = 0. / Conversely if supp(xx) ∩ supp(yy) = 0/ then deg(yy − x ) < n and y − x ∈ Bn−1 . We have shown that deg(yy − x ) < n ⇔ supp(xx) ∩ supp(yy) = 0. / Now if x and y are in the same Bn−1 -equivalence class Ft ,k , then there exists a sequence of states x = x 0 , x 1 , . . . , x L = y , such that deg(xx j − x j−1 ) < n, j = 1, . . . , L, or equivalently supp(xx j ) ∩ supp(xx j−1 ) = 0. / Therefore the equivalence classes in (5.3) can be understood as the connected components of the following graph G. The set of vertices of G is the fiber Ft and G has an undirected edge between x and y ∈ Ft if and only if deg(yy − x ) < n.
5.2 Structure of Minimal Markov Basis and Indispensable Moves A Markov basis B is minimal if no proper subset of B is a Markov basis. A minimal Markov basis always exists, because from any Markov basis, we can remove redundant elements one by one, until none of the remaining elements can
5.2 Structure of Minimal Markov Basis and Indispensable Moves
67
be removed any further. From this definition, a minimal Markov basis is not signinvariant in the sense that for each z ∈ B, −zz is not a member of B when B is a minimal Markov basis, because −zz can be omitted from B without affecting the connectivity. At this point, we discuss the signs of moves in a Markov basis. In discussing minimality of Markov bases, it is sometimes more convenient if both (or neither of) z and −zz belong to a Markov basis. We call a set of moves B sign-invariant if z ∈ B implies −zz ∈ B. We call a sign-invariant Markov basis minimal if no proper sign-invariant subset of B is a Markov basis. If a sign-invariant Markov basis B is minimal, then for each move z ∈ B, we can leave exactly one of z and −zz in the Markov basis and have a minimal Markov basis without the requirement of sign invariance. Conversely if B is a minimal Markov basis, then B ∪ (−B), where −B = {−zz | z ∈ B} is a minimal sign-invariant Markov basis. For each t , let n = degtt and consider the Bn−1 -equivalence classes of Ft in (5.3). Let x j ∈ Ft , j , j = 1, . . . , Kt , be representative elements of the equivalence classes and let z j1 , j2 = x j1 − x j2 , j1 = j2 be a move connecting Ft , j1 and Ft , j2 . Note that we can connect all equivalence classes with Kt − 1 moves of this type, by forming a tree, where the equivalence classes are interpreted as vertices and connecting moves are interpreted as edges of an undirected graph. Now we state the main theorem of this chapter. The following result was already known to algebraists in Theorem 2.5 of [28]. For the rest of this chapter we write |tt | for degtt . Theorem 5.1. Let B be a minimal Markov basis. For each t , B ∩ Bt consists of Kt − 1 moves connecting different B|tt |−1 -equivalence classes of Ft , in such a way that the equivalence classes are connected into a tree by these moves. Conversely choose any Kt − 1 moves zt ,1 , . . . , zt ,Kt −1 connecting different B|tt |−1 equivalence classes of Ft , in such a way that the equivalence classes are connected into a tree by these moves. Then B=
{zt ,1 , . . . , zt ,Kt −1 }
(5.5)
t :Kt ≥2
is a minimal Markov basis. Proofs of this theorem and the following corollaries are given at the end of this section. Note that no move is needed from Ft with Kt = 1, including the case where Ft is a one-element set. If Ft = {xx} is a one-element set, no nonzero move is applicable to x , but at the same time we do not need to move from x at all for such an Ft . In principle this theorem can be used to construct a minimal Markov basis from below as follows. As the initial step we consider t with the sample size n = |tt | = 1. Because B0 consists only of the zero move B0 = {00}, each point x ∈ Ft , |tt | = 1, is isolated and forms an equivalence class by itself. For each t with |tt | = 1, we choose
68
5 Structure of Minimal Markov Bases
Kt − 1 degree 1 moves to connect Kt points of Ft into a tree. Let B˜ 1 be the set of chosen moves. B˜ 1 is a subset of the set B1 of all degree 1 moves. Every degree 1 move can be expressed by nonnegative integer combination of chosen degree 1 moves, thus it follows that B˜ 1 and B1 induce the same equivalence classes for each Ft with |tt | = 2. Therefore as the second step we consider B˜ 1 -equivalence classes of Ft for each t with |tt | = 2 and choose representative elements from each equivalence class to form degree 2 moves connecting the equivalence classes into a tree. We add the chosen moves to B˜ 1 and form a set B˜ 2 . We can repeat this process for n = |tt | = 3, 4, . . .. By the Hilbert basis theorem there exists some n0 such that for n ≥ n0 no new moves need to be added. Then a minimal Markov basis B of (5.5) is written as B = B˜ n0 . The difficulty with this approach is that the known theoretical upper bound for n0 in Proposition 5.3 below is large. Theorem 5.1 clarifies to what extent the minimal Markov basis is unique. If an equivalence class consists of more than one element, then any element can be chosen as the representative element of the equivalence class. Another indeterminacy is how to form a tree of the equivalence classes. In addition there exists a trivial indeterminacy of a Markov basis B in changing the signs of its elements. We say that a minimal basis B is unique if all minimal bases differ only by sign changes of the elements; that is, if B ∪ (−B) is the unique minimal sign-invariant Markov basis. In terms of binomials, a unique minimal Markov basis corresponds to a unique minimal system of binomial generators of a toric ideal. In view of Lemma 5.3 below, we have the following corollary to Theorem 5.1. Corollary 5.1. A minimal Markov basis is unique if and only if for each t , Ft itself constitutes one B|tt |−1 -equivalence class or Ft is a two-element set. In this corollary the importance of a two-element set Ft = {xx, y } is suggested. Therefore we make the following definition. Definition 5.1. A move z = y − x is indispensable if Ft = {xx, y } is a twoelement set. In this definition we are not assuming that the supports of x and y are disjoint. However min(xx, y ) is canceled in z = y − x . We also call a binomial + − uz − u z indispensable if z = z+ − z− is an indispensable move. The notion of the indispensable move was given in Takemura and Aoki [142]. Ohsugi and Hibi [110] proved some properties of indispensable moves, in particular for configurations arising from finite graphs. In Theorem 2.4 of [111], Ohsugi and Hibi showed that the set of indispensable binomials is characterized as the intersection of binomials in the reduced Gr¨obner bases with respect to all lexicographic term orders. Using the notion of indispensability, we state another corollary, which is more convenient to use. Corollary 5.2. The unique minimal Markov basis exists if and only if the set of indispensable moves forms a Markov basis. In this case, the set of indispensable moves is the unique minimal Markov basis.
5.2 Structure of Minimal Markov Basis and Indispensable Moves
69
From these corollaries it seems that the minimal Markov basis is unique only under special conditions. It is therefore of great interest whether the minimal Markov basis is unique for some standard problems in m-way (m ≥ 2) contingency tables with fixed marginals. On the other hand for the simplest case of oneway contingency tables, the minimal Markov basis is not unique. These facts are confirmed in Sect. 5.4, Chaps. 8 and 9. For the rest of this section we give a proof of Theorem 5.1 and its corollaries. We also state some lemmas, which are of some independent interest. Lemma 5.1. If a move z is applicable to at least one element of Ft , then deg z ≤ |tt |,
(5.6)
where the equality holds if and only if t = Azz+ = Azz− . Proof. Let z be applicable to x ∈ Ft . Then by (4.12), x(ii) ≥ z− (ii), ∀ii ∈ I . Summing over I yields (5.6). Concerning the equality, if z is applicable to x ∈ Ft and the equality holds in (5.6), then x(ii ) = z− (ii), ∀ii ∈ I and t = Axx =
∑ a(ii)x(ii) = ∑ a(ii)z− (ii) = Azz− .
i ∈I
i ∈I
Conversely if t = Azz+ = Azz− , then deg z = |tt | by the definition of degzz and |tt |.
Lemma 5.1 implies that in considering mutual accessibility between x , y ∈ Ft , we only need to consider moves of degree smaller than |tt | or moves z with t = Azz+ = Azz− . Lemma 5.1 also implies the following simple but useful fact. Lemma 5.2. Suppose that Ft = {xx, y } is a two-element set and suppose that the supports of x and y are disjoint. Then Kt = 2 and x , y are B|tt |−1 -equivalence classes by themselves. Furthermore z = y − x belongs to each Markov basis. Proof. Suppose that y is accessible from x by B|tt |−1 . Then there exists a nonzero move z with deg z ≤ |tt | − 1 such that z is applicable to x . If x + z = y , then z = y − x and deg z = |tt |, because the supports of x and y are disjoint. Therefore x + z = y and Ft contains a third element x + z , which is a contradiction. Therefore y and x are in different B|tt |−1 -equivalence classes, implying that y and x are B|tt |−1 -equivalence classes by themselves. Now consider moving from x to y . Because they are B|tt |−1 -equivalence classes by themselves, no nonzero move z of degree deg z < |tt | is applicable to x . By Lemma 5.1, only moves z with t = Azz+ = Azz− are applicable to x . If any such move is different from y − x , then as above Ft contains a third element. It follows that in order to move from x to y , we have to move by exactly one step using the move z = y − x . Therefore z has to belong to any Markov basis.
Lemma 5.2 can be slightly modified to yield the following result for the case where supports of x and y are not necessarily disjoint.
70
5 Structure of Minimal Markov Bases
Lemma 5.3. Suppose that Ft = {xx, y } is a two-element set. Then z = y − x belongs to each Markov basis. Proof. If the supports of x and y are disjoint, then the result is already contained in Lemma 5.2. Otherwise let v = min(xx, y ) and consider y − v and x − v . Then the supports of y − v and x − v are disjoint and by Lemma 5.2 again z = (yy − v) − (xx − v ) = y − x belongs to each Markov basis.
The following lemma concerns replacing a move by series of moves. Lemma 5.4. Let B be a set of moves and let z 0 ∈ B be another nonzero move. − Assume that z + is accessible from z by B. Then for each x , to which z 0 is 0 0 applicable, x + z 0 is accessible from x by B. − This lemma shows that if z + 0 is accessible from z 0 by B, then we can always replace z 0 by a series of moves from B.
Proof. Suppose that z 0 is applicable to x . Then we assume x ≥ z − 0 without loss of + generality. By the definition of accessibility (cf. (5.1)), we can move from z − 0 to z 0 by moves from B without causing negative elements on the way. Then the same sequence of moves can be applied to x without causing negative elements on the way, leading from x to x + z 0 .
Now we are ready to prove Theorem 5.1 and its corollaries. Proof (Theorem 5.1). Let B be a minimal Markov basis. For each z ∈ Bn \ (B ∩ Bn ), z + is accessible from z − by B ∩ Bn , because no move of degree greater than n is applicable to z + as stated in Lemma 5.1. Considering this fact and Lemma 5.4, it follows that Bn and B ∩ Bn induce the same equivalence classes in Ft , |tt | = n + 1. Fix a particular t . Write {zz1 , . . . , z L } = B ∩ Bt , where Bt is the set of moves belonging to Ft defined in (5.4). For any j = 1, . . . , L, let x = z +j , y = z −j . If x and y are in the same B|tt |−1 -equivalence class, then by Lemma 5.4, z j can be replaced by a series of moves of lower degree from B and B \ {zz j } remains to be a Markov basis. This contradicts the minimality of B. Therefore z +j and z −j are in two different B|tt |−1 -equivalence classes connecting them. Now we consider an undirected graph whose vertices are B|tt |−1 -equivalence classes of Ft and whose edges are moves z 1 , . . . , z L . Considering that B is a Markov basis, and no move of degree greater than |tt | is applicable to each element of Ft as stated in Lemma 5.1, this graph is connected. On the other hand if the graph contains a cycle, then there
5.3 Minimum Fiber Markov Basis
71
exist z j , such that z +j and z −j are mutually accessible by B \ {zz j }. By Lemma 5.4 again, this contradicts the minimality of B. It follows that the graph is a tree. Because any tree with Kt vertices has Kt − 1 edges, L = Kt − 1. Reversing the above argument, it is now easy to see that if Kt − 1 moves zt ,1 , . . . , zt ,Kt −1 connecting different B|tt |−1 -equivalence classes of Ft are chosen in such a way that the equivalence classes are connected into a tree by these moves, then B=
{zt ,1 , . . . , zt ,Kt −1 }
t :Kt ≥2
is a minimal Markov basis.
Proof (Corollary 5.1). From our argument preceding Corollary 5.1, it follows that if the minimal Markov basis is unique, then for each t , Ft itself constitutes one B|tt |−1 -equivalence class or Ft is a two-element set {xxt ,1 , xt ,2 }, such that xt ,1 ∼ xt ,2 (mod B|tt |−1 ). Therefore we only need to prove the converse. Suppose that for each t , Ft itself constitutes one B|tt |−1 -equivalence class or Ft is a two-element set. By Lemma 5.3, for each two-element set Ft = {xx, y }, the move z = y − x belongs to each Markov basis. However, by Theorem 5.1 each minimal Markov basis consists only of these moves. Therefore a minimal Markov basis is unique.
Proof (Corollary 5.2). By Lemma 5.3, indispensable moves belong to each Markov basis. Therefore if the set of indispensable moves forms a Markov basis, then it is the unique minimal Markov basis. On the other hand if the set of indispensable moves does not constitute a Markov basis, then there is a term with Kt ≥ 3 in (5.5) and in this case a minimal Markov basis B is not unique as discussed after Theorem 5.1. From these considerations it is obvious that if the unique minimal Markov basis exists, it coincides with the set of indispensable moves.
5.3 Minimum Fiber Markov Basis In this section we discuss the union of all minimal Markov bases and define the minimum fiber Markov basis. Let z = z + − z − be a move of degree n. We call z nonreplaceable by lower degree moves if z+ ∼ z−
(mod Bn−1 ),
(5.7)
that is, if z connects different Bn−1 -equivalence classes of Ft z + . Clearly an indispensable move is nonreplaceable by lower degree moves. Let BMF = {zz | z is nonreplaceable by lower degree moves}.
(5.8)
72
5 Structure of Minimal Markov Bases
We call BMF the minimum fiber Markov basis. In actuality, we have the following fact. Proposition 5.1. BMF is the union of all minimal Markov bases. Proof. From the argument of the previous section, for all minimal Markov bases B, the set of sufficient statistics {tt | t = Azz+ = Azz− , z ∈ B} = {tt | Ft is not a single B|tt |−1 -equivalence class} is common and it is equal to the set of fibers of the moves in BMF . A minimal Markov basis is constructed by arbitrarily choosing moves z connecting different B|tt |−1 -equivalence classes of Ft into a tree. Because BMF is the union of all these moves, it is the union of all minimal Markov bases.
By construction the minimum fiber Markov basis is invariant in the sense of Chap. 7.
5.4 Examples of Minimal Markov Bases 5.4.1 One-Way Contingency Tables We start with the simplest case of one-way contingency tables. Let x = (xi ) be an I-dimensional frequency vector and A = (1, . . . , 1) = 1 I . In this case, t is the sample size n. This situation corresponds to testing the homogeneity of mean parameters for I independent Poisson variables conditional on the total sample size n. In this case, a minimal Markov basis is formed as a set of I − 1 degree 1 moves, but is not unique. A minimal Markov basis is constructed as follows. First consider the case of n = |tt | = 1. There are I elements in Ft as Ft = {ee1 , . . . , e I }. Each element x ∈ Ft forms an equivalence class by itself. To connect these points into a tree, there are I I−2 ways of choosing I − 1 degree 1 moves by Cayley’s theorem (see, e.g., Chap. 4 of Wilson [149]). One example is B = {ee1 − e2 , e 2 − e 3 , . . . , e I−1 − e I }. It is easily verified that no move of degree larger than 1 is needed.
5.4 Examples of Minimal Markov Bases
73
5.4.2 Independence Model of Two-Way Contingency Tables The next example is the independence model of two-way contingency tables discussed in Chaps. 1 and 2. In Theorem 2.1 we have shown that the set of degree 2 moves displayed as +1 −1 −1 +1
(5.9)
is a Markov basis. In addition, this is the unique minimal Markov basis. Indeed, consider marginal frequencies, where the ith row sum, the i th row sum, the jth column sum, and the j th column sum are ones and all other marginal frequencies are zeros. Then the relevant marginal frequencies are displayed as follows. j j i 1 . i 1 1 1 2 Clearly there are only two elements of this fiber: 10 , 01
01 . 10
The move in (5.9) is the difference of these two elements and hence it is indispensable.
5.4.3 The Unique Minimal Markov Basis for the Lawrence Lifting One remarkable example of the existence of the unique minimal Markov basis is the Lawrence lifting Λ (A) in Sect. 4.6. In Proposition 4.3 we gave the Graver basis of Λ (A). We now show that it is actually the unique minimal Markov basis of Λ (A), by showing that each move in (4.25) is indispensable. This also shows that the Graver basis for the configuration A is finite, because a minimal Markov basis for Λ (A) is finite. Proposition 5.2. For the Lawrence lifting Λ (A), the Graver basis given by (4.25) is the unique minimal Markov basis Proof. Let z = z + − z − be a conformally primitive move for A and let t = Azz+ . Consider the fiber of z −zz
74
5 Structure of Minimal Markov Bases
for Λ (A). Note that + + z z = − , z −zz
− − z z = + . z −zz
Then ⎞ + − t z ⎠ = Λ (A) z ⎝ t Λ (A) = , − z z+ + − z +z ⎛
⎛
⎞ A 0 where Λ (A) = ⎝ 0 A ⎠ . Eη Eη
Now suppose that ⎞ ⎛ ⎞ t Axx x ⎠ = Λ (A) ⎝ t = ⎝ Ayy ⎠ . y z + + z− x+y ⎛
/ Then z + , z − , x , y ∈ Ft for the configuration A. Note that supp(zz+ ) ∩ supp(zz− ) = 0. Now decomposing x , y into these disjoint supports write x = x 1 + x2 ,
y = y 1 + y2 ,
s.t. supp(xx1 ), supp(yy1 ) ⊂ supp(zz + ), supp(xx2 ), supp(yy2 ) ⊂ supp(zz − ).
Then by z + + z − = x + y we have z + = x 1 + y1 ,
z − = x 2 + y2 .
This implies t = Axx1 + Ayy1 = Axx2 + Ayy2 . On the other hand t = Axx = Axx1 + Axx2 . Similarly t = Ayy1 + Ayy2 . Hence 0 = (Axx1 + Ayy1 ) − (Axx1 + Axx2 ) = A(yy1 − x2 ). Similarly 0 = A(xx1 − y 2 ). Then z = z + − z − = (xx1 + y 1 ) − (xx2 + y 2 ) = (yy1 − x 2 ) + (xx1 − y 2 ), which is a conformal sum. By primitiveness of z , either y 1 − x 2 = 0 or x 1 − y2 = 0. It follows that + −
z z , + z− z is a two-element fiber for Λ (A).
5.5 Indispensable Monomials
75
Concerning the finiteness of the Graver basis, we cite the following important theoretical upper bound to the highest degree of elements in the Graver basis. In Definition 3.3 the toric ideal is defined in terms of kerZ A. When we remove linearly dependent rows from the configuration A, kerZ A does not change. Hence we can assume without loss of generality that the rows of A : ν × η are linearly independent. Proposition 5.3 (Corollary 4.15 of Sturmfels [139]). The degree of primitive moves for the configuration A with linearly independent rows is bounded from above by 1 (ν + 1)(η − ν )D(A), 2 where D(A) = max{|det(aai1 , . . . , a iν )| | 1 ≤ i1 < · · · < iν ≤ η } is the maximum of the absolute value of the determinants of ν × ν submatrices of A. For the case that the semigroup generated by the columns of A is normal, the following much better upper bound on Markov bases is known. Proposition 5.4 (Theorem 13.14 of Sturmfels [139]). Let A : ν × η be a configuration such that the semigroup generated by columns of A is normal. Then the toric ideal IA is generated by binomials of degree at most ν .
5.5 Indispensable Monomials Extending the notion of indispensable moves, in this section we define indispensable monomials of a toric ideal and establish some of their properties. Indispensable monomials were introduced in [19]. They are useful for searching for indispensable binomials of a toric ideal and for proving the existence or nonexistence of a unique minimal system of binomial generators of a toric ideal. In this section we identify a frequency vector x with a monomial ux and use two notations interchangeably. First we define an indispensable monomial. Hereafter, we say that a Markov basis B contains x if it contains a move z containing x (i.e., z+ = x or z − = x holds) by abusing the terminology. Definition 5.2. A monomial u x is indispensable if every system of binomial generators of IA contains a binomial f such that u x is a term of f . We also call a frequency vector x indispensable if u x is an indispensable monomial. From this definition, any Markov basis contains all indispensable monomials. Therefore the set of indispensable monomials is finite. Note that both terms of an + − indispensable binomial u z − u z are indispensable monomials, but the converse does not hold in general.
76
5 Structure of Minimal Markov Bases
Now we present an alternative definition. Definition 5.3. A frequency vector x is a minimal multielement if |FAxx | ≥ 2 and |FA(xx−eei ) | = 1 for every i ∈ supp(xx). Here x − e i is the frequency vector obtained by subtracting one frequency from x in the cell i . Proposition 5.5. x is an indispensable monomial if and only if x is a minimal multielement. Proof. First, we suppose that x is a minimal multielement and want to show that it is an indispensable monomial. Let n = deg x and consider the fiber FAxx . We claim that {xx} forms a single Bn−1 -equivalence class. In order to show this, we argue by contradiction. If {xx} does not form a single Bn−1 -equivalence class, then there exists a move z with degree less than or equal to n − 1, such that x + z = (xx − z − ) + z + ∈ FAxx . Inasmuch as deg x = n, deg z ≤ n − 1, we have 0 = x − z − and supp(xx) ∩ supp(xx + z ) = 0. / Therefore we can choose i ∈ supp(xx ) ∩ supp(xx + z ) such that A(xx − e i ) = A(xx + z − e i ),
x − ei = x + z − ei .
This shows that |FA(xx−eei ) | ≥ 2, which contradicts the assumption that x is a minimal multielement. Therefore we have shown that {xx} forms a single Bn−1 -equivalence class. Because we are assuming that |FAxx | ≥ 2, there exists some other Bn−1 -equivalence class in FAxx . By Theorem 5.1 each Markov basis has to contain a move connecting {xx} to another equivalence class of FAxx , which implies that each Markov basis has to contain a move z containing x . We now have shown that each minimal multielement has to be contained in each Markov basis; that is, a minimal multielement is an indispensable monomial. Now we show the converse. It suffices to show that if x is not a minimal multielement, then x is a dispensable monomial. Suppose that x is not a minimal multielement. If x is a 1-element (|FAxx | = 1), obviously it is dispensable. Hence assume |FAxx | ≥ 2. In the case that FAxx is a single Bn−1 -equivalence class, no move containing x is needed in a minimal Markov basis by Theorem 5.1. Therefore we only need to consider the case that FAxx contains more than one Bn−1 -equivalence class. Because x is not a minimal multielement, there exists some i ∈ supp(xx) such that |FA(xx−eei ) | ≥ 2. Then there exists y = x − e i , such that Ayy = A(xx − ei ). Noting that deg y = deg(xx − e i ) = n − 1, a move of the form z = y − (xx − e i ) = (yy + e i ) − x
5.5 Indispensable Monomials
77
satisfies 0 < deg z ≤ n − 1. Then y + ei = x + z = and x and y + e i belong to the same Bn−1 -equivalence class of FAxx . Because x y + e i , Theorem 5.1 states that we can construct a minimal Markov basis containing y + ei , but not containing x . Therefore x is a dispensable monomial.
We give yet another definition. Definition 5.4. x is a minimal i -lacking 1-element if |FAxx | = 1, |FA(xx+eei ) | ≥ 2, and |FA(xx+eei −ee j ) | = 1 for each j ∈ supp(xx). We then have the following result. Proposition 5.6. The following three conditions are equivalent: (1) x is an indispensable monomial, (2) for each i ∈ supp(xx), x −eei is a minimal i -lacking 1-element, (3) for some i ∈ supp(xx), x − e i is a minimal i -lacking 1-element. By the previous theorem we can replace the condition (1) by the condition that x is a minimal multielement. Proof. (1) ⇒ (2). Suppose that x is a minimal multielement. Then for any i ∈ supp(xx), x − e i is a 1-element and |FA((xx−eei )+eei ) | = |FAxx | ≥ 2. If x − e i is not a minimal i -lacking 1-element, then for some j ∈ supp(xx − e i ), |FA(xx−ee j ) | ≥ 2. However, j ∈ supp(xx − e i ) ⊂ supp(xx) and |FA(xx−ee j ) | ≥ 2 contradicts the assumption that x is a minimal multielement. It is obvious that (2) ⇒ (3). Finally we prove (3) ⇒ (1). Suppose that for some i ∈ supp(xx), x − e i is a minimal i -lacking 1-element. Note that |FAxx | = |FA((xx−eei )+eei ) | ≥ 2. Now consider j ∈ supp(xx). If j ∈ supp(xx − e i ) then |FA(xx−ee j ) | = |FA((xx−eei )+eei −ee j ) | = 1. On the other hand if j ∈ supp(xx − e i ), then j = i because j ∈ supp(xx). In this case |FA(xx−ee j ) | = 1. This shows that x is a minimal multielement.
Proposition 5.6 suggests the following: Find any 1-element x . It is often the case that each e i , i = 1, . . . , η , is a 1-element. Randomly choose 1 ≤ i ≤ η and check whether x + e i remains to be a 1-element. Once |Fx +eei | ≥ 2, then subtract e j s, j = i , one by one from x such that it becomes a minimal i -lacking 1-element. We can apply this procedure for finding indispensable monomials of some actual statistical problem. We illustrate this procedure with an example of a 2 × 2 × 2 contingency table. Consider the following problem where η = 8, ν = 4, and A is given as ⎛
11 ⎜1 1 A=⎜ ⎝1 1 10
11 11 00 10
11 00 11 10
⎞ 11 0 0⎟ ⎟. 0 0⎠ 10
78
5 Structure of Minimal Markov Bases
We write indeterminates as u = (u111 , u112 , u121 , u122 , u211 , u212 , u221 , u222 ). To find indispensable monomials for this problem, we start with the monomial u x = u111 and consider x + e i , i ∈ I . Then we see that • u2111 , u111 u112 , u111 u121 , u111 u211 are 1-element monomials • u111 u122 , u111 u212 , u111 u221 are 2-element monomials • u111 u222 is a 4-element monomial. From this, we found four indispensable monomials, u111 u122 , u111 u212 , u111 u221, and u111 u222, because each of u122 , u212 , u221 , u222 is a 1-element monomial. Starting from the other monomials, similarly, we can find the following list of indispensable monomials: • u111 u122 , u111 u212 , u111 u221 , u112 u121 , u112 u211, u112 u222 , u121 u222 , u121 u211 , u122 u221 , u122 u212 , u211 u222 , u212 u221 , each of which is a 2-element monomial. • u111 u222 , u112 u221 , u121 u212 , u122 u211 , each of which is a 4-element monomial. The next step is to consider the newly produced 1-element monomials, u2111 , u111 u112, u111 u121, u111 u211 , and so on. For each of these monomials, consider adding e i , i ∈ I one by one, checking whether they are multielement. For example, we see that the monomials such as u3111 , u2111 u112 , u111 u2112 , . . . are again 1-element monomials (and we have to consider these 1-element monomials in the next step). On the other hand, monomials such as u2111 u122 , u111 u112 u122 , u2111 u222 , u111 u112u221 , . . . are multielement monomials. However, it is seen that they are not minimal multielement, inasmuch as u111 u122 , u112 u122 , u111 u222 , u112 u221, . . . are not 1-element monomials. To find all indispensable monomials for this problem, we have to repeat the above procedure for monomials of degree 4, 5, . . .. Indispensable monomials belong to any Markov basis, in particular to the Graver basis, therefore Proposition 5.3 again gives an upper bound for the degree of indispensable monomials and we can stop at this bound. We mention that analogous to Theorem 2.4 in [111], the set of indispensable monomials is characterized as the intersection of monomials in reduced Gr¨obner bases with respect to all lexicographic term orders. Further characterizations of indispensable binomials and indispensable monomials are given in [31, 63, 115].
Chapter 6
Method of Distance Reduction
6.1 Distance Reducing Markov Bases Throughout this book, we use the method of distance reduction (or a distancereducing argument) of Takemura and Aoki [143] for finding a Markov basis for a given configuration. We have already seen a typical example in Sect. 2.1 for proving that the set of basic moves in (2.4) forms a Markov basis for the independence model of I × J contingency tables. In this section we first formalize the idea of distance reduction by a set of moves. Consider a metric d(xx, y ) on a fiber Ft . Although we are mainly concerned with the 1-norm in the following, here we consider a general metric. A metric d = dt on Ft can be defined in various ways. If a metric d is defined on the whole space of frequency vectors X = N|I | , we can consider the restriction of d to each Ft dt (xx, y ) = d(xx, y ),
x , y ∈ Ft .
If d(·) is a norm on the set Z|I | of integer vectors, such as the 1-norm |zz| = ∑i ∈I |z(ii)|, dt is defined by dt (xx, y ) = d(xx − y ). For notational simplicity we suppress the subscript t in dt hereafter. Now we introduce the notion of a distance reduction by a set of moves. Let B be a set of moves. We call B d-reducing for x , y ∈ Ft if there exists an element z ∈ B and ε = ±1 such that ε z is applicable to x or y and we can decrease the distance; that is, x + ε z ∈ Ft
and d(xx + ε z , y ) < d(xx, y ),
y + ε z ∈ Ft
and d(xx, y + ε z ) < d(xx, y ).
or (6.1)
We simply call B d-reducing if B is d-reducing for every x , y ( = x ) ∈ Ft and for every t . Alternatively we say that B is norm-reducing if it is clear which metric d is used in the context. S. Aoki et al., Markov Bases in Algebraic Statistics, Springer Series in Statistics 199, DOI 10.1007/978-1-4614-3719-2 6, © Springer Science+Business Media New York 2012
79
80
6 Method of Distance Reduction
We call B strongly d-reducing for x , y ∈ Ft if there exist elements z 1 , z 2 ∈ B and ε1 , ε2 = ±1 such that x + ε1 z 1 ∈ Ft , y + ε2 z 2 ∈ Ft and d(xx + ε1 z 1 , y ) < d(xx, y ) and d(xx, y + ε2 z 2 ) < d(xx, y ).
(6.2)
We call B strongly d-reducing if B is strongly d-reducing for every x , y ( = x ) ∈ Ft and for every t . Clearly if B is strongly d-reducing, then B is d-reducing. The following fact on d-reducing set of moves B is obvious, but very useful. Proposition 6.1. Let a metric d be given on each fiber Ft . A set of finite moves B is a Markov basis if it is d-reducing. Instead of a formal proof, we give the following argument on how two states are connected by a set of moves from B. If B is d-reducing, then for every x , y ( = x ) ∈ Ft , there exist k > 0, εl = ±1, z l ∈ B, x l ∈ Ft , y l ∈ Ft , l = 1, . . . , k, with the following properties. (i) x k = y k . (ii) d(xxl , y l ) < d(xxl−1 , y l−1 ), l = 1, . . . , k, where x 0 = x and y 0 = y . (iii) (xxl , y l ) = (xxl−1 + εl z l , y l−1 ) or (xxl , y l ) = (xxl−1 , y l−1 + εl z l ), l = 1, . . . , k. Given the above sequence of frequency vectors, we can move from x to x k = y k and then reversing the moves we can move from y k to y . Thus y is accessible from x by B. Note that in this sequence of moves the distances d(xx, x 1 ), . . . , d(xx, x k ), d(xx , y k−1 ) . . . , d(xx, y ) might not be monotone increasing. On the other hand, if B is strongly d-reducing, then starting from y , we can always decrease the distance by moving from the side of y ; that is, we can find k > 0 and y = y 0 , y 1 , . . . , y k−1 , y k = x in Ft such that y l = y l−1 + εl z l , εl = ±1, z l ∈ B, l = 1, . . . , k, and d(xx, y k−1 ), d(xx, y k−2 ), . . . , d(xx, y ) are strictly increasing. Note that a Markov basis is not necessarily d-reducing. By a Markov basis, we can connect any two states x , y in the same fiber as x = x 0 → x 1 → · · · → x k−1 → x k = y by the moves in a Markov basis. Hence by moving from x to y , we can eventually decrease the distance between x and y to 0. However, in difficult cases, we might need to make a detour, so that initially the distance increases or stays the same as d(xx, y ) ≤ d(xx1 , y )
and
d(xx, y ) ≤ d(xx, x k−1 ).
6.2 Examples of Distance-Reducing Proofs
81
6.2 Examples of Distance-Reducing Proofs In this section we give two examples of distance-reducing arguments. The first example is a minimal Markov basis for a complete independence model of threeway contingency tables. The second example is a minimal Markov basis for the Hardy–Weinberg model. These examples are treated again in Sect. 7.2 from the viewpoint of symmetry of Markov bases.
6.2.1 The Complete Independence Model of Three-Way Contingency Tables Consider I × J × K contingency tables. Under the complete independence model the probability of the cell (i, j, k) is written as pi jk = pi++ p+ j+ p++k , where pi++ , p+ j+ , p++k denote one-dimensional marginal probabilities. With lexicographic ordering of indices, the configuration A for the complete independence model of three-way tables is written as ⎛ ⎞ 1 I ⊗ 1J ⊗ EK A = ⎝1 I ⊗ EJ ⊗ 1 K ⎠ . EI ⊗ 1J ⊗ 1 K A sufficient statistic consists of one-dimensional marginal frequencies. In this case, we construct a minimal Markov basis as follows. There are two obvious patterns of moves of degree 2. An example of moves of type I is z111 = z222 = 1, z211 = z122 = −1, with the other elements being 0. For the case of a 2 × 2 × 2 table, this move can be displayed as follows. +1 0 −1 0 . 0 −1 0 +1 All the other moves of type I are obtained by permutation of indices or axes. An example of moves of type II is z111 = z122 = 1,
z112 = z121 = −1,
with the other elements being 0. For the case of a 2 × 2 × 2 table, this move can be displayed as follows.
82
6 Method of Distance Reduction
+1 −1 −1 +1
00 . 00
All the other moves of type II are obtained by permutation of indices or axes. Let B ∗ be the set of type I and type II degree 2 moves. Then we have the following proposition. Proposition 6.2. B ∗ is a Markov basis for three-way contingency tables with fixed one-dimensional marginals. Proof. In this problem it is obvious that no degree 1 move is applicable to any frequency vector. Furthermore it is easy to verify that every degree 2 move is either of type I or type II. It remains to verify that for degtt ≥ 3, Ft itself constitutes one B ∗ -equivalence class. Suppose that for some t , Ft consists of more than one B ∗ equivalence class. Let F1 , F2 denote two different B ∗ -equivalence classes. Choose x ∈ F1 , y ∈ F2 such that |zz| = |yy − x | =
∑ |yi jk − xi jk |
i, j,k
is minimized. Because x and y are chosen from different B ∗ -equivalence classes, this minimum has to be positive. In the following we let z111 > 0 without loss of generality. Case 1. Suppose that there exists a negative cell zi0 11 < 0, i0 ≥ 2. Then because ∑ j,k zi0 jk = 0, there exists ( j, k), j + k > 2, with zi0 jk > 0. Then the four cells (1, 1, 1), (i0 , 1, 1), (i0 , j, k), (1, j, k) are in the positions of either a type I move or a type II move. In either case we can apply a type I move or a type II move to x or y and make |zz| = |yy − x | smaller, which is a contradiction. This argument shows that z cannot contain both positive and negative elements in any one-dimensional slice. Case 2. Now we consider the remaining case, where no one-dimensional slice of z contains both positive and negative elements. Because ∑ j,k z1 jk = 0, there exists ( j1 , k1 ), j1 , k1 ≥ 2, such that z1 j1 k1 < 0. Similarly there exists (i1 , k2 ), i1 , k2 ≥ 2, such that zi1 1k2 < 0. Then the four cells (1, j1 , k1 ), (1, 1, k1 ), (i1 , 1, k2 ), (i1 , j1 , k2 ) are in the positions of a type II move (if k1 = k2 ) or a type I move (if k1 = k2 ) and we can apply a degree 2 move. By doing this, |zz| = |yy − x | may remain the same, but now z11k1 becomes negative and this case reduces to Case 1. Therefore Case 2 itself is a contradiction.
We show in the following that B ∗ is not a minimal Markov basis. Let z be a degree 2 move and let t = Azz+ . If z is a type II move, it is easy to verify that Ft is
6.2 Examples of Distance-Reducing Proofs
83
a two-element set {zz+ , z − }. Therefore degree 2 moves of type II are indispensable. On the other hand, if z is a type I move, Ft is a four-element set. For the 2 × 2 × 2 case, let t = (z1++ , z2++ , z+1+ , z+2+ , z++1 , z++2 ) = (1, 1, 1, 1, 1, 1). Then we have 10 00 01 00 00 01 00 10 F(1,1,1,1,1,1) = , , , . 00 01 00 10 10 00 01 00 To connect these elements to a tree, only three moves of type I are needed. In the 2 × 2 × 2 case, there are 44−2 = 16 possibilities, such as
or
+1 −1 0 0
+1 −1 0 0
0 0 −1 +1
0 +1 −1 0
,
0 0 −1 +1
,
0 −1 +1 0
+1 0 −1 0
0 −1 0 +1
0 0 +1 −1
,
,
+1 0 0 −1
−1 +1 0 0
−1 0 0 +1
and so on. From these considerations, a minimal Markov basis for I × J × K tables consists of I J K 3 2 2 2 degree 2 moves of type I and J K I K I J I +J +K 2 2 2 2 2 2 degree 2 moves of type II.
6.2.2 Hardy–Weinberg Model We next discuss the Hardy–Weinberg model. It is a standard model in population genetics. Consider a multiallele locus with alleles A1 , A2 , . . . , AI . The allele frequency data are usually given as the genotype frequency. The probability of the genotype Ai A j in an individual from a random breeding population is given by P(Ai A j ) =
q2i
(i = j)
2qi q j
(i = j),
where qi is the proportion of the allele Ai , i = 1, . . . , I. Because the Hardy–Weinberg law plays an important role in the field of population genetics and often serves as a basis for genetic inference, much attention has been paid to tests of the hypothesis
84
6 Method of Distance Reduction
that a population being sampled is in the Hardy–Weinberg equilibrium against the hypothesis that disturbing forces cause some deviation from the Hardy–Weinberg ratio. See [43, 67, 90]. For the Hardy–Weinberg model, the frequency vector is written as x = (x11 , x12 , . . . , x1I , x22 , x23 , . . . , x2I , x33 , . . . , xII ) . If the frequencies are written in a matrix, only the upper triangular part has the frequencies. A sufficient statistic t = (t1 , . . . ,tI ) is the frequencies of the alleles Ai , i = 1, . . . , I, and is given as ti = 2xii + ∑ xi j , j =i
i = 1, . . . , I,
where we write xi j = x ji for i > j. The configuration A is an I × I(I + 1)/2 matrix. In terms of the standard basis vectors, the columns of A are written as 2eei , i = 1, . . . , I and e i + e j , 1 ≤ i < j ≤ I. Guo and Thompson [67] constructed a connected Markov chain over any fiber. Their basis consists of three types of degree 2 moves, namely, type 0, type 1, and type 2. Here “type” refers to the number of nonzero diagonal cells in the move. The examples of the moves are displayed as 0 +1 −1 0 0 0 −1 type 0: , 0 +1 0
+1 −1 −1 0 0 +1 0 type 1: , 0 0 0
+1 −2 0 0 +1 0 0 type 2: . 00 0
By the distance-reducing argument we first show that these moves form a Markov basis. Proposition 6.3. The above three types of moves form a Markov basis for the Hardy–Weinberg model. Proof. Suppose that x and y (yy = x ) are in the same fiber. First consider the case that xii = yii , i = 1, . . . , I. We look at the type 0 move above. Because y = x , there exist some i < j such that xi j > yi j . By relabeling the levels we can assume that i = 1, j = 2. Then because 2x11 + ∑Ij=2 x1 j = 2y11 + ∑Ij=2 y1 j , there exists some j > 2 such that x1 j < y1 j . We can again assume that j = 3. Also because x12 + 2x22 + ∑Ij=3 x2 j = y12 + 2y22 + ∑Ij=3 y2 j , there exists some j > 2 such that x2 j < y2 j . If j = 3, then we can add a type 0 move to y and reduce the distance |yy − x |. In the case j = 3, we can then find j > 3 such that x3 j > y3 j . We can put j = 4. In this case we can subtract a type 0 move from x and reduce the distance |yy − x |. Now consider the case that there exists some i, such that xii > yii . We can assume i = 1. If there are 1 < j < j such that x1 j < y1 j , x1 j < y1 j , then letting j = 2, j = 3,
6.3 Graver Basis and 1-Norm Reducing Markov Bases
85
we can add a type 1 move to y and reduce |yy − x|. On the other hand, if there is only one j > 1 satisfying x1 j ≤ y1 j , then y1 j ≥ 2 holds and we can add a type 2 move to y and reduce |yy − x |.
We now show the above basis is not minimal and a minimal basis is not unique. Consider Ft with degtt = 2 for the above three types of moves. If t = Azz+ = Azz− for moves z of type 1 or type 2, there are two elements in Ft and the move of type 1 or type 2 is the difference of these two elements. Hence type 1 and type 2 moves are indispensable. But if t = Azz+ = Azz− for a move z of type 0, there are three elements in Ft . Then to connect these three elements to form a tree, we can choose two moves to construct a minimal Markov basis. (There are three ways of doing this.) For example, consider the case of I = 4 and t = (1, 1, 1, 1) . F(1,1,1,1) is written as
F(1,1,1,1)
⎧ 0100 0010 0001 ⎪ ⎪ ⎨ 000 001 010 = , , ⎪ 0 1 0 0 00 ⎪ ⎩ 0 0 0
⎫ ⎪ ⎪ ⎬ ⎪ ⎪ ⎭
.
To connect these three elements to a tree, any two of the following type 0 moves of degree 2, 0 +1 −1 0 0 +1 0 −1 0 0 −1 +1 0 0 −1 0 −1 0 0 +1 −1 , , , 0 +1 0 +1 0 0 0 0 0 can be included in a minimal Markov basis. Accordingly, I(I − 1)(I − 2)(I − 3)/12 moves of type 0, I(I − 1)(I − 2)/2 moves of type 1 and I(I − 1)/2 moves of type 2 constitute a minimal Markov basis.
6.3 Graver Basis and 1-Norm Reducing Markov Bases The 1-norm |zz| = ∑i ∈I |zi | = 2 deg z on the set Z|I | of integer vectors is a natural norm to consider for Markov bases. In this section we discuss the relation between the Graver basis and the 1-norm reducing Markov bases. We show that the Graver basis is always 1-norm reducing. We have the following proposition. Proposition 6.4. The Graver basis is strongly 1-norm reducing. = x, be in the same fiber. Express y − x as a conformal sum Proof. Let x, y ∈ Ft , y of nonzero elements of the Graver basis: y − x = z1 + · · · + zm .
86
6 Method of Distance Reduction
Then |yy − x | = |zz1 | + · · · + |zzm |. Now z 1 can be subtracted from y and at the same time z 1 can be added to x to give |(yy − z 1 ) − x | = |yy − (xx + z 1 )| = |zz2 | + · · · + |zzm | < |yy − x |.
Note that the Graver basis is rich enough that we can take z 1 = z 2 in the definition of strong distance reduction in (6.2). Proposition 6.5. A set of moves B is 1-norm reducing if and only if for every element z = z + − z − of the Graver basis, B is 1-norm reducing for z + , z − . Proof. We only have to prove sufficiency. Let x , y ∈ Ft be arbitrarily given and let y − x = z 1 + · · · + z m be a conformal sum of elements of the Graver basis. By − assumption B is 1-norm reducing for z + 1 , z 1 . Among four possible cases, without loss of generality, consider the case that z ∈ B is applicable to z + 1 and − + − |(zz+ 1 + z ) − z1 | < |zz1 − z 1 | = |zz1 |.
(6.3)
+ − y − x)+ . Furthermore (6.3) implies that Because z is applicable to z + 1 , z ≤ z 1 ≤ (y + y − x )− ). 0/ = supp(zz+ ) ∩ supp(zz− 1 ) ⊂ supp(zz ) ∩ supp((y
It follows that z is applicable to y and |(yy + z ) − x| < |yy − x |.
Note that the same statement holds for strong 1-norm reduction with exactly the same proof. Proposition 6.6. A set of moves B is strongly 1-norm reducing if and only if for every element z = z + − z − of the Graver basis, B is strongly 1-norm reducing for z+, z−.
6.4 Some Results on Minimality of 1-Norm Reducing Markov Bases In this section we discuss minimality of 1-norm reducing Markov bases. Inasmuch as the material in this section is not used in other parts of this book, the reader can skip this section. Suppose that B is a 1-norm reducing Markov basis. Then any B ⊃ B is a 1norm reducing Markov basis as well. In view of this, it is of interest to consider minimality of 1-norm reducing Markov bases. A 1-norm reducing Markov basis B is minimal if no proper subset of B is a 1-norm reducing Markov basis. For a 1-norm reducing Markov basis B, we can examine each element z of B one by one, and see whether B \ {zz} remains to be a 1-norm reducing Markov basis. If B \ {zz} remains to be 1-norm reducing, we remove z , recursively, until none of
6.4 Some Results on Minimality of 1-Norm Reducing Markov Bases
87
the remaining elements can be removed any further. Then we arrive at a minimal 1-norm reducing Markov basis. Therefore every 1-norm reducing Markov basis B contains a minimal 1-norm reducing Markov basis. Exactly the same argument holds concerning minimality of strongly 1-norm reducing Markov bases. Every strongly 1-norm reducing Markov basis contains a minimal strongly 1-norm reducing Markov basis. In Chap. 5 we considered minimality of Markov bases. A similar argument can be applied to the question of minimality of 1-norm reducing Markov bases. In order to study this minimality question we introduce three closely related notions of degree reduction of a move z by other moves. We say that a move z = z + − z − is 1-norm reducible by another move z = ±zz if z is applicable to + − z and |zz + z | < |zz| or z is applicable to z and | − z + z | = |zz − z | < |zz|. We say that a move z = z + − z − is strongly 1-norm reducible by a pair of (other) moves z1, z2 = ±zz if z 1 is applicable to z + and |zz + z 1 | < |zz| and furthermore z 2 is applicable − to z and |zz − z 2 | < |zz|. Finally we say that z is 1-norm reducible by a lower degree move z if |zz | < |zz| and z is 1-norm reducible by z . Consider the implications among these notions. If z is strongly 1-norm reducible by z 1 , z 2 , then z is clearly 1-norm reducible by z 1 (or z 2 ). Now we show that if z is 1-norm reducible by a lower degree move z , then z is strongly 1-norm reducible either by the pair z , z + z or by the pair z − z , z . To show this, first consider the case that z is applicable to z + and |zz + z | < |zz|. Let z = z + z . Then |zz − z | = |zz | < |zz| and we only need to check that z is applicable to z − . In fact z = z + − z − + (zz )+ − (zz )− = (zz + − (zz )− ) + (zz )+ − z − ≥ (zz )+ − z − . This implies that (zz )− ≤ z − and z is applicable to z − . Similarly if z is applicable to z − , we can check that z is strongly 1-norm reducible by the pair z − z , z . Based on the above observation, we define three notions of irreducibility of a move. We call z 1-norm irreducible if it is not 1-norm reducible by any other move z = z . We call z strongly 1-norm irreducible if it is not strongly 1-norm reducible by any pair of other moves. Finally we call z 1-norm lower degree irreducible if it is not 1-norm reducible by any lower degree move. We state the above implications of the properties of moves, as well as further implications among indispensability and conformal primitiveness, in the following proposition. Proposition 6.7. For a move z , the following implications hold. indispensable ⇒ 1-norm irreducible ⇒ strongly 1-norm irreducible ⇒ 1-norm lower degree irreducible ⇒ conformally primitive.
(6.4)
88
6 Method of Distance Reduction
Proof. If z is not conformally primitive, then z is clearly 1-norm reducible by a lower degree move. This proves the last implication. If z is 1-norm reducible by z = ±zz, then z − = z + + z ∈ Ft or z + = z − + z ∈ + Ft , where t = Azz . Therefore Ft is not a two-element fiber. Therefore z is not indispensable. This proves the first implication. Other implications hold by definition.
We now state two lemmas. Lemma 6.1. If z is 1-norm reducible by another move z = ±zz, then there exists a conformally primitive move z = z , |zz | ≤ |zz |, such that z is 1-norm reducible by z . Proof. If z is itself conformally primitive, just let z = z . If z is not conformally primitive, write z as a conformal sum z = z 1 + · · · + z m of nonzero elements of the Graver basis. Among two possible cases, without loss of generality, consider the case that z is applicable to z + and |zz + z | < |zz|. In this case (zz )− ≤ z + + and supp((zz )+ ) ∩ supp(zz− ) = 0. / Because supp((zz )+ ) = supp(zz+ 1 ) ∪ · · · ∪ supp(zzm ), − + − there exists some l such that supp((zz l ) ) ∩ supp(zz ) = 0. / Furthermore z l ≤ (zz )− ≤ z + and |zzl | < |zz | ≤ |zz|. This implies that z is 1-norm reducible by z = z l = z.
Lemma 6.2. Let z be a 1-norm irreducible move. Then either z or −zz belongs to every 1-norm reducing Markov basis. Proof. We argue by contradiction. Let z = z + − z − be 1-norm irreducible and let B be a 1-norm reducing Markov basis containing neither z nor −zz. Because B is 1-norm reducing, B is 1-norm reducing for z + , z − . But this contradicts the 1-norm irreducibility of z in view of (6.1).
We say that there exists a unique minimal 1-norm reducing Markov basis if all minimal 1-norm reducing Markov bases coincide except for sign changes of their elements. We now state the following proposition. Proposition 6.8. There exists a unique minimal 1-norm reducing Markov basis if and only if 1-norm irreducible moves form a 1-norm reducing Markov basis. Proof. Every 1-norm irreducible move (or its sign change) belongs to every 1-norm reducing Markov basis, thus if the set of 1-norm irreducible moves is a 1-norm reducing Markov basis, then it is clearly the unique minimal 1-norm reducing Markov basis ignoring the sign of each move. Conversely suppose that 1-norm irreducible moves do not form a 1-norm reducing Markov basis. Then every 1-norm reducing Markov basis contains a 1norm reducible move. Let B be a minimal 1-norm reducing Markov basis and let z 0 ∈ B be 1-norm reducible. Consider B˜ = (B ∪ BGraver ) \ {zz0 , −zz0 },
6.4 Some Results on Minimality of 1-Norm Reducing Markov Bases
89
where BGraver is the Graver basis. We show that B˜ is a 1-norm reducing Markov basis. If this is the case, B˜ contains a minimal 1-norm reducing Markov basis different from B even if we change the signs of the elements. Now by Propositions 6.1 and 6.5, it suffices to show that for every z = z + − − z ∈ BGraver , B˜ is 1-norm reducing for z + , z − . If z 0 is not conformally primitive, B˜ ⊃ BGraver and B˜ is 1-norm reducing. Therefore let z 0 be conformally primitive. Each conformally primitive z = z + − z − = z 0 is already in B˜ and B˜ is 1-norm reducing for z + , z − . The only remaining case is z = z 0 itself, but by Lemma 6.1, z 0 ˜ is 1-norm reducible by a conformally primitive z = ±zz0 , z ∈ B.
Chapter 7
Symmetry of Markov Bases
7.1 Motivations for Invariance of Markov Bases In this chapter we study properties of Markov bases from the viewpoint of invariance. This is partly motivated by the fact that Gr¨obner bases depend on a given term order and a reduced Gr¨obner basis does not preserve the symmetry inherent in a given statistical model. For example, hierarchical models for multiway contingency tables (cf. Sect. 1.5) are symmetric with respect to permutations of the levels of each axis of the table. In group-theoretic terminology, the direct product of symmetric groups acts on the set of multiway tables and hierarchical models are invariant with respect to this group action. By utilizing invariance we can give a concise description of Markov bases by orbit lists. To illustrate this, we consider the no-three-factor interaction model for three-way tables, which is treated in Chap. 9. In Table 7.1 we list the numbers of the elements of the unique minimal Markov basis, along with the numbers of the reduced Gr¨obner basis elements calculated by 4ti2 [1] and the numbers of the orbits with respect to the action of the direct product of symmetric groups for the problem of 3 × 3 × K (K ≤ 7) contingency tables with fixed two-dimensional marginals. As we show in Sect. 7.6, a set of moves is partitioned into orbits that are equivalence classes by the action of the group. As we show in Chap. 9, there are at most six orbits of indispensable moves for these problems. In these examples, a minimal Markov basis is unique. Furthermore it is minimal invariant in the sense of Sect. 7.6. Therefore the representative basis elements for each orbit contain all the information of the minimal Markov basis. To perform the Markov chain Monte Carlo simulations using these orbit lists, users can first randomly choose an orbit, and then apply a random group action to the representative basis element for each step of the chain. Another interesting consideration is how to choose a minimal Markov basis if it is not unique. For such cases, different minimal Markov bases contain different numbers of orbits in general, and some basis elements in these orbits are not necessarily needed for connectivity.
S. Aoki et al., Markov Bases in Algebraic Statistics, Springer Series in Statistics 199, DOI 10.1007/978-1-4614-3719-2 7, © Springer Science+Business Media New York 2012
91
92
7 Symmetry of Markov Bases Table 7.1 Number of the elements of the unique minimal Markov bases, the reduced Gr¨obner bases, and orbits for 3 × 3 × K, K ≤ 7, tables with fixed two-dimensional marginals K 3 4 5 6 7 Number of the elements in the unique minimal Markov basis Number of the elements in the reduced Gr¨obner basis Number of orbits in the unique minimal Markov basis
81
450
2,670
10,665
31,815
110
622
3,240
12,085
34,790
4
5
6
6
6
In Table 7.1 we have considered permutation of the levels for each axis. If the number of levels of the axes is common and if in addition the hierarchical log-linear model considered is symmetric with respect to permutations of axes, we can further consider the permutation of the axes themselves. For example in the case of the 3 × 3 × 3 contingency tables with no three-factor interactions, we can consider the permutation of the axes. As we show in Chap. 9, if this additional symmetry of axes is considered, there are only two orbits corresponding to moves of degree 4 and degree 6, whereas if this additional symmetry is not considered there are four orbits as indicated in Table 7.1. This question leads to the notion of the largest symmetry in a given model, which we define in Sect. 7.4.
7.2 Examples of Invariant Markov Bases In this section we consider two simple examples of invariant Markov bases. They are the 2 × 2 × 2 contingency tables with fixed one-dimensional marginals and the Hardy–Weinberg model, which were already treated in Sect. 6.2. We use the following notation for our moves. Moves in minimal bases contain many zero cells. Furthermore, often the nonzero elements of a move contain either 1 or −1. Therefore a move can be concisely denoted by locations of its nonzero cells. We express a move z of degree n as z = i 1 · · · i n − j 1 · · · j n , where i 1 , . . . , i n are the cells of positive frequencies of z and j 1 , . . . , j n are the cells of negative frequencies of z . In the case z(ii) > 1, i is repeated z(ii) times. Similarly j is repeated −z( j ) times if z( j ) < −1. We use a similar notation for contingency tables as well. x with deg x = n is simply denoted as x = i 1 · · · i n . First consider the 2 × 2 × 2 contingency tables with fixed one-dimensional marginals. As shown in Sect. 6.2, the minimal Markov basis for this problem is not unique. Each minimal Markov basis contains six indispensable elements and three dispensable elements. Consider dispensable elements. The reduced Gr¨obner basis with respect to the graded reverse lexicographic order contains three dispensable moves (binomials) such as (121)(212) − (111)(222), (122)(211) − (111)(222), (112)(221) − (111)(222).
7.3 Action of Symmetric Group on the Set of Cells
93
It is seen that these three dispensable basis elements are in different orbits with respect to permutation of levels. On the other hand, another minimal basis is constructed from three dispensable basis elements such as (121)(212) − (111)(222), (122)(211) − (111)(222), (112)(221) − (121)(212). In this basis, the second and the third binomials are in the same orbit. In fact, we see that (112)(221) − (121)(212) can be produced from (122)(211) − (111)(222) by interchanging the cell indices 1, 2 in the second axis. Accordingly, if we consider an action of the direct product of symmetric groups, only two basis elements such as (121)(212) − (111)(222), (122)(211) − (111)(222) have to be included in our list, because the third basis element can be produced by permuting the second axis. Furthermore, because the number of levels is common for three axes in the 2 × 2 × 2 case, we can also permute the axes. If we consider invariance with respect to this larger group, then a single representative element among dispensable ones such as (112)(221) − (111)(222) is sufficient to describe an invariant Markov basis. We now consider the Hardy–Weinberg model of Sect. 6.2.2 for I alleles. The direct product of symmetric groups is not appropriate in this case, because the contingency table x = {xi j }1≤i≤ j≤I is of an upper triangular form. However, it is clear that this problem has the symmetry with respect to a simultaneous permutation of the levels (i.e., alleles). It can be checked (see [12]) that a minimal invariant Markov basis with respect to this group action consists of three orbits, with the representative moves given as (11)(22) − (12)(12), (11)(23) − (12)(13), (12)(34) − (13)(24).
(7.1)
In this case, the unique minimal Markov basis does not exist as we have seen in Sect. 6.2.2. However, the minimal invariant Markov basis given in (7.1) can be shown to be the unique minimal invariant Markov basis.
7.3 Action of Symmetric Group on the Set of Cells In this section we formulate the symmetry of a given toric model in terms of the action of a group on the set of cells. As the most important example we consider the direct product of symmetric groups acting on the cells of multiway contingency tables by permutations of levels for each axis. Decomposable models (Chap. 8) and more general hierarchical models (Chap. 9) are invariant with respect to this group.
94
7 Symmetry of Markov Bases
First we give a brief list of definitions and notations of a group action. Basic facts on group action in statistical problems are found in Chap. 6 of [98] or Chap. 4 of [58]. More comprehensive treatment is given in [56]. Let a group G act (from the left) on a set I . This means that each element g ∈ G is a map I → I sending i to gii, and the following conditions are satisfied eii = i , (g1 g2 )ii = g1 (g2 i),
∀ii ∈ I ,
(7.2)
∀g1 , g2 ∈ G, ∀ii ∈ I ,
(7.3)
where e is the identity element of G. Under these conditions, the inverse element g−1 of g in G is also the inverse of g as a map from I to I . This implies that each g is a bijection from I to I . In this book as I we are considering the set of cells, which is a finite set. Hence each g ∈ G is just a permutation of the cells of I and G is a subgroup of the symmetric group Sη , η = |I |, which is the group of all permutations of the cells of I . Define G(ii) = {gii | g ∈ G} as the orbit through i . Let I /G denote the orbit space, that is, the set of orbits. The action of G is called transitive, if the whole G is one orbit. Let Gi = {g | gii = i } denote the isotropy subgroup (pointwise stabilizer) of i in G. If G acts on I , the action of G on the set of functions f on I is induced by (g f )(ii) = f (g−1 i ). Because the frequency vector x is considered as a function I → N, the action of G on the set X = Nη of frequency vectors is defined as (gxx)(ii) = x(g−1 i ). This is again just a permutation of elements of x . Let us write out the permutation matrix for g. (gxx)(ii) = x(g−1 i ) means that the i th element of gxx is the (g−1 i )th element of x . Let Pg = {pi j } = {δi ,g j }
(7.4)
denote an η × η permutation matrix, where δ is Kronecker’s delta. Then the i th row of Pg has 1 at the column j = g−1 i and hence the i th element of Pg x is the (g−1 i )th element of x . Equivalently, the (gii)th element of Pg x is the i th element of x . Therefore we have gxx = Pg x . From this it follows that Pg1 g2 = Pg1 Pg2 ,
∀g1 , g2 ∈ Sη and Pg−1 = Pg ,
∀g ∈ Sη .
(7.5)
Similarly G acts on a move z by (gzz)(ii) = z(g−1 i ). If we write z = z + − z − , then gzz = Pg z = Pg z + − Pgz − = gzz+ − gzz− . We also call a move z = z + − z − symmetric with respect to G if z + = gzz− for some g ∈ G. Conversely, a move z is asymmetric if G(zz+ ) ∩ G(zz− ) = 0. /
7.3 Action of Symmetric Group on the Set of Cells
95
As G consider the direct product of symmetric groups, which is our main example. Let G = SI denote the symmetric group on {1, . . . , I } for = 1, . . . , m and let G = G 1 × G2 × · · · × Gm
(7.6)
be the direct product. We write an element of g ∈ G as g = g1 × · · · × gm =
1 · · · I1 σ1 (1) · · · σ1 (I1 )
× ···×
1 · · · Im σm (1) · · · σm (Im )
.
G acts on the set of cells I of m-way contingency tables by i = gii = (g1 i1 , . . . , gm im ) = (σ1 (i1 ), . . . , σm (im )) .
(7.7)
Now we consider the action of G on the set of sufficient statistics T in (4.6). We go back to the general definition of a group action. Let h : I → T be a surjection. If the following condition holds, h(ii) = h(ii ) ⇒ h(gii) = h(gii ),
∀g ∈ G,
(7.8)
then the action of G on T is induced by defining gtt = h(gii), where t = h(ii).
(7.9)
Indeed gtt is well defined, because by (7.8) gtt does not depend on i such that t = h(ii). Then by choosing i for each t ∈ T , (7.2) and (7.3) for t are easily verified as ett = h(eii) = h(ii) = t , (g1 g2 )tt = h((g1 g2 )ii) = h(g1 (g2 i )) = g1 h(g2 i ) = g1 (g2 h(ii)) = g1 (g2t ). Note that (7.9) is written as h(gii) = gh(ii),
∀g ∈ G, ∀ii ∈ I .
(7.10)
We call h satisfying (7.10) equivariant. We can also say that h and the group action commute (i.e., hg = gh). Conversely suppose that G acts on both I and T and the surjection h : I → T is equivariant. Then h(ii) = h(ii ) ⇒ gh(ii) = gh(ii ) ⇒ h(gii) = h(gii ) and (7.8) holds.
96
7 Symmetry of Markov Bases
Consider again the direct product of symmetric groups in (7.6). A sufficient statistic for hierarchical models for contingency tables consists of various marginal frequencies. Note that G acts on the marginal cells i D , D = {s1 , . . . , sk } ⊂ [m] = {1, . . . , m}, by i D = giiD = (gs1 is1 , . . . , gsk isk ) = (σs1 (is1 ), . . . , σsk (isk )). Hence G acts on marginal tables by x D = gxxD = {xD (g−1 i D )}iD ∈ID . Considering this action simultaneously for various marginals D1 , . . . , Dr ⊂ [m], the action of G on the sufficient statistic t = (xxD1 , . . . , x Dr ) of a hierarchical model is defined by gtt = (gxxD1 , . . . , gxxDr ). An important point here is that the map of taking marginal frequencies is equivariant; that is, we have the following lemma. Lemma 7.1. (gxx)D = gxxD for all g ∈ G and x ∈ Nη . This lemma clearly holds, because taking the marginal sums after permutation of levels of axes is the same as first taking the marginal sums and permuting the axes in the marginal cells. By Lemma 7.1 and the above argument, the action of G on the set T of marginal frequencies t is induced from the action of G on x .
7.4 Symmetry of a Toric Model and the Largest Group of Invariance In the previous section we considered the direct product of symmetric groups acting on the set of multiway contingency tables as our main example. In the case of the Hardy–Weinberg model, the set of cells was an upper triangular matrix and the symmetry was not described by the direct product of symmetric groups. Now we consider how to define a symmetry of a given toric model or a configuration A. Consider the toric model in (4.5). The probability distribution of x depends on θ A, and θ A is an element of the row space of A, rowspan(A). Assuming that θ ∈ Rν is a free parameter vector, the set of probability distributions of the toric model is identified with rowspan(A). In this sense, when we consider the symmetry of a given toric model, it is reasonable to require that the symmetry be defined in terms of rowspan(A). Multiplying A from the right by Pg results in a matrix APg whose columns are permutations of columns of A by g ∈ Sη . The reason we take the transpose Pg is to preserve the action of G “from the left.” By defining gA = APg , we have (g1 g2 )A = APg 1 g2 = A(Pg1 Pg2 ) = APg 2 Pg 1 = g1 (g2 A).
(7.11)
7.4 Symmetry of a Toric Model and the Largest Group of Invariance
97
In gA = APg , we can think of Pg as multiplying each row of A from the right, namely G acts on the set of η -dimensional row vectors α by gα = α Pg . Note that the space of row vectors can be considered as the dual vector space of the space of column vectors. In Sect. 7.2, the complete independence model of 2 × 2 × 2 contingency tables is clearly invariant with respect to the direct product of symmetric groups S2 × S2 × S2 . The Hardy–Weinberg model for I alleles is invariant with respect to permutation of alleles SI . In view of these examples we make the following definition. Definition 7.1. Let G ⊂ Sη be a subgroup of Sη . A configuration A is invariant with respect to G if rowspan(A) = rowspan(APg ) for all g ∈ G. In Sect. 4.2 we discussed the relation rowspan(A)⊥ = ker A. Then we have rowspan(A) = rowspan(APg ) ⇔ ker A = ker(APg ). Also note that ker(APg ) = {zz | APg z = 0} = {Pgz | Azz = 0} = Pg ker A,
(7.12)
where on the right-hand side Pg is now multiplying column vectors from the left. Hence an equivalent definition of invariance with respect to G is given as follows. Definition 7.2. A configuration A is invariant with respect to G if ker A = Pg ker A for all g ∈ G. So far we have defined the invariance of the configuration A with respect to a given G. When A is given first, it is natural to consider all g ∈ SI such that rowspan(A) = rowspan(APg ), or equivalently ker A = Pg ker A. Here the notion of setwise stabilizer [136] is useful. Let a group G act on a set X from the left. For a subset V of X , let GV = {g | gV = V } denote the setwise stabilizer of V . (Note that GV forms a subgroup of G.) As we have discussed already, G acts on the set of η -dimensional column vectors by g : x → Pg x and the set of η -dimensional row vectors by g : α → α Pg . From this viewpoint, the set of g such that rowspan(A) = rowspan(APg ) is the setwise stabilizer Growspan(A) . Equivalently it is the setwise stabilizer Gker A . Therefore we are led to the following definition. Definition 7.3. For a given configuration A, the largest group of invariance is the setwise stabilizer Gker A of ker A in the symmetric group Sη , where G acts on the set of η -dimensional column vectors. Alternatively, it is the setwise stabilizer Growspan(A) , where G acts on the set of η -dimensional row vectors. From now on, among two equivalent definitions, we mainly consider Gker A . The notion of the largest group of invariance was introduced in [12].
98
7 Symmetry of Markov Bases
In Sect. 7.3 we considered the induced action on the set of sufficient statistics. We want to check that the induced action on the set of sufficient statistics T is defined also for the largest group of invariance. We try to define gtt , t = Axx, by gtt = Agxx. If t = Axx = A˜x , then x − x˜ ∈ ker A. For g ∈ Gker A , g(xx − x˜ ) ∈ ker A and hence Ag(xx − x˜ ) = 0. Therefore gtt does not depend on the choice of x in t = Axx. Hence the induced action of Gker A on T is well defined. It should be noted that a configuration A is invariant with respect to a group H if and only if H is a subgroup of the largest group of invariance Gker A . Also because Gker A acts on T , any subgroup H of Gker A also acts on T . We have given the definition of the largest group of invariance for a general configuration A. For many configurations A it is often surprisingly hard to determine the largest group of invariance Gker A , although some obvious subgroup of Gker A is easy to find. We discuss one simple example in the next section.
7.5 The Largest Group of Invariance for the Independence Model of Two-Way Tables As we have stated above, it is often surprisingly hard to determine the largest group of invariance G for a given A. The symmetry in the independence model of twoway tables seems to be trivial. However, to prove that the obvious symmetry is the largest, we need some careful arguments. For showing that a given candidate group is the largest group of invariance, in [135], we developed a “perturbation method.” Here we illustrate the perturbation method with the independence model of two-way tables. Consider I × J contingency tables with fixed row sums and column sums. The configuration is clearly invariant with respect to the direct product SI × SJ , which seems to be the largest group of invariance if I = J. In the case of square tables I = J, there is an additional symmetry of interchanging the two axes. Although this is again a symmetric group S2 , for clarity we denote the group of interchanging the axes by H2 . Then the largest group of invariance for the square case seems to be the subgroup of SI 2 generated by SI × SI and H2 . In the square case, we can first decide whether to flip the axes, and then we can arbitrarily and independently permute the levels of two axes. This is called the wreath product (e.g., [129]) of groups and written as SI wr H2 . Now we have the following result. Proposition 7.1. The largest group of invariance for I × J contingency tables with fixed row sums and column sums is SI × SJ if I = J and SI wr H2 if I = J.
7.5 The Largest Group of Invariance for the Independence Model. . .
99
The rest of this section is devoted to a sketch of a proof of this proposition. We mainly consider the case I = J. In the process of proving the case I = J, it will become clear that the additional symmetry in the square case is given by H2 . We now assume I > J without loss of generality. By the form of the configuration A in (1.20), an element of rowspan(A) is an I × J-dimensional (row) vector with components of the form {αi + β j }1≤i≤I,1≤ j≤J ,
αi , β j ∈ R.
Write γi j = αi + β j . For four distinct pairs (i1 , j1 ), (i2 , j2 ), (i3 , j3 ), (i4 , j4 ) consider the following linear combination
δ = γi1 j1 + γi2 j2 − γi3 j3 − γi4 j4 . Inasmuch as ker A is spanned by the basic moves in (2.4), δ = 0 if (i1 , j1 ), (i2 , j2 ), (i3 , j3 ), (i4 , j4 ) are the four cells in a rectangular position as in (2.4). By taking αi , β j sufficiently “generic” it is clear that δ = 0 unless the four cells are in a rectangular position. This can be made explicit as follows. Let b > 0 be a large positive integer. For our case b = 5 is good enough. Let
αi = bi ,
β j = bI+ j .
(7.13)
Then by the uniqueness of the base b expansion of a positive integer δ , for this choice of αi and β j , δ = 0 unless the four cells are in a rectangular position. Let γ = {bi + bI+ j } denote the I × J-dimensional vector with these elements. In the following we consider γ as an I × J table. Now consider g ∈ Growspan(A) and let γ˜ = gγ . By the consideration immediately following (7.4), the g(i, j) element of γ˜ is the (i, j) element of γ . Consider the lowerright cell (I, J) of γ . Then the value bI +bI+J is in the g(i, j) cell of γ˜ . Write (i∗ , j∗ ) = g(I, J) and consider the following linear combination of elements of γ˜ :
δ˜ (i2 , j2 ) = γ˜i∗ j∗ + γ˜i2 j2 − γ˜i∗ j2 − γ˜i2 j∗ = γIJ + γg−1 (i2 , j2 ) − γg−1(i∗ , j2 ) − γg−1 (i2 , j∗ ) , where the four cells (i∗ , j∗ ), (i2 , j2 ), (i∗ , j2 ), (i2 , j∗ ) are distinct; that is, i2 = i∗ and j2 = j∗ . Because g and g−1 are bijections, the four cells (I, J), g−1 (i2 , j2 ), g−1 (i∗ , j2 ), g−1 (i2 , j∗ ) are distinct as well. Furthermore, because γ˜ ∈ rowspan(APg ) = rowspan(A), δ˜ (i2 , j2 ) = 0 for all choices of (i2 , j2 ). By our particular choice (7.13) of γ , we see that (I, J), g−1 (i2 , j2 ), g−1 (i∗ , j2 ), g−1 (i2 , j∗ ) have to be in a rectangular position. This means that, either 1. g−1 (i∗ , j2 ) is in the last row and g−1 (i2 , j∗ ) is in the last column, or 2. g−1 (i∗ , j2 ) is in the last column and g−1 (i2 , j∗ ) is in the last row. We note that these two cases cannot mix. In fact, if g−1 (i2 , j∗ ) and g−1 (i∗ , j2 ) are both in the last column for some i2 and j2 , then it can be easily seen that δ˜ (i2 , j2 ) cannot be zero for this i2 and j2 . It follows that either g−1 (i2 , j∗ ), i2 = i∗ , are all in
100
7 Symmetry of Markov Bases
the last column, or they are all in the last row. Because we have assumed I > J, the second case is impossible and we have {g−1 (i2 , j∗ )}i2 =1,...,I = {(i, J)}i=1,...,I . This means that the last column of γ is moved to the j∗ th column of γ˜ . Now we can similarly argue for other columns and other rows of γ . Then it follows that g moves columns of γ to columns of γ˜ and rows of γ to rows of γ˜ . Hence g is an element of SI × SJ .
7.6 Characterizations of a Minimal Invariant Markov Basis Now we go back to invariant Markov bases. Let B ⊂ kerZ A be a set of moves. For convenience in this section we assume that B is sign invariant (see Sect. 5.2). Let the configuration A be invariant with respect to a group G. We call B G-invariant if G(B) = B. Note that B is G-invariant if and only if g ∈ G, z ∈ B =⇒ gzz ∈ B.
In other words, B is G-invariant if and only if it is a union of orbits B = z ∈B∗ G(zz) for some subset B ∗ ⊂ kerZ A of moves. A finite sign invariant set B ⊂ kerZ A is an invariant Markov basis if it is a Markov basis and it is G-invariant. An invariant Markov basis is minimal if no proper sign invariant and G-invariant subset of B is a Markov basis. A minimal invariant Markov basis always exists, because from any invariant Markov basis, we can remove orbits one by one, until none of the remaining orbits can be removed any further. Partition X = Nη by the degree (total sample size) of the frequency vectors as X = ∪∞ n=1 Xn ,
Xn = {xx ∈ X | deg x = n}.
Similarly partition the set of sufficient statistics as T = ∪∞ n=1 Tn . In considering the orbits of G acting on X , we note that deg x = deg(gxx), ∀g ∈ G, and hence G(Xn ) = Xn for all n. Therefore we can consider the action of G on each Xn separately. Similarly we can consider the action of G on each Tn separately because degtt = deg(gtt ), ∀g ∈ G. Consider a particular sufficient statistic t ∈ Tn . As in (5.4) let Bt = {zz ∈ kerZ A | z + , z − ∈ Ft } be the set of moves whose positive and negative parts belong to the fiber Ft . Let G(tt ) ∈ Tn /G be the orbit through t . Let BG(tt ) =
Bt
t ∈G(tt )
denote the union of the set of moves Bt over the orbit G(tt ) through t .
7.6 Characterizations of a Minimal Invariant Markov Basis
101
Let B ⊂ kerZ A be a finite set of moves. An important observation is that B is partitioned as B=
Bn,α ,
(7.14)
n α ∈Tn /G
where we define Bn,α = B ∩ Bα ,
α ∈ Tn /G.
Inasmuch as B is invariant if and only if it is a union of orbits G(zz ), the following lemma holds. Lemma 7.2. B is invariant if and only if Bn,α is invariant for each n and α ∈ Tn /G. Proof. Let z ∈ Bn,α and t = Azz+ ∈ α . Then it follows that gzz ∈ Bgtt ⊂ Bα and the lemma is proved. This lemma shows that we can restrict our attention to each Bn,α in studying the invariance of a Markov basis. In characterizing a Markov basis and its minimality, in Chap. 5 we argued that it is essential to consider B|tt |−1 -equivalence classes of Ft , where Bn is the set of moves of degree less than or equal to n defined in (5.2). As in Chap. 5 we write |tt | for degtt . Considering group actions on the set of moves and each fiber, we characterize the structure of a minimal invariant Markov basis. As we show in the following, the relation between the action of the isotropy subgroup Gt and B|tt |−1 -equivalence classes of Ft is important. For the rest of this section, we write the set of B|tt |−1 equivalence classes of Ft as Ht for simplicity; that is, Ht = Ft /B|tt |−1 . Now we state the following theorem. Theorem 7.1. Let B be a minimal G-invariant Markov basis and let B = n α ∈Tn /G Bn,α be the partition in (7.14). Then each Bn,α , α ∈ Tn /G, is a minimal invariant set of moves, where Bn,α ∩ Bt , t ∈ α , connects B|tt |−1 equivalence classes of Ft and
Bn,α = G(Bn,α ∩ Bt )
(7.15)
for any t ∈ α . Conversely, from each α ∈ Tn /G with |Ht | ≥ 2, where t ∈ α is a representative sufficient statistic, choose a minimal Gt -invariant set of moves B ∗ ⊂ Bt connecting B|tt |−1 -equivalence classes of Ft , where Gt ⊂ G is the isotropy subgroup of t , and extend B ∗ to G(B ∗ ). Then B=
n
α ∈Tn /G |Ht |≥2,tt ∈α
is a minimal G-invariant Markov basis.
G(B ∗ )
102
7 Symmetry of Markov Bases
This theorem only adds a statement of minimal G-invariance to the structure of a minimal Markov basis considered in Chap. 5 In principle this theorem can be used to construct a minimal invariant Markov basis by considering α ∈Tn /G Bn,α , n = 1, 2, 3, . . . step by step. By the Hilbert basis theorem, there exists some n0 (cf. Proposition 5.3), such that for n ≥ n0 no new moves need to be added. Then a minimal invariant Markov basis is written as n0 n=1 α ∈Tn /G Bn,α . To prove Theorem 7.1, we prepare some lemmas in the following. First, we derive some basic properties of orbits of G acting on each fiber. As we stated before, we consider the action of G on each Xn separately. Let FG(tt ) =
Ft
t ∈G(tt )
denote the union of fibers over the orbit G(tt ) through t . Let x ∈ Ft . Because t (gxx) = gtt , it follows that gxx ∈ Fgtt ⊂ FG(tt ) . Therefore G(FG(tt ) ) = FG(tt ) . This implies that Xn is partitioned as Xn =
Fα ,
(7.16)
α ∈Tn /G
where α runs over the set of different orbits and we can consider the action of G on each FG(tt ) separately. Consider a particular FG(tt ) . An important observation is that there is a direct product structure in FG(tt ) . Write G(tt ) = {tt 1 , . . . ,tt a },
(7.17)
where a = a(tt ) = |G(tt )| is the number of elements of the orbit G(tt ) ⊂ Tn . Let b = b(tt ) = |FG(tt ) /G| be the number of orbits of G acting on FG(tt ) and let x1 , . . . , xb be representative elements of different orbits; that is, FG(tt ) = G(xx1 ) ∪ · · · ∪ G(xxb )
(7.18)
gives a partition of FG(tt ) . Then we have the following lemma. Lemma 7.3. We use the notations (7.17) and (7.18). Then FG(tt ) is partitioned as FG(tt ) =
a b
Ft i ∩ G(xx j ),
(7.19)
i=1 j=1
where each Ft i ∩ G(xx j ) is nonempty. Furthermore if t i = gtt i , then Ft i x → gxx ∈ Ft i gives a bijection between Ft i ∩ G(xx) and Ft i ∩ G(xx).
7.6 Characterizations of a Minimal Invariant Markov Basis
103
Proof. Let FG(tt ) = Ft 1 ∪ · · · ∪ Ft a be a partition. Intersecting this partition with FG(tt ) = bj=1 G(xx j ) gives the partition of (7.19). Let x ∈ Ft . Then the orbit G(xx) intersects each fiber; that is, G(xx) ∩ Ft i = 0/ for i = 1, . . . , a. Because every g ∈ G is a bijection of FG(tt ) to itself and g(Ft ∩ G(xx)) = Fgtt ∩ G(xx), g gives a bijection between Ft i ∩ G(xx) and Ft i ∩ G(xx).
In particular for each j, Ft i ∩ G(xx j ), i = 1, . . . , a, have the same number of elements |Ft 1 ∩ G(xx j )| = · · · = |Ft a ∩ G(xx j )|. In addition, for t i ,tt i ∈ G(tt ) such that t i = gtt i , the map g : Gt i → gGt i g−1 gives an isomorphism between Gt i and Gt i = gGt i g−1 , where Gt i and Gt i are the isotropy subgroup of t i and t i in G, respectively. Therefore there exists the following isomorphic structures in Ft i , (Gt i , Ft i ) (Gt i , Ft i ).
(7.20)
Considering the isomorphic structure of (7.20), now we can focus our attention on each fiber. Consider a particular fiber Ft . Here we can restrict our attention to the action of Gt on Ft . As we have stated before, the relation between the action of Gt and Ht = Ft /B|tt |−1 (the B|tt |−1 -equivalence classes of Ft ) is essential. First we show the following lemma. Lemma 7.4. For any integer n, if x is accessible from x by Bn , then gxx is accessible from gxx by Bn . Proof. Note that deg z ≤ n if and only if deg(gzz) ≤ n. If x is accessible from x by Bn , then there exist L > 0, z1 , . . . zL ∈ Bn , ε1 , . . . , εL ∈ {−1, 1}, satisfying L
x = x + ∑ εs z s , s=1
l
x + ∑ εs z s ∈ Ft
for 1 ≤ l ≤ L.
s=1
Applying g to both sides of the equations we get L
gxx = gxx + ∑ εs gzzs , s=1
l
gxx + ∑ εs gzzs ∈ Fgtt
for 1 ≤ l ≤ L .
s=1
Because gzzs ∈ Bn for s = 1, . . . , L, the lemma is proved.
This lemma holds for all g ∈ G. In particular, gxx ∈ Ft (xx) if g ∈ Gt . This implies that an action of Gt is induced on Ht . In the sequel let Xγ ∈ Ht denote each equivalence class: Ht = {Xγ }1≤γ ≤|Ht | .
104
7 Symmetry of Markov Bases
Fig. 7.1 A direct product structure of FG(tt ) (a = 3, b = 2, p = 1, qi = 2, ri = 2)
G t(x)
Xγ
x Ft g
gx
g’
g’x
G(x) FG(t)
Let π : x → Xγ denote the natural projection of x to its equivalence class. Then Lemma 7.4 states
π (xx) = π (xx ) ⇒ π (gxx) = π (gxx ). Let x ∈ Xγ and g ∈ Gt . Then gxx belongs to some Bn−1 -equivalence class Xγ . By Lemma 7.4, this γ does not depend on the choice of x ∈ Xγ and we may write γ = gγ . By definition a group action is bijective, therefore the following lemma holds. Lemma 7.5. g ∈ Gt : Xγ → Xγ is a bijection of Ht to itself. Now we give a proof of Theorem 7.1. Proof (Theorem 7.1). Let B be a minimal invariant Markov basis and consider the partition (7.14). Then each Bn,α , α ∈ Tn /G, is G-invariant from Lemma 7.2. Moreover, from the argument of Chap. 5, each z = z + − z − ∈ Bn,α is a move connecting Xγ ∈ Ht and Xγ ∈ Ht , γ = γ , that is, z + ∈ Xγ and z − ∈ Xγ , from the minimality of B. In this case, gzz = gzz+ − gzz− is a move connecting Xgγ and Xgγ . Applying g−1 the converse is also true. This implies that the way Bn,α ∩ Bt connects the Bn−1 -equivalence classes Ht is the same for all t ∈ α and hence the relation (7.15) holds. Conversely, to construct a minimal invariant Markov basis, we only have to consider sets of moves connecting B|tt |−1 -equivalence classes of each Ft from the argument of Chap. 5. Considering the isomorphic structure (7.20) of Lemma 7.3 and Lemma 7.5, we see that the structure of Ht is common for all t ∈ G(tt ). Therefore it suffices to consider the Gt -invariant set of moves Bt for some representative sufficient statistic t ∈ α satisfying |Ht | ≥ 2 for each α ∈ Tn /G. Here we give an illustration of a direct product structure of FG(tt ) . Figure 7.1 shows a structure of FG(tt ) where a = a(tt ) = |G(tt )| = 3 and b = b(tt ) = |FG(tt ) / G| = 2. In each Ft ⊂ FG(tt ) , there are two B|tt |−1 -equivalence classes: |Ht | = 2.
7.6 Characterizations of a Minimal Invariant Markov Basis
105
Figure 7.1 also shows Gt orbits in each Ft . In fact, Fig. 7.1 is derived from an example of 2 × 2 × 2 × 2 contingency tables, where the following marginals are fixed: D1 = {1, 2},
D2 = {1, 3},
D3 = {2, 3},
D4 = {3, 4}.
We see the above structure by considering x = (1111)(1221)(2122)(2212), for example. In this case, Ft (xx) is an eight-element set as follows. (1111)(1221)(2122)(2212), (1111)(1222)(2121)(2212), Xγ ( x ) (1112)(1222)(2121)(2211), (1112)(1221)(2122)(2211), (1121)(1211)(2112)(2222), (1121)(1212)(2111)(2222), (1122)(1212)(2111)(2221), (1122)(1211)(2112)(2221).
Gt (xx)
In this chapter we have discussed properties of minimal invariant Markov bases. Then a natural question is to seek some conditions for the uniqueness of a minimal invariant Markov basis. In [13] we gave some characterizations of the uniqueness of a minimal invariant Markov basis. However, the characterizations are not simple and the argument is rather long. Therefore we omit discussion of uniqueness of a minimal invariant Markov basis.
Part III
Markov Bases for Specific Models
In Part III of this book, we present results on Markov bases for some specific models, which are important for applications. We give many numerical examples to illustrate the application of Markov basis methodology to practical statistical problems. In Chap. 8 we give a thorough discussion of Markov bases for decomposable models of contingency tables. For decomposable models we have a complete description of minimal Markov bases and minimal invariant Markov bases. In Chap. 9 we discuss Markov bases for no-three-factor interaction models of three-way contingency tables and some other hierarchical models. We see that for general hierarchical models the structure of Markov bases is very complicated. In Chap. 10 we discuss two-way tables with structural zeros and fixed subtable sums. We give explicit forms of Markov bases and give some numerical examples of a running Markov chain with the obtained Markov bases. In Chap. 11 we explain applications of the Markov basis approach to experimental designs, where the response variables are discrete. In standard textbooks on experimental design, the response variables are usually assumed to be normally distributed. When response variables are discrete it is more appropriate to use exact tests. We give many numerical examples, because this topic is of practical importance. In Chap. 12 we introduce groupwise selection models, where the Gr¨obner basis approach works particularly well and testing these models can be performed easily. We illustrate the use of these models by analyzing educational and allele frequency data. Finally in Chap. 13 we study the problem of connecting some specific fibers by a subset of a Markov basis. In some problems, when we consider connectivity of specific fibers, it is possible to describe a subset of a Markov basis that connects these fibers. A typical example is the logistic regression model with positive sample size for each level of a covariate.
Chapter 8
Decomposable Models of Contingency Tables
8.1 Chordal Graphs and Decomposable Models In this section we summarize some properties of the decomposable model and chordal graphs according to Lauritzen [97] and Hara and Takemura [74, 75, 76]. We use the notation of hierarchical models introduced in Sect. 1.5. Let Δ = [m] = {1, . . . , m} denote the set of variables of an m-way contingency table x = {x(ii) | i ∈ I }. Let D = {D1 , . . . , Dr } be the set of facets of a simplicial complex K such that Δ = ∪rj=1 D j . Let p(ii) denote the cell probability for i . Then the hierarchical model for D is defined as log p(ii) = ∑ μD (ii), D∈D
where μD depends only on i D . D is called a generating class for the model. In this chapter, we often identify a hierarchical model with its generating class D. As defined in Sect. 1.4, for a subset of the variables V ⊂ Δ , let xV and zV denote the V -marginal sums of x and z with entries given by xV (iiV ) =
∑
iV C ∈IV C
x(iiV , iV C ),
zV (iiV ) =
∑
iV C ∈IV C
z(iiV , iV C )
for iV ∈ IV = ∏δ ∈V Iδ . We often denote i = (iiV , iV C ) by appropriately reordering indices. A sufficient statistic t for D is the set of marginal sums for all D ∈ D, t = {xxD | D ∈ D}. Hence a move z for the generating class D satisfies zD = 0 for all D ∈ D. Marginal tables xD1 , . . . , xDr are called consistent if, for any r1 , r2 , (Dr1 ∩ Dr2 )marginal of xDr1 is equal to the (Dr1 ∩ Dr2 )-marginal of xDr2 ([52]). The consistency of the marginal tables is obviously a necessary condition for the existence of x. However, it does not necessarily guarantee the existence of x in general (e.g., [46, 91, 148]). This is closely related to the notion of normality of semigroups given in Sect. 4.3. We again discuss normality in Sect. 9.5. S. Aoki et al., Markov Bases in Algebraic Statistics, Springer Series in Statistics 199, DOI 10.1007/978-1-4614-3719-2 8, © Springer Science+Business Media New York 2012
109
110
8 Decomposable Models of Contingency Tables
Let G be a graph with the vertex set Δ and an edge between δ , δ ∈ Δ if and only if there exists D ∈ D such that δ , δ ∈ D. G is called the independence graph of D (e.g., Dobra and Sullivant [54]). For V ⊂ Δ , denote by G (V ) the subgraph induced by V ; that is, V is the set of vertices of G (V ) and the edges of G (V ) are those in G restricted to V . V ⊂ Δ is called a clique if G (V ) is complete; that is, every pair of vertices in V is an edge of G . A clique V of G is called maximal if every proper superset of V is not a clique of G . A hierarchical model for D is called graphical if there exists a graph whose set of maximal cliques is given by D (e.g., Edwards [57]). A graphical model is called decomposable if G is chordal; that is, every cycle of G with length greater than three has a chord. A clique tree T = (D, E ) of G is a tree with the vertex set D satisfying D ∩ D ⊂ D
for all D on the path between D and D in T .
A graph is chordal if and only if there exists a clique tree of it ([30, 64]). When (D, D ) ∈ E , S = D ∩ D is called a minimal vertex separator of G . Let S be the multiset S := {D ∩ D | (D, D ) ∈ E }, where the same minimal vertex separator may be included several times (e.g., [97]). Denote a marginal probability for i D by pD (iiD ). Then p(ii) and its maximum likelihood estimator p(i ˆ i) are written by p(ii) =
∏D∈D pD (iiD ) , ∏S∈S pS (iiS )
p(i ˆ i) =
∏D∈D xD (iiD ) , n ∏S∈S xS (iiS )
respectively. A vertex is called simplicial if its adjacent vertices form a clique of G . Any chordal graph with at least two vertices has at least two simplicial vertices and if the graph is not complete, these can be chosen to be nonadjacent ([51]). For D ∈ D of a decomposable model, let Simp(D) denote the set of simplicial vertices in D and let Sep(D) denote the set of nonsimplicial vertices in D. If Simp(D) = 0, / D is called a simplicial clique. A simplicial clique D is called a boundary clique if there exists another clique D ∈ D such that Sep(D) = D ∩ D ([137]). Simplicial vertices in boundary cliques are called simply separated vertices. A maximal clique D is a boundary clique if and only if there exists a clique tree such that D is its endpoint ([74]). Dobra [52] showed that decomposable models have a Markov basis consisting of only square-free moves of degree 2. In the next section, we give a proof of this fact. For convenience, denote a square-free move z of degree 2 with z(ii) = z( j ) = 1 and z(ii ) = z( j ) = −1 by z = i j − i j . This notation was already used in Sect. 7.2. Similarly, x = i j denotes a frequency vector with one frequency at cells i and j .
8.2 Markov Bases for Decomposable Models
111
8.2 Markov Bases for Decomposable Models The simplest decomposable model is the two-way complete independence model D = {{1}, {2}}. Consider an R ×C table. A sufficient statistics for this model is the set of row sums and column sums, thus every move z = {zi j } satisfies zi+ =
C
∑ zi j = 0,
j=1
R
z+ j = ∑ zi j = 0. i=1
In Theorem 2.1 we saw that the set of the following degree 2 moves z (i, i ; j, j ) := (i j)(i j ) − (i j )(i j),
1 ≤ i < i ≤ R,
1 ≤ j < j ≤ C
(8.1)
forms a Markov basis for R × C two-way complete independence models. Consider a decomposable model consisting of two maximal cliques D = {D, D }. Denote A := D \ D , B := D \ D, and S := D ∩ D . When A = {1}, B = {2}, and S = 0, / the model coincides with the two-way complete independence model. When D = {1, 2}, D = {2, 3}, A = {1}, B = {3}, and S = {2}, the model coincides with the conditional independence model of three-way contingency tables in Sect. 1.4. Proposition 8.1. Define B(D, D ) by the following set of square-free moves of degree 2, B(D, D ) = {(iiA i S i B )(ii A i S i B ) − (iiA i S i B )(iiA i S i B ) | i A , i A ∈ IA , i B , i B ∈ IB , i S ∈ IS }. Then B(D, D ) forms a Markov basis for D = {D, D }. Proof. Let x , y (yy = x ) be two tables in the same fiber of the model D and let z = y − x . Denote by z i S the i S -slice of z : z i S = {z(iiA i S i B ) | i A ∈ IA , i B ∈ IB }. Assume z i S = 0 without loss of generality. Consider z i S as a two-way integer array i i with the set of levels IA × IB . Let z AS and z BS denote the A-marginal table and the Bmarginal table of z i S , respectively. The assumption that z D = 0 and z D = 0 implies i i that z AS = 0 and z BS = 0 . Therefore z i S is regarded as a move of a two-way complete independence model. Hence from Theorem 2.1 we can reduce |zz|1 by a square-free move of degree 2 of the form (iiA i S i B )(iiA i S i B ) − (iiA i S i B )(iiA i S i B ).
The above arguments are generalized to general decomposable models. Let T be a clique tree of G . Denote by Te = (De , Ee ) and Te = (De , Ee ) the two induced subtrees of T obtained by removing an edge e ∈ E from T . Let Ve and Ve be Ve =
D∈De
D,
Ve =
D∈De
D.
112
8 Decomposable Models of Contingency Tables
Then Te and Te are clique trees of chordal graphs G (Ve ) and G (Ve ), respectively. Define the set B T of square-free moves of degree 2 as BT =
B(Ve ,Ve ).
(8.2)
e∈E
Denote Se := Ve ∩Ve , Re := Ve \ Se and Re := Ve \ Se . Lemma 8.1. Suppose that z ∗ = (iiRe i Se )(iiRe i Se ) − ( j Re i Se )( j Re i Se ) ∈ B Te , i Re , i Re , j Re , j Re ∈ IRe ,
i Se , i Se ∈ ISe
is a move of De . Then z = (iiRe i Se i Re )(iiRe i Se i Re ) − ( j Re i Se i Re )( j Re i Se i Re ) ∈ B T for any i Re , i Re ∈ IRe . Proof. Obviously zVe = 0 and hence zD = 0 for all D ∈ De . Because zVe = z∗ , we also have z D = 0 for all D ∈ De . Hence z is a move for D = De ∪ De . Because zVe = 0, there exists an edge e∗ ∈ Ee such that z ∈ B(Ve∗ ,Ve∗ ).
Theorem 8.1 (Dobra [52]). B T forms a Markov basis of the decomposable model D. Proof. The proof is by induction on the number of maximal cliques r. When r = 2, B T coincides with the Markov basis in Proposition 8.1. Suppose that the theorem holds for any decomposable models with r − 1 maximal cliques. Let x , y (yy = x ) be two tables in the same fiber F of the decomposable model D. Let D ∈ D be an endpoint of T and suppose that e := (D, D ) ∈ E . Then we can set Ve = Δ \ (D \ D),
Ve = D,
De = D \ {D},
De = {D}
and define Te as above. Then the marginal tables xVe and yVe lie in the same fiber F of De . From the inductive assumption, B Te is a Markov basis for De . Hence there exists a sequence of moves zV1e , . . . , zVl e such that l
yVe = xVe + ∑ zVk e , k=1
l
xVe + ∑ zVk e ∈ F k=1
for 1 ≤ l ≤ l. From Lemma 8.1, there exists a sequence of moves z 1 , . . . , z l of D such that l
x + ∑ zk ∈ F k=1
8.3 Structure of Degree 2 Fibers
113
for 1 ≤ l ≤ l. Define y = x + ∑lk=1 z k . Then y and y satisfy yVe = yV e and yVe = yV . e Hence y and y are accessible by moves in B(Ve ,Ve ).
Dobra [52] proposed the following algorithm for generating moves from B T . Algorithm 8.1 (Dobra [52]) 1. For each edge e ∈ E of T : a. Define Ve , Ve and Se as above. b. Calculate the weights we representing the number of degree 2 moves: ∏δ ∈Se Iδ Iδ Iδ . we ← 2 · ∏ · ∏ 2 2 δ ∈V \S δ ∈V \S e
e
e
e
2. Normalize the weights w2 , . . . , wr . 3. Randomly select an edge e ∈ E with probability we . 4. Uniformly pick up a move in B(Ve ,Ve ).
8.3 Structure of Degree 2 Fibers In the previous section we showed that every decomposable model has a Markov basis consisting of square-free moves of degree 2. As discussed in Sect. 5.3, the set of fibers of the minimum fiber Markov basis for a decomposable model coincides with the set of degree 2 fibers with more than one element. Therefore the structure of degree 2 moves is equivalent to that of degree 2 fibers. In this section, we discuss the structure of such fibers in detail this section is mainly based on [71]. Let Ft be a fiber with degtt = 2. For a given t we say that a variable δ ∈ Δ is degenerate if there exists a unique level iδ such that x{δ } (iδ ) = 2. Otherwise, if there exist two levels iδ = iδ such that x{δ } (iδ ) = x{δ } (iδ ) = 1, then we say that δ is nondegenerate. Degeneracy or nondegeneracy of δ does not depend on a particular x ∈ Ft , because one-dimensional marginals are determined from marginals of the facets x D , D ∈ D. If all the variables δ ∈ Δ are degenerate, then Ft = {xx} is a one-element fiber with frequency x(ii) = 2 at a particular cell i . This case is trivial, therefore below we consider the case that at least one variable is nondegenerate. From the fact that there exist at most two levels with positive one-dimensional marginals for each variable, it follows that we only need to consider 2 × · · · × 2 tables for studying degree 2 fibers. Therefore we set I1 = · · · = Im = 2, I = {0, 1}m without loss of generality. For a given t of degree 2, let Δ¯t denote the set of nondegenerate variables. As noted above, we assume that Δ¯t = 0. / Each x ∈ Ft is of the form x(ii) = x(ii ) = 1 for i = i and remaining entries are 0. For nondegenerate δ ∈ Δ¯t the levels of the variable δ in i and i are different: {iδ , iδ } = {0, 1},
∀δ ∈ Δ¯t ,
114
8 Decomposable Models of Contingency Tables
or equivalently iδ = 1 − iδ , ∀δ ∈ Δ¯t . In the following, we use the notation i∗δ = 1 − iδ . More generally, for a subset of variables V = {δ1 , . . . , δk } and a marginal cell iV = (iδ1 , . . . , iδk ), we write iV∗ ≡ (i∗δ1 , . . . , i∗δk ) = (1 − iδ1 , . . . , 1 − iδk ). Let us identify x = i i ∈ Ft with the set {ii, i } of its two cells of frequency one. ¯ Then we see that the number of elements of fibers |Ft | is at most 2|Δt |−1 . Let G (Δ¯t ) ¯ be the subgraph of G induced by Δt ⊂ Δ . Lemma 8.2. Suppose that t is a set of consistent marginal frequencies of a contingency table with degtt = 2. Let Γ be any subset of a connected component in G (Δ¯t ). Then the marginal table xΓ = {xΓ (iiΓ ) | iΓ ∈ IΓ } is uniquely determined. = 0. / We Proof. Let r(Γ ) be the number of generating sets D ∈ D satisfying Γ ∩ D prove this lemma by induction on r(Γ ). When r(Γ ) = 1, the lemma obviously holds. Suppose that the lemma holds for all r(Γ ) < r and we now assume that r(Γ ) = r . Let Γ1 ⊂ Γ and Γ2 ⊂ Γ satisfy
Γ1 ∪ Γ2 = Γ ,
Γ1 ∩ Γ2 = 0, /
r(Γ1 ) < r ,
r(Γ2 ) < r .
Because r(Γ1 ) < r and r(Γ2 ) < r , both xΓ1 and xΓ2 are uniquely determined. Suppose that xΓ1 (iiΓ1 \Γ2 , iΓ1 ∩Γ2 ) = 1, xΓ1 (iiΓ∗1 \Γ2 , iΓ∗1 ∩Γ2 ) = 1. (8.3) Then from the consistency of t , there uniquely exists iΓ2 \Γ1 ∈ IΓ2 \Γ1 , such that xΓ2 (iiΓ2 \Γ1 , iΓ1 ∩Γ2 ) = 1,
xΓ2 (iiΓ∗2 \Γ1 , iΓ∗1 ∩Γ2 ) = 1.
(8.4)
Hence the table xΓ = {x( j Γ ) | j Γ ∈ IΓ } with entries x( j Γ ) =
1, if j Γ = (iiΓ1 \Γ2 , iΓ1 ∩Γ2 , iΓ2 \Γ1 ) or j Γ = (iiΓ∗1 \Γ2 , iΓ∗1 ∩Γ2 , iΓ∗2 \Γ1 ), 0, otherwise
is consistent with t . Suppose that there exists another marginal table xΓ which is consistent with t such that xΓ ( j Γ ) = xΓ ( j Γ∗ ) = 1 and j Γ = (iiΓ1 \Γ2 , iΓ1 ∩Γ2 , iΓ2 \Γ1 ). Then we have at least one of xΓ1 (iiΓ1 ) = 0 or xΓ2 (iiΓ2 ) = 0.
This contradicts (8.3) and (8.4).
= 0/ and let c(tt ) be the number Theorem 8.2. Let Ft be a degree 2 fiber such that Δ¯t of connected components of G (Δ¯t ). Then |Ft | = 2c(tt )−1 .
8.4 Minimal Markov Bases for Decomposable Models
115
Proof. Denote by Γ1 , . . . , Γc , c = c(tt ), the connected components of G (Δ¯t ). Define Γc+1 := Δ \ Δ¯ t . By definition, there exists iΓc+1 such that iΓc+1 = {iiδ | δ ∈ Γc+1 , x{δ } (iiδ ) = 2}. From Lemma 8.2, the marginal cells iΓk , k = 1, . . . , c, satisfying xΓk (iiΓk ) = xΓk (iiΓ∗k ) = 1 uniquely exist. Now define It by It = {iiΓ1 , iΓ∗1 } × {iiΓ2 , iΓ∗2 } × · · · × {iiΓc , iΓ∗c } × {iiΓc+1 }, where × denotes the direct product of sets. Suppose that j ∈ It . Define 1, if i = j or i = j ∗ , j j j x = {x (ii) | i ∈ I }, x (ii) = 0, otherwise. Then we have F (It ) := {xx j | j ∈ It } ⊂ Ft and |F (It )| = 2c−1 . Suppose that x ∈ F (It ). If there exists x = {x (ii) | i ∈ I } such that x ∈ Ft , x ∈ / F (It ), there exists a cell j ∈ I and 1 ≤ k ≤ c + 1 such that x ( j ) = 1 and j Γk = iΓk . This implies that there exists Dl ∈ D such that x Dl (iiDl ) = x Dl (iiDl ). Hence we have |Ft | = 2c−1 .
As mentioned in Sect. 8.1, for a consistent t such that degtt > 2, the fiber Ft may be empty in general. However Theorem 8.2 shows that, in the case of degtt = 2, if a consistent t such that Δ¯t = 0/ is given, then Ft = 0/ for any hierarchical model. We also note that Theorem 8.2 holds for general hierarchical models.
8.4 Minimal Markov Bases for Decomposable Models In this section we discuss Markov bases for decomposable models from a viewpoint of minimality (cf. Chap. 5). Let degtt = 2. Let Tt be any tree whose nodes are elements of Ft . Denote the set of edges in Tt by BTt . We can identify each edge (xx, x ) ∈ BTt with a move z = x − x . So we identity BTt with a set of moves for Ft . In this section, we consider only sign invariant Markov bases. Hence identify z = x − x with −zz = x − x and consider the edges in Tt as undirected. Let Bnd be Bnd = {tt | degtt = 2, |Ft | ≥ 2}.
(8.5)
As mentioned above, the set of fibers of the minimum fiber Markov basis for decomposable models coincides with the set of degree 2 fibers with more than one element. Hence we can provide the complete description of minimal Markov bases for decomposable models as follows. Theorem 8.3. Define B 0 by B0 =
t ∈Bnd
BTt .
(8.6)
116
8 Decomposable Models of Contingency Tables BTt
1
(000)(111)
(001)(110)
(011)(100)
(010)(101)
BTt
BTt
2
3
(000)(110) BTt
(001)(111)
(011)(101)
BTt
4
5
(000)(011)
BTt
(010)(100)
(010)(001)
(100)(111)
(110)(101)
BTt
6
7
(000)(101)
(001)(100)
(010)(111)
(011)(110)
Fig. 8.1 BTt l in the complete independence model of three-way contingency tables
Then B 0 is a minimal Markov basis and (8.6) is a disjoint union. Conversely every minimal Markov basis can be written as in (8.6). Example 8.1 (The complete independence model of 2 × 2 × 2 contingency tables). Consider the model D = {{1}, {2}, {3}}. Bnd for the model has seven elements. Denote them by t 1 , . . . ,tt 7 . Figure 8.1 shows an example of BTt l for t = 1, . . . , 7. t 1 , . . . ,tt 7 satisfy
Δ¯t 1 = {1, 2, 3},
Δ¯t 2 = Δ¯t 3 = {1, 2},
Δ¯t 4 = Δ¯t 5 = {2, 3}, Δ¯t 6 = Δ¯t 7 = {1, 3}.
(8.7)
The union of all these moves is a minimal Markov basis for the model. Inasmuch as Ft 1 is a four-element fiber, Tt 1 is not uniquely determined. Hence minimal Markov bases are not unique for this model. As seen from this example, minimal Markov bases are not necessarily uniquely determined. The following corollary provides a necessary and sufficient condition on decomposable models to have the unique minimal Markov basis. Corollary 8.1. There exists the unique minimal Markov basis for a decomposable model if and only if the number of connected components in any induced subgraph of G is less than three. Proof. Suppose that G (Δ¯t ) has more than two connected components. Then because |Ft | ≥ 4 from Theorem 8.2, Tt is not uniquely determined. For a different tree Tt , BTt = BTt . Hence minimal Markov bases are not unique either.
8.4 Minimal Markov Bases for Decomposable Models
117
2
1
2
3
4
1
4
3
6
5
Fig. 8.2 Examples of chordal graphs satisfying the condition of Corollary 8.2
Conversely assume that the number of connected components of G (Δ¯t ) for any t ∈ Bnd is two. Then Tt for any t ∈ Bnd is uniquely determined. Hence the minimal Markov basis is also unique.
Corollary 8.2. For a decomposable model, there exists the unique minimal Markov basis if and only if G has only two boundary cliques D and D such that D ⊂ D ∪ D for all D ∈ D. Proof. Suppose that G has two boundary cliques D and D such that D ⊂ D ∪ D for all D ∈ D. Then any vertex in D is adjacent to D or D . Hence the number of connected components for any induced subgraph of G is at most two. Conversely suppose that there exists D ∈ D such that D ⊂ D ∪ D . Then the subgraph induced by the union of D \ (D ∪ D ), Simp(D) and Simp(D ) has three connected components.
The graphs with r = 2 always satisfy the conditions of the corollary. For r ≥ 3 the graph with D = {{1, . . ., r − 1}, {2, . . ., r}, . . . , {r, . . . , 2r − 2}}
(8.8)
satisfies the conditions of the corollary. Figure 8.2 shows the graphs satisfying (8.8) for r = 3, 4. We can easily see that any induced subgraph of the graphs in the figure has at most two connected components. From a viewpoint of minimality, Dobra’s Markov basis B T is characterized as follows. Theorem 8.4. A decomposable model has a clique tree T such that B T is a minimal Markov basis if and only if the model has the unique minimal Markov basis. Proof. When a decomposable model has a unique minimal Markov basis, B T coincides with it. Suppose that there exist three vertices in G which are not adjacent to one another. Let 1, 2, and 3 be three such vertices and assume that l ∈ Dl , Dl ∈ D, for l = 1, 2, 3. Define {1, 2, 3}C = Δ \ {1, 2, 3}. Consider a degree 2 fiber Ft such that Δ¯t = {1, 2, 3} and x{1,2,3}C (ii{1,2,3}C ) = 2 for some i {1,2,3}C . Then |Ft | = 4 from Theorem 8.2 and we can denote the four elements by
118
8 Decomposable Models of Contingency Tables
Fig. 8.3 T in Example 8.2
D1
Fig. 8.4 BtT for t such that Δ¯ t = {1, 2, 3, 4}
D2
D3
D4
(0000)(1111) (0111)(1000)
(0001)(1110)
(0110)(1001)
(0010)(1101)
(0101)(1010)
(0011)(1100) (0100)(1011)
x 1 = (000 i {1,2,3}C )(111 i {1,2,3}C ), x 2 = (001 i {1,2,3}C )(110 i {1,2,3}C ), x 3 = (010 i {1,2,3}C )(101 i {1,2,3}C ), x 4 = (011 i {1,2,3}C )(100 i {1,2,3}C ). (8.9) A minimal Markov basis connects these four elements by three moves. Let T = (D, E ) be any clique tree for G and T = (D , E ) be the smallest subtree of T satisfying Dl ∈ D for l = 1, 2, and 3. Then we can assume that T satisfies either of the following two conditions, (i) D2 is an interior point and D1 and D3 are endpoints on the path. (ii) All of D1 , D2 , and D3 are endpoints of T . In both cases there exists e ∈ E such that D1 , D2 ⊂ Ve and D3 ⊂ Ve . Then B T (Ve ,Ve ) includes the following two moves, z1 = x1 − x2 ,
z2 = x 3 − x4 .
exists e
∈ E such that D1 ⊂ Ve and D2 , D3 ⊂ Ve . In On the other hand there also this case B T (Ve ,Ve ) includes the following two moves, z3 = x1 − x4 ,
z4 = x 2 − x3 .
Thus B T includes at least four moves for the fiber Ft , which implies that B T is not minimal for the model which does not have the unique minimal Markov basis.
Example 8.2 (The complete independence model of 2 × 2 × 2 × 2 contingency tables). Consider the model D = {{1}, {2}, {3}, {4}} and Dl = {l} for l = 1, . . . , 4. Let Ft be the fiber with Δ¯t = {1, 2, 3, 4}; that is, c(tt ) = 4 and |Ft | = 8. Consider B T for T in Fig. 8.3. Denote the set of moves for Ft belonging to B T by BtT . Figure 8.4 shows BtT . As seen from Fig. 8.4, BtT includes 12 moves. Because |Ft | = 8, 7 moves are sufficient to connect Ft .
8.5 Minimal Invariant Markov Bases
119
8.5 Minimal Invariant Markov Bases In this section we discuss Markov bases for decomposable models from the viewpoint of invariance under the action of the direct product of symmetric groups G = GI1 ,...,Im = SI1 × · · · × SIm on the levels of the variables and provide a minimal G-invariant Markov basis. Here we denote c = c(tt ) for simplicity. Let Γl , l = 1, . . . , c, be connected components of G (Δ¯t ) and let Γc+1 = Δ \ Δ¯t . For a subset of vertices V ⊂ Δ , denote |V |
0V := 0 · · · 0,
|V |
1V := 1 · · · 1 .
As a representative fiber Ft0 , we can consider t such that the levels of all degenerate variables are determined as 0: Ft0 xt0 ≡ (00 Δ )(11Δ¯t 0Γc+1 ). Then any x ∈ Ft0 is expressed as follows, x = (00Γ1 iΓ2 · · · iΓc 0Γc+1 )(11Γ1 iΓ∗2 · · · iΓ∗c 0Γc+1 ), iΓl = 0Γl
or iΓl = 1Γl ,
l = 2, . . . , c. |Γ |
Let GΓl , l = 2, . . . , c, be the diagonal subgroup of S2 l defined by |Γ |
GΓl = {g¯ = (g, . . . , g) | g ∈ S2 } ⊂ S2 l . Define Gt = GΓ2 × · · · × GΓc and let g ∈ Gt act on x ∈ Ft0 by g(xx) = (00Γ1 g¯2 (iiΓ2 ) · · · g¯c (iiΓc ) 0Γc+1 )(11Γ1 g¯2 (iiΓ∗2 ) · · · g¯c (iiΓ∗c ) 0Γc+1 ). Clearly g(xx) ∈ Ft0 for x ∈ Ft0 and furthermore for any x ∈ Ft0 there exists g ∈ Gt such that x = g(xxt0 ). This shows that Gt ⊂ GI1 ,...,Im is the setwise stabilizer of Ft0 acting transitively on Ft0 . Then Gt ⊂ GI1 ,...,Im is isomorphic to a c-fold direct product of S2 s: S2c = S2 × · · · × S2 . Therefore the structure of Ft is equivalent to the structure of the fiber Ft with Δ = Δ¯t = {1, . . . , c}. Let BGt be a minimal Gt -invariant set of moves that connects Ft0 . Let κ (tt ) be the number of Gt -orbits included in BGt . As representative moves of Gt -orbits in BGt we can consider ztk = xt0 − xtk ∈ Bt ,
xtk ∈ Ft0 ,
k = 1, . . . , κ (tt ).
120
8 Decomposable Models of Contingency Tables
This is because we can always send x in z = x − x to xt0 by the transitivity of Gt . Denote BG0 t = {zzt1 , . . . , ztκ (tt ) }. Define the set of t that induces representative fibers by 0 = {tt | xt0 ∈ Ft0 } ⊂ Bnd . Bnd
(8.10)
By Theorem 7.1 a minimal GI1 ,...,Im -invariant Markov basis can be expressed by BG =
(tt ) κ
GI1 ,...,Im (zztk ),
(8.11)
0 k=1 t ∈Bnd
where GI1 ,...,Im (zztk ) denotes the GI1 ,...,Im -orbit through ztk . Hence in order to clarify the structure of BG , it suffices to consider 2 × · · ·× 2 tables and investigate κ (tt ) and BG0 t for each Ft0 . As mentioned above, the structure of Ft0 is equivalent to the one of the fiber with Δ¯t = Δ = {1, . . . , c} and G (Δ¯t ) is totally disconnected. We first consider the structure of such a fiber. Ft0 satisfies Ft0 = {(0 i 2 · · · i c )(1 i ∗2 · · · i ∗c ) | (ii2 · · · i c ) = i Δ \{1} ∈ IΔ \{1} }
(8.12)
and (0 · · · 0)(1 · · · 1) ∈ Ft0 . Then we can identify Gt with S2c−1 . For g ∈ S2c−1 , we write g = (g1 , . . . , gc ), where gl ∈ S2 for l = 1, . . . , c. A representative move of an S2c−1 -orbit is written by zt = (0 · · · 0)(1 · · · 1) − (0 i Δ \{1} )(1 i ∗Δ \{1}) for some i Δ \{1} ∈ IΔ \{1} . We first consider deriving κ (tt ) and BGt . Let V c−1 = {0, 1}c−1 denote the (c − 1)-dimensional vector space over the finite field GF(2), where the addition of two vectors is defined to be XOR of the elements. Let ⊕ denote the XOR operation. Let ◦ denote the group operation of S2c−1 . Lemma 8.3. S2c−1 is isomorphic to V c−1 . Proof. Consider the map φ : S2c−1 → V c−1 such that φ (g) = v = (v2 , . . . , vc ) ∈ V c−1 , where 0, if gl (iil ) = i l , vl = 1, if gl (iil ) = i ∗l , for l = 2, . . . , c and {iil , i ∗l } = {0, 1}. For g = (g2 , . . . , gc ) ∈ S2c−1 , gl ∈ S2 , and v ∈ V c−1 , define φ (g ) = v = (v2 , . . . , vc ). Then we have φ (g ◦ g ) = v˜ = (v˜2 , . . . , v˜c ), v˜ ∈ V c−1 , where 0, if gl ◦ gl (iil ) = i l , v˜l = 1, if gl ◦ gl (iil ) = i ∗l for l = 2, . . . , c. Hence we have v˜l = vl ⊕ vl ,
l = 2, . . . , c.
8.5 Minimal Invariant Markov Bases
121
Therefore φ is a homomorphism. It is obvious that φ is a bijection. Therefore S2c−1 is isomorphic to V c−1 .
Based on this lemma, we can show the equivalence between S2c -orbits in a minimal S2c -invariant set of moves that connects Ft0 and a (vector space) basis of V c−1 . Theorem 8.5. Let V 0 = {vvk = (vk2 , . . . , vkc ), k = 2, . . . , c} be any basis of V c−1 . Define xt0 , xtv k ∈ Ft0 by xt0 = (00 · · · 0)(11 · · · 1),
xtv k = (0 vk2 · · · vkc )(1 v∗k2 · · · v∗kc ),
where v∗kl = 1 ⊕ vkl . Let BGt be an S2c−1 -invariant set of moves in Ft0 . Then BGt is a minimal S2c−1 -invariant set of moves that connects Ft0 if and only if the representative moves of the S2c−1 -orbits in BGt are expressed by ztv k = xt0 − xtv k , k = 2, . . . , c. Hence κ (tt ) = c − 1. Proof. Suppose that BGt is a minimal S2c−1 -invariant set of moves that connects Ft and that BGt includes κ (tt ) orbits S2c−1 (zzt1 ), . . . , S2c−1 (zztκ (tt ) ), where ztk = xt0 − xtk ,
xtk = (0 i k2 · · · i kc )(1 i ∗k2 · · · i ∗kc )
for i kl ∈ Il , k = 1, . . . , κ (tt ), l = 2, . . . , c. Let gk ∈ S2c−1 satisfy gk (xxt0 ) = xtk for k = 1, . . . , κ (tt ). We write gk = (gk2 , . . . , gkc ), gkl ∈ S2 for l = 2, . . . , c. Let Ht = {g1 , . . . , gκ (tt ) } ⊂ S2c−1 be a subset of S2c−1 . As mentioned above, Ft0 can be expressed as in (8.12). Hence for any x ∈ Ft0 there exists g ∈ S2c−1 satisfying x = g(xxt0 ). BGt connects Ft0 if and only if there exists p ≤ κ (tt ) such that x = xt0 − ztk1 − gk1 (zztk2 ) − · · · − gk p−1 ◦ · · · ◦ gk1 (zztk p ) and g = gk p ◦ · · · ◦ gk1 . Hence BGt is a minimal S2c−1 -invariant set of moves that connects Ft if and only if Ht satisfies ∀g ∈ S2c−1 , ∃p ≤ κ (tt ), ∃gk1 ∈ Ht , . . . , ∃gk p ∈ Ht
s.t. g = gk p ◦ · · · ◦ gk1
(8.13)
and no proper subset of Ht satisfies (8.13). Denote V 0 = φ (Ht ) ⊂ V c−1 . From Lemma 8.3, (8.13) is equivalent to ∀vv ∈ V ,
∃vv1 ∈ V 0 , . . . , ∃vv p ∈ V 0
s.t.
v = v1 ⊕ · · · ⊕ v p.
(8.14)
From the minimality of BGt no proper subset of V 0 satisfies (8.14). This implies that V 0 is a basis of V c−1 and hence κ (tt ) = c − 1. If we define gk = φ −1 (vvk+1 ) for k = 1, . . . , c − 1, we have gkl (0) = vk+1,l and hence gk (xxt0 ) = xtk = xtv k+1 . Therefore ztv k , k = 2, . . . , c, are the representative moves of the S2c−1 -orbits in BGt .
122
8 Decomposable Models of Contingency Tables
Conversely suppose that the representative moves of BGt are ztv k , k = 2, . . . , c. V 0 satisfies (8.14) and no proper subset of V 0 satisfies (8.14). Hence if we define gk = φ −1 (vvk+1 ) and Ht = {g1 , . . . , gc−1 }, Ht satisfies (8.14) and no proper subset of Ht satisfies (8.14). Hence BGt is a minimal S2c−1 -invariant set of moves that connects Ft .
For example, we can set V 0 = {vv2 , . . . , v c } as v 2 = (11 · · · 11),
v 3 = (01 · · · 11),
...,
v c−1 = (00 · · · 011),
v c = (00 · · · 01),
and then the representative moves in a minimal G-invariant Markov basis are z 02 = (00 · · · 0)(11 · · · 1) − (011 · · ·11)(100 · · ·00), z 03 = (00 · · · 0)(11 · · · 1) − (001 · · ·11)(110 · · ·00), .. .
.. .
.. .
z 0c = (00 · · · 0)(11 · · · 1) − (000 · · ·01)(111 · · ·10).
(8.15)
So far we have focused on Ft such that Δ¯t = Δ = {1, . . . , c} and G (Δ¯t ) is totally disconnected. Now we consider a fiber for a general t of a general decomposable model. Define g¯kl ∈ GΓl by 0 · · · 0 if vkl = 0, (8.16) g¯kl (00Γl ) = 1 · · · 1 if vkl = 1 for k = 2, . . . , c and l = 2, . . . , c and define gk ∈ Gt by gk (xx) = (00Γ1 g¯k2 (iiΓ2 ) · · · g¯kc (iiΓc ) 0Γc+1 )(11Γ1 g¯k2 (iiΓ∗2 ) · · · g¯kc (iiΓ∗c ) 0Γc+1 ).
(8.17)
Denote xtv k = gk (xxt0 ) and ztv k = xt0 − xtv k . By following (8.11) and Theorem 8.5, we can easily obtain the following result. Theorem 8.6. BGt is a minimal S2c−1 -invariant set of moves that connects Ft0 if and only if the representative moves of the S2c−1 -orbits in BGt are expressed as ztv k , k = 2, . . . , c. Hence κ (tt ) = c − 1. Then BG =
c
GI1 ,...,Im (zztk )
0 k=2 t ∈Bnd
is a minimal GI1 ,...,Im -invariant Markov basis. Conversely every minimal GI1 ,...,Im invariant Markov basis can be written in this form. Example 8.3 (The complete independence model of three-way contingency tables). 0 = {tt ,tt ,tt ,tt }. Figure 8.5 Define t 1 , . . . ,tt 7 as in Fig. 8.1 of Example 8.1. Then Bnd 1 2 4 6
8.5 Minimal Invariant Markov Bases
123 BG
BG t1
G(BGt ) 1
(000)(111)
(001)(110)
G ((000)(111))
G ((001)(110))
(011)(100)
(010)(101)
G ((011)(100))
G ((010)(101))
G(BG ) t2
BG t2 (000)(110)
(010)(100)
G ((000)(110))
G ((010)(100))
G(BG ) t4
BG 4 t (000)(011)
(010)(001)
G ((000)(011))
G((010)(001))
G(BG ) t6
BG t6 (000)(101)
(001)(100)
G((000)(101))
G((001)(100))
Fig. 8.5 The structure of minimal G2,2,2 -invariant Markov bases for the complete independence model of three-way contingency tables
shows a structure of BG for the I1 × I2 × I3 complete independence model of threeway contingency tables. The left half of the figure shows the structure of BGt t for 2 × 2 × 2 tables. c(tt 1 ) = 3 and hence κ (tt 1 ) = 2. If we set vt1 = (10) and vt2 = (01), we have zt11 = (000)(111) − (010)(101), t
zt21 = (000)(111) − (001)(110).
t
The orbits S22 (zzt11 ) and S22 (zzt21 ) are expressed in dotted lines and solid lines, respectively, in the figure. c(tt l ) = 2 and κ (tt l ) = 1 for l = 2, 4, 6. There exists one orbit in BGt l for l = 2, 4, 6. Then from Theorem 8.6 a minimal G2,2,2 -invariant Markov basis is expressed by t
t
t
t
t
BG = G(zzt11 ) ∪ G(zzt21 ) ∪ G(zzt12 ) ∪ G(zzt14 ) ∪ G(zz16 ). Dobra’s Markov basis B T is characterized from a viewpoint of invariance as follows. Because B T does not depend on the levels of the variables, B T is GI1 ,...,Im invariant. Based on the result of Theorem 8.5, we can show that B T is not always a minimal invariant Markov basis.
124
8 Decomposable Models of Contingency Tables
Fig. 8.6 The clique tree with two endpoints
D2
D1
···
Dr
Theorem 8.7. B T is minimal invariant if and only if T has only two endpoints. Proof. It suffices to show that the theorem holds for 2 × · · · × 2 tables. Suppose that T = (D, E ) has more than two endpoints. Let D1 , D2 , and D3 be three of them. Then they are boundary cliques. Suppose 1, 2, 3 ∈ Δ are simply separated vertices in D1 , D2 , and D3 , respectively. In the same way as the argument in the proof of Theorem 8.4, there exist e, e , e ∈ E such that D1 , D2 ∈ Ve , D3 ∈ Ve , D2 , D3 ∈ Ve , D1 ∈ Ve , D3 , D1 ∈ Ve , D2 ∈ Ve . Consider the moves for the fiber Ft0 for t such that Δ¯t = {1, 2, 3}. Define z 5 and z 6 by z 5 = x1 − x 3 ,
z6 = x2 − x4 ,
where x 1 , . . . , x 4 are defined in (8.9). Then we have z 1 , z 2 ∈ B T (Ve ,Ve ),
z 3 , z 4 ∈ B T (Ve ,Ve ),
z 5 , z 6 ∈ B T (Ve ,Ve ).
We note that {zz1 , z2 }, {zz3 , z4 }, and {zz5 , z6 } are S22 -orbits in BtT . Because κ (tt ) = 2, B T is not minimal invariant. Suppose that T has only two endpoints. Then T is expressed as in Fig. 8.6. Let Γ1 , . . . , Γc be the c connected components of G (Δ¯t ). Suppose that δl ∈ Γl . The structure of Ft0 is equivalent to the one of Ft0 such that Δ¯t = {δ1 , . . . , δc−1 } and G (Δ¯t ) is totally disconnected. So we restrict our consideration to such a fiber. Denote by Ft0 the representative fiber for t . Let = x } Bt = {xx − x | x , x ∈ Ft0 , x denote the set of all moves in Ft0 . Without loss of generality we can assume that δl ∈ Dπ (l) , where π (1) < · · · < π (c(tt )). Define el = (Dl−1 , Dl ) ∈ E , Sl = Dl−1 ∩ Dl , Vl = Vel \ Sl and Vl = Vel \ Sl for l = 2, . . . , c(tt ). Then the moves in B T (Vl ,Vl ) are expressed as z = (iiVl iV i Sl )( j Vl j V , i Sl ) − (iiVl j V i Sl )( j Vl iV , i Sl ), l
iVl , j Vl ∈ IVl ,
l
l
iV , j V ∈ IV , l
l
l
l
i Sl ∈ ISl .
(8.18)
If Vel ∩ Δ¯t = 0/ or Vel ∩ Δ¯t = 0, / then we have B T (Vel ,Vel ) ∩ Bt = 0. / If Vel ∩ Δ¯t = 0/ and Vel ∩ Δ¯t = 0, / then there exists 2 ≤ k(el ) ≤ c(tt ) satisfying δk ∈ Vl for all k < k(el ) and δk ∈ Vl for all k ≥ k(el ). Then
8.5 Minimal Invariant Markov Bases
125
Fig. 8.7 Clique trees for the four-way complete independence model
D1 D2 D1
D2
D3
D4
D3
T1
D1
D1
D2
D2
D3
D3
Fig. 8.8 The structure of BtT
D4 T2
(0000)(1110)
(0010)(1100)
(0110)(1000)
(0100)(1010)
(0000)(1110)
(0010)(1100)
(0110)(1000)
(0100)(1010)
D4
D4
1
B T (Vel ,Vel ) ∩ Bt = S2c−1 (zz0k(el ) ), where z 0k(e ) is defined as in (8.15). Hence we have l
BtT =
el ∈E
B T (Vel ,Vel ) ∩ Bt =
c(tt )
S2c−1 (zz 0k ),
k=2
0 . Hence B T is minimal G which contains c(tt ) − 1 orbits for all t ∈ Bnd I1 ,...,Im invariant.
Example 8.4 (The complete independence model of four-way contingency tables). As an example we consider the 2 × 2 × 2 × 2 complete independence model D = {Dl = {i}, i = 1, . . . , 4}. Both T 1 and T 2 in Fig. 8.7 are clique trees for 1 D. From Theorem 8.7, B T is a minimal S23 -invariant Markov basis. Consider the 0 representative fiber Ft such that Δ¯t = {1, 2, 3}. For j = 1, 2, denote the two induced subtrees of T j obtained by removing the edge el by Telj and Telj . Figure 8.8 shows
126
8 Decomposable Models of Contingency Tables
D1 (0000)(1110)
(0010)(1100)
(0110)(1000)
(0100)(1010)
(0000)(1110)
(0010)(1100)
(0110)(1000)
(0100)(1010)
(0000)(1110)
(0010)(1100)
(0110)(1000)
(0100)(1010)
D4
D3
D2
D1
D4
D2
D3
D1
D4
D2
D3
Fig. 8.9 The structure of BtT
2
Te1l , Te1l and B T (Vel ,Vel ) ∩ Bt . If we remove e3 from T 1 , 1, 2, and 3 are still 1
/ Therefore BtT contains κ (tt ) = 2 connected and hence B T (Ve3 ,Ve3 ) ∩ Bt = 0. orbits. 2 On the other hand because T 2 has three endpoints, B T is not a minimal S23 2 invariant Markov basis. Figure 8.9 shows Te2l , Te2l , and B T (Vel ,Vel ) ∩ Bt . We 1
1
can see that BtT contains three orbits. As seen from this example, in general the minimality of M T depends on clique trees T . 2
Example 8.5. We consider the model defined by the chordal graph in Fig. 8.10. The clique tree of this graph is uniquely determined by T 2 in Fig. 8.7. As seen from this example, there exist decomposable models such that B T for every clique tree T is not minimal GI1 ,...,Im -invariant.
8.6 The Relation Between Minimal and Minimal Invariant Markov Bases
127
Fig. 8.10 A chordal graph whose clique tree is uniquely determined
D1
D2
D4
D3
8.6 The Relation Between Minimal and Minimal Invariant Markov Bases From a practical point of view a GI1 ,...,Im -invariant Markov basis is useful because its representative moves give the most concise expression of a Markov basis. On the other hand a minimal Markov basis is also important because the number of moves contained in it is minimum among Markov bases. Here we consider the relation between a minimal and a minimal GI1 ,...,Im -invariant Markov basis and give an algorithm to obtain a minimal Markov basis from representative moves of a minimal GI1 ,...,Im -invariant Markov basis. As mentioned in the previous section, the set of Gt -orbits in a minimal Gt invariant set BGt of moves that connects Ft0 has a one-to-one correspondence to a basis V 0 of V c−1 . Define g¯kl ∈ GΓl and gk ∈ Gt as in (8.16) and (8.17). Let Ht = {g1 , . . . , gc−1 } ⊂ Gt . Now we consider generating a set of moves Bt∗ in Ft by the following algorithm. Algorithm 8.2 Input: Ft , Ht = {g1 , . . . , gc−1 } Output: Bt∗ begin Bt∗ ← 0; / Choose any element x 1 in Ft ; for k = 2 to c do begin for l = 1 to 2k−2 do begin x l+2k−2 := gk−1 (xxl ); z l+2k−2 := x l − xl+2k−2 ; Bt∗ ← Bt∗ ∪ {zzl+2k−2 }; end end return Bt∗ ; end
128
8 Decomposable Models of Contingency Tables x1
x8
z5
(0111)(1000) x4
x1
(0000)(1111) z3
z8
z2
(0110)(1001) x6
z4
x5
x8
(0001)(1110)
(0111)(1000)
x3
x4
(0010)(1101)
(0110)(1001)
x7
z7
z3 z2
x5 (0001)(1110) x3 (0010)(1101)
z4
x7
z7
(0011)(1100)
(0101)(1010)
x2
z6
z5
z8
x6
(0011)(1100)
(0101)(1010)
(0000)(1111)
x2
z6
(0100)(1011)
(0100)(1011)
BG
Bt∗
t
Gt (z 2 ) Gt (z 3 ) Gt (z 4 )
Fig. 8.11 BGt and Bt∗ generated by Algorithm 8.2
Theorem 8.8. Bt∗ generated by Algorithm 8.2 is a minimal set of moves that connects Ft . Proof. Inasmuch as |Bt∗ | = 20 + 21 + · · · + 2c−1 = 2c − 1, it suffices to show that xl = xl for l = l . Suppose that there exist l and l such that l = l and xl = xl and x x that l and l are expressed as x l = gk p ◦ · · · ◦ gk1 (xx1 ),
x l = g
kp
◦ · · · ◦ gk1 (xx1 ),
where k1 < k2 < · · · < k p ≤ c − 1 and k1 < k2 < · · · < kp ≤ c − 1. Without loss of generality we can assume p ≤ p . Then we have gk p ◦ · · · ◦ gk1 = g
kp
◦ · · · ◦ gk1
(8.19)
= kl . From Lemma 8.3 (8.19) is equivalent to and there exists l ≤ p such that kl v k1 ⊕ · · · ⊕ vk p = v k ⊕ · · · ⊕ vk , 1
p
xl for l = l .
which contradicts that V 0 is a basis of V c−1 . Hence we have xl = From (8.6) we obtain the following result. Corollary 8.3. B ∗ =
∗ t ∈Bnd Bt
is a minimal Markov basis.
Example 8.6 (The complete independence model of a four-way contingency table). We consider the same fiber as in Example 8.2. Define V 0 = {vv2 , v 3 , v 4 } by v 2 = (100), v 3 = (010), and v 4 = (001). Figure 8.11 shows BGt and Bt∗ generated by Algorithm 8.2 with x 1 = (0000)(1111).
Chapter 9
Markov Basis for No-Three-Factor Interaction Models and Some Other Hierarchical Models
9.1 No-Three-Factor Interaction Models for 3 × 3 × K Contingency Tables The no-three-factor interaction model for three-way contingency tables is one of the simplest nondecomposable hierarchical models. In this chapter, we write I × J × K contingency tables as x = {xi jk | i = (i jk) ∈ I } where I = {1, . . . , I} × {1, . . . , J} × {1, . . ., K}. The generating class of no-three-factor interaction models is D = {{1, 2}, {1, 3}, {2, 3}}. Therefore the cell probability for i = (i jk) is written as log pi jk = μ{1,2} (i j) + μ{1,3} (ik) + μ{2,3}( jk). With lexicographic ordering of indices, the configuration A for this model is written as ⎛ ⎞ EI ⊗ EJ ⊗ 1K A = ⎝ EI ⊗ 1J ⊗ EK ⎠ . 1I ⊗ EJ ⊗ EK As we see below, the structure of Markov bases for this model is very complicated. In fact, the closed-form expression of Markov bases for this model of general I × J × K tables is not yet obtained at present. Instead, we show the structure of minimal Markov basis for I = J = 3 cases (i.e., 3 × 3 × K contingency tables) given in [10]. The arguments in [10] are based on the distance-reducing proofs in Chap. 6: first we give a candidate set of moves B ∗ , and in order to show that Ft constitutes one B ∗ equivalence class for any t , we suppose F1 and F2 are different B ∗ -equivalence classes of Ft for some t , then choose x ∈ F1 and y ∈ F2 such that |yy − x | =
∑ |yi jk − xi jk | > 0
i, j,k
is minimized, and finally derive a contradiction. S. Aoki et al., Markov Bases in Algebraic Statistics, Springer Series in Statistics 199, DOI 10.1007/978-1-4614-3719-2 9, © Springer Science+Business Media New York 2012
129
130
9 No-three-factor interaction models and other hierarchical models
For an I × J × K table x = {xi jk }, i-slice (or i = i0 slice) of x is the twodimensional slice {xi0 jk }1≤ j≤J,1≤k≤K , where i = i0 is fixed. We similarly define j-slice and k-slice. In this chapter, to display I × J × K contingency tables, we write I i-slices of size J × K as follows. x111 · · · x11K .. .. . . x1J1 · · · x1JK
x211 · · · x21K xI11 · · · xI1K .. .. .. .. . ··· . . . . x2J1 · · · x2JK xIJ1 · · · xIJK
We also use the concise expression of moves in Chap. 7 by the locations of nonzero cells. For example, a move of 3 × 3 × 3 table displayed as +1 −1 0 −1 +1 0 00 0 −1 +1 0 0 −1 +1 +1 0 −1 0 0 0 +1 0 −1 −1 0 +1 is also written as (111)(122)(212)(223)(231)(321)(333) − (112)(121)(211)(222)(233)(323)(331). In this chapter, it is always assumed that the indices are integers such that 1 ≤ i1 , i2 , . . . , iI ≤ I, i1 , i2 , . . . , iI all distinct, 1 ≤ j1 , j2 , . . . , jJ ≤ J, j1 , j2 , . . . , jJ all distinct, 1 ≤ k1 , k2 , . . . , kK ≤ K, k1 , k2 , . . . , kK all distinct.
9.2 Unique Minimal Markov Basis for 3 × 3 × 3 Tables First we define the most elementary eight-entry move. Definition 9.1. A move of degree 4 is a move m 4 (i1 i2 , j1 j2 , k1 k2 ) ∈ kerZ A written as (i1 j1 k1 )(i1 j2 k2 )(i2 j1 k2 )(i2 j2 k1 ) − (i1 j1 k2 )(i1 j2 k1 )(i2 j1 k1 )(i2 j2 k2 ). We call this move a basic move for the no-three-factor interaction model. Figure 9.1 gives a three-dimensional view of the basic move. From the definition, the relation m4 (i1 i2 , j2 j1 , k2 k1 )=m m4 (i2 i1 , j1 j2 , k2 k1 )=−m m4 (i2 i1 , j1 j2 , k1 k2 ) m 4 (i1 i2 , j1 j2 , k1 k2 )=m holds. These moves of degree 4 are the most elementary moves in the sense that all the other moves of higher degrees in kerZ A are written as linear combinations
9.2 Unique Minimal Markov Basis for 3 × 3 × 3 Tables
131
Fig. 9.1 2 × 2 × 2 move of degree 4 (basic move)
-1
+1
-1
+1
-1
+1
+1
-1
of these degree 4 moves with integral coefficients. Namely, the set of basic moves contains a lattice basis of kerZ A. It is seen that the basic moves are indispensable (see Definition 5.1) because {(i1 j1 k1 )(i1 j2 k2 )(i2 j1 k2 )(i2 j2 k1 ), (i1 j1 k2 )(i1 j2 k1 )(i2 j1 k1 )(i2 j2 k2 )} constitutes a two-element fiber. For I × J × K tables with fixed two-dimensional marginals, the set of basic moves is not a Markov basis when at least two of I, J, K are larger than 2. To see this, consider 3 × 3 × 3 contingency tables having two-dimensional marginals as xi j· = xi·k = x· jk = 2 for all 1 ≤ i, j, k ≤ 3. There are 132 elements in this fiber, but elements such as 200 020 002 020 002 200 002 200 020 are not connected to any other element in the fiber by the basic moves. This simple example suggests that the following moves of degree 6 are needed for the Markov basis. Definition 9.2. Moves of degree 6 are a move m I6 (i1 i2 , j1 j2 j3 , k1 k2 k3 ) ∈ kerZ A written as (i1 j1 k1 )(i1 j2 k2 )(i1 j3 k3 )(i2 j1 k2 )(i2 j2 k3 )(i2 j3 k1 ) −(i1 j1 k2 )(i1 j2 k3 )(i1 j3 k1 )(i2 j1 k1 )(i2 j2 k2 )(i2 j3 k3 ), a move m J6 (i1 i2 i3 , j1 j2 , k1 k2 k3 ) ∈ kerZ A written as (i1 j1 k1 )(i1 j2 k2 )(i2 j1 k2 )(i2 j2 k3 )(i3 j1 k3 )(i3 j2 k1 ) −(i1 j1 k2 )(i1 j2 k1 )(i2 j1 k3 )(i2 j2 k2 )(i3 j1 k1 )(i3 j2 k3 ) and a move m K6 (i1 i2 i3 , j1 j2 j3 , k1 k2 ) ∈ kerZ A written as (i1 j1 k1 )(i1 j2 k2 )(i2 j2 k1 )(i2 j3 k2 )(i3 j1 k2 )(i3 j3 k1 ) −(i1 j1 k2 )(i1 j2 k1 )(i2 j2 k2 )(i2 j3 k1 )(i3 j1 k1 )(i3 j3 k2 ).
132
9 No-three-factor interaction models and other hierarchical models
Similarly to the basic move, the relations m I6 (i1 i2 , j1 j2 j3 , k1 k2 k3 ) = m I6 (i1 i2 , j2 j3 j1 , k2 k3 k1 ) = m I6 (i2 i1 , j1 j3 j2 , k2 k1 k3 ), mI6 (i2 i1 , j1 j2 j3 , k1 k2 k3 ) m I6 (i1 i2 , j1 j2 j3 , k1 k2 k3 ) = −m and similar relations for m J6 (i1 i2 i3 , j1 j2 , k1 k2 k3 ) and m K6 (i1 i2 i3 , j1 j2 j3 , k1 k2 ) are derived from the definition. We see that all the moves of degree 6 are indispensable. Note that the moves of degree 6 are obtained as combinations of two basic moves. To see this, we provide a complete list of the patterns that are obtained by the sum of two overlapping basic moves. For basic moves m 4 (i1 i2 , j1 j2 , k1 k2 ) and m 4 (i1 i2 , j1 j2 , k1 k2 ), define
ΔI = δi1 i1 + δi1 i2 + δi2 i1 + δi2 i2 , ΔJ = δ j1 j1 + δ j1 j2 + δ j2 j1 + δ j2 j2 , ΔK = δk1 k1 + δk1 k2 + δk2 k1 + δk2 k2 and
Δ = ΔI + ΔJ + ΔK , where δi j = 1 if i = j, and = 0 otherwise. Because two moves are overlapping, ΔI , ΔJ , ΔK ≥ 1. Furthermore, ΔI ≤ 2, because i1 = i2 and i1 = i2 . Similarly, ΔJ , ΔK ≤ 2, therefore Δ ∈ {3, 4, 5, 6}. Corresponding to the values of Δ , all the patterns are classified as follows. • Δ = 3: m 4 (i1 i2 , j1 j2 , k1 k2 ) and m 4 (i1 i2 , j1 j2 , k1 k2 ) overlap at one nonzero entry. In this chapter, we call this case a combination of type 1 or a type-1 combination. If the signs of this overlapping cell are opposite, a move of degree 7 is obtained. Figure 9.2 gives a three-dimensional view of this type of move. • Δ = 4: m 4 (i1 i2 , j1 j2 , k1 k2 ) and m 4 (i1 i2 , j1 j2 , k1 k2 ) overlap at two nonzero entries. We call this case a combination of type 2 or a type-2 combination. If the pairs of signs of these two cells are opposite, an indispensable move of degree 6 in Definition 9.2 is obtained. Figure 9.3 gives a three-dimensional view of this type of move. • Δ = 5: m 4 (i1 i2 , j1 j2 , k1 k2 ) and m 4 (i1 i2 , j1 j2 , k1 k2 ) overlap at four nonzero entries along a two-dimensional rectangle. If all the pairs of signs are canceled, a basic move is again obtained as m 4 (i1 i2 , j1 j2 , k1 k2 ) = m 4 (i1 i2 , j1 j2 , k1 k3 ) + m4 (i1 i2 , j1 j2 , k3 k2 ) = m 4 (i1 i2 , j1 j3 , k1 k2 ) + m4 (i1 i2 , j3 j2 , k1 k2 ) = m 4 (i1 i3 , j1 j2 , k1 k2 ) + m4 (i3 i2 , j1 j2 , k1 k2 ). • Δ = 6: m 4 (i1 i2 , j1 j2 , k1 k2 ) and m 4 (i1 i2 , j1 j2 , k1 k2 ) overlap completely.
9.2 Unique Minimal Markov Basis for 3 × 3 × 3 Tables
133
Fig. 9.2 3 × 3 × 3 move of degree 7
+1 -1
-1 +1
+1
-1 -1
+1 -1
+1
-1
+1
+1
-1
Fig. 9.3 2 × 3 × 3 move of degree 6
+1
-1 +1
-1
-1
+1 +1
-1
+1
-1 +1
-1
In the above list, the cases of Δ = 3 and Δ = 4 yield so called “two-step moves;” that is, two basic moves are needed to construct these moves of degree 6 and degree 7. As we see in Theorem 9.1 below, the moves of degree 7 in Fig. 9.2 are not needed for a minimal Markov basis. To demonstrate this point, consider the following two 3 × 3 × 3 tables.
134
9 No-three-factor interaction models and other hierarchical models
Fig. 9.4 2 × 3 × 3 move of degree 6 (as another combination of type 2)
-1
+1
+1
-1
+1
-1
-1
+1
-1
+1
+1
-1
000 x: 010 001
001 100 010
100 001 000
000 y: 001 010
100 010 001
001 100 . 000
These two tables are the negative part and the positive part of the move of degree 7, (123)(132)(211)(222)(233)(313)(321) − (122)(133)(213)(221)(232)(311)(323), and mutually accessible by this move: y − x . However, instead of adding y − x to x , m 4 (23, 12, 13) can be added to x , and then, m 4 (12, 23, 32) can be added to x + m 4 (23, 12, 13), to obtain y . Note that the move y − x is a type-1 combination of m 4 (23, 12, 13) and m 4 (12, 23, 32). x + m 4 (23, 12, 13) does not contain a negative element, whereas x + m 4 (12, 23, 32) contains a negative element (2, 2, 3). Note also that m 4 (23, 12, 13) and m 4 (12, 23, 32) overlap at this cell. Because the two basic moves are canceling at this cell, it is obvious that at least one of these basic moves (that has +1 at this cell) can be added without causing negative elements. On the other hand, because the type-2 combination has two overlapping cells, it cannot be avoided that one of these two elements becomes negative in adding basic moves one by one. For this reason, the type-2 combination is essential. We also note that the expression of the move of degree 6 as a type-2 combination of two basic moves is not unique. Figure 9.4 illustrates the same move of degree 6 shown in Fig. 9.3, but the overlapping cells of the two basic moves are different. Now we give a unique minimal Markov basis for 3 × 3 × 3 case.
9.2 Unique Minimal Markov Basis for 3 × 3 × 3 Tables
135
Theorem 9.1. The set of basic moves m 4 (i1 i2 , j1 j2 , k1 k2 ) and moves of degree 6, m I6 (i1 i2 , j1 j2 j3 , k1 k2 k3 ), m J6 (i1 i2 i3 , j1 j2 , k1 k2 k3 ), m K6 (i1 i2 i3 , j1 j2 j3 , k1 k2 ) constitutes a unique minimal Markov basis for 3 × 3 × 3 tables. Note that the minimality and the uniqueness directly hold if this set of moves constitutes a Markov basis, because it is composed of indispensable moves only. See Corollary 5.2 in Chap. 5. Following the distance-reducing proofs in Chap. 6, we consider the pattern of z = y − x where x and y have the same two-dimensional marginals. Before we give a proof of Theorem 9.1, we show a useful lemma concerning the patterns of twodimensional slices of y − x for general 3 × 3 × K cases. Definition 9.3. Let C be a two-dimensional matrix with elements ci j . Then a rectangle is a set of four entries (ci1 j1 , ci2 j1 , ci2 j2 , ci1 j2 ) with alternating signs. Similarly, a 6-cycle is a set of six entries (ci1 j1 , ci2 j1 , ci2 j2 , ci3 j2 , ci3 j3 , ci1 j3 ) with alternating signs. Using the fact that all the marginal totals of z = y − x are zeros, it can be easily shown that any nonzero entry of z has to be a member of either a rectangle or a 6-cycle in all of the i-, j-, and k-slices when x and y are 3 × 3 × K contingency tables. Lemma 9.1. Let x and y be 3 × 3 × K contingency tables and let z = y − x . Consider z after minimizing |zz| by applying the basic moves and the moves of degree 6 without causing negative entries on the way. Then (a) No k-slice of z contains 6-cycles. (b) There is at least one rectangle in either an i-slice or a j-slice unless z = 0. Proof. In the proof of this lemma, we display k-slices of z instead of our usual display of i-slices. To prove (a), suppose that, without loss of generality, k = 1 slice of z contains the following 6-cycle i\ j 1 2 3
1 2 +− − ∗ ∗ +
3 ∗ . + −
Because z11· = 0, there exists at least one negative element in z112 , z113 , . . . , z11K . Let z112 < 0 without loss of generality. As is shown above, z112 has to be an element of either a rectangle or a 6-cycle in the k = 2 slice. These two cases are considered, respectively, as follows. Case 1. z112 is an element of a 6-cycle. It is seen that the negative entries in the 6-cycle in the k = 2 slice, which includes z112 , can be either (i) (z112 , z222 , z332 ) or (ii) (z112 , z232 , z322 ). In case (i), m 4 (12, 12, 12) can be added to x without causing negative entries to make |zz| smaller
136
9 No-three-factor interaction models and other hierarchical models
because x121 , x211 , x112 , x222 > 0. On the other hand, in case (ii), m K6 (132, 123, 12) can be added to x without causing negative entries to make |zz| smaller because x121 , x211 , x331 , x112 , x232 , x322 > 0. These imply that Case 1 is a contradiction. Case 2. z112 is an element of a rectangle. It is seen that the negative entries in the rectangle, which includes z112 , can be either (i) (z112 , z222 ), (ii) (z112 , z232 ), (iii) (z112 , z322 ), or (iv) (z112 , z332 ). In case (i), m 4 (12, 12, 12) can be added to x without causing negative entries and |zz| can be made smaller as in (i) of Case 1. In case (ii), it follows that z132 , z212 > 0 and m 4 (12, 13, 21) can be added to y without causing negative entries and make |zz| smaller because y111 , y231 , y132 , y212 > 0. Case (iii) is the symmetric case of (ii). In case (iv), the two k-slices, {zi j1 } and {zi j2 }, are represented as i\ j 1 {zi j1 } : 2 3
1 + − ∗
2 3 − ∗ ∗ + +−
i\ j 1 {zi j2 } : 2 3
1 2 −∗ ∗ ∗ +∗
3 + . ∗ −
In this case, because z331 , z332 < 0, at least one of z333 , . . . , z33K has to be positive. Let z333 > 0 without loss of generality. Here, z333 is again an element of either a rectangle or a 6-cycle. But as already seen in Case 1, there cannot be another 6-cycle in the k = 1 slice. Thus z333 has to be a member of a rectangle. Moreover, for the same reason as (i)–(iii) of Case 2, the k = 3 slice has to be a mirror image of the k = 2 slice: i\ j 1 {zi j1 } : 2 3
1 2 +− − ∗ ∗ +
3 ∗ + −
i\ j 1 {zi j2 } : 2 3
1 2 3 −∗+ ∗ ∗ ∗ +∗−
i\ j 1 {zi j3 } : 2 3
1 + ∗ −
2 3 ∗− . ∗ ∗ ∗+
However, m 4 (13, 13, 23) can be added to y or m 4 (13, 13, 32) can be added to x without causing negative entries and |zz| can be made smaller, which contradicts the assumption. These imply that Case 2 also is a contradiction. These considerations indicate that the 6-cycle cannot be included in any 3 × 3 slices and the proof of (a) is completed. Next (b) is proved. Suppose z has nonzero entries and let z111 > 0 without loss of generality. It is known that z111 is a member of a rectangle in the k = 1 slice from (a). Then let the k = 1 slice be represented as i\ j 1 2 3
1 + − ∗
2 3 −∗ +∗ ∗ ∗
9.2 Unique Minimal Markov Basis for 3 × 3 × 3 Tables
137
without loss of generality. We are assuming that there exists no rectangle in the 3×K i-slices or j-slices of z . Write z112 < 0 without loss of generality because z11· = 0. i\ j 1 {zi j1 } : 2 3
1 + − ∗
2 3 −∗ +∗ ∗ ∗
i\ j 1 {zi j2 } : 2 3
1 2 −∗ ∗ ∗ ∗ ∗
3 ∗ . ∗ ∗
From the assumption, it follows that z122 , z212 ≤ 0 because otherwise either i = 1 slice or j = 1 slice has a rectangle. We also have z222 ≥ 0 because otherwise we can add m4 (12, 12, 12) to x without causing negative entries and make |zz| smaller. Hereafter we display nonnegative elements by 0+ and nonpositive elements by 0−. i\ j 1 {zi j1 } : 2 3
1 + − ∗
2 3 −∗ +∗ ∗ ∗
i\ j 1 {zi j2 } : 2 3
1 2 3 − 0− ∗ . 0− 0+ ∗ ∗ ∗ ∗
Inasmuch as z112 has to be an element of a rectangle in a k = 2 slice, z132 > 0, z312 > 0 and z332 < 0 are derived. i\ j 1 {zi j1 } : 2 3
1 2 3 +−∗ −+∗ ∗ ∗ ∗
i\ j 1 {zi j2 } : 2 3
1 2 3 − 0− + . 0− 0+ ∗ + ∗ −
It is seen that if z131 < 0, there appears a rectangle in the i = 1 slice; and if z311 < 0, there appears a rectangle in the j = 1 slice. These contradict the assumption. Then it follows that z131 , z311 ≥ 0. Here we write z123 > 0 without loss of generality, because z12· = 0. i\ j 1 {zi j1 } : 2 3
1 2 3 + − 0+ − + ∗ 0+ ∗ ∗
i\ j 1 {zi j2 } : 2 3
1 2 3 − 0− + 0− 0+ ∗ + ∗ −
i\ j 1 {zi j3 } : 2 3
1 ∗ ∗ ∗
2 3 +∗ . ∗ ∗ ∗ ∗
It is seen that if z113 < 0, there appears a rectangle in the i = 1 slice; and if z223 < 0, there appears a rectangle in the j = 2 slice. These contradict the assumption. Then it follows that z113 , z223 ≥ 0. i\ j 1 {zi j1 } : 2 3
1 2 3 + − 0+ − + ∗ 0+ ∗ ∗
i\ j 1 {zi j2 } : 2 3
1 2 3 − 0− + 0− 0+ ∗ + ∗ −
i\ j 1 {zi j3 } : 2 3
1 2 3 0+ + ∗ . ∗ 0+ ∗ ∗ ∗ ∗
138
9 No-three-factor interaction models and other hierarchical models
z123 has to be an element of a rectangle in the k = 3 slice, therfore z133 , z323 < 0 and z333 > 0 are derived. i\ j 1 {zi j1 } : 2 3
1 2 + − − + 0+ ∗
3 0+ ∗ ∗
i\ j 1 {zi j2 } : 2 3
1 2 3 − 0− + 0− 0+ ∗ + ∗ −
i\ j 1 {zi j3 } : 2 3
1 2 3 0+ + − . ∗ 0+ ∗ ∗ − +
But then a rectangle (z132 , z133 , z333 , z332 ) appears in the j = 3 slice, which contradicts the assumption and the proof of (b) is completed.
Now we carry out a proof of Theorem 9.1 using Lemma 9.1. Proof (Theorem 9.1). As we have stated, we only need to show that the elements of z = y − x have to be all zero after minimizing |zz| by applying the basic moves or the moves of degree 6 without causing negative entries on the way. Suppose z has nonzero entries. Let z111 > 0 without loss of generality. From Lemma 9.1(a), z111 has to be an element of rectangles, in each of the i = 1, j = 1, and k = 1 slices. We can take one of these rectangles in the i = 1 slice as (z111 , z112 , z122 , z121 ) without loss of generality. +−∗ −+∗ ∗ ∗ ∗
∗∗∗ ∗∗∗ ∗∗∗
∗∗∗ ∗∗∗ . ∗∗∗
Next consider the j = 1 slice. We claim that z111 and z112 are elements of the same rectangle in j = 1 slice. To prove this, consider the sign of z113 . If z113 ≥ 0, the rectangle containing z111 in the j = 1 slice contains z112 , and if z113 < 0, the rectangle containing z112 in the j = 1 slice contains z111 . Therefore, z111 and z112 are elements of the same rectangle in the j = 1 slice and the rectangle can be taken as (z111 , z112 , z212 , z211 ) without loss of generality. +−∗ −+∗ ∗ ∗ ∗
−+∗ ∗ ∗ ∗ ∗ ∗ ∗
∗∗∗ ∗∗∗ . ∗∗∗
Now consider the rectangle in the k = 1 slice containing z111 . For a similar reason as above, this rectangle also contains z121 . In addition, if z221 > 0, m 4 (12, 12, 21) can be added to y without causing negative entries and |zz| can be made smaller, which contradicts the assumption. Hence, the rectangle in the k = 1 slice including z111 has to be (z111 , z121 , z321 , z311 ). +−∗ −+∗ ∗ ∗ ∗
− +∗ 0− ∗ ∗ ∗ ∗ ∗
−∗∗ +∗∗ . ∗ ∗∗
9.3 Unique Minimal Markov Basis for 3 × 3 × 4 Tables
139
Next consider the rectangle in the j = 2 slice including z121 . For a similar reason as above, this rectangle also contains z122 . Hence, the rectangle in the j = 2 slice including z121 has to be (z121 , z122 , z322 , z321 ). +−∗ −+∗ ∗ ∗ ∗
− +∗ 0− ∗ ∗ ∗ ∗ ∗
− ∗ ∗ +−∗ . ∗ ∗ ∗
However, m 4 (13, 12, 12) can be added to x without causing negative entries and |zz| can be made smaller, which contradicts the assumption. From these considerations, a set of the basic moves and the moves of degree 6 is shown to be a Markov basis for the 3 × 3 × 3 case. All these moves are indispensable, therefore the minimality and the uniqueness also follow. This completes the proof of Theorem 9.1.
9.3 Unique Minimal Markov Basis for 3 × 3 × 4 Tables The next indispensable move is constructed as a three-step move. For the case of a general I × J × K contingency table, there are several types of such a move. One is a 2 × 4 × 4 move of degree 8 and another is a 3 × 4 × 4 move of degree 9. We consider these moves in Sect. 9.5. For the 3 × 3 × K case, the following type of move is needed. Definition 9.4. A move of degree 8 is a move m 8 (i1 i2 i3 , j1 j2 j3 , k1 k2 k3 k4 ) ∈ kerZ A written as (i1 j1 k1 )(i1 j2 k2 )(i2 j1 k3 )(i2 j2 k1 )(i2 j3 k4 )(i3 j1 k2 )(i3 j2 k4 )(i3 j3 k3 ) −(i1 j1 k2 )(i1 j2 k1 )(i2 j1 k1 )(i2 j2 k4 )(i2 j3 k3 )(i3 j1 k3 )(i3 j2 k2 )(i3 j3 k4 ). Figure 9.5 gives a three-dimensional view of this type of move. From the definition, the relation m8 (i1 i3 i2 , j2 j1 j3 , k2 k1 k4 k3 ) m 8 (i1 i2 i3 , j1 j2 j3 , k1 k2 k3 k4 ) = −m = m 8 (i1 i3 i2 , j1 j2 j3 , k2 k1 k3 k4 ) is derived. Now we state a theorem for the 3 × 3 × 4 case. Theorem 9.2. The set of basic moves m 4 (i1 i2 , j1 j2 , k1 k2 ), moves of degree 6, m I6 (i1 i2 , j1 j2 j3 , k1 k2 k3 ), m J6 (i1 i2 i3 , j1 j2 , k1 k2 k3 ), m K6 (i1 i2 i3 , j1 j2 j3 , k1 k2 ), and moves of degree 8, m 8 (i1 i2 i3 , j1 j2 j3 , k1 k2 k3 k4 ) constitutes a unique minimal Markov basis for 3 × 3 × 4 tables.
140
9 No-three-factor interaction models and other hierarchical models
+1 -1 +1
-1 +1
-1 +1
-1 -1
+1 -1
+1
-1 +1
+1 -1
Fig. 9.5 3 × 3 × 4 move of degree 8
Proof. Similarly to the proof of Theorem 9.1, we only need to show that the set of the moves of degree 4, 6, and 8 above constitutes a Markov basis; that is, the pattern of z = y − x has to be of all zero entries after minimizing |zz| by adding the basic moves, the moves of degree 6 or degree 8, without causing negative entries on the way. Suppose z has nonzero entries. Let z111 > 0 without loss of generality. From Lemma 9.1(b), we can also assume that there is a rectangle including z111 in either an i = 1 slice or a j = 1 slice. We can take one of these rectangles in the i = 1 slice as (z111 , z112 , z121 , z122 ) without loss of generality. Moreover, z211 < 0, z221 > 0 without loss of generality because it is known from Lemma 9.1(a) that z111 is an element of a rectangle in the k = 1 slice. +−∗∗ −+∗∗ ∗ ∗ ∗∗
−∗∗∗ +∗∗∗ ∗ ∗∗∗
∗∗∗∗ ∗∗∗∗ . ∗∗∗∗
As in the proof of Theorem 9.1, by considering the sign of z132 , we see that z112 and z122 are members of the same rectangle in the k = 2 slice. Then (z212 , z222 ) and/or (z312 , z322 ) has to be (+, −). But if z212 > 0, m 4 (12, 12, 21) can be added to y without causing negative entries; and if z222 < 0, m 4 (12, 12, 12) can be added to x without causing negative entries; and |zz| can be made smaller. These imply that z312 > 0, z322 < 0 and z212 ≤ 0, z222 ≥ 0. Similarly, if z311 < 0, m 4 (13, 12, 12) can be added to x without causing negative entries; and if z321 > 0, m 4 (13, 12, 21) can be added to y without causing negative entries; and |zz| can be made smaller, which forces z311 ≥ 0 and z321 ≤ 0.
9.3 Unique Minimal Markov Basis for 3 × 3 × 4 Tables
+−∗∗ −+∗∗ ∗ ∗ ∗∗
− 0− ∗ ∗ + 0+ ∗ ∗ ∗ ∗ ∗∗
141
0+ + ∗ ∗ 0− − ∗ ∗ . ∗ ∗ ∗∗
Inasmuch as z21· = 0, let z213 > 0 without loss of generality, which forces z123 ≤ 0, otherwise, m 4 (12, 12, 31) can be added to y without causing negative entries and |zz| can be made smaller. The fact that z213 > 0 also forces z323 ≤ 0, otherwise, m J6 (132, 21, 123) can be added to y without causing negative entries and |zz| can be made smaller. +− ∗ ∗ − + 0− ∗ ∗ ∗ ∗ ∗
− 0− + ∗ + 0+ ∗ ∗ ∗ ∗ ∗ ∗
0+ + ∗ ∗ 0− − 0− ∗ . ∗ ∗ ∗ ∗
Because z·23 = 0, it follows that z223 ≥ 0. This implies z224 , z233 < 0 because z22· = z2·3 = 0. +− ∗ ∗ − 0− + ∗ 0+ + ∗ ∗ − + 0− ∗ + 0+ 0+ − 0− − 0− ∗ . ∗ ∗ ∗ ∗ ∗ ∗ − ∗ ∗ ∗ ∗ ∗ From symmetry (in interchanging roles of + and −), z114 , z314 ≥ 0, otherwise, m 4 (12, 12, 14) can be added to x without causing negative entries or m J6 (132, 12, 124) can be added to x without causing negative entries and |zz| can be made smaller. These also imply z214 ≤ 0, z234 > 0 because z·14 = z2·4 = 0. + − ∗ 0+ − + 0− ∗ ∗ ∗ ∗ ∗
− 0− + 0− + 0+ 0+ − ∗ ∗ − +
0+ + ∗ 0+ 0− − 0− ∗ . ∗ ∗ ∗ ∗
(9.1)
Because z31· = z32· = z3·3 = z3·4 = 0, it follows that z313 < 0, z324 > 0, z333 > 0, z334 < 0. + − ∗ 0+ − + 0− ∗ ∗ ∗ ∗ ∗
− 0− + 0− + 0+ 0+ − ∗ ∗ − +
0+ + − 0+ 0− − 0− + . ∗ ∗ + −
But m8 (132, 123, 2134) can be added to y (or m8 (123, 123, 1234) can be added to x) without causing negative entries and |zz| can be made smaller. From these considerations, a set of the basic moves, the moves of degree 6 and degree 8, is shown to be a Markov basis for the 3 × 3 × 4 case. This is a set of indispensable moves, thus this is a unique minimal Markov basis and Theorem 9.2 is proved.
142
9 No-three-factor interaction models and other hierarchical models
9.4 Unique Minimal Markov Basis for 3 × 3 × 5 and 3 × 3 × K Tables for K > 5 Continuing the above discussion, next we consider a four-step move. For the case of a 3 × 3 × K contingency table, only a move of the following type needs to be considered. Definition 9.5. A move of degree 10 is a move m 10 (i1 i2 i3 , j1 j2 j3 , k1 k2 k3 k4 k5 ) ∈ kerZ A written as (i1 j1 k1 )(i1 j2 k2 )(i1 j2 k5 )(i1 j3 k4 )(i2 j1 k3 )(i2 j2 k1 )(i2 j3 k5 )(i3 j1 k2 )(i3 j2 k4 )(i3 j3 k3 ) −(i1 j1 k2 )(i1 j2 k1 )(i1 j2 k4 )(i1 j3 k5 )(i2 j1 k1 )(i2 j2 k5 )(i2 j3 k3 )(i3 j1 k3 )(i3 j2 k2 )(i3 j3 k4 ). Figure 9.6 gives a three-dimensional view of this type of move. From the definition, the relation m 10 (i1 i2 i3 , j1 j2 j3 , k1 k2 k3 k4 k5 ) = m 10 (i1 i3 i2 , j3 j2 j1 , k4 k5 k3 k1 k2 ) m10 (i1 i2 i3 , j3 j2 j1 , k5 k4 k3 k2 k1 ) = −m is derived. As for a connected Markov chain, the next theorem holds for the 3 × 3 × 5 case. Theorem 9.3. The set of basic moves m 4 (i1 i2 , j1 j2 , k1 k2 ), moves of degree 6, m I6 (i1 i2 , j1 j2 j3 , k1 k2 k3 ), m J6 (i1 i2 i3 , j1 j2 , k1 k2 k3 ), m K6 (i1 i2 i3 , j1 j2 j3 , k1 k2 ), moves of degree 8, m 8 (i1 i2 i3 , j1 j2 j3 , k1 k2 k3 k4 ), and moves of degree 10, m 10 (i1 i2 i3 , j1 j2 j3 , k1 k2 k3 k4 k5 ), constitutes a unique minimal Markov basis for the 3 × 3 × 5 tables.
+1 -1 +1
-1 +1
-1
-1
+1
+1
-1 +1
-1
-1 +1
+1 -1
Fig. 9.6 3 × 3 × 5 move of degree 10
+1 -1
-1 +1
9.4 Unique Minimal Markov Basis for 3 × 3 × 5 and 3 × 3 × K Tables for K > 5
143
Interestingly, this set of moves is shown to be a unique minimal Markov basis for the general 3 × 3 × K(K ≥ 5) case. We give a main result of this section. Theorem 9.4. The set of basic moves m 4 (i1 i2 , j1 j2 , k1 k2 ), moves of degree 6, m I6 (i1 i2 , j1 j2 j3 , k1 k2 k3 ), m J6 (i1 i2 i3 , j1 j2 , k1 k2 k3 ), m K6 (i1 i2 i3 , j1 j2 j3 , k1 k2 ), moves of degree 8, m 8 (i1 i2 i3 , j1 j2 j3 , k1 k2 k3 k4 ), and moves of degree 10, m 10 (i1 i2 i3 , j1 j2 j3 , k1 k2 k3 k4 k5 ), constitutes a unique minimal Markov basis for 3 × 3 × K(K ≥ 5) tables. Proof (Theorem 9.3). Again all we have to show is that the pattern of z = y − x must be of all zero entries after minimizing |zz| by adding the basic moves, the moves of degree 6, degree 8, or degree 10, without causing negative entries on the way. Suppose z has nonzero entries. For a similar reason leading to (9.1) in the proof of Theorem 9.2, the patterns can be restricted to + − ∗ 0+ ∗ − + 0− ∗ ∗ ∗ ∗ ∗ ∗ ∗
− 0− + 0− ∗ + 0+ 0+ − ∗ ∗ ∗ − + ∗
0+ + ∗ 0+ ∗ 0− − 0− ∗ ∗ ∗ ∗ ∗ ∗ ∗
without loss of generality. Because z31· = z32· = 0, at least one of z313 and z315 has to be negative and at least one of z324 and z325 has to be positive. But we have already seen that (z313 , z324 ) = (−, +) contradicts the assumption. In addition, if (z315 , z325 ) = (−, +), it follows that z115 ≤ 0, z125 ≥ 0 (otherwise m 4 (13, 12, 25) can be added to y without causing negative entries and m 4 (13, 12, 52) can be added to x without causing negative entries and |zz| can be made smaller) and (z215 , z225 ) = (+, −) because z·15 = z·25 = 0. But m J6 (132, 21, 125) can be added to y without causing negative entries and m J6 (132, 12, 125) can be added to x without causing negative entries and |zz| can be made smaller. All of these contradict the assumption. The remaining patterns are (z313 , z325 ) = (−, +) or (z315 , z324 ) = (−, +). Considering the symmetry, we write (z313 , z325 ) = (−, +) without loss of generality. Then the patterns are, without loss of generality, summarized as + − ∗ 0+ ∗ − + 0− ∗ ∗ ∗ ∗ ∗ ∗ ∗
− 0− + 0− ∗ + 0+ 0+ − ∗ ∗ ∗ − + ∗
0+ + − 0+ 0+ 0− − 0− 0− + . ∗ ∗ ∗ ∗ ∗
z·24 = z1·4 = z3·3 = z3·5 = 0, thus it follows that z124 > 0, z134 < 0, z333 > 0, z335 < 0. + − ∗ 0+ ∗ − + 0− + ∗ ∗ ∗ ∗ − ∗
− 0− + 0− ∗ + 0+ 0+ − ∗ ∗ ∗ − + ∗
0+ + − 0+ 0+ 0− − 0− 0− + . ∗ ∗ + ∗ −
144
9 No-three-factor interaction models and other hierarchical models
If z225 < 0, m 8 (123, 123, 1235) can be added to x without causing negative entries and |zz| can be made smaller, which contradicts the assumption. Similarly, if z235 > 0, m 8 (123, 213, 1253) can be added to y without causing negative entries and |zz| can be made smaller, which contradicts the assumption. These imply z225 ≥ 0, z234 ≤ 0, which also imply z125 < 0, z135 > 0 inasmuch as z·25 = z·35 = 0. + − ∗ 0+ ∗ − + 0− + − ∗ ∗ ∗ − +
− 0− + 0− ∗ + 0+ 0+ − 0+ ∗ ∗ − + 0−
0+ + − 0+ 0+ 0− − 0− 0− + . ∗ ∗ + ∗ −
But m 10 (123, 321, 45321) can be added to y (or m 10 (123, 123, 12354) can be added to x ) without causing negative entries and |zz| can be made smaller, which contradicts the assumption. From these considerations, the set of the basic moves, the moves of degree 6, degree 8, and degree 10 is shown to be a Markov basis for the 3×3×5 case. Because this is a set of indispensable moves, this is a unique minimal Markov basis and Theorem 9.3 is proved.
Proof (Theorem 9.4). Again we can begin with the following pattern. + − ∗ 0+ ∗ ∗ − + 0− ∗ ∗ ∗ ∗ ∗ ∗ ∗ ∗∗
− 0− + 0− ∗ ∗ + 0+ 0+ − ∗ ∗ ∗ ∗ − + ∗∗
0+ + ∗ 0+ ∗ ∗ 0− − 0− ∗ ∗ ∗ . ∗ ∗ ∗ ∗ ∗∗
As we have seen in the proof of Theorem 9.3, z313 has to be nonnegative and z324 has to be nonpositive, because either one of (z313 , z326 ) = (−, +) and (z316 , z324 ) = (−, +) also contradicts the assumption. The case of (z316 , z326 ) = (−, +) also contradicts the assumption for a similar reason to that of (z315 , z325 ) = (−, +). Hence the remaining patterns are (z315 , z326 ) = (−, +) and (z316 , z325 ) = (−, +). We write (z315 , z326 ) = (−, +) without loss of generality. + − ∗ 0+ ∗ ∗ − + 0− ∗ ∗ ∗ ∗ ∗ ∗ ∗ ∗∗
− 0− + 0− ∗ ∗ + 0+ 0+ − ∗ ∗ ∗ ∗ − + ∗∗
0+ + 0+ 0+ − ∗ 0− − 0− 0− ∗ + . ∗ ∗ ∗ ∗ ∗ ∗
According to the symmetry in interchanging the roles of {+, −}, the roles of {z2 jk , z3 jk }, and the roles of {(zi j3 , zi j4 ), (zi j5 , zi j6 )}, the patterns can be restricted to + − ∗ 0+ ∗ 0− − + 0− ∗ 0+ ∗ ∗ ∗ ∗ ∗ ∗ ∗
− 0− + 0− 0− 0− + 0+ 0+ − 0+ 0+ ∗ ∗ − + ∗ ∗
0+ + 0+ 0+ − 0+ 0− − 0− 0− 0− + ∗ ∗ ∗ ∗ + −
9.5 Indispensable Moves for Larger Tables
145
for a similar reason to the proof of Theorem 9.2. Because z·13 = z·15 = z·24 = z·26 = 0, it follows that z113 < 0, z115 > 0, z124 > 0 and z126 < 0. z1·3 = z1·4 = z1·5 = z1·6 = 0 also forces z133 > 0, z134 < 0, z135 < 0 and z136 > 0. + − − 0+ + 0− − + 0− + 0+ − ∗ ∗ + − − +
− 0− + 0− 0− 0− + 0+ 0+ − 0+ 0+ ∗ ∗ − + ∗ ∗
0+ + 0+ 0+ − 0+ 0− − 0− 0− 0− + . ∗ ∗ ∗ ∗ + −
But this pattern includes moves of degree 6. We can add m I6 (21, 132, 134) to y , m I6 (12, 132, 134) to x , m I6 (13, 132, 256) to y , or m I6 (31, 132, 256) to x without causing negative entries and make |zz| smaller, which contradicts the assumption. From these considerations, it is shown that the set of the basic moves, the moves of degree 6, degree 8, and degree 10 is also a Markov basis for the 3 × 3 × K (K ≥ 5) case. The minimality and the uniqueness directly hold again. Note that although we have displayed 3 × 3 × 6 tables, the above argument does not involve k slices for k ≥ 7. Therefore we obtain the same contradiction for the 3 × 3 × K (K ≥ 7) tables. This completes the proof.
A result corresponding to Theorem 9.4 for Gr¨obner bases was given in [27].
9.5 Indispensable Moves for Larger Tables The fact that the structure of a minimal Markov basis for 3 × 3 × K tables is essentially explained by 3 × 3 × 5 tables is very attractive. Such a theoretical result seems important because even if we can obtain the reduced Gr¨obner basis for the 3 × 3 × 6 table by an algebraic algorithm, we have to carry out new calculations to obtain results for 3 × 3 × 7 or 3 × 3 × 8 problems. In fact, the following result is shown in [131] as a special case of the Graver complexity of the higher Lawrence lifting (see Sect. 9.8 below). Proposition 9.1 (Corollary 2 of [131]). For any positive integers I, J, there exists a positive integer m such that every element of a minimal Markov basis for the I × J × K tables with fixed two-dimensional marginal frequencies is included in I × J × m. The above m is called a Markov complexity for the configuration A. The values of m for some cases are computed in [131]. For the example of the 3 × 3 × K table with fixed two-dimensional marginal frequencies, an upper bound of the Markov complexity is given by the Graver complexity, which is computed to be 9 by [131]. Therefore no new type of conformally primitive move appears for K ≥ 10. The results of the previous section are not derived by algebraic algorithms but by “thoroughly checking symmetry by inspection” ([10]) based on the distancereducing proofs in Sect. 6. Of course, if we carry out a similar method for problems of larger sizes, the number of the cases we have to consider becomes huge. In [9], a
146
9 No-three-factor interaction models and other hierarchical models
similar approach of distance-reducing proofs is carried out for the 3 × 4 × K cases. The result of [9] is that “for 3 × 4 × K problems, it is sufficient to consider up to K = 8, and the set of the 20 kinds of moves up to 16 degree forms a unique minimal Markov basis.” The truth of this proposition has not yet been confirmed by algebraic algorithms. On the other hand, as for the development of the algorithms for calculating Gr¨obner bases, calculating Markov bases for the 4 × 4 × 4 tables has been used as a benchmark problem since about 2002. This problem is first solved completely by [83] using 4ti2 ([1]). In [83], it is reported that “148, 654 elements in 15 kinds of moves form a minimal Markov basis for the 4× 4 × 4 problem.” See [15] for an overview of the history of calculating Markov bases for 4 × 4 × 4 problems. We have already pointed out that the type-2 combination is essential in Sect. 9.2. In fact, from the three-dimensional views of the indispensable moves in the unique minimal Markov basis for 3 × 3 × K cases in Figs. 9.3, 9.5, and 9.6, we see that they are constructed as the type-2 combination of several basic moves. However, structure of the indispensable moves for general I × J × K cases can be more complicated. To see this point, we show the unique minimal Markov basis for the 3 × 4 × 4 case. It is composed of basic moves, moves of degree 6 (2 × 3 × 3, 3 × 2 × 3, 3 × 3 × 2), and moves of degree 8 (3 × 3 × 4, 3 × 4 × 3), and moves of degree 8 (2 × 4 × 4) like +1 −1 0 0 0 +1 −1 0 0 0 +1 −1 −1 0 0 +1
−1 +1 0 0 0 −1 +1 0 0 0 −1 +1 +1 0 0 −1
00 00 00 00
00 00 , 00 00
(9.2)
moves of degree 9 (3 × 4 × 4, Fig. 9.7) like +1 −1 0 0 −1 0 +1 0 0 +1 −1 0 0 0 0 0
−1 +1 0 0 +1 0 0 −1 0 0 0 0 0 −1 0 +1
0 0 0 0 0 0 −1 +1 , 0 −1 +1 0 0 +1 0 −1
and moves of degree 10 (3 × 4 × 4, Fig. 9.8) like +1 −1 0 0 −1 +1 0 0 0 0 +1 −1 0 0 −1 +1
−1 +1 0 0 0 0 0 0 +1 0 −1 0 0 −1 +1 0
0 0 0 0 +1 −1 0 0 . −1 0 0 +1 0 +1 0 −1
Among the newly obtained moves, the 3 × 4 × 4 move of degree 10 is interpreted as a type-2 combination of a basic move and a move of degree 8, which is similar to the 3 × 3 × 5 move of degree 10 shown in Sect. 9.4. However, the 3 × 4 × 4 move of degree 9 is new in the sense that this is a type-2 combination of a basic move and a move of degree 7. Recall that the move of degree 7 itself is not needed to construct
9.5 Indispensable Moves for Larger Tables
147
-1
+1
+1
-1 +1 -1
-1 +1
+1
-1
-1
+1
+1
-1
-1
+1
+1
-1
Fig. 9.7 3 × 4 × 4 move of degree 9
+1
-1
-1 +1
+1
-1
-1
+1
+1
-1 -1
+1
+1 -1 +1 +1
-1
-1 +1
-1
Fig. 9.8 3 × 4 × 4 move of degree 10
148
9 No-three-factor interaction models and other hierarchical models
a connected Markov chain. In this chapter, we have only considered combinations of basic moves that happen “one at a time.” But it might be worthwhile to think of this degree 9 move as a combination of three basic moves that happens “all at once,” and every two of these basic moves are type-1 combinations. The move of degree 9 suggests the difficulty in forming a conjecture on a minimal Markov basis for larger tables. On the other hand, we see that the 2 × 4 × 4 moves of degree 8 displayed in (9.2) are conformally primitive (see Definition 4.2 of Sect. 4.6). From Proposition 4.2, for general 2 × J × K tables, each conformally primitive move of degree 2m can be written as (1 j1 k1 )(1 j2 k2 ) · · · (1 jm km )(2 j1 k2 )(2 j2 k3 ) · · · (2 jm k1 ) −(2 j1 k1 )(2 j2 k2 ) · · · (2 jm km )(1 j1 k2 )(1 j2 k3 ) · · · (1 jm k1 ) and it is also indispensable. Therefore, for 2 × J × K tables, the unique minimal Markov basis exists as the Graver basis. This is the consequence of the fact that 2 × J × K no-three-factor interaction model is the Lawrence lifting (cf. Sects. 4.6 and 5.4.3) of the J × K two-way complete independence model. As another difficulty in larger tables, we show that a minimal Markov basis can include non-square-free indispensable moves. It is easy to check that the following two 3 × 4 × 6 moves are indispensable. +1 −1 0 0 0 0 +1 −1 0 0 0 0 +1 0 0 −1 0 0 0 0
0 0 −1 +1
+1 −1 0 0 0 0 0 +1 −1 0 0 0 0 0 +1 −1 0 0 −1 0 0 +1 0 0
0 0 0 0
+1 0 −1 0 0 −1 0 0 +1 0 0 0 +1 0 −1 0 0 0 −1 +1
−1 0 0 0 0 0 +1 0
0 0 0 +1 +1 0 −1 0 −1 0 0 +1 0 0 +1 −2
−1 0 0 0 0 0 +1 0
0 +1 0 0 +1 0 −1 0 , −1 −1 0 +2 0 0 +1 −2
(9.3)
0 +1 0 0 0 −1 0 −1 0 0 +1 0 . 0 0 0 +1 0 −1 0 0 0 −1 −1 +2
Though the complete structure of the minimal Markov bases for general I × J × K problems is not obtained at present, all the minimal Markov bases for 3 × 3 × K cases, 4 × 4 × 4 cases by [83], and also 3 × 4 × K cases by [9] turned out to be unique. These results suggest the following conjecture. Conjecture 9.1. For no-three-factor interaction models of three-way contingency tables, there exists a unique minimal Markov basis. The indispensable move in (9.3) has +2 and −2; that is, both the positive part and the negative part are non-square-free. It is known (cf. Lemma 6.1 of [111]) that existence of an indispensable move with both parts non-square-free implies that the semigroup associated with the configuration is not normal. The example of a hole for the 3 × 4 × 6 contingency table in Sect. 10 of [148] corresponds to the indispensable move (9.3). Results on normality of semigroups for larger tables are summarized in [113]. Normality for the 3 × 5 × 5 case was recently established by [29].
9.6 Reducible Models
149
9.6 Reducible Models Let [m] = {1, . . . , m} be the set of variables and let K denote a simplicial complex on [m]. Denote by D the set of maximal elements (i.e., facets) of K . Then, as seen in Sects. 1.5 and 8.1, the hierarchical model associated with K is defined as log p(ii) =
∑
μD (iiD ).
(9.4)
D∈D
A sufficient statistic t for this model is the set of marginal frequencies for each facet t = {xD (iiD ), i D ∈ ID , D ∈ D}. In the following we identify K with the hierarchical model (9.4). We note that D is considered as a hypergraph such that each facet in D is a hyperedge of D. Here we introduce some notions on hypergraphs according to Badsberg and Malvestuto [20] and Malvestuto and Moscarini [101]. A subset D of a hyperedge of D is called a partial edge. We note that the submodel induced by a partial edge is saturated. A partial edge S is a separator of D if the subhypergraph of D induced by [m] \ S is disconnected. A partial edge separator S of D is called a divider if there exist two vertices u, v ∈ [m] that are separated by S but by no proper subset of S. When D is graphical, a partial edge separator and a divider are the clique separator and clique minimal vertex separator [97], respectively (e.g., Hara and Takemura [78], Leimer [99]). If two vertices u, v ∈ [m] are not separated by any partial edge, u and v are called tightly connected. A subset C ⊂ [m] is called a compact component if every pair of variables in C is tightly connected. Denote the set of maximal compact components of D by C . Then there exists a sequence of maximal compact components C1 , . . . ,C|C | such that (C1 ∪ · · · ∪Ck−1 ) ∩Ck = Sk
(9.5)
and Sk , k = 2, . . . , |C |, are dividers of D. We denote S = {S2 , . . . , S|C | }. S is a multiset in general. The property (9.5) is called the running intersection property. Then cell probability p(ii) is expressed as a rational form of marginal probabilities, p(ii) =
∏C∈C p(iiC ) . ∏S∈S p(iiS )
MLE is expressed as a rational form of the MLE of marginal probabilities, p(i ˆ i) =
ˆ iC ) ˆ iC ) ∏C∈C p(i ∏C∈C p(i = . ˆ iS ) ∏S∈S (x(iiS )/n) ∏S∈S p(i
(9.6)
150
9 No-three-factor interaction models and other hierarchical models
If D does not have a divider (i.e., if |C | = 1), the corresponding hierarchical model (9.4) is called prime. On the other hand, if D has a divider, the corresponding hierarchical model is called reducible (e.g., Develin and Sullivant [49], Hos¸ten and Sullivant [88]).
9.7 Markov Basis for Reducible Models From (9.6), in order to compute the MLE of a reducible model, it suffices to compute the MLEs of marginal models p(iiC ) for each maximal compact component. In the same way, a Markov basis for a reducible hierarchical model is also constructed from Markov bases of marginal models for maximal compact components. In this section we discuss the divide-and-conquer approach to the computation of a Markov basis for reducible models by Hos¸ten and Sullivant [88] and Dobra and Sullivant [54]. For a subset of variables D ⊂ [m], denote by K (D) the submodel induced by D. Let (A1 , A2 , S) be a decomposition of D and define V1 := A1 ∪ S and V2 := A2 ∪ S. Denote by AV1 = {aaV1 (iiV1 )}iV ∈IV and AV2 = {aaV2 (iiV2 )}iV ∈IV the configurations 1 2 1 2 for the marginal models K (V1 ) and K (V2 ), where aV1 (iiV1 ) and aV2 (iiV2 ) denote column vectors of AV1 and AV2 , respectively. Noting that iV1 = (iiA1 i S ) and iV2 = (iiS i A2 ), the configuration A for a reducible model is written by A = AV1 ⊕S AV2 = {aaV1 (iiA1 i S ) ⊕ aV2 (iiS i A2 )}iA
1
where
aV1 (iiA1 i S ) ⊕ aV2 (iiS i A2 ) =
∈IA1 ,ii S ∈IS ,iiA2 ∈IA2 ,
aV1 (iiA1 i S ) aV2 (iiS i A2 )
denotes the stacked vector (1.21). As in previous sections we denote a move z with degree d by z = i 1 · · · i d − i1 · · · i d , where i 1 , . . . , i d ∈ I are cells of positive elements of z and i 1 , . . . , i d ∈ I are cells of negative elements of z . i k appears z(iik ) times in {ii1 , . . . , i d } and in the same way i k appears |z(iik )| times in {ii1 , . . . , i d }. Assume that B(V1 ) and B(V2 ) are Markov bases for K (V1 ) and K (V2 ), respectively. Let z 1 = {z1 (iiV1 )}iV ∈IV and z 2 = {z2 (iiV2 )}iV ∈IV be degree d moves 1 1 2 2 in B(V1 ) and B(V2 ), respectively. Because S is a partial edge separator, K (S) is saturated and we have
∑
iV \S ∈IV \S 1 1
z1 (iiV1 ) = 0,
∑
iV \S ∈IV \S 2 2
z2 (iiV2 ) = 0.
9.7 Markov Basis for Reducible Models
151
Hence z 1 and z 2 can be written as z 1 = (ii1 j 1 ) · · · (iid j d ) − (ii1 j 1 ) · · · (iid j d ), z 2 = ( j 1 k 1 ) · · · ( j d k d ) − ( j 1 k 1 ) · · · ( j d k d ),
(9.7)
respectively, where i l , i l ∈ IA1 , j l ∈ IS and k l , k l ∈ IA2 for l = 1, . . . , d. Definition 9.6 (Dobra and Sullivant [54]). Define z 1 ∈ B(V1 ) as in (9.7). Let k := {kk1 , . . . , k d } ∈ IA2 × · · · × IA2 . Define z k1 by z k1 := (ii1 j 1 k 1 ) · · · (iid j d k d ) − (ii1 j 1 k 1 ) · · · (iid j d k d ). Then we define Ext(B(V1 ) → K ) by Ext(B(V1 ) → K ) := {zzk1 | k ∈ IA2 × · · · × IA2 }. Lemma 9.2. Suppose that z 1 ∈ B(V1 ) as in (9.7). Then Ext(B(V1 ) → K ) is the set of moves for K . Proof. Let z ∈ Ext(B(V1 ) → K ). Then we have Azz = where zV1 (iiV1 ) =
∑iV1 ∈IV1 aV1 (iiV1 )zV1 (iiV1 ) ∑iV2 ∈IV2 aV2 (iiV2 )zV2 (iiV2 )
∑
iV C ∈IV C 1
z(ii),
1
zV2 (iiV2 ) =
,
∑
iV C ∈IV C 2
z(ii ).
2
Because zV1 (iiV1 ) = z1 (iiV1 ) and z 1 ∈ B(V1 ), ∑iV ∈IV aV1 (iiV1 )zV1 (iiV1 ) = 0. From 1 1 Definition 9.6, zV2 (iiV2 ) = 0 for all iV2 ∈ IV2 . Hence Azz = 0. The following theorem by Dobra and Sullivant [54] shows that a Markov basis for K is computed recursively from Markov basis for the induced submodel K (C), C ∈ C. Theorem 9.5 (Dobra and Sullivant [54]). Let B(V1 ) and B(V2 ) be Markov bases for K (V1 ) and K (V2 ), respectively. Let BV1 ,V2 be a Markov basis for the decomposable model with two cliques V1 and V2 . Then B := Ext(B(V1 ) → K ) ∪ Ext(B(V2 ) → K ) ∪ BV1 ,V2
(9.8)
is a Markov basis for K . Proof. Let x , x ∈ Ft be two tables in the same fiber Ft . As in the previous sections we denote x and x as x 1 = (ii1 j 1 k 1 ) · · · (iin j n k n ),
x 2 = (ii1 j 1 k 1 ) · · · (iin j n k n ),
152
9 No-three-factor interaction models and other hierarchical models
Fig. 9.9 The independence graph for D
2 1
4 3
where i k , i k ∈ IA1 , j k , j k ∈ IS , k k , k k ∈ IA2 . Let xV1 and xV 1 be V1 -marginal sum of x and x , respectively. Because xV1 and xV 1 belong to the same fiber for K (V1 ), there exists a sequence of moves zV11 , . . . , zVt 1 connecting xV1 and xV 1 . We note that zV11 is written as zV11 = (ii1 j 1 ) · · · (iim j m ) − (ii1 j 1 ) · · · (iim j m ), where 0 < m < n. Define z 1 by z 1 = (ii1 j 1 k 1 ) · · · (iim j m k m ) − (ii1 j 1 k 1 ) · · · (iim j m k m ).
(9.9)
Then z 1 ∈ Ext(B(V1 ) → K ) and x + z 1 ≥ 0 . By iterating this procedure with zV21 , . . . , zVt 1 , we can define z 2 , . . . , zt ∈ Ext(B(V1 ) → K ) in the same way as (9.9) such that x 1 + ∑tl=1 z l ≥ 0 for all t ≤ t and V1 - and V2 -marginal sums of y := x + ∑tl=1 z s are yV1 = xV 1 and yV2 = xV2 , respectively. On the other hand xV 2 and yV2 also belong to the same fiber for K (V2 ) and hence xV2 and yV2 are connected by moves in B(V2 ). By using the same argument as above, there exist moves w 1 , . . . , w s ∈ Ext(B(V2 ) → K ) such that x 1 + ∑sl=1 w l ≥ 0 for all s ≤ s and V1 - and V2 -marginal sums of y := x + ∑sl=1 w l are yV1 = xV 1 and yV2 = xV2 , respectively.
In general it is not easy to obtain an explicit list of a Markov basis for a hierarchical model. This theorem shows that when we study the structure of Markov bases for hierarchical models, we only need to focus on Markov bases for prime models. Hara et al. [73] extend this result to a more general class of log affine models which is called the hierarchical subspace model. Example 9.1. Consider the model D = {{1, 2}, {1, 3}, {2, 3}, {2, 4}, {3, 4}}. Let K be the corresponding simplicial complex. Assume that the number of levels for all variables are two; that is, Ik = 2 for k = 1, 2, 3, 4. The independence graph for D is described as in Fig. 9.9. In this model S := {2, 3} is a divider and hence K is reducible. Then D is decomposed by S into two no-three-factor interaction models D1 := {{1, 2}, {1, 3}, {2, 3}} and D2 := {{2, 3}, {2, 4}, {3, 4}}. Let V1 := {1, 2, 3} and V2 := {2, 3, 4}.
9.8 Markov Complexity and Graver Complexity
153
A Markov basis for D1 consists of one move i1 \i2 1 2 1 1 −1 2 −1 1 i3 = 1
i1 \i2 1 2 1 −1 1 2 1 −1 i3 = 2
(Develin and Sullivant [49]). This move are described as (111)(212)(221)(122) − (211)(112)(121)(222). By Definition 9.6, moves in Ext(B(V1 ) → K ) are described as (111k1)(212k2 )(221k3 )(122k4) − (211k1)(112k2 )(121k3 )(222k4 ), where k1 , k2 , k3 , k4 ∈ I4 = {1, 2}. For example, when (k1 , k2 , k3 , k4 ) = (1, 1, 2, 2), the corresponding move in Ext(B(V1 ) → K ) is i1 \i2 1 2 i1 \i2 1 2 1 1 0 1 −1 0 2 −1 0 2 1 0 i3 = 1 i3 = 2 i4 = 1
i1 \i2 1 2 i1 \i2 1 0 −1 1 2 0 1 2 i3 = 1 i4 = 2
1 2 0 1 0 −1 . i3 = 2
Moves in Ext(B(V2 ) → K ) are obtained in the same way. BV1 ,V2 is a Markov basis for the decomposable model associated with the graph in Fig. 9.9. BV1 ,V2 is obtained by following the argument in Chap. 8. Then B := Ext(B(V1 ) → K ) ∪ Ext(B(V2 ) → K ) ∪ BV1 ,V2 forms a Markov basis for D.
9.8 Markov Complexity and Graver Complexity In this section we give a brief review of the recent progress on the evaluation of complexity of Markov bases and the Graver basis for hierarchical models. In Sect. 4.6 we discussed the Lawrence lifting of a configuration A : ν × η . Santos and Sturmfels [131] introduced the rth Lawrence lifting (or the rth Lawrence configuration) Λ (r) (A) as the following configuration.
154
9 No-three-factor interaction models and other hierarchical models
⎞ A 0 0 ··· 0 ⎜ 0 A 0 ··· 0 ⎟ ⎟ ⎜ ⎟ ⎜ (r) Λ (A) = ⎜ ... ... ... . . . ... ⎟ : (rν + η ) × (rη ). ⎟ ⎜ ⎝ 0 0 0 ··· A ⎠ Eη Eη Eη · · · Eη ⎛
(9.10)
As in the ordinary Lawrence lifting, we can omit one block of rows from Λ (r) (A) and write the the rth Lawrence lifting also as ⎛ A ⎜0 ⎜ ⎜ Λ˜ (r) (A) = ⎜ ... ⎜ ⎝0
r−1
0 ··· 0 A 0 ··· .. .. .. . . .
0 0 .. .
⎞
⎟ ⎟ ⎟ ⎟ : ((r − 1)ν + η ) × (rη ). ⎟ ··· 0 A 0 ⎠ Eη Eη · · · Eη Eη
(9.11)
It can be easily seen that the configuration of no-three-factor interaction model for I × J × r tables is the rth Lawrence lifting of the configuration A for the I × J twoway independence model. From a statistical viewpoint, as we saw in Sect. 4.6, the Lawrence lifting corresponds to a logistic regression. Similarly the rth Lawrence lifting corresponds to an unordered multinomial logistic regression, where the response variable Y = Yi can take r different levels for each cell i and the probability P(Yi = k), k = 1, . . . , r, is expressed as exp(θ k a(ii)) P(Yi = k) = p(ii, k) = r , (9.12) ∑h=1 exp(θ h a(ii)) where θ k is a ν -dimensional parameter vector, k = 1, . . . , r. For each cell i we observe ni independent trials, each trial taking one of the r levels. Let x(ii, k), k = 1, . . . , r, denote the frequency of the level k among ni trials. Then (x(ii , 1), . . . , x(ii , r)) has the multinomial distribution Mult(ni , (p(ii, 1), . . . , p(ii, r))). For each cell i , the lowest block of rows (Eη , . . . , Eη ) of Λ (r) (A) corresponds to fixing the total number of frequencies ni = x(ii, 1) + · · · + x(ii, r). As explained in Sect. 4.6, when ni s are allowed to vary over nonnegative integers, the rth Lawrence lifting is the configuration for the multinomial logistic regression model. The result on the bound of complexity of Markov basis for 3 × 3 × K tables in Theorem 9.4 led to many investigations of its generalizations. Santos and Sturmfels [131] gave a first general result on the complexity of the Graver basis of Λ (r) (A), which was already mentioned in Proposition 9.1. Note that z ∈ kerZ Λ (r) (A) is an
9.8 Markov Complexity and Graver Complexity
155
η r-dimensional integer column vector and we can express it as z = (zz1 , . . . , z r ). z belongs to kerZ Λ (r) (A) if and only if Azzi = 0, i = 1, . . . , r, and
r
∑ z i = 0.
i=1
We call z i the ith slice of z . Define the type of z by type(zz) = |{i | z i = 0}|, which is the number of nonzero slices of z. Let BGr (Λ (r) (A)) denote the Graver basis of Λ (r) (A). For a given configuration A define the Graver complexity g(A) by g(A) = max{type(zz ) | z ∈ BGr (Λ (r) (A))}. r≥1
As we see below in Proposition 9.2, there is an explicit charact