Series Foreword
The goal of building systems that can adapt to their environments and learn from their experience has attracted researchersfrom many fields , including computer science, engineering, mathematics, physics, neuroscience, and cognitive science. Out of this research has come a wide variety of learning techniques that have the potential to transform many industrial and scientific fields . Recently, several research communities have begun to converge on a common set of issues surrounding supervised, unsupervised, and reinforcement learning problems. The MIT Press Series on Adaptive Computation and Machine Learning seeksto unify the many diverse strands of machine learning research and to foster high-quality research and innovative applications. This book collects recent research on representing, reasoning, and learning with belief networks . Belief networks (also known as graphical models and Bayesian networks) are a widely applicable formalism for compactly representing the joint probability distribution over a set of random variables. Belief networks have revolutionized the development of intelligent systems in many areas. They are now poised to revolutionize the development of learning systems. The papers in this volume reveal the many ways in which ideas from belief networks can be applied to understand and analyze existing learning algorithms (especially for neural networks). They also show how methods from machine learning can be extended to learn the structure and parameters of belief networks . This book is an exciting illustration of the convergence of many disciplines in the study of learning and adaptive computation.
Preface
Graphical models are a marriage between probability theory and graph theory . They provide a natural tool for dealing with two problems that occur throughout applied mathematics and engineering - uncertainty and complexity - and in particular they are playing an increasingly important role in the design and analysis of machine learning algorithms . Fundamental to the idea of a graphical model is the notion of modularity - a complex system is built by combining simpler parts . Probability theory provides the glue whereby the parts are combined , insuring that the system as a whole is consistent , and providing ways to interface models to data . The graph theoretic side of graphical models provides both an intuitively appealing interface by which humans can model highly -interacting sets of variables as well as a data structure that lends itself naturally to the design of efficient general-purpose algorithms . Many of the cl~ sical multivariate probabilistic systems studied in fields such as statistics , systems engineering , information theory , pattern recogni tion and statistical mechanics are special cases of the general graphical model formalism - examples include mixture models , factor analysis , hid den Markov models , Kalman filters and Ising models . The graphical model framework provides a way to view all of these systems as instances of a common underlying formalism . This has many advantages- in particular , specialized techniques that have been developed in one field can be trans ferred between research communities and exploited more widely . Moreover , the graphical model formalism provides a natural framework for the design of new systems . This book presents an in -depth exploration of issues related to learn ing within the graphical model formalism . Four of the chapters are tutorial articles (those by Cowell, MacKay , Jordan , et al., and Heckerman ). The remaining articles cover a wide spectrum of topics of current research in terest . The book is divided into four main sections: Inference , Indepen dence , Foundations for Learning , and Learning from Data . While the sections can be read independently of each other and the articles are to a large extent self-contained , there also is a logical flow to the material . A full appreciation of the material in later sections requires an understanding 1
2 of the material in the earlier sections . The book begins with the topic of probabilistic inference . Inference refers to the problem of calculating the conditional probability distribu tion of a subset of the nodes in a graph given another subset of the nodes. Much effort has gone into the design of efficient and accurate inference algorithms . The book covers three categories of inference algorithms - exact algorithms , variational algorithms and Monte Carlo algorithms . The first chapter , by Cowell , is a tutorial chapter that covers the basics of exact infer ence, with particular focus on the popular junction tree algorithm . This material should be viewed as basic for the understanding of graphical models . A second chapter by Cowell picks up where the former leaves off and covers advanced issues arising in exact inference . Kjcerulff presents a method for increasing the efficiency of the junction tree algorithm . The basic idea is to take advantage of additional independencies which arise due to the partic ular messages arriving at a clique ; this leads to a data structure known as a "nested junction tree ." Dechter presents an alternative perspective on exact inference , based on the notion of "bucket elimination ." This is a unifying perspective that provides insight into the relationship between junction tree and conditioning algorithms , and insight into space/ time tradeoffs . Variational methods provide a framework for the design of approximate inference algorithms . Variational algorithms are deterministic algorithms that provide bounds on probabilities of interest . The chapter by Jordan , Ghahramani , Jaakkola , and Saul is a tutorial chapter that provides a general overview of the variational approach , emphasizing the important role of convexity . The ensuing article by Jaakkola and Jordan proposes a new method for improving the mean field approximation (a particular form of variational approximation ) . In particular , the authors propose to use mixture distributions as approximating distributions within the mean field formalism . The inference section closes with two chapters on Monte Carlo meth ods. Monte Carlo provides a general approach to the design of approximate algorithms based on stochastic sampling . MacKay 's chapter is a tutorial presentation of Monte Carlo algorithms , covering simple methods such M rejection sampling and importance sampling , M well as more sophisticated methods based on Markov chain sampling . A key problem that arises with the Markov chain Monte Carlo approach is the tendency of the algorithms to exhibit random -walk behavior ; this slows the convergence of the algorithms . Neal presents a new approach to this problem , showing how a sophisticated form of overrelaxation can cause the chain to move more systematically along surfaces of high probability . The second section of the book addresses the issue of Independence . Much of the aesthetic appeal of the graphical model formalism comes from
3 the "Markov properties " that graphical models embody . A Markov prop erty is a relationship between the separation properties of nodes in a graph
(e.g., the notion that a subset of nodes is separated from another subset of nodes, given a third subset of nodes) and conditional independenciesin the family of probability distributions associated with the graph (e.g., A is independent of B given C , where A , Band C are subsets of random
variables). In the case of directed graphs and undirected graphs the relationships are well understood (cf . Lauritzen , 1997) . Chain graphs , however, which are mixed graphs containing both directed and undirected edges, are less well understood . The chapter by Richardson explores two of the Markov properties that have been proposed for chain graphs and identifies natural "spatial " conditions on Markov properties that distinguish between these Markov
properties
and those for both directed
and undirected
graphs .
Chain graphs appear to have a richer conditional independence semantics than directed and undirected graphs The chapter by Studeny and Vejnarova addresses the problem of characterizing stochastic dependence. Studeny and Vejnarova discuss the proper ties of the multiinformation function , a general information -theoretic func tion from which many useful quantities can be computed , including the conditional
mutual
information
for all disjoint
subsets of nodes in a graph .
The book then turns to the topic of learning . The section on Founda tions for Learning contains two articles that cover fundamental concepts that are used in many of the following articles . The chapter by H eckerman is a tutorial
article
that
covers
many
of the
basic
ideas
associated
with
learning in graphical models . The focus is on Bayesian methods , both for parameter learning and for structure learning . Neal and Hinton discuss the
expectation-maximization (EM) algorithm . EM plays an important role in the graphical model literature , tying together inference and learning prob lems. In particular , EM is a method for finding maximum likelihood (or maximum a posteriori ) parameter values, by making explicit use of a prob abilistic inference (the "E step" ) . Thus EM - based approaches to learning generally make use of inference algorithms as subroutines . Neal and Hinton describe the EM algorithm as coordinate ascent in an appropriately -defined cost function . This point of view allows them to consider algorithms that take partial E steps, and provides an important justification for the use of approximate inference algorithms in learning . The section on Learning from Data contains a variety of papers concerned with the learning of parameters and structure in graphical models . Bishop provides an overview of latent variable models , focusing on prob abilistic principal component analysis , mixture models , topographic maps and time series analysis . EM algorithms are developed for each case . The
article by Buhmann complements the Bishop article , describing methods
4 for dimensionality reduction , clustering , and data visualization , again with the EM algorithm providing the conceptual framework for the design of the algorithms . Buhmann also presents learning algorithms based on approxi mate inference and deterministic annealing . Friedman and Goldszmidt focus on the problem of representing and learning the local conditional probabilities for graphical models . In partic ular , they are concerned with representations for these probabilities that make explicit the notion of "context -specific independence ," where , for example , A is independent of B for some values of C but not for others . This representation can lead to significantly more parsimonious models than standard techniques . Geiger, Heckerman , and Meek are concerned with the problem of model selection for graphical models with hidden (unobserved ) nodes. They develop asymptotic methods for approximating the marginal likelihood and demonstrate how to carry out the calculations for several cases of practical interest . The paper by Hinton , Sallans , and Ghahramani describes a graphical model called the "hierarchical community of experts " in which a collection of local linear models are used to fit data . As opposed to mixture models , in which each data point is assumed to be generated from a single local model , their model allows a data point to be generated from an arbitrary subset of the available local models . Kearns , Mansour , and Ng provide a careful analysis of the relationships between EM and the K -means algorithm . They discuss an "information -modeling tradeoff ," which characterizes the ability of an algorithm to both find balanced assignments of data to model components , and to find a good overall fit to the data . Monti and Cooper discuss the problem of structural learning in networks with both discrete and continuous nodes. They are particularly concerned with the issue of the discretization of continous data , and how this impacts the performance of a learning algorithm . Saul and Jordan present a method for unsupervised learning in layered neural networks based on mean field theory . They discuss a mean field approximation that is tailored to the case of large networks in which each node has a large number of parents . Smith and Whittaker discuss tests for conditional independence tests in graphical Gaussian models . They show that several of the appropriate statistics turn out to be functions of the sample partial correlation coefficient . They also develop asymptotic expansions for the distributions of the test statistics and compare their accuracy as a function of the dimen sionality of the model . Spiegelhalter , Best , Gilks , and Inskip describe an application of graphical models to the real-life problem of assessing the effectiveness of an immunization program . They demonstrate the use of the graphical model formalism to represent statistical hypotheses of interest and show how Monte Carlo methods can be used for inference . Finally ,
5 Williams provides an overview of Gaussian processes , deriving the Gaussian process approach from a Bayesian point of view , and showing how it can be applied to problems in nonlinear regression, classification , and hierarchical modeling . This volume arose from the proceedings of the International School on Neural
Nets
" E .R . Caianiello
," held
at
the
Ettore
Maiorana
Centre
for
Scientific Culture in Erice , Italy , in September 1996. Lecturers from the school contributed chapters to the volume , and additional authors were asked to contribute chapters to provide a more complete and authoritative coverage of the field . All of the chapters have been carefully edited , following a review process in which each chapter WM scrutinized by two anonymous reviewers and returned to authors for improvement . There are a number of people to thank for their role in organizing the Erice meeting . First I would like to thank Maria Marinaro , who initiated the ongoing series of Schools to honor the memory of E .R . Caianiello , and who co- organized the first meeting . David Heckerman was also a co- organizer of the school , providing helpful advice and encouragement throughout . Anna Esposito at the University of Salerno also deserves sincere thanks for her help in organizing the meeting . The staff at the Ettore Maiorana Centre were exceedingly professional and helpful , initiating the attendees of the school into the wonders of Erice . Funding for the School was provided by the NATO Advanced Study Institute program ; this program provided generous support that allowed nearly 80 students to attend the meeting . I would
also like
to thank
Jon Heiner
, Thomas
Hofmann
, Nuria
Oliver
,
Barbara Rosario , and Jon Yi for their help with preparing the final docu ment
.
Finally , I would like to thank Barbara Rosario , whose fortuitous atten dance as a participant at the Erice meeting rendered the future condition ally independent of the past .
Michael I. Jordan
INTRODUCTION TO INFERENCE FOR BAYESIAN NETWORKS
ROBERT
COWELL
City University , London .
The School of Mathematics, Actuarial Science and Statistics, City University, Northampton Square, London EC1E OHT
1. Introduction The field of Bayesian networks , and graphical models in general , has grown enormously over the last few years, with theoretical and computational developments in many areas. As a consequence there is now a fairly large set of theoretical concepts and results for newcomers to the field to learn . This tutorial aims to give an overview of some of these topics , which hopefully will provide such newcomers a conceptual framework for following the more detailed and advanced work . It begins with revision of some of the basic axioms of pro babili ty theory .
2. Basic axioms of probability Probability
theory
ing
under
degree
data
,
of
at
of
using
of
recent
us
of
obeys
P
(
A
)
,
absence
is
a
of
as
proposition
,
event
1
if
but
system
of
certainty
a
.
reason
-
Within
numerical
the
measure
consistency
of
being
,
denoted
with
and
only
if
A
P
of
(
A
)
:
is
,
the
certain
.
9
,
is
,
was
and
the
abandoned
.
algorithms
AI
number
uncertainty
prohibitive
community
It
that
.
probability
a
encapsulated
with
systems
the
axioms
axioms
logic
cope
computational
within
by
following
expert
efficient
revival
basic
,
to
became
in
of
a
Boolean
made
calculations
inference
had
or
were
the
some
A
the
,
Attempts
for
with
an
deductive
.
,
has
begin
=
the
interpreted
a
development
theory
ability
used
theory
the
which
in
rules
theory
probability
Let
logic
under
is
systems
production
probability
with
is
belief
probability
use
that
inductive
.
expert
sets
as
probability
consistent
hand
Early
by
known
,
framework
the
1
also
uncertainty
Bayesian
is
,
theory
in
the
.
interval
The
prob
[ 0
-
,
1
]
,
10
ROBERT COWELL
2 If A and B are mutually exclusive , then P (A or B ) = P (A ) + P (B ). We will be dealing exclusively with discrete random variables and their probability distributions . Capital letters will denote a variable , or perhaps a set of variables , lower case letter will denote values of variables . Thus suppose A is a random variable having a finite number of mutually exclusive states (al , . . ' , an) . Then P (A ) will be represented by a vector of non negative real numbers P (A ) = (Xl " , . , xn ) where P (A = ai ) = Xi is a scalar , and Ei Xi = 1. A basic concept is that of conditional probability , a statement of which takes the form : Given the event B = b the probability of the event A = a is X, written P (A == a I B == b) = x . It is important to understand that this is not saying : "If B = b is true then the probability of A = a is x " . Instead it says: "If B = b is true , and any other information to hand is irrelevant to A , then P (A == a) == X" . (To see this , consider what the probabilities would be if the state of A was part of the extra information ) . Conditional probabilities are important for building Bayesian networks , as we shall see. But Bayesian networks are also built to facilitate the calculation of conditional probabilities , namely the conditional probabilities for variables of interest given the data (also called evidence) at hand . The fundamental rule for probability calculus is the product rulel
P(A andB) = P(A I B)P(B).
(1)
This equationtells us how to combineconditionalprobabilitiesfor individual variablesto definejoint probabilitiesfor setsof variables. 3. Bayes ' theorem The -
simplest
form
written
marginal
of
Bayes
as
P (A , B )
and
conditional
-
' theorem of
two
relates
events
or
probabilities
the
joint
probability
hypotheses
A
and
P (A B
is This
with
Bayes can
a prior
lOr
more
' theorem be
(2)
probability
I B ) =
P (B
(3)
I A ) P (A ) P (B ) ,
.
interpreted
generally
of
we easily obtain P (A
which
terms
B )
:
P(A, B) = P(A I B)P(B) = P(B I A)P(A). By rearrangement
in
and
as
follows
P ( A ) for
P ( A and
. our
We are interested in A , and we begin belief
about
A ,
B I C ) :=: P ( A I B , C ) P ( B I C ) .
and then
we observe
INTRODUCTION TO INFERENCEFORBAYESIANNETWORKS 11 B . Then Bayes' theorem, (3), tells us that our revisedbelief for A, the posterior probability P(~ I B) is obtainedby multiplying the prior P (A) by the ratio P (B I A)/ P(B ). The quantity P(B I A), asa functionof varying A for fixed B , is calledthe likelihoodof A. We canexpressthis relationship in the form: posterior
cx:
prior
x likelihood
P(A IB) cx : P(A)P(B IA). Figure 1 illustrates this prior -to- posterior inferenceprocess. Each diagram
0 0 P (A ,B )
Figure 1.
0f P (0 B )P (A IB )
Bayesian inference as reversing the arrows
represents in different ways the joint distribution P (A , B ) , the first repre sents the prior beliefs while the third represents the posterior beliefs . Often , we will think of A as a possible "cause" of the "effect" B , the downward arrow represents such a causal interpretation . The "inferential " upwards arrow then represents an "argument against the causal flow " , from the observed effect to the inferred cause. (We will not go into a definition of "causality " here.) Bayesian networks are generally more complicated than the ones in Figure 1, but the general principles are the same in the following sense. A Bayesian network provides a model representation for the joint distri bution of a set of variables in terms of conditional and prior probabilities , in which the orientations of the arrows represent influence , usually though not always of a causal nature , such that these conditional probabilities for these particular orientations are relatively straightforward to specify (from data or eliciting from an expert ) . When data are observed, then typically an inference procedure is required . This involves calculating marginal prob abilities conditional on the observed data using Bayes' theorem , which is diagrammatically equivalent to reversing one or more of the Bayesian network arrows . The algorithms which have been developed in recent years
12
ROBERT COWELL
allows these calculations to be performed in an efficient and straightfor ward
manner
4 . Simple
.
inference
problems
Let us now consider some simple examples of inference . The first is simply Bayes' theorem with evidence included on a simple two node network ; the remaining examples treat a simple three node problem . 4 .1 .
PROBLEM
I
Supposewe have the simple model X - + Y , and are given: P (X ), P (Y I X ) and Y == y . The problem is to calculate P (X I Y == y ) .
Now from P (X ), P (Y I X ) we can calculate the marginal distribution P (Y ) and hence P (Y = y ). Applying Bayes' theorem we obtain
P (X I Y = y) = P (Y = yIX )P (X ) P (Y = y) .
4 .2 .
PROBLEM
(4)
II
Suppose now we have a more complicated model in which X is a par -
ent of both Y and Z : Z +- X -::,. Y with specified probabilities P (X ), P (Y I X ) and P (Z I X ) , and we observe Y = y . The problem is to calculate P (Z I Y = y ) . Note that the joint distribution is given by P (X , Y, Z ) = P (Y I X )P (Z I X )P (X ) . A 'brute force ' method is to calculate :
1. The joint distribution P (X , Y, Z ). 2. The marginal distribution
P (Y ) and thence P (Y = y ) .
3. The marginal distribution P (Z, Y ) and thence P (Z, Y = y). 4. P (Z I Y = y) = P (Z, Y = y)/ P (Y = y). An alternative method is to exploit the given factorization :
1. Calculate P (X I Y = y) = P (Y = y I X )P (X )/ P (Y = y) using Bayes' theorem, where P (Y = y) = Ex P (Y = y I X )P (X ).
2. Find P(Z I Y = y) = Ex P(Z I X )P(X I Y = y). Note that the first step essentially reverses the arrow between X and Y . Although the two methods give the same answer, the second is generally more efficient . For example , suppose that all three variables have 10 states .
Then the first method in explicitly calculating P (X , Y, Z ) requires a table of 1000 states . In contrast the largest table required for the second method has size 100. This gain in computational efficiency by exploiting the given factorizations is the basis of the arc-reversal method for solving influence
INTRODUCTION
diagrams
,
example
shows
4 .3 .
TO
and
of
the
Suppose
calculate
that
P
The
calculational
' sent
'
( Z
I Y
2 .
Find
P
3 .
Find
P
4 .
Find
P
( Z
=
) ,
) .
Note
P
( Z
, X
P
( Y
, X
( X
, Y
( X
)
( Z
, X
)
==
( Z
, Y
=
, Z
the
P
I Y
==
( X
undirected
)
and
( Y
, X
=
P
( Z
I X
) P
( X
)
)
=
P
( Y
I X
) P
( X
)
)
=
P
( Z
, X
proceeds
) P
( Y
using
y
, X
= )
( X
, Y
) P
( X
ExP =
= ) j
( Z
P
( Z
, Y
y
P
=
we
had
( X
, Y
, Z
)
=
P
likewise
the
for , X
=
P x
)
( Y
I Z
Z the
- E-
X
)
=
y
) j
P
( X
-
X
-
problem
XY is
, to
) .
' message
'
in
step
1
which
is
) .
Ez
I X
P
P
) P
( X P
I Y
P
( Y
, X I X
)
=
P
- +
P
P
=
x
conditional X
joint
graphs
( Z
=
of
graph
directed
) j
the
( Z
, Y
=
y
)
( Z
I X
) P
( X
) ,
get
example
fact
.
) .
( Y
=
In
tree
) .
, Y
( Z
( Y
, Z
, X
I X
( Y
( Z )
I X
and
I X
) . P
( Z
) P
Y
with
this can
( Z
( Z
I X
, X
) P
)
)
Hence I Y
given , X
=
X x
( Dawid
distribution be
( X
)
independence
probability
)
)
P
I Z
following
that
=
an
The junction
ZX
Again
, X
a
( X
, X
,
( Y
) .
)
P
( Z
)
we
P
is
a
structure
P
:
Ey
P
=
example
which
P
on
that
now
y
P
and
.
propagation
13
independence
IMt
from
algorithms
using
given
, X
y
P
Conditional
the
NETWORKS
2 :
Calculate
In
are
( Z
steps
step
1 .
.
BAYESIAN
propagation
calculation
we P
P
5
- tree
same
probabilities
in
FOR
III
now
and
junction
the
PROBLEM
INFERENCE
,
factorized
)
=
= P
( 1979 though
x ( Z ) ) .
this
according
Z
+ -
X
- t
Y
:
P
( X
, Y
, Z
)
=
P
( X
Z
- t
X
- t
Y
:
P
( X
, Y
, Z
)
=
P
( Y
) P I X
( Y ) P
I X ( X
Z
+ -
X
+ -
Y
:
P
( X
, Y
, Z
)
=
P
( X
I Y
) P
( Z
) P I Z I X
( Z
I X
) .
) P
( Z
) .
) P
( Y
) .
,
I X
three
we =
We is
to
:
say
obtain x
) .
This
associated not
unique distinct
.
ROBERT COWELL
14
Each of thesefactorizations follows from the conditional independenceproperties which each graph expresses , viz Z 11 Y I X , (which is to be read as "Z is conditionally independent of Y given X" ) and by using the general factorization property : P (X1 , . . . Xn ) = = -=
P (X11 X2, . . . , Xn )P (X2, . . . , Xn ) P (X 1 I X2, . . . , Xn )P (X2 \ X3, . . . , Xn )P (X3, . . . , Xn ) . .. P (X1 / X2, . . . , Xn ) . . . P (Xn - ll Xn )P (Xn ).
Thus for the third example P (X , Y, Z ) = P (Z I X , Y )P (X I Y )P (Y ) = P (Z / X )P (X I Y )P (Y ). Note that the graph Z - t X ~ Y does not obey the conditional independence property Z lL Y I X and is thus excluded from the list ; it factorizes as P (X , Y, Z ) = P (X I Y, Z )P (Z )P (Y ). This example showsseveralfeaturesof generalBayesiannetworks. Firstly , the use of the conditional independenceproperties can be used to simplify the general factorization formula for the joint probability . Secondly, that the result is a factorization that can be expressedby the use of directed acyclic graphs (DAGs). 6. General specification
in DAGs
It is these features which work together nicely for the general specification of Bayesian networks. Thus a Bayesiannetwork is a directed acyclic graph, whosestructure definesa set of conditional independenceproperties. These properties can be found using graphical manipulations, eg d-separation (see eg Pearl(1988)). To each node is associateda conditional probability distri bution , conditioning being on the parents of the node: P (X I pa(X )). The joint density over the set of all variables U is then given by the product of such terms over all nodes: P (U) = IIp x
(X I pa(X )).
This is called a recursivefactorization according to the DAG ; we also talk of the distribution being graphical over the DAG. This factorization is equivalent to the general factorization but takes into account the conditional independenceproperties of the DAG in simplifying individual terms in the product of the general factorization. Only if the DAG is complete will this formula and the general factorization coincide, (but even then only for one ordering of the random variables in the factorization).
INTRODUCTION TO INFERENCE FORBAYESIAN NETWORKS15 6.1. EXAMPLE Considerthe graph of Figure 2.
P(A,B,C,D,E,F,G,H,I) =P(A)P(B)P(C) P(D IA)P(E IA,B)P(F IB,C) P(GIA,D,E)P(H IB,E,F)P(I IC,F). Figure 2.
It
to
is
simply
marginalising
useful
to
removing
note
that
it over
the
and
Nine node example.
marginalising
any variable
over
edges H
to in
it the
from
a
its
above
childless
node
parents gives
. Thus
is
for
equivalent
example
,
:
P(A, B, C, D, E, F, G, I ) = L P(A,B, C, D, E, F, G,H, I ) H = L P(A)P(B)P(C)P(D IA)P(E IA, B)P(F IB, C) H P(GIA, D,E)P(H IB, E, F)P(I IC, F) = P(A)P(B)P(C)P(D IA)P(E IA, B)P(F IB, C) P(GIA, D, E)P(I IC, F) L P(H IB, E, F) H
= P (A)P (B )P (C)P (D I A)P (E I A, B )P (F I B , C) P (G I A , D , E )P (I I C, F ), which can be represented by Figure 2 with H and its incident edges removed .
Directed acyclic graphs can always have their nodes linearly ordered so
that for each node X all of its parents pa(X ) precedesit in the ordering. Such and ordering is called a topological ordering of the nodes. Thus for example (A , B , C, D , E , F , G, H , I ) and (B , A , E , D , G , C, F , I , H ) are two of the many topological orderings of the nodes of Figure 2. A simple algorithm to find a topological ordering is as follows : Start with the graph and an empty list . Then successively delete from the graph any node which
does not have any parents , and add it to the end of the
list . Note that if the graph is not acyclic , then at some stage a graph will be obtained in which no node has no parent nodes, hence this algorithm can be used as an efficient way of checking that the graph is acyclic .
16
ROBERTCOWELL
Another equivalent way is to start with the graph and an empty list , and successively delete nodes which have no children and add them to the
beginning of the list (cf. marginalisation of childless nodes.) 6 .2 . DIRECTED
MARKOV
PROPERTY
An important property is the directed Markov property . This is a condi tional independence property which states that a variable is conditionally independent of its non-descendents given it parents :
X Jl nd(X ) I pa(X ). Now recall that the conditional probability P (X I pa (X )) did not necessarily
mean that if pa(X ) = 7r* say, then P (X = x ) = P (x 17r*), but included the
caveat
For
the
that
DAGs
any this
other
information
' other
information
is irrelevant
to X
' means , from
the
for
this
to hold .
directed
Markov
property , knowledge about the node itself or any of its descendents. For if all of the parents of X are observed, but additionally observed are one or more descendents Dx of X , then because X influences Dx , knowing
D x and pa(X ) is more informative than simply knowing about pa(X ) alone . However having information about a non-descendent does not tell us anything more about X , because either it cannot influence or be influenced by X either directly or indirectly , or if it can influence X indirectly , then only through influencing the parents which are all known anyway . For example , consider again Figure 2. Using the previous second topological ordering we may write the general factorization as:
P (A , B , C, D , E , F, G, I , H ) = P (B ) * P (A I B ) * P (EIB , A ) * P (D I B , A , E ) * P (G f B , A , E , D ) * P (C I B , A , E , D , G) * P (FIB , A , E , D , G, C) * P (I I B , A , E , D , G, C, F ) * P (HIB , A , E , D , G, C, F , I )
(5)
but now we can use A lL B from the directed Markov property to simplify
P (A I B ) - t P (A ), and similarly for the other factors in (5) etc, to obtain the factorization
in Figure 2. We can write the general pseudo- algorithm of
what we have just done for this example as
INTRODUCTION
TO INFERENCE
Topological
FOR BAYESIAN
ordering
General factorization Directed :~
7.
Making
the
inference
Markov
17
NETWORKS
+ +
property
Recursive factorization
.
engine
We shall now move on to building the so called " inference engine " to in troduce new concepts and to show how they relate to the conditional in dependence / recursive factorization ideas that have already been touched upon . Detailed justification of the results will be omitted , the aim here is to give an overview , using the use the fictional ASIA example of Lauritzen and Spiegelhalter . 7 .1 .
ASIA : SPECIFICATION
Lauritzen lows :
and Spiegelhalter
describe their fictional
problem
domain
as fol -
Shortness -of -breath (Dyspnoea ) may be due to Tuberculosis , Lung can cer or Bronchitis , or none of them , or more than one of them . A recent visit to Asia increases the chances of Tuberculosis , while Smoking is known to be a risk factor for both Lung cancer and Bronchitis . The results of a single X -ray do not discriminate between Lung cancer and Tuberculosis , as neither does the presence or absence of Dyspnoea .
@ / @ I
P(U) =P(A)P(S) P(T IA)P(L I S) P(B I S)P(E I L, T) P(D IB, E)P(X I E)
Figure3. ASIA
18
ROBERTCOWELL
The network for this fictional example is shown in Figure 3. Each vari able is a binary with the states ( " yes" , " no " ) . The E node is a logical node taking value " yes" if either of its parents take a " yes" value , and " no " oth erwise ; its introduction facilitates Lung cancer and Tuberculosis .
modelling
the relationship
of X -ray to
Having specified the relevant variables , and defined their dependence with the graph , we must now assign (conditional ) probabilities to the nodes . In real life examples such probabilities may be elicited either from some large database (if one is available ) as frequency ratios , or subjectively from the expert from whom the structure has been elicited (eg using a fictitious gambling scenario or probability wheel ) , or a combination of both . However M this is a fictional example we can follow the third values . (Specific values will be omitted here .) 7.2. CONSTRUCTING
THE INFERENCE
route and use made - up
ENGINE
With our specified graphical model we have a representation density in terms of a factorization : P (U )
=
IIP v
(Vlpa
(V ) )
=
P (A ) . . . P (X I E ) .
of the joint
(6) (7)
Recall that our motivation is to use the model specified by the joint distri bution to calculate marginal distributions conditional on some observation of one or more variables . In general the full distribution will be computa tionally difficult to use directly to calculate these marginals directly . We will now proceed to outline the various stages that are performed to find a representation of P ( U ) which makes the calculations more tractable . (The process of constructing the inference engine from the model specification is sometimes called compiling the model .) The manipulations required are almost all graphical . There are five stages in the graphical manipulations . Let us first list them , and then go back and define new terms which are introduced . 1. Add undirected edges to all co- parents which are not currently joined (a process called marrying parents ) . 2. Drop all directions in the graph obtained from Stage 1. The result is the so- called moral graph . 3 . Triangulate the moral graph , that is , add sufficient additional undi rected links between nodes such that there are no cycles (ie . closed paths ) of length 4 or more distinct nodes without a short -cut . 4 . Identify the cliques of this triangulated graph . 5. Join the cliques together to form the junction tree .
INTRODUCTION TO INFERENCEFOR BAYESIAN NETWORKS 19 Now let us go through these steps, supplying somejustification and defining the new terms just introduced as we go along. Consider first the joint density again. By a changeof notation this can be written in the form
P(U)
-
-
(8)
II a(V,pa(V)) v a(A) ...a(X,E).
(9)
where a(X , pa(X )) == P (V I pa(V )). That is, the conditional probability factors for V can be consideredas a function of V and its parents. We call such functions potentials. Now after steps 1 and 2 we have an undirected graph, in which for each node both it and its set of parents in the original graph form a complete subgraph in the moral graph. (A complete graph is one in which every pair of nodes is joined together by an edge.) Hence, the original factorization of P (U) on the DAG G goesover to an equivalent factorization on these complete subsetsin the moral graph Cm. Technically we say that the distribution is graphical on the undirected graph Gm. Figure 4 illustrates the moralisation processfor the Asia network. Now let us de-
0
0
j
0
@
@
""
@
@
Figure4. MoralisingAsia: Two extra links arerequired, A - Sand L - B . Directionality is droppedafter all moral edgeshavebeenadded. note the set of cliques of the moral graph by om . (A clique is a complete subgraph which is not itself a proper subgraph of a complete subgraph , so it is a maximal complete subgraph .) Then each of the complete subgraphs formed from { V } U pa (V ) is contained within at least one clique . Hence we can form functions ac such that
P(U) =
nac(Vc) cEcm
where ac (Vc ) is a function of the variables in the clique C . Such a factoriza tion can be constructed as follows : Initially define each factor as unity , i .e.,
20 ac
ROBERT COWELL
(
Vc
)
and
only
and
multiply
to
=
1
for
one
a
Note
not
that
by
Those
:
fact
an
are
ancestor
is
separates
a
there
the
the
First
)
one
( V
I
pa
the
of
{
)
is
a
functions
)
V
of
result
of
( V
function
find
}
one
U
pa
that
( V
)
clique
potential
on
rep
the
in
the
-
cliques
of
set
A
and
B
A
is
S
the
of
The
.
B
The
A
or
in
node
a
definitions
Y
A
E
we
A
:
the
moral
an
node
of
a
A
set
and
have
is
is
a
.
-
A
of
set
nodes
a
)
set
ancestral
the
node
( ii
ancestral
of
these
condi
.
parent
.
between
With
exploit
elicudating
definitions
sets
path
.
a
B
ancestors
every
of
to
.
.
for
more
of
its
is
original
sets
ancestral
if
node
used
later
construction
( i )
it
the
specification
are
described
parents
the
numerical
graph
be
some
of
of
process
of
the
ancestral
either
the
moralisation
in
moral
require
if
the
"
powerful
of
the
some
buried
will
we
union
sets
a
in
independences
on
B
and
is
"
"
is
.
node
least
through
Lemma
done
conditional
which
itself
Y
passes
the
terms
edges
the
still
graph
( at
node
nodes
of
properties
of
the
of
is
in
" visible
moral
of
P
subgraph
into
extra
all
Markov
the
factor
complete
this
the
of
independence
ancestor
When
computations
Aside
tional
adding
remain
local
In
.
they
which
.
each
the
distributions
read
though
efficient
for
distribution
joint
to
,
Then
.
possible
DAG
.
contains
function
the
am
am
conditional
new
of moral
in
which
this
obtain
the
cliques
clique
resentation
8
all
b
set
S
E
B
1
Let
P
factorize
recursively
according
to
g
.
Then
AlLEIS
whenever
of
A
the
A
,
Band
separates
sets
To
B
of
S
in
( QAn
A
can
and
,
we
in
subgraph
the
of
if
is
look
U
if
( AUBUS
BUS
)
) m
,
graph
.
d
to
Y
.
is
When
into
them
is
in
.
set
which
or
not
ancestral
,
Suppose
a
no
possible
any
set
.
S
) )
let
set
of
children
longer
m
in
the
.
-
smallest
calculation
us
that
of
Then
( AUBUS
of
picture
have
An
.
conditional
ways
the
Q
( Q
check
alternative
ancestral
it
minimal
B
properties
are
G
graph
to
- separation
from
acyclic
from
want
come
the
,
A
finding
find
the
directed
we
they
set
nodes
left
that
for
wish
set
separates
-
ancestral
a
S
at
graphs
delete
not
the
us
algorithm
that
successively
are
only
either
simple
G
and
moral
why
following
Then
if
subsets
tell
the
understand
graph
nodes
by
containing
disjoint
lemmas
we
ancestral
be
from
these
dependences
they
separated
set
S
A
What
the
are
2
Let
-
B
ancestral
Lemma
d
and
smallest
.
consider
we
have
nodes
Y
,
the
~
U
provided
delete
any
.
INTRODUCTION TO INFERENCEFOR BAYESIAN NETWORKS 21 Now recall that deleting a childless node is equivalent to marginalising over that node. Hence the marginal distribution of the minimal ancestral set containing A lL B I S factorizes according to the sub-factors of the original joint distribution . So these lemmas are saying that rather than go through the numerical exercise of actually calculating such marginals we can read it off from the graphical structure instead, and use that to test conditional independences. (Note also that the directed Markov property is also lurking behind the sceneshere.) The "moral" is that when ancestral sets appear in theorems like this it is likely that such marginals are being considered. 9. Making the junction
tree
The remaining three steps of the inference-engine construction algorithm seem more mysterious , but are required to ensure we can formulate a consistent and efficient message passing scheme. Consider first step 3 - adding edges to the moral graph am to form a triangulated graph Gt . Note that adding edges to the graph does not stop a clique of the moral graph formed from being a complete subgraph in at . Thus for each clique in am of the moral graph there is at least one clique in the triangulated graph which contains it . Hence we can form a potential representation of the joint prob ability in terms of products of functions of the cliques in the triangulated graph :
P(U) =
II cECt ac(Xc )
by analogy with the previous method outline for the moral graph . The point is that after moralisation and triangulation there exists for each a node-parent set at least one clique which contains it , and thus a potential representation can be formed on the cliques of the triangulated graph . While the moralisation of a graph is unique , there are in general many alternative triangulations of a moral graph . In the extreme , we can always add edges to make the moral graph complete . There is then one large clique . The key to the success of the computational algorithms is to form triangulated graphs which have small cliques , in terms of their state space . SIze. Thus after finding the cliques of the triangulated graph - stage 4 - we are left with joining them up to form a junction tree . The important prop erty of the junction tree is the running intersection property which means that if variable V is contained in two cliques , then it is contained in every clique along the path connecting those two cliques . The edge joining two cliques is called a separator . This joining up property can always be done , not necessarily uniquely for each triangulated graph . However the choice of
22
ROBERT COWELL
tree
is
immaterial
junction
except
tree
pendence
of
the
( not
.
necessarily
this
.
sage
The
running
algorithms
.
possible
If
.
the
ie
,
5
shows
of
a
!
)
@
~
"
-
j
Inference
:
~ ~ - _/
"
"
manageable
in
the
of
size
then
version
basic
the
is
the
mes
unit
of
-
the
computational
local
of
It
computation
Asia
and
a
possible
......
, , " "' .. .-..
the
summarise
tree
joint
consistence
.
becomes
@
on
unction
the
them
passing
become
to
between
@
FIgure
will
edges
between
granularity
-
some
~
@
We
loses
@
V
.
It
.
/
10
cliques
The
inde
extra
message
ensures
triangulated
.
adding
separators
the
.
conditional
independence
with
the
the
DAG
of
given
define
are
0
(
cliques
and
they
cliques
Figure
tree
. ,
of
original
process
property
cliques
,
,
conditional
computation
intersection
between
the
the
retain
)
local
all
on
does
considerations
necessarily
by
it
that
computation
junction
efficiency
not
distribution
However
fact
passing
local
the
but
neighbouring
of
possible
,
independences
graph
because
is
of
conditional
moral
computational
many
properties
the
for
captures
.
We
have
Junction
junction
some
probability
5 .
seen
the
basic
Asia
that
results
we
can
functions
P
for
tree
of
using
tree
( U
of
form
a
defined
)
=
II
the
( Xc
passing
potential
on
ac
message
)
on
the
representation
cliques
of
:
.
CEct
This
sections
potential
can
be
of
generalized
to
neighbouring
representation
cliques
include
functions
)
to
form
on
the
the
following
:
P(U)=D _ CECt ac Xc llSESt bs ((Xs )).
separators
so
( the
called
generalized
inter
-
INTRODUCTION TO INFERENCE FORBAYESIAN NETWORKS23 (for instanceby makingthe separatorfunctionsthe identity). Now, by sendingmessages betweenneighbouring cliquesconsistingof functionsof the separatorvariablesonly, whichmodifythe interveningseparatorand the cliquereceivingthe message , but in sucha waythat the overallratio of productsremainsinvariant,wecanarriveat the followingmarginal representation : p(U) = llCECp(C). llSESp(S)
(10)
Marginalsfor individualvariablescanbe obtainedfrom theseclique(or separator ) marginalsby furthermarginalisation . Suppose that weobserve "evidence " , : X A = xA' Definea newfunction P* by . P*(x) = { 0 P(x) otherwIse if XA ~ xA
(11)
ThenP*(U) = P(U, ) = P([' )P(U I[,). Wecanrewrite(11) as P*(U) = P(u) n l (v), vEA
(12)
wherel (v) is 1 if Xv==x~, 0 otherwise . Thusl(v) is the likelihood function basedon the partialevidence Xv = x~. Clearlythis alsofactorizes on the junctiontree, andby message passingwemayobtainthe followingclique marginalrepresentation p(VI ) = llCECP(CIt :) . llSESP(SI )
(13)
or by omittingthe normalization stage , p(V, ) =: rICECP (C, ) . rIsEsp(S, )
(14)
Againmarginaldistributionsfor individualvariables , conditionaluponthe evidence , canbe obtainedby furthermarginalisation of individualclique tables, as can the probability(accordingto the model) of the evidence , P( ). 11. Why the junction tree? Giventhat themoralgraphhasniceproperties , whyis it necessary to goon to formthejunctiontree? Thisis bestillustratedby anexample , Figure6:
24
ROBERTCOWELL A
@-----( )---- @ ) E Figure 6.
A non-triangulated graph
The cliquesare (A, B , C), (A, C, D), (C, D, F ), (C, E, F ) and (B , C, E) with successive intersections(A, C), (C, D), (C, F ), (C, E) and (B , C). Suppose we havecliquemarginalsP (A, B, C) etc.. WecannotexpressP(A, B , C, D) in terms of P(A, B , C) and P (A, C, D) - the graphicalstructure doesnot imply B Jl D I(A, C). In generalthere is no closedfor expressionfor the joint distribution of all six variablesin terms of its cliquesmarginals. 12 . Those extra
edges again
Having explained why the cliques of the moral graph are generally not up to being used for local message p~ sing , we will now close by indicating where the extra edges to form a triangulated graph come from . Our basic message passing algorithm will be one in which marginals of the potentials in the cliques will form the messages on the junction tree . So let us begin with our moral graph with a potential representation in terms of functions on the cliques , and suppose we marginalise a variable Y say,which belongs to more than one clique of the graph , say two cliques , 01 and O2, with variables Y U Zl and Y U Z2 respectively . They are cliques , but the combined set of variables do not form a single clique , hence there must be at least one pair of variables , one in each clique , which are not joined to each other , Ul and U2 say. Now consider the effect of marginalisation of the variable Y . We will have
L aCt(Y UZl)aC2 {Y UZ2) ==f {Zl UZ2), y a function of the combined variables of the two cliques minus Y . Now this function cannot be accommodated by a clique in the moral graph because the variables ul and U2 are not joined (and there may be others ) .
INTRODUCTION TOINFERENCE FORBAYESIAN NETWORKS25 Hence
we cannot
P (U the
Y ) on
missing
is
can
P (U why
such
one
adds
Pearl
is one
reasoning book
properties
good
collection
which
covers
for
be
doing
scheme
in this
representation extra
able
out
fill
, then
edges
to
that
. This
accommodate one
so results
must
fill - in
in being
able
.
Bayesian intelligence
of
its
. This
in
junction
expert
systems
for
graphical
connected
DAGs with
is Shafer but
also
and
other
overviews
selected
by
papers
number
text
-
prob
-
models
;
( ie prior them
Pearl
.) A
( 1990 ) ,
formalisms
for
the
ex -
. An
is ( Spiegelhalter
a large
. His
introducing
propagating
good
of the
contain
community , from
reasoning
contains
uncertain
and
reasoning
also
for
singly
trees
uncertain
significance
references
material
use ; axiomatics
propagation
probabilistic
methods
editors
introductory
et ai . , 1993 ) . Each
of
references
for
further
. ( 1979 ) introduced
dependence phasis
. More
on
showed
Asia how
' s using
J unction in
do
relational
on this
given
in other
also
the
by
for
treating
conditional
independence
properties
latter
Lauritzen
also
are
in -
with
given
contains
discussion
textbook
reprinted
and
) ; see
by
proofs
Spiegelhalter
em Whit of
-
the
( Lauritzen section
Bayesian
( 1988 ) , who
in multiply
in
are known
in junction on
and
calculations
is also
areas
of propagation
introductory
Markov
probability , ( it
databases
and
their
basis conditional
8 .)
was
consistent
arise
of
( 1996 ) . ( The
section
propagation
trees
formulation
A recent
in
to
and
Lauritzen
example
axiomatic
accounts
models
and
stated
The
the
recent
graphical
( 1990 )
lemmas
more
, and
artificial
for
on
probabilistic
Dawid
eral
these
, to
a wealth
of making
only
a potential
of
, if we
cliques
find
turns
helped
in the
; etc , to
historical
three
reading
who
arguments
uncertainty
these
taker
popular
not
two
having
. It
distribution
. However
of the
graph
graph
joint
reading
of papers
the
moral
passing
pioneers
and
removed
graph
expressions
, 1988 ) contains
development
review
the
Y
we can
moral
message
the
to the
node
of the
of variables
, and
to
further
theory
plaining
pairs
a triangulated
become
handling
with
the
marginal
Markov
trees
edges
of
( Pearl
ability
graph
reduced
a consistent
Suggested
DAG
the
to form
up
representation
be accommodated
13 .
of
moral
intermediate
set
a potential
between
Y ) on
sufficiently to
the
edges
marginal for
form
( Shafer
and
by different and
connected Pearl
, 1990 ) ) .
names
( eg join
Spiegelhalter
of that
paper
trees
is given
networks
, 1988 ) for
. A recent by
and
Dawid
is ( Jensen
gen -
( 1992 ) .
, 1996 ) .
References Dawid , A . P. (1979) . Conditional independence in statistical theory (with discussion). Journal of the Royal Statistical Society, Series B, 41 , pp. 1- 31.
26
ROBERTCOWELL
Dawid
, A . P . ( 1992 expert
Jensen
Applications
) .
An ) .
, S . L . and
on
graphical
Journal , J . Mateo
( 1988 .
the ) .
, G . R . and mann
Spiegelhalter analysis Whittaker Sons
and introduction
, San
Pearl
Mateo
expert
, J . ( 1990 , Chichester
) . .
) .
application
, J . ( ed . ) ( 1990
Graphical
, Series in
) .
Local to
. UCL
B , 50
, pp
systems
in
Press
probabilistic
, London
computations expert
intelligent
Readings
for
uncertain
systems . 157 - 224 . Morgan
reasoning
with ( with
.
probabilities discussion
) .
. Kaufmann
, San
. Morgan
Kauf
.
, A . P . , Lauritzen
systems
networks
, D . J . ( 1988
Society
algorithm
. 25 - 36 .
Bayesian .
inference
, California
propagation , 2 , pp
. CUP
their
Statistical
, D . J . , Dawid in
to
and
Probabilistic
general
models
Spiegelhalter
Royal
a
Computing
Graphical
structures of
of
Statistics
, S . L . ( 1996
Lauritzen
Shafer
.
, F . V . ( 1996
Lauritzen
Pearl
) .
systems
.
Statistical models
in
, S . L . , and Science
, 8 , pp
applied
multivariate
Cowell
, R . G . ( 1993
) .
Bayesian
. 219 - 47 . statistics
. John
Wiley
and
-
ADVANCED INFERENCE IN BAYESIAN NETWORKS
RO BERT
COWELL
City University , London .
The Schoolof Mathematics , Actuarial Scienceand Statistics, City University, NorthamptonSquare , LondonEC"lE OHT
1. Introduction The previous chapter introduced inference in discrete variable Bayesian net works . This used evidence propagation on the junction tree to find marginal distributions of interest . This chapter presents a tutorial introduction to some of the various types of calculations which can also be performed with the junction tree , specifically : -
Sampling . Most likely configurations . Fast retraction . Gaussian and conditional Gaussian models .
A common theme of these methods is that of a localized message-passing algorithm , but with different 'marginalisation ' methods and potentials tak ing part in the message passing operations . 2 . Sampling Let us begin with the simple simulation problem . Given some evidence t: on a (possibly empty ) set of variable X E , we might wish to simulate one or more values for the unobserved variables .
2.1. SAMPLING IN DAGS Henrion proposedan algorithm called probabilistic logic sampling for DAGs, which works as follows. One first finds a topological ordering of the nodes of the DAG Q. Let us denote the ordering by (Xl , X2, . . . , Xn ) say after relabeling the nodes, so that all parents of a node precedeit in the ordering, hence any parent of X j will have an index i < j . 27
28
ROBERT COWELL Assume
from
if
at
P
X2
( XI
)
h ~
no
Otherwise
h
the
X
stage will
state
1[ * .
*
=
, x2
the
we
over
Now
suppose can
samples are
say
that
.
x ;
at
Xl
rejection
.
,
will
x
is
This a
.
We
Let
us
of
the
see
we the
( U
small
junction
To
draw
an
that
made
directed
rators
such
graph and that
analogy
a
which
clique as that
have
=
ordering
: all it
II
P
( C
the
I
) j
is
the
, 01 a
,
because
the
Then
,
correct
if
it
otherwise
the
the
.
that
This
when
a
distribution
networks
, of
can
the
start
case so
number
is
we
current
of sampling
let
.
alone
nodes
be
large bearing
used
to
sample
and
edges
topological
a
P
tree
( S
from that
a
marginal
:
I t : ) .
tree , 8m
the also
, Om ordering
)
DAG
edges
of
directed
from is
the
the
are
away
, . . .
have
s
,
also
and
junction
n
sampling
root
point is
, 81
,
from
evidence on
direct as
follows
( 00
suppose evidence
continue
.
ensure
Thus
logic
tree
propagated
I t : )
the
they
to
TREE
probability
fixed
because
separators this
to is
previous
the
probabilistic
the
c
pose
) .
from
with
, .
The
from
node
quite
JUNCTION
joint
P
I
say
drawn
cases
.
THE
that
the
.
probabilities
' s
even
how
evidence
USING
of
of
such
steps
( U
for
case
independently
then
next
is
of
nodes
xi
the
the
correctly
set
random
generated
exponentially now
and
obtain
Henrion
with
}
the
definite
sampled
known
at
it is
is
all
some
rejection
x
to
generated sampling
with
assume
j
because
sampled
more
.
no
general
obtain
a
P
balancing
that
or
j
=
values
correct
is
one
X
ie
say
are In
I
Now
x2
) .
in
will
* ) ,
be
drawn
sample
the
successfully
x ;
X
to
all
shall
representation
set
proceed
problem
even
SAMPLING
we
u
xi
be
generate
will
of
been
samples
increases
efficiently
2 .2 .
,
=
to
.
there
1[ * )
we
X
X2
obtain
=
=
,
distribution state
to
Xl
)
introduces
correct
have
one
rejection
rejection
evidence
}
now
simply
,
( U
on
one
the
not
ensures
However ones
cannot
discarding
case
interest
We
( Xj
nodes
case
evidence
the
( X21
and
the
such
, but
I pa
wish
)
sample
node
ordering P
( Xj
of
we
Each
that
case
== ,
.
If
for
( X2
themselves
P
) .
have
but
Instead
step
complete
Xl
from ,
P
can
state
P
from
each
( U
one
a
topological
probability
applied
j
.
again
sampled
we
drawn
distribution
case
from
been
Then
from
the
sample
sample
P
be
resulting
can
with
with
stage
x }
~ )
.
samples
sample ( by
sampled
that
still
at
be
the
, x
again
are
we
. . .
can
we
distribution
begin
scheme
to
'
full
evidence one
parent
hence
have
no
Then
we
a
can
we
.
then
already
When
is
say
as
we
have
( xi
from
I
) ;
jth
there xi
,
~
possibilities
parents
u
that
obtain
parents
i t
other in
first
to
,
the
between root acyclic
such
that of
the
let
us
now
junction cliques
.
The
.
Let 00 nodes
result us is
and is
label the in
sup
tree
a the
root the
-
are sepa
-
directed cliques clique directed
,
/"/-.[~ / ~ ~ /]-".'Dir ["jun ]'~ -tre .".
ADVANCED INFERENCE IN BAYESIAN NETWORKS
29
tree, and with Si the parent of Ci ; seeFigure 1. (Note that this notation
Figure 1.
has a subtlety that the tree is connected , but the disconnected case is easily dealt with .) Then we may divide the contents of the parent separator Si into the clique table in Ci to obtain the following representation :
m P(UIf;) = P(XcoIf;) 11P(XCi\SiIXSi,f;). i=l
This is called the set-chain representation , and is now in a form similar to the recursive factorization on the DAG discussed earlier , and can be sampled from in a similar manner . The difference is that instead of sampling individual variables at a time , one samples groups of variables in the cliques . Thus one begins by sampling a configuration in the root clique , drawing from P (Xco , ) to obtain xco say. Next one samples from P (XCl \ Sl I XS1-' ) where the states of the variables for XS1 are fixed by xco because XS1 C XCo . One continues in this way, so that when sampling in clique Ci the variables XSi will already have been fixed by earlier sampling , as in direct sampling from the DAG . Thus one can sample directly from the correct distribution and avoid the inefficiencies of the rejection method .
3. Most likely configurations One contributing reason why local propagation on the junction tree to find marginals "works " is that there is a "commutation behaviour " between the
30
ROBERTCOWELL
operation of summation and the product form of the joint density on the tree , which allows one to move summation operations through the terms in the product , for example :
L,B,Cf(A,B)f(B,C)=2 :::,Bf(A,B)LCf(B,C). A A However summation is not the only operation which has this another very useful operation is maximization , for example :
property ;
A ,B,Cf{A,B)f{B,C)=max A,B(f{A,B)maxf C {B,C)) , max provided the factors are non-negative, a condition which will hold for clique and separator potentials representing probability distributions .
3.1. MAX-PROPAGATION So suppose we have a junction tree representation of a probability distri -
nca(C)j nsb(S) b*(8)=max C\Sa(C), (which can be performed
locally through
the commutation
property
above )
what do weget? The answeris the max-marginalrepresentation of thejoint density:
-=rIG maxU \G P ( U ,& ) rIs maxU \SP(S,).
P(U,&) = no (c, nspmax pmax (s,t:)) The interpretation isthatforeach configuration c* ofthevariables inthe clique C , the value Pcax (c* ) is the highest probability value that any configuration of all the variables can take subject to the constraint that the variables of the clique have states c* . (One simple consequence is that this most likely value appears at leMt once in ever). clique and separator .) To see how this can come about , consider a simple tree with two sets of variables in each clique :
1 P(A,B,C,E:) = a(A,B)~
a(B,C).
ADVANCED INFERENCE IN BAYESIAN NETWORKS
31
Now recall that the messagep~ sing leaves invariant the overall distribution . So take ~ maximization
to be the root clique , and send then the first message, a over C :
b*(B) = maxa c (B,C). after "collecting" this messagewe have the representation:
P(A,B,C, ) = (a(A,B)b (B} b;(:1B)a(B,C). b*(B)) The root clique now holds the table obtained over C , because pmax (A , B , )
by maximizing
P (A , B , C , )
:=
max c P (A , B , C , )
=
mgx ( a (A , B ) -:r;rii b* (B ) ) biB1 ) a (B , C )
=
B ) )- ) mgx b; (1:B ) a (B , C ) ( a (A , B ) b b*("(B
=
( a (A , B ) -:r;rii b* (B ) ) .
By symmetry the distribute message results in the second clique table hold ing the max - marginal value maxA P (A , B , C , ) and the intervening separa tor holding maxA ,C P (A , B , C , E ) . The more general result can be obtained by induction on the numbers of cliques in the junction tree . (Note that one can pass back to the sum - marginal representation from the max - marginal representation by a sum -propagation .) A separate but related task is to find the configuration of the variables which takes this highest probability . The procedure is as follows : first from a general potential representation with some clique Co chosen as root , per form a collect operation using maximization instead of summation . Then , search the root clique for the configuration of its variables , Co say, which has the highest probability . Distribute this as extra " evidence " , fixing succes sively the remaining variables in the cliques further from the root by finding a maximal configuration consistent with the neighbouring clique which has also been fixed , and including the states of the newly fixed variables as evi dence , until all cliques have been so processed . The union of the " evidence " yields the most likely configuration . If there is " real " evidence then this is incorporated in the usual way in the collect operation . The interpretation is that the resulting configuration acts as a most likely explanation for the data .
32
ROBERTCOWELL
Note the similarity to simulation, where one first does a collect to the root using ordinary marginalisation, then does a distribute by first randomly selecting a configuration from the root , and then randomly selecting configurations from cliques successivelyfurther out.
3.2. DEGENERACY OFMAXIMUM It is possible to find the degeneracy of the most likely configuration , that is the total number of distinct configurations which have the same maximum probability pmax (U I ) = p* by a simple trick . (For most realistic applica tions there is unlikely to be any degeneracy, although this might not be true for ego genetic -pedigree problems .) First one performs a max -propagation to obtain the max -marginal representation . Then one sets each value in each clique and separator to either 0 or 1, depending on whether or not it has attained the maximum probability value , thus :
1 if Pcax (XG1 ) = p* Ic (xc 1 ) = { 0 otherwise , and
.1 Is(xs 1)={I0otherwise ifp.s;n(xs ax .)=p* Then
I (UI ) = 11Ic(xcI )j n Is(Xs1 ) c s is a potential representation of the indicator function of most likely config urations , a simple sum-propagation on this will yield the degeneracy as the normalization .
3.3. TOP N CONFIGURATIONS In finding the degeneracy of the most likely configuration in the previous section , we performed a max -propagation and then set clique elements to zero which did not have the value of the highest probability . One might be tempted to think that if instead we set to zero all those elements which are below a certain threshold p < 1 then we will obtain the number of configurations having probability ~ p . It turns out that one can indeed find these configurations after one max -propagation , but unfortunately not by such a simple method . We will discuss a simplified version of an algorithm by Dennis Nilsson which allows one to calculate the top N configurations
ADVANCED
by
a
sequence
can
be To
have (X
by
of
propagations
found
after
a
begin
with
an
, . . .
do
a
M1
M2
= state
, Xn
max
=
the
of
(X ~ 2
(x ~
) ,
that
and
let
propagations
least
we ,
)
has
no
the
j - th find
recently
evidence
ordering
, and
33
shown
how
they
.)
have
any
do
have
the
.
will
node
Necessarily
must
one
with
( 1997
NETWORKS
kj
most
The
.
first
Let
states
likely
step
is
us
write
( xJ
, . . . x ~j ) .
to
this
configuration
as We
, denoted
1
l , . . . x ~ n ) . 2
at
BAYESIAN
- propagation
nodes
1 , . . . x ~ n ) ,
of
max
the
- propagation 1
IN
. ( Nilsson
single
assume
ordering
1 , X2
now
INFERENCE
the
differ
from
variable
" pseudo
second
.
the
So
- evidence
- most most
now
"
j
we
as
likely
configuration
likely
configuration
perform
follows
,
a
in
further
n
max
-
.
ml & 1
=
Xl
#
Xl
1 ml
& 2
=
Xl
=
Xl
ml 1 and
X
2
#
X2
2
ml & 3
=
X
1 =
XlI
1 =
Xl
ml an
d
X
2
=
ml
X 2 2
an
d
X
3
#
X 3 3
. . . 1 C "' n
By
this
ing one
procedure
the ,
of
found
the
we
most
likely
them
has
by
& j
third
set
of
at
least
the
most
=
one
or
. To
using
=
it
and
. . .
Xj
with
which
we
for
and
" " j
an
M2
,
found
Xj
as
= F Xj
Xj
=
=
to
be
found
M1
and
need
each
,
and
which
to
x7
in
can
be
the
jth
set
, ie
1
~ 11
and
Xj
either M2
perform
follows
-
only
i = x7j
. Now
in
other
the
configuration a
further
in n
-
j
+
1
:
2 . J
m ej
one
.
- l
the
m
=
, exclud
Hence
each was
is
2 j
.
of
- J. ml - ; - Xn n
n
configurations
configuration
M3
and
of
X
1
1 j
d
an
found
configuration
X ~ l
"
- l 1
set
likely
likely 1
out
=
mn -
- ) normalizations
" evidence
j
Xn
already
disagrees
find
1 =
remaining
configuration ,
place
propagations
Xl
n -
have
most
( max most
likely
propagations
we
second the
X
. . .
the
MI
second
propagating
1 d
an
partition
at
the
ml
one
looking
Suppose
by
X
=
Xj
2 .
2 .+ l
m
J and
Xj
+ 1 #
Xj
~ l
,
. . .
" n - j + 1 " j
This find
further the
partitions third
most
likely
configuration
most
likely
to
develop
=
the likely by
suitable
X . . .
allowed
essentially etc
states
notation
. . We
performing .
The to
m ; - l X n - 1 an
n - 1 =
configuration
configuration a
d
idea keep
After can a
is track
quite of
d
X
-. L m2 I X n n .
n
propagating then similar simple which
these
find
the
partition ,
the
partition
we
fourth on main one
can most
the
third
problem is
up
to
.
34
ROBERT COWELL
If we have prior evidence , then we simply take this into account at the beginning , and ensure that the partitions do not violate the evidence . Thus for example if we have evidence about m nodes being in definite states , then instead of n propagations being required to find M2 after having found M1 , we require instead only n - m further propagations . One application of finding a set of such most likely explanations is to explanation , ie, answering what the states of the unobserved variables are likely to be for a particular case. We have already seen that the most likely configuration offers such an explanation . If instead we have the top 10 or 20 configurations , then in most applications most of these will have most variables in the same state . This can confirm the diagnosis for most variables , but also shows up where diagnosis is not so certain (in those variables which differ between these top configurations ) . This means that if one is looking for a more accurate explanation one could pay attention to those variables which differ between the top configurations , hence serve to guide one to what could be the most informative test to do (cf. value of information ) . The use of partitioned "dummy evidence" is a neat and quite general idea , and will probably find other applications .!
4. A unification One simple comment to make is that minimization can be performed in a similar way to maximization . (In applications with logical dependencies the minimal configuration will have zero probability and there will be many such configurations . For example in the ASIA example half of the 256 configurations have zero probability .) Another less obvious observation is that sum , max and min -propagation are all special cases of a more general propagation based upon P norms , used in functional analysis . Recall that the LP norm of a non-negative real val ued function is defined to be
1 LP (f)=(lEX fP (X)dX )P
For p = 1 this gives the usual integral , for p - t 00 this give the maximum of the function over the region of integration , and for p - t - 00 we obtain the minimum of f . We can use this in our message propagation in our junction tree : the marginal message we pass from clique to separator is the P marginal , lSee for example (Cowell, 1997) for sampling without replacement from the junction tree.
ADVANCED
INFERENCE
IN BAYESIAN
35
NETWORKS
defined by : 1 p
bs(Xs) =
L a~(Xc) Xc \ Xs
So that we can obtain the LP marginal representation :
P (U I ' ) =
llCEC pbP ( X I t : c ) llSES PfP (X1s)
which is an infinite -family of representations . Apart from the L2 norm , which may have an application to quadratic scoring of models , it is not clear if this general result is of much practical applicability , though it may have theoretical
5.
uses .
F' ast retraction
Suppose that for a network of variables X we have evidence on a subset of
k variables U: , = { 'u : u E U*} , with ,U of the form "Xu = x~" Then it can be useful to compare
each item of evidence
with
the probabilistic
prediction given by the system for Xu on the basis of the remaining evidence
\ {u} : "Xv = x~ for v E U \ { u} " , as expressedin the conditional density of Xu given E\ { u} . If we find that
abnormally
low probabilities
are being
predicted by the model this can highlight deficiencies of the model which could need attention or may indicate a rare case is being observed . Now one "brute force" method to calculate such probabilities is to per form k separate propagations , in which one takes out in turn the evidence on each variable
in question
and propagates
the evidence
for all of the
remaining variables . However it turns out that yet another variation of the propagation algorithm allows one to calculate all of these predictive probabilities in one p.ropagation , at le~ t for the c~ e in which the joint probability is strictly
positive, which is the casewe shall restrict ourselvesto here. (For probabilities with zeros it may still be possible to apply the following algorithm ; the matter depends upon the network and junction tree . For the Shafer-Shenoy message passing scheme the problem does not arise because divisions are not necessary.) Because of the computational savings implied , the method is called fast -retraction . 5 .1.
QUT
- MARGINALISATION
The basic idea is to work with a potential representation of the prior joint probability even when there is evidence. This means that , unlike the earlier
36
ROBERT COWELL
sections , we do not modify the clique potentials by multiplying them by the evidence likelihoods . Instead we incorporate the evidence only into forming the messages, by a new marginalisation method called out-marginalisation , which will be illustrated for a simple two- clique example :
~
~
Here A , Band C are disjoint sets of variables , and the clique and separator potentials are all positive . Suppose we have evidence on variables a E A , ,B E Band ' Y E C . Let us denote the evidence functions by ha , hp and h'Y' where ha is the product of the evidence likelihoods for the variables a E A etc . Then we have
P(ABC) P(ABC, 0.) P(ABC, "1) P(ABC,[,0., "1)
-
-
-
1 g(AB )~9(B)~g(BC ) P(ABC)ha P(ABC)h'Y P(ABC)hah'Y.
where the g 's are the clique and separator We message
take from
the
clique C! ! : QJ to
~
as ~
root defined
g* (B) =
. Our
first
potentials step
is to
. send
an
out
- marginal
as :
LCg(BC )h"Y.
That is we only incorporate into the message that subset of evidence about the variables in C , thus excluding any evidence that may be relevant to the separator variables B . Note that because we are using the restriction that the joint probability is non zero for every configuration , this implies that the potentials and messages are also non zero. Sending this message leaves the overall product of j unction tree potentials invariant as usual :
P(ABC )=g(AB )g*)(B)@ --g*(B -_!..-)@ g(BC ). g(B
ADVANCED
N ow
let
us
use
INFERENCE
this
representation
pout
( AB
,
\ AUB
-
shows
by
that
symmetry
of
the
the
joint
g ( B ) is
joint
P
Further
out
desired
of
, with an
thus
)
and
the
clique
~
arrive
pout
at
( AB
,
the
have
)
these
follows on
Fast
retraction
predictive ing
re
with
the
following
evidence
alter
,
of between
, the
of
=
then
which
be
over
: EA
pout
two
back
the
( AB
of , and
out - margin
A . The
separator
, t : \ AUB ) ha
==
representation
to
because
.
( C
, E \ c
( S
,
,
.
the ,
use the
besides
,
.d
.
by
To
way
multiplied one do
to
this
,
by
,
the
one
would
deal ev
require
potential
- initialise
-
a
with represen
the
-
might
evidence a
re
of by
case
propagating
retains need
.
comparing
previous
are
case
no
the
describe
about
always is
yield evidence :
previously
another
However
tree there
then having
,
Consider
another
.
) .
)
potentials
at
\ BUG
.
evidence
tree
)
\ s
tree
has
clique
look
will
case
the
, [
representation
pout
in
( BC
variables
- marginal
evidence
junction
) pout
potentials
- clique
cliques
the
, [ ; \ B
pout
propagating
probability es
also
individual
lls
applicable
After
the
joint c ~
)
simple
evidence
of
- retraction
( U
where
in .
the
out - margin
out - marginal
( ~
out
against
likelihoods to
tation
the number ,
- initialisation
fast
tree
from
the
probabilities
idence wish
,
uction
will
clique
for the
pout
llc
ind
the
message
taken
following
\ AUB
of
P
which
is simply out - margin
out - marginalisation
probabilities we
~
send
marginalisation
- marginalisation
general
clique
now
:
=
predictive In
content
probability
( ABC
of the can
also
( B , & \ B ) . We
of the
content . We
probability
potential pout
the
probability
calculate
g(AB )g* (B ) g(B ) .
-
joint
37
NETWORKS
~ g*)(B)I:& >.g_*.(!B _-)I:& >g(BC )h'Y L.., g()B cg(AB g(AB g*)(B)0.g._ ..-) >c=g(BC )h'Y==g*(B) g()B *(!.B
-
the
to
BAYESIAN
Lc P(ABC , "'() L P(ABC )h"'(
)
-
This
IN
junction
-
38
ROBERTCOWELL
6 . Modelling
with
continuous
variables
All examples and discussion has have been restricted to the special case of discrete random variables . In principle , however, there is no reason why we should not build models having continuous random variables as well as, or instead of , discrete random variables , with more general conditional probability densities to represent the joint density , and use local message passing to simplify the calculations . In practice the barrier to such general applicability is the inability of performing the required integrations in closed form representable by a computer . (Such general models can be analyzed by simulation , for example Gibbs sampling .) However there is a case for which such message pa8sing is tractable , and that is when the random variables are such that the overall distribution is multivariate - Gaussian . This further extends to the situation where both discrete and continuous random variables coexist within a model having a so called conditional -gaussian joint distribution . We will first discuss Gaussian models , and then discuss the necessary adj ustments to the theory enabling analysis of mixed models with local computation . 7.
Gaussian
models
Structurally , the directed Gaussian model looks very much like the discrete models we have already seen. The novel aspect is in their numerical specifi cation . Essentially , the conditional distribution of a node given its parents is given by a Gaussian distribution with expectation linear in the values of the parent nodes, and variance independent of the parent nodes. Let us take a familiar example :
[ YJ--t [ KJ-t ~ . NodeY, whichhasno parents , hasa normaldistributiongivenby
Ny(J1 ,y;a})cx : exp ( -(y20 -'2 yJ.L~)2) , where ,uy and ay are constants . Node X has node Y as a parent , and has
the conditionaldensity:
Nx(J1 ,x+fJx ,yy;uk)cx :exp (-(x- Jl ,x2u2 -X{3x ,yy)2) '
ADVANCEDINFERENCEIN BAYESIANNETWORKS
39
whereJi,x , 'sx,y and ax are constants . Finally, node Z has only Xasa parent ; its conditional
density
is given by
Nz(Jlz+f3z ,xx;O "~)cx :exp ( -(z- Jl ,z2O -2(lZ'XX )2) . "z In general, if a node X had parents { Y1, . . . , Yn} it would have a conditional density:
Nx (J.Lx+
Li (3X ,~Yi;ak)cx :exp (-(x- J.Lx2a2 - Ei ,~Yi )2) ' x (3X
Now the joint density is obtained by multiplying together the separate component
Gaussian distributions
:
P(X, Y, Z) = Ny (Jl,y; a} )Nx (Jl,x + f3x,yY; alr)Nz(J1 ,z + f3z,xx ; a~)
exp (-~(x- /lx,Y- /lY,Z- /lz)K(x- /lx,Y- /lY,Z- /lZ)T) ,
cx:
where K is a symmetric (positive definite ) 3 x 3 matrix , and T denotes transpose . In a more general model with n nodes, one obtains a similar expression with an n x n symmetric (positive definite ) matrix . Expanding the exponential , the joint density can be written as:
exp h y)-2 K yxKxy K yyKXZ K yZ)(X Y hz Kzx Kzy Kzz z)) ((XYz)(hX 1(xyz)(KXX where the
hx
most
will
= useful
consist
for
of
properties
7 .1 .
J.Lx / a ~
we
Suppose
we in
exp
jjz
, Bz ,x / a ~
constructing
functions shall
GAUSSIAN
potential
+
of
be
using
local this
type
etc
. This
form
messages . Let
us
of
, and now
the
joint
indeed
define
density
local them
is
messages and
list
the
.
POTENTIALS
have
n
a subset
9 + (Yl
hI K 1 1 . . K 1 k Yl , , 1 . . . . Yk )hk :.--2(YIKk ,l..Kk ,kYk
continuous
random
{ YI , . . . , Yk }
of
variables
variables
Xl
is
, . . . , Xn
a function
. A
of
the
Gaussian form
:
Yk)
where K is a constant positive definite k x k matrix , h is a k dimen sional constant vector and 9 is a number . For shorthand we write this as a
40 triple respective
ROBERT COWELL ,
(g , h , K triples
) .
G
. aUSSlan potentials
together
can be multiplied by adding their
:
4>1 * 4>2 = (91 + 92, hI + h2 , K1 + K2 ) . Similarly
division
is
easily
handled
:
These operations will be used in passing the "update factor " from separator to clique . To initialize cliques we shall require the extension operation combined with multiplication . Thus a Gaussian potential defined on a set of variables Y is extended to a larger set of variables by enlarging the vector hand matrix K to the appropriate size and setting the new slots to zero. Thus for example : <jJ(x ) = exp (g + xTh - ! xTK x ) extends to
cf >(x,y) = cf >(x) = exp(g+ (x y) (~) - ~(x y) (~ ~) (:) ) . Finally , toformthemessages wemust define marginalisation , which isnow anintegration . LetustakeY1andY2tobetwosets ofdistinct variables , and 4>(Yl,Y2 ) =exp(9+(Yl Y2 ) (~~) - ~(Yl Y2 ) (~~:~ ~~:~) (~~) ) sothatthehandK areinblocks . Then integrating over Y1yields anew vector h andmatrix K asfollows : h = h2- K2,lK~th1 K = K22 , - K21K1 , -11K12o , ' (Discussion of the normalization will be omitted , because it is not required except for calculating probability densities of evidence .) Thus integration has a simple algebraic structure . 7.2. JUNCTION TREES FOR GAUSSIAN NETWORKS Having defined the directed Gaussian model , the construction of the junc tion tree proceeds exactly as for the discrete case, as far as the structure is concerned . The difference is with the initialization . A Gaussian potential of correct size is allocated to each clique and separator . They are initialized with all elements equal to zero.
ADVANCED INFERENCE IN BAYESIAN NETWORKS
41
Next for each conditional density for the DAG model , a Gaussian poten tial is constructed to represent it and multiplied into anyone clique which contains the node and its parents , using extension if required . The result is a junction tree representation of the joint density . Assum ing no evidence , then sending the clique marginals as messages results in the clique marginal representation , as for the discrete case:
P(U) = II p(Xc )/ll
sP(Xs).
c
Care must be taken to propagate evidence. By evidence [; on a set of nodes Y we mean that each node in Y is observed to take a definite value . (This is unlike the discrete case in which some states of a variable could be excluded but more than one could still be entertained .) Evidence about a variable must be entered into every clique and separator in which it occurs . This is because when evidence is entered on a variable it reduces the dimensions of every h vector and K matrix in the cliques and separators in which it occurs . Thus for example , let us again take Y1 and Y2 to be two sets of distinct variables , and
K2 ,2)(Yl Y2 Yk )(~~)-~(YlY)(Kl kK2 ,l1Kl ))
>(Yl, Y2) <Xexp ( (Yl so
that
the
variables
h
=
hI
-
After
the
Y
to
Y2K2
, 1
such
Gaussian
are
take
and
K
evidence
evidence
again
in
values
=
has
propagation
with
(
K
2
standard
tation
the
hand
of
Y2
KI
, I
distributions
blocks
Then
been
entered
yield
.
on
.
the
Suppose
we
potentials
now
observe
become
the
modified
to
.
will
included
)
'
in
the
Further
every
clique
within
individual
clique
-
and
marginal
clique
nodes
separator
,
density
marginals
then
represen
then
-
gives
.
7.3. EXAMPLE Let us take out three node example again , with initial tions as follows :
[Y]- +[RJ - +~ N(Y) N(X I Y) N(Z IX)
-
N(O,l ) N(y,l ) N(x,l )
conditional
distribiU-
42
ROBERTCOWELL
The cliquesfor this treeare[ K: rJ and[ K! ] . After initializingand propagating , the clique potentials are
exp (-~(x y)(!1~1 )(:)) cx exp (-~(x z)(~~~1 )(:))
<jJ(x , y) cx <jJ(x , z)
with separator >(x ) cxexp(- x2/ 4); Now if we enter evidenceX = 1.5, say, then the potentials red lice to :
4>(X = 1.5, y) cx : exp(1.5y - y2) and 1
2
4>(X = 1.5, z) cx : exp(1.5z - 2z ), because in this example X makes up the separator
between the two cliques .
The marginal densities are then :
P (Y ) = N (0.75, 0.5) and P (Z ) = N (1.5, 1). Alternatively idence
that
cj>( X on
,
)
cx
Z
exp
[ ! : : XJ
is
=
As as
we
and
have
(X
(
)
seen
( x
=
,
message
so
y )
N
the
that
(
the
from
after
105 )
( l , 2 / 3 )
root
clique
~
to
propagation
,
and
[ K : rJ the
enter is
ev
given
clique
-
by
potential
-
~
( x
y )
(
!
1
~
1 )
( :
)
)
. ,
and
P
(Y
)
=
N
( 1 / 2 , 2 / 3 ) .
models
treatment
models employed
The
and
of minor
(2 )
Gaussian
networks
differences
evidence
are
has
to
be
is
in
( 1 )
entered
much
the
the
nature
into
every
same of
the
clique
.
passage . The
)
as
:
Gaussian
separator The
ences
[ K : rJ
densities
discrete
potentials
take the
O . 75x2 form
Conditional
for
Then
< X exp
P
8 .
-
the
marginal
we
1 .5 .
( I . 5x of
cf>( x , y )
with
suppose
first
to
mixed
is
a
restriction
models
proceeds in
the
with modeling
some stage
more :
important
continuous
differ variables
-
ADVANCEDINFERENCE IN BAYESIAN NETWORKS are
not
allowed
discrete
to
parents
differ
in
ple
,
the
of
to
as
for
discrete
the
on
following
original
8
is
paper
. 1
.
CG
The
by
set
of
element
of
X
( r
joint
y
( x
)
E
( i
)
,
O
f
,
( i
I
}
,
tentials
0
,
for
sim
to
,
the
sub
to
theory
,
we
but
( i
)
( i
and
)
=
the
)
)
( i
)
parent
-
but
con
-
configurations
include
more
indicator
careful
for
more
follow
( i
)
=
{
with
details
closely
)
( i
the
)
g
( i
{
any
4 > ( i
-
y
the
see
here
the
.
,
E
)
( i
)
-
we
+
yTh
is
}
)
exp
of
{
g
-
at
lrllog
( i
The
.
define
)
discrete
joint
( i
density
The
/
2
}
f
the
is
with
,
triple
it
is
form
) y
and
typical
the
the
K
i
a
of
;
given
( i
( t : J. )
denote
has
yT
( g
only
,
h
,
K
)
defined
moment
charac
-
by
=
K
( i
)
are
( 27r
)
-
l
K
-
~
CG
the
)
distribution
can
~
y
values
it
positive
and
,
,
.
( i
the
generalize
4 >
X
)
( i
that
one
( i
( i
,
=
~
variables
=
the
characteristics
logdetK
)
,
lh
functions
,
)
case
canonical
+
x
variables
f
{ p
K
discrete
Let
means
of
is
the
.
denoting
which
exp
networks
=
i
r
continuous
triple
the
( i
are
u
whether
the
Gaussian
4 > ( x
X
that
have
which
the
,
=
~
with
of
y
by
logp
=
space
when
we
,
into
V
characteristics
denoted
Inverting
As
again
potentials
by
need
have
notation
thus
indicates
~
g
Gaussian
we
we
the
whose
distribution
canonical
>
teristics
CG
=
{
the
) ~
to
partitioned
,
values
a
)
is
)
state
the
be
f
( i
called
( i
,
V
to
where
K
are
certain
,
and
former
indexed
because
allowed
guide
Lauritzen
variables
and
assumed
X
,
have
nodes
.
brief
variables
the
variables
for
are
only
discrete
- POTENTIALS
continuous
is
a
can
for
The
K
Also
be
nodes
.
latter
potentials
g
discrete
matrices
.
not
Gaussian
the
and
parents
constants
The
,
h
might
the
normalization
.
nodes
models
discrete
ie
specified
continuous
vectors
variables
functions
of
discrete
,
,
probabilities
those
g
of
children
conditional
constants
figurations
discrete
The
character
tables
with
have
.
43
( i
.
( i
) T
)
=
K
( i
E
( i
)
-
) ~
( i
)
distributions
l
}
to
,
/
h
2
CG
( i
)
=
.
po
-
form
)
+
yTh
( i
)
-
yT
K
( i
) y
/
2
}
.
K (i ) is restricted to be symmetric , though not necessarily invertible . However we still call the triple (g, h , K ) the canonical characteristics , and if for all i , x (i ) > 0 and K (i ) is positive definite then the moment characteristics are given M before . Multiplication , division and extension proceed as for the Gaussian potentials have already been discussed. Marginalisation is however different ,
44
ROBERT COWELL
because adding two CG potentials in general will result in a mixture of CG potentials - a function of a different algebraic structure . Thus we need to distinguish two types of marginalisation - strong and weak.
8.2. MARGINALISATION
Marginalising continuous variables corresponds tointegration . Let Y2) , h = ( hI h2) , K = ( KI1 K21K22 y = ( YI K12) withYlhaving dimensionp andY2dimension qwithK11ispositive definite . Thentheintegral f ~(i,Yl,Y2 )dYlisfiniteandequal toa CGpotential ~ withcanonical characteristics given as g(i) = g(i) + {plog(27r ) - logdetK11 (i) + h1(i)TKll(i)-lh1(i)} /2 h(i) = h2(i) - K21 (i)K11 (i)-lh1(i) K(i) = K22 (i) - K21 (i)K11 (i)-lK12 (i). Marginalising discrete variables corresponds to summation . Since in general addition of CG potentials results is a mixture of CG potentials , an alternative definition based upon the moment characteristics {p , ~, E } is used which does result in a CG potential ; however it is only well defined for K (i-, j ) positive definite . Specifically , the marginal over the discrete states - of ~ is defined as the CG potential with moment characteristics {j), ~, E } where and
p(i) = LP (i,j ), ~(i) = L ~(i,j )p(i,j )jjj(i),
J
J
t (i) = 2:::: . ~(i,j )p(i,j )/p(i) +L. (~(i,j ) - ~(i))T(~(i,j ) - ~(i))p(i,j )/p(i). J
Note
that
J
the
latter
can
be
written
as
p(i) (~(i) +~(i)T~(i)) =~ p(i,j) (~(i,j)+~(i,j)T~(i,j)) . J
so that if E (i , j ) and ~(i , j ) are independent of j then they can be taken through the summations as constants . This observation is used to define a marginalisation over both continuous and discrete variables : First marginalise over the the continuous variables and then over the discrete variables . If , after marginalising over the continuous variables the result ing pair (h, K ) is independent of the discrete variables to be marginalised over , (summation over these discrete variables then leaves the pair (h , K )
ADVANCED INFERENCE IN BAYESIAN NETWORKS
45
unmodified ) , we say that we have a strong marginalisation . Otherwise one sums over the discrete variables using the moment characteristics , and the overall marginalisation is called a weak marginalisation . Weak and strong marginalisation satisfy composition :
LA LB
AuBuC = AUB L AUBUC , but in general only the strong marginalisation satisfies
L (ifJAUB 'l/JB ) = 'l/JB L ifJAUB . A
Under
both
the
type
correct
8
( I
the
. 3
.
=
)
p
( i
of
we
in
moralize
cg
the
,
sulting
cliques
to
it
.
Unlike
the
then
Any
can
the
R
tree
to
,
when
discrete
,
extra
20ne
way
nodes and
tion orderings
to the
cases
joining . .
Finding
any
of
V
( Y
take
density
will
I
I
=
i
)
=
then
~
expectations
have
( i
)
,
.
\
A
we
A
~
r
or
B
of
B
satisfies
( B
two
we
n
A
)
Ll
the
from
select
has
restricted
the
a
clique
defined
re
-
root
of
as
.
the
follows
:
neighbours
cliques
root
First
.
neighbouring
from
Then
themselves
~
and
.
continuous
any
root
cliques
a
the
must
choose
strong
,
of
.
Now
freely
called
than
by
all
.
tree
tree
triangulate
variables
tree
variables
junction
eliminate
cannot
discrete
junction
the
Then
first
so
R
)
of
the
discrete
pair
to
away
only
is
not
purely
continuous
vertices
.
a
good
.
we
we
between
' unmarked pairs
,
construct
junction
a
closer
triangulate
all
ordering
,
choose
separator
status
)
construct
to
the
a
furthest
the
( i
to
we
way
eliminate
separator
clique
beyond
the it
a
the
~
used
is
,
( B
Thus
=
'
marginalisation
usual
for
A
)
i
step
the
which
' marginalised
how
Specifically
we
with
=
construct
Instead
clique
on
I
a
.
under
pure
.
. e
is
first
we
we
tree
I
adjust
in
. 2
previous
junction
( Y
i
TREE
The
DAG
and
E
,
- potentials
ordering
variables
,
2
distribution
have
messages
,
order
JUNCTION
- closure
elimination
)
CG
THE
that
we
=
correct
non
means
pass
to
i
MAKING
The
marginalisation
moments
P
where
of
A
graph ' .
its
unmarked triangulations
is One
to
take
then
an works
neighbours is
ordering through .
then
The
equivalent
of
the
each
nodes node
ordering
is to
finding
and in called good
give
turn
, an
all
of
marking elimina
elimination
-
46
ROBERT COWELL
8.4. PROPAGATION ONTHEJUNCTION TREE The point of this restriction is that on the collect operation , only strong marginalisations are required to be performed . This is because our restricted elimination ordering - getting rid of the continuous variables first , is equiva lent to doing the integrations over the continuous variables before marginal ising any of the discrete variables . Thus our message passing algorithm takes the form : 1. Initialization : Set all clique and separator potentials to zero with unit indicators , and multiply in the model specifying potentials using the extension
operation
where appropriate
.
2. Enter evidence into all clique and separator potentials , reducing vector and
matrix
sizes
as necessary
.
3. Perform a collect operation to the strong root , where the messages are formed by strong marginalisation by first integrating out the redundant continuous
variables , and then summing
over discrete variables .
4. Perform a distribute operation , using weak marginalisation where appropriate discrete
when mixtures variables
might
be formed
on marginalising
over the
.
The result is a representation of the joint CG-distribution including evidence , because of the invariant nature of the message passing algorithm . Furthermore , because of the use of weak marginalisation for the distribute operation , the marginals on the cliques will themselves be CG -distributions whose first two moments match that of the full distribution . The following is an outline sketch of why this could be. First by the construction of the junction tree , all collect operations are strong marginals , so that after a collect -to-root operation the root clique contains a strong marginal . Now suppose, for simplicity , that before the
distribute operation we move to a set-chain representation (cf. section 2.2). Then
apart
from
the strong
root , each clique will
have the correct
joint
density P (XCi\ Si l Xsi ) where Si is the separator adjacent to the clique Ci on the path between it and the strong root . Now on the distribute oper ation the clique Ci will be multiplied by a CG-potential which will either be a strong marginal or a weak marginal . If the former then the clique potential will be the correct marginal joint density . If the latter then we
may write the clique potential as the product P (XCi\ Si) * Q(Xsi ) where Q is the correct weak marginal for the variables X Si. Now consider taking an expectation of any linear or quadratic function of the XCi with respect to this "density " . We are free to integrate by parts . However , choosing to inte -
grate wrt . XCi\ Si first meansthat we form the expectation wrt the correct CG-density P (XCi\ Si I Xsi ), and will thus end up with a correct expectation (which will be a linear or quadratic function in the XSi ) multiplied
ADVANCED INFERENCE IN BAYESIAN NETWORKS
47
by the correct weak marginal Q (X Si) . Hence performing these integrations we will obtain the correct expectation of the original function wrt the true joint density . For brevity some details have been skipped over here, such as showing that the separator messages sent are correct weak marginals . Detailed justifications and proofs use induction combined with a careful analysis of the messages sent from the strong root on the distribute operation . See the original paper for more details . 9 . Summary This tutorial has shown the variety of useful applications to which the junction -tree propagation algorithm can be used. It has not given the most general or efficient versions of the algorithms , but has attempted to present the main points of each so that the more detailed descriptions in the orig inal articles will be easier to follow . There are other problems , to which the junction -tree propagation algorithm can be applied or adapted to , not discussed here, such as: - Influence diagrams : Discrete models , with random variables , decisions and utilities . Potentials are now doublets representing probabilities and utilities . Junction tree is generated with a restricted elimination generalising that for cg-problems to emulate solving the decision tree . - Learning probabilities . Nodes presenting parametrisations of probabil ities can be attached to networks , and Bayesian updating performed using the same framework . - Time series. A network can represent some state at a given time , and they can be chained together to form a time -window for dynamic mod elling . The junction tree can be expanded and contracted to allow forward - prediction or backward smoothing . Doubtless new examples will appear in the future .
10. Suggested further
reading
Probabilistic logic sampling for Bayesian networks is described by Henrion (1988) . A variation of the method - likelihood -weighting sampling in which rejection steps are replaced by a weighting scheme is given by Shachter and Peot (1989) . Drawing samples directly from the junction tree is described by Dawid (1992), which also shows how the most likely config uration can be found from the junction tree . The algorithm for finding the N - most likely configurations is due to Nilsson (1994) , who has also developed a more efficient algorithm requiring only one max -propagation on the junction tree . V -propagation is not described anywhere but here.
48
ROBERT COWELL
Fast retraction is introduced in (Dawid , 1992) and developed in more detail in (Cowell and Dawid , 1992) .
Gaussian networks are described by Shachter and Kenley(1989), who use arc-reversal and barren -node reduction algorithms for their evaluation .
(The equivalenceof various evaluation schemesis given in (Shachter et al., 1994) .) The treatment
of Gaussian and conditional -gaussian networks is
based on the original paper by Lauritzen( 1992). For pedagogical reasons this chapter specialized the conditional -gaussian presentation of (Lauritzen , 1992) to the pure gaussian case, to show that the latter is not so different from the pure discrete case. Evaluating influence diagrams by junction -trees is treated in (Jensen et al., 1994) . For an extensive review on updating probabilities (Buntine , 1994) . Dynamic junction trees for handling time
series is described by Kjrerulff (1993). See also (Smith et al., 1995) for an application using dynamic junction trees not derived from a DAG model . References Buntine , W . L . ( 1994) . Operations for learning with graphical models . Journal ficial Intelligence Research , 2 , pp . 159- 225.
of Arti -
Cowell, R . G. (1997) . Sampling without replacement in junction trees, Research Report 15, Department of Actuarial Science and Statistics , City University , London . Cowell , R . G . and Dawid , A . P. ( 1992) . Fast retraction of evidence in a probabilistic expert system . Statistics and Computing , 2 , pp . 37- 40 .
Dawid , A . P. (1992) . Applications of a general propagation algorithm for probabilistic expert systems . Statistics
and Computing , 2 , pp . 25- 36.
Henrion , M . (1988) . Propagation of uncertainty by probabilistic logic sampling in Bayes' networks. In Uncertainty in Artificial Intelligence, (ed. J. Lemmer and L . N . Kanal ), pp . 149 - 64 . North
- Holland
, Amsterdam
.
Jensen, F ., Jensen, F . V ., and Dittmer , S. L . (March 1994). From influence diagrams to junction trees . Technical Report R -94-2013 , Department Computer Science Aalborg University , Denmark .
of Mathematics
and
Kj cerulff, U . (1993) . A compu tational scheme for reasoning in dynamic pro babilistic networks . Research Report R -93-2018 , Department Science , Aalborg University , Denmark .
of Mathematics
and Computer
Lauritzen , S. L . (1992) . Propagation of probabilities , means and variances in mixed graphical
association models . Journal
of the American
Statistical
Association , 87 ,
pp . 1098 - 108 .
Nilsson , D . ( 1994) . An algorithm for finding the most probable configurations of discrete variables that are specified in probabilistic expert systems . M .Sc. Thesis , Department of Mathematical Statistics ) University of Copenhagen .
Nilsson, D . (1997) . An efficient algorithm for finding the M most probable configurations in a probabilistic
expert system . Submitted
to Statistics and Computing .
Shachter, R. D ., Andersen, S. K ., and Szolovits, P. (1994). Global conditioning for probabilistic Uncertainty
inference in belief networks . In Proceedings of the Tenth Conference on in Arlifical Intelligence , pp 514- 522.
Shachter, R . and Kenley, C. (1989) . Gaussian influence diagrams. Management Science, 35 , pp . 527 - 50 .
Shachter, R . and Peot, M . (1989) . Simulation approaches to general probabilistic inference on belief networks . In Uncertainty in Artificial Intelligence 5, (ed . M . Hen rion , R . D . Shachter , L . Kanal , and J . Lemmer ) , pp . 221- 31. North -Holland , North -
ADVANCED
Holland
INFERENCE
IN
BAYESIAN
NETWORKS
49
.
Smith , J. Q., French, S., and Raynard , D . (1995). An efficient graphical algorithm for updating
the estimates of the dispersal of gaseous waste after an accidental release . In
Probabilistic reasoning and Bayesian belief networks, (ed. A . Gammerman) , pp. 12544 . Alfred Waller , Henley -on - Thames .
INFERENCE IN BAYESIAN NETWORKS USING NESTED JUNCTION TREES
UFFE
KJ
lERULFF
Department
of
Fredrik
Abstract
.
Shenoy
architectures
lations to
clique
sent
in
and
reduced
.
junction
is large
Introduction
Kim
and
was
are
passed
1990
) .
More
variable
a
tree
be
leaves
but
distributions
( or ,
,
a
Shafer
this
That
is
,
the
we
show of
may
the
the
nested
usefulness
of
evaluation
both
that
methods
The
Hugin
-
the
messages
exploiting .
-
mes of
paper
of
re
factorization
computation
empirical
the
root
;
nodes ( at
least
tree
)
Jensen
the
involving and
the
at
Shafer
of
up
a
51
,
-
. ,
1990
the
) normalizing
the to
;
for
then
a
Shenoy
,
particular
the
leaf
of
interest
.
toward
the
root
contain constant
Later
network
and
from
the
.
messages
the
Shafer
variable from
will
network only
where
inward
messages
cliques to
Bayesian
distribution
messages containing
( or
a
networks
corresponding et
probability
node
in
connected networks
sending
propagation all
inference singly
join
1988
posterior by
,
In
reductions
connected
tree
computed
performed
.
the
propagation
and
for
multiply
outward is
. a
way
formulated ,
junction
toward
subsequent
posterior
first
Spiegelhalter precisely
can the
)
to
and
clique
the
thorough
networks
passing
in
( Lauritzen
tree
and independence
.
( 1983
extended
ugin
via
in
such a
Bayesian
a
structured
achieve
H
Denmark
the
conventional a
algorithms
message
this
of
- world
the
computed
junction
the
University (!),
exploiting
trees
through
Pearl
through
of
to
both
of
be
a
Aalborg
Aalborg
messages
presents
technique
inference
1 .
by
of
costs
paper
,
- 9220
in
junction
emphaBized real
Shenoy
DK
inference
can
form
time
trees
method ten
the
,
improved
clique
nested
The
7E
incoming a
such
space
be
be
the
from
exploiting
both
of can
by
be
Science
Vel
efficiency
potential
by
a
The
induced
sage
Computer
Bajers
the ) .
nodes
correct
If
52
UFFEKJlERULFF
The Hugin and the Shafer-Shenoy propagation methods will be reviewed briefly in the following ; for more in -depth presentations , consult above references. We shall assume that all variables of a Bayesian network are discrete . 2. Bayesian
Networks
and Junction
Trees
A Bayesian network can be defined as a pair (Q,p), where Q = (V, E ) is an acyclic , directed graph or , more generally , a chain graph (Frydenberg , 1989) with vertex set V and edge set E , and p is a discrete probability function with index set V . To each vertex v E V corresponds a (discrete ) random variable
Xv with domain
Xv . Similarly , a subset A ~ V corresponds
to a set
of variables XA with domain XA = XvEAXv. Elements of XA are denoted XA = (xv ) , v E A .
A probability function "' v(xv, Xpa(v)) is specified for each variable Xv , where pa(v) denotesthe parents of v (i.e., the set of vertices of 9 from which there are directed links to v). If pa(v) is non-empty, "'v is a conditional probability distribution for Xv given Xpa(v); otherwise, "'v is a marginal probability distribution for Xv . The joint probability function pv = P for X V is Markov with respect to the acyclic , directed graph g . That is, Q is a map of the independence relations represented by p : each pair of vertices not connected by a directed edge represents an independence relation . Thus , 9 is often referred to as the independence graph ofp . The probability function p being Markov with respect to 9 implies that p factorizes according to g :
p = n ~V' vEV
For a more detailed account of Markov fields over directed graphs , consult
e.g. the work of Lauritzen et al. (1990. Exact inference in Bayesian networks involves the computation of marginal probabilities for subsets C ~ V , where each C induces a fully connected (i .e., complete ) subgraph of an undirected graph derived from Q. (Formally , a set A ~ V is said to induce a subgraph QA = (A , EnAxA ) ofQ = (V, E ) .) A fixed set C of such complete subsets can be used no matter which variables have been observed
and no matter
for which variables
we want posterior
marginals . This is the idea behind the junction tree approach to inference . The derived undirected graph is created through the processes of moral ization and triangulation . In moralization an undirected edge is added between each pair of disconnected vertices with common children and , when this has been completed , all directed edges are replaced by undirected ones. In the triangulation process we keep adding undirected edges (fill -ins ) to the moral graph until there are no cycles of length greater than three with -
NESTEDJUNCTIONTREES
53
out a chord (i .e., an edge connecting two non-consecutive vertices of the cycle ). The maximal complete subsets of the triangulated graph are called cliques. It can easily be proved that an undirected graph is triangulated if and only if the cliques can be arranged in a tree structure such that the intersection between any pair of cliques is contained in each of the cliques on the path between the two . A tree of cliques with this property is referred to as a junction tree. Now , inference can be performed by passing messages between neighbouring cliques in the junction tree .
3. Inference in Junction
Thees
Inthefollowing wewillbriefly review thejunction tree approach toinfer ence .For amore in-depth treatment ofthesubject consult e.g.thebook of Jensen (1996 ). IntheHugin architecture , apotential table >c isassociated with each clique , C.The linkbetween any pair ofneighbouring cliques ,say Cand D, shall bedenoted aseparator , and itislabelled byS=CnD.FUrther ,with each separator , S, isassociated one ortwo potential tables ('mailboxes '). The Hugin algorithm uses one mailbox perseparator , referred to~ 8 and \f~'out S. Inaddition ,weassign each function I'\.v,vEV,toaclique , C,such that {v} Upa(V) EC. That is,foreach clique , C, isassociated asubset ofthe conditional probabilities specified fortheBayesian network .LetKcdenote thissubset , and define ~c=KE n.K :c1'\.. As mentioned above, inference in a j unction tree is based on message passing . The scheduling of message passes is controlled by the following rule : a clique , C , is allowed to send a message to a neighbour , D , if C has received messages from all of its neighbours , except possibly from D , and C has not previously sent a message to D . Thus propagation of messages is initiated in leaf cliques and proceeds inwards until a 'root ' clique , R , has received messages from all of its neighbours . (Note that any clique may be root .) The root clique is now fully informed in the sense that it has got information (via its neighbours ) from all cliques of the junction tree , and thus PR cx cPR,with the proportionality constant being the probability of the evidence (i .e., observations ) , if any. An outward propagation of messages from R will result in Pc cx cPc for all C E C, where C denotes the set of cliques .
54
UFFEKJ.lERULFF
A messageis passedfrom clique C to clique D via separator S = C n D , as follows: 1. C generatesthe messageV = >; , where >V is the clique potential for D after the absorption. Now, assumethat we are performing the inward pass, and that clique C is absorbing messages4JS1 ' . . . , 4Jsnfrom neighbouring cliques C1, . . . , Cn and sending a message
1. Absorb messages
L\S< Pc C i=l
2. Generate message
\f' Si
f -
,.,h*
. -
\f' Si ' ~ -
1
, . . . , n
4. If algorithm = Hugin, then store clique potential : c/Jc f - c/Jc
5. DiscardcPO , Sl' . . . , cPsn 6. SendcfJSo to Co (Note that if C is the root clique , we skip steps 2 and 6.) Thus , considering the inward p~ s, the only difference between the H ugin and the ShaferShenoy algorithms is that the Hugin algorithm stores the clique potential . Storing the clique potential increases the space cost, but (most often ) reduces the time cost of the outward pass, as we shall see shortly . Next , assume that we are performing the outward pass, and that clique C receives a message, 4>80' from its 'parent clique ' Co (i .e., the neighbouring clique on the path between C and the root clique ) , and sends messages 4>Sl ' . . . , 4>sn to its remaining neighbours , 01 , . . . , On. (The * 's should not be confused by the similar potentials of the inward pass.) This is done as follows .
1. Absorb messagecPSo : if algorithm = Hugin then
~* -- 'fiG ~ 4>80 'fiG
NESTEDJUNCTIONTREES
55
.C L\Si ~ 1,..,i-nl,i+1,..,n< ~Si =C L\Si
else
2. Store
if algorithm = Hugin then
4>Sot - 4>80 else ~ out ' f ' So
3
.
If
4
.
Discard
5
.
Send
+
=
< Pc
c / > Sl
'
4
index
)
,
{
sets
=
and
~
time
c
Ui
Ui
the
}
U1
)
'
Thus
,
,
potential
:
4 > c
+
-
4 > c
,
if
very
the
expensive
generation
be
of
obtained
approach
due
.
to
the
these
This
is
generation
can
what
be
is
relaxed
,
exploited
in
.
Trees
U
.
be
can
trees
( g
( fam
v
clique
may
savings
network
=
store
trees
subgraph
( v
then
ciJSn
junction
Bayesian
fam
( U
,
Junction
complete
1 -
.
junction
Nested
a
.
and
nested
In
.
,
< PSo
potentials
space
.
Hugin
and
in
clique
both
the
~ * ' f ' So
algorithm
Inference
of
-
.
where
pa
.
,
( v
Urn
( v
)
.
~
each
=
)
( V
,
In
,
fam
( v
general
V
Ui
)
)
an
induces
, p
x
,
induce
~
E
)
each
fam
a
potential
( v
set
)
)
of
undirected
a
complete
of
~
the
v
,
moral
,
subgraph
E
V
,
graph
potentials
graph
v
,
~
1i
Ul
of
'
=
induces
.
.
.
( U
( Ui
'
,
,
Q
~
Ui
Ui
a
,
um
where
,
with
Ui
x
x
Ui
Vi
)
)
of
potential
~u = II ~Ui 1;
is Markov with respect to 1 (i.e., 1 is an independencegraph of ~u). Thus, if triangulating 1 does not result in a complete graph, we can build a junction tree corresponding to the triangulated graph, and then exploit the messagepassing principle for computing marginals of ~u . Assume that ~Ul' . . . , ~um are the potentials involved in computing a message(i .e., a marginal), say >80' in a clique C. That is, ~Ul' . . . , ~Umare the incoming messagesand the I'\,v'S associatedwith C. Then, instead of computing the
56
UFFEKJlERULFF
clique potential ~* \fiG -
n ~Ui 'l;
(cf. Step 1 of the above inward algorithm ), inducing a complete graph, and
marginalizingfrom so (cf. Step2 of the aboveinwardalgorithm ), we might be able to exploit conditional independencerelationships between variables of So given the remaining variables of C in computing
cPSo through messagepassingin a junction tree residinginside C (which, remember , is a clique of another , larger junction tree) . This , in essence, is the idea pursued in the present paper . In principle , this nesting of junc tion trees can extend to arbitrary depth . In practice , however, we seldomly encounter instances with depth greater than three to four . As a little sidetrack , note the following property of triangulations . Let 0 be a clique of a junction tree corresponding to a triangulation of a moral
graph 9 , let KG = { ~Vl' . . . '
the graph induced by
graph , unless the triangu -
lation of (] contains a superfluous fill -in : a fill -in between vertices u and v is
superfluous if {u, v} ~ Si for each i = 1, . . . ,n , becausethen C can be split into two smaller neighbouring cliques 0 ' = C \ {u} and C" = C \ { v} with Ci , i == 1, . . . , n , connected to either
to C ' if v E Ci , to C " if u E Ci , or , otherwise ,
0 ' or C " .
Therefore , assuming the triangulation to be minimal in the sense of not containing any superfluous fill -ins, the nested junction tree principle cannot be applied
when a clique , C , receives messages from all of its neighbours ,
or when a clique potential cPais involved in the computation (cf. the out ward pass of the Hugin algorithm ) . However , in the inward pass and in the outward pass of the Shafer-Shenoy algorithm , any non -root clique receives messages from all but one, say Co, of its neighbours , making it possible to exploit the junction tree algorithm for generating the message to Co. To illustrate the process of constructing nested junction trees, we shall consider the situation where clique 016 is going to send a message to clique 013 in the junction tree of a subnet , here called Munin1 , of the Munin net -
work (Andreassenet ai., 1989). Clique 016 and its neighbours are shown in Figure 1. The variables of 016 = { 22, 26, 83, 84, 94, 95, 97, 164, 168} (named corresponding to their node identifiers in the network ) have 4, 5, 5, 5, 5, 5, 5, 7, and 6 possible states , respectively .
The undirectedgraph inducedby the potentialscPs1 ' cPs2 , cPs3 , and Vl may be depicted as in Figure 2. At first sight this graph looks quite messy, and it might be hard to believe that its triangulated graph will be anything but complete . However , a closer examination reveals that the graph
NESTED JUNCTION TREES G19
G13
I/>s~ c/J~S Figure
1
.
Clique
respectively
a
message
is
22
must
be
26
,
83
,
Figure
2
SO
,
table
and
-
Thus
4 >
82
so
. e
.
,
95
,
,
625
,
clique
cPs1
)
-
to
'
clique
its
and
< ! > S3
from
probability
cliques
G19
potential
013
,
,
cPVl
026
=
,
P
{
and
G63
97122
,
,
26
)
,
.
cliques
are
{
83
,
84
,
97
,
164
,
168
}
and
induced
i
.
e
.
been
,
by
clique
potentials
containing
reduced
tables
of
cps1
a
size
< / JS2
nine
to
total
,
,
< / JS3
tree
000
(
'
and
< / JV1
variables
junction
381
'
)
with
a
including
a
'
with
5
-
a
clique
separator
.
be
shall
,
the
further
cPV1
try
5
to
-
)
continue
clique
broken
three
and
4 > S2
{ 22, 26, 83, 84, 94, 95, 168} { 83, 84, 97, If >4, 168} { 94, 95, 97} { 22, 26, 97}
.
(
has
we
,
}
'
the
that
168
clique
000
remaining
cPs3
,
tree
cannot
,
sent
graph
with
750
the
97
undirected
junction
it
got
i
,
encouraged
clique
'
94
9
2
size
and
and
original
8
of
has
generated
The
size
an
table
,
.
< ! > Sl
messages
and
84
the
of
-
messages
these
triangulated
,
two
on
81 = 82 = 83 = VI =
I'\-S3
receives
Based
already
{
(
016
.
57
down
potentials
induce
our
has
clique
associated
.
The
8
-
clique
associated
the
graph
break
in
it
,
with
shown
-
with
on
it
Figure
the
.
down
only
other
These
3
.
the
hand
potentials
.
In
potential
,
NESTEDJUNCTIONTREES
59
(Cb n So) \ Ca = { 22, 26, 94, 95} ; that is, 4 x 5 x 5 x 5 = 500 messages
must be sentvia separator{83, 84,97, 168} in order to generate>80. Sending a message from Cb to Ca involves inward propagation of messages in
the (Cc, Cd) junction tree, but , again, since neither Cc nor Cd contain all variables of the (Ca, Cb) separator, we need to send multiple messagefrom Cd to Cc (or vice versa). For example, letting Cc be the root clique of the inward pMS at this level, we need to send 5 messagesfrom Cd (i .e., one for each instantiation of the variables in (Cd n { 83, 84, 97, 168} ) \ Cc = { 97} ) for each of the 500 messages to be sent from Cb (actually from Cc) to Ca. Similarly , for each message sent from Cd to Cc, 20 messages must be sent
from Ce to Cf (or 25 messagesfrom Cf to Ce). Clearly, it becomesa very
time consumingjob to generate4>80' exploitingnestingto this extent.
525
872, 750,000
000
22 , 26 , 83 , 84 , 94 , 95 , 164 , 168
2t625
5t250
( 7x )
, 000
75 , 000
( 150x
. .
500
83
22
97
84
94
168
95
26
4
!100 ( 20x )
..
( 500x .
It 210t 000
)
~
750
)
17 , 000
( 5x )
/' ..I '
~
,."
, ,
Figure 5. The nested junction tree for clique C16 in Muninl . Only the connection to neighbour C13 is shown. The small figures on top of the cliques and separators indicate table sizes, assuming no nesting. The labels attached to the arrows indicate (1) the time cost of sending a single message, and (2) the number of messagesrequired to compute the separator marginal one nesting level up.
A proper balance between space and time costs will most often be of interest . We shall address that issue in the next section . Finally , however, let us briefly analyze the case where Cb is chosen as root instead of Ca (see Figure 6) . First , note that , since Cb contains
60
UFFEKJlERULFF
the
three
potentials
4>1 , . . . , 4>1 , from subgraph
.
Ca
The
via
time
S
and
=
cost
of
and
receives
message
{ 83 , 84 , 97 , 168 } , Cb
collapses
computing
clique
the
first
potentials to
a
,
complete
potential
, 4>Cb
=
525 , 000 7, 429 , 500
22 , 26 , 83 , 84 , 94 , 95 , 164 , 168
2, 625 , 000
5, 250 ( 7x ) .. 750 83 84 97 168
Figure
6.
The
( i .e . , the with
Cb as root
a single
4>81
one
*
4>83
the
( 750
+
cost
of
and
using
Cb 4
The
nesting
is
2 , 625
, 000
, using
43 % , respectively
order must
junction
~
C16 in Muninl is computed
the of
time
arrows
cost
is
tables
, with
via
of
4
x
375
marginalization
of
measure
, the
total
time
.
the
clique
cost
clique
of sending
the
separator
the
clique
Hugin
potentials
potentials
based
time
cost
Using
six
root
propagation
compute
, 000
remaining
a conservative
) . Thus
to
the
message
( I ) the
required
the
Cb being
inward
indicate
messages
computing
) . Each
( 750
costs
of
and
5 x
nested
Time
on of
has
the
a
larger
generating
of 4>80 '
375
, 000
) +
conventional 2 , 625
, 000
space
525
, 000
( i . e . , non =
13 , 125
trees and
7 x
time
costs
- nested
, 000
approach
=
7 , 427
, 500
) message
, respectively provides
savings
. gener
-
. Thus
, in
of
85 %
.
Costs the
its
+
junction
, of
evaluate
comnare -
a of
6 x
the
and to
to
number
( this
time
and
we
has
cost
+
case
In
4>1 ,
clique C16
, is
, 000
and
Space
up .
, 000
this
5.
level
, 000
root
375
space
ation
375
the
as x
*
for by
attached
( 2 ) the
time
525
tree
generated
labels
, and
, the
6 x
time
be
) . The
*
algorithm
junction
to
message
marginal
is
nested
message
applicability
~
snace -
tree approaches .
and
of time
the
costs
nested with
junction those
of
tree the
approach
conventional
,
S331 :IL NOI~~Nflr a3LS~N
' HddV'IVNOI~N'aANOO log
. -
r
L _
11-
-
1881
.
1
1
1 So I
. J -,
1
C
n
19
HOVO
62
UFFEKJ.lERULFF
space
required
pass
,
In
be
cost
The
clique
,
Co
of
<1>c
shall
assume
is
it
.
) .
For
( m
n
) IXcl
time
during
that
' l / Jc
,
.
is
is
the
outward
stored
in
the
clique
Note
potential
that
if
utilized
shall
,
,
,
,
information
this
however
< Pc
cost
,
may
refrain
from
.
c/ Jc
only
not
generated
we
compute
will
the
are
combination
always
C
+
simplicity
of
If
to
potentials
will
is
message
the
and
compute root
store
c/ Jc
clique
it
in
when
( i .e . ,
< Pc
, whereas
more
So
=I
than
0 ) ,
the
one
message
,
.
processing
8 .
Note
if
message
,
always
Using
that
have
to <1>so
n
times
algorithm
an
Note
marginalize when
V; c
, <1>80
' Jig ! , . . . , J~
So
=
or
0
generating
Kc
=
0
once
that the
-
( but
! , J~ not
=
,
to
.
both
. . . , Jign ) ,
and
the
The
)
if n
to
# 1
if
in
0
pass
,
# =
case
the
Hugin
algorithm which
+
Kc So
.
do
case
we
contributing
n
and
we
which
tables
equals
both
in :
4>c of
Ci
0
the that
inward
- Shenoy
to
So -
#
Shafer
Note
clique
algorithms
number '
an
clique on
.
leaf
Kc two
shown
a
based
by
unless the
4 > Si
was
a
is be
generated
being
contributes
,
will
preceded
whereas
table
+ 1 '
be
C
'
pass
there pass
must be
Jso
table
message
outward ,
inward )
between
one
the
with
Jc
only
from
4> 0
since
only
directly
to
the < Pc
pass
difference
unless
in ( =
algorithm
4> c ,
during algorithm
C
4Jc
outward
, the
computes it
,
- Shenoy
C
Hugin in
otherwise
anything
.
clique
the
processing ;
Shafer
do
v; c
in using
assume
the
=
does
1996
method
shall
4> c
we
extra
Outward
cost
not
little
contributing
algorithm to
conventional we
a
tables c/ Jc
algorithm - Shenoy
Figure
table
n
methods
contribute
5 . 1 .2 .
use
contributing
,
refined
Hugin
to
+
the
( Shenoy
Shafer
'
,
computing of
such
table
and
Anyway
m
of
reduced
The
in
with
domain
using
< Pso
,
time
.
I,
.
general
the
the
" ' UVm ' l / Jc
pass
the
IXvlu
recomputing
inward
on
,
0
1
0 , and
( i .e . ,
n Kc
if
tables
either =
0 .
5.2. NESTEDAPPROACH In describing the costs associated with the nested junction tree approach , we shall distinguish between message processing at level 1 (i .e., the outer most level) and at deeper levels.
5.2.1. Levell The processing of messages in a non-root clique C (i .e., receiving messages from neighbours C 1, . . . , Cn via separators 81, . . . , 8n , and sending to clique Co) may involve inference in a junction tree induced by VI U . . . U Vm U 81 U . . . U 8n (see Figure 9). Note that , in this setup , C may either be involved in inward message passing using either algorithm , or it may be involved in outward Shafer-Shenoy message passing . In the inward case, C1, . . . , Cn are
63
NESTEDJUNCTIONTREES I I
I
ct >So
I
r
I
I So I
t
-
L
I-
-
I-
,
~
I
C
c/>ST&
c!>Sl . .
81
Sn
ConvCost~ut (C) if algorithm
= Hugin
if So # 0
{conventional inward messageprocessing in C }
if
{ computing ~c = 4>0 ~ ""So}
Ct +- lx501 + IXcl else
Ct +- ( m + n + l ) IXcl { computing
c = so I17 = 14>Si I17 = 1
for i = 1 to n Ct +- Ct + IXcl
{ computing
else if ( So # 0 ) 1\ (n = 0 ) 1\ ( m > 0 ) Ct +- Ct + 21Xc I
{ C is a leaf clique
{ computing
with
Kc
# 0}
CPo = 'l/JcCPso}
for i = 1 to n if r > 1
{ r is the number
Ct f - Ct + rlXcl
of tables
contributing
to ~ c }
{ computing
4>0 = Il/Jc
Ct f - Ct + IXcl
{ computing
<1>Si = EC \ Si c/>c }
Cs f - Cs + IXsi I
{ storing
c/>Si in
Figure 8 . Space and time costs of receiving a message from the inward neighbour and sending to the outward neighbours . Note that r = n + 1 if m > 0 1\ So # 0 , r = n if m > 0 V So :1: 0 , and r = n - 1 if m = 0 1\ So = 0 .
the
outward
outward the
neighbours neighbour
inward
neighbour
assuming cannot The
done
message , the inward
and
Co
which
C
plus
a minimal be
tree . Thus performing
to
the
inference
of
message
propagation
inward
is going
remaining
triangulation
through
the
. In to
outward
, computing in
an
through processing towards
the
outward
send , and
case , Co
C1 , . . . , Cn
neighbours a marginal
induced
inference in a root
clique clique
. ( Recall
that
a root
clique
in
junction
is the
includes ,
tree . )
in
the C
level
equals
in the
level
2 junction the
cost
2 junction
of
64
UFFEKJlERULFF
cPSo
4>51
c/JSn .
.
.
NestedCostfn /out(C) CSf - c~oot
{space cost of inward propagation towards clique 'root '} { time cost of inward propagation towards clique 'root '} {storing ~So}
Ct -+- c~oot
Csf - Cs+ Ix801 Figure 9.
Spaceandtimecostsof receivingn messages andsending0 or 1, wherethe marginalis computed in a junctiontreeinducedby VI U. . . UVmU81U. . . U8n.
tree
( see
below
describe clique
is
5 .2 .2 . We
shall
inference
the
clique
2
( more
outward
operation Now
, since
at
cPSo . In
Section
propagation
of
situation
, say
separator
levell
6 , we
towards
shall a
root
10
processing
an
S * - marginal
that , and
the
message
purpose
generate tree
where
to
of
perform
a message
, we
-
that
going
at
1 , wanting
the tree
to
clique tree
C * can
j unction is
to
clique
junction
levell
S * . Note
defining
1 is
conventional
the
should
from
only
compute
-
per
-
a marginal
.
a
C * , at
of
junction a root
Figure in
costs . Since
>
the
marginals in
those
time deeper
towards
neighbours
message are
2 or
tree
passing
a clique
and
level
1 containing
) a set
the
via
or
l -
outward in
storing inward
space
at
a junction
typically
neighbour
of
conventional
the
message
contained
cost
performing
-
trees
level
Consider its
deeper
analyze
in
inward
from
or
j unction
at
space
of
.
now in
the
cost
calculated
ing
or
plus
the
Level
passing
form
)
how
E::: ~
relevant
be
be
C
receives
level
l
to
send
engaged
generated
1
which
a message in
potentials containing
messages
>
either
involved clique and
C
is to
inward in
that
C . might
share
a
NESTEDJUNCTIONTREES
65
ConvCost ~1(C) if algorithm
= Hugin
if m + n >
1
Ct t - (m + n) IXcl
{
computing
Cs t - IXcl
{
storing
{
comp
{
a
{
computing
{
storing
{
computing
the
c
.
=
rem
first
c
}
2:::~=1(ri - 1)n~~~rj 4>0'8}
.
number
of
1/ ; 0
1 / ; 0
~
}
absorptions
=
I1i
}
CPVi
}
}
I1i
Ti
CPo
' s
}
if (a > 0) V (m > 1) Ct-+- Ct+ (m + n) f1i rilXcl {computingf1i Ti I } if So =F0 Cs-+- Cs+ IXSoI {storing
{computingf1i ri marginals}
~
~
Figure 10. Spaceand time costsof receivingmessages from outward neighboursand sendingto the inward neighbourin a junction tree at nestinglevel greaterthan 1.
66
UFFEKJlERULFF
variables to
with
send
( assuming
That to
be
message
able
to
the
of of
This
way as
of
that
in
the
Denote to
by
send
that
to
chrononizes C
of
to
ICI
<
to
the
first
) 4>0
remaining
Ili
replacing messages
C
the
messages
Co .
for
Co
* ) \ co
neighbour
messages
will
be
from
the
outward
in
a junction
alternative
C ,
by
the
neighbours tree
will
to the
to
multiplied
called
, but
, or
, if , for
is
.
referred
variable
shall
that
outward
that
the
messages
are
messages
in
one
, and
such
, for
prop
refrain
-
from
each
>
neighbours
combination
of
C 1 syn
received
, for
either of
the
messages
.
( Note
-
from
, we
each
sends
part
have such
that
C2 , . . . , Cn
1 . Thus , C
the
Ci
scheduled
message
from
i
== 0 , generates
neighbour
batch
each
messages
I for
outward
So
that of
I ) I XSi
actual
algorithm
=
I1i
: l >~
r i
-
I
Using of
greater
. time
the
time
except once .
1
of
marginal
and
of
is
the
the
IIi
IXcl
>
root ri time .
not
have
time
need
combina
IX ( cns
that
-
* ) \ co 1
S * - marginal
=
=
( ri
( ii ) IIi
4>~ is
a
we
cor
-
, typically
>
+ IIi
S * - marginal
we
must an
is
(m
+
-
0
,
I Xc
m
>
1 ) IIi 4>~
TilXcl is
is
compute
case
the
re -
:
.
(i)
be
( i ) , it
( ii ) to
be
is
similar
computed
composed
the
pays 4>0 ' 8 ,
. For
corresponding equals
the
, a ) is
multiple
. Case
to
that
cases
the
going
by
absorptions
1 . In
going
. The
performed
. Note
two
only
S * - marginal
n ) IXcl
I multiplications
of
computing
compute
processed
1 ) Il ; ~ l rj
number or
to
are
distinguish
before (n
C
are
and
( i . e . , the
, since
, the
,
cost
messages
I divisions
algorithm
by
r1 . replacements
ri
I XSi
therefore
of
taken
that
Ci , where
computed
cost
step
of
such
1 , and
clique
first
messages
1/ Jc
which is
. The and
m
compute
1/ Jc
time
involves
of
cost
C
Ci
the
= l 4>Sj ' which
from
from
- Shenoy
to
,
r7 . combinations a
Shafer
)
that
at
combinations
than
( wrt
Ilj Ei
originating
the
number
=
message
a message
each
with
from
Hugin
one
placing
If
to
x ( cns
IS * I . )
Using ( the
of
their
(r i -
the
C
is going
* ) \ co I messages
multiple
from
) ; an
it
. Furthermore
considering
combinations
Co , or , if
responding
sent
which
inward
sending
marginals
of
size
messages
messages
IXsol
activity
of
its
clique
received
( 1996
to
.
all
all
IX ( cns
to
root
be
Co
configuration
are
worth
number
send
its
storage
tion
for
the
processes
extra
the
to
Jensen be
send
each
== 1 , . . . , n . Assuming
C2 , . . . , Cn
C 1,
by
paper
r i
C , i
is
messages
) might
present
Co
arbitrary
firing , 1994
clique
messages
needed
computing
( Xu
to
for
neighbours
of
with
have
sent
if
number
variable
agation
share
appropriate
messages
the
will
be
, C ' s outward
number
not
0) , C
S * - marginal
reason
product
#
does
must
generate
the
same
it
So
is , a
generate
to
S * which
S *larger
of
NESTED JUNCTION TREES
67
5.2.3. Level .2 or deeper- nested The processing of messagesin a non-root clique C at levell > 1, where the message, <1>80' to be sent is generated through inference in an induced junction tree at levell + 1, is shown in Figure 11. This situation resembles
8 .
(ro x)4>80 t So C
4>V1' . ' " 4>v'tJI .' 4>81' . ' " 4>sn
(rl x) rPs1 I / 81
8n .
.
\ . rPSn(TnX) '\
.
NestedCostt ; (C) CS-+- c~oot
{ space cost of inward prop . towards 'root '}
Ct -+- c~oot Cs-+- Cs+ IXso I
{ time cost of inward prop . towards 'root '} { storing 4>50}
Ct -+- CtIX(cns. )\ col lli Ti Cs-+- Cs+ E ~=2(ri - l )IXsi I
{inward prop. IX(cns. )\ col lli Ti times} {storing multiple messages for eachi > I } and
.
Figure 11. Space and time costs of receiving messages from outward neighbours sending to the inward neighbour in a junction tree at nesting level greater than 1.
~
the situation shown in Figure 9, the only difference being that C may receive multiple messagesfrom each outward neighbour, and that it may have to send multiple messagesto clique Co. Since C needs to perform IIi ri absorptions, with each absorption correspondingto an inward passin
UFFE KJlERULFF
68
the junction tree at levell + 1, and IX(cns*)\coI marginalizationsfor each combinationof messages , a total of IX(cns*)\col ni ri inward passesmust be performedin the levell + 1 tree. 5.3. SELECTINGCOSTFUNCTION Now, depending on the level, directionality of propagation, and algorithm used, we should be able to select which of the five cost functions given in Figures 7- 11 to use. In addition , we need a function for comparing two pairs of associated space and time costs to select the smaller of the two. To determine which of two costs, say c = (cs, Ct) and c' = (c~, c~), is the smaller, we compare the linear combinations Cs+ , Ct and C~ + 'Yc~, where the time factor , is chosenaccording to the importance of time cost. The algorithm Cost(C) for selecting the minimum cost is shown in Figure 12, where ' -<' refers to the cost comparison mentioned above.
Cost(C) if level = 1 if (direction
= inward
if ConvCostfn
) V ( algorithm
( C ) - < NestedCostfn
c = ConvCostfn
= Shafer - Shenoy ) / out ( C )
(C )
else c = NestedCostfn
/ out ( C )
else c = ConvCost
~ut ( C )
else if ConvCost
~ l ( C ) - < NestedCost
c = ConvCost
~ l (C )
~ 1( C )
else c = NestedCost
~ l (C )
Figure 12 .inclique Selecting functions and the minimum cost associated with message processing C.cost 5.4. SUMOF COSTS
Undera giventimefactor," theoverallminimumspaceandtimecosts of inwardor outwardpropagation of messages towards /froma givenroot
NESTEDJUNCTIONTREES clique, R, can
now be computed
69
as
CR= } : Cost (C) + C;emp , CEC
(1)
where
max IXsl +{CIct Hugill prop . SES > cmax not stored }IXclifoutward c;emp=
max CECIXc I
if Shafer-Shenoyprop.
0
otherwise.
During outward H ugin propagation we need auxiliary spacewhen generating messages ; thus, a space of size maxSESIXsl suffices. Further , for each clique, C, for which
Costs
All of the cost functions mentioned aboveare relative to a given root clique, R, and we are therefore only able to compute the cost of probability propagation (inward or outward) with R as root . However, we want to select the root clique such that the associatedcost is minimal , and, therefore, we must be able to compute the 'root cost' cC for each clique C E C. Assuming that root clique R has neighbours C1, . . . , On, Equation 1 can be re-expressedas n cR = Cost(R) + L cCi\ R + c~emp, i=l
(2)
where cCi\ R denotes the root cost in the subtree rooted at Ci and with the R-branch cut off. Note that , since cCi\ R = Cost(Ci ) +
L CC\Ci, CEneighbours (Ci)\ { R}
70
UFFEKJlERULFF
the root costscan be computedthrough inwardpropagationof costs. This is illustrated in Figure 13, whereeachclique C sendsthe cost message Cost(C) + l::~ 1CCi\C to its inward neighbourCo.
CCl \C
. . .
Cost(G) + E ~=l CCi\C . . .
.
.
CCYL \C
Figure 13.
Propagating costs of probability propagation .
Thus, what lacks to compute the root cost for Ci , i = 1, . . . , n, is cR\ Ci (i.e., the root cost for the subtree rooted at R and with the Ci-branch cut off ). However, this is nothing but the cost messagesent from R to Ci if we perform outward propagation of costs from R. That is, after a full propagation of costs (i.e., inward and outward) we can easily compute the root cost of any clique. Note that Cost() dependson the directionality (i.e., inward or outward). So, to compute the root costs for both inward and outward probability propagation we need to perform two full cost propagations. 7. Experiments To investigate the practical relevance of nested junction trees, the cost propagation schemedescribed above has been implemented and run on a variety of large real- world networks. The following networks were selected. The KK network (50 variables) is an early prototype model for growing barley. The link network (724 variabIes) is a version of the LQT pedigreeby ProfessorBrian Suarezextended for linkage analysis (Jensenand Kong, 1996). The Pathfindernetwork (109 variables) is a tool for diagnosing lymph node diseases(Heckerman et al., 1992). The Pignet network (441 variables) is a small subnet of a pedigree of breeding pigs. The Diabetesnetwork (413 variables) is a time-sliced network for determining optimal insulin dose adjustments (Andreassen et al., 1991). The Muninl - 4 networks (189, 1003, 1044, and 1041variables, respectively ) are different subnets of the Munin system (Andreassenet al., 1989).
NESTED JUNCTION TREES
71
The Water network (32 variables ) is a time -sliced model of the biological processes of a water treatment plant (Jensen et al., 1989) . The average space and time costs of performing a probability propaga tion is measured for each of these ten networks . Tables 1- 4 summarize the results obtained for inward Hugin propagation , inward Shafer-Shenoy prop agation , full H ugin propagation (i .e., inward and outward ) , and full ShaferShenoy propagation , respectively . All space/ time figures should be read as millions of floating point numbers / arithmetic operations . The first pair of space/ time columns lists the costs associated with conventional junction tree propagation . The remaining three pairs of space/ time columns show, respectively , the least possible space cost with its associated time cost , the costs corresponding to the highest average relative saving , and the least possible time cost with its associated space cost. The largest average relative savings were found by running the algorithm with various , -values for each network . The optimal values, , * , are shown in the rightmost columns .
TABLE
1
tional
(
.
Space
and
approach
,
minimum
space
and
(
iv
)
time
and
cost
minimum
costs
the
)
for
nested
,
(
iii
time
)
inward
propagation
trees
maximum
cost
Hugin
junction
average
relative
Space
Time
saving
15
. 1
50
. 3
0
. 7
28
. 1
83
. 3
0
. 3
0
. 7
0
. 1
2
. 2
Diabetes
. 8
. 0
207
Munin3
Munin4
As
728
9
3
. 7
12
.
. 3
64
. 6
28
. 3
anticipated
ated
with
costs
of
minimum
,
1119
. 9
because
cost
may
,
be
ii
(
)
i
)
the
conven
maximum
space
-
nesting
and
that
. 8
0
. 3
0
. 9
0
. 5
4
. 9
1128
0
. 6
3532
. 7
is
not
time
costs
,
(
1 '
. 0
2
. 7
26
.
9
. 5
9
. 9
Pathfinder
. 8
0
. 25
.
1
0
. 15
. 3
0
. 30
8
. 4
0
. 30
9
. 25
2
. 4
2
. 4
3
. 4
0
. 0
0
. 30
. 9
22
. 3
0
.
time
costs
general
than
,
since
network
very
limited
15
associ
the
-
time
nesting
Pathfinder
a
315
50
larger
to
1
29
. 9
maximum
The
0
. 8
12
the
much
although
possible
68
1
,
in
only
. 25
. 3
63
recommended
is
0
. 6
. 3
.
. 30
. 6
5
. 5
large
. 30
0
0
1
are
0
. 7
. 6
6
)
. 5
61
. 0
. 8
0
37
2
. 9
*
. 3
. 2
315
,
. 3
0
. 1
,
12
. 6
. 5
=
Thus
100
Time
0
34
. 4
=
8
1
but
unacceptably
nesting
68
. 7
85
.
. 5
. 0
1
costs
. 9
62
90
203
,
Space
40
61
. 2
.
. 5
. 2
95818
networks
,
. 7
0
. 2
it
11
. 1
. 5
space
6
. 3
1
propagation
cost
of
Time
. 1
29692
. 5
=
Space
0
all
minimum
space
time
19
. 7
for
conventional
,
. 04
0
. 7
. 2
8
0
. 1
3
18
Water
33
. 7
Munin2
differently
0
11
Munin1
ated
. 2
0
Time
link
0
=
Space
KK
Pignet
(
Nested
,
Pathfinder
with
.
Conventional
Network
using
approach
yields
the
associ
-
behaves
degree
.
72
UFFEKJlERULFF
TABLE 2. Space and time costs for inward Shafer-Shenoy propagation using (i) the conventional approach, and the nested junction trees approach with (ii ) maximum nesting (minimum space cost) , (iii ) maximum average relative saving of space and time costs, and (iv ) minimum time cost. Conventional
Nested 1 = 0
Network
Space
Time
Space
'Y=
Time
' Y*
'Y =
Space
Time
100
Space
Time
-
--y* .
KK
7 .9
52 . 2
6 .0
787 .9
6 .8
42 .7
7 .0
42 .4
0 . 15
link
4 .5
83 .3
2 .1
587 .4
3 .6
76 . 1
4 .0
73 .8
0 .05
Pathfinder
0 .1
0 .7
0 .1
1 .3
0 .1
0 .7
0 .1
0 .7
0 . 10
Pignet
0.3
2.2
0.2
4.0
0.2
2.4
0.3
2.1
0.15
Diabetes
1 .2
33 .9
0 .5
97 .7
728 .8
79 .2
Munin2
0 .7
9 .9
0 .2
Muninl
51 .0
0 .7
36 .2
1.2
33 .3
0 .05
.7
85 .3
379 .3
88 . 0
368 .6
0 . 10
50 .7
0 .4
11 .1
0 .7
9 .1
0 .05
31260
Munin3
0 .7
12 .2
0 .2
324 .0
0 .5
12 .0
0 .7
10 .5
0 .05
M unin4
3 .0
64 .3
1.1
235 . 7
2 .1
52 .6
2 .4
51 .4
0 .05
Water
6 .1
29 .2
5 .3
115 .8
5 .9
27 . 1
6 .1
26 .9
0 .15
However , as the , = , * columns show, a moderate increase in the space costs tremendously reduces the time costs. (The example in Figure 5 demonstrates the dramatic effect on the time cost as the degree of nesting is varied .) In fact , for , = , * , the time costs of nested computation are either roughly identical or smaller than those of conventional computation , while space costs are still significantly reduced for most of the networks . Interestingly , for all networks the minimum time costs (, = 100) are less than the time costs of conventional propagation , and , of course, the associated space costs are also less than in the conventional case, since the saving on the time side is due to nesting which inevitably reduces the space cost . Comparing Tables 3 and 4 with , = , * , we note , somewhat surprisingly , that the time costs of a full H ugin propagation are consistently smaller than those obtained using the Shafer-Shenoy algorithm , while the space costs are either comparable or smaller for the H ugin algorithm . Note , however , that the , * 's are significantly smaller in the Shafer-Shenoy case, indicating an attempt to keep the space costs under control .
ACKNOWLEDGEMENTS I wish to thank Steffen L . Lauritzen for suggesting the cost propagation scheme, Claus S. Jensenfor providing the link and Pignet networks, David
73
JUNCTION TREES NESTED
TABLE3. Space andtimecostsfora fullHuginpropagation using(i) theconventional approach , andthenested junctiontreesapproach with(ii) maximum nesting (minimum space cost), (iii) maximum average relativesavingof space andtimecosts , and(iv) minimum timecost. Conventional Nested , =0 1= , . l' = 100 Network Space Time Space Time SpaceTime SpaceTime , . KK 15.8 93.3 1.4 1162 .2 2.7 104 .1 9.0 80.5 0.20 link 28.4 164 .1 0.5 29773 .2 7.2 164 .6 12.6 142 .6 0.20 Pathfinder 0.2 1.2 0.1 1.6 0.2 1.1 0.2 1.1 0.15 Pignet 0.9 4.5 0.1 64.1 0.4 4.3 0.6 4.1 0.20 Diabetes 11.0 59.5 0.6 116 .5 1.0 61.0 5.3 55.5 0.15 Munin1 213.3 1362 .3 25.5 96451 .8 74.0 949.6 74.4 948.8 0.15 Munin2 3.2 18.7 0.6 212.6 1.4 19.0 2.4 17.3 0.20 Munin3 3.8 23.6 0.5 97.3 1.4 21.6 2.5 20.8 0.15 Munin4 18.4 119 .4 5.0 1183 .2 5.5 122 .1 13.0 105 .2 0.20 Water 9.0 46.3 1.0 3550 .7 3.2 44.0 4.4 40.3 0.15
TABLE 4. Spaceand time costsfor a full Shafer-Shenoypropagationusing (i) the conventional approach, and the nestedjunction trees approachwith (ii) maximumnesting (minimum spacecost) , (iii ) maximumaveragerelativesavingof spaceand time costs, and (iv) minimum time cost. Conventional
Network KK link Pathfinder Pignet Diabetes Munin1 Munin2 Munin3 Munin4 Water
Nested , =0 "Y= , * , = 100 Space Time Space Time Space Time Space Time
,*
9.0 6.8 0.1
153.6 263.1 2.1
7.1 4.5 0.1
889.3 767.1 2.6
7.9 6.0 0.1
114.9 222.5 2.1
8.1 6.4 0.2
114.4 220.8 2.0
0.05 0.05 0.05
0.5 1.8 117.0 1.1 1.2 4.9 6.7
7.0 80.9 2411.9 31.6 44.8 209.9 60.6
0.3 1.1 98.4 0.6 0.7 3.0 5.9
8.9 97.9 32943.8 72.4 356.6 381.3 147.3
0.4 1.3 105.8 0.8 1.0 3.1 6.5
7.1 82.0 1309.1 31.2 44.1 199.4 55.5
0.5 1.8 110.9 1.1 1.2 5.7 6.6
6.8 79.0 1263.4 29.2 42.5 154.9 55.3
0.05 0.05 0.05 0.05 0.05 0.01 0.15
74
UFFEKJlERULFF
References Andreassen, S., Hovorka, R., Benn, J., Olesen, K . G. and Carson, E. R. (1991) A modelbased approach to insulin adjustment , in M . Stefanelli , A . Hasman , M . Fieschi and
J. Talmon (eds), Proceedings of the Third Conference on Artificial Intelligence in Medicine , Springer -Verlag , pp . 239- 248. Andreassen , S., Jensen , F . V ., Andersen , S. K ., Falck , B ., Kjrerulff , U ., Woldbye , M .,
S0rensen, A . R., Rosenfalck, A . and Jensen, F . (1989) MUNIN - an expert EMG assistant, in J. E. Desmedt (ed.) , Computer-Aided Electromyography and Expert Systems, Elsevier Science Publishers B. V . (North -Holland ) , Amsterdam , Chapter 21. Frydenberg, M . ( 1989) The chain graph Markov property , Scandinavian Journal of Statistics , 17 , pp . 333 - 353 .
Jensen , F . V . ( 1996) An Introduction
to Bayesian Networks . UCL Press , London .
Jensen, F . V ., Kjrerulff , U., Olesen, K . G. and Pedersen, J. (1989) Et forprojekt til et ekspertsystem for drift af spildevandsrensning (an expert system for control of waste water treatment - a pilot project ) , Technical report , Judex Datasystemer A / S, Aalborg , Denmark . In Danish .
Jensen, C. S. and Kong , A . (1996) Blocking Gibbs sampling for linkage analysis in large pedigrees with many loops , Research Report R -96-2048 , Department of Computer Science , Aalborg University , Denmark , Fredrik Bajers Vej 7, DK -9220 Aalborg 0 .
Jensen, F. V ., Lauritzen , S. L . and Olesen, K . G. (1990) Bayesian updating in causal probabilistic
networks by local computations , Computational
Statistics
Quarterly , 4 ,
pp . 269 - 282 .
Heckerman , D ., Horvitz , E . and Nathwani , B . ( 1992) Toward normative expert systems : Part I . The Pathfinder project , Methods of Information in Medicine , 31 , pp . 90- 105.
Kim , J. H . and Pearl, J. (1983) A computational model for causal and diagnostic reasoning in inference systems . In Proceedings of the Eighth International on Artificial Intelligence , pp . 190- 193.
Joint Conference
Lauritzen , S. L ., Dawid , A . P., Larsen, B. N . and Leimer, H.-G. (1990) Independence properties of directed Markov fields , Networks , 20 , pp . 491- 505. Lauritzen , S. L . and Spiegelhalter , D . J . ( 1988) Local computations with probabilities on graphical structures and their application to expert systems . J oumal of the Royal Statistical Society , Series B , 50 , pp . 157- 224.
Shafer, G. and Shenoy, P. P. (1990) Probability propagation , Annals of Mathematics and Artificial -
Intelli -.qence,. 2 ., -pp- . 327- 352 .
Shenoy, P. P. (1996) Binary Join Trees, in D . Geiger and P. Shenoy (eds.) , Proceedingsof the Twelfth Conference on Uncertainty Publishers
, San Francisco
, California
in Artificial
,- -pp- . 492 - 499 .
Intelligence , Morgan Kaufmann
Xu , H . (1994) Computing marginals from the marginal representation in Markov trees, in Proceedings of the Fifth International
Conference on Information
Processing and
Management of Uncertainty in Knowledge-Based Systems (IPMU ) , Cite Interna tionale
Universitaire
, Paris
, France
, pp . 275 - 280 .
BUCKET
ELIMINATION
PROBABILISTIC
: A
UNIFYING
FRAMEWORK
FOR
INFERENCE
R . DECHTER Department
of Information
and
Computer
Science
University of California , Irvine dechter @ics.uci .edu
Abstract . Probabilistic inference algorithms for belief updating , finding the most probable explanation , the maximum a posteriori hypothesis , and the maxi mum expected utility are reformulated within the bucket elimination frame work . This emphasizes the principles common to many of the algorithms appearing in the probabilistic inference literature and clarifies the relation ship of such algorithms to nonserial dynamic programming algorithms . A general method for combining conditioning and bucket elimination is also presented . For all the algorithms , bounds on complexity are given as a function of the problem 's structure .
1. Overview Bucketeliminationis a unifyingalgorithmicframework that generalizes dynamicprogramming to accommodate algorithms for manycomplex problem solvingandreasoning activities , includingdirectionalresolution for propo sitionalsatisfiability(Davisand Putnam, 1960 ), adaptiveconsistency for constraintsatisfaction(Dechterand Pearl, 1987 ), Fourierand Gaussian eliminationfor linearequalitiesand inequalities , and dynamicprogram mingfor combinatorial optimization(BerteleandBrioschi , 1972 ) . Here, after presenting theframework , wedemonstrate that a numberof algorithms for probabilisticinference canalsobe expressed asbucket -eliminationalgorithms. The main virtuesof the bucket -eliminationframeworkare simplicity and generality . By simplicity , we meanthat a completespecification of 75
76
R. DECHTER
bucket -elimination algorithms is feasible without introducing extensive ter -
minology (e.g., graph conceptssuch as triangulation and arc-reversal) , thus making the algorithms accessible to researchers in diverse areas. More im portant , the uniformity of the algorithms facilitates understanding , which encourages cross-fertilization and technology transfer between disciplines . Indeed , all bucket -elimination algorithms are similar enough for any im provement to a single algorithm to be applicable to all others expressed in this framework . For example , expressing probabilistic inference algorithms a.g bucket -elimination methods clarifies the former 's relationship to dynamic programming and to constraint satisfaction such that the knowledge accumulated in those areas may be utilized in the probabilistic framework . The generality of bucket elimination can be illustrated with an algorithm in the area of deterministic reasoning . Consider the following algo-
rithm for deciding satisfiability . Given a set of clauses (a clause is a disjunction of propositional variables or their negations) and an ordering of the propositional variables , d = Ql , ..., Qn, algorithm directional resolution
(DR) (Dechter and Rish, 1994) , is the core of the well-known Davis-Putnam algorithm for satisfiability (Davis and Putnam, 1960). The algorithm is described using buckets partitioning the given set of clauses such that all the clauses containing Q i that do not contain any symbol higher in the ordering are placed in the bucket of Q i , denoted bucketi .
The algorithm (seeFigure 1) processesthe buckets in the reverseorder of d. When processing bucketi , it resolves over Q i all possible pairs of clauses in the bucket and inserts the resolvents into the appropriate
lower buckets .
It was shown that if the empty clause is not generated in this process then the theory is satisfiable and a satisfying truth assignment can be generated in time linear in the size of the resulting theory . The complexity of the
algorithm is exponentially bounded (time and space) in a graph parameter called induced width (also called tree-width) of the interaction graph of the theory , where a node is associated with a proposition and an arc connects
any two nodes appearing in the same clause (Dechter and Rish, 1994) . The belief-network algorithms we present in this paper have much in common with the resolution procedure above. They all possess the prop erty of com piling a theory into one from which answers can be extracted easily and their complexity is dependent on the same induced width graph parameter . The algorithms are variations on known algorithms and , for the most part , are not new , in the sense that
the basic ideas have existed for
some time (Cannings et al., 1978; Pearl, 1988; Lauritzen and Spiegelhalter, 1988 ; Tatman
and
Shachter
Favro , 1990 ; Bacchus
and
, 1990 ; Jensen van
et al . , 1990 ; R .D . Shachter
Run , 1995 ; Shachter
, 1986 ; Shachter
and
, 1988 ;
Shimony and Charniack, 1991; Shenoy, 1992) . What we are presenting here is a syntactic and uniform exposition emphasizing these algorithms ' form
BUCKETELIMINATION
77
Algorithm directional resolution Input : A set of clausescp, an ordering d == Q! , ..., Qn. Output : A decision of whether
Algorithm directional resolution
as a straightforward elimination algorithm . The main virtue of this presentation , beyond uniformity , is that it allows ideas and techniques to flow across the boundaries between areas of research. In particular , having noted that elimination algorithms and clustering algorithms are very similar in the context of constraint processing (Dechter and Pearl , 1989) , we find that this similarity carries over to all other tasks . We also show that the idea of conditioning , which is as universal as that of elimination , can be incorporated and exploited naturally and uniformly within the elimination framework . Conditioning is a generic name for algorithms that search the space of partial value assignments , or partial conditionings . Conditioning means splitting a problem into subproblems based on a certain condition . Al gorithms such as backtracking and branch and bound may be viewed as conditioning algorithms . The complexity of conditioning algorithms is exponential in the conditioning set, however , their space complexity is only linear . Our resulting hybrid of conditioning with elimination which trade off time for space (see also (Dechter , 1996b; R . D . Shachter and Solovitz , 1991)) , are applicable to all algorithms expressed within this framework . The work we present here also fits into the framework developed by Arnborg and Proskourowski (Arnborg , 1985; Arnborg and Proskourowski , 1989) . They present table -based reductions for various NP -hard graph prob lems such as the independent -set problem , network reliability , vertex cover , graph k-colorability , and Hamilton circuits . Here and elsewhere (Dechter and van Beek, 1995; Dechter , 1997) we extend the approach to a different set of problems .
78
R. DECHTER Following
preliminaries
algorithm
for
we
extend
(
4
and
to
)
.
2
-
9
.
tree
present
and
of
We
(
section
8
Then
poly
,
-
a
-
pos
-
utility
tree
for
.
.
expected
' s
)
)
maximum
schemes
section
3
explana
maximum
Pearl
describe
elimination
probable
the
the
to
then
most
finding
finding
-
(
the
algorithms
elimination
bucket
performance
finding
taBks
for
the
algorithms
combining
the
Conclusions
are
given
in
.
provide
of
nodes
a
signify
,
the
and
.
A
of
Xi
,
of
parent
Xi
in
)
E
E
E
,
we
G
,
of
while
(
Xi
its
)
pai
Xi
Let
=
acyclic
=
,
graph
between
having
i
Ipai
Xl
.
.
.
,
,
and
)
the
}
.
=
)
(
.
Xi
)
=
{
IXi
V
,
,
(
Xi
)
,
of
chi
.
.
i
:
#
For
,
{
of
the
arcs
.
.
.
,
Xn
set
(
,
are
is
a
set
edges
,
.
the
If
set
pointing
Xi
)
,
,
to
comprises
,
Pi
has
.
}
of
Xi
arise
Xi
it
,
variables
Ch
if
Xl
the
can
family
graph
variable
denoted
confusion
of
=
.
linked
conditional
directed
is
each
acyclic
directions
}
the
Xi
is
=
j
comprises
The
graph
a
V
,
domains
by
of
the
we
abbreviate
includes
no
Xi
directed
ignored
:
-
over
the
quantified
where
Xi
no
by
,
V
to
nodes
the
}
E
Whenever
directed
E
Xj
given
between
notion
un
graph
from
are
points
child
,
.
Dn
and
Xi
X
Ch
A
G
Xj
value
the
beliefs
acyclic
influences
influences
pa
to
takes
on
partial
directed
and
cycles
(
Xi
,
Xj
.
)
and
.
{
D1
,
Xi
of
identical
X
,
(
set
.
are
,
Xi
denoted
graph
)
domains
,
and
undirected
,
these
pair
(
points
variables
an
Xj
{
Xi
Xi
by
child
a
=
a
causal
relies
that
the
that
is
say
nodes
variables
=
about
by
direct
network
graph
and
Xi
of
reasoning
defined
that
of
belief
directed
elements
is
variables
strength
A
It
existance
the
probabilities
for
.
random
arcs
variables
formalism
uncertainty
representing
The
X
.
with
networks
P
the
the
clustering
conditions
In
of
to
)
relates
we
Preliminaries
Belief
pa
5
7
,
its
taBks
it
section
)
analyze
the
extend
method
der
{
and
Section
join
section
(
,
(
6
conditioning
(
)
2
and
to
hypothesis
section
section
updating
algorithm
section
teriori
(
belief
the
tion
(
.
product
.
,
Xn
A
P
its
The
.
}
be
a
belief
=
=
{
Pi
parents
belief
set
of
random
network
network
}
,
,
namely
is
where
Pi
a
variables
pair
(
G
denotes
over
P
)
where
probability
is
a
directed
relationships
probability
a
multivalued
G
probabilistic
conditional
represents
,
matrices
distribution
Pi
: . . . :
over
form
P(Xl, ...., Xn) ==Ili =lP (Xilxpai ) where an assignment (X I = Xl , ..., Xn = xn ) is abbreviated to x = (Xl , ..., xn) and where Xs denotes the projection of a tuple X over a subset of variables S . An evidence set e is an instantiated subset of variables . A = a denotes a partial assignment to a subset of variables A from their respective domains . We use upper case letter for variables and nodes in a graph and lower case letters for values in variable 's domains .
79
BUCKETELIMINATION
(a)
Figure
2.
Example
belief
2 .1
(b)
network
P (g , f , d , c , b, a ) = P ( glf ) P ( flc , b) P ( dlb , a ) P ( bla ) P ( cla )
Consider
the
belief
network
P ( g , f , d , c , b , a ) == P ( glf Its
acyclic The
graph following
namely
, given
each
proposition
some
observed
rest
of the
are
some
function
evidence
an
is known
that
multiply
, and
small
cycle
elimination relationship We partial
with
conclude tuple
( us , xp ) to Definition subset
an
tasks
variables
are
or for
existing this
singly
the
cycle
. In above
methods
will
be
with
some
of
tuple
S , where
variables Us
, and
functions XES
Xp by )
, the
for
sparse
called -
networks
sections be
,
bucket
presented
-
and
.
a variable a value
functions
( Pearl
Spiegelhal
conventions
Given
permit
, also
and
will
discussed
that
algorithm
approach
subsequent tasks
a util -
networks
well
notational
appended
all
propagation
- cutset
work
hypothesis also
, they
, 1988 ; Lauritzen
the
( map ) ,
variables
- connected this
the
section
the
. Nevertheless
clusters of
decision
NP - hard
extending
of
, 4 . given
( meu ) .
methods
small
a subset
of
to
hypothesis
finally
a subset
( Pearl
each
( mpe ) , or , given
problem
for to
, 1986 ) . These
2 . 2 ( elimination of
are
networks
, S a subset denote
the
of
Msignment
aposteriori to
updating
probability
explanation
, and
{ B ,C } .
: 1 . belief
posterior
probability
assignment
to
of
algorithm
- cutsets
networks
probable
probability
approaches
algorithms
case , pa ( F ) =
the
maximum
tree - clustering
ter , 1988 ; Shachter
belief
their
these
- connected
conditioning
the
utility
main
over
, finding
propagation two
this
most
assignment
expected
2a . In
a maximum
3 . Finding
, finding
a polynomial
with
the
by
, b ) P ( dlb , a ) P ( bla ) P ( cla ) .
, computing
, finding
maximizes
the
1988 ) . The
defined
observations
variables
maximizes
Figure
, 2 . Finding
that
to
queries
variables
variables
It
in
a set
or , given
ity
is given
) P ( flc
defined
not
xp
. Let in
u be
S . We
a
use
of Xp .
a function ( minxh
h defined ),
( maxxh
over ),
80
R. DECHTER
(meanxh), and (Ex h) are definedover U = S - {X } as follows. For every U = u, (minxh )(u) = minxh(u, x), (maxxh) (u) = maxxh(u, x), (Ex h)(u) = Ex h(u, x), and (meanxh)(u) =" Ex~ , where' ,XI is Ithe I cardinality of X 's domain. Given a set of functions h1, ..., hj defined over the subsets81, ..., 8j , the productfunction (lljhj ) and EJ hj are defined over U == UjSj . For every U = u, (lljhj )(u) = lljhj (usj)' and (Ej hj ) (u) = Ej hj (uSj) . 3 . An Elimination
Algorithm
for Belief
Assessment
Belief updating is the primary inference task over belief networks . The task is to maintain the probability of singleton propositions once new evidence arrives . Following Pearl 's propagation algorithm for singly -connected net works (Pearl , 1988) , researchers have investigated various approaches to belief updating . We will now present a step by step derivation of a general variable - elimination algorithm for belief updating . This process is typical for any derivation of elimination algorithms . Let X I = XI be an atomic proposition . The problem is to assess and update the belief in Xl given some evidence e. Namely , we wish to compute P (X I = Xl ie) = Q' . P (X I = Xl , e) , where Q' is a normalization constant . We will develop the algorithm using example 2.1 (Figure 2) . Assume we have the evidence 9 = 1. Consider the variables in the order d1 = A , C , B , F , D , G . By definition we need to compute
P(gif )P(flb,c)P(dla,b)P(cla)P(bla)P(a)
L
P (a , 9 == 1) ==
c,b,j ,d ,g = l
We can now apply some simple symbolic manipulation , migrating each conditional probability table to the left of summation variables which it does not reference, we get == P ( a ) L
P ( cla
) L
C
Carrying pute defined
the the
P ( bla ) L b
computation
rightmost by : AG ( f ) =
from
summation Lg
P ( flb
, c ) LP
f
= 1 P ( glf
right which ) and
( dlb
, a ) L
d
to
left
( from
generates place
it
P ( glf
)
( 1)
9= 1
as
to
A ) , we
a
function
G
over
far
to
the
left
first
com
-
f , AG ( f ) as
possible
,
yielding
P(a) L P(cla)L P(bla ) L P(flb,c)Aa(f) L P(dlb,a) C b f d
(2)
BUCKETELIMINATION bucketa bucketD bucketF bucketB bucketc bucketA
= = = = = =
81
P(glf), 9 = 1 P(dlb,a) P(flb, c) P(bla) P(cla) P(a)
The answer to the query P (alg == 1) can be computed by evaluating the last product and then normalizing . The bucket -elimination algorithm mimics the above algebraic manipu lation using a simple organizational devise we call buckets, as follows . First , the conditional probability tables (CPT s, for short ) are partitioned into buckets , relative to the order used d1 == A , C , B , F , D , G , as follows (going from last variable to first varaible ) : in the bucket of G we place all functions mentioning G . From the remaining CPTs we place all those mentioning D in the bucket of D , and so on . The partitioning rule can be alternatively stated as follows . In the bucket of variable X i we put all functions that mention Xi but do not mention any variable having a higher index . The resulting initial partitioning for our example is given in Figure 3. Note that observed variables are also placed in their corresponding bucket . This initialization step corresponds to deriving the expression in Eq . ( 1) . Now we process the buckets from top to bottom , implementing the
82
R. DECHTER
Bucket G Bucket D Bucket F Bucket B Bucket C
P(a )
Bucket A Figure 4.
Bucket elimination along ordering d 1 = A , C, B , F, D , G.
right to left computation of Eq. (1) . Bucketa is processedfirst . Processing a bucket amounts to eliminating the variable in the bucket from subsequent computation
. To eliminate
G , we sum over all values of g . Since , in this case
we have an observed value 9 = 1 the summation is over a singleton value .
Namely, AG(f ) = L9 =1P(glf ), is computedand placedin bucketF (this correspondsto deriving Eq. (2) from Eq. (1)). New functions are placed in lower buckets using the same placement rule . BucketD is processed next . We sum-out D getting AD(b, a) = Ed P (dlb, a) ,
that is computed and placed in bucketB, (which correspondsto deriving Eq. (3) from Eq. (2)). The next variable is F . BucketF contains two functions P (fJb, c) and Aa (f ), and thus, following Eq. (4) we generate the function
AF(b, c) := Ll P(flb , c) . Aa(f ) whichis placedin bucketB(this corresponds to deriving Eq. (4) from Eq. (3)) . In processingthe next bucketB, the function AB(a, c) == Lb (P (bla) . An (b, a) . AF(b, c)) is computed and placed in bucketc (deriving Eq. (5) from Eq. (4)). In processing the next bucketc, Ac (a) = LCECP (cla) . AB(a, c) is computed (which correspondsto deriving Eq. (6) from Eq. (5)). Finally , the belief in a can be computed in bucketA, P (alg == 1) == P (a) . AC(a) . Figure 4 summarizes the flow of computation of the bucket elimination algorithm for our example . Note that since throughout this process we recorded two -dimensional functions at the most ,
the complexity the algorithm using ordering d1 is (roughly) time and space quadratic in the domain sizes. What will occur if we use a different variable ordering ? For example , lets apply the algorithm using d2 = A , F , D , C , B , G . Applying algebraic manipulation from right to left along d2 yields the following sequence of deri vations
:
P(a, 9 = 1) = P (a) Ll Ld Lc P(cla) Lb P(bla) P(dla, b)P(flb , c) Lg =1P (glf )=
BUCKETELIMINATION
bucket
B
bucket
C
bucket
D
bucket
F
bucket
A
83
-
-
= P(a)
(a) Figure 5.
P(a) Ll P(a) Ll P(a) Ll P(a) Ll
(b)
The buckets output
when processing along d2 = A , F , D , C , B , G
AG(f ) Ld Lc P(cla) Lb P(bla) P(dla, b)P(flb , c)== Aa(f ) Ld Lc P(Cla)AB(a, d, c, f ) == A9(f ) Ld Ac(a, d, f ) == Aa(f )AD(a, f ) =
P (a)AF(a) The bucket elimination
process for ordering
d2 is summarized
in Figure
5a. Each bucket contains the initial CPTs denoted by P 's, and the functions generated throughout the process, denoted by AS. We summarize with a general derivation of the bucket elimination algo-
rithm , called elim-bel. Consider an ordering.ofthe variablesX = (Xl , ..., Xn). Using the notation Xi = (Xl ' ..., Xi) and xi = (Xi, Xi+l , ..., Xj), where Fi is the family of variable Xi , we want to compute :
P(Xl, e) = ~ P(Xn,e) = ~ ~ IIiP(xi, elXpai )= X= X2n
_(nX2
l ) Xn
Seperating X n from the rest of the variables we get :
P(Xn,elxpan )llxiEchnP (xi,elXpai )= }=: IIXiEX - FnP(Xi, elXpai ) .E xn _ (n -l) X=X2
where
An(XUn ) = L P(xn, eIXpan )I1XiEChnP (Xi, e!Xpai ) xn
(7)
84
R. DECHTER
Figure 6.
Algorithm
elim - bel
Where Un denoted the variables appearing with X n in a probability component , excluding Xn . The process continues recursively with Xn - l . Thus , the computation performed in the bucket of Xn is captured by Eq . (7) . Given ordering Xl , ..., Xn , where the queried variable appears first , the C PTs are partitioned using the rule described earlier . To process each bucket , all the bucket 's functions , denoted AI , ..., Aj and defined over subsets SI , ..., Sj are multiplied , and then the bucket 's variable is eliminated by summation . The computed function is Ap : Up - t R , Ap == }::;x ni = IAi , where Up = UiSi - Xp . This function is placed in the bucket of it : largest index variable in Up. The procedure continues recursively with the bucket of the next variable going from last variable to first variable . Once all the buckets are processed, the answer is available in the first bucket . Algorithm elim -bel is described in Figure 6.
Theorem 3.1 Algorithm elim-bel compute the posterior belief P (xlle ) for any given ordering of the variables. 0 Both the peeling algorithm for genetic trees (Cannings et al., 1978), and Zhang and Poole's recent algorithm (Zhang and Poole, 1996) are variations of elim-bel.
BUCKETELIMINATION
(a )
85
(c)
(b)
Figure 7.
Two ordering of the moral graph of our example problem
3.1. COMPLEXITY We see that although elim -bel can be applied using any ordering , its complexity varies considerably . Using ordering d1 we recorded functions on pairs of variables only , while using d2 we had to record functions on four variables
(see Bucketc in Figure 5a) . The arity of the function recorded in a bucket equals the number of variables appearing in ing the bucket 's variable . Since recording a space exponential in r we conclude that the exponential in the size of the largest bucket
that processed bucket , exclud function of arity r is time and complexity of the algorithm is which depends on the order of
.
processIng
.
Fortunately , for any variable ordering bucket sizes can be easily read in advance
from
an ordered
associated
with
the elimination
process . Consider
the moral graph of a given belief network . This graph has a node for each propositional variable , and any two variables appearing in the same CPT are connected in the graph . The moral graph of the network in Figure 2a is given in Figure 2b . Let us take this moral graph and impose an ordering on its nodes. Figures 7a and 7b depict the ordered moral graph using the two orderings d1 = A , C , B , F , D , G and d2 = A , F , D , C , B , G . The ordering is pictured from bottom up . The width of each variable in the ordered graph is the number of its earlier neighbors in the ordering . Thus the width of G in the ordered graph along d1 is 1 and the width of F is 2. Notice now that using ordering d1, the
number
of variables
in the
initial
buckets
of G and
2 respectively . Indeed , in the initial partitioning
F , are also
1 , and
the number of variables
mentioned in a bucket (excluding the bucket's variable) is always identical to the width of that node in the corresponding ordered moral graph . During processing we wish to maintain the correspondance that any
86
R. DECHTER
two nodes in the graph are connected if there is function (new or old ) deefined on both . Since, during processing , a function is recorded on all the variables apearing in a bucket , we should connect the corresponding nodes in the graph , namely we should connect all the earlier neighbors of a processed variable . If we perform this graph operation recursively from last node to first , (for each node connecting its earliest neighbors ) we get the the induced graph . The width of each node in this induced graph is identical to the bucket 's sizes generated during the elimination process (see Figure 5b) . Example 3 .2 The induced moral graph of Figure 2b, relative to ordering d1 == A , C , B , F , D , G is depicted in Figure 7a. In this case the ordered graph and its induced ordered graph are identical since all earlier neighbors of each node are already connected. The maximum induced width is 2. Indeed, in this case, the maximum arity of functions recorded by the elimination algorithms is 2. For d2 == A , F , D , C , B , G the induced graph is depicted in Figure 7c. The width of C is initially 1 (see Figure 7b) while its induced width is 3. The maximum induced width over all variables for d2 is 4, and so is the recorded function 's dimensionality . A formal definition of all the above graph concepts is given next . Definition 3 .3 An ordered graph is a pair (G , d) where G is an undirected graph and d = Xl , ..., Xn is an ordering of the nodes. The width of a node in an ordered graph is the number of the node 's neighbors that precede it in the ordering . The width of an ordering d, denoted w (d) , is the maximum width over all nodes. The induced width of an ordered graph , w* (d) , is the width of the induced ordered graph obtained as follows : nodes are processed from last to first ,. when node X is processed, all its preceding neighbors are connected. The induced width of a graph , w * , is the minimal induced width over all its orderings . The tree-width of a graph is the minimal induced width plus one (Arnborg , 1985) . The established connection between buckets ' sizes and induced width motivates finding an ordering with a smallest induced width . While it is known that finding an ordering with the smallest induced width is hard (Arnborg , 1985) , usefull greedy heuristics as well as approximation algorithms are available (Dechter , 1992; Becker and Geiger , 1996) . In summary , the complexity of algorithm elim -bel is dominated by the time and space needed to process a bucket . Recording a function on all the bucket 's variables is time and space exponential in the number of variables mentioned in the bucket . As we have seen the induced width bounds the arity of the functions recorded ; variables appearing in a bucket coincide with the earlier neighbors of the corresponding node in the ordered induced moral graph . In conclusion :
BUCKETELIMINATION
87
Theorem 3 .4 Given an ordering d the complexity of elim -bel is (time and space) exponential in the induced width w* (d) of the network 's ordered moral graph . 0
3.2. HANDLINGOBSERVATIONS Evidence should be handled in a special way during the processing of buck ets. Continuing with our example using elimination on order d1, suppose we wish to compute the belief in A = a having observed b = 1. This observation is relevant only when processing bucket B . When the algorithm arrives at that bucket , the bucket contains the three functions P (bla) , AD (b, a) , and AF (b, c) , as well as the observation b == 1 (see Figure 4) . The processing rule dictates computing AB(a, c) == P (b = 1Ia) AD(b == 1, a) AF(b == 1, c) . Namely , we will generate and record a two -dimensioned function . It would be more effective however , to apply the assignment b == 1 to each function in a bucket separately and then put the resulting func tions into lower buckets . In other words , we can generate P (b == 11a) and AD (b == 1, a) , each of which will be placed in the bucket of A , and AF (b = 1, c) , which will be placed in the bucket of C . By so doing , we avoid increasing the dimensionality of the recorded functions . Processing buckets containing observations in this manner automatically exploits the cutset conditioning effect (Pearl , 1988) . Therefore , the algorithm has a special rule for processing buckets with observations : the observed value is assigned to each function in the bucket , and each resulting function is moved individ ually to a lower bucket . Note that , if the bucket of B had been at the top of our ordering , as in d2, the virtue of conditioning on B could have been exploited earlier . When processing bucketB it contains P (bla) , P (dlb, a) , P (flc , b) , and b == 1 (see Figure 5a) . The special rule for processing buckets holding observations will place P (b == lla ) in bucket A, P (dlb == 1, a) in bucketD , and P (flc , b == 1) in bucketF . In subsequent processing , only one-dimensional functions will be recorded . We see that the presence of observations reduces complexity . Since the buckets of observed variables are processed in linear time , and the recorded functions do not create functions on new subsets of variables , the corresponding new arcs should not be added when computing the induced graph . Namely , earlier neighbors of observed variables should not be connected . To capture this refinement we use the notion of adjusted induced graph which is defined recursively as follows . Given an ordering and given a set of observed nodes, the adjusted induced graph is generated by processing from top to bottom , connecting the earlier neighbors of unobserved nodes only . The adjusted induced width is the width of the adjusted induced graph .
88
R. DECHTER
Theorem
3 .5 Given a belief network having n variables , algorithm elim -
bel when using ordering d and evidencee, is (time and space) exponential in the adjusted inducedwidth w* (d, e) of the network'8 orderedmoral graph. 0
3 .3 .
FOCUSING
ON
RELEVANT
SUBNETWORKS
We will now present an improvement to elim -bel whose essenceis restricting the computation to relevant portions of the belief network . Such restrictions are already available in the literature in the context of existing algorithms
(Geiger et al., 1990; Shachter, 1990) . Since summation over all values of a probability function is 1, the recorded functions of some buckets will degenerate to the constant 1. If we could recognize such cases in advance, we could avoid needless compu tation by skipping some buckets . If we use a topological ordering of the
belief network's acyclic graph (where parents precede their child nodes) , and assuming that the queried variable starts the ordering ! , we can recognize skipable buckets dynamically , during the elimination process. Proposition 3 .6 Given a belief network and a topological ordering ~Y 1, ..., Xn , algorithm elim -bel can skip a bucket if at the time of processing, the bucket contains no evidence variable , no query variable and no newly computed function
. 0
Proof : If topological ordering is used, each bucket (that does not contain the queried variable) contains initially at most one function describing its probability conditioned on all its parents . Clearly if there is no evidence, summation will yield the constant 1. 0 Example 3 .7 Consider again the belief network whose acyclic graph is given in Figure 2a and the ordering d1 = A , C , B , F , D , G , and assume we want to update the belief in variable A given evidence on F . Clearly the buckets of G and D can be skipped and processing should start with bucket F . Once the bucket of F is processed, all the rest of the buckets are not skipable. Alternatively , the relevant portion of the network can be precomputed by using a recursive marking procedure applied to the ordered moral graph .
(seealso (Zhang and Poole, 1996)). Definition 3 .8 Given an acyclic graph and a topological ordering that starts with the queried variable , and given evidence e, the marking process works as follows . An evidence node is marked, a neighbor of the query variable is marked, and then any earlier neighbor of a marked node is marked . 1otherwise , the queried variable can be moved to the top of the ordering
BUCKETELIMINATION Algorithm .
.
89
elim -bel
.
2 . Backward
: For p -(- n downto
1 , do
for all the matrices AI , A2, ..., Aj in bucketp, do . (bucket with observed variable) if Xp == xp appears in bucketp, then substitute Xp = xp in eachmatrix Ai and put eachin appropriate bucket.
. else, if bucketpis NOT skipable , t~en
Up-(- Ui=1Si - {Xp} Ap== LXp lli =l Ai. Add Apto the largest -index variable in Up' .
.
.
Figure 8.
Improved algorithm elim-bel
The marked belief subnetwork, obtained by deleting all unmarkednodes, can be processed now by elim-bel to answer the belief-updating - query. - It is eaBY to see that Theorem ponential graph .
3 .9 The complexity of algorithm elim - bel given evidence e is exin the adjusted induced width of the marked ordered moral sub -
Proof : Deleting the unm larked nodes from the belief network results in a belief subnetwork whose distribution is identical to the marginal distribu ~~ tion over the marked variables . o .
4. An Elimination
Algorithm
for mpe
In this section we focus on the task of finding the most probable explana tion . This task appears in applications such as diagnosis and abduction . For example , it can suggest the disease from which a patient suffers given data on clinical findings . Researchers have investigated various approaches to finding the mpe in a belief network . (See, e.g., (Pearl , 1988; Cooper , 1984; Peng and Reggia , 1986; Peng and Reggia, 1989)) . Recent proposals include best first -search algorithms (Shimony and Charniack , 1991) and algorithms based on linear programming (Santos, 1991) . The problem is to find XOsuch that P (xO) == maxx IIiP (xi , elxpai) where x == (Xl ' ..., xn ) and e is a set of observations . Namely , computing for a given ordering Xl , ..., Xn ,
M==llXn !axP(x)==IJlax ) Xn -l max Xnlli=lP(Xi,elXpai
(8)
This can be accomplished as before by performing the rnxirnization operation along the ordering from right to left , while migrating to the left , at
90
R. DECHTER
each step , all components that do not mention the maximizing variable . We get ,
M == m~x P (xn , e) == Jllax max IIiP (xi , elXpai) == X = Xn
X (n - l )
Xn
ill_ax IlXiEX - FnP(Xi, elXpai) . max P (Xn, elxpan)IlXiEchnP(Xi, elXpai) =
X =
Xn
-
l
Xn
ill_ax IIXiEX - FnP(Xi, elXpai) . hn(xun)
X =
Xn
-
l
where
hn(xun) == ~ ~x P(Xn, eIXpan )I1XiEChnP (Xi, elXpai ) Where Un are the variables appearing in components defined over X n. Clearly , the algebraic manipulation of the above expressions is the same as the algebraic manipulation for belief assessment where summation is replaced by maximization . Consequently , the bucket -elimination procedure elim -mpe is identical to elim -bel except for this change. Given ordering Xl , ..., X n, the conditional probability tables are partitioned as before . To process each bucket , we multiply all the bucket 's matrices , which in this case
are denoted hi , ..., hj and defined over subsets51, ..., 5j , and then eliminate the bucket 's variable by maximi .zation . The computed function in this case
is hp : Up-t R, hp = maxxpni =lhi , whereUp= UiSi - Xp. The function obtained by processing a bucket is placed in the bucket of its largest -index
variable in Up. In addition , a function x~(u) = argmaxxphp (u) , which relates an optimizing value of Xp with each tuple of Up, is recorded and placed in the bucket
of X p .
The procedure continues recursively , processing the bucket of the next variable while going from last variable to first variable . Once all buckets are processed, the mpe value can be extracted in the first bucket . When this backwards phase terminates the algorithm initiates a forwards phase to compute an mpe tuple . Forward phase: Once all the variables are processed, an mpe tuple is computed by assigning values along the ordering from X I to Xn , consult ing the information recorded in each bucket . Specifically , once the partial
assignment x == (Xl , ..., Xi- I) is selected, the value of Xi appended to this tuple is xi (X) , where XOis the function recorded in the backward phase. The algorithm is presented in Figure 9. Observed variables are handled as in elim - bel .
Example 4 .1 Consider again the belief network of Figure 2. Given the or dering d == A , C , B , F , D , G and the evidence g == 1, process variables from last to the first after partitioning the conditional probability matrices into
buckets, such that bucketa = { P (glf ), 9 = 1} , bucketD == { P (dlb, a)} ,
BUCKETELIMINATION
Figure 9.
bucketF bucket
A
Ilf
) ,
is
placed
==
{ P
( f
==
{
( a ) }
and
maxd
( dl
( dlb
matrices
..
and
place
the
function .
P
( a )
tension
of
==
.
is
buckets
,
we
in
F
,
to
determined
the ,
1 ,
bucket be
( bl
B
.
h D hc
mpe
.
( a )
==
==
the
( f
=
.
)
==
also ,
now
maxi
h F
p
P
given
( flb
tuple
( c /a )
( g
== ( f
( b , a )
DO
( b , a )
by
.
hc
we
hB
==
( f
it
( a A
going
) ,
record
place .
) ==
two
, c )
bucket ,
P
and
hD
,
and
in
,
contains
B
( b , c )
maxc
mpe
( c /a ) }
argmaxhG
eliminate
is
with
)
Record
To
( b , a )
ha ( f
next
value
along
.
( b , c ) B
{ P
computing
processed
hF
a )
get GO
by
bucket
compute ,
==
==
function
Compute
P
bucketc
bucketD
in
Finally
, c ) ,
in and
M
==
forward
.
process
which
) .
maxb
C
( a ) ,
( f
,
we
partial
can
compile
tuples
be
viewed
as
information to
a
compilation
regarding
variables
higher
in
the the
( or most
o .rdering
learning
)
probable ( see
ex
also
-
section
.2 ) .
Similarly bounded are
in
hG
) } 9
The
result of
( bla
assign
Process
function
A
backward ,
and
, .
the
eliminate
. hG
.
bucket
( a , c )
bucket
the
The
7
, c )
G
{ P
bucketF
well
resulting
To in
through
phase
putting
h B
it
maxa
as
The
==
process in
and
Algorithm elim-mpe
bucketB
To
, a ) .
( flb the
bucketc place
P
,
result
bucketc
b , a )
argmaxDP
.
the
in P
P
place
/b , c ) }
91
to
the
case
the
induced
exponentially bounded
by
of in
the
belief
updating
dimension width
, of
w
the
the
* ( d , e )
complexity recorded
of
the
of matrices
ordered
elim
- mpe
, and moral
graph
is
those .
In
92
R. DECHTER
summary : Theorem 4 .2 Algorithm elim -mpe is complete for the mpe task. Its complexity (time and space) is O (n . exp (w* (d, e) )) , where n is the number of variables and w * (d, e) is the e-adjusted induced width of the ordered moral graph . 0
5. An Elimination
Algorithm
for MAP
We next present an elimination algorithm for the map task . By its defini tion , the task is a mixture of the previous two , and thus in the algorithm some of the variables are eliminated by summation , others by maximization . Given a belief network , a subset of hypothesized variables A = { AI , ..., Ak } , and some evidence e, the problem is to find an assignment to the hypoth esized variables that maximizes their probability given the evidence. For- . mally , we wish to compute maXa;k P (x , e) = maXa;k EXk +l IIi = lP (Xi , elxpai) where x == (aI , ..., ak, Xk+ l , ..., xn) . In the algebraic manipulation of this expression , we push the maximization to the left of the summation . This means that in the elimination algorithm , the maximized variables should initiate the ordering (and therefore will be processed last ) . Algorithm elim map in Figure 10 considers only orderings in which the hypothesized vari ables start the ordering . The algorithm has a backward phase and a forward phase, but the forward phase is relative to the hypothesized variables only . Maximization and summation may be somewhat interleaved to allow more effective orderings ; however , we do not incorporate this option here. Note that the relevant graph for this task can be restricted by marking in a very similar manner to belief updating case. In this case the initial mark ing includes all the hypothesized variables , while otherwise , the marking procedure is applied recursively to the summation variables only . Theorem 5 .1 Algorithm elim -map is complete for the map task. Its complexity is O (n . exp (w* (d, e) ) , where n is the number of variables in the relevant marked graph and w* (d, e) is the e-adjusted induced width of its marked moral graph . 0 6 . An Elimination
Algorithm
for MEU
The last and somewhat more complicated task we address is that of find ing the maximum expected utility . Given a belief network , evidence e, a real-valued utility function u (x ) additively decomposable relative to func tions 11, ..., Ij defined over Q = { Ql , ..., Qj } , Qi <; X , such that u (x ) = LQjEQ fj (xQj ) ' and a subset of decision variables D = { D1 , ...Dk } that are assumed to be root nodes, the meu task is to find a set of decisions
BUCKETELIMINATION
Figure 10.
93
Algorithm elim-map
dO== (dOl, ..., dOk) that maximizes the expected utility . We assumethat the variables not appearing in D are indexed Xk+l , ..., Xn . Formally, we want to compute E ==
and
max L1,..Xn d1 ,..,dXk k+
lli =l P (Xi, elXpai' d1, ..., dk)U(X) ,
do == argmaxDE
As in the previous tasks , we will begin by identifying the computation associated with Xn from which we will extract the computation in each bucket . We denote an assignment to the decision variables by d == (d1, ..., dk) and xi == (Xk, ..., xi ) . Algebraic manipulation yields
E=m ;xXk IIi=!P(X,elXpai i 'd)Q }:J"EQ -}:n+ -ll}:Xn fj (xQj)
We can now separate the components in the utility functions into those mentioning X n, denoted by the index set tn , and those not mentioning X n, labeled with indexes in = { I , ..., n} - tn. Accordingly we get
E=max :+l)}= :n~lP(Xi ,elxpai 'd).(}= :nfj(xQj )+JEtn }= dXk .1 .:fj (xQj)) _(}= n -l Xn JE
94
R. DECHTER
E == m; x[_(nL L IIi : lP (Xi, elXpai ' d} jEln L fj (xQj} Xk+l l) Xn L L IIi=lP (Xi, e\Xpai ' d) L fj (xQj)] _(n- l) Xn jEtn Xk+l By migrating to the left of Xn all of the elements that are not a function of Xn , we get
max L-l llXiEX -Fn P(Xi,elxpai ' d).(L fj(xQj ))LXnllXiEFnP (Xi,elxpai ' d) d [Xk . l -n JE n +l (9) +-n L-l IIxiEx -FnP (xi,elXpai ' d).LXnIIxiEFnP (xi,elXpai ' d)jEtn L fj (xQj )] Xk +l
An(xunld)==L llXiEFiP (Xi, elxpai ' d) Xn
We define en over W n as
8n(xWnld ) ==L llXiEFnP (Xi, elXpai ' d) L Ij (XQj)) Xn
jEtn
1111111111111
We denote by Un the subset of variables that appear with Xn in a proba bilistic component , excluding Xn itself , and by W n the union of variables that appear in probabilistic and utility components with Xn , excluding Xn itself . We define An over Un as (x is a tuple over Un U Xn )
After substituting Eqs. (10) and (11) into Eq. (9), we get """""
"""""
en (x WnId) ]
E == m: x -Ln.., llXiEX- FnP(Xi, elxpai'd).'\n(xUnld)[~ Ij (xQj)+ An (XUn Id) - l JEln Xk + l
(12) The functions (In and An compute the effect of eliminating Xn . The result
(Eq. (12)) is an expression, which does not include Xn , where the product has one more matrix An and the utility
components have one more
element Tn== ~ . Applyingsuchalgebraic manipulation to therestof the variables in order yields the elimination algorithm elim -meu in Figure 11. We assume here that decision variables are processed last by elim -meu . Each bucket contains utility components f)i and probability components , Ai . When
there is no evidence , An is a constant
and we can incorporate
the marking modification we presented for elim -bel . Otherwise , during processing, the algorithm generates the Ai of a bucket by multiplying all its
BUCKETELIMINATION
Figure 11.
Algorithm
95
elim - meu
probability components and summing over Xi . The () of bucket Xi is computed as the average utility of the bucket ; if the bucket is marked , the average utility of the bucket is normalized by its A. The resulting () and A are placed into the appropriate buckets . The maximization over the decision variables can now be accomplished using maximization as the elimination operator . We do not include this step explicitly , since, given our simplifying assumption that all decisions are root nodes, this step is straightforward . Clearly , maximization and summation can be interleaved to some degree, thus allowing more efficient orderings . As before , the algorithm 's performance can be bounded as a function of the structure of its augmented graph. The augmented graph is the moral graph augmented with arcs connecting any two variables appearing in the same utility component Ii , for some i . Theorem 6 .1 Algorithm elim -meu computes the meu of a belief network augmented with utility components (i . e., an influence diagram ) in 0 (n .
96
R. DECHTER
Z3 Z2 ~
VI V2 U3 Xl (a)
(b)
Figure 12.
(a) A poly-tree and (b) a legal processing ordering
exp (w * (d , e) ) , where w * (d , e) is the induced width along d of the augmented moral graph . 0 Tatman and Schachter (Tatman and Shachter , 1990 ) have published an algorithm that is a variation of elim - meu , and Kjaerulff 's algorithm (Kjreaerulff , 1993 ) can be viewed as a variation of elim - meu tailored to dynamic probabilistic networks .
7. Relation of Bucket Elimination
to Other Methods
7.1. POLY-TREE ALGORITHM When the belief network is a poly -tree , both belief assessment, the mpe task and map task can be accomplished efficiently using Pearl 's poly -tree
algorithm (Pearl, 1988) . As well, when the augmentedgraph is a tree, the meu can be computed efficiently . A poly -tree is a directed acyclic graph whose underlying undirected graph has no cycles. We claim that if a bucket elimination algorithm process variables in a
topological ordering (parents precedetheir child nodes), then the algorithm coincides (with someminor modifications) with the poly-tree algorithm . We will demonstrate the main idea using bucket elimination for the m pe task . The arguments are applicable for the rest of the tasks . Example
7 .1 Consider the ordering Xl , U3, U2, VI , YI , ZI , Z2, Z3 of the poly -
tree in Figure 12a, and assumethat the last four variablesare observed(here we denote an observed value by using primed lowercase letter and leave other
variables in lowercase ) . Processingthe bucketsfrom last to first , after the first four buckets have been processed as observation buckets, we get
bucket(U3) = P (U3) , P (xllul , U2, U3) , P (ZI3Iu3) bucket(U2) = P (U2), P (z/2Iu2)
BUCKETELIMINATION
97
bucket(U1) = P(Ul), P(Z/IIU1) bucket(,X 1)- := P(y/llx1) . -. . When processing bucket(U3) by elim-mpe, we get hU3(Ul , u2, U3), which is placed in bucket(U2) . The final resulting bucketsare bucket(U3) == P (U3), P (xllul , u2, U3) , P (ZI3Iu3) bucket(U2) == P (U2), P (ZI2Iu2), hu3(Xl , U2, UI) bucket(UI) == P (UI) , P (z' lluI ), hu2(Xl , UI) bucket(XI ) == P (Y' llxI ), hUl(XI) We can now choose a value Xl that maximizes
the product
in X I 's bucket ,
then choose a value Ul that maximizes the product in VI 's bucket given the selected value of X I , and so on. I t is easy to see that if elim -mpe uses a topological ordering of the
poly-tree, it is time and spaceO(exp(IFI )), where IFI is the cardinality of the maximum family size. For instance , in Example 7.1, elim -mpe records
the intermediate function hU3(Xl , U2, Ul ) requiring O(k3) space, where k bounds
the
domain
size for
each
variable
. Note , however
, that
Pearl ' s al -
gorithm (which is also time exponential in the family size) is better , as it records functions on single variables only . In order to restrict space needs, we modify elim -mpe in two ways . First , we restrict processing to a subset of the topological orderings in which sibling nodes and their parent appear consecutively as much as possible. Second, whenever the algorithm reaches a set of consecutive buckets from the same family , all such buckets are combined and processed aE one superbucket. With this change, elim -mpe is similar to Pearl 's propagation algorithm on poly -trees .2 Processing a super-bucket amounts to eliminating all the super-bucket 's variables without recording intermediate results . Example 7 .2 Consider Example 7.1. Here, instead of processing each b1tcket OfUi separately, we compute by a br'ute-force algorithm the function hu1,U2,U3(Xl ) in the super-bucket of VI , V2, U3 and place the function in the bucket of XI We get the unary function hCI1 ,U2,(T3(XI ) == maXttl,U211t3 P (U3)P (XII 'UI, U2, 'U3)P (Z/3Iu3)P ('lt2)P (Z/2Iu2)P ('ltl ) P (Z/1Iul ) .
The details for obtaining an ordering such that all families in a poly -tree can be processed a5 super-buckets can be worked out , but are beyond the scope
of this
Proposition
paper . In summary
,
7 .3 There exist an ordering of a poly-tree, such that bucket-
elimination algorithms (elim-bel, elim-mpe, etc.) with the super-bucketmodification have the same time and space complexity as Pearl 's poly -tree algorithm for the corresponding tasks. The modified algorithm 's time complexity is exponential in the family size, and it requires only linear space. 0 2Actually , Pearl 's algorithm rooted
tree
in
order
to
be identical
should be restricted with
ours .
to message passing relative
to one
98
R. DECHTER
F
Figure 13 .
7 .2 .
@
Clique - tree associated with the induced graph of Figure 7a
JOIN-TREECLUSTERING
Join -tree clustering (Lauritzen and Spiegelhalter , 1988) and bucket elimi nation are closely related and their worst -case complexity (time and space) is essentially the same. The sizes of the cliques in tree-clustering is identical to the induced -width plus one of the corresponding ordered graph . In fact , elimination may be viewed as a directional (i .e., goal- or query -oriented ) version of join - tree clustering . The close relationship between join -tree clustering and bucket elimination can be used to attribute meaning to the intermediate functions computed by elimination . Given an elimination ordering , we can generate the ordered moral induced graph whose maximal cliques (namely , a maximal fully -connected subgraph ) can be enumerated as follows . Each variable and its earlier neighbors are a clique , and each clique is connected to a parent clique with whom it shares the largest subset of variables (Dechter and Pearl , 1989) . For example , the induced graph in Figure 7a yields the clique-tree in Figure 13, If this ordering is used by tree-clustering ,the same tree may be generated . The functions recorded by bucket elimination can be given the following meaning (details and proofs of these claims are beyond the scope of this paper ) . The function hp(u) recorded in bucketp by elim -mpe and defined over UiSi - { Xp } , is the maximum probability extension of u , to variables appearing later in the ordering and which are also mentioned in the clique subtree rooted at a clique containing Up' For instance , hF (b, c) recorded by elim -mpe using d1 (see Example 3.1) equals maxf ,g P (b, c, f , g) , since F and G appear in the clique-tree rooted at (FC B ) . For belief assessment, the function Ap = l:::xp ni =lAi , defined over Up = UiSi - Xp , denotes the probability of all the evidence e+P observed in the clique subtree rooted at a clique containing Up, conjoined with u . Namely , Ap(U) = P (e+P, u) .
BUCKETELIMINATION
99 P(d=llb,a)P(g=Olf =O) P(d=lib,s)P(g=Olf=1) P(d=lib,s)P(g=Olf=O)
...
...
...
...
P(d=Ilb,a)P(g=Olf=1) Figure 14. probability tree
8 . Combining
Elimination
and Conditioning
A serious drawback of elimination algorithms is that they require considerable memory for recording the intermediate functions . Conditioning , on the other hand , requires only linear space. By combining conditioning and elimination , we may be able to reduce the amount of memory needed yet still have performance guarantee . Conditioning can be viewed as an algorithm for processing the algebraic expressions defined for the task , from left to right . In this case, partial results cannot be assembled; rather , partial value assignments (conditioning on subset of variables ) unfold a tree of subproblems , each associated with an assignment to some variables . Say, for example , that we want to compute the expression for m pe in the network of Figure 2: M
=
max P (glf ) P ( flb , c ) P ( dla , b) P ( cla ) P ( bla ) P ( a ) a,c,b,f ,d,g
= max P ( a ) maxP a c
( cla ) maxP b
( bla ) maxP f
( flb , c ) maxP d
( dlb , a ) maxP 9
(glf ) . ( 13 )
We can compute along
the
traversed algorithms
the
ordering either such
expression
from
breadth
first - first
as best - first
by traversing variable
to
or depth - first search
the tree
last
variable
and will
and branch
in Figure .
result
and bound
The
14 , going
tree
in known .
can
be
search
100
R. DECHTER
Algorithm
elim
Input d ;
: A a
subset :
:
1 .
For
2 .
of
The
Initialize
PI
.
P
P
=
=
probable
We
C
output
max
of
{ p , PI p
and to
will
on
and
}
V C
mpe
a
== X
-
C
. Clearly
m :
x
P
responding
value
denote
by
v
== m ' i(x
m ~ x
every
the
partial
a
of
the
is
computed 0
When graph
( n
Given
. exp
on
be
the
variables ordered
P
with
conditioned
( c , v , e ) = = maXc
c , we
elimina
variables
V
and
by
, vlliP
compute
= lP
( Xi
maxv
while
, C
c an
-
~
X
,
assignment
, c , v , elXpai
P
kept
. The
the
a
( v , c , e )
)
and
a
cor
-
without variables
of
( w * ( d , eUc ordered
) +
O ICI
moral
in such
e
U
that
( n ) ) ,
U
all
variables
of
conditioned
15
.
complexity
the c )
the
Figure
space of
the
nodes
connecting
earlier
adjusted
, C
( w * ( d , cUe
where
constitute its
in
and
variables . exp
graph
c
for retaining
is
ordered .
In
this
neighbors
.
conditioning is
tuple
rest
w * ( d , e
and
variables
enumerated
presented
time the
width
observed
- mpe
the
conditioned
the
is
over
)
be
, and
c ,
generated
set
the will
algorithm
induced
conditioned
- cond
, e , CIXpai
treating
variables
both is
( Xi
computation
assignment
for
elim
the can
. ) .
- mpe
to
probability
graph
8 . 1
O
be
by
and
algorithm
c .
observations
conditioning of
assignment
basic
value
induced
is
an
- cond
combining
argmaxvIIi
maximum
evidence
plexity
as
probability
elim
conditioned
will
adjusted
Theorem of
the
particular
graph
=
This
exponentially
both
,
.
subset
algorithm .
the
bounded
ity
) (c )
probability
Given
of
of a
tuple
elimination
computing
case
variables
tuple
variables
maximum
moral
idea be
combinations
of
cUe
Algorithm
C
( Xv
using
with maximum
tuple
15 .
Let
maximizing
observed
the
c , do
maximizing
.
. We
( x , e )
, for
as
of e .
,
Therefore
by
ordering
.
- mpe the
the
task
=
elim
( update
demonstrate
the
} ; an
; observations
assignment
Figure
tion
, . . . , Pn
o .
The
Return
{ PI
variables
assignment
f ~
BN
conditioned
most
every
.
- mpe
network
C
Output
- cond
belief
the
that
a
induced
was
adjusted
cycle
- cutset
induced
,
the
space
) ) , while
width
its
width
-
com
w * ( d , cUe
relative
of
complex time
the equals
to
e
graph 1 . In
and
,
) ,
the this
BUCKETELIMINATION case
elim
- cond
- mpe
1988 ; Dechter Clearly take
ables
. There that
can
an
by
the
one
super
some
, during
We
mon
bucket
the
of
elim - mpe
possible
The
for
are
and to
belief
algorithms be
more
performance
reduced
suffer
of from
: exponential
using
which
avoiding
into
recording
Dechter
, 1996 ) .
in
aB similar
probabilis
-
to
-
observing
appeared
bucket the
both
com -
in the
past
et al . , 1997 ) .
shown
bucket the
and
, that
- tree
toward
by
the
standard
- elimination
and
usual and
difficulty exponential
Rish
, 1994 ; Dechter . We
have
portions - width
tree - clustering with in
and
that
which
the
the allows of the
complexity bound
.
algorithms dynamic
worst
constraint
, 1997 ) . Space
shown
, and
- based
induced
time
resolution
algo -
( always
highlights
relevant graph
associated
plague
the
.
clustering
the
accompanied
than
then orderings
framework
join
on
algorithms
elim - meu , for
derived
ex -
effort
if
network some and
pro -
expressing
, algorithms
example
proposed and
of
conscience
elim - map
also
conditioning
framework
explicitly
the
dynamic
way
without
, 1988 ) for
procedures
refined
this
- connected
to
not
were
uniform
, for
( Pearl
of
generalizes
and
. In
- elimination
space
deficiencies ( Dechter
also
network
applies
- assessment
are
performance
elimination
buckets
algorithms
, which
a singly
elegance
which
gramming
given
bucket
bounds
to
have
were
enhancements
The
decides
method
and
viewed
( Bistarelli
of the
same
. Such
likely
, thus
frameworks
a concise
algorithms
common
be
had
in
' s algorithms
network
is
by
elim -
method
and
consecutive
many
can
reasoning
) . The
simplicity
paper
framework
. We
elim - bel
trees
focusing
of
vari and
- mpe . One
a bucket
; EI - Fattah
, unifying
presented
to Pearl
on
conditioned
Conclusion
properties
tree - propagation
features
,
effectively
recorded
conditioning
that
recently
designer
and
reduces
the
, 1996 ) ) . Another a set
algorithms
probabilistic
the
more
elim - cond
to process
, 1996b
this
- elimination
topological
part
rithm
more
have
for
the
in
Rish
by
( Dechter
various
and
, we
algorithms
( Pearl
conditioning
functions
, collects
processes
to
between
of
and
. In addition
between
the
assignments
arity
reaBoning
, 1992 ) and
Using
be implemented
hybrids
, whether
throughout
Summary
ploit
it
algorithms
gramming
algorithms
k
mentioned
features
10 .
loop - cutset
procedure
( Dechter
results
deterministic
( Shenoy
can
the
approach
that
war
elimination
- mpe
basic
processing
- bucket
- bucket
had and
known
partial
on
( see
super
Related
the
bound
intermediate
9 .
the
of possible
refine
upper
conditioning
uses
tic
of shared
is a variety
dynamically or
elim - cond
advantage
imposes
to
, 1990 ) .
, algorithm
if we
ination
reduces
101
pro -
case . Such - satisfaction
complexity
conditioning
can can
be
102
R. DECHTER
implemented
naturally
quirement
ing
and
still
Finally
for
,
be
.
no
attempt
These
was
.
ditional
In
can
made
and
particular
reducing
.
the
space
Combining
the
,
1996
re
-
condition
virtues
paper
exploit
of
be
addressed
Poole
to
optimize
-
forward
as
,
and
)
the
the
be
run
-
bucket
-
structure
recently
can
algorithms
vs
within
presented
1997
the
compilation
exploiting
,
;
this
to
improvements
matrices
Boutilier
elimination
thus
combining
in
nor
should
,
probability
;
a5
,
issues
framework
,
features
viewed
computation
sources
elimination
.
distributed
press
of
topological
can
search
top
exploiting
elimination
backward
time
of
in
(
incorporated
re
the
Santos
con
et
on
-
elimination
top
al
of
.
-
,
in
bucket
-
.
In
summary
eral
,
tasks
,
which
what
we
applicable
the
importantly
buckets
,
Dechter
11
,
.
A
This
as
have
1997
)
exposition
bucket
done
,
-
by
via
approximation
areas
of
associated
-
,
research
with
algorithms
combining
sev
reasoning
several
elimination
either
or
across
deterministic
between
benefit
the
be
uniform
and
ideas
organizational
shown
a
.
the
to
use
be
conditioning
with
algorithms
of
improved
as
elimi
is
-
shown
in
.
Acknowledgment
preliminary
like
version
to
thank
of
Irina
different
grant
IRI
F49620
-
-
of
9157636
96
-
America
-
0224
,
and
paper
and
this
,
1
this
Rish
versions
of
is
probabilistic
of
all
can
we
both
the
allow
.
nation
here
transfer
,
should
uniformly
provide
to
facilitates
More
(
on
while
paper
Air
appeared
Nir
.
Force
This
work
Electrical
,
1996a
.
I
on
by
Research
ACM
-
grant
20775
and
Institute
would
comments
supported
Scientific
Research
)
useful
partially
grants
Power
Dechter
their
was
of
MICRO
(
for
Office
Rockwell
in
Freidman
AFOSR
95
grant
NSF
-
043
,
RP8014
-
Amada
06
.
References
S
.
Arnborg
and
to
S
. A
.
A
partial
k
Arnborg
-
.
.
Proskourowski
trees
.
of
A
.
S
.
.
and
.
D
In
.
Bertele
and
,
Boutilier
Artificial
van
.
Run
.
.
BIT
.
A
.
.
.
Brioschi
N
,
Journal
of
95
UAI
-
96
onserial
A
,
1985
Rossi
Cassis
ssociation
-
11
)
,
pages
France
-
csps
,
for
81
89
.
of
problems
24
-
In
1995
,
restricted
1989
.
graphs
with
bounded
finding
,
and
close
1996
to
optimal
Academic
jnmction
Press
constraint
Computing
Practice
.
.
based
Principles
.
Programming
Semiring
hard
-
on
in
,
algorithm
.
:
.
ordering
,
Dynamic
F
np
23
problems
23
)
fast
(
and
the
-
variable
AI
.
Montanari
: 2
-
sufficiently
in
F
U
25
Dynamic
CP
for
,
combinatorial
,
(
Geiger
algorithms
Mathematics
for
survey
Uncertainty
timization
C
.
a
time
Applied
Programming
Bistarelli
1997
P
Constraints
Becker
trees
U
and
Linear
and
algorithms
-
Bacchus
Discrete
Efficient
decomposability
F
.
.
,
satisfaction
Machinery
(
J
A
C
M
)
,
to
1972
.
and
op
-
appear
,
.
.
Context
Intelligence
-
specific
independence
(
UAI
-
96
)
in
,
pages
115
-
bayesian
123
,
networks
1996
.
.
In
Uncertainty
in
BUCKETELIMINATION
103
C. Cannings, E.A . Thompson, and H .H. Skolnick. Probability functions on complex pedigrees. Advances in Applied Probability , 10:26- 61, 1978. G.F . Cooper. Nestor: A computer-based medical diagnosis aid that integrates causal and probabilistic knowledge. Technical report , Computer Science department , Stanford University , Palo-Alto , California , 1984. M . Davis and H . Putnam . A computing procedure for quantification theory. Journal of the Association of Computing Machinery , 7(3) , 1960. R. Dechter and J. Pearl. Network-based heuristics for constraint satisfaction problems. A rtificial Intelligence , 34:1- 38, 1987. R. Dechter and J. Pearl. Tree clustering for constraint networks. Artificial Intelligence , pages
353
R . Dechter In
- 366
and
Principles
1994
.
of
.
Directional
resolution
I ( nowledge
: The
Representation
davis
and
- putnam
procedure
Reasoning
, revisited
( I ( R - 94 ) , pages
.
134 - 145
,
.
R . Dechter
and
I . Rish
Constraint
. To
Practice R . Dechter
and
P . van
of
Constraint
Beek
.
to
think
- 96 ) , 1996
Local
and
. Constraint
for
Artificial .
for
sat
consistency
- 95 ) , pages
constraint
240 - 257
processing
Intelligence
networks
algorithms
relational
( CP
schemes .
? hybrid
.
In
Principles
of
.
global
programming
decomposition
R . Dechter
or
( CP
. Enhancement
cutset
1992
guess
Programming
R . Dechter
, 41 : 273
Encyclopedia
of
. , 1995
In
Principles
: Backjumping
- 312
, 1990
Artificial
and
. , learning
and
.
Intelligence
, pages
276
- 285
,
algo
-
.
R . Dechter
.
rithms
.
R . Dechter
Bucket In
A rtificial
Ijcai
: A
reasoning
and :
R . Dechter
results
on
- 96 ) , pages
D . Geiger
, 20 : 507
F . V . Jensen
, S .L
networks U . Kjreaerulff
by . A
structures
An
J . Pearl
, 1990
of
the
inference
tradeoffs
. In
211
- 219
Uncertainty
, 1996 in
.
Artificial
. generating
Fifteenth
evaluation
of
circuits
, 1996
approximations
in
International
Joint
automated
Conference
on
.
structural In
parameters
Uncertainty
in
for
probabilistic
Artificial
Intelligence
. .
Identifying
independence
in
bayesian
networks
.
Net
-
. , and
computation
computational in
of
probabilistic
- 96 ) , pages
. .
Lauritzen local
Uncertainty
S . L . Lauritzen
, and
- 534
for ( UAI
- space
, 1996
scheme
benchmark
244 - 251
, T . Verma
works
time
220 - 227
general
, 1997
framework
Intelligence
- 97 : Proceedings
Intelligence
Y . El - Fattah
unifying
for
- 96 ) , pages
- buckets
. In
A
Artificial
parameters
( UAI . Mini
reasoning
: in
. Topological
R . Dechter
( UAI
elimination
Uncertainty
Intelligence
In
, 1989
I . Rish
Artificial
K . G . Olesen .
scheme
for
Intelligence
Bayesian
and
D . J . Spiegelhalter
. Local
their
to
expert
updating
Statistics
reasoning ( UAI
and
application
.
Computational
in
dynamic
- 93 ) , pages
computation systems
with .
in
causal
Quarterly
Journal
probabilistic
, 4 , 1990
probabilistic
121 - 149
, 1993
the
.
.
probabilities of
. networks
on Royal
graphical Statistical
Society , Series B , 50( 2) :157- 224, 1988. J . Pearl . Probabilistic Reasoning in Intelligent Systems . Morgan Kaufmann , 1988. Y . Peng and l .A . Reggia . Plausability of diagnostic hypothesis . In National Conference on Artificial Intelligence (AAAI86 ) , pages 140- 145, 1986. Y . Peng and l .A . Reggia . A connectionist model for diagnostic problem solving , 1989. D . Poole . Probabilistic partial evaluation : Exploiting structure in probabilistic inference . In Ijcai - 97 : Proceedings of the Fifteenth International Joint Conference on Artificial Intelligence , 1997. S.K . Anderson R . D . Shachter and P. Solovitz . Global conditioning for probabilistic inference in belief networks . In Uncertainty in Artificial Intelligence ( UAI - 91) , pages 514- 522 , 1991. B . D 'Ambrosio R .D . Shachter and B .A . Del Favro . Symbolic probabilistic inference in belief networks . A utomated Reasoning , pages 126- 131, 1990. E . Santos , S.E . Shimony , and E . Williams . Hybrid algorithms for approximate belief updating in bayes nets . International Journal of Approximate Reasoning , in press .
104
R. DECHTER
E. Santos. On the generation of alternative explanations with implications for belief revision. In Uncertainty in Artificial Intelligence ( UAI -91) , pages 339- 347, 1991. R.D . Shachter. Evaluating influence diagrams. Operations Research, 34, 1986. R.D . Shachter. Probabilistic inference and influence diagrams. Operations Research, 36, 1988. R. D . Shachter. An ordered examination of influence diagrams. Networks, 20:535- 563, 1990. P.P. Shenoy. Valuation -based systems for bayesian decision analysis. Operations Research, 40:463- 484, 1992. S.E. Shimony and E. Chamiack . A new algorithm for finding map assignments to belief networks. In P. Bonissone, M. Henrion , L . Kanal , and J. Lemmer ed., Uncertainty in Artificial Intelligence , volume 6, pages 185- 193, 1991. J.A . Tatman and R.D . Shachter. Dynamic programming and influence diagrams. IEEE Transactions on Systems, Man, and Cybernetics, 1990. N .L . Zhang and D. Poole. Exploiting causal independencein bayesian network inference. Journal of Artificial Intelligence Research(JAIR ) , 1996.
AN INTRODUCTION TO V.t\.RIATION AL METHODS FOR GRAPHICAL MODELS
MICHAEL
I . JORDAN
Massachusetts Institute of Technology Cambridge , MA ZO UBIN
G HAHRAMANI
University of Toronto Toronto
, Ontario
TOMMI
S. JAAKKOLA
University Santa
of California
Cruz , CA
AND LAWRENCE
K . SAUL
AT & T Labs
- Research
Florham
Park
, NJ
Abstract . This paper presents a tutorial introduction to the use of varia tional methods for inference and learning in graphical models . We present a number of examples of graphical models , including the QMR -DT database , the sigmoid belief network , the Boltzmann machine , and several variants of hidden Markov models , in which it is infeasible to run exact inference algorithms . We then introduce variational methods , showing how upper and lower bounds can be found for local probabilities , and discussing methods for extending these bounds to bounds on global probabilities of interest . Finally we return to the examples and demonstrate how variational algorithms can be formulated in each case.
105
MICHAELI. JORDAN ET AL.
106 1. The
Introduction problem
of probabilistic
computing the nodes nodes
( the
( the
probability
" hidden
" evidence
set of hidden wish
inference
a conditional
to calculate
" or " unobserved
" or " observed
nodes
and
letting
P ( HIE
General
exact
inference
( Cowell
tematic
We often
volume
of the
as inferred
of the parameters quantity
evaluation
of the likelihood
tor
and
maximize
complexity have recourse tion
tree
of P ( HIE are many
of which
we discuss
in this
calculation
to approximation
construction
clique
architectural
Even
in particular
the exact
resentation
of the joint
model
another
; put
the
of distributions implied
or clusters
of nodes
algorithms
probability
particular
pendencies
to consider
probability that
exact
produce
provide
, there
are other
the time
and
or space
it is necessary
the context
complexity
of the junc -
of the exact
is man -
procedures
associated
with
" nearly " conditionally
consideration the
rep -
a graphical
have the same complexity
is consistent
. Note
numerical with
regard within
conditional
be situations independent
are
cliques .
algorithms
no use of the
in
see , there
lead to large
under
to
is exponential
approximation
may
, al make
algorithms
problems
distribution
by the graph . There are
the
necessarily
make
-
) . Moreover
tree . As we will
distribution
way , the algorithms
less of the family
that
).
.
. Within time
of P ( HIE
) generally
paper , in which
the complexity
can be reason that
, the
as
the numera
generally
quantities
is unacceptable
in the junction
assumptions
in cases in which
ageable , there
in fact
learning
procedures
, for example
the size of the maximal natural
cases in which
mod -
E , P ( E ) is an
compute
of P ( HIE
) as a subroutine and
exact
simply
, they
joint
by Eq . ( 1) , the
to the calculation
( and related
to inference
in the
in graphical
model , for fixed
divide
sys -
, P ( E ) . Viewed
. As is suggested
do not
this
take
edges in the graph .
evidence
of the calculation
likelihood
to perform
algorithms
probabilities
related
solution of the
the
nodes , we
present
of missing
marginal
algorithms
gorithms
cases , several
of other
H represent
developed
of the observed
of Eq . ( 1 ) and
there
the values
( 1)
independencies
is closely
inference
use of the calculation
of
of some of
set of evidence
been
as the likelihood
as a by - product
Although
have
of the graphical
known
denominator
a satisfactory
.
the pattern
the likelihood that
) = % r2
to calculate
important
, although
the
; Jensen , 1996 ) ; these
the probability
a function
Indeed
" nodes ) , given
E represent
conditional
from
also wish
els , in particular
is the problem
" nodes ) . Thus , letting
algorithms
, this
advantage
distribution
models
over the values
):
P ( HIE
calculation
in graphical distribution
in which
inde nodes
, situations
in
AN INTRODUCTION TO VARIATIONAL METHODS
107
which node probabilities are well determined by a subset of the neighbors of the node , or situations in which small subsets of configurations of variables contain most of the probability mass. In such cases the exactitude achieved by an exact algorithm may not be worth the computational cost . A variety of approximation procedures have been developed that attempt to identify and exploit such situations . Examples include the pruning algorithms of Kjrerulff (1994) , the "bounded conditioning " method of Horvitz , Suermondt , and Cooper (1989) , search-based methods (e.g., Henrion , 1991), and the "localized partial evaluation " method of Draper and Hanks (1994) . A virtue of all of these methods is that they are closely tied to the exact meth ods and thus are able to take full advantage of conditional independencies . This virtue can also be a vice , however, given the exponential growth in complexity of the exact algorithms . A
related
approach
cations
of graphical
MacKay
, &
as an iterative
graphical
Another
approach
Carlo
for
design
and
in
in
appli -
( McEliece
Pearl ' s algorithm
inference
, for
in non - singly - connected
( see MacKay
problem
Kjrerulff
can be slow this
ational
of the
to converge chapter
approach
generally
provide
underlying averaging
It
yields bounds
exploit solution
to diagnose
can come
with into
of values
phenomena
their
yet
. an -
. Vari -
procedures basic
that
intuition
can be probabilis
connectivity
play , rendering
can lead
convergence
provide
. The
there
nodes
neighbors
relatively
. Taking
to simple , accurate
are
advan approxi
. to emphasize
outlined
complementary to any given
are
features problem
that
the
by no means
various
approaches
mutually
exclusive
of the graphical may well
involve
model
to inference ; indeed
formalism
an algorithm
.
algorithms
algorithms
graphs
of their
the
, which
dense
their
of convergence
inference
complex
, in graphs
include
are that
of interest
is that
settings
guarantees
approximation
on probabilities
that
& Luby , 1993 ; Fung
algorithms
methods
deterministic
methods
averaging
is important we have
design
; in particular
procedures
it can be hard
involves algorithms
, 1994 ; Jensen , Kong , &
approach
of approximate
to particular
of these
and
theoretical Carlo
Carlo
, and Neal , 1993 ) and applied ( Dagum
of these
variational
phenomena
insensitive
and Monte
algorithms
of Monte
, & Spiegelhalter
the
variational
simple
mation
models
we discuss
to
methodology
tically
, this volume
in graphical
of implementation
disadvantages
In
. A variety
, 1995 ; Pearl , 1988 ) . Advantages
simplicity
other
of approximation
methods
Favero , 1994 ; Gilks , Thomas
that
, Kim
arisen decoding
( Pearl , 1988 ) has been used successfully
method
to the
use of Monte
to the inference
tage
has
to error - control
particular
models
approximate
have been developed
The
inference
inference
.
making
&
approximate
Cheng , 1996 ) . In
singly - connected graphs
to
model
that
they
. The best combines
-
108
MICHAELI. JORDANET AL.
aspects of the different methods . In this vein , we will present variational methods in a way that emphasizes their links to exact methods . Indeed , as we will see, exact methods often appear as subroutines within an overall variational approximation (cf . Jaakkola & Jordan , 1996; Saul & Jordan , 1996) . It should be acknowledged at the outset that there is as much "art " as there is "science" in our current understanding of how variational meth ods can be applied to probabilistic inference . Variational transformations form a large , open-ended class of approximations , and although there is a general mathematical picture of how these transformations can be exploited to yield bounds on probabilities in graphical models , there is not as yet a systematic algebra that allows particular variational transformations to be matched optimally to particular graphical models . We will provide illustrative examples of general families of graphical models to which varia tional methods have been applied successfully, and we will provide a general mathematical framework which encompasses all of these particular examples, but we are not as yet able to provide assurance that the framework will transfer easily to other examples . We begin in Section 2 with a brief overview of exact inference in graph ical models , basing the discussion on the junction tree algorithm . Section 3 presents several examples of graphical models , both to provide motivation for variational methodology and to provide examples that we return to and develop in detail as we proceed through the chapter . The core material on variational approximation is presented in Section 4. Sections 5 and 6 fill in some of the details , focusing on sequential methods and block methods , respectively . In these latter two sections, we also return to the examples and work out variational approximations in each case. Finally , Section 7 presents conclusions and directions for future research.
2. Exact inference In this section we provide a brief overview of exact inference for graphical models , as represented by the junction tree algorithm (for relationships between the junction tree algorithm and other exact inference algorithms , see Shachter , Andersen , and Szolovits , 1994; see also Dechter , this volume , and Shenoy, 1992, for recent developments in exact inference ) . Our intention here is not to provide a complete description of the junction tree algorithm , but rather to introduce the "moralization " and "triangulation " steps of the algorithm . An understanding of these steps, which create data structures that determine the run time of the inference algorithm , will suffice for our
AN INTRODUCTION TO VARIATIONALMETHODS
109
Figure 1. A directed graph is parameterized by associating a local conditional probability with each node. The joint probability is the product of the local probabilities .
purposes.l For a comprehensiveintroduction to the junction tree algorithm see Cowell (this volume) and Jensen (1996). Graphical models come in two basic flavors- directed graphical models and undirected graphical models. A directed graphical model is specified numerically by associating local conditional probabilities with each of the nodes in an acyclic directed graph. These conditional probabilities specify the probability of node Si given the values of its parents, i .e., P (8iI87r(i)), where 1[ (i ) representsthe set of indices of the parents of node 8i and 87r(i) representsthe correspondingset of parent nodes (seeFig. 1).2 To obtain the joint probability distribution for all of the N nodesin the graph, i .e., P (8 ) == P (8l , 82, . . . , SN), we take the product over the local node probabilities :
N P(S) = n P(SiIS7r (i)) i=l Inference involves the calculation of conditional
(2) probabilities
under this
joint distribution . An undirected graphical model (also known as a "Markov random field " ) is specified numerically by associating "potentials " with the cliques of the graph .3 A potential is a function on the set of configurations of a clique lOur presentation will take the point of view that moralization and triangulation , when combined with a local message-passing algorithm , are sufficient for exact inference . It is also possible to show that , under certain conditions , these steps are necessary for exact inference . See Jensen and Jensen ( 1994) . 2Here and elsewhere we identify the ith node with the random variable Si associated with the node . 3We define a clique to be a subset of nodes which are fully connected and maximal ; i .e., no additional node can be added to the subset so that the subset remains fully connected .
110
MICHAELI. JORDANET AL.
S4 (/
- .' ,
I
\
S
51(". -- - '
3
,= 3(C3 ) .~) S6 -
/
"
S2 (
-
-
,
--'.'
.--Ss -)
Figure 2. An undirected graph is parameterized by associating a potential with each clique in the graph. The cliques in this example are C1 = { 81, 82, 83} , C2 = { 83, 84, 85} , and C3 = { 84, 85, 86} . A potential assigns a positive real number to each configuration of the corresponding clique. The joint probability is the normalized product of the clique potentials .
(that is, a setting of values for all of the nodes in the clique ) that Msociates a positive real number with each configuration . Thus , for every subset of nodes Ci that forms a clique , we have an associated potential
(3) where M is the total number of cliques and where the normalization factor Z is obtained by summing the numerator over all configurations :
Z= L II c/Ji(Ci)} . {5} { i=l
(4)
In keeping with statistical mechanical terminology we will refer to this sum as a "partition function ." The junction tree algorithm compiles directed graphical models into undirected graphical models ; subsequent inferential calculation is carried out in the undirected formalism . The step that converts the directed graph into an undirected graph is called "moralization ." (If the initial graph is already undirected , then we simply skip the moralization step). To under stand moralization , we note that in both the directed and the undirected cases, the joint probability distribution is obtained as a product of local
AN INTRODUCTION
TO VARIATIONAL
A
DAD
B
C
METHODS
B
(a)
111
C
(b)
Figure 3. (a) The simplest non-triangulated graph. The graph has a 4-cycle without a chord. (b) Adding a chord between nodes B and D renders the graph triangulated .
functions
. In
the
directed
case , these
functions
are the
node
conditional
probabilities P (Si IS7r (i)). In fact, this probability nearly qualifies as a potential function ; it is certainly a real-valued function on the configurations
of the set of variables { Si, S7r(i)} . The problem is that these variables do not always appear together within a clique . That is, the parents of a common child are not necessarily linked . To be able to utilize node conditional probabilities as potential functions , we "marry " the parents of all of the nodes with undirected edges. Moreover we drop the arrows on the other edges in the graph . The result is a "moral graph ," which can be used to represent the probability distribution on the original directed graph within the
undirected
formalism
.4
The second phase of the junction tree algorithm is somewhat more complex . This phase, known as "triangulation ," takes a moral graph as input and produces as output an undirected graph in which additional edges have (possibly ) been added . This latter graph has a special property that allows recursive calculation of probabilities to take place . In particular , in a triangulated graph , it is possible to build up a joint distribution by proceeding sequentially through the graph , conditioning blocks of interconnected nodes only on predecessor blocks in the sequence. The simplest graph in which this is not possible is the "4-cycle ," the cycle of four nodes shown in Fig . 3(a). If we try to write the joint probability sequentially as, for
example, P (A)P (BIA )P (CIB )P (DIC ), we seethat we have a problem. In particular , A depends on D , and we are unable to write the joint probability as a sequence
of conditionals
.
A graph is not triangulated if there are 4-cycles which do not have a chord, where a chord is an edge between non-neighboring nodes. Thus the 4Note in particular
that Fig . 2 is the moralization
of Fig . 1.
112
MICHAELI. JORDAN ET AL.
graph
in
chord
as
Fig
.
in
sequentially More
in
A
any
,
in cliques
The
the
local
data
In models
models
.
all
graph need
If
a
that
to
junction
node
lie
appears
on
based
cliques in
the
path that
on
achieving
assign
common
,
a
consequence be
the
possible
the
) . ,
local
In
a
same
junction .,
consistency
implies
will by
cliques
,
will
able
.
will
either to
,
of
be
achieve
of
in
per
particular
the
For
for
potential
efficient
is
inference
specific exact
able
lower
,
it
for Kjrerulff
for
display the
in
the
the
size moral
1990
these
" obvious
of
"
cliques
graph
triangulation ,
graphical
inference
to
bound
cliques
e .g . ,
;
investigate
algorithms see
tree to
complexity
represent
clique
junction as
.
we
the
,
cliques
costs
specific
algorithms
time
the to
the
the so
The
of
required in
considering
consider
.
size
paper
we be
on potentials
cliques the
computational
cases
we
performed clique
.
( for
in Thus
a we
discussion
) .
Examples
In
this
section
inference
is
system
in
remaining is
to
these
or
triangulation
.
,
is as
:
cliques
have
are
small
this
the
of
triangulated not
is
the
nodes
obtain
of
,
will
) . it
important
property
values
of
to
triangulation
of
, D
a
probability
known
can
they
on
of
consider
In
all the
( That
that
number
remainder and
. that
neighboring
number
the
critical
the
in has
nodes
( CIB
structure
inference
depends
the in
therefore
) P
adding
joint
property
appears
rescaling
between
discrete
it
by the
triangulated
data
intersection
and
calculation
exponential
, DIA
been a
triangulated write
intersection
calculations
consistency this
( B
has
cliques
running
marginalizing
forming
) P
property
the
be can
.
probabilistic
involve
,
This
can we
probabilistic
to
of
( A
into
tree .
for
consistency
P
running
between
because
=
graph
the
it
graph
graph
the
probability
,
)
a the
has
two
;
latter
, D
once
algorithm
bal
the , C
of
consistency
tree
triangulated
In
, B
cliques the
marginal
3
( A
tree
two
local
glo
not
) .
cliques
general
is
P
junction
between a
is
3 ( b
generally the
.
.
as
arrange tree
3 ( a )
Fig
fit
we
generally which
data
and
examples
infeasible a
examples to
present
fixed involve
subsequently
of
.
graphical
Our
graphical
first model
estimation used
example is
problems for
models
prediction
in
which
involves
used
to in
which or
a
answer
diagnostic
queries a
diagnosis
exact
graphical
.
The
model .
3.1. THE QMR-DT DATABASE The QMR-DT database is a large-scale probabilistic database that is intended to be used as a diagnostic aid in the domain of internal medicine.5
5The acronym "refers tothe "Decision Theoretic "version ofthe "Quick Medical Reference ."QMR " -DT
113
AN INTRODUCTION TO VARIATIONALMETHODS
diseases
symptoms Figure
. 4
.
The
evidence
structure
nodes
We
provide
see
The
per
-
of
(
Fig
.
4
,
use
of
, 6
the
.
a
.
The
the
a
of
"
bipartite
i
f
d
)
=
probabilities
diseases
OR
of
archival
data
,
"
model
P
{
fild
.
.
)
That
,
The
and
obtained
,
the
;
represent
di
the
for
further
which
the
up
-
represent
disease
nodes
and
are
binary
P
(
[
fid
Il
P
by
diseases
(
d
)
P
expert
nodes
.
bipartite
are
Making
form
nodes
findings
{
dj
]
~
,
of
we
the
obtain
:
)
,
P
(
were
probabilities
probability
with
All
variables
the
to
)
fild
,
refer
findings
.
symptom
P
(
of
diseases
random
and
)
we
vector
of
unobserved
from
conditional
represent
nodes
henceforth
the
vector
diseases
conditional
were
is
the
;
in
of
600
the
=
prior
and
over
,
model
layer
symptoms
over
(
nodes
here
lower
implied
probability
shaded
database
approximately
"
marginalizing
joint
DT
the
denotes
f
The
.
findings
d
.
graphical
and
observed
components
P
from
-
independencies
following
The
QMR
are
symbol
conditional
and
the
database
set
model
. "
.
There
the
as
thus
the
graph
)
in
symptoms
f
)
graphical
findings
diseases
is
symbol
DT
"
is
nodes
observed
1991
database
evidence
binary
(
represent
see
-
~
of
ale
DT
symptom
The
the
et
nodes
symptoms
QMR
to
overview
,
QMR
the
referred
brief
Shwe
layer
4000
are
a
details
of
and
dj
)
.
obtained
of
by
the
Shwe
findings
assessments
the
ith
a
symptom
5
)
(
6
)
et
given
under
that
,
(
ale
the
"
noisy
-
is
6In particular , the pattern of missing edgesin the graph implies that (a) the diseases are marginally independent, and (b) given the dise~ es, the symptoms are conditionally independent .
114
MICHAEL
I . JORDAN
ETAL.
absent, P (fi = Old), is expressedas follows:
P(fi = Old ) = (1- qiO ) II (1 - qij)dj
(7)
jE1r (i )
where the qij are parameters
obtained
from the expert
Msessments . Con -
sidering casesin which only one diseMe is present, that is, { dj = I } and { dk = 0; k ~ j } , we see that qij can be interpreted M the probability that the ith finding is present if only the jth disease is present . Considering the case in which
all diseases
are absent , we see that
the qiO parameter
can be
interpreted as the probability that the ith finding is present even though no disease
is present
.
We will find it useful to rewrite the noisy -OR model in an exponential
form :
P (fi = Old) = e- EjE1r (i) Bijdj- Bio
(8)
where (}ij := - In ( 1 - qij ) are the transformed parameters . Note also that the probability of a positive finding is given as follows :
P (fi = lid ) = I - e- EjE7r(i) Oijdj- lJio
(9)
These forms express the noisy - OR model as a generalized
If we now form the joint probability
distribution
linear model .
by taking products of
the local probabilities P (fild ) as in Eq. (6), we seethat negative findings are benign with respect to the inference problem . In particular ~ . a nroduct -
of exponential factors that are linear in the diseases(cf. Eq. (8)) yields a joint probability that is also the exponential of an expression linear in the diseases. That is, each negative finding can be incorporated into the joint probability in a linear number of operations . Products of the probabilities of positive findings , on the other hand , yield cross products terms that are problematic for exact inference . These cross product terms couple the diseases (they are responsible for the "explaining away" phenomena that arise for the noisy -OR model ; see Pearl , 1988) . Unfortunately , these coupling terms can lead to an exponential growth in inferential complexity . Considering a set of standard diagnos-
tic cases (the "CPC cases" ; see Shwe, et al. 1991), Jaakkola and Jordan (1997c) found that the median size of the maximal clique of the moralized QMR -DT graph is 151.5 nodes. Thus even without considering the trian gulation step , we see that diagnostic calculation under the QMR - DT model is generally
infeasible .7
7Jaakkola and Jordan (1997c) also calculated the median of the pairwise cutset size. This
value was found
for the QMR - DT .
to be 106 .5 , which
also rules out exact
cutset
methods
for inference
115
ANINTRODUCTION TOVARIATIONAL METHODS Input
Hidden Output Figure
5
output
nodes
3
. 2
.
.
The
NEURAL
Neural
graphical the
networks
are
each
logistic
j
( z
model
interpreting
the
,
)
=
Fig
/
(
.
5
)
.
e
-
of
one
the
of
us
one
Z
)
.
,
We
The
input
nodes
nonlinear
consider
those
and
treat
such
a
Si
.
from
neural
example
as
node
the
,
the
network
each
that
For
"
functions
with
probability
values
activation
obtained
variable
the
"
activation
as
can
as
two
a
such
binary
node
its
with
Let
and
+
.
MODELS
a
activation
write
1
network
endowed
associating
takes
we
1
neural .
GRAPHICAL
zero
by
variable
function
( see
between
a
nodes
graphs
node
function
graphical
of
evidence
layered
bounded
binary
of
AS
are
at
that
structure
set
NETWORKS
function
a
layered comprise
and
associated
using
the
logistic
:
1
P
where
j
( ) ij
and
is
"
i
,
=
a
is
neural
on
,
way
,
=
1
+
"
parameter
~
e
"
bias
"
to
requires
that
6
'
(
parent
with
(
1992
include
,
and
.
problem
)
.
The
i
.
This
advantages
to
treat
perform
unsupervised
Realizing
be
nodes
node
ability
to
learning
inference
)
the
data
10
10
between
Neal
manner
supervised
the
- J
edges
missing
as
- ' S 1. J
associated
by
this
handle
footing
( i )
with
introduced
in
,
same
6
L . . . jE7r
these
solved
in
bene
an
-
efficient
.
In
ered
fact
,
it
neural
parents
is
all
of
these
ticular
,
nodes
,
cally
during
are
of
of
are
inference
a
do
N
in
of
exact
hidden
least
in
the
0
( 2N
the
a
)
general
( see
is
nodes
hidden
,
the
-
are
.
in
par
additional
)
.
-
-
layers
layer
6
evidence
probabilisti
hidden
particular
Fig
clear
become
preceding
-
as
neural
layer
output
ignoring
lay
has
moralized
this
layer
in
in
the
in
general
generally
Thus
penultimate
units
at
.
nodes
network
in
network
inference
the
infeasible
neural
the
ancestors
is
is
a
layer
all
neural
their
in
preceding
for
units
inference
node
the
necessary
as
there
exact
A
between
hidden
,
if
in
links
training
the
that
.
nodes
links
thus
see
models
has
dependent
Thus
to
the
graph
That
easy
network
network
complexity
the
network
the
however
)
network
calculations
learning
( i )
associated
( ) iO
belief
treating
1IS7r
parameters
and
sigmoid
diagnostic
fits
the
node
the
of
are
( Si
.
,
the
time
growth
116
MICHAELI. JORDAN ET AL.
Hidden
OD J
j
. /
..;". /
..
Output Figure 6. Moralization of a neural network . The output nodes are evidence nodes during training . This creates probabilistic dependencies between the hidden nodes which are captured by the edges added by the moralization .
Figure 7. A Boltzmann machine . An edge between nodes Si and Sj is associated with a factor exp ( (Jij Si Sj ) that contributes multiplicatively to the potential of one of the cliques containing the edge. Each node also contributes a factor exp ((}iOSi) to one and only one potential .
in clique
size due to triangulation
or even hundreds of hidden neural network using exact
. Given
that
neural
networks
with
units are commonplace , we see that inference is not generally feasible .
dozens
training
a
3.3. BOLTZMANNMACHINES A Boltzmann nodes
and
ular , the
machine
is an undirected
a restricted clique
potentials
graphical
set of potential
functions
are formed
by taking
model
with
binary
( see Fig . 7 ) . products
In
- valued partic
-
of " Boltzmann
factors " - exponentials of terms that are at most quadratic in the Si ( Hin ton & Sejnowski , 1986 ) . Thus each clique potential is a product of factors exp { 8ijSiSj
} and factors
exp { (JiOSi } , where
Si E { O, 1} .8
8It is also possible to consider more general Boltzmann machines with multivalued nodes , and potentials that are exponentials of arbitrary functions on the cliques . Such models are essentially equivalent to the general undirected graphical model of Eq . (3)
ANINTRODUCTION TOVARIATIONAL METHODS
117
A given pair of nodes 8i and 8j can appear in multiple , overlapping cliques. For each such pair we ~ sume that the expressionexp{ OijSiSj} appears as a factor in one and only one clique potential . Similarly , the factors
exp{(}io8i } are ~ sumed to appear in one and only one clique potential . Taking the product over all such clique potentials (cf. Eq. (3)), we have:
P (S) =
e~ i<j BijSiSj+ ~ i BiOSi Z '
( 11)
where we have set (}ij = 0 for nodes Si and Sj that are not neighbors in the graph - this convention allows us to sum indiscriminately over all pairs
Si and Sj and still respect the clique boundaries. We refer to the negative of the exponent in Eq . (11) as the energy. With this definition the joint probability in Eq . (11) has the general form of a Boltzmann distribution . Saul and Jordan (1994) pointed out that exact inference for certain special cases of Boltzmann machine - such as trees, chains , and pairs of coupled chains- is tractable and they proposed a decimation algorithm for this purpose . For more general Boltzmann machines , however, decimation is not immune to the exponential time complexity that plagues other exact methods . Indeed , despite the fact that the Boltzmann machine is a special class of undirected graphical model , it is a special class only by virtue of its parameterization , not by virtue of its conditional independence struc ture . Thus , exact algorithms such as decimation and the junction tree algorithm , which are based solely on the graphical structure of the Boltzmann machine , are no more efficient
for Boltzmann
machines
than they are for
general graphical models . In particular , when we triangulate generic Boltz mann machines , including the layered Boltzmann machines and grid -like Boltzmann machines , we obtain intractably large cliques . Sampling algorithms have traditionally
been used to attempt to cope
with the intractability of the Boltzmann machine (Hinton & Sejnowski, 1986). The sampling algorithms are overly slow, however, and more recent work has considered the faster "mean field" approximation (Peterson & Anderson , 1987) . We will describe the mean field approximation for Boltz mann machines later in the paper - it is a special form of the variational approximation approach that provides lower bounds on marginal probabili ties . We will also discuss a more general variational algorithm that provides upper and lower bounds on probabilities (marginals and conditionals ) for
Boltzmann machines (Jaakkola & Jordan, 1997a). (although the latter can represent zero probabilities while the former cannot) .
118
MICHAEL I. JORDAN ET AL.
XT
Xj
7t ,
~ I
B
-
-
~ (,'
B
.
.
.
B
~
('-) B . YT
Figure 8. A HMM represented as a graphical model . The left -to -right spatial dimension represents time . The output nodes Yi are evidence nodes during the training process and the state
3 .4 .
nodes
HIDDEN
Xi
are hidden .
MARKOV
MODELS
In this section , we briefly review hidden Markov models . The hidden Markov model (HMM ) is an example of a graphical model in which exact inference is tractable ; our purpose in discussing HMMs here is to lay the groundwork for the discussion of intractable variations on HMMs in the following sections . See Smyth , Heckerman , and Jordan (1997) for a fuller discussion of the HMM as a graphical model . An HMM is a graphical model in the form of a chain (see Fig . 8) . Consider
a sequence of multinomial
conditional probability
" state " nodes Xi and assume that the
of node Xi , given its immediate predecessor Xi - I ,
is independent of all other precedingvariables. (The index i can be thought of as a time index). The chain is assumedto be homogeneous ; that is, the matrix of transition probabilities , A = P (Xi !Xi - I ), is invariant acrosstime . We also require a probability
distribution
7r = P (XI ) for the initial state
Xl . The HMM
model also involves a set of "output " nodes
i and an emis -
sion probability law B = P ( i !Xi ), again assumedtime-invariant. An HMM is trained by treating the output nodes as evidence nodes and the state nodes as hidden nodes. An expectation -maximization (EM ) algorithm (Baum , et al ., 1970; Dempster , Laird , & Rubin , 1977) is generally used to update the parameters A , B , 7r; this algorithm involves a simple iter -
ative procedure having two alternating steps: (1) run an inferencealgorithm to calculate the conditional probabilities P (Xil { i } ) and P (Xi , Xi - ll { i } ); (2) update the parameters via weighted maximum likelihood where the weights are given by the conditional probabilities calculated in step (1) . It
is easy to see that
exact
II
alization and triangulation
inference
is tractable
for HMMs
. The
mor -
steps are vacuous for the HMM ; thus the time
ANINTRODUCTION TOVARIATIONAL METHODS ~J J)
~v<J)
119
~3J) . . .
. . .
. . .
YJ
Y2
Yj
Figure 9. A factorial HMM with threechains. The transition matricesare A (1), A (2), and A (3) associatedwith the horizontal edges , and the output probabilitiesare determined by matricesB (1), B (2), and B (3) associatedwith the vertical edges .
complexity can be read off from Fig. 8 directly . We see that the maximal clique is of size N2 , where N is the dimensionality of a state node. Inference therefore scalesas O(N2T ), where T is the length of the time series.
3.5. FACTORIALHIDDENMARKOVMODELS In many problem domains it is natural to make additional structural assumptions about the state space and the transition probabilities that are not available within the simple HMM framework. A number of structured variations on HMMs have been considered in recent years (see Smyth, et al., 1997); generically these variations can be viewed as "dynamic belief networks" (Dean & Kanazawa, 1989; Kanazawa, Koller , & Russell, 1995). Here we consider a particular simple variation on the HMM theme known as the "factorial hidden Markov model" (Ghahramani & Jordan, 1997; Williams & Hinton , 1991). The graphical model for a factorial HMM (FHMM ) is shown in Fig. 9. The system is composedof a set of M chains indexed by m . Let the state node for the mth chain at time i be representedby Xi(m) and let the transition matrix for the mth chain be representedby A (m). We can view the effective state space for the FHMM as the Cartesian product of the state spacesassociatedwith the individual chains. The overall transition probability for the system by taking the product acrossthe intra -chain transition
120
MICHAELI. JORDANET AL. . . .
. . .
YJ
~
~
Figure 10 . A triangulation of an FHMM with two component chains . The morali step links states at a single time step . The triangulation step links states diago between neighboring time steps . probabilities : M P(XiIXi -l)=m n=lA(m )(X(m )IXi i (~{), (12 ) where the symbol Xi stands for the M -tuple (Xi (l ), Xi (2), . . . , Xi (M)) . Ghahramani and Jordan utilized a linear -Gaussian distribution for the emission probabilities of the FHMM . In particular , they assumed: P (~ IXi ) = N (L B (m)Xi (m), E ), m
(13)
where the B (m) and ~ are matrices of parameters . The FHMM is a natural model for systems in which the hidden state is realized via the joint configuration of an uncoupled set of dynamical systems . Moreover , an FHMM is able to represent a large effective state space with a much smaller number of parameters than a single unstructured Cartesian product HMM . For example , if we have 5 chains and in each chain the nodes have 10 states , the effective state space is of size 100,000, while the transition probabilities are represented compactly with only 500 parameters . A single unstructured HMM would require 1010parameters for the transition matrix in this case. The fact that the output is a function of the states of all of the chains implies that the states become stochastically coupled when the outputs are observed . Let us investigate the implications of this fact for the time complexity of exact inference in the FHMM . Fig . 10 shows a triangulation for the case of two chains (in fact this is an optimal triangulation ) . The cliques for the hidden states are of size N3 ; thus the time complexity of
ANINTRODUCTION TOVARIATIONAL METHODS
121
. . .
Figure 11. A triangulation of the state nodes of a three -chain FHMM with three com ponent chains . (The observation nodes have been omitted in the interest of simplicity ) .
Figure 12.
This graph is not a triangulation
.
.
.
.
.
.
.
.
.
of a three -chain FHMM .
exact inference is 0 (N3T ) , where N is the number of states in each chain
(we assumethat each chain has the same number of states for simplicity ). Fig . 11 shows the case of a triangulation
of three chains ; here the triangula -
tion (again optimal ) creates cliques of size N4 . (Note in particular that the graph in Fig . 12, with cliques of size three , is not a triangulation ; there are 4-cycles without a chord ) . In the general case, it is not difficult to see that cliques of size NM + l are created , where M is the number
of chains ; thus
the complexity of exact inference for the FHMM scales as O (NM + IT ) . For a single unstructured Cartesian product HMM having the same number of
states as the FHMM - i .e., NM states- the complexity scalesas O(N2MT ), thus exact inference
for the FHMM
is somewhat
less costly , but the expo -
111111111111
nential growth in complexity in either case shows that exact inference is infeasible for general FHMMs .
122
MICHAELI. JORDANET AL.
Uj
U2
U3 .
Yj
~
.
.
Y3
Figure 13. A hidden Markov decision tree . The shaded nodes { Ui } and { i } represent a time series in which each element is an (input , output ) pair . Linking the inputs and outputs are a sequence of decision nodes which correspond to branches in a decision tree . These decisions are linked horizontally to represent Markovian temporal dependence .
3 .6 .
HIGHER
- ORDER
HIDDEN
MARKOV
MODELS
A related variation on HMMs considers a higher -order Markov model in which each state depends on the previous K states instead of the single previous state . In this case it is again readily shown that the time complex ity is exponential in K . We will not discuss the higher - order HMM further in this chapter ; for a variational algorithm for the higher - order HMM see Saul and Jordan (1996) .
3.7. HIDDENMARKOVDECISIONTREES Finally , we consider a model in which a decision tree is endowed with Marko vian dynamics (Jordan , et al ., 1997) . A decision tree can be viewed as a graphical model by modeling the decisions in the tree as multinomial ran dom variables , one for each level of the decision tree . Referring to Fig . 13, and focusing on a particular time slice, the shaded node at the top of the diagram represents the input vector . The unshaded nodes below the input nodes are the decision nodes. Each of the decision nodes are conditioned on the input and on the entire sequence of preceding decisions (the vertical arrows in the diagram ) . In terms of a traditional decision tree diagram , this dependence provides an indication of the path followed by the data point as it drops through the decision tree . The node at the bottom of the diagram is the output variable . If we now make the decisions in the decision tree conditional not only
AN INTRODUCTION
TO VARIATIONAL
METHODS
123
on the current data point , but also on the decisions at the previous moment in time , we obtain a hidden Markov decision tree (HMDT ) . In Fig . 13, the horizontal edges represent this Markovian temporal dependence. Note in particular that the dependency is assumed to he level-specifIc- the proba bility of a decision depends only on the previous decision at the same level of the decision
tree .
Given a sequence of input vectors Ui and a corresponding sequence of output vectors i , the inference problem is to compute the conditional probability distribution over the hidden states . This problem is intractable for general HMDTs - as can be seen by noting that the HMDT includes the FHMM as a special case. 4 . Basics of variational
methodology
Variational methods are used as approximation methods in a wide variety of settings , include finite element analysis (Bathe , 1996), quantum mechanics (Sakurai , 1985) , statistical mechanics (Parisi , 1988) , and statistics (Rustagi , 1976). In each of these cases the application of variational methods converts a complex problem into a simpler problem , where the simpler problem is generally characterized by a decoupling of the degrees of freedom in the original problem . This decoupling is achieved via an expansion of the problem to include additional parameters , known as variational parameters , that must be fit to the problem
at hand .
The terminology comes from the roots of the techniques in the calculus of variations . We will not start systematically from the calculus of varia tions ; instead , we will jump off from an intermediate point that emphasizes the important role of convexity in variational approximation . This point of
view turns out to be particularly well suited to the development of varia tional methods for graphical models . 4 .1 .
EXAMPLES
Let us begin by considering a simple example . In particular , let us express the logarithm function variationally : In (x ) = min { "\x - In ''\ - I } . A
(14)
In this expression A is the variational parameter , and we are required to perform the minimization for each value of x . The expression is readily verified by taking the derivative with respect to A, solving and substituting . The situation is perhaps best appreciated geometrically , as we show in Fig . 14. Note that the expression in braces in Eq . (14) is linear in x with slope A. Clearly , given the concavity of the logarithm , for each line having
124
MICHAELI. JORDAN ET AL. 5
. .
.'
3 .
,,
.
"
,
I
.
.
.
.
,
,
.
,
.
,
.
.
.
.
.
.
.
.
.
'
'
'
'
'
'
,..'
,
.
.
"
,...,..,.. ..,.,.,.-' -~'.~." .,,,-' ,
.'
.
,
.
.
.
.-
,
.
.
.
-
-
.
'
. -'
-
.
~
-
_
.
-
-
-
~
. , ' ','.'~ . , ' : : '. '. ', '~'. ~, ' . . " , . ' - . ~ - - - . '
-1
-3
-5 0
0 .5
1
1 .5
2
2 .5
3
X
Figure 14.
Variational
transformation
of the logarithm
function . The linear functions
(Ax - In A - I ) form a family of upper bounds for the logarithm , each of which is exact for a particular
value of x .
slope .,\ there is a value of the intercept such that the line touches the logarithm at a single point . Indeed , - In A - I in Eq . (14) is precisely this intercept . Moreover , if we range across "\ , the family of such lines forms an upper envelope of the logarithm function . That is, for any given x , we have:
In(x) ~ AX - In A - I ,
(15)
for all A. Thus the variational transformation provides a family of upper bounds on the logarithm . The minimum over these bounds is the exact value of the logarithm . The pragmatic converted
justification
a nonlinear
have obtained
function
a free parameter
for such a transformation into
a linear
A that
function
. The
is that cost
we have
is that
we
must be set , once fOl' each x . For
any value of A we obtain an upper bound on the logarithm ; if we set A well we can obtain a good bound . Indeed we can recover the exact value of logarithm for the optimal choice of A. Let us now consider a second example that is more directly relevant to graphical models . For binary -valued nodes it is common to represent the probability that the node takes one of its values via a monotonic nonlinear ity that is a simple function - e .g., a linear function - of the values of the parents of the node . An example is the logistic regression model :
1 f (x) = 1+ e-X'
(16)
AN
which
we
the
have
values
seen
of
The
the
TO
previously
in
parents
logistic
bound
of
function
will
the
INTRODUCTION
not
a
is
work
.
VARIATIONAL
Eq
node
( 10
) .
Here
x
is
the
weighted
sum
of
.
neither
However
.
125
METHODS
convex
,
the
=
-
nor
logistic
concave
,
function
is
so
log
a
simple
linear
concave
.
That
is
,
function
g
is
a
concave
second
function
derivative
of ) .
functions
and
thereby
particular
,
can
we
( x
Thus
)
x
( as
we
can
( 1
can
+
e -
X )
readily
bound
bound
write
In
the
( 17
be the
verified
log
logistic
by
logistic
calculating
the
function
function
by
with
the
)
linear
exponential
.
In
:
g
( x
)
=
min
{ Ax
-
H
( A ) }
,
( 18
)
A
where
H
( We
( A )
will
is
the
binary
explain
suffices
to
logistic
think
function
the
entropy
how
minimum
the
of
it
) .
We
now
the
exponential
and
is
a
plotted
variational
in
obtain
Fig
an
as
.
15
.
)
for
of
i ty
in
a
graphical
with
( x
)
=
( 1
a
exponentials the
e -
on
CONVEX
Can
we
find variational
~
This
is
H
( A ) ]
-
In
.
,
Such
a
.
is
function
joint
Eq
For
.
to
x
it
the
log
noting
that
examples
any
of
of x
A
we
:
( 20
)
are the
a
a
.
If
variational as
obtained
by
,
-
the
of
form
the
local repre
functions
simple
including
is
the
transformations
more that
have
instead
we
parameters in
Eq
.
( 20
taking
-
) -
we
see
products
particularly
so
systematically been
utilized
-
form
of
given
that
.
transformations
)
probabil
over probabilities
of
in
in
joint
product
conditional
not
significant
DUALITY
variational
)
are
value
values
obtain
take
computationally in
;
for all
variationally
probability
tractable
linear
function
products
by
A ) .
( 20
,
obtain
product
,
-
now
for
sides
that
to
( 2 ) ) .
( l
.
particular
we
term
In
for
( A ) .
required
Eq
A ) ;
( 19
for
in
are
-
:
again
H
( I
below
.
function
eAx
A -
both
logistic
bounds
. we
logistic
the
-
once
representation
are
4 .2 .
the
X ) .
of
transformation
( cf
each
.
exponents
)
regression
network
bound
( x
models
representing
that
of
+
our
note
In
intercept
the
logistic
A
commute
[ eAX
better the
model
logistic
1 /
augment i .e .
of
we
the
provide
probabilities
sented
f
A
graphical
conditional
min
-
arises
exponential
for
, of
advantages
context
the
=
==
appropriate
function
( x
Finally
bound
( A )
function
transformation
upper
choices The
the
take
f
Good
, H
entropy
simply
f
This
function
binary
? in
Indeed the
, literature
many
126
MICHAELI. JORDAN ET AL. 1. 8
,
,
,
,
,
,
,
,
.
,
,
,
,
,
,
,
'
,
, .
,
,
,
.
,
,
,
,
.
,
1 4
,
'
,
'
,
,
"
.
,
,
,
.
,
,
,
,
,
,
,
.
' ,
.
,
,
,
"
,
,
,
.
.
-
" ,
,
,
,
'
"
.
,
,
'
,
"
.
.
'
' )
"
'
. . ..
r
,
. ,. ,," ' -, 1"
. '
,
. - - _. - ' _
" " "
,
,
"
,
,
, ,
..
,' ,
,
,
"
,
, ,
0 6
"
" , ,
"
,, , ~~.::,;~.:~.$"~;'=----'-------_._,
,
,
,
, , '
'
"
.
,
,
, ,
,
.
,
,
.
1 0
,
,
,
"
s -
..' - , ' .
' )
----------~~~~~~~~~~~-I'.-'.-I,I~~~~" 0 .2
----' ----- -------------
- 0 ..2 -3
-2
-1
0
1
2
3
x
Figure 15.
Variational transformation of the logistic function .
on graphical models are examplesof the general principle of convexduality. It is a general fact of convex analysis (Rockafellar, 1972) that a concave function f (x ) can be representedvia a conjugateor dual function as follows:
j (x) = min{ A ATx - j *(A)} ,
(21)
where we now allow x and A to be vectors . The conjugate function j * (A) can be obtained from the following dual expression :
j *(A) = min{ ATx - j (x )} . x
(22)
This relationship is easily understood geometrically , as shown in Fig . 16. Here we plot f (x ) and the linear function AX for a particular value of A. The short vertical segments represent values AX- f (x ) . It is clear from the figure that we need to shift the linear function AX vertically by an amount which is the minimum of the values AX - f (x ) in order to obtain an upper bounding line with slope A that touches f (x ) at a single point . This observation both justifies the form of the conjugate function , as a minimum over differences Ax - f (x ) , and explains why the conjugate function appears as the intercept
in Eq. (21). I t is an easy exercise to verify that the conj ugate function for the logarithm is f * (A) = In A + 1, and the conjugate function for the log logistic function is the binary entropy H (A) . Although we have focused on upper bounds in this section , the frame work of convex duality applies equally well to lower bounds ; in particular
AN INTRODUCTION TO VARIATIONALMETHODS
127
f (x)
x Figure
16
tions
-
.
The
conjugate
represented
function
as
dashed
f
lines
-
*
(
A
)
is
between
obtained
by
AX
and
f
(
x
minimizing
)
across
the
devia
-
.
for convex f (x ) we have : j
(
x
)
=
maX
{
ATx
-
j
*
(
A
)
}
(23)
,
. , \
where f
*
(
A
)
=
max
{
AT
x
-
f
(
x
)
(24)
}
x
is
the
conjugate
We
not
function
have
restricted
on
to
transforming
linear
the
of
the
in
x2
function
we
.
focused
write
bounds
bounds
.
argument
(
can
linear
Jaakkola
of
&
the
Jordan
in
More
this
section
general
of
1997a
but
bounds
function
,
,
)
.
interest
For
convex
can
duality
be
rather
example
obtained
by
than
,
if
f
is
the
(
x
)
is
value
concave
:
f (x) = min{AX2 - l *(A)} , .,\
(
25
)
where J*('\) is the conjugatefunction of J(x) == f (x2). Thus the transformation yields a quadratic bound on f (x). It is also worth noting that suchtransformationscanbe combinedwith the logarithmictransformation utilized earlier to obtain Gaussianrepresentations for the upper bounds. This can be useful in obtaining variational approximationsfor posterior distributions (Jaakkola& Jordan, 1997b). To summarize , the generalmethodologysuggestedby convexduality is the following. We wish to obtain upper or lower boundson a function of interest. If the function is already convexor concavethen we simply
128
MICHAELI. JORDANET AL.
calculate
the
then
we
convex
the
a
then
also
,
such
the
function
is
as
For
or
function
approach
in
to
logarithm
,
whose
be
concave
the
of
conjugate
this
the
convex
renders
transformations
the
.
not
that
consider
calculate
back
transform
If
transformation
may
transform
properties
the
argument
the
transformed
useful
we
inverse
has
,
function
need
to
useful
to
algebraic
.
. 3
.
APPROXIMATIONS
FOR
CONDITIONAL
discussion
thus
ability
far
distributions
has
at
proximations
interest
,
in
interest
AND
our
Let
in
us
lower
on
P
(
and
(
Si
SiIS7r
and
I
S7r
i
)
(
At
are
the
(
'
(
the
.
That
)
,
providing
,
assume
product
of
)
(
S
(
HIE
ap
-
of
)
that
is
P
(
that
E
our
)
we
conditional
forms
that
,
(
Si
I S7r
(
respectively
n
P
(
upper
bound
SiIS7r
(
i
a
)
,
-
Af
)
and
where
Af
appropriate
the
upper
i
,
parameterizations
first
an
have
probabil
pU
bounds
Consider
is
=
Suppose
local
have
variational
.
)
the
lower
bounds
P
of
we
and
bounds
upper
-
these
probability
.
each
that
upper
lower
do
prob
probabilities
P
marginal
local
How
global
concreteness
for
different
and
the
for
bound
is
.
the
?
graphs
upper
)
model
the
distribution
and
directed
generally
upper
that
i
graphical
conditional
problems
an
At
a
for
for
problem
learning
focus
bound
ities
the
inference
interest
of
approximations
approximations
for
the
on
nodes
into
particular
in
for
PROBABILITIES
focused
the
translate
pL
JOINT
PROBABILITIES
The
is
We
We
.
invertible
.
.
and
find
an
concave
function
space
function
for
or
of
4
conjugate
look
,
bounds
we
.
have
Given
:
)
'"
: : : ;
n
pU
(
SiIS7r
(
i
)
'
AY
(26)
)
' "
for
Eq
any
.
fixed
ing
For
settings
(
26
,
sums
example
)
of
must
thus
values
hold
upper
of
for
any
bounds
over
the
,
on
and
H
(
E
)
parameters
S
whenever
on
be
a
the
disjoint
Af
some
probabilities
form
E
P
variational
of
marginal
variational
letting
the
subset
other
can
right
-
partition
{LH }P(H,E)
hand
be
~ L n pU(SiIS7r (i)' Af), {H} i
Moreover
,
is
obtained
side
of
.
subset
of
S
,
by
the
we
equation
have
held
tak
-
.
:
(27)
where , as we will see in the examples to discussed below , we choose the vari ational forms pU (Si IS7r(i ), Af ) so that the summation over H can be carried
AN INTRODUCTION
TO VARIATIONAL
METHODS
129
out efficiently (this is the key step in developing a variational method). In either Eq. (26) or Eq. (27), given that these upper bounds hold for any
settingsof valuesthe variationalparametersAf , they hold in particular for optimizing settings of the parameters . That is, we can treat the right -hand
side of Eq. (26) or the right -hand side Eq. (27) as a function to be minimized
with respectto Af . In the latter case,this optimizationprocesswill induce interdependencies betweenthe parametersAf . Theseinterdependencies are desirable ; indeed they are critical for obtaining a good variational bound on the marginal probability of interest . In particular , the best global bounds are obtained when the probabilistic dependencies in the distribution are reflected in dependencies in the approximation . To clarify the nature of variational bounds , note that there is an im portant distinction to be made between joint probabilities (Eq . (26) ) and
marginal probabilities (Eq. (27)). In Eq. (26), if we allow the variational parameters to be set optimally
for each value of the argument S , then it
is possible (in principle ) to find optimizing settings of the variational parameters that recover the exact value of the joint probability . (Here we assume that the local probabilities P (SiIS -rr(i)) can be represented exactly via a variational transformation , as in the examples discussed in Section
4.1). In Eq. (27), on the other hand, we are not generally able to recover exact values of the marginal by optimizing over variational parameters that depend only on the argument E . Consider , for example , the case of a node Si E E that has parents in H . As we range across { H } there will be summands on the right -hand side of Eq . (27) that will involve evaluating the
local probability P (SiIS-rr(i)) for different values of the parents S-rr(i). If the
variationalparameterAf dependsonly on E, we cannotin generalexpect to obtain an exact representation for P (SiIS-rr(i)) in each summand. Thus, some of the summands in Eq . (27) are necessarily bounds and not exact values
.
This observation provides a bit of insight into reasons why a variational bound might be expected to be tight in some circumstances and loose in others . In particular , if P (SiISrr(i)) is nearly constant as we range across Srr(i )' or if we are operating at a point where the variational representation
is fairly insensitiveto the setting of Af (for examplethe right-hand sideof the logarithm in Fig . 14) , then the bounds may be expected to be tight . On the other hand , if these conditions are not present one might expect that the bound would be loose. However the situation is complicated by the
interdependencies betweenthe Af that areinducedduringthe optimization process . We will
return
to these
issues in the discussion
.
Although we have discussed upper bounds , similar comments apply to lower bounds , and to marginal probabilities obtained from lower bounds on the joint
distribution
.
130
MICHAELI. JORDAN ET AL.
The two
conditional
per
and
and
distributions lower
lower
bounds
, then
fewer
est
and
Finally
, it
simply
as
strict
a
ability
to
conditional bound
on
substitute (H
tion
)
and
to
P
( HIE
In
the
been
as
important
forward
to
ease
and
on
case
a
utility
.
have
simplify
more
readily
, the
in
development
require
9Note necessarily
way
for
a
-
to
inter
this
is
prob
obtained , we
into
might
the
obtain
a
parameters
. We
variational
form
an
-
provide do
marginal
calculate
we
practical
for
approxima
-
choices
effective
a
in
of
node
properties
.
Also
,
Eq
. ( 27 are
variational
)
are
simple
depend functions
which
the
transfor
certain
approximations
-
architectures others
functions not
,
model
probability
variational
than
currently
-
. The
section
conditional
transformation
issues
this
in
well
; in some
par cases
understood can
in
some
and cases
.
P ( H , E ) in E
general jointly
a. s a marginal exhaust
the
probability set
of
nodes
-
.
architecture
in
to
con
straight
probability
others
of
architectures
new
node
-
All
examples
necessarily
regime
than
variational in
for
of
the
related
not
parameter
algebraic
and
to is
.
provide
degree
outlined
choice
and
certain
it
frame
examples
interest a
that
methods
variational - out
generalized ,
readily
. These
H
be
general
worked
. To
certain more
creativity
that
of
the
bounds
treat
the of
particular ,
others of
assume
the
under
substantial
that
of
that
One
example
approximation
the
useful
marginal
complex
the
be
methods
) .
to
illustrate
however
, including
particular
that
and
also
variational
number
can ,
applying
themselves
mations
ticular
that
and
In
lend
a
variational
of
topology
operated
denomi
involves
parameterized
methodology
details
functions
the
numerator
thus
form
architectures
emphasize
develop
graph
on
upper
Generally
, the
bound
fitting
the
will in
histories to
the
we
variational
architectural
the
a
, for
by
variational
applied
involve of
serve is
this
sections
has
examples
also It
it
into
utilize
.
-
) .
examples
crete
parameters
then
have
involves
as
used
is
. Thus )
of up
numerator
parameters
(E
must
can
than are
variational
P
ratio
obtain
the
methods
that
likelihood
the
To
.
rather
distribution
the
following
as
these
the
these
, E
work
is
substitute
) .9
bounds
= = HUE
m ~ thods
approximation
probability
lower
S
variational
sampling
variational
, and
can
as
lower
evaluation
, is
(E
denominator
, because
which
that
hand
) j P
the
and
approximations
( much
obtain
upper
in
, E
, we
and
finished
case
other
(H
distribution
obtain
noting
tractable
P
numerator
function
the
=
conditional
essentially
a
worth
) , on )
the
the
simply
is
bounds
to
, in
is
( HIE ( HIE
can
is
. Indeed
sums
we
labor
P ; i .e . , P the
both
, if
our
sums
no
on
, however
nator
on
bounds
speaking
P
distribution
marginal
; that S .
is , we
do
not
-
AL METHODS ANINTRODUCTION TOVARIATION 4.4. SEQUENTIAL Let
us now
AND BLOCK
consider
in
131
METHODS
somewhat
more
detail
how
variational
methods
can be applied to probabilistic inference problems . The basic idea is that suggested above - we wish to simplify the joint probability distribution by transforming the local probability functions . By an appropriate choice of variational transformation , we can simplify the form of the joint probability distribution and thereby simplify the inference problem . We can transform some or all of the nodes. The cost of performing such transformations is that we obtain bounds or approximations to the probabilities rather than exact
results
.
The option of transforming only some of the nodes is important implies a role for the exact methods as subroutines within a variational
; it ap -
proximation . In particular , partial transformations of the graph may leave
someof the original graphical structure intact and/ or introduce new graphical structure to which exact methods can be fruitfully applied . In general , we wish to use variational approximations in a limited way, transforming the graph into a simplified graph to which exact methods can be applied . This will in general yield tighter bounds than an algorithm that transforms the entire graph without regard for computationally tractable substructure . The majority of variational algorithms proposed in the literature to date can be divided into two main classes: sequential and block. In the sequential approach , nodes are transformed in an order that is determined during the inference process. This approach has the advantage of flexibility and generality , allowing the particular pattern of evidence to determine the best choices of nodes to transform . In some cases, however, particularly when there are obvious substructures in a graph which are amenable to exact methods , it can be advantageous to designate in advance the nodes to be transformed . We will see that this block approach is particularly natural
in the setting
5 . The sequential
of parameter
estimation
.
approach
The sequential approach introduces variational transformations for the nodes in a particular order . The goal is to transform the network until the result ing transformed
network
is amenable
to exact methods . As we will see in
the examples below , certain variational transformations can be understood graphically as a sparsification in which edges are removed from the graph . A series of edge removals eventually renders the graph sufficiently sparse that an exact method becomes applicable . Alternatively , we can variation ally transform all of the nodes of the graph and then reinstate the exact node probabilities sequentially while making sure that the resulting graph stays computationally tractable . The first example in the following section
132
MICHAELI. JORDAN ET AL.
illustrates
the latter
approach Many example run
approach
time
of the exact
methods
provide
, one can run
a greedy
triangulation
of the junction
is sufficiently
small , in terms
the exact Ideally
made
the
the resulting be as small
of the
is , an ordering
would
as possible
order
clique
) . Thus
have been . In
the
Jaakkola
and
Jordan
is a bipartite
findings
are based
QMR - DT graph
i .e ., symptoms of the joint
impact
on inference
present
no difficulties
associated
Repeating
. Moreover
nodes
and
would
be
so that
( in particu graph
, particularly
-
would
given
that
1 -
to
approach
-
of a positive
there
for the findings
therefore
they
, the negative form that
been made
have
findings
probabilities updates
and focus findings
representation
:
concave ; thus , as in the
to express
the
on
.
= lid ) = I - e - EjE1r (i ) 6ijdj - 6iO
e - x is log
no
of the prob the
are positive
, we have the following
are not
be marginal
on the disease
have already
varia -
nodes that
assume
for
QMR - DT
probabilities
the exponential
findings
when
finding
can be used
can simply
and
of
QMR - DT
of sequential
symptom
us therefore
findings inference
context
the
( Eq . ( 8 ) for the negative that
observed
given
Eq . ( 9 ) for convenience
, we are able
we return
the conditional
of negative
.
in the
, as we have discussed -
orderings
. As we have seen , the
) . Note
time . Let
the negative
node
presented
by omission
for inference
of performing
probability
function
function
inference
be chosen
at each step
an application
network
are not
distribution
P ( fi The
the
run time
to the
triangulated
variational
findings that
in linear
with
the problem for the
section
in which
in Eq . ( 8 ) , the effects be handled
bound
transformations
would
problem
best
on the noisy - OR model
and Eq . ( 9 ) for the positive
can
allotted
as possible
( 1997c ) present
to the
network
ability
time . For
estimated
to transform
used to choose
following
QMR - DT NETWORK
out
time
run
to upper
variational
of the resulting
is perhaps
5 .1. THE
-
their
. If this
of the nodes
and show how a sequential in this network .
methods
bound
algorithm
in which
is a difficult
approach
example
network inference
ized
the former
is unlikely to produce the simplest graph at each step ; partial orders must be considered . In the literature to date
sequential
a specific
that
overall
be as simple
the maximal
procedures
The
findings
illustrates
.
, that
a single ordering that is , different heuristic
of the
choice
graph
lar , such that
tional
example
algorithm
can stop introducing
procedure
optimally
tests
tree inference
proced ure , the system run
and the second
.
variational
upper
( 28 ) case of the bound
logistic
in terms
of
134
MICHAELI. JORDANET AL.
The sequential methodology utilized by Jaakkola and Jordan for infer ence in the QMR -DT network actually proceeds in the opposite direction . They first transform all of the nodes in the graph . They then make use of a simple heuristic to choose the ordering of nodes to reinstate , basing the choice on the effect of reinstating each node individually starting from the completely transformed state . (Despite the suboptimality of this heuristic , they found that it yielded an approximation that was orders of magnitude more accurate than that of an algorithm that used a random ordering ). The algorithm then proceeds as follows : (1) Pick a node to reinstate , and consider the effect of reintroducing the links associated with the node into the current graph . (2) If the resulting graph is still amenable to exact methods , reinstate the node and iterate . Otherwise stop and run an exact method . Finally , (3) we must also choose the parameters Ai so as to make the approximation as tight as possible . It is not difficult to verify that products of the expression in Eq . (32) yield an overall bound that is a convex function of the Ai parameters (Jaakkola & Jordan , 1997c) . Thus standard optimization algori thms can be used to find good choices for the Ai. Jaakkola and Jordan (1997c) presented results for approximate inference on the " CPC cases" that were mentioned earlier . These are difficult cases which have up to 100 positive findings . Their study was restricted to upper bounds because it was found that the simple lower bounds that they tried were not sufficiently tight . They used the upper bounds to determine vari ational parameters that were subsequently used to form an approximation to the conditional posterior probability . They found that the variational approach yielded reasonably accurate approximations to the conditional posterior probabilities for the CPC cases, and did so within less than a minute of computer time .
5.2. THEBOLTZMANN MACHINE Let us now consider a rather different example . As we have discussed, the Boltzmann machine is a special subset of the class of undirected graph ical models in which the potential functions are composed of products of quadratic and linear "Boltzmann factors ." Jaakkola and Jordan (1997a) in troduced a sequential variational algorithm for approximate inference in the Boltzmann machine . Their method , which we discuss in this section , yields both upper and lower bounds on marginal and conditional probabilities of interest . Recall the form of the joint probability machine :
distribution
for the Boltzmann
P(S)=eLi <j8ij Si Sj +Li8iOSi Z .
(33)
AN INTRODUCTION TO VARIATIONALMETHODS
135
To obtain marginal probabilities such as P (E ) under this joint distribu tion , we must calculate sums over exponentials of quadratic energy func tions . Moreover , to obtain conditional probabilities such as P (HIE ) := P (H , E )/ P (E ), we take ratios of such sums, where the numerator requires fewer sums than the denominator . The most general such sum is the par tition function itself , which is a sum over all configurations { S } . Let us therefore focus on upper and lower bounds for the partition function as the general case; this allows us to calculate bounds on any other marginals or conditionals of interest . Our approach is to perform the sums one sum at a time , introducing variational transformations to ensure that the resulting expression stays computationally tractable . In fact , at every step of the process that we describe , the transformed potentials involve no more than quadratic Boltz mann factors . (Exact methods can be viewed as creating increasingly higher order terms when the marginalizing sums are performed ) . Thus the trans formed Boltzmann machine remains a Boltzmann machine . Let us first consider lower bounds . We write the partition function as follows :
~ eEj
InSiE L{O ,l} -
-
>
L .(}jkSjSk + j:l:i L (}joSj+ In SiE L{O,l}eLj~i (}ijSiSj +(}iOSi {J.
(}jkSjSk+ L
j#i
(35)
(JjoSj + Af L (}ijSj+ (JiO+ H{Af), (36) j#i
where the sum in the first term on the right -hand side is a sum over all pairs j < k such that neither j nor k is equal to i , where H (.) is as before the binary entropy function , and where Af is the variational parameter associated with node Si . In the first line we have simply pulled outside of the sum all of those terms not involving Si , and in the second line we
136
MICHAEL10JORDANET ALo
~ I;~ =::=:~:I
sl. (a)
(b)
Figure 18. The transformation of the Boltzmann machine under the approximate marginalization over node Si for the case of lower bounds . ( a) The Boltzmann machine before the transformation . (b ) The Boltzmann machine after the transformation , where Si has become delinked . All of the pairwise parameters , ()jk , for j and k not equal to i , have remained unaltered . As suggested by the wavy lines , the linear coefficients have changed for those nodes that were neighbors of Si .
have perfoI 'med the sum over the two values of Si . Finally , to lower bound the expression in Eq . (35) we need only lower bound the term In (1 + eX) on the right -hand side. But we have already -found variational bounds for a related expression in treating the logistic function ; recall Eq . (18) . The upper bound in that case translates into the lower bound in the current case:
In(l + e- X) ~ AX+ H(A).
(37)
This is the bound that we have utilized in Eq . (36) . Let us consider the graphical consequences of the bound in Eq . (36) (see Fig . 18) . Note that for all nodes in the graph other than node Si and its neighbors , the Boltzmann factors are unaltered (see the first two terms in the bound ) . Thus the graph is unaltered for such nodes. From the term in parentheses we see that the neighbors of node Si have been endowed with new linear terms ; importantly , however, these nodes have not become linked (as they would have become if we had done the exact marginalization ) . Neighbors that were linked previously remain linked with the same (}jk parameter . Node Si is absent from the transformed partition function and thus absent from the graph , but it has left its trace via the new linear Boltzmann factors associated with its neighbors . We can summarize the effects of the transformation by noting that the transformed graph is a new Boltzmann machine with one fewer node and the following parameters :
AN INTRODUCTION TOVARIATIONAL METHODS -
j ,k
()jk ()jO+ Af()ij
j
#
137
. 'l
=1= i .
Note finally that we also have a constant term Af (JiO+ H (Af ) to keep track of. This term will have an interesting interpretation when we return to the Boltzmann machine later in the context of block methods. Upper bounds are obtained in a similar way. We again break the partition function into a sum over a particular node Si and a sum over the configurations of the remaining nodes S\ Si. Moreover, the first three lines of the ensuing derivation leading to Eq. (35) are identical. To complete the derivation we now find an upper bound on In( 1 + eX). Jaakkola and Jordan (1997a) proposed using quadratic bounds for this purpose. In particular , they noted that : In(l + eX) = In(ex/ 2 + e- x/2) + x / 2
(38)
and that In(eX/2 + e- x/2) is a concavefunction of x2 (as can be verified by taking the secondderivative with respect to x2). This implies that In(l + eX) must have a quadratic upper bound of the following form: In(l + eX) S Ax2 + x/ 2 - g* (A).
(39)
where g* (A) is an appropriately defined conjugate function . Using these upper bounds in Eq. (35) we obtain:
InSiE L{O
(}jkSjSk + L OjoSj
+
j :f=i
Afj# LifJijSj +fJio
+_ 21 jL# i (JijSj+ (JiG - g*(Af), (40)
-
where Af is the variational parameter associated with node Si . The graphical consequences of this transformation are somewhat dif ferent than those of the lower bounds (see Fig . 19) . Considering the first two terms in the bound , we see that it is still the case that the graph is unaltered for all nodes in the graph other than node Si and its neighbors , and moreover neighbors of Si that were previously linked remain linked . The quadratic term , however, gives rise to new links between the previ ously unlinked neighbors of node Si and alters the parameters between previously linked neighbors . Each of these nodes also acquires a new linear term . Expanding Eq . (40) and collecting terms , we see that the approxi mate marginalization has yielded a Boltzmann machine with the following par ameters :
138
MICHAELI. JORDANET AL.
41 ;~~:=~~ 1
.
(a) Figure
19
.
The
transformation
marginalization
over
chine
before
where
the
Si
have
edges
,
The
with
a
+
2AfOjiOik
OjO
=
=
OjO
+
Oij
is
,
vantage
Jaakkola
does
given
as
by
reveal
6 . The block
the
/
AfO
2
,
a
ma
the
transformation
the
neighbors
-
,
of
Si
values
linear
algorithm
)
.
coefficients
that
= 1 =
i
j
= 1 =
i
g
upper
In
*
(
As
.
All
.
when
at
are
nodes
this
point
un
an
,
of
This
seeming
is
a
tighter
a
-
exact
transformation
.
bound
the
nodes
neighbors
readily
that
delinks
;
-
given
transformations
bound
as
.
transforma
,
links
the
)
bound
these
revealed
upper
Af
particular
upper
the
k
simply
is
between
structure
fact
-
combine
tree
The
,
additional
the
.
links
)
approximate
Boltzmann
parameter
new
Aforo
and
to
subroutine
1997a
of
[ j
+
lower
no
as
the
The
new
have
.
,
such
tractable
by
Jordan
Oio
natural
introducing
mitigated
&
a
the
)
all
have
also
+
by
particular
(
called
not
is
Si
2AfOiOOij
of
In
,
linked
of
+
more
.
,
formerly
a
after
suggest
consequences
structure
hand
edges
introduces
methods
tractable
other
node
2
is
somewhat
(
.
consequences
is
.
j
/
term
under
bounds
machine
dashed
computational
it
algorithm
the
(
Ojk
upper
Boltzmann
unaltered
=
exact
til
The
were
transformation
,
)
machine
of
neighbors
=
bound
Boltzmann
case
the
the
Ojk
have
delinked
b
As
are
constant
also
lower
,
parameters
the
(
that
lines
graphical
tions
those
wavy
the
the
.
and
the
and
Finally
for
delinked
linked
by
other
Si
.
become
become
of
node
transformation
has
suggested
(b)
on
delinked
disad
-
bound
.
approach
An alternative approach to variational inference is to designate in advance a set of nodes that are to be transformed . We can in principle view this "block approach " as an off-line application of the sequential approach . In the case of lower bounds , however, there are advantages to be gained by
AN INTRODUCTION
TO VARIATIONAL
METHODS
139
developing a methodology that is specific to block transformation . In this section , we show that a natural global measure of approximation accuracy can
be obtained
for
lower
bounds
via
a block
version
of the
variational
formalism . The method meshes readily with exact methods in cases in which tractable substructure can be identified in the graph . This approach was first presented by Saul and Jordan (1996) , as a refined version of mean field theory
for Markov
random
fields , and has been developed
further
in a
number of recent studies (e.g., Ghahramani & Jordan, 1997; Ghahramani & Hinton , 1996; Jordan, et al., 1997). In the block approach , we begin by identifying a substructure in the graph of interest that we know is amenable to exact inference methods (or , more generally , to efficient approximate inference methods ) . For example , we might pick out a tree or a set of chains in the original graph . We wish to use this simplified structure to approximate the probability distribution on the original graph . To do so, we consider a family of probability distri butions that are obtained from the simplified graph via the introduction of variational
parameters . We choose a particular
approximating
distribution
from the simplifying family by making a particular choice for the variational parameters . As in the sequential approach a new choice of variational parameters
must
be made
each time
new evidence
is available
More formally , let P (S) represent the joint distribution
.
on the graphical
model of interest , where as before S represents all of the nodes of the graph and Hand E are disjoint subsets of S representing the hidden nodes and
the evidence nodes, respectively . We wish to approximate the conditional probability P (HIE ). We introduce an approximating family of conditional probability distributions , Q (HIE , A) , where A are variational parameters . The graph representing Q is not generally the same as the graph repre senting P ; generally it is a sub-graph . From the family of approximat ing distributions
Q , we choose a particular
distribution
by minimizing
the
Kullback -Leibler (KL ) divergence, D (QIIP), with respect to the variational parameters :
A* == argminA D (Q(HIEix ) II P (HIE )), where for any probability is defined
as follows
distributions
(41)
Q (S) and P (S) the KL divergence
:
Q (S)
D(QIIP) = L Q(S) InP( S) " { S} The minimizing
values of the variational
(42)
parameters , A* , define a partic -
ular distribution , Q(HIE , A*), that we treat as the best approximation of P (HIE ) in the family Q (HIE , A).
140
MICHAELI. JORDANET AL.
One
simple
justification
approximation
for
accuracy
ability
of
the
mations
Q
evidence
( HIE
inequality
is
as
P
, ,\ ) .
( E
Indeed
follows
using
that
it
) ,
( i .e . , we
the
the
( E
)
==
In
=
the
L
In
~
a
between
seen
KL
to
be
L
lower
obtain
We
can
also
,
as ,
the
the
numbers
log ,
. ,
for
vector
the
"
in
Eq
.
( HIE
) .
KL
indeed Eq
convex .
( 23
( E
of )
using
prob
-
approxi
-
Jensen
' s
in
the
values
) .
~~..:~
2
( HIE
)
]
sides
Thus
the
,
right
.
( 43
of by
this
the side
according
)
equation
is
positivity
- hand A
of
of to
by
,
( H
for
simplicity
( HIE
) .
can
Eq Eq
. .
the
( 43
( 41
the
)
) ,
viewed
} : =
P
( H
f In
eln
P
( x
P
( E
( H
,E
case
)
H in as
H
.
to
Theat be
to
the
variables
be
define
appeal
viewed
" >" "
also
for
the
be
of
,
In
In
can
configurations
Finally
=
,
, >" ) of
)
an with
parameter
, E
of
making
approach
configuration
)
~ .:~
[ ~
block
Q
set
( E
In
divergence the
expression
P
:~
is we
DIVERGENCE
variational
( 23
.
hand
choosing
Consider
P
following
)
( QIIP
{ H
in
family
ofP
of
the
)
)
by
KL
each
the
( HIE
) ,
THE
of
In
In
is
measure on
.
- valued
on
Q
1991
distribution
one
, E
right
linking
probability
" x that
,
choice
1997
( H
D ,
bound
The
defined
variable
verified
the
logarithm
P
and
Moreover
thereby ,
H
numbers
vector
) .
the
theory
left
AND
( Jaakkola
real
the
( E lower
nodes
over
P
justify
approach valued
a
bound
in
Q
Q
Thomas
DUALITY
duality
)
}
divergence &
tightest
CONVEX
vex
of
on
the
6 .1 .
KL
( Cover
bound
~
the
the
divergence
as
lower
}
{ H
easily
best
likelihood
bound
{ H
difference
divergence
the
:
InP
The
KL
yields
InP
Eq a
.
of
discrete
as
a
.
-
( 23
) .
this
-
vector
Treat
vector
( E
con
sequential
this More of
real
vector ) .
It
as can
be
) :
)
( 44
)
}
, E
) .
Moreover
,
by
direct
substitution
) :
f * (Q) = mill
(HIE ,;\)InP(H,E)-lnP (E) {2:::H}Q
(45)
AN INTRODUCTION TO VARIATIONAL METHODS
141
and minimizing with respect to In P (H , E ), the conjugate function f * (Q) is seento be the negative entropy function ~ {H} Q(HIE ) In Q(HIE ). Thus, using Eq. (23), we can lower bound the log likelihood as follows: In P (E ) ~ :2:::: Q(HIE ) In P (H , E ) - Q(HIE ) In Q(HIE ) {H}
(46)
This is identical to Eq. (43). Moreover, we seethat we could in principle recover the exact log likelihood if Q were allowed to range over all probability distributions Q(HIE ). By ranging over a parameterized family Q(HIE , A), we obtain the tightest lower bound that is available within the family. 6.2. PARAMETER ESTIMATION VIA VARIATIONAL METHODS Neal and Hinton (this volume) have pointed out that the lower bound in Eq. (46) has a useful role to play in the context of maximum likelihood parameter estimation. In particular , they make a link between this lower bound and parameter estimation via the EM algorithm . Let us augment our notation to include parameters0 in the specification of the joint probability distribution P (SIO). As before, we designatea subset of the nodes E as the observedevidence. The marginal probability P (EIO), thought of as a function of (), is known as the likelihood. The EM algorithm is a method for maximum likelihood parameter estimation that hillclimbs in the log likelihood . It does so by making use of the convexity relationship between In P (H , EIO) and In P (EIO) described in the previous section. In Section 6 we showedthat the function
(Q,8) = L Q(HIE) InP(H, EIO) - Q(HIE) InQ(HIE) {H}
(47)
is a lower bound on the log likelihood for any probability distribution Q(HIE ). Moreover, we showed that the difference between InP (EI (}) and the bound (Q, O) is the KL divergence between Q(HIE ) and P (HIE ). Supposenow that we allow Q(HIE ) to range over all possible probability distributions on H and minimize the KL divergence. It is a standard result (cf. Cover & Thomas, 1991) that the KL divergenceis minimized by choosing Q(HIE ) == P (HIE , 0), and that the minimal value is zero. This is verified by substituting P (HIE , ()) into the right -hand side of Eq. (47) and recovering In P (E I0) . This suggeststhe following algorithm . Starting from an initial parameter vector 0(0), we iterate the following two steps, known as the "E (expectation) step" and the "M (maximization) step." First , we maximize the bound (Q, O) with respect to probability distributions Q. Second, we fix
142
MICHAELI. JORDAN ET AL.
Q and maximize the bound I:,(Q, 0) with respect to the parameters O. More formally , we have: (E step) :
Q(k+ l )
=
argmaxQ
(Q , (}(k))
(48)
(M step) : fJ(k+l ) =
argmax(}
(Q(k+l ), fJ)
(49)
which is coordinate ascent in (Q , ()) . This can be related to the traditional presentation of the EM algorithm (Dempster , Laird , & Rubin , 1977) by noting that for fixed Q , the right -
hand side of Eq. (47) is a function of fJonly through the In P (H , ElfJ) term . Thus ma:ximizing (Q , ()) with respect to () in the M step is equivalent to ma:ximizing the following function :
L P(HIE,(}(k)) InP(H, ElfJ).
(50)
{H }
Maximization
of this function , known as the "complete log likelihood " in
the EM literature Let
us now
, defines the M step in the traditional return
to
the
situation
in
which
presentation
we are
unable
of EM . to
com -
pute the full conditional distribution P (HIE , fJ). In such casesvariational methodology suggests that we consider a family of approximating distribu tions . Although we are no longer able to perform a true EM iteration given
that we cannot avail ourselvesof P (HIE , fJ), we can still perform coordinate ascent in the lower bound imizing
the KL divergence
(Q , fJ) . Indeed , the variational strategy of min with respect to the variational
parameters
that
define the approximating family is exactly a restricted form of coordinate ascent in the first argument of (Q, (J) . We then follow this step by an "M step " that increases the lower bound with respect to the parameters
(J.
This point of view , which can be viewed as a computationally tractable approximation to the EM algorithm , has been exploited in a number of recent architectures , including the sigmoid belief network , factorial hidden Markov
model
and
cuss in the following
hidden
Markov
decision
tree
architectures
sections , as well as the " Helmholtz
that
we
dis -
machine " of Dayan ,
et ale (1995) and Hinton , et ale (1995). 6 .3 .
EXAMPLES
We now return to the problem of picking a tractable variational parame terization for a given graphical model . We wish to pick a simplified graph which is both rich enough to provide distributions that are close to the true distribution , and simple enough so that an exact algorithm can be uti lized efficiently for calculations under the approximate distribution . Similar consideration ~ hold for the variational parameterization : the variational parameterization must be representationally rich so that good approximations
AN
are
available
KL
divergence
stuck
and
that
some
6 .3 . 1 .
Mean
. In
. It
field
Section and
bounds
machine
bounds
Recall written
also
the
as follows
Consider in
lJij8iSj Hand that Si
0 for
now
the
Sj we
for
the
E E
also
context
to
the
two
become
sums
8i
and
node
sum The
mann
, we
are
" mean machines
<j OijSiSj
+ Li
(JiOSi
Sj
reasonably
examples
ac -
.
that now
yielded
revisit and
the
discuss
machine
can
be
and
not
E E
. In
range
contributions
Zc
over
.lize . If
norma
a linear associated
summary
, we
, lJ )
Si
is given
on
E
contribution with
can
nodes
express
the
( 52 )
in
with
H the
and
the
evidence
updated nodes
} : = 8ijSjo jEE
machine
:
( 53 )
as follows
the
( Peterson form
we
terms
nodes
< j (Jij Si Sj + Ei
" approximation
P ( HIE contribution
.
:
associated
8io +
} : = { eEi {H }
graph
LIt . O~ t.OS t. < j Oij SiSj + """"' Zc '
eLi
function
the
E E , the
becomes
vanish
to
8j
when
, linear
in
distribution
and
vanishes
( 51 )
neighbors
conditional
8i
, 0 ) as follows
, (} ) =
is a particular
are
contribution
a Boltzmann field
'
that
Si . Finally
restricted
partition
have
found
. Boltzmann
, which
P ( HIE
Zc =
In
been
approach
the
nodes
constants
io include
updated
has
. We
block
for
of the
. For
8io =
The
the
Z
quadratic
with
distribution
the
it
algorithm
approaches
e
a constant
E E , the
parameters
of these
yield
machine
of
probability
representation
P ( HIE
where
getting
all
such
variational
Boltzmann
the
relate
nodes
associate
conditional
not
machine
joint
machine
reduces
and
to realize
of cases can
the
:
=
a Boltzmann
minimizes
parameters
several
143
that
possible
we discuss
P ( SlfJ ) = Oij
good
, in a number
Li
where
a procedure
necessarily
a sequential
within
. We
that
METHODS
approximations
section
discussed
lower
Boltzmann
of finding
is not
Boltzmann
5 .2 we
so that
variational this
VARIATIONAL
enough
; however
simple
solutions
TO
hope
minimum
relatively
lower
simple
simultaneously
curate
upper
yet
has
in a local
desiderata
In
INTRODUCTION
of variational
:
(JiOSi } 0
subset
( 54 )
H .
& Anderson
, 1987 ) for
approximation
Boltz
in which
a
144
MICHAELI. JORDAN ET AL.
0
f =:~~'
.
(b)
(a)
Figure 20. (a) A node Si in a Boltzmann machine with its Markov blanket . (b) The approximating mean field distribution Q is based on a graph with no edges. The mean field equations yield a deterministic relationship , represented in the figure with the dotted lines, between the variational parameters Jl,i and J.l.j for nodes j in the Markov blanket of node i .
completely factorized distribution is used to approximate P (HIE , fJ). That is, we consider the simplest possible approximating distribution ; one that is obtained by dropping all of the edges in the Boltzmann graph (see
Fig. 20). For this choice of Q(HIE , J..L), (where we now use J..L to represent the variational parameters ) , we have little choice as to the variational parameterization - to represent as large an approximating family as possible we endow each degree of freedom Si with its own variational parameter J..Li. Thus Q can be written as follows :
Q(HIE,J..L) = II J..Lfi(1- J..Li)l- Si,
(55)
iEH
wherethe product is takenoverthe hiddennodesH . Formingthe KL divergencebetweenthe fully factorizedQ distribution and the P distribution in Eq. (52), we obtain: D (QIIP) = L [JLiIn JLi+ (1 - JLi) In(l - JLi)] t -
L ijJ .LiJ -jL.". OioJLi+ In Zc, .J.(J.L t<
(56)
where the sums range across nodes in H . In deriving this result we have used the fact that , under the Q distribution , Si and Sj are independent random variables with mean values J1 ,i and J1 ,j . We now take derivatives of the KL divergencewith respectto Jli- noting that Zc is independent of J1 ,i- and set the derivative to zero to obtain the following equations:
JLi =a LJ 8ijJLj +(JiO,
(57)
145
AN INTRODUCTION TO VARIATIONAL METHODS
where a (z) = 1/ (1 + e- Z) is the logistic function and we define (}ij equal to (Jji for j < i . Eq . (57) defines a set of coupled equations known as the "mean field equations ." These equations are solved iteratively for a fixed point solution . Note that each variational parameter J1 ,i updates its value based on a sum across the variational parameters in its Markov blanket (cf . Fig . 20b) . This can be viewed as a variational form of a local message passing algorithm . The mean field approximation for Boltzmann machines can provide a reasonably good approximation to conditional distributions in dense Boltz mann machines , and is the basis of a useful approach to combinatorial opti mization known as "deterministic annealing ." There are also cases, however, in which it is known to break down . These cases include sparse Boltzmann machines and Boltzmann machines with "frustrated " interactions ; these are networks
whose
ing In
nodes the
that case
this
potential cannot
of
mean
us
machines
to
now .
"
P
of
() )
~
bound
in
( JijJ
Eq
. LiJ . Lj
+
2: : :
[ J. li
In
Galland
.
2: : :
)
( JioJ
. Li
;
and
Jordan
this
-
In
1993
) . ,
within
for
for
-
indeed
subroutines
Saul
( 47
,
help
problem
i < j
-
as by
neighbor
provide
algorithms
estimation
2: : :
also
can
pursued
lower
( EI
exact
parameter
the
between ( see
algorithms
approach
the
In
exact
use
field
out
constraints satisfied
,
the
consider
Writing
embody
simultaneously
networks led
" structured Let
be
sparse
observation
the
functions
case
( 1996
) .
Boltzmann ,
we
have
:
Z
i
, ui
+
( 1
-
, ui
)
In
( 1
-
JLi
) ]
( 58
)
1.
Taking
the
" Hebbian with where
derivative "
term
respect
to
the
brackets
distribution
performing
P
with J1, iJ1 , j Oil
.
respect
as It
well is
signify ( SIO
) .
Thus
an approximate
to as
not
a
hard an
average
we
have
Oij
yields
a
contribution to
from
show
that
with the
gradient
respect following
which the
this
has
a
derivative
derivative
is
the
unconditional
to gradient
simple of
algorithm
In
Z
( SiSj
) ;
for
M step :
~ (}ij CX(J.LiJ .Lj - (BiBj)).
(59)
Unfortunately , however , given our assumption that calculations under the Boltzmann distribution are intractable for the graph under consideration , it is intractable to compute the unconditional average. We can once again appeal to mean field theory and compute an approximation to (SiSj ) , where we now use a factorized distribution on all of the nodes; however , the M step is now a difference of gradients of two different bounds and is therefore no longer guaranteed to increase . There is a more serious problem , moreover, which is particularly salient in unsupervised learning problems . If the
146
MICHAELI. JORDANET AL.
data set of interest is a heterogeneous collection of sub-populations , such as in unsupervised classification problems , the unconditional distribution will generally be required to have multiple modes. Unfortunately the factorized mean field approximation is unimodal and is a poor approximation for a multi -modal distribution . One approach to this problem is to utilize multi -modal Q distributions within the mean-field framework ; for example , Jaakkola and Jordan (this volume ) discuss the use of mixture models as approximating distributions . These issues find a more satisfactory treatment in the context of di rected graphs , as we see in the following section . In particular , the gradient for a directed graph (cf . Eq . (68)) does not require averages under the uncondi tional distribution . Finally , let us consider the relationship between the mean field approx imation and the lower bounds that we obtained via a sequential algorithm in Section 5.2. In fact , if we run the latter algorithm until all nodes are eliminated from the graph , we obtain a bound that is identical to the mean field bound (Jaakkola , 1997) . To see this , note that for a Boltzmann machine in which all of the nodes have been eliminated there are no quadratic and linear terms ; only the constant terms remain . Recall from... Section 5.2 that the ,f ) , " constant that arises when node i is removed is JLf(JiO+ H (J1 where (JiGrefers to the value of (JiGafter it has been updated to absorb the linear terms from previously eliminated nodes j < i . (Recall that the latter update is given by BiD== BiG+ J1 ,tBij for the removal of a particular node j ). Collecting together such updates for j < i , and summing across all nodes i , we find that the resulting constant term is given as follows :
2:::i {OiOJ1 ,i +H(J1 ,i)} =2 :::j 8ijJ1 ,iJ.L+j 2::: ,i i< i 8ioJ1 - L [JLiInJLi+ (1 - JLi)In(l - J.Li)] 1,
(60)
This differs from the lower bound in Eq. (58) . . only by the term In Z , which disappears when we maximize with respect to J1 ,i . 6.3.2. Neural networks As discussed in Section 3, the "sigmoid belief network " is essentially a (directed ) neural network with graphical model semantics . We utilize the logistic function as the node probability function :
1 ) , P(Si==l/S7r (i) ==1+e-EjE7r (i)lJijSj -lJiO
(61)
where we assume that (Jij = 0 unless j is a parent of i . (In particular , Oij =1= 0 * Oji = 0) . Noting that the probabilities for both the Si = 0 case
ANINTRODUCTION TOVARIATIONAL METHODS
147
andthe Si = 1 casecanbe writtenin a singleexpression asfollows : P(S.IS (i)OijSj +OiO ) Si t 1r(t). ) - e(EjE7r 1+ eEjE7r (i)BijSj+BiO '
(62)
weobtainthe followingrepresentation for thejoint distribution: P(SI8) ==II i
(EjE7r (i)(}ij Sj+0;0) Si e . , 1+ eEjE7r (i)(}ijSj+()iO
(63)
Wewishto calculateconditionalprobabilities underthisjoint distribution. Aswehaveseen(cf. Fig. 6), inference for general sigmoidbeliefnetworks is intractable , andthusit is sensible to consider variationalapproximations . Saul, Jaakkola , andJordan(1996 ) andSaulandJordan(this volume ) have exploredtheviabilityof thesimplecompletely factorized distribution.Thus onceagainweset: Q(HIE, /1) == II /17i(1 - /1i)l - Si, iEH
(64)
andattemptto find thebestsuchapproximation by varyingtheparameters Jl,i. The computationof the KL divergence D(QIIP) proceeds muchas it doesin the caseof the meanfieldBoltzmannmachine . The entropyterm (QInQ) is the sameasbefore . Theenergyterm (QInP) is foundby taking the logarithmof Eq. (63) andaveraging with respectto Q. Puttingthese resultstogether , weobtain: InP(EIO) ~ L OijJ .liJ.lj + L OiOJ .li i<j i - ~1- ( In [1+ e2::jE7r (i)OijSj +OiO ]) - L [J.li In(J.li) + (1 - J.li) In(l - /li )] 1;
(65)
where(.) denotes anaverage with respectto the Q distribution.Notethat, despitethe fact that Q is factorized , weareunableto calculatethe average of In[l + eZi], wherezi denotesEjE7r(i) OijSj+ OiOo This is an important term whicharisesdirectlyfrom the directednatureof the sigmoidbelief network(it arisesfrom the denominator of the sigmoid , a factor which is necessary to definethe sigmoidas a localconditionalprobability). To
148
MICHAELI. JORDAN ET AL.
deal
with
this
term
parameters
of
~
Jensen
(
In
[
'
l
+
eZi
Jensen
'
this
to
s
.
)
(
given
Saul
works
[
.
and
in
+
65
(
the
For
fixed
this
In
[ e
{
iZie
=
~
i
(
Zi
)
+
(
~
~
i
(
Zi
)
+
In
for
~
i
Kij
i
=
is
is
the
)
a
the
( )
Boltzmann
via
their
ijJ1
, j
also
lize
Jordan
-
+
( JiG
+
a
~
)
+
iZi
+
The
]
e
(
65
found
tighter
)
)
.
that
bound
due
)
(
e
(
l
-
l
~
-
final
~
i
i
)
)
]
Zi
)
)
,
the
(
can
on
in
Zi
result
bound
the
be
log
of
a -
(
Zi
.
of
parents
i
has
)
,
where
)
utilized
case
~
66
likelihood
limiting
number
,
by
,
a
so
net
-
that
a
probabilistic
a
differentiating
(
.
the
J1 , i
( } ji
Yet
J1 , j
-
and
(
,
we
~
j
)
+
)
is
again
KL
obtain
the
diver
-
following
Kij
-
(
21
)
1996
)
67
)
we
Thus
,
see
that
the
from
as
in
the
case
are
(
a
term
Eq
.
(
67
)
of
linked
)
can
be
.
and
Saul
and
Jordan
~
ways
and
,
is
i
.
(
The
presented
this
two
obtain
for
bounds
the
is
contributions
parameters
lower
,
and
second
parameters
approximation
and
i
,
term
the
equation
different
j
first
and
node
.
algorithm
variational
upper
of
.
child
the
,
variational
variational
slightly
i
consistency
(
its
involves
Fig
the
,
that
node
again
see
i
Given
children
passing
Jordan
node
.
of
that
the
the
on
the
find
message
another
.
(
j
node
and
in
)
i
parents
node
we
update
both
1996
iZi
of
node
given
the
,
local
to
including
(
eZi
parameter
L
of
from
,
how
)
the
blankets
parameters
.
,
-
.
depends
"
a
of
Jaakkola
show
~
large
the
~
parents
for
as
equations
work
co
machine
these
e
O
a
,
that
"
Markov
,
ale
.
.
expression
the
blanket
Saul
-
+
parameters
contributions
interpreted
et
on
Eq
J
equation
the
(
1
that
has
invoked
from
over
Markov
[ e
=
parameters
contributions
sum
(
In
show
variational
L
an
consistency
iZi
expectation
the
a
(
over
in
:
parents
sum
Saul
form
bound
sign
a
tight
upper
negative
however
lower
.
other
{
tractable
volume
be
-
J
where
a
an
introduced
(
node
of
to
equations
J1 ,
,
providing
require
a
bound
and
approximate
respect
consistency
as
we
variational
.
values
with
viewed
=
a
can
function
gence
)
hidden
as
logistic
additional
with
a
Jensen
theorem
interpretation
the
be
tight
provide
each
introduced
:
]
to
Jordan
limit
such
eZi
)
)
appears
particular
l
1996
that
term
standard
(
which
central
In
to
Eq
(
particular
sufficiently
.
In
reduces
in
in
this
not
)
.
can
Note
that
1995
(
directly
al
parameters
.
was
(
et
provides
bound
which
These
inequality
Seung
Saul
inequality
]
s
i
,
volume
papers
different
the
uti
-
update
sigmoid
belief
in
)
Jaakkola
net
and
-
ANINTRODUCTION TOVARIATIONAL METHODS
149
.//~=:~~~ I
~jO 0 0 ,
I
,
I
I ,
I
I
I
,
I ,
I
,
I
I ,
,
I ,
I
I
I
,
I
~. ' I ,
O ----_tO ------O ,
,
,
,
,
,
,
,
,
,
,
,
,
,
I
,
,
,
0
0
(a) Figure
(
21
b
)
The
with
.
(
a
)
mean
the
A
lines
blanket
,
fixed
(
we
can
there
a
first
(
both
is
of
ten
,
(
and
Jordan
this
. 3
.
see
Fig
.
probability
22
(
a
)
)
Using
.
)
,
as
blanket
Jlj
for
.
the
nodes
figure
j
in
the
the
)
(
case
l
of
-
J . Li
of
and
a
"
of
weight
one
)
on
were
"
.
a
handwrit
-
with
advantage
of
data
in
.
the
graph
Indeed
-
,
Saul
performance
with
comparative
empirical
,
including
(
a
decay
competitive
missing
Dayan
the
parameters
degradation
,
work
comparisons
1996
)
.
models
(
notation
FHMM
)
is
developed
for
the
FHMM
a
earlier
is
given
)
machine
and
that
68
under
variational
architectures
Hinton
(
network
further
related
ij
and
.
Boltzmann
zero
with
the
)
is
important
For
Saul
appearance
the
results
.
the
equation
belief
that
( }
by
interesting
the
deal
parameters
parameters
between
An
the
obtained
J . Li
the
sigmoid
model
P({x1m )},{Yt}(8) =
i
values
slight
Markov
~
also
to
,
to
result
:
-
in
.
and
the
The
1
in
obtaining
Frey
distribution
(
extreme
report
is
see
i
Note
ability
digits
~
bounded
Markov
.
)
)
-
its
hidden
hidden
Markov
in
and
variational
systems
,
Factorial
factorial
8ij
the
networks
sampling
its
represented
respect
.
form
-
SI9
,
volume
the
belief
Gibbs
(
tested
is
in
sigmoid
3
)
~
calculate
are
1996
and
term
non
learning
(
J . Lj
1992
problem
pixels
with
)
second
for
.
i
P
,
the
approach
missing
(
-
supervised
model
joint
al
Jli
with
,
following
to
,
recognition
ical
.
~
parameters
et
digit
other
6
-
Neal
maximal
these
Saul
The
J . Li
need
by
term
that
on
(
no
noted
regularization
term
gradient
the
distribution
fact
with
,
parameters
Jl
takes
:
is
machine
relationship
variational
the
)
cx
network
deterministic
.
compute
volume
unconditional
(
belief
a
parameters
this
that
sigmoid
the
i
Ll8ij
Note
a
between
node
variational
Jordan
in
yield
,
of
Finally
for
Si
equations
dotted
Markov
node
field
(b)
multiple
chain
(
by
see
Section
:
structure
3
. 5
)
,
the
150
MICHAELI. JORDANET AL.
~ .
.
.
.
.
.
.
.
.
0
..0
..( )-........ ...
0
..0
..( )-....-.... ...
0
..0
..() -........ ...
.
.
.
Figure 22. (a) The FHMM . (b) A variational approximation for the FHMM can be obtained by picking out a tractable substructure in the FHMM graph. Parameterizing this graph leads to a family of tractable approximating distributions .
M[7f III (m )(X~m ))gTA(m )(Xlm )IXI ~{)]gTP(Yt I{Xlm )}~=1)(69 ) Computation under this probability distribution is generally infeasible , because, as we saw earlier , the clique size becomes unmanageably large when the FHMM chain structure is moralized and triangulated . Thus it is necessary to consider approximations . For the FHMM there is a natural substructure on which to base a variational algorithm . In particular , the chains that compose the FHMM are individually tractable . Therefore , rather than removing all of the edges, as in the naive mean field approximation discussed in the previous two sections , it would seem more reasonable to remove only as many edges as are necessary to decouple the chains . In particular , we remove the edges that link the state nodes to the output nodes (see Fig . 22(b )) . Without these edges the moralization process no longer links the state nodes and no longer creates large cliques . In fact , the moralization process on the delinked graph in Fig . 22(b ) is vacuous, as is the triangulation . Thus the cliques on the delinked graph are of size N2 , where N is the number of states for a single chain . Inference in the approximate graph runs in time O (MT N2 ) , where M is the number of chains and T is the length of the time series. Let us now consider how to express a variational approximation using the delinked graph of Fig . 22 (b) as an approximation . The idea is to intro duce one free parameter into the approximating probability distribution , Q , for each edge that we have dropped . These free parameters , which we denote as A~m), essentially serve as surrogates for the effect of the observation at time t on state component m . When we optimize the divergence D (QIIP ) with respect to these parameters they become interdependent ; this (deterministic ) interdependence can be viewed as an approximation to the probabilistic dependence that is captured in an exact algorithm via the moralization process.
AN INTRODUCTION TO VARIATIONALMETHODS
151
Referringto Fig. 22(b), we write the approximatingQ distribution in the followingfactorizedform:
M T Q{{xfm)}1{Yt},0,;\) = IIir )(xim ))tII=2A(m )(xfm )Ixf ~{), m =l(m
(70)
where .,\ is th ~ vector of variational parameters .,\~m). We define the transition matrix A (m) to be the product of the exact transition matrix A (m) and the variational parameter A~m) : A(m) (X ;m) IX;~{ ) = A (m) (X ;m) IX;~{ )A~m),
(71)
and similarly for the initial state probabilities ir(m): ir(m) (X ~m)) = 1[(m) (X ~m) )Aim).
(72)
This family of distributions respects the conditional independencestatements of the approximate graph in Fig. 22, and provides additional degrees of freedom via the variational parameters. Ghahramani and Jordan (1997) present the equations that result from minimizing the KL divergencebetween the approximating probability distribution (Eq. (70)) and the true probability distribution (Eq. (69)). The result can be summarized as follows. As in the other architectures that we have discussed, the equation for a variational parameter (A~m)) is a function of terms that are in the Markov blanket of the correspondingdelinked node (i .e., Yt). In particular , the update for A~m) dependson the parameters A~n), for n # m, thus linking the variational parameters at time t . Moreover, the update for A~m) depends on the expected value of the states X ;m), where the expectation is taken under the distribution Q. Given that the chains are decoupled under Q, expectations are found by running one of the exact algorithms (for example, the forward-backward algorithm for HMMs ), separately for each chain. These expectations of course depend on the current values of the parameters ,\~m) (cf. Eq. (70)), and it is this dependencethat effectively couples the chains. To summarize, fitting the variational parameters for a FHMM is an iterative , two-phaseprocedure. In the first phase, an exact algorithm is run as a subroutine to calculate expectations for the hidden states. This is done independently for each of the M chains, making referenceto the current values of the parameters A~m). In the second phase, the parameters ,\~m) are updated based on the expectations computed in the first phase. The procedure then returns to the first phase and iterates.
AN INTRODUCTION TO VARIATIONALMETHODS
.
Figure 24.
.
153
.
The "forest of trees approximation " for the HMDT . Parameterizing this
graphleadsto anapproximating familyof Q distributions .
We can also consider a "forest of trees approximation " in which the horizontal links are eliminated (see Fig . 24) . Given that the decision tree is a fully connected graph , this is essentially a naive mean field approximation on a hypergraph . Finally , it is also possible to develop a variational algorithm for the HMDT that is analogous to the Viterbi algorithm for HMMs . In particular , we utilize an approximation Q that assigns probability one to a single path in the state space. The KL divergence for this Q distribution is particularly easy to evaluate , given that the entropy contribution to the KL divergence (i .e., the Q In Q term ) is zero. Moreover , the evaluation of the energy (i .e., the Q In P term ) reduces to substituting the states along the chosen path into the P distribution . The resulting algorithm involves a subroutine in which a standard Viterbi algorithm is run on a single chain , with the other chains held fixed . This subroutine is run on each chain in turn . Jordan , et ale (1997) found that performance of the HMDT on the Bach chorales was essentially the same as that of the FHMM . The advantage of the HMDT was its greater interpretability ; most of the runs resulted in a coarse-to -fine ordering of the temporal scales of the Markov processes from the top to the bottom of the tree .
7. Discussion We have described a variety of applications of variational methods to prob lems of inference and learning in graphical models . We hope t .o have convinced the reader that variational methods can provide a powerful and elegant tool for graphical models , and that the algorithms that result are simple and intuitively appealing . It is important to emphasize, however, that research on variational methods for graphical models is of quite recent origin , and there are many open problems and unresolved issues. In this
154
MICHAELI. JORDAN ET AL.
section we discuss a number of these issues. We also broaden the scope of the presentation and discuss a number of related strands of research. 7 .1 .
RELATED RESEARCH
The methods that we have discussed all involve deterministic , iterative approximation algorithms . It is of interest to discuss related approximation schemes that are either non-deterministic or non-iterative . 7.1.1. Recognition models and the Helmholtz machine All of the algorithms that we have presented have at their core a nonlinear optimization problem . In particular , after having introduced the variational parameters , whether sequentially or as a block , we are left with a bound such as that in Eq . (27) that must be optimized . Optimization of this bound is generally achieved via a fixed -point iteration or a gradient -based algorithm . This iterative optimization process induces interdependencies between the variational parameters which give us a "best" approximation to the marginal or conditional probability of interest . Consider in particular a problem in which a directed graphical model is used for unsupervised learning . A common approach in unsupervised learning is to consider graphical models that are oriented in the "generative " direction ; that is, they point from hidden variables to observables. In this case the "predictive " calculation of P (E [H ) is elementary . The calculation of P (HIE ) , on the other hand , is a "diagnostic " calculation that proceeds backwards in the graph . Diagnostic calculations are generally non-trivial and require the full power of an inference algorithm . An alternative approach to solving iteratively for an approximation to the diagnostic calculation is to learn both a generative model and a "recognition " model that approximates the diagnostic distribution P (HIE ) . Thus we associate different parameters with the generative model and the recognition model and rely on the parameter estimation process to bring these parameterizations into register . This is the basic idea behind the "Helmholtz machine " (Dayan , et al ., 1995; Hinton , et al ., 1995) . The key advantage of the recognition -model approach is that the calculation of P (HIE ) is reduced to an elementary feedforward calculation that can be performed quickly . There are some disadvantages to the approach as well . In particular , the lack of an iterative algorithm makes the Helmholtz machine unable to deal naturally with missing data , and with phenomena such as "explaining away," in which the couplings between hidden variables change as a function of the conditioning variables . Moreover , although in some cases there is a clear natural parameterization for the recognition model that is induced
AN INTRODUCTION TO VARIATIONAL METHODS
155
from the generative model (in particular for linear models such as factor analysis ) , in general it is difficult to insure that the models are matched appropriately .l0 Some of these problems might be addressed by combining the recognition -model approach with the iterative variational approach ; essentially treating the recognition -model as a "cache" for storing good initializations for the variational parameters . 7.1.2. Sampling methods In this section we make a few remarks on the relationships between vari ational methods and stochastic methods , in particular the Gibbs sampler . In the setting of graphical models , both classes of methods rely on extensive message-passing . In Gibbs sampling , the message-passing is par ticularly simple : each node learns the current instantiation of its Markov blanket . With enough samples the node can estimate the distribution over its Markov blanket and (roughly speaking ) determine its own statistics . The advantage of this scheme is that in the limit of very many samples, it is guaranteed to converge to the correct statistics . The disadvantage is that very many samples may be required . The message-passing in variational methods is quite different . Its pur pose is to couple the variational parameters of one node to those of its Markov blanket . The messages do not come in the form of samples, but rather in the form of approximate statistics (as summarized by the varia tional parameters ) . For example , in a network of binary nodes, while the Gibbs sampler is circulating messages of binary vectors that correspond to the instantiations of Markov blankets , the variational methods are cir culating real-valued numbers that correspond to the statistics of Markov blankets . This may be one reason why variational methods often converge faster than Gibbs sampling . Of course, the disadvantage of these schemes is that they do not necessarily converge to the correct statistics . On the other hand , they can provide bounds on marginal probabilities that are quite difficult to estimate by sampling . Indeed , sampling -based methods - while well -suited to estimating the statistics of individual hidden nodes- are ill equipped to compute marginal probabilities such as P (E ) = EH P (H , E ) . An interesting direction for future research is to consider combinations of sampling methods and variational methods . Some initial work in this direction has been done by Hinton , Sallans , and Ghahramani (this volume ), who discuss brief Gibbs sampling from the point of view of variational approximation . laThe particular recognition model utilized in the Helmholtz machine is a layered graph , which makes weak conditional independence assumptions and thus makes it possible , in principle , to capture fairly general dependencies .
156
MICHAELI. JORDAN ET AL.
7 . 1 .3 .
Bayesian
methods
Variational
inference
parameter additional
.
nodes
thereby
treat
inference
in
a ,
A
in
each
a
setting
of
ensemble
tional
of
by
the
the
L
, KL
is
model
copying
the
ture
of
models they
do
not
best
a
family
Type
II
)
has
and inference approximation
In
marginal
this
The
,
) a
logistic
in
data
,
member
represent ) .
of
a
The
-
where
varia
-
ensemble
is
.
( 73
we
we
( O ) dO
;
a
know a
that
lower
find
the
et
aI
,
that
)
this
bound
.
In
minimizing
following
quantity
:
,
( 74
key
quantity
has
been
1996
aspect
) of
parametric
in
)
Bayesian
and
prior
on
mix
-
Markov
applications for itself
between
to
hidden
these family
the
connection
have
applied
minimization
inference
( 1997b
for
(O ) P
and a
using
)
of
on
variational
likelihood
,
originally to
6 ,
approach
factorization
Jordan
was
( OIE
) ) dlJ
6 ,
interesting
described
( OIE
Section
likelihood
particular .
often
. learning
way
( fJIE ( 91 ~E
bound
( E
is
different
maximization
P
averaging
any
Q ~p
Section
f
and
:
in
the
lower
=
One
Q
a
P
In
( Waterhouse
maximum
Bayesian
parameters
the
a ) .
specific
Jaakkola
tractable
)
ensemble
1997
given
)
as
best
( E
assume
( 1997b
and
for
in
( lJIE
from
architectures
factorizes
MacKay
the
,
Q
to
model
,
( MacKay
that
the
of
recently experts
P
Let
divergence
argument
the
logarithm and
f
=
argument
yields
selection More
Q
the
)
) .
as
)
probabilistic
networks of
1993
volume
.
"
neural
as
problem
useful
learning of
distribution
equivalent
In
which
,
KL
of
be
divergence
Camp
this
generic
inference be
thought
posterior
( QIIP
line
must
particular
van the
as
Bayesian
parameters
,
footing
" be
appropriate
same
minimization
Heckerman
" ensemble
" ensemble can
& to the
Following
.
same
of
treat
can
as
an
problem
probabilistic
known
( Hinton
K
( cf
the
. This
parameters
minimizing
general generally
approximations
fitting
the
the
quite
model
model
approximation
fit
can
on
variational
way
to
we
graphical
method
as
applied
inference
variational
troduced
a
graphical
and
be
Indeed
Bayesian
intractable
the
can
estimation
(J .
In
Q
is ,
just
that
determines related
variational
work
,
inference
. also
variational
developed
variational
approach regression
to with
a
find
Gaussian
methods an
analytically prior
on
the
.
7.1.4. Perspective and prospectives Per haps the key issue that faces developers of variational methods is the issue of approximation accuracy . At the current state of development of variational methods for graphical models , we have little theoretical insight
AN INTRODUCTION TO VARIATIONAL METHODS
157
into conditions under which variational methods can be expected to be accurate and conditions under which they might be expected to be inaccurate . Moreover , there is little understanding of how to match variational transformations to architectures . One can develop an intuition for when variational methods work by examining their properties in certain well -studied cases. For mean field meth ods, a good starting point is to understand the examples in the statistical mechanics literature where this approximation gives not only good , but indeed exact , results . These are densely connected graphs with uniformly weak (but non -negative ) couplings between neighboring nodes (Parisi , 1988) . The mean field equations for these networks have a unique solution that determines the statistics of individual nodes in the limit of very large graphs . In more general graphical models , of course, the conditions for a mean field approximation may not be so favorable . Typically , this can be diagnosed by the presence of multiple solutions to the mean field equations . Roughly speaking , one can interpret each solution as corresponding to a mode of the posterior distribution ; thus , multiple solutions indicate a mul timodal posterior distribution . The simplest mean field approximations , in particular those that utilize a completely factorized approximating distri bution , are poorly designed for such situations . However , they can succeed rather well in applications where the joint distribution P (H , E ) is multimodal , but the posterior distribution P (HIE ) is not . It is worth emphasizing this distinction between joint and posterior distributions . This is what allows simple variational methods - which make rather strong assumptions of conditional independence - to be used in the learning of nontrivial graphical models . A second key issue has to do with broadening the scope of variational methods . In this paper we have presented a restricted set of variational techniques , those based on convexity transformations . For these techniques to be applicable the appropriate convexity properties need to be identified . While it is relatively easy to characterize small classes of models where these properties lead to simple approximation algorithms , such as the case in which the local conditional probabilities are log-concave generalized lin ear models , it is not generally easy to develop variational algorithms for other kinds of graphical models . A broader characterization of variational approximations is needed and a more systematic algebra is needed to match the approximations to models . Other open problems include : (1) the problem of combining variational methods with sampling methods and with search based methods , (2) the problem of making more optimal choices of node ordering in the case of sequential methods , (3) the development of upper bounds within the block framework , (4) the combination of multiple variational approximations for
158 the
MICHAEL I. JORDAN ET AL.
same
model
tectures
,
that
Similar
on
of
and
for
pruned
values
variational
discrete
sampling
of
of
the
of
a
large
underlying
the
of
degree
.
these
on
probability
graph
for
methods
all
The
difficulty
lies
actual
model
-
methods
cases
the
archi
.
and
exact
in
to
for
variables
methods
versions
contingent
methods
random
foundations
is
properties
.
development
theoretical
accuracy
discrete
the
or
solid
that
)
exist
incomplete
probability
8
5
problems
providing
fact
(
continuous
open
based
in
and
combine
in
the
conditional
rather
than
on
the
.
Acknowledgments
We
wish
to
always
)
thank
Brendan
Peter
Frey
Dayan
for
,
David
helpful
Heckerman
,
comments
on
the
Uffe
Kjrerulff
,
manuscript
and
(
as
.
References
Bathe
,
K
.
Baum
,
L
. E
ring
J
in
of
.
(
. ,
1996
)
.
Petrie
the
T
,
T
Cowell
.
,
(
Ed
,
(
in
.
,
press
)
Luby
is
P
. ,
T
. ,
NP
-
Hinton
&
M
.
,
R
.
(
.
.
plete
in
press
In
M
A
. P
,
I
D
.
L
. ,
B
.
.
.
.
and
,
&
C
,
Ghahramani
(
,
,
Z
,
29
,
Thomas
E
B
.
.
.
.
S
,
.
.
.
A
,
of
Prentice
-
Hall
.
technique
Markov
occur
chains
Theory
for
Norwell
.
Bayesian
,
Zemel
model
142
. )
,
&
for
-
150
.
The
-
Annals
MA
,
60
,
R
New
York
networks
:
Kluwer
,
141
-
S
.
(
In
153
John
M
Academic
Wiley
.
I
.
.
Jordan
Publishers
inference
.
:
.
in
.
Bayesian
belief
.
1995
)
.
The
Helmholtz
Machine
.
reasoning
about
causality
and
persistence
.
.
:
,
A
Learning
unifying
in
Rubin
,
(
.
1994
)
framework
Graphical
.
D
. B
.
(
Journal
for
Models
1977
of
Localized
)
.
the
.
Maximum
-
Royal
partial
:
Dayan
In
D
.
,
D
.
S
P
.
probabilistic
Norwell
in
,
MA
likelihood
:
from
Statistical
Society
evaluation
Proceedings
of
.
(
(
1996
)
.
Does
Touretzky
,
of
the
the
M
Processing
1994
)
.
:
The
379
-
Kluwer
Tenth
incom
,
belief
B39
1
networks
Conference
.
-
,
-
.
San
38
.
Un
-
Mateo
,
.
C
wake
.
Systems
Backward
in
of
,
8
simulation
Proceedings
-
Mozer
the
Tenth
.
sleep
algorithm
&
M
.
E
learn
.
Cambridge
,
Bayesian
(
MA
:
networks
Conference
good
Hasselmo
.
.
San
MIT
Eds
. )
Press
,
.
Uncertainty
Mateo
,
CA
:
,
limitations
of
&
Hinton
,
G
.
Report
.
deterministic
Boltzmann
machine
learning
.
Net
-
.
,
&
,
245
-
A
,
273
. ,
E
.
(
CRG
Jordan
,
modelling
&
:
.
Information
)
-
Learning
Ghahramani
,
NJ
.
1993
Z
.
,
maximization
.
Neural
355
Technical
A
Information
inference
Intelligence
.
,
Toronto
.
Ed
.
probabilistic
904
5
)
.
.
R
)
Intelligence
.
4
171
of
algorithm
?
Favero
,
W
1989
. M
,
Kaufmann
,
N
EM
G
Artificial
Morgan
-
elimination
Artificial
in
work
(
-
(
,
estimators
Galland
889
Bucket
Hanks
Hinton
Advances
,
K
,
Kaufmann
density
R
7
,
Cliffs
1970
.
&
Morgan
164
Intelligence
Neal
Jordan
the
and
:
Bayesian
.
Laird
via
certainty
,
. ,
Englewood
(
Approximating
,
)
.
. ,
data
,
.
.
functions
Elements
Artificial
E
,
)
N
probabilistic
,
.
,
Models
1993
.
G
Publishers
,
CA
41
)
.
Weiss
to
,
Academic
Dempster
Draper
1991
(
hard
,
&
Introduction
Kanazawa
ference
Gilks
(
,
of
Intelligence
Dechter
.
Graphical
,
Computational
Fung
.
.
Computation
,
G
,
J
in
&
Neural
Dean
,
Learning
P
,
,
analysis
Thomas
networks
Dayan
Procedures
Soules
statistical
&
.
. )
,
,
Statistics
,
R
Dagum
Element
.
Mathematical
Cover
Frey
Finite
,
M
1996
-
.
I
.
)
TR
(
-
.
Switching
96
1997
-
)
.
3
,
state
Department
-
of
Factorial
Hidden
space
models
Computer
.
University
of
Science
Markov
.
models
.
Machine
.
&
.
Spiegelhalter
The
,
Statistician
D
.
,
(
43
1994
,
)
169
.
A
-
178
language
.
and
a
program
for
complex
ANINTRODUCTION TOVARIATIONAL METHODS
159
Heckerman , D. (in press). A tutorial on learningwith Bayesiannetworks. In M. I. Jordan (Ed.), Learningin GraphicalModels. Norwell, MA: Kluwer AcademicPublishers. Henrion, M. (1991). Search -basedmethodsto bound diagnosticprobabilities in very large belief nets. Uncertaintyand Artificial Intelligence: Proceedings of the Seventh Conference . San Mateo, CA: MorganKaufmann. Hinton, G. E., & Sejnowski , T. (1986). Learningandrelearningin Boltzmannmachines . In D. E. Rumelhart & J. L. McClelland, (Eds.), Parallel distributedprocessing : Volume 1, Cambridge, MA: MIT Press. Hinton, G.E. & van Camp, D. (1993). Keepingneural networkssimple by minimizing the descriptionlength of the weights. In Proceedings of the 6th Annual Workshopon ComputationalLearning Theory, pp 5-13. New York, NY: ACM Press. Hinton, G. E., Dayan, P., Frey, B., and Neal, R. M. (1995). The wake-sleepalgorithm for unsupervisedneural networks. Science , 268:1158- 1161. Hinton, G. E., Sallans, B., & Ghahramani, Z. (in press). A hierarchicalcommunity of experts. In M. I . Jordan (Ed.), Learningin GraphicalModels. Norwell, MA: Kluwer AcademicPublishers. Horvitz, E. J., Suermondt, H. J., & Cooper, G.F. (1989). Boundedconditioning: Flexible inferencefor decisionsunderscarceresources . Conferenceon Uncertaintyin Artificial Intelligence: Proceedingsof the Fifth Conference . Mountain View, CA: Association for UAI . Jaakkola, T. S., & Jordan, M. I . (1996). Computingupperandlowerboundson likelihoods in intractable networks. Uncertaintyand Artificial Intelligence: Proceedingsof the Twelth Conference . SanMateo, CA: MorganKaufmann. Jaakkola, T . S. (1997). Variational methodsfor inferenceand estimation in graphical models . Unpublisheddoctoral dissertation, Massachusetts Institute of Technology . Jaakkola, T. S., & Jordan, M. I . (1997a ) . Recursivealgorithmsfor approximatingprobabilities in graphical models. In M. C. Mozer, M. I. Jordan, & T . Petsche(Eds.), Advancesin Neural Information ProcessingSystems9. Cambridge, MA: MIT Press. Jaakkola, T. S., & Jordan, M. I . (1997b). Bayesianlogistic regression : a variational approach. In D. Madigan & P. Smyth (Eds.), Proceedingsof the 1997Conferenceon Artificial Intelligenceand Statistics, Ft . Lauderdale , FL. Jaakkola, T . S., & Jordan. M. I . (1997c ) . Variationalmethodsandthe QMR-DT database . Submitted to: Journal of Artificial IntelligenceResearch . Jaakkola, T . S., & Jordan. M. I . (in press). Improvingthe meanfield approximationvia the useof mixture distributions. In M. I . Jordan(Ed.), Learningin GraphicalModels. Norwell, MA: Kluwer AcademicPublishers. Jensen, C. S., Kong, A., & Kjrerulff, U. (1995). Blocking-Gibbs samplingin very large probabilistic expert systems. International Journal of Human-ComputerStudies, 42, 647- 666. Jensen,F. V., & Jensen,F. (1994). Optimal junction trees. Uncertaintyand Artificial Intelligence: Proceedings of the Tenth Conference . SanMateo, CA: MorganKaufmann. Jensen, F. V. (1996). An Introductionto BayesianNetworks. London: UCL Press. Jordan, M. I . (1994). A statistical approachto decisiontree modeling. In M. Warmuth (Ed.), Proceedings of the SeventhAnnual ACM Conferenceon ComputationalLearning Theory. New York: ACM Press. Jordan, M. I ., Ghahramani, Z., & Saul, L. K. (1997). Hidden Markov decisiontrees. In M. C. Mozer, M. I . Jordan, & T . Petsche(Eds.), Advancesin Neural Information ProcessingSystems9. Cambridge, MA: MIT Press. Kanazawa, K ., Koller, D., & Russell, S. (1995). Stocha .gtic simulation algorithms for dynamic probabilistic networks. Uncertaintyand Artificial Intelligence: Proceedings of the EleventhConference . SanMateo, CA: MorganKaufmann. Kj ~rulff, U. (1990). Triangulationof graphs- algorithmsgiving small total state space. ResearchReport R-90-09, Departmentof Mathematicsand ComputerScience , Aalborg University, Denmark. Kj ~rulff, U. (1994). Reduction of computational complexity in Bayesiannetworks
160
MICHAEL
through
removal
ceedings
of
the
I.
of
weak
Tenth
MacKay, D.J.C. (1997a). manuscript. Department MacKay, D.J.C. (1997b). eters. Submitted to Neural MacKay, D.J.C. (1997b). Learning
in
McEliece, of
Pearl s
Areas
belief
in
Merz, [ http:
dependences.
Ensemble Physics, Comparison Computation. Introduction
of
Models.
propagation
Department
R.
of
(1992).
R.
(1993).
Norwell,
MA:
Kluwer
G. (1988). J. (1988).
Statistical Probabilistic
San
C.,
networks.
Mateo,
Rockafellar, Rustagi,
CA:
inference
(1972).
(1976).
Report press). In
J.
L.
1,
(1987).
Convex
networks. Neural
Saul,
neural
ings
In
D.
K.,
&
networks.
S.
(1995). R. D.,
Networks:
inference
of
the
Tenth
Shenoy, P. 40,P. Research, Shwe, M. A.,
S.
P.,
for hidden Waterhouse,
of
experts.
Neural Williams,
Artificial
M.
I.
Touretzky, Processing
In
Publishers.
M.
Ma
M.
in
Analysis.
Methods
I. (in
I. Jordan
M.
Princeton
in
Intelligence
(1996).
Red In
Statistics.
Mechanics. Learning M. I.
Exploiting
C.
Systems
Resear
Mozer, 8.
press).
Cam
A mean
(Ed.),
Learning
Annealed theories of learning The Statistical Mechanics P Andersen, S. K., & Szolovits, in
belief
Conference.
(1992). 463 484. Middleton,
H. P., & Cooper, INTERNIST-1/QMR
Smyth,
of
Jordan,
Academic
Seung, Neural Shachter, bilistic
Jordan,
Information
L.
Kluwer
&
bel
A mean
995 1019.
Variational
Journal
K.,
of
CRG-TR-93-1 A view of M. I. Jordan
Sakurai, J. (1985). Modern Quantum Saul, L. K., & Jordan, M. I. (1994). 6, 1173 1183. Saul, L. K., Jaakkola, T. S., & Jordan, networks.
Computer
Kaufmannn.
R.
re
using
Publishers.
Morgan
Systems,
R.
J.
and
UCI
Field Theory. Reasoning
& Anderson,
Complex
J.
Subm
learning
Academic
Kl
Cheng,
algorithm.
Information
Probabilistic
MA:
Peterson,
Saul,
learning of appr to M
University
&
Connectionist
versity of Toronto Technical Neal, R., & Hinton, G. E. (in tal, sparse, and other variants. Inference,
Mateo,
Norwell,
D.J.C.,
AL.
Unce
San
J., & Murphy, P. M. (1996). /www. ics . uci/ - m1earn/MLRepository
fornia,
Parisi, Pearl,
ET
Conference.
MacKay,
Communication.
C.
Neal, 113. Neal,
Graphical
R.J.,
JORDAN
San
Mateo,
Valuation-based B., Heckerman,
&
F.
Uncertainty CA:
systems D. E
(1991). Probabilistic d knowledge base. Meth. Heckerman, D., & Jordan, M. I. (19 Markov probability models. Neural S., MacKay, D.J.C. & Robinson, T In D. S. Touretzky, M. C. Mozer,
Information C. K. I.,
G.
networks.
Processing Hinton,
G.
Systems E. (1991).
8.
Cambrid Mean
AN INTRODUCTION TO VARIATIONAL METHODS
161
temporallydistortedstrings.In Touretzky , D. S., Elman,J., Sejnowski , T., & Hinton, G. E., (Eds.), Proceedings of the 1990Connectionist ModelsSummerSchool . San Mateo, CA: MorganKaufmann . 9. Appendix In this section, we calculate the conjugate functions for the logarithm function and the log logistic function . For f (x) = In x , we have: j *(.;\) = min{ x .;\x - lnx } .
(75)
Taking the derivative with respect to x and setting to zero yields x = >..- 1. Substituting back in Eq . (75) yields :
f * (A) = In A + 1,
(76)
which justifies the representation of the logarithm given in Eq. (14). For the log logistic function g(x) == - In(l + e- X), we have:
g*(>..) = min x {>..x + In(l + e-X)}.
(77)
Taking the derivative with respect to x and setting to zero yields: A
=
=
e
i
-
+
(78)
X
e
-
x
'
from which we obtain : x
=
=
In
(79)
~
A
and
1 In(1+e-X)=1--=-"':\.
(80)
Pluggingtheseexpressions backinto Eq. (77) yields: f * (A) = - AIn;;\ - (1 - A) In(l - A),
(81)
whichis the binary entropyfunctionH (A). This justifiesthe representation of the logistic function givenin Eq. (19).
IMPROVING THE MEAN FIELD APPROXIMATION VIA THE USE OF MIXTURE DISTRIBUTIONS
TOMMI
S . JAAKKOLA
University Santa
of Cruz
,
California CA
AND
MICHAEL I . JORDAN MassachusettsInstitute of Technology Cambridge, MA
Abstract . Mean field methods provide computationally efficient approxi mations to posterior probability distributions for graphical models . Simple mean field methods make a completely factorized approximation to the posterior , which is unlikely to be accurate when the posterior is multi modal . Indeed , if the posterior
is multi - modal , only one of the modes can
be captured . To improve the mean field approximation in such cases, we employ mixture models as posterior approximations , where each mixture component
is a factorized
distribution
. We describe efficient
methods
for
optimizing the parameters in these models .
1.
Introduction
Graphical models provide a convenient formalism in which to express and manipulate conditional independence statements . Inference algorithms for graphical models exploit these independence statements , using them to compute conditional probabilities while avoiding brute force marginaliza tion over the joint probability table . Many inference algorithms , in particu lar the algorithms that construct a junction tree , make explicit their usage of conditional independence by constructing a data structure that captures the essential Markov properties underlying the graph . That is, the algorithm groups interacting variables into clusters , such that the hypergraph of clusters has Markov properties that allow simple local algorithms to be 163
164
TOMMI S. JAAKKOLA AND MICHAEL I . JORDAN
employed for inference. In the best case, in which the original graph is sparse and without long cycles, the clusters are small and inference is efficient. In the worst case, such as the caseof a densegraph, the clusters are large and inference is inefficient (complexity scalesexponentially in the size of the largest cluster). Architectures for which the complexity is prohibitive include the QMR database (Shwe, et al., 1991), the layered sigmoid belief network (Neal, 1992), and the factorial hidden Markov model (Ghahramani & Jordan, 1996). Mean field theory (Parisi, 1988) provides an alternative perspective on the inference problem. The intuition behind mean field methods is that in dense graphs each node is subject to influences from many other nodes; thus, to the extent that each influence is weak and the total influence is roughly additive (on an appropriate scale), each node should roughly be characterized by its mean value. In particular this characterization is valid in situations in which the law of large numbers can be applied. The mean value is unknown, but it is related to the mean values of the other nodes. Mean field theory consistsof finding consistencyrelations betweenthe mean values for each of the nodes and solving the resulting equations (generally by iteration ). For graphical models these equations generally have simple graphical interpretations ; for example, Peterson and Anderson (1987) and Saul, Jaakkola, and Jordan (1996) found, in the casesof Markov networks and Bayesian networks, respectively, that the mean value of a given node is obtained additively (on a logit scale) from the mean values of the nodes in the Markov blanket of the node. Exact methods and mean field methods might be said to be complementary in the sensethat exact methods are best suited for sparsegraphs and mean field methods are best suited for densegraphs. Both classesof methods have significant limitations , however, and the gap between their respective domains of application is large. In particular , the naive mean field methods that we have referred to above are based on the approximation that each node fluctuates independently about its mean value. If there are a strong interactions in the network, i .e. if higher-order moments are important , then mean field methods will generally fail . One way in which such higher-order interactions may manifest themselvesis in the presence of multiple modes in the distribution ; naive mean field theory, by assuming independent fluctuations , effectively assumesa unimodal distribution and will generally fail if there are multiple modes. One approach to narrowing the gap between exact methods and mean field methods is the "structured mean field" methodology proposedby Saul and Jordan (1996). This approach involves deleting links in the graph and identifying a core graphical substructure that can be treated efficiently via exact methods (e.g., a forest of trees). Probabilistic dependenciesnot
IMPROVING THE MEAN FIELD APPROXIMATION
165
accounted for by the probability distribution on this core substructure are treated via a naive mean field approximation. For architectures with an obvious chain-like or tree-like substructure, such as the factorial hidden Markov model, this method is natural and successful. For architectures without such a readily identified substructure, however, such as the QMR databMe and the layered sigmoid belief network, it is not clear how to develop a useful structured mean field approximation. The current paper extends the basic mean field methodology in a different direction . Rather than basing the approximation on the assumption of a unimodal approximating distribution ,1 we build in multimodali ty by allowing the approximating distribution to take the form of a mixture distribution . The components of the mixture are assumedto be simple factorized distributions , for reasonsof computational efficiency. Thus, within a mode we assumeindependent fluctuations , and multiple modes are used to capture higher-order interactions. In the following sections we describe our mixture -based approach that extends the basic mean field method. Section 2 describesthe basicsof mean field approximation , providing enough detail so as to make the paper selfcontained. Section 3 then describeshow to extend the mean field approximation by allowing mixture distributions in the posterior approximation . The following sections develop the machinery whereby this approximation can be carried out in practice. 2.
Mean
field
approximation
Assume we have a probability model P (S, S*), where S* is the set of instantiated or observed variables (the "evidence" ) and the variables S are hidden or unobserved. We wish to find a tractable approximation to the posterior probability P (SIS*). In its simplest form mean field theory assumesthat nodes fluctuate independently around their mean values. We make this assumption explicit by expressingthe approximation in terms of a factorized distribution Qmj (SIS*): Qmf (SIS*) =
II7.Qi(SiIS *),
(1)
where the parameters Qi (SiIS * ) depend on each other and on the evidence S* (i .e., we must refit the mean field approximation for each configuration of the evidence ) . lBy "unimodal distributions" we meandistributionsthat are log-concave . Meanfield distributions that are products of exponentialfamily distributions are unimodal in this sense .
166
TOMMI S. JAAKKOLAAND MICHAELI. JORDAN
The mean field approximationcan be developedfrom severalpoints of view, but a particularly usefulperspectivefor applicationsto graphical modelsis the "variational" point of view. Within this approachwe use the KL divergenceas a measureof the goodnessof the approximationand choosevaluesof the parametersQi(SiIS*) that minimizethe KL divergence :
KL( Qmj (SIS *) IIP(Sls*) ) = ~ Qmf (SIS *)log~
.
(2)
Why do we useKL(QmfIIP) rather than KL (PIIQmf)? A pragmaticanswer is that the use of KL (QmfIIP) implies that averagesare taken using the tractable Qmf distribution rather than the intractableP distribution (cf. Eq. (2)); only in the former casedo we haveany hopeof finding the best approximationin practice. A moresatisfyinganswerarisesfrom considering the likelihood of the evidence , P(S*). Clearly calculationof the likelihood permits calculationof the conditionalprobability P(SIS*). UsingJensen 's inequality we can lower bound the likelihood (cf. for exampleSaul, et al., 1996):
logP(S*)
-
-
>
log LsP(S,s*) P(S,S*) log L Q(SIS *)(J(SIS ;)S P(S,S*) LSQ(SIS *)log (J(SIS ;)'
for arbitrary Q(SIS*), in particular for the meanfield distribution Qmj (SIS*). I t is easily verified that the difference between the left hand side and the right hand side of this inequality is the KL divergence KL (QIIP ); thus, minimizing this divergence is equivalent to maximizing a lower bound on the likelihood . For graphical models, minimization of KL (QmfIIP ) with respect to Qi (SiIS*) leads directly to an equation expressing Qi (SiIS*) in terms of the Qj (Sj IS*) associatedwith the nodes in the Markov blanket of node i (Peterson & Anderson, 1987; Saul, et al., 1996). These equations are coupled nonlinear equations and are generally solved iteratively . 3. Mixture
approximations
In this section we describe how to extend the mean field framework by utilizing mixture distributions rather than factorized distributions as the approximating distributions . For notational simplicity , we will drop the
167
IMPROVING THE MEAN FIELD APPROXIMATION
dependenceof Q(SIS*) on S* in this section and in the remainder of the paper, but it must be borne in mind that the approximation is optimized separately for each configuration of the evidence S*. We also utilize the notation F (Q) to denote the lower bound on the right hand side of Eq. (3):
:F(Q)=:LsQ(S)log :~i~ .~):l Q(~ S When
Q
( S
)
takes
the
approximation
a
corresponding
I ~_
the
as
I
OUf distribution
factorized
is
as
to
Qmf
mean
bound
proposal
form
" factorized F
utilize
( S
field
( Qmj
"
)
in
(3)
Eq
.
( 1 ) ,
we
approximation
will
refer
and
will
to
the
denote
) .
a
mixture
=
1: : :
distribution
as
the
approximating
:
Qmix
( S
)
amQmj
( Slm
) ,
( 4 )
m
where
each
butions
.
and of
In
F
( Qmix
F
( Qmf
sider
the
The
sum the
of
mixing
to
one
KL
divergence
the
remainder
)
-
1m
)
component
the
distributions
proportions
,
are
Qmf
am
additional
,
which
( 81m are
parameters
to
)
are
factorized
constrained be
fit
distri
to via
the
be
-
positive
minimization
. of
mixture
corresponding
this
section
bound to
we
the
and mixture
focus
the
on
the
relationship
factorized components
between
mean .
To
field this
bounds end
,
con
-
:
P(8,8 *) :F(Qm1 ..x)-- ~ L...SQmix (8)logQ ( mix 8) }:,Sam [Qmj (Slm )log ~ .~~.!-~ m Qmix (8:2)] }:,Sam [Qmf (8Im )log ~( S 8~)+Qmf (Slm )log 2Qmix ~l(SI ~))-] m Qmj (,Slm (S }:mam :F(Qmflm )+m }:,SamQmf (Slm )log 2Qmix ~f(SI ~)1 (S (5) }:mG :m :F(Qmflm )+I(m;S) -
-
where I (m ; S) is the mutual information between the mixture index m and the variables S in the mixture distribution2 . The first term in Eq. (5) 2Recall that the mutual
l (x ;y) = ~ LIz
,y
information . between any two random variables x and y is
P (x , y) logP (ylx )fP (y ).
168
TOMMI
is a convex
S. JAAKKOLA
combination
AND
of factorized
MICHAEL
mean
field
I . JORDAN
bounds
and
therefore
it
yields no gain over factorized mean field as far ~ the KL divergence is concerned
. It
is the
second
term , i .e ., the
mutual
information
, that
char -
acterizes the gain of using the mixture approximation .3 As a non-negative quantity , I (m ; S) incre ~ es the likelihood bound and therefore improves the approximation . We note , however, that since I (m ; S) :::; log M , where M is the number of mixture components , the KL divergence between the mixture approximation and the true posterior can decrease at most logarithmically in the number
of mixture
4 . Computing
components
the mutual
.
information
To be able to use the mixture bound of Eq. (5) we need to be able to compute the mutual information term . The difficulty with the expression M it stands is the need to average the log of the mixture distribution over all configurations of the hidden variables . Given that there are an exponential number of such configurations , this computation is in general intractable . We will make use of a refined form of Jensen's inequality to make headway. Let
us first
rewrite
the mutual
"""
information
as
Qmf (8lm )
I(m;S) = L..., O :'mQmj (Slm )logQ . (8) m
,
S
(6)
m1 .X
=EO :'mQmj (8Im )[-lOg ~ ] (7) = ~ :'mR {SIm """O:'mQmj(8Im)[- logOO :'m{8fm {Slm R))Qmj Qmix (8)) ] (8) = L O :'mQmj (Slm) logR(8Im) - LGmlogam m ,S
"""
m
( I )[ R(Slm ) Qmix (8)] (9)
+~ QmQmj 8m - log_._~~~---Qmj{8Im )
where we have introduced additional smoothing distributions R(Slm ) whose role is to improve
the convexity
bounds that we describe below .4 With
some
foresight we have also introduced am into the expression ; as will become 3Of course , this is only the potential gain to be realized from the mixture mation . We cannot rule out the possibility that it will be harder to search for approximating distribution in the mixture approximation than in the factorized particular the mixture approximation may introduce additional local minima KL divergence .
approxi the best case. In into the
4In fact , if we use Jensen's inequality directly in Eq. (6), we obtain a vacuous bound of zero , as is easily verified .
IMPROVING THE MEAN FIELD APPROXIMATION
169
apparent in the following section the forced appearance of am turns out to be convenient in updating the mixture coefficients . To avoid the need to average over the logarithm of the mixture distri bution , we make use of the following convexity bound :
- log(x) ~ - .,\ x + log("\) + 1,
(10)
replacing the - log(.) terms in Eq. (9) with linear functions, separately for eachm . These substitutions imply a lower bound on the mutual information given by:
I (m; S) ~ } : amQmf (Slm)logR(Slm) - } : amlogam
-
m,S m - } : Am}: R(Slm)Qmix (S) + } : amlogAm + 1 (11) m S m IA(m; B).
(12)
The tightness of the convexity bound is controlled by the smoothing functions R (Stm) , which can be viewed as "flattening" the logarithm function such that Jensen's inequality is tightened. In particular , if we set R(Slm ) cx Qm/ (Slm )/ Qmix(S) and maximize over the new variational parameters Amwe recoverthe mutual information exactly. Sucha choicewould not reduce the complexity of our calculations, however. To obtain an efficient algorithm we will instead assumethat the distributions R have simple factorized forms, and optimize the ensuing lower bound with respect to the parameters in these distributions . It will turn out that this assumption will permit all of the terms in our bound to be evaluated efficiently. We note finally that this bound is never vacuous. To seewhy, simply set R(Slm ) equal to a constant for all { S, m} in which casemax,\ I ,\(m; S) = 0; optimization of R can only improve the bound. 5 . Finding
the mixture
parameters
In order to find a reasonable mixture approximation we need to be able to optimize the bound in Eq . (5) with respect to the mixture model . Clearly a necessary condition for the feasibility of this optimization process is that the component factorized mean field distributions can be fit to the model tractably . That is, we must assume that the following mean field equations :
88Qj :F(Qm /))=constant (Sj
(13)
170
TOMMI S. JAAKKOLA AND MICHAEL I. JORDAN
can be solved efficiently (not necessarily in closed form) for any of the marginals Qj (Sj ) in the factorized mean field distribution Qmf .5 Examples of such mean field equations are provided by Petersonand Anderson (1987) and Saul, et all (1996). Let us first consider the updates for the factorized mean field components of the mixture model. We needto take derivatives of the lower bound: F (Qmix) ~ L amF (Qmflm ) + IA(m; S) == FA (Qmix) m
(14)
with respect to the parameters of the component factorized distributions . The important property of I A(m; S) is that it is linear in any of the marginals Qj (Sjlk ), where k indicates the mixture component (seeAppendix A). As a result, we have:
8I).(m;S) = 8Qj(Sjlk) where
the
ally
constant
is
dependent
field
on
distribution
)
8
:
F ; \
can
be
the
us
I
; \
can
now
the
( m
;
)
.
write
ak8
for
the
turn
to
same
F ;\
+
8I
; \
( m
8Qj
specific
;
the
S
( Sjlk
is
gener
)
equations
=
-
mean
:
0
(
16
)
)
marginal
by
( and
Qj
( Sjlk
iteratively
)
.
We
can
optimizing
thus
each
of
mixture
-
Ek
coefficients
ak
log
therefore
CXk
ak
,
true
these
.
We
note
coefficients
for
F
; \
first
that
appear
( Qmix
)
( see
Eq
,
linearly
.
(
14
)
)
,
apart
in
and
we
:
:
F ; \
( Qmix
)
=
L
cxk
( -
E
( k
)
k
where
that
)
)
component
.
the
is
)-
( Sjlk
( Sjlk
particular
assumption
( Qmflk
any
Qj
the
components
model
term
The
: 8Qj
mixture
entropy
B
=
our
(15)
marginal
in
from
)
fitting in
Let
)
the
marginals
follows
efficiently
best
marginals
from
It
( Sjlk
solved
the
.
of
other
( Qmix
8Qj
find
independent
the
constant
-
E
( k
)
-
is
E
the
( k
collection
)
= =
of
F
( Qmflk
terms
)
linear
+
)
-
Lk aklogak +1 in
CXk
(17 )
:
L Qmj(Slk) logR(Slk) s
+
LAmL R(Slm}Qmf (Slk} + logAk m S
(18)
5In the ordinary mean field equations the constant is zero, but for many probability models of interest this slight difference posesno additional difficulties .
IMPROVING THE MEAN FIELD APPROXIMATION
171
Now Eq. (17) has the form of a free energy, and the mixture coefficientsthat maximize the lower bound : FA(Qmix) therefore come from the Boltzmann distri bution : e- E(k) ak=}=:kl e-E(k'). ( This with
fact
respect Finally
Am and ter
is easily
verified
updates
parameters
Lagrange
multipliers
to optimize
Eq . ( 17 )
to am ) .
, we must the
using
(19)
optimize
parameters
the
bound
of the smoothing
in Appendix Am , we readily
A . Differentiating obtain
with
respect
distribution the
bound
to the
parameters
. We discuss with
respect
the lat to the
:
;\m= EkakEsRam (Slm )Qmf (Slk )'
( 20 )
where a simplified form for the expressionin the denominator is presented in the Appendix . 6. Discussion We have presenteda general methodology for using mixture distributions in approximate probabilistic calculations. We have shown that the bound on the likelihood resulting from mixture -basedapproximations is composedof two terms, one a convexcombination of simple factorized mean field bounds and the other the mutual information between the mixture labels and the hidden variables S. It is the latter term which representsthe improvement available from the use of mixture models. The basic mixture approximation utilizes parameters am (the mixing proportions ), Qi (Silm ) (the parametersof the factorized component distri butions). These parameters are fit via Eq. (19) and Eq. (16), respectively. We also introduced variational parameters Am and Ri (Si 1m) to provide tight approximations to the mutual information . Updates for these parameters were presented in Eq. (20) and Eq. (29). Bishop, et al. (1998) have presentedempirical results using the mixture approximation . In experiments with random sigmoid belief networks, they showed that the mixture approximation provides a better approximation than simple factorized mean field bounds, and moreover that the approximation improves monotonically with the number of components in the mixture . An interesting direction for further research would involve the study of hybrids that combine the use of the mixture approximation with exact methods. Much as in the caseof the structured mean field approximation of
172
TOMMI S. JAAKKOLAAND MICHAELI. JORDAN
Sauland Jordan(1996), it shouldbe possibleto utilize approximationsthat identify core substructures that are treated exactly , while the interactions between
these substructures
are treated
via the mixture
approach . This
III
would provide a flexible model for dealing with a variety of forms of high order interactions , while remaining tractable computationally .
II
References
Bishop, C., Lawrence, N ., Jaakkola, T . S., & Jordan, M . I . Approximating posterior in belief networks using mixtures . In M . I . Jordan , M . J . Kearns , & S. A . III
distributions
Solla, Advances in Neural Information ProcessingSystems 10, MIT Press, Cambridge MA (1998) . Ghahramani , Z . & Jordan , M . I . Factorial
Hidden Markov models . In D . S. Touretzky ,
II
M . C. Mozer, & M . E. Hasselmo (Eds.) , Advances in Neural Information Processing Systems 8, MIT Press, Cambridge MA (1996) . Neal , R . Connectionist
learning
of belief networks , Artificial
Intelligence , 56: 71- 113
(1992) . Parisi , G. Statistical Field Theory. Addison-Wesley: Redwood City (1988) . Peterson , C . & Anderson , J . R . A mean field theory learning algorithm
for neural net -
works. Complex Systems 1:995- 1019 (1987) . Saul , L . K ., Jaakkola , T . S., & Jordan , M . I . Mean field theory for sigmoid belief networks . Journal of Artificial Intelligence Research, 4:61- 76 ( 1996) . Saul , L . K . & Jordan , M . I . Exploiting tractable substructures in intractable networks .
In D . S. Touretzky , M . C. Mozer, & M . E. Hasselmo (Eds.) , Advances in Neural Information Processing Systems 8, MIT Press, Cambridge MA (1996). Shwe
, M . A . , Middleton
, B . , Heckerman
P., & Cooper , G . F . Probabilistic
, D . E . , Henrion
, M . , Horvitz
diagnosis using a reformulation
, E . J . , Lehmann
, H .
of the INTERNIST
-
l / QMR knowledge base. Meth. Inform . Med. 30: 241-255 (1991).
A . Optimization of the smoothing distribution Let usfirst adoptthe followingnotation: 7rR ,Q(m, m') :=
Ls R(Slm )Qmf (Slm ')
(21)
II L Ri(Silm )Qi(Silm ') i Si H(Q IIRim)
H(m)
LSQmj (81m )logR(8Im ) L L Qi(8ilm )logRi(Silm ) i Si - Lam m logam
(22) (23)
(24) (25)
IMPROVINGTHE MEANFIELDAPPROXIMATION
173
wherewe haveevaluatedthe sumsoverS by makinguseof the assumption that R(Slm) factorizes.We denotethe marginalsof thesefactorizedforms by Ri(Siim). Eq. (11) now readsas: IA(m; S) = L amH(Q II Rim ) + H (m) m - ~ Am[~ Q(m') 7rR ,Q(m, m/)] + ~ amlogAm+ ~26)
Tooptimize thebound IA(m;S) with respect toR,weneed thefollowing derivatives : ~a-Rj (Q(IISjlk Ri~_ )) (27 ) ) l = 8k ,mQj Rj((Sjlk Sjlk ,Q n ~."Si ) --{}{}7rR R SJI .,Ikm )') = 8k,mQj (Sj Im')1 .#J. L .;Ri(SiIm)Qi(SiIm') .(28 J.((m Denoting the optimal
the product
in Eq . ( 28 ) by 7r~ ,Q ( m , m ' ) , we can now characterize
R via the consistency
equations
I ,X((m ; 81} = Q k 2j a Rj Sjlk Rj ( Sjlk }) - Ak [ ~ Note
that
the
second
term
does
:
Q (m , ) ~ R. ,Q ( k , m , ) Qj ( Sjlm ' ) ] = 0 ( 29 ) not
depend
on the
smoothing
Rj ( Sjlk ) and therefore these consistency equations are easily any specific marginal . The best fitting R is found by iteratively ing each of its not necessarily
marginal solved for optimiz -
marginals . We note finally that the iterative solutions normalized . This is of no consequence , however , since
information bound of Eq . ( 11 ) - once maximized invariant to the normalization of R .
over the A parameters
are the -
is
INTRODUCTION
TO
MONTE
CARLO
METHODS
D .J .C . MACKAY
Department of Physics , Cambridge University . Cavendish Laboratory , Madingley Road, Cambridge , CB3 OHE. United Kingdom .
ABSTRACT
This chapter describes a sequence of Monte Carlo methods : impor tance sampling , rejection sampling , the Metropolis method , and Gibbs sampling . For each method , we discuss whether the method is expected to be useful for high- dimensional problems such as arise in inference with graphical models . After the methods have been described , the terminology of Markov chain Monte Carlo methods is presented . The chapter
concludes
with
a discussion
of advanced methods , includ -
ing methods for reducing random walk behaviour . For details of Monte Carlo methods , theorems and proofs and a full
list of references, the reader is directed to Neal (1993) , Gilks, Richardson and Spiegelhalter (1996) , and Tanner (1996). 1. The problems The aims of Monte
to be solved Carlo methods
are to solve one or both of the following
problems .
Problem 1: to generatesamples{x(r)}~=l from a givenprobability distribution p (x ) .l Problem 2 : to estimate expectations of functions under this distribution , for example
==/>(x))=JdNx P(x)>(x).
(1)
1Please note that I will use the word "sample" in the following sense: a sample from a distribution P (x ) is a single realization x whose probability distribution is P (x ) . This contrasts with the alternative usage in statistics , where "sample" refers to a collection of realizations { x } . 175
176 The
D.J.C. MACKAY probability
might
distribution
be
a
arising
in
model
from
data
' s
that
P
distribution
modelling
-
parameters
x
is
given
an
N
sometimes
-
will
solved
{
x
,
(
r
}
=
l
can
to
which
example
,
call
first
target
.
density
conditional
,
distribution
posterior
real
also
the
a
the
data
with
the
the
will
or
observed
the
solve
give
we
physics
spaces
on
we
~
,
vector
discrete
then
)
)
some
concentrate
it
ples
x
for
dimensional
consider
We
(
statistical
probability
We
will
of
generally
components
a
assume
Xn
,
but
we
will
.
problem
(
second
sampling
)
problem
by
,
because
using
if
the
we
have
random
sam
-
estimator
< i >
.
=
i
L
< I > (
x
(
r
)
)
.
(
2
)
r
Clearly
.if. .
tion
of
< i >
the
< 1 >
will
vectors
is
< 1 >
.
{
Also
decrease
,
as
x
(
as
~
,
is
one
of
The
the
accuracy
of
of
x
to
We
,
it
may
be
find
given
1
.
,
We
will
can
can
< I >
If
The
we
can
(
at
x
)
is
goes
as
first
is
to
that
a
that
we
)
j
>
(
x
)
x
)
~
.
-
< 1
2
dozen
expecta
-
variance
of
.
(
Carlo
methods
(
t
the
the
< / > ,
Monte
So
then
,
estimate
he
regardless
(
sam
of
3
)
.
equation
space
independent
high
pled
the
2
.
)
)
is
To
be
dimensionality
samples
{
P
x
(
(
x
obtain
typically
)
*
,
(
x
)
r
)
}
suffice
(
x
)
(
why
-
a
wish
to
multiplicative
draw
samples
constant
,
;
that
P
" is
(
x
,
)
,
we
that
)
can
=
=
P
*
we
samples
do
we
a
x
dif
from
?
which
such
other
samples
.
HARD
within
cause
independent
easy
from
to
can
Obtaining
not
density
least
*
dimensionality
.
often
P
P
x
of
as
,
function
difficult
(
(
increases
of
of
methods
the
evaluate
general
variance
Carlo
P
in
the
P
FROM
,
a
is
P
R
.
that
evaluate
2
dNx
few
SAMPLING
evaluated
from
samples
dimensionality
Carlo
assume
be
generated
Monte
he
as
P
IS
are
of
J
however
Monte
WHY
=
the
t
that
distribution
.
1
satisfactorily
later
for
=
of
< I >
will
=
0 '
2
variance
estimate
ficulties
1
the
~
number
properties
of
,
}
important
independent
precise
)
where
0 '
This
r
the
not
from
not
know
(
x
)
/
Z
.
(
easily
P
solve
(
the
x
)
?
normalizing
z ==JdNXp*(x).
problem
There
1
are
two
?
Why
difficulties
is
4
)
it
.
constant
(5)
178
D.J.C. MACKAY
N ? Let us concentrate on the initial
cost of evaluating Z . To compute
Z (equation (7)) we have to visit every point in the space. In figure Ib there are 50 uniformly spaced points in one dimension . If our system had N dimensions , N = 1000 say, then the corresponding number of points would be 501000, an unimaginable number of evaluations of P * . Even if each component xn only took two discrete values, the number of evaluations of P * would be 21000, a number that is still horribly huge, equal to the fourth power of the number of particles
in the universe .
One system with 21000states is a collection of 1000 spins , for example ,
a 30 X 30 fragment of an Ising model (or 'Boltzmann machine' or 'Markov field') (Yeomans 1992) whose probability distribution is proportional to P* (x ) -= exp [- ,sE(x )]
(9)
where Xn E { :i:1} and
E(x)==- [~LJmnXmXn +LHnxn ]. (10 ) m
, n
n
The energy function E (x ) is readily evaluated for any x . But if we wish to evaluate this function at all states x , the computer 21000 function evaluations .
time required
would be
The Ising model is a simple model which has been around for a long time ,
but the task of generating samplesfrom the distribution P (x ) = P* (x )jZ is still an active research area as evidenced by the work of Propp and Wilson
(1996). 1 .2 .
UNIFORM
SAMPLING
Having agreed that we cannot visit every location x in the state space, we
might consider trying to solve the second problem (estimating the expec-
tation of a function <J>(x)) by drawingrandomsamples{x(r)}~=l uniformly from the state spaceand evaluating P* (x ) at those points. Then we could introduce ZR , defined by
R ZR =rL=lp*(x(r)), and estimate < I>==JdN x< />(x)P(x)by p * ( ( r ) < i>==rLR < / > ( x ( r ) ) --;=l R.
(11)
(12)
MONTECARLOMETHODS Is
anything
and
wrong
with
P * ( x ) . Let
and
us
concentrate
is
often
this
on
that
the
set
Shannon
T
,
in
whose
- Gibbs
? Well ( x )
nature
concentrated
typical
strategy
assume
of
a
given
the
depends
benign
region
is
of
, it
a
P * (x ) .
small
volume
entropy
is
by
179
on
the
, smoothly
A
high
- dimensional
of
the
state
ITI
~
probability
2H
functions
<j >( x )
varying
distribution
space
(X ) ,
function
known
where
distribution
P
H
as ( X
)
its
is
the
(x ) ,
1
H
If
almost
is
a
all
the
benign
by
sampling the
the
is
the
2N
of
of
a
set
,
what
Ising H
is
H
model
max
=
N
sampling
?
the
once
is
. So
set
of
bits
so be
a
not
is
melts
temperature
set
of
is
of
of
for particles
the
study
, if
unlikely
1 .3 .
1000
in
the Ising
the to
to
Under
these for
phase Ising
model of
, <1> .
a is
at
disordered
But
N
required
high are
which
phase
roughly
samples
to
uniform
interesting
temperature to
an
tends
estimating more
critical
of
entropy conditions
Considerably
the
number
the
/ 2
.
bits
.
simply
)
to
the At
this
For
this
hit
the
order
is
about
universe
.
models
useful
~
1015
2N
- N / 2 =
, This
Thus
of
distribution be
chance
not
( 15
the
sampling
size
( x ) is
/ 2
roughly
uniform
modest P
is
2N
.
And
actually
square is
in
utterly
most
uniform
of
high
the
)
number
useless
for
- dimensional
, uniform
sampling
.
OVERVIEW
Having bution
==
of
problems is
N
a
required
distribution
and
.
as
Rmin
which
Let
space
has
samples
probability
technique
an
?
state
( 14
the
1 .
ordered
the
of
we hit
required the
sample
number
if to
- H .
,
interest
an
entropy
once
2N
order
such
distribution
typical
of
each
of
likely
are
size
uniform
of are
samples total
. SO
. So
estimate we
<jJ ( x )
principally
set
that
distribution
great
from
the
probability
~
satisfactory
of
temperatures model
2H
and
be
typical
good
many
set
will
)
order
uniform
Rmin
well are
intermediate
a
a
The
temperatures
to ,
may
temperatures
Ising
high
.
typical (x )
the
. The
size
set
the
large
, how
has
typical thus
in
giving
again
( 13
<jJ ( x ) P on
of
.
in
fdNx
takes
model
typical
At
located
~
sufficiently
times
Ising
in
tends
is =
Rmin
So
( x ) log2
chance R
of
the the
P
4> ( x )
samples
falling
typical
of
that
number of
, and
/ 2N
the
a
~
value
stand of
case
states
2H
hit
only
set
take
=
mass
the
values
number
typical
us
,
the
will
make
)
probability
function
determined
( X
established P
(x )
that
== P * ( x ) / Z
drawing is
difficult
samples even
from if
a
P * ( x ) is
high easy
- dimensional to
evaluate
distri , we
will
180
D.J.C. MACKAY
Q * (x ),' I I I " ~ ,
x
Figure 2. Functions involved in importance sampling. We wish to estimate the expectation of (x ) under P (x ) cx : P *(x ) . We can generate samples from the simpler distribution Q(x ) cx : Q*(x ). We can evaluate Q* and P * at any point .
now study a sequenceof Monte Carlo methods: importance sampling , rejection sampling , the Metropolis method , and Gibbs sampling . 2. 1mportance sam pIing Importance sampling is not a method for generating samples from P (x ) (problem 1) ; it is just a method for estimating the expectation of a func tion (x ) (problem 2) . It can be viewed as a generalization of the uniform sampling method . For illustrative purposes, let us imagine that the target distribution is a one- dimensional density P (x ) . It is assumed that we are able to evaluate this density , at least to within a multiplicative constant ; thus we can evaluate a function P * (x ) such that
P(x) = P* (x)/ Z.
(16)
But P (x ) is too complicated a function for us to be able to sample from it directly . We now assume that we have a simpler density Q (x ) which we can evaluate to within a multiplicative constant (that is, we can evaluate Q * (x ) , where Q (x ) = Q * (x ) / ZQ) , and from which we can generate samples. An example of the functions P * , Q * and is shown in figure 2. We call Q the sampler density . In importance sampling , we generate R samples { x (r)} ~=l from Q (x ) . If these points were samples from P (x ) then we could estimate ~ by equation (2) . But when we generate samples from Q , values of x where Q (x ) is greater than P (x ) will be over- represented in this estimator , and points
MONTECARLOMETHODS - 6 .2
- 6 .2
- 6 .4
- 6 .4
- 6 .6
- 6 .6
-6 .8
- 6 .8
-7
-7
- 7 .2
181
- 7 .2 10
1 00
1 000
1 0000
1 00000
1 000000
10
(a)
1 00
1 000
1 0000
1 00000
1 000000
(b)
Figure 9. Importance sampling in action: a) using a Gaussian sampler density; b) using a Cauchy sampler density. Horizontal axis showsnumber of samples on a log scale. Vertical " axis shows the estimate cI>. The horizontal line indicates the true value of
where Q (x) is less than P (x) will be under- represented. To take into account the fact that we have sampled from the wrong distribution , we introduce 'weights' - p * (x(r))
Wr= (J;{; (;)) which we use to adjust the ' importance
(17)
' of each point in our estimator
thus :
= l : r Wr(x(r)) . -
(18)
l : r Wr
If Q (x) is non- zero for all x where P (x) is non- zero, it can be proved that , . .
the estimator converges to <1>, the mean value of 4>(x ) , as R increases. A practical difficulty estimate
how
reliable
the
with importance sampling is that it is hard to "
estimator
is . The
"
variance
of is hard
to
estimate, becausethe empirical variancesof the quantities wr and wr4>(x(r)) are not necessarily a good guide to the true variances of the numerator and
denominator in equation (18). If the proposal density Q(x) is small in a region where 14 >(x) P* (x) I is large then it is quite possible, even after many points x (r ) have been generated , that none of them will have fallen in that region . This leads to an estimate of that is drastically wrong , and no indication in the empirical variance that the true variance of the estimator is large . , . .
MONTECARLOMETHODS
183
will fall outside Rp and will have weight zero.] Then we know that most samples from Q will have a value of Q that lies in the range
(217ra2 )N /2exp -/2-:N (-2 N:f:-V :").
(2)2
Thus the weights Wr = P*/ Q will typically have values in the range
(27r0 '2)N/2exp (~2:i:~2) .
(23)
So if we draw a hundred samples, what will the typical range of weights be? We can roughly estimate the ratio of the largest weight to the median weight by doubling the standard deviation in equation (23) . The largest weight and the median weight will typically be in the ratio :
wmax r ~ =exp (J2N ).
In
N
==
1000
samples
is
Thus very
likely
to
an
importance
likely
be
utterly
conclusion
,
In from
two
the
dimensions
set
the , still
of
P ,
Rejection
We
assume
again
a
function have
that
we
( within
a
multiplicative
by
a few
, we
clearly
this
may
, even
of
a
we
obtain
in
order
long
are
a typical exp
samples
time
unless in
to
set
.
vary
Q
by
, although
suffers
that
the
.
will
weights often
samples likely
weight
problem
huge
obtain
hundred
median
dimensions
to
take if
the
with
high
need
one
- dimensional
samples in
after
than
a high
samples
points of
from
. We
one
- dimensional for
us
is
lie a
good
typical large
in
set
,
factors
similar
to
,
each
( . . ; N) .
be
proposal
factor further
density
to
a simpler
ZQ
able
before
that
we
== P * ( x ) / Z
sample
density
, as
assume
P (x ) to
from
it
Q ( x ) which ) , and
know
which the
we we
value
of
that
is
directly can can
too . We
evaluate generate
a constant
c
that for
A
greater
for
those
factors
a
assume
such
with
by
times
weight
sampling
complicated
samples
and
probabilities
largest
sampling
P . Second
differ
3 .
1019
dominated
associated
because other
, the
estimate
importance . First
to
weights
roughly
sampling
approximation the
be
difficulties
typical
therefore
(24)
schematic We
proposal
picture
generate density
of
two
the
random
Q ( x ) . We
all
x , cQ * ( x ) >
two
functions numbers
then
evaluate
is . The
P * (x ) .
( 25 )
shown
in
first
, x , is
cQ * ( x ) and
figure
4a .
generated
generate
from a uniformly
the
MONTECARLOMETHODS
185
-
-4
Figure
c
5
such
.
A
that
As
a
tions
(
x
case
P
)
~
P
(
other
.
no
[
aQ
c
if
the
/
(
(
N
/
and
,
N
the
rejection
the
acceptance
volume
=
=
1000
under
c
grows
)
,
,
is
a
2
)
N
/
2
(
27r0
'
Q ~
)
N
/
2
=
create
(
a
x
1
.
scaled
up
by
a
01
,
/
we
.
The
=
the
our
the
. ]
N
c
c
?
0 ' ~
=
=
exp
(
The
Q
are
technique
x
)
,
useful
for
)
at
single
is
and
similar
of
the
10
density
P
)
~
20
,
000
.
What
:
curve
P
(
x
)
implies
is
1
/
20
,
.
Q
x
(
)
x
.
)
In
that
large
has
complex
this
property
to
the
In
the
general
,
.
method
only
and
)
since
that
000
26
will
immediate
the
for
generating
sampling
(
is
(
is
this
c
origin
one
-
samples
dimensional
from
high
.
rejection
to
is
value
.
normalized
study
a
the
there
)
under
N
whilst
Q
x
,
set
answer
volume
and
(
to
than
case
is
Q
need
two
larger
the
what
of
we
log
case
,
,
from
these
percent
not
So
with
samples
that
one
is
dimensionality
therefore
(
is
density
find
P
For
P
x
exp
of
that
c
aQ
obtain
-
one
method
)
factor
distribu
assume
this
all
P
=
of
fact
1
for
value
practical
sampling
Q
)
from
to
us
,
(
ratio
the
Metropolis
Importance
density
x
Gaussian
Let
say
bound
'
distributions
The
(
samples
if
?
-
27r0
the
,
not
1000
this
be
sampling
problems
to
x
Q
dimensional
ape
-
(
with
dimensional
.
(
exponentially
Rejection
4
is
will
4
sampling
P
=
=
-
because
upper
fur
N
value
bounds
N
~
be
cQ
rate
3
generating
ap
to
-
and
rate
acceptance
of
Imagine
in
-
cQ
rate
2
Gaussian
is
close
upper
for
1
rejection
than
C
With
.
using
larger
so
pair
)
deviation
cQ
2
0
broader
a
is
)
-1
slightly
5
are
be
that
~
a
figure
aQ
dimensionality
27ra
and
standard
must
such
)
consider
deviations
ap
x
-2
.
zero
whose
standard
)
,
mean
deviation
the
(
x
study
with
standard
1
Gaussian
cQ
-3
work
well
problems
.
if
the
it
proposal
is
difficult
-
186
D.J.C. MACKAY
r"\Q(x;X(l))
. . . . . .
. . . I . .
.
.
.
.
.
.
.
.
.
I
I
.
.
.
.
.
.
.
,
.
,
.
.
.
.
.
'
. "
.
.
"
.
.
'
.
-
-
, -
, -
-
X(l )
x
---.
Figure 6. Metropolis method in one dimension. The proposal distribution Q(x ' ; x ) is here shown as having a shape that changes as x changes, though this is not typical of the proposal densities used in practice.
The
Metropolis
which
algorithm
depends
the
on
simplest
the
case
current
is
x
not
of
a
Q
( x
.
for
,
x
whether
'
to
we
( t ) )
in
new
a
~
1
then
the
Otherwise
If
the
step
set
x
is
( t +
l
)
sampling
= =
,
samples
to
,
{
be
label
current
is
a
point
of
( t +
then
( x
' ;
a
in
in
( x
on
density
)
.
An
shows
.
T
' )
It
example
the
density
)
for
any
Q
( x
x
' ;
x
.
A
( t ) )
tentative
.
To
decide
quantity
' )
l
)
= =
x
' .
,
a
Markov
( t ) )
the
no
.
factor
.
.
.
the
.
( x
we
rejection
the
list
current
It
R
to
( t )
label
of
state
is
important
to
T
are
;
to
x
unity
' )
/
=
1
note
Q
( x
as
a
and
x
to
( t ) )
. T
that
.
compute
If
the
Gaussian
the
.
.
able
' ;
.
independent
correlated
be
,
that
t
produce
such
points
superscript
need
is
)
then
in
on
the
samples
we
Q
,
:
influence
not
The
and
1
and
chain
(
.
= =
does
.
rejected
causes
density
latter
is
sampling
time
,
a
step
have
distribution
( x
probability
the
r
)
.
rejection
iterations
P
)
28
If
another
a
( t
rejection
and
Here
P
/
x
.
probability
( x
( x
with
from
symmetrical
,
Q
X
accepted
distribution
P
)
superscript
acceptance
simple
( t ) ;
accepted
points
the
of
ratios
density
( t
is
.
states
target
the
probability
the
of
the
compute
( x
x
from
sequence
*
the
( x
Q
might
.
P
Q
( t ) )
fixed
P
figure
( 2 )
x
centred
any
to
density
discarded
used
simulation
from
posal
list
have
samples
the
*
collected
the
I
' )
difference
are
we
onto
:
Metropolis
To
on
that
written
samples
the
}
( X
be
this
x
compute
is
set
the
points
)
independent
to
a
( r
*
state
we
Note
rejected
x
Notation
are
,
( t ) .
can
' ;
Gaussian
similar
evaluate
we
state
new
accepted
x
)
a
density
( x
( 27 p
new
the
x
proposal
,
proposal
Q
a5
;
and
a
-
If
' ;
6
)
can
P a
( l
the
state
of
density
all
figure
we
from
( x
at
x
that
the
The
such
look
states
assume
use
.
Q
to
shown
generated
accept
( t )
density
x
different
is
x
distribution
' ;
is
two
before
state
simple
( x
density
( t ) )
As
new
Q
makes
state
proposal
for
proposal
x
a
The
necessary
' ;
current
be
( t )
instead
the
centred
Metropolis
pro
-
Q (
MONTECARLOMETHODS
187
Figure 7. Metropolis method in two dimensions, showing a traditional proposal density that has a sufficiently small step size ~ that the acceptance frequency will be about 0.5.
method simply involves comparing the value of the target density at the two points . The general algorithm for asymmetric Q , given above, is often called the Metropolis - Hastings algorithm . It can be shown that for any positive Q (that is, any Q such that Q (x ' ; x ) > 0 for all x , x') , as t - + 00, the probability distribution of x (t) tends to P (x ) == P * (x ) / Z . [This statement should not be seen as implying that Q has to assign positive probability to every point x ' - we will discuss examples later where Q (x' ; x ) == 0 for some x , x' ; notice also that we have said nothing about how rapidly the convergence to P (x ) takes place.] The Metropolis method is an example of a ' Markov chain Monte Carlo ' method (abbreviated MCMC ) . In contrast to rejection sampling where the accepted points { x (r)} are independent samples from the desired distribution , Markov chain Monte Carlo methods involve a Markov process in which a sequence of states { x (t)} is generated , each sample x (t) having a probability distribution that depends on the previous value, x (t - 1). Since successive samples are correlated with each other , the Markov chain may have to be run for a considerable time in order to generate samples that are effectively independent samples from P . Just as it was difficult to estimate the variance of an importance sampling estimator , so it is difficult to assess whether a Markov chain Monte Carlo method has 'converged ' , and to quantify how long one has to wait to obtain samples that are effectively independent samples from P .
188
D.J.C. MACKAY
4.1. DEMONSTRATION OF THE METROPOLIS METHOD The Metropolis method is widely used for high- dimensional problems . Many implementations of the Metropolis method employ a proposal distribution with a length scale f that is short relative to the length scale L of the prob able region (figure 7) . A reason for choosing a small length scale is that for most high - dimensional problems , a large random step from a typical point (that is, a sample from P (x )) is very likely to end in a state which hM very low probability ; such steps are unlikely to be accepted. If f is large , movement around the state space will only occur when a transition to a state which has very low probability is actually accepted, or when a large random step chances to land in another probable state . So the rate of progress will be slow , unless small steps are used. The disadvantage of small steps, on the other hand , is that the Metropo lis method will explore the probability distribution by a random walk , and random walks take a long time to get anywhere . Consider a one- dimensional random walk , for example , on each step of which the state moves randomly to the left or to the right with equal probability . After T steps of size f , the state is only likely to have moved a distance about . ; Tf . Recall that the first aim of Monte Carlo sampling is to generate a number of inde pendent samples from the given distribution (a dozen, say) . If the largest length scale of the state space is L , then we have to simulate a random walk Metropolis method for a time T ~ (L / f ) 2 before we can expect to get a sam pIe that is roughly independent of the initial condition - and that 's assuming that every step is accepted: if only a fraction / of the steps are accepted on average, then this time is increased by a factor 1/ / . Rule of thumb : lower bound on number of iterations ora Metropo lis method . If the largest length scale of the space of probable states is L , a Metropolis method whose proposal distribution generates a random walk with step size f must be run for at least T ~ (L / f )2 iterations to obtain an independent sample . This rule of thumb only gives a lower bound ; the situation may be much worse, if , for example , the probability distribution consists of several islands of high probability separated by regions of low probability . To illustrate how slow the exploration of a state space by random walk is, figure 8 shows a simulation of a Metropolis algorithm for generating samples from the distribution :
1 21 x E { a, 1, 2 . . . , 20} P(x) = { 0 otherwise
(29)
MONTECARLOMETHODS
189
~~~III ~ . . - III.-~
(a)
I111111111111111111111 1111 !!1
(b) Metropolis
=1!!llliiilll -
Figure 8. Metropolis method for a toy problem . (a) The state sequence for t = 1 . . . 600 . Horizontal direction = states from 0 to 20; vertical direction = time from 1 to 600 ; the cross bars mark time intervals of duration 50. (b ) Histogram of occupancy of the states after 100, 400 and 1200 iterations . (c) For comparison , histograms resulting when successive points are drawn independently from the target distribution .
D.J.C. MACKAY
190 The proposal distribution is
X '
Q(x';x) = { ~ Because
the
when
0
and
state
end
state
end
x
states
is
in
4 .2 .
METROPOLIS
The
rule
{ O , 1 , 2 , . . . 20 steps
of
thumb
is of
{ un
,
}
in
the
such
that
each
variable
dom
walk
the
directions
with
step
least
need
T T
~
Now comes
( L
how ,
fall
~
from
but sharply
/ /
big if
f
the
) 2
;
iterations
to
.
random of
same
a
lower
step
the sizes
,
case
of
equal
and us
,
to all
umin
be
to will
an
section
a
largest
this
is
and
assumption
, a
by
where
independent
ran
-
effectively
controlled ,
-
adjusted
executing
generate be
loss dis
deviations
f
,
-
spherical
Without
the
Under others
distri
separable
that
taken
previous
obtain
standard
1 . the
time
the
has
to
target
. a
the
applies
is
to
assume
close
a
is
it
on
also
that
that
Let
The
bound
method
distribution in
abolish
the
distribution
of
as
other
iterations
to
using
giving
umax
is
target
just
to
the
we
sample
the
needed ,
here
we
f ) 2 . can
is .
Let
.
required
independent
try
simplest
and
.
about
amax
( amax
,
independently sizes
lengthscale
}
.
both
exploration
direction
probability
evolves
,
target
deviations
acceptance
samples
largest
n
standard
xn
each
{ xn
,
an
visit
hundred
distribution
the
axes
to
an
into
!
proposal
that
the
it
the
in
assume
with
these
independent
will
can
take
effectively
Metropolis
Consider
deviation
we
step
with
systematic
above
a
first
are
four to
A
above reach
DIMENSIONS
walk
and
Thus
states
to
it
is
end thumb
encounter
about
hundred
discussed
.
iterations
only
evolution
the of
The
400
around
HIGH
random
,
different
of
a
Gaussian
aligned
smallest
at
a
standard
generality
tribution
we of
.
four
.
important
get
of
problems
that
Gaussian
could
IN
that
dimensional
bution
}
instead
iterations
is
its of
iterations
does
.
methods
and
rule
100
occur
.
one
first
for it
10
long
the
iteration
21
the
~
How
will
=
=
steps
about
that
'
example
indeed
Carlo
rejections
reach
T
simulating
shows Monte
10
.
540th by
(30)
x
Xo
present
And
METHOD
of
higher
the
example
twenty
number
on
or
to
is
predicts .
generated
space
about
thumb
1
1
,
-
time
iteration
space
place
only
simple
state
of state
behaviour
toy
178th
=
take
a
the
:!:
state
it
distance
in
X
uniform '
the
take
the
rule
takes
This walk
the
is x
does
confirmed on
The
are
Since
)
to in
long
typically
whole
state
samples
?
( x
state
started
How
will
occurs ?
.
20
This
the
end
of
= it
.
traverse
in
8a
that
end
the
was
figure
predicts
P
takes
simulation in
=
distribution
proposal
The shown x
target
the
=
otherwise
too It
be big seems
? -
The
bigger bigger
plausible
it than that
is
,
amin the
the
smaller -
optimal
then
this the must
number
T
acceptance be
be
-
rate similar
to
MONTECARLOMETHODS
X2
191
X2
P(x) ;:.::.::-. -
(a)
Xl
(b) X2
X2
P (x2IXI)
(c)
;:
Xl
(d)
.............. t))
tX X ( + 2 ) (t+ l)X (t) Xl
Xl
Figure 9. Gibbs sampling. (a) The joint density P (x ) from which samples are required. (b) Starting from a state x (t), Xl is sampled from the conditional density P (xllx ~t)). (c) A sample is then made from the conditional density P (x2IxI ) . (d) A couple of iterations of Gibbs sampling.
umin. Strictly , this may not be true ; in special cases where the second small est an is significantly greater than amin, the optimal f may be closer to that second smallest an . But our rough conclusion is this : where simple spherical proposal distributions are used, we will need at least T ~ (am.ax/ umin)2 iterations to obtain an independent sample , where umax and amln are the longest and shortest lengthscales of the target distribution . This is good news and bad news. It is good news because, unlike the cases of rejection sampling and importance sampling , there is no cat &Strophic dependence on the dimensionality N . But it is bad news in that all the same, this quadratic dependence on the lengthscale ratio may force us to make very lengthy simulations . Fortunately , there are methods for suppressing random walks in Monte Carlo simulations , which we will discuss later .
192
D.J.C. MACKAY
5. Gibbs sampling We introduced importance sampling , rejection sampling and the Metropo lis method using one- dimensional examples . Gibbs sampling , also known as the heat bath method, is a method for sampling from distributions over at least two dimensions . It can be viewed as a Metropolis method in which the proposal distri bu tion Q is defined in terms of the conditional distri butions of the joint distribution P (x ) . It is assumed that whilst P (x ) is too complex to draw samples from directly , its conditional distributions P (Xi I{ Xj } jli ) are tractable to work with . For many graphical models (but not all ) these one- dimensional conditional distributions are straightforward to sample from . Conditional distributions that are not of standard form may still be sampled from by adaptive rejection sampling if the conditional distribution satisfies certain convexity properties (Gilks and Wild 1992) . Gibbs sampling is illustrated for a cage with two variables (Xl , X2) = x in figure 9. On each iteration , we start from the current state x (t), and Xl is sampled from the conditional density P (xllx2 ) ' with X2 fixed to x ~t). A sample x2 is then made from the conditional density P (x2IxI ) , using the new value of Xl . This brings us to the new state X(t+l ), and completes the iteration . In the general case of a system with I( variables , a single iteration involves sampling one parameter at a time :
X(t+l) 1 X(t+l) 2 X3 (t+l)
r.....I
(t),X3 (t),...XK (t)} P(XIIX2 I (t+l),X3 (t),...XK (t)} P(X2IXl IXl (t+l),X2 (t+l)'...XK (t)},etc P(X3 .
erty that every proposal is always accepted . Because Gibbs sampling is a Metropolis method , the probability distribution of x (t) tends to P (x ) as t - + 00, as long as P (x ) does not have pathological properties .
5.1. GIBBSSAMPLING IN HIGHDIMENSIONS Gibbs sampling suffers from the same defect as simple Metropolis algorithms - the state space is explored by a random walk , unless a fortuitous parameterization has been chosen which makes the probability distribution P (x ) separable . If , say, two variables x 1 and X2 are strongly correlated , having marginal densities of width L and conditional densities of width f , then it will take at least about (L / f ) 2 iterations to generate an indepen dent sample from the target density . However Gibbs sampling involves no adjustable parameters , so it is an attractive strategy when one wants to get
MONTECARLOMETHODS
193
a model running quickly . An excellent software package, BUGS, is available which makes it easy to set up almost arbitrary probabilistic models and simulate them by Gibbs sampling (Thomas , Spiegelhalter and Gilks 1992) . 6
.
Terminology
We
for
now
spend
method
A
p
a
and
( O )
( x
)
Mar
few
a
transition
is
given
.
construct
The
A
( t +
l
)
( X
.
distribution
7r
chain
is
must
such
( x
)
is
ergodic
the
MetropoliR
initial
( x
' ;
state
probability
x
)
distribution
.
at
the
( t
+
l
)
th
iteration
of
the
T
( x
' ;
x
) p
( t ) ( x
)
.
( 34
)
:
( x
)
is
the
invariant
distribution
of
the
often
( t )
)
convenient
all
density
-
t
= =
f
dNx
7r
( x
)
as
( x
' )
)
.
of
( x
that
t
-
T
which
( x
T
,
construct
of
P
' )
distribution
ergodic
to
B
desired
( X
be
( x
invariant
t
00
by
' ;
x
is
,
,
for
)
7r
( x
)
any
or
( x
' ;
x
)
if
.
p
mixing
T
( O )
( x
)
.
concatenating
( 35
)
( 36
)
( 37
)
simple
satisfy
=
JdNX
B
These
base
( x
' ;
x
)
P
( x
transitions
)
,
need
not
be
individually
.
Many
erty
the
that
an
also
transitions
the
an
JdNX
P
P
for
= =
distribution
p
base
which
.
The
It
by
T
' )
chain
7r
2
specified
of
on
by
the
desired