ASPECTS
OF INDUCTIVE LOGIC
Edited by
JAAKKO HINTIKKA Professor of Philosophy, University of Helsinki and Stanford University
PATRICK SUPPES Professor of Philosophy and Statistics, Stanford University
cp~c
~
~ 1966
NORTH-HOLLAND PUBLISHING COMPANY AMSTERDAM
© North-Holland Publishing Company - Amsterdam - 1966
All rights reserved No part of this book may be reproduced in any form by print, photoprint, microfilm or any other means without written permission from the publisher
PRINTED IN THE NETHERLAN DS
PREFACE
Of the fourteen papers included in this volume, eight were read, in a form not always identical with the present one, at an International Symposium on Confirmation and Information which was held in Helsinki, Finland, from September 30 to October 2, 1965. These are the papers by Black, Tornebohm, Walk, Von Wright, Hintikka, Hintikka and Hilpinen, Hintikka and Pietarinen, and Tuomela. The present volume thus in effect incorporates the proceedings of this symposium. The Helsinki symposium was held under the auspices of the International Union for the History and Philosophy of Science (Division of Logic, Methodology, and Philosophy of Science) and of the Finnish Philosophical Society. It was financially supported by the International Union and by the Finnish Ministry of Education, and helped in other ways by the University of Helsinki. The present volume thus owes an indirect debt to all these institutions. Many of its contributors are undoubtedly also grateful to the other participants ofthe Symposium whose remarks helped to shape their papers. Among these participants, a special mention should perhaps be made of those speakers whose contributions are not published here: Dagfinn Follesdal (Oslo), Yrjo Reenpaa (Helsinki), and Wolfgang Stegmuller (Munich). The Helsinki symposium was partly inspired by the informal seminar on induction which was held at Stanford University during the Spring term of 1965. This seminar was attended by members of the Stanford Philosophy Department and also by Max Black, who at the time was a Fellow of the nearby Center for Advanced Study in the Behavioral Sciences. The earliest form of several of the papers included in this volume was presented at the Stanford Seminar. The papers included in the present volume do not fall within any single approach to induction and to its logic. On the contrary, a wide spectrum of views are represented, not only in the sense that the main allegiance of the different authors to the several well-established schools of thought is often somewhat different, but also in the sense that new approaches are suggested and explored in some of the papers. For instance, Ernest Adams explores the
VI
PREFACE
logic of conditionals in its relation to probability much more systematically than has been done before, Suppes exhibits some unexamined connections between the problems of concept formation and the logic of induction; and Peter Krauss and Dana Scott develop in detail the model theory that arises from assigning probabilities rather than truth values to first-order formulas. In some other papers, the concept of information is brought to bear on the logic of induction in a novel fashion. One of them (by H intikka and Pietarinen) suggests a more hopeful view of the possibility of conceiving of induction in a decision-theoretic spirit as a maximization of certain "epistemic utilities" than earlier attempts in this direction have indicated. In Walk's paper, some "episternic utilities" different from information are studied and related to the concept of information. The extensions of Carnapian techniques and results outlined in Hintikka's paper may bring out a need of modifying the underlying philosophical viewpoint so as to do fuller justice to the critics of Carnap's earlier work. The new points of view frequently enable the authors to put earlier work into a fresh perspective. Thus Suppes appraises critically the relevance of the notion of total evidence to probabilistic inference, while Black surveys the paradoxes of confirmation for which Von Wright and Suppes suggest new treatments. Hintikka and Hilpinen report certain new positive results concerning the possibility of essentially probabilistic rules of acceptance. It is our hope that some of the novel approaches suggested by the different authors, and the results they obtain, will turn out to lead to new and better ways of understanding the subtle and difficult processes of induction. Stanford, California March 1966
THE EDITORS
KNOWLEDGE, ACCEPTANCE, AND INDUCTIVE LOGIC JAAKKO HINTIKKA University of Helsinki, Helsinki, Finland and Stanford University, Stanford, California and RISTO HILPINEN University of Jyvdskyld, Finland
1. According to a widespread philosophical view, most of our knowledge of empirical matters of fact is logically speaking not absolutely certain, but only probable. This feature is perhaps seen best by considering empirical generalizations and statements concerning future events. It is impossible to verify a universally quantified sentence with factual content in a logically conclusive way (except trivially by exhausting the whole universe). Such sentences may be more or less confirmed by available evidence, but not entailed by it. In the same way, singular predictions concerning future events may in some cases be very probable, but usually not absolutely certain. However, most of us are presumably ready to claim that we know the truth of many generalizations as well as the truth of many statements concerning future events. It thus lies close at hand to try to define knowledge in terms of truth and high probability, and many philosophers have in fact tried to do so. In this paper we shall consider some of the difficulties encountered by these attempts, and propose a partial solution to them. In addition, we shall consider the corresponding difficulties which arise in connection with probabilistic rules of acceptance. Other aspects of the interrelation of knowledge, certainty, probability, and entailment will not be taken up here. 2. Roderick M. Chisholm has defined the concept of knowledge in the following way! : "S knows that h is true" means: (i) S accepts h, (Dl) (ii) S has adequate evidence for h, and (iii) h is true. 1
Chisholm [1957] p. 16.
2
JAAKKO HINTIKKA AND RISTO HILPINEN
Part (i) of the definition (01) is ignored here, because it is not essential for those logical aspects of the concept of knowledge in which we are here interested. What we are here interested in are the conditions on which people are justified in making knowledge-claims. Parts (ii) and (iii) of (01) can be said to define "S is in a position to know that h is true" or simply "h is knowable to S". For the sake of simplicity, we shall in the sequel omit the reference to a person S, because it is immaterial for the purposes of our argument. "Having adequate evidence for h" in part (ii) of (01) means of course that there is a body of evidence e that gives h support strong enough to make it acceptable. We express this in short by "Ac(h, e)". The acceptability of h makes it rational to believe that h is true". If evidence e makes h acceptable and, in addition, h is true, its truth is knowable on the basis of e. This will be expressed by" K (h, e)". (01) thus yields another definition ofknowledge: (02) K(h,e)
=df.
Ac(h,e)&h.
When is evidence e adequate for the acceptance of a hypothesis h? As we suggested earlier, it is tempting to require only that e makes h probable enough. "Probable" here refers of course to logical probability, i.e. to a degree of confirmation. According to the usual probabilistic analysis of empirical knowledge, the acceptability of a proposition can thus be defined as follows: (03) Ac(h,e)
=df.
P(h,e) > I-I;.
Probability 1 - I; is supposed to be relatively high, in other words, 0 < I; ~ 0.5. According to Chisholm, I; = 0.5 is sufficient. Similar views have been put forward by Hempel, too '. We shall here leave the question concerning the precise value of I; open. (D3) purports to be a definition of acceptability, and it may be called a putative rule of acceptance for empirical hypotheses. It says that it is reasonable to accept a hypothesis if and only if its degree of confirmation is higher than 1 -I;. The question whether it is reasonable to incorporate such probabilistic rules of acceptance as (D3) into inductive logic has recently been " The concept of acceptability used here is different from the concept used by Chisholm [1957]. According to Chisholm, a hypothesis h is acceptable for S if and only if S does not have adequate evidence for the contradictory of h. We use the expression "h is acceptable on the basis of e" as a synonym for the expression "e gives adequate evidence for h", We might distinguish these two concepts by calling the concept used by us acceptability in the strong sense, whereas Chisholm uses the word "acceptable" in the weak sense. See pp. 8-9. 3 See Chisholm [19571 p, 28 and Hempel [1962] p. ISS.
KNOWLEDGE, ACCEPTANCE, AND INDUCTIVE LOGIC
3
subject to a great deal of discussion. Carnap and many other modern writers on probability and induction, partly influenced by modern statistical decision theory, have argued against their usefulness 4. On the other hand, many philosophers of science have stressed the importance of the tentative acceptance of hypotheses in scientific enquiry e.g. in connection with the "hypothetico-deductive" method 5. We are not going to discuss the importance of rules of acceptance from the point of view of the methodology of science. Instead, we shall consider the possibility of incorporating a probabilistic rule of acceptance into a system of quantitative inductive logic, because such a rule would in our view be very helpful for the purpose of explicating in terms of inductive probabilities such classificatory expressions used by most philosophers as "empirical knowledge", "practical certainty" and "rational belief". 3. In spite of the intuitive plausibility of definitions (02) and (03), they lead to difficulties, as Keith Lehrer and R. C. Sleigh (among others) have recently emphasized 6. These difficulties are connected with two closure conditions that are usually presented as principles of epistemic logic. The conditions in question are: (CKl) If K(h l,e)&K(h 2,e)& ... &K(hk,e) and if I- (h l & h 2 & ... & hk) :::J h a, then also K (h a, e). The condition (CKl) is very natural, because all sentences entailed by a set of true sentences are true, too. In order to know that a proposition is true it should therefore suffice that it be entailed by other propositions that are known to be true, which is just what (CKl) says. Another very obvious condition is (CK2) The set K={hi:K(h i, is logically consistent.
en
(CKl) and (CK2) together say that the set K defined in (CK2) is consistent and logically closed. (CKl) and (CK2) are adopted as epistemic principles 4 According to one school of thought, inductive reasoning about a proposition h should lead, not to its acceptance or rejection, but to the assignment of a credibility-value, i.e. degree of confirmation, to the proposition. By using credibility-values it is possible to determine in the usual decision-theoretic way how one should act in each particular situation to maximize one's expected utility. For this kind of conception of inductive inference, see Carnap [1962] pp. 316-317, and Jeffrey [1956]. 5 See e.g. Kyburg [1965] pp. 301-310, and Popper [1959] e.g, pp. 22, 418-419. 6 The difficulties and contradictions that arise in connection with the attempt to define knowledge in terms of truth and high probability have recently been subject to a great deal of discussion. See e.g, Lehrer [1964] and Sleigh [1964].
4
JAAKKO HINTIKKA AND RISTO HILPINEN
by Roderick M. Chisholm and Richard M. Martin (in the form (CAl)(CA2) given below). They are assumed to hold for the concept of knowledge by Jaakko Hintikka in his book Knowledge and belief? It is easy to see that definitions (D2) and (D3) together with conditions (CK 1) and (CK2) give rise to a contradiction, when used without any restrictions. The contradiction in question is sometimes called the lottery paradox s, In its simplest form it comes about as follows: Suppose that the following sentences are true: h[
(1)
hz
(2)
P(hbe»l-e
(3)
P(hz,e»l-e.
(4)
According to (D2), (D3), and (CKl) sentences (1)-(4) together entail K(h 1 &hz,e).
(5)
Because of definitions (D2) and (D3), (5) entails P(h[&hz,e»
I-e.
(6)
On the other hand, because of the multiplication theorem of probabilities, it is possible that although (1)-(4) are true,
rt», &hz,e) ~ l-e, (7) which contradicts (6). Conditions (CK 1)-(CK2) do not concern the concept of knowledge alone. Carl G. Hempel has put forward similar requirements of consistency and of logical closure as necessary conditions of rationality in the formation of any beliefs 9. Because "h is acceptable" means just that it is reasonable to believe that h is true, Hempel's conditions can be expressed as follows: (CAl) If Ac(hl>e)&Ac{h z,e)& ... &Ac(hk,e) and if f-(h 1 &h z & ... & h k ) =:> h o, then also Ac(h o, e). (CA2) The set A = {hi:Ac(hj,e)} is logically consistent. 7 See Chisholm [1957] p. 13, Martin [1963] pp. 95-101, and Hintikka [1962] chapters 2 and 3. 8 The expression "lottery paradox" has been used by Kyburg [1965] p. 305. The contradiction (1)-(7) is a special case of the lottery paradox. Another form of the contradiction in question is exemplified in formulas (8.1)-(9). 9 Hempel [1962] p. 149.
KNOWLEDGE, ACCEPTANCE, AND INDUCTIVE LOGIC
5
These conditions are very natural and their intuitive justification corresponds closely to the justification of (CKl) and (CK2). In addition, Hempel has stated the requirement of total evidence as third condition of adequacy for the acceptability of hypotheses. This condition creates of course special problems, but in the present discussion it can be supposed to be satisfied 10. It is clear that any purely probabilistic definition of acceptability, e.g. (D3), will contradict (CAl) and (CA2), because of the multiplication theorem of probabilities. For instance, it is possible that the following k + 1 sentences are true: (8.l) P(h I,e»l-e
and
P(h z,e»l-e
(8.2)
P(h k,e) > 1-e
(8.k)
P(~hI v~hz
v ... v~hk>e»
I-e.
(9)
According to (8.1)-(8.k), (9) and (D3), every member of an inconsistent set of sentences {hI' hz, ... , hk> ~hI V ~hz v ... v ~hk} is acceptable, which contradicts (CA2). The contradiction in question will come about, although s is very small, if the number k is sufficiently large. 4. The contradictions discussed in the preceding section show that it is not possible to define acceptability in terms of high probability alone and at the same time require that the concept in question has to satisfy conditions of adequacy as strong as (CAl) and (CA2). Moreover, if we wish to stick to the corresponding epistemic principles (CK1) and (CK2), (D3) will not do as a definition of acceptability. Because of this, Herbert Heidelberger has concluded that it is a mistake to assimilate probability to knowledge, in other words, he has rejected the usual probabilistic analysis of empirical knowledge altogether t-. Henry E. Kyburg has tried to tackle the problem in another way, although perhaps not entirely successfully so far. He has replaced (CAl) and (CA2) by weaker conditions and retained in his system of inductive logic a purely probabilistic rule of acceptance 12. Kyburg's studies are not conHempel 11962] p. 15!. Heidelberger [1963]. 12 Kyburg takes (CAl) to mean not that every logical consequence of the conjunction of the sentences in A belongs to A, but only that every logical consequence of each single element of A belongs to A. He interprets (CA2) in the same way. See Kyburg [1965] p, 307. However, the lottery paradox still arises in the system of inductive logic presented by Kyburg [1961], though in a somewhat peculiar way. See Schick [1963] p. I I.
10
11
6
JAAKKO HINTIKKA AND RISTO HILPINEN
cerned with the concept of knowledge, but with the concept of rational belief. However, it seems to us that although it perhaps is possible to speak of the rationality of beliefs in a way that does not presuppose (CAl) and (CA2), the concept of empirical knowledge in any case. has to satisfy (CKl) and (CK2). Nevertheless, the conclusion drawn by Heidelberger is in our view too hasty. What the contradictions and paradoxes show is only that high probability cannot be a sufficient condition of acceptability, if "acceptability" is understood in the strong sense required by (CAl) and (CA2). As we shall show in the sequel, it is possible in certain interesting cases to incorporate in a system of quantitative inductive logic a rule of acceptance which satisfies (CAl) and (CA2). The rule in question is probabilistic, but not purely probabilistic. It is obtained from (03) by means of a very simple additional condition. Because it fulfils Hempel's requirements, the concept of knowledge defined by means of it and (02) satisfies (CK1) and (CK2). 5. The problem of the acceptability of hypotheses is especially interesting in the case of general propositions, for such propositions provide clear examples of the probabilistic character of empirical knowledge. Because of the rule (CA 1), it will also be possible to justify the acceptability of many singular sentences, if the acceptability of the corresponding generalizations can first be justified. This aspect of the problem is also of special interest from the point of view of the philosophy and methodology of science, where general laws loom especially large. Therefore we shall first consider the acceptability of general sentences, and turn later to singular sentences. Dealing with general propositions by means of a probabilistic rule of acceptance presupposes of course that it is possible to attach probabilities to such sentences in a reasonable way. Carnap's well-known system of inductive logic is of little use here, because according to it all universal sentences with factual content receive negligible degrees of confirmation if evidence does not contain a relatively large part of the individuals in the whole universe. In particular, in infinite domains of individuals Carnap's confirmation function c* gives all generalizations zero probability. In other words, according to c*, no factual generalization concerning an infinite universe has any credibility whatsoever, on the contrary, we ought to believe that there are all possible kinds of individuals in our universe 13. These disadvantages cannot be wholly 13 The same holds for all the other systems in Catnap's A-continuum of inductive methods, except for A = O. See R. Carnap [1950] pp. 570--571; and [1952].
KNOWLEDGE, ACCEPTANCE, AND INDUCTIVE LOGIC
7
eliminated by using the notion of instance-confirmation 14. In discussing the applicability of inductive logic Carnap has stressed its importance for practical decisions. A system based on c* may perhaps be applicable for practical decision-making, because in these decisions it normally suffices to consider only the instance-confirmation of a generalization 15. However, such a system cannot be used as a rational reconstruction of the more theoretical aspects of scientific activity, because of the weakness just mentioned. Jaakko Hintikka has recently constructed a system of inductive logic that seems to give fairly natural degrees of confirmation to general sentences. Our solution of the paradoxes of acceptability makes use of Hintikka's system. It is in principle applicable to all first-order languages. We shall here restrict our attention mainly to the case in which only monadic predicates are used 16. 6. Let us consider a simple language L k that contains k primitive monadic predicates Pi (i= 1,2, ... , k). By means of these predicates and propositional connectives it is possible to define exactly K = 2k different kinds of individuals. These kinds of individuals are specified by certain complex predicates CtAj= 1,2, ... , K), which we shall call attributive constituents or Ct-predicates. In L k they are simply Carnap's Q-predicates in a new guise. By specifying of each attributive constituent Ct , whether it is instantiated or not it is possible to define 2K different constituents. Constituents describe all the different kinds of "possible worlds" that can be specified by means of our monadic predicates, quantifiers, and propositional connectives 17. Let us assume that the whole domain of individuals of which we are speaking in L k contains N individuals. Suppose that we have observed n individuals sampled from the universe in question and that the observed individuals exemplify c different attributive constituents. Let e be a singular sentence that describes our sample. If attributive constituents are numbered in a suitable way, an arbitrary 14 Hintikka has argued that it is not possible to explain our preferences among generalizations by means of the notion of instance-confirmation. See Hintikka [1965a] pp. 274288, especially p. 277. 15 See e.g. Carnap [1950] pp. 571-573. 16 For the system of inductive logic used here, see Hintikka [1965a]. 17 Constituents, attributive constituents, and distributive normal forms have been characterized in greater detail in a number of papers by Hintikka. See e.g, Hintikka [1953], and [1965b] pp. 47-90.
8
JAAKKO HINTTKKA AND RISTO HILPINEN
constituent compatible with evidence e can be written as follows:
(EX)Ct i1(X) & (Ex)Ctjx) & & (x) (Cti,(x) v Ct i2(X) v
& (Ex)Ctic(x) & ... &(Ex)Ctdx) V Ctic(x) v ... v Ctdx)),
(10)
where c;':2 w;':2 K and where Ct.,(x), Cti,(x), ... , Ctic(x) are all the attributive constituents instantiated in our sample. We shall call constituent (10) CWo In Hintikka's system of inductive logic, a priori probabilities are first distributed among the 2K constituents. The probability of each constituent is then divided evenly among the state-descriptions that make the constituent in question true. In the simplest case, we may assume that all constituents have received an equal a priori probability 1/2K • We shall discuss this assumption later, but meanwhile we shall base our calculations on it. A posteriori probabilities, or degrees of confirmation, are given to the constituents by Bayes' well-known formula
(11) where the sum in the denominator is taken over all constituent compatible with the evidence e. For a given value of i, K~i~c, there are (~.:::-;) such constituents c; If the number of those state-descriptions which make the constituent C w true, given the evidence e, is expressed by m(C w ) , and the corresponding number in the absence of any evidence is expressed by M (C w ) , and if all the constituents have an equal a priori probability, the degree of confirmation of C w with respect to e is (12) The case in which the universe in question is infinite is the easiest to deal with, and it also seems the most interesting from the point of view of inductive logic. Hence we shall assume in the sequel that we are considering an infinite (or at any rate very large) universe. In this case (12) becomes (approximately) (13) It is easy to see that the value of (13) is the greater the smaller w is, and that it
assumes its greatest value when W=C. In other words, according to (13), e gives the strongest support to the constituent C; which says that in the
KNOWLEDGE, ACCEPTANCE, AND INDUCTIVE LOGIC
9
whole universe there exist only such kinds of individuals as are already exemplified in our evidence e. This result is very plausible from the intuitive point of view, at least if n is large in relation to K. According to (13) the degree of confirmation of Ce is
P(Ce,e) =
Ij{1 + ~f(K ~ c)(_c .)n}. + 1=1
C
I
I
(14)
Because every consistent general sentence of L k can be transformed into a distributive normal form, viz. into a disjunction of constituents, the degree of confirmation of any general sentence h is obtained as the sum of the probabilities of the constituents in its normal form, i.e,
P(h,e) = IP(Clhl,e),
(15)
i
where "C?)" denotes an arbitrary constituent which occurs in the normal form of h. What formulas (13) and (14) say can also be expressed as follows: Prior to any evidence, each of the 2K constituents of L k had an equal probability 1(2K • Evidence e changes these probabilities in the following way: All constituents incompatible with e now have zero for their a posteriori probability. All constituents of the kind described in (10), with c;;2;w;;2;K, are compatible with e and have thus positive degrees of confirmation with respect to it. Of these constituents, the most highly confirmed is the constituent C; that says that in the whole universe there are only such kinds of individuals as are already instantiated in experience. In addition to this, it is easy to see from formulas (13) and (14) that p( Co e) grows when n grows and that the probabilities of all the constituents C w with w> c become correspondingly smaller. Moreover, when n grows without limit, p(C e , e) approaches one, and the probabilities of all the other constituents compatible with e approach the value zero. We might express these results in a concise form as follows: (Ll) lim P (C w , e) = 0 (Ll) entails (L2), because
L
when
w
>
C•
P(C w,e)=1.
c~w~K
As in (11), there are in (16) for each w exactly (~=~) equal terms.
(L2)
(16)
10
JAAKKO HINTIKKA AND RISTO HILPINEN
The two lemmas (Ll) and (L2) concerning the behavior of Hintikka's confirmation function are important for our argument, because they imply that the value of P (Cc' e) can be raised arbitrarily close to one by making n large enough. In other words, if we let n become sufficiently large, say n > no, inequality (17) where 0no. According to (14), (17) is logically equivalent to
Ij{1 + ~£C(K ~ c)(~)n} > 1-8. .=1
I
C+ I
(18)
(18) can be expressed in a simpler way if we define a new constant 8':
,
(04)
1 -1, 1-8
8 =df.--
i.e,
1 1-8=--. 1 +/;'
(19)
In virtue of definition (04), (18) and therefore also (17) are equivalent to (20) As soon as n is large enough to make (20) true, (17) holds, too, because of the equivalence of (17) and (20). According to (20), the critical value no depends only on the values of K and c, provided the value of 8 and thus also of 8' is fixed. A specific value of K is a characteristic of the language L k used in our generalizations, but the value of c depends on the number of attributive constituents which happen to be exemplified in evidence e. The critical value of n can be made independent of the specific value of c by replacing (20) by a stronger condition
/;' > max C
K-C(K-. c)(--.)nJ . [I + C
i= 1
I
C
I
(21)
In other words, the critical value of n is computed by using that value of c, o~ c ~ K -1, which makes the right-hand side of (20) as large as possible. (21) entails (20), and therefore it entails (17), too. We now define the critical value of n as follows:
KNOWLEDGE, ACCEPTANCE, AND INDUCTIVE LOGIC
(05) no =
df.
11
the largest integer n for which B
f
~ max e
I-e(K-. c)(--.)nJ . [K + C
i= 1
I
I
C
In virtue of (05), (21) can be expressed in a very simple way: n > no.
(C.Ac)
The acceptability of a general sentence of the language L k can now be defined in the following way: (O.Ac) Ac(h,e)
=df.
(i) P(h,e) > I-B, where 0 < B ~ 0.5 (ii) n > no.
(O.Ac) is a probabilistic definition of acceptability, but it is not purely probabilistic. According to part (ii) of (O.Ac), the number n of individuals included in the sample described by e must exceed no in order for e to make h acceptable. Our definition (O.Ac) is easily shown to satisfy Hempel's conditions (CAl) and (CA2). The proof is as follows: Because of (05), part (ii) of (O.Ac) is equivalent to (21). (21) entails (17), and therefore (O.Ac) entails (17). Moreover, because of (16), (17) entails
I
w>e
Because we have assumed that
I
w>e
P ( C w- e) <
(22)
B.
0
entails
P(Cw,e) < I-B.
(23)
In other words, part (ii) of (O.Ac) assures that one and only one constituent, viz. the constituent Ce , has a degree of confirmation higher than 1 -B, i.e. (L3) If and only ifn>no, Ac(Cw , e) holds for one and only one constituent. This constituent is Ce • Because of (23), not even the disjunction of all constituents other than C; has a degree of confirmation higherthan 1- B, if part (ii) of(O.Ac) is satisfied, on the contrary, according to (22) it is always less than B. Thus (15) and (23) together entail another condition concerning the acceptability of hypotheses: (L4) Acih, e) holds if and only if the constituent C; occurs in the distributive normal form of h, and Ac(Ce , e). (L4) makes it easy to prove that our definition (O.Ac) satisfies Hempel's conditions (CAl) and (CA2):
12
JAAKKO HINTIKKA AND RISTO HILPINEN
(i) According to (L4), every member of any set of sentences acceptable on the basis of e has in its normal form the constituent C e , where C; is itself acceptable. Now obviously any consequence of such a set of sentences has C; in its distributive normal form. Because Ce is acceptable, this arbitrary consequence of our set of acceptable sentences is according to (L4) acceptable, too. (CAl) is thus satisfied. (ii) According to (L4), every member of the set A= {hi:Ac(h;, e)} has in its distributive normal form the acceptable constituent Ce • This obviously implies that they are all jointly consistent, i.e. that A is consistent. (CA2) is thus satisfied. As we mentioned earlier, as soon as the definition of acceptability satisfies Hempel's conditions, the concept of knowledge defined by (02) satisfies the corresponding epistemic principles (CKl) and (CK2). Because our definition of acceptability (O.Ac) satisfies (CAL) and (CA2), the concept of knowledge defined by means of (02) and (O.Ac) conforms to the principles (CKl) and (CK2), and all the contradictions discussed in section 3 are avoided. E.g. sentences (24) K (hi' e) K (hz, e)
(25)
are according to (02) equivalent to
Ac(h t , e) Sch ,
(26)
Ac(hz,e)&h z
(27)
and entail according to (O.Ac) and (L4), The constituent C; occurs in the distributive normal form of both h , and hz, and p(C c , e) > 1- 8.
(28)
Because of (28), it is clear that C c occurs in the distributive normal form of h] & hz . We therefore have according to (16) and (28) P(h]&hz,e»
1-8.
(29)
Now (26), (27), (29) and (O.Ac) together with (02) entail K(h] &hz,e).
(30)
In other words, the contradiction exemplified by formulas (1)-(8) will not arise any more. In the same way, the consistency of the set of acceptable
KNOWLEDGE, ACCEPTANCE, AND INDUCTIVE LOGIC
13
statements entails the consistency of the set of knowable statements. To sum up: In the system of inductive logic developed by Hintikka it is possible to define the concept of empirical knowledge for general (closed) statements along the usual probabilistic lines without violating the principles (CKl) and (CK2). (D.Ac) gives us a rule of acceptance for factual generalizations. According to it, it is reasonable to accept a general hypothesis if it is probable enough and if its high probability is due to strong factual evidence in the sense of a sufficient number of observations of individuals. It is possible that a hypothesis h should have a very high probability although e is based on few observations, provided that h has a relatively low informational content, i.e. provided that almost all of the 2K constituents occur in the distributive normal form of h. Then h is not acceptable, however, for a high probability is not a sufficient condition of acceptability, as we have seen. In the preceding discussion we assumed that a priori probabilities are distributed evenly among the constituents. This assumption was adopted because it led to very simple formulas. As Hintikka has pointed out, however, an even distribution of a priori probabilities leads to considerably overoptimistic numerical values of p(Cc , e) for small values of n 18. This shortcoming affects also our argument, especially our definition (05). The critical number no seems in many cases unrealistically small from the intuitive point of view. This weakness can be easily removed by assigning different a priori probabilities to different constituents. There are of course many ways of doing this. One rather natural method is to make the a priori probability of C w proportional to (wi Kt According to this method, the a priori probabilities of constituents are proportional to the «th power of the number of attributive constituents that are, according to the constituent in question, instantiated in the universe. By giving different values to the parameter a it is possible to specify how uneven the distribution of a priori probabilities is. Different values of a define different inductive methods as far as inductive generalizations are concerned. In this way we obtain acontinuum of inductive methods that is in certain respects similar to Carnap's A-continuum 19. In this a-continuum we obtain instead of the definition (05) Hintikka [l965a] pp.284-285. Hintikka has recently constructed a two-dimensional continuum of inductive methods with two parameters, a and A., where A. corresponds to Carnap's A.. For this two-dimensional continuum, see Hintikka's article A two-dimensional continuum of inductive methods in the present volume. The results of the present paper are established for some special cases of this a-A.-continuum, but they can be extended to the whole continuum. 18
19
14
JAAKKO HINTlKKA AND RISTO HILPINEN
(050:) no =df. the largest integer n for which 8' ;;:;
max C
K-C(K L _ . C)( --. )n-il C
i= 1
I
C
+I
Formulas (13), (14), (18), (20) and (21) should be modified in the same way. The effect of (050:) is that it makes the critical number no larger than (D5), depending on the choice of the value of 0:. All our theorems concerning Hintikka's confirmation function and the concept of acceptability remain valid for all finite values of 0:, however. Our argument in section 6 concerned the special case in which 0: = 0, but our main result is independent of this restriction. Although all our principal results hold for all finite values of 0:, the case in which 0: is large has certain interesting features that do not appear when 0: = o. If 0: is large, constituent CK has a very high a priori probability, i.e. it is possible that (31) Instead of using (O.Ac), we could try to define acceptability in the following way: (O.Ac+) Ac(h,e)
=df.
(i) P(h,e) > 1-8 (ii) there is a constituent C w ' 0;;:; w;;:; K, such that P(Cw,e) > 1-8.
Rule (O.Ac+) would satisfy Hempel's conditions just as well as (O.Ac). In the case 0: = 0 (O.Ac) and (D.Ac +) are equivalent. But if 0: is large enough to make (31) true, it is possible that a hypothesis is acceptable although n is very small or zero. In this case the hypothesis in question would be acceptable because of the high a priori probability of C K • However, (O.Ac +) will lead to implausible and unreasonable consequences. If a constituent has a high degree of confirmation on the basis of a very small number of observations, the constituent in question can be only C K . In other words, only C K could according to (C.Ac+) be acceptable if evidence e contains few or no observations. C K says that individuals of all the possible kinds exist in our universe. This implies that no contingent general law, i.e. no general implication (x) (R (x):» Q(x)) which is not logically true, holds in the universe in question. If a scientist accepted CK when it has high probability, viz. when n is very small or zero, he would prior to any investigation accept the view that no general implications with factual content hold in our universe. Such a procedure would of course be eminently
15
KNOWLEDGE, ACCEPTANCE, AND INDUCTIVE LOGIC
pessimistic, because general laws often are just what scientists are looking for. Thus we have to reject the rule (D.Ac+). 8. So far we have considered only the acceptability of general hypotheses. According to the principle (CAl), all singular hypotheses that are substitution instances of acceptable generalizations are acceptable, too. In this section we shall inquire whether there are other kinds of singular propositions that can be accepted without contradicting Hempel's conditions. The singular sentence e that describes our evidence is of course trivially acceptable, because the acceptability of factual hypotheses is decided on the basis of e. In other words, all observational reports that are used in testing hypotheses are assumed to be true. Our simple language L k forces us here into a considerable oversimplification, because it is not possible to discuss here the reliability of measurements and other questions concerning the acceptability of observation reports. This simplification is not perhaps too serious, however. When we speak of singular hypotheses, we are in the first place interested in singular propositions concerning unobserved individuals. Let us consider a singular hypothesis "A (ai)", i.e. "An unobserved individual ai is A", where A is an arbitrary primitive or complex predicate of the language L k • Any predicate A of L k can be transformed into a disjunction of attributive constituents. We shall call such a disjunction "the normal form of A". The number of Ct-predicates that occur in the normal form of A is called "the logical width of the predicate A". If "A (a,)" is a substitution instance of an acceptable generalization, i.e. if "(x)A(x)" is acceptable, the normal form of A contains at least all the attributive constituents Ct., Ct i 2, ... , Ct., that are already exemplified in the evidence described bye, and possibly other Cz-predicates as well. The probability that an unobserved individual a, will exemplify an attributive constituent Clij' given the evidence e, is according to Bayes' formula
LP(Cj)P [(Ctij(ai) & e), CJ P(Clij(a;),e) = j L ( ) -(j
P
c,
P
(32)
«c;
where the sums include all constituents Cj compatible with e. In the system of inductive logic with IX = 0 and N = 00 which was described above, the probability that a, will exemplify an attributive constituent Ct., (1 ~j ~ c) that is already exemplified in the evidence e is according to (32)
P(Ctij(ai),e)
=
K-C(K L . i=O
I
c)(--: 1 )n+ ljK-C(K c)( 1 )n L . -. C
+I
1=0
I
C
+I
(33)
16
JAAKKO HINTIKKA AND RISTO HILPINEN
Accordingly, the probability that a, will exemplify the predicate Ac(x)= ci., (x) v Cli2(x) V ... V Clic (x) is peA (a.) C
I'
e) =
e.K£C(K - e)(_1 )n+ljK£C(K - e)(_1 )n i=O
i
e+i
i=O
i
e+i
(34)
It is easy to see that for large values of n the value of (33) is approximately 1/ c. When n grows without limit, the value of (34) approaches one. Conversely, the probability that a, will exemplify an attributive constituent different from the c attributive constituents already exemplified in the evidence e differs only very slightly from zero when n is large in comparison with K. Suppose now that "(x)A (x)" is an acceptable generalization. In this case the normal form of A includes the attributive constituents Clil, Cli,' ... , Clic plus possibly other attributive constituents as well. However, if the generalization in question is acceptable, the number n must be fairly large, and consequently the probability that a, will exemplify an attributive constituent Ct, _ with j > c is negligible. Therefore we shall assume, for the sake of simplicity, that A (x) is logically equivalent to Ct., (x) v Ct., (x) V ... v Cldx), and that the logical width of A is therefore equal to c. To indicate this we shall attach the subscript "c" to "A". In this case the degree of confirmation of the prediction "Ac(a;)" is expressed by (34). If "(x) Ac(x)" is acceptable, we ought according to (CAl) to accept not only the prediction that a specified individual, say at, is A c ' but also the corresponding prediction concerning any unobserved individual a.. Consequently, we ought according to (CAl) to accept the proposition that any number r of unexamined individuals will exemplify A c ' The probability of such a hypothesis is given by
P[AcCat)&Ac(az)&··· &Ac(ar),e] =
When r grows without limit, (35) approaches the value
Irt:(K~e)C:iY (36) is equal to P (C c '
(36)
e). In other words,
(L5) lim P [Ac(a 1) &A c(a2) & ... &Ac(a r), e] = P(C c, e). r---"
00
In fact, when r becomes infinite, (35) says just the same thing as Cc , viz. that there are in the whole universe only such kinds of individuals as are already
17
KNOWLEDGE, ACCEPTANCE, AND INDUCTIVE LOGIC
exemplified in the evidence e. If "(x)Ac(x)" is an acceptable generalization, ric; e) is according to (D.Ac) and (L4) higher than I-e. Therefore the acceptance of "Ac(ai)" will not lead to the lottery paradox, because the conjunction of any number of such propositions has always a degree of confirmation higher than 1 - e. What happens if "A (a i) " is not a substitution instance of an acceptable generalization? We shall consider again a situation in which n is large enough to make (C.Ac) true. If"A(ai)" is not a substitution instance of an acceptable generalization, although n is large, not all attributive constituents Ct., l-;i;j-;i; c, occur in the normal form of A. Instead, A may be equivalent to, say, the disjunction of c-d such Ct-predicates. It is of course possible that the normal form of A contains also Ct-predicates not exemplified in the evidence e, but such Ct-predicates can be left out of consideration for the same reason as in the previous case. In other words, let us assume that the logical width of A is c- d, where l-;i; d < c, and A (a;) is logically equivalent to Cti.(a;) v ... v Ctic_d(a;)' The degree of confirmation of the prediction in question is
P(AC-d(aJ,e) = (c -
K-C(K _ c)(
d)·.I
.
1=0
Z
1
-.
)n+ljK-C(K _ c)( .L
C+Z
.
1=0
1
-.
Z
C+Z
)n
(37)
Given suitable values of e, c and d, it is possible that
p(Ac-d(ai),e) > 1- e.
(38)
In other words, it is possible that a singular hypothesis may have a very high degree of confirmation although it is not a substitution instance of an acceptable generalization. (38) may hold again, not only for a specific individual, say aI' but for any unobserved individual. However, the degree of confirmation of an arbitrarily long conjunction of such singular hypotheses is never higher than I-e. The probability of such an r-termed conjunction is
P [(Ac-ial) &Ac-d(a Z) & ... &Ac-d(a,»), e] =
, K-C(K _. =(c-d)'I ;=0
Z
c)(-1.)n+'jK-c(K -. c)(-1.)n I c
+ Z
i=O
Z
C +
Z
(39)
(39) is easily seen to approach zero when r grows without limit. The acceptance of Ac-d(a;) will give rise to the lottery paradox, because it is possible that (40.1) p(AC-d(al),e) > 1 - s , (40.2) P(Ac-d(aZ),e) > 1 - s ,
P(AC-d(a,),e) > 1 -
E,
(40.r)
18
JAAKKO HINTIKKA AND RISTO HILPINEN
but, however,
In fact, (39) shows that the value of (41) may be pushed arbitrarily close to zero by choosing r large enough. The formulas (40.1)-(41) violate Hempel's conditions. Since similar results are forthcoming always when A (a;) is not a substitution instance of an acceptable generalization, we are led to define the acceptability of singular hypotheses in the following way: (O.Ac.sing) A singular hypothesis A (a;) is acceptable if and only if the generalization (x)A (x) is acceptable. The system of inductive logic which was used in the preceding argument is not particularly well-suited for the study of the probabilities of singular predictions. As formula (33) shows, it does not take into account the observed relative frequencies of the different kinds of individuals. It makes only the rough distinction whether there are observed individuals that exemplify a given attributive constituent or not. The degree of confirmation of the prediction "Ctij(aJ" is thus independent of the number of individuals exemplifying Ct ij in the evidence e, provided that this number is ~ 1. In the system of inductive logic in question, one does not learn anything from observed relative frequencies. Our principal results are, however, independent of the weakness just mentioned. Hintikka [l965c] has sketched another system in which these shortcomings are corrected. In this system, the degree of confirmation of a singular prediction "Ctij(ai)" depends on the number n j of individuals that have exemplified Ct., in the evidence e. Also in this system the satisfiability of Hempel's conditions (CAl) and (CA2) can be warranted only by the definition (O.Ac.sing) 20. The definition (O.Ac.sing) shows that Hempel's conditions are very strong indeed. Their strength does not, however, seem quite unreasonable from the point of view of the concept of knowledge. Because (O.Ac.sing) satisfies Hempel's conditions (CAl) and (CA2), it also satisfies the corresponding epistemic principles (CKl) and (CK2). According to (02) and (O.Ac.sing), it is illegitimate to use the expression "knowledge" of singular propositions concerning unobserved events unless they can be deduced from acceptable generalizations, a doctrine well-known from the history of philosophy. The concept of knowledge defined by (02), (O.Ac) and (O.Ac.sing) recalls e.g. "n For this system, see Hintikka [1965c] pp. 21-30.
KNOWLEDGE, ACCEPTANCE, AND INDUCTIVE LOGIC
19
the Aristotelian dictum that one can only have knowledge of the general 21. 9. Our results concerning the acceptability of hypotheses are established for a simple language L, only. Is it possible to generalize these results to richer languages? It is possible to characterize and define attributive constituents, constituents, and distributive normal forms in the whole first-order logic, and in suitable higher-order logics. However, it is much more difficult to extend the principles of inductive logic which we have used in such a way that we can see clearly their consequences. It is especially difficult to construct for a language covering the whole first-order logic a system of inductive logic that would assign probabilities to general sentences in a reasonable way. Independently of the richness of the language considered, if it were possible to show in the language in question that the degree of confirmation of one constituent approaches asymptotically one when more and more individuals are examined, results similar to those obtained in the present paper would follow. This is, however, very questionable as regards to the full first-order logic. If there were a finite number of constituents such that the sum of their degrees of confirmation approaches one when n grows, it would also be possible to obtain positive results comparable to those obtained in this paper. In this case, however, the definition of acceptability would be somewhat more complicated than (D.Ac) or (D.Ac.sing). The generalizability of our results thus remains an open question. To some extent even the relevant questions remain unasked. Up to the present time it has been possible to deal with inductive generalizations in languages more complex than L k by means of rules of acceptance, but not by means of a quantitative concept of confirmation 22. Therefore the problem of the relation between probability and acceptability simply has not yet come up in the case of richer languages 23.
21 This does not of course imply that there are no important differences between the concept of knowledge defined by means of (D2), (D.Ac) and (D.Ac.sing) and the Aristotelian conception of knowledge. 22 For systems of inductive logic based on rules of acceptance, see e.g. Kemeny [1953] and Putnam [1963]. 23 This study has been supported by a grant from the Finnish State Council for Humanities iValtion humanistinen toimikunta). The contribution of the senior author has been facilitated by a Finnish State Fellowship (Va/tion apuraha varttuneille tieteenharjoittajillei. The work has been carried out independently by the junior author on the basis of suggestions from the senior author.
20
JAAKKO HINTIKKA AND RISTO HILPINEN
References CARNAP, R., 1950, The logical foundations of probability (University of Chicago Press, Chicago; second edition, 1963) CARNAP, R., 1952, The continuum of inductive methods (University of Chicago Press, Chicago) CARNAP, R., 1962, The aim of inductive logic, in: Logic, Methodology, and Philosophy of Science, eds. Nagel, Suppes and Tarski (Stanford University Press, Stanford, California) pp. 303-318 CHISHOLM, R. M., 1957, Perceiving: A philosophical study (Cornell University Press, Ithaca, N.Y.) HEIDELBERGER, H., 1963, Knowledge, certainty, andprobability, Inquiry, vol. 6, pp. 245-255 HEMPEL, C. G., 1962, Deductive-nomological versus statistical explanation, in: Minnesota Studies in the Philosophy of Science, vol. 3, eds. H. Feigl and G. Maxwell (University of Minnesota Press, Minneapolis, Minnesota) pp. 98-169 HINTIKKA, J., 1953, Distributive normal forms in the calculus of predicates, Acta Philosophica Fennica, vol. 6 HINTIKKA, J., 1962, Knowledge and belief (Cornell University Press, Ithaca, N.Y.) HINTIKKA, J., 1965a, Towards a theory of inductive generalization, in: Proc. 1964 Intern. Congress for Logic, Methodology, and Philosophy of Science, ed. Y. Bar-Hillel (NorthHolland Publ. Co., Amsterdam) pp. 274-288 HINTIKKA, J., 1965b, Distributive normal forms in first-order logic, in: Formal Systems and Recursive Functions, Proc. Eighth Logic Colloquium, Oxford 1963, eds. J. N. Crossley and M. A. E. Dummett (North-Holland Publ. Co., Amsterdam) pp. 47-90 HINTIKKA, J., 1965c, On a combined system of inductive logic, in: Studia logico-rnathematica et philosophica in honorem Rolf Nevanlinna, Acta Philosophica Fennica, vol. 18, pp, 21-30 JEFFREY, R. C.; 1956, Valuation and acceptance of scientific hypothesis, Philosophy of Science, vol. 23, pp. 237-246 KEMENY, J. G., 1953, The use of simplicity in induction, Philosophical Review, vol. 62, pp. 391-408 K YBURG, H. E., 1961, Probability and the logic ofrational belief(Wesleyan University Press, Middletown, Conn.) KYBURG, H. E., 1965, Probability, rationality and a rule of detachment, in: Proc. 1964 Intern. Congress for Logic, Methodology, and Philosophy of Science, ed. Y. Bar-Hillel, (North-Holland Publ. Co., Amsterdam) pp. 301-310 LEHRER, K., 1964, Knowledge and probability, The Journal of Philosophy, vol. 61, pp. 368372 MARTIN, R. M., 1963, Intension and decision (Prentice-Hall, Englewood Cliffs, N. J.) POPPER, K. R., 1959, The logic of scientific discovery (Hutchinson, London) PUTNAM, H., 1963, Degree of confirmation and inductive logic, in: The Philosophy of RudolfCarnap, ed. P. A. Schilpp (Open Court Publ. Co., La Salle, Illinois) pp. 761-783. SCHICK, F., 1963, Consistency and rationality, The Journal of Philosophy, vol. 60, pp. 5-19 SLEIGH, R. C., 1964, A note on knowledge and probability, The Journal of Philosophy, vol. 61, p. 478
CONCEPT FORMATION AND BAYESIAN DECISIONS* PATRICK SUPPES Stanford University, Stanford. California
1. Introduction. The primary aim of this paper is to examine and develop some relations between decision theory and recent work on concept formation by learning theorists. Some of the ground rules of this investigation perhaps need to be stated at the very beginning. Let me first try to make clear how I conceived in a general way the relation between decisions and concept formation. If we examine the structure of decision theory as expounded for example, in the excellent book of Savage [1954], we find that there is really no place for the formation of new concepts by the decision-maker. The theory is conceived in such a way that the decision-maker has a probability distribution over all possible states of the world and a utility function over all possible future histories of the universe. As observations are made or experiments performed by the decision-maker, the information received is brought into his formal decision framework by appropriate modifications of initial probabilities as new conditional probabilities. The important thing I wish to emphasize is that the theory provides no place for the decision-maker to acquire a new concept on the basis of new information received. The theory is static in the sense that it is assumed the decision-maker has a fixed conceptual apparatus available to him throughout time. There are, it seems to me, two important ways in which concept formation enters in the making of actual decisions. The first kind of modification in the decision structure that may be introduced by concept formation is a relatively straightforward refinement or at least modification of the initial partition of the possible states of nature by the consideration of additional concepts. The consideration of these additional concepts is almost always brought about by the reception of a cue or stimulus resulting from some new observation. The essential thing however in this kind of modification is that the concepts newly introduced are already a part of the conceptual apparatus of the decision-
*
The work on this paper was supported by a contract between ARPA, U.S. Department of Defense and the System Development Corporation.
22
PATRICK SUPPES
maker. It is just that he has not been using them to partition the space of the states of nature until a particularly critical, or new, sort of observation was obtained. Because the new concept brought into focus is actually one already known to the decision-maker, it is becoming customary in the psychological literature to call this process concept identification, rather than concept formation, and we shall so in fact refer to it here. Advocates like de Finetti and Savage of Bayesian theory would indeed claim that this first kind of concept formation is already taken care of by considering all the possible states of the world and all possible future histories. From a theoretical standpoint, or at least one theoretical standpoint, there is indeed a good argument to back up this claim. Yet from a more behavioristic viewpoint it is quite unrealistic, for no actual decisionmaker is able in any genuine way to define an a priori distribution over all possible states of the world or a utility function over all future histories. His powers of discrimination and analysis, even in terms of the empirical data available to him, are inadequate to this task. In actual practice the decisionmaker is always operating with what Savage has termed a small-world situation. The decision-maker operates with a fairly small number ofconcepts and the partition of the possible states of nature generated by these concepts. I want to emphasize that it is not necessary that the partition itself be finite for some of the concepts may be conceived by the experimenter as being measured on a continuum. The crucial thing is that the concept space is always finite-dimensional and in fact, the number of dimensions is a relatively small integer. The second way in which concept formation modifies the decision structure is the genuine case of concept formation proper. In this instance the decisionmaker actually forms a concept he did not previously have in his repertoire. Numerous examples of this kind of concept formation are to be found in the learning experience of anyone. It may be rightly claimed that this kind of concept formation is essential to any major advance in science or technology. In the next section] turn to some simple examples of concept identification and attempt to show how they disturb the simple Bayesian picture of decision making. In the following section I consider some common problems besetting Bayesian and stimulus-sampling learning models. In the final section I sketch a possible line of attack on the structural or combinatorial problems facing any theory of concept formation. I also try to show that Bayesian considerations are not central to the most pressing problems of the theory of concept formation, and that no theory of complex problem solving is possible without an approximate solution to these problems.
CONCEPT FORMATION AND BAYESIAN DECISIONS
23
Before embarking on these somewhat detailed considerations, I would like to indicate in a general way how the subject matter of this paper relates to the more standard literature of inductive logic. The most important and also the most subtle point centers around the conception of rational behavior back of the general criteria used to evaluate an inductive logic or procedure. In the case of deductive logic the response is simple and clear. The criteria of soundness and completeness make no allowance for an imperfect or limited knower. The inattention to the obvious finite capacity of any actual knower is a simplifying abstraction that makes the mathematical theory of deductive inference a manageable subject in the tradition of classical mathematics. To a large extent the same sort of simplifying abstraction has been assumed in deductive logic, but with an important difference. No adequate inductive criteria corresponding to the deductive criteria of soundness and completeness are as yet available. Bayesian decision theory provides a possible answer, but certainly not one that is as yet uniformly acceptable. In my own judgment the problem of finding such criteria in inductive logic is not as interesting as in deductive logic, because the finite capacity of the learner (when talking about induction it seems more natural to speak of a learner rather than a knower) is central to the fundamental problem of making an induction from a finite sample. Put another way, problems ofinduction seem continually to run up against massive combinatorial problems that do not play the same essential role in deduction. And once we begin talk about, say, 10 (2)2 possibilities it is natural to ask about the kind oflearner that is going to "look over" possibilities whose number is of this order of magnitude, and then to try to inject some semi-realism into the discussion. Now that it is generally recognized that even the biggest conceivable computers could not attack by brute force methods the combinatorial problems of playing a winning game of chess, for example, the crucial role of concept formation in providing a powerful method of introducing new and necessary structure is more easily made apparent. In addition, the continued concern in the literature of inductive logic with overly simplified, unrealistic problems suggests that there is a useful place for an explicit analysis of why even the relatively powerful Bayesian methods of induction are far too weak to solve most complex problems. From a more general standpoint, then, an objective of this paper is to make a contribution to the analysis of the concept of rationality. The discussions of rationality in the literature of induction or ethics seem to have largely ignored the difficult problems of concept formation that must be faced by any agent that does not have an unlimited memory and unlimited powers of analysis.
24
PATRICK SUPPES
2. Some simple examples of concept identification. To illustrate some of the comparisons I want to draw between a Bayesian approach to information processing and decision making, on the one hand, and psychological models of behavior on the other, I shall begin with an experiment that is really too simple to be described as a concept experiment, but because of its very simplicity will be a satisfactory paradigm for the making of certain initial distinctions. I have in mind a simple paired-associate experiment. The task for the subject is to learn to associate each one of a list of nonsense syllables with an appropriate response. In a typical setup the list might consist of twenty nonsense syllables of the form CVC. The responses are given by pressing one of two keys. On a random basis ten of the syllables are assigned to key 1 and ten to key 2. The subject is shown each nonsense syllable in turn, is asked to make a response, and is then shown the correct response by one of several devices, for example, by the illumination of a small light above the correct key. After the subject has proceeded through the list once, he is taken through the Jist a second time but the order of presentation of the twenty items is randomized. A criterion of learning is set, for example, four times through the list without a mistake. The subject is asked to continue to respond until he satisfies this criterion. The criterion is selected so as to give substantial evidence that the subject has indeed learned the correct association of each stimulus item and its appropriate response - at least this language of association is the one ordinarily used by many psychologists concerned with this type of experiment. Let me describe two simple psychological models for this experiment before discussing the obvious Bayesian model and its defects. The simple stimulus-association model to be applied to the phenomena is the following. The subject begins the experiment by not knowing the arbitrary association established by the experimenter between individual stimuli and the response keys. He is thus in the unconditioned state U. On each trial there is a constant probability c that he will pass from the unconditioned state to the conditioned state C. It is postulated that this probability c is constant over trials and independent of responses on preceding trials. Once the subject has passed into the conditioned state it is also postulated that he remains there for the balance of the experiment. A simple transition matrix for the model, which is a firstorder Markov chain with two states U and C, is the following:
CONCEPT FORMATION AND BAYESIAN DECISIONS
25
To complete the model for the analysis of experimental data it is also necessary to state what the probabilities of response are in the two states U and C. When the subject is in the unconditioned state, it is postulated that there is a guessing probability p of making a correct response, and that this guessing probability is independent of the trial number and the preceding pattern of responses. When the subject is in the conditioned state, the probability of making a correct response is postulated to be 1. The most striking psychological aspect of the stimulus-association model just described is the all-or-none character it postulates for the learning process. The organism responds with a constant guessing probability until the correct conditioning association is established on an all-or-none basis. From that point on he responds correctly with probability 1. This means that for an individual subject the individual learning curve has the following simple appearance.
o
.
»~ p
...- - - - - - - - - -
+' U
~ L
UQ)
-'" OC +,'"
=1Il
~~
o
.0
o
t:
0
•
'-----------------Trials
The important thing to note about this curve is that it is perfectly flat until conditioning occurs and at that point there is a strong discontinuity. The second psychological model is a linear-incremental model that postulates that the probability of making a correct response increases each time the subject is exposed to the stimulus and is shown the correct response. Let P« be the probability of a correct response on trial n, and let qn= 1- POl that is, let qn be the probability of an incorrect response or error on trial n. The simplest way of formulating this model is in terms of q.: It is postulated that the following recursion will describe the course of learning: qn + 1 = aq.. This linear model can be put within the framework of stimulus-association theory in a rather simple way. Instead of postulating that a single stimulus is being sampled and conditioned in connection with each nonsense syllable displayed, it may be postulated that there are a large num ber of stimuli being sampled and conditioned. These are simple and reasonable assumptions
26
PATRICK SUPPES
about sampling and conditioning. As the number of stimuli becomes quite large, the linear model emerges as an asymptotic limit. (For a detailed derivation of the linear model from stimulus-sampling and conditioning assumptions, see Estes and Suppes [1959J.) The learning curve postulated for an individual subject by the linear models looks something like the following .
....v
Ol
L
L
o
VOl
_VI
oC o
>,0-
....
VI
. - Q)
i';LP o
.a C L
a,
o
•
Trials
As is evident enough from the two theoretical learning curves for individual subjects predicted by the two models, there are quite sharp behavioral differences in the predictions of the one-element stimulus-association model and the linear-incremental model. On the other hand, it is worth noting that the matter of discriminating the two models must be approached with some care. For example, the mean learning curve obtained by averaging data over a group of subjects, or a group of subjects and a list of items as well, is precisely the same for the two models. In the linear model it would naturally be written: (1) In the one-element stimulus-association model the same mean learning curve would naturally be written: qn+ t = (1 - c)"-t q . (2) In estimating parameters from behavioral data it is natural to equate Pl and p (or q land q) and, on that basis, the estimate of a will simply be the same as the estimate of 1 - c; there is no behavioral difference between the two models in the prediction of the mean learning curve. On the other hand, perhaps the most striking difference between the two models can be obtained by looking at data prior to the last error, that is, we sum data over subjects and items, but we restrict that summation to response data occurring before
CONCEPT FORMATION AND BAYESIAN DECISIONS
27
the last error on a given subject-item. When data are summed in this fashion, the one-element stimulus-association model predicts the discontinuous learning curve shown above for an individual subject, whereas the linear-incremental model predicts a smooth incremental learning curve. That the data from experiments of this kind favor very much the one-element stimulusassociation models over the linear-incremental models has been shown by a number of experiments (see, e.g., Bower [1961]). Let us now attempt to apply Bayes' theorem in a correspondingly direct way to an analysis of the paired-associate experiment. Without any loss of generality we may restrict the analysis to a single item, that is, to the learning of a single association between a given nonsense syllable and the correct response. Let HI be the hypothesis that the correct response for the single syllable is response 1, and let Hz be defined similarly. It is natural to assume that the a priori probabilities P (HI) and P (Hz) are each a half. (As we shall see, for this simple. situation the particular assumption made about the a priori probabilities is of no real importance.) In the present case, the "evidence events" are easy to describe and amount essentially to a complete confirmation of one of the two hypotheses. Let us define the evidence event E, as the event of being shown that the nonsense syllable is associated with the response i. It should be clear how to define the conditional probability P (Ej!H i) which is called the likelihood of Hi when E, is observed. In the present simple case the likelihoods must either be 0 or 1. The likelihoods are 1 when i=j, and 0 when i=fI We may then compute thea posteriori probabilities P (HiIEj) according to the usual Bayes formula:
Again in the present case the computation of these a posteriori probabilities is simple and immediate. If i = j the a posteriori probability is 1 and if i # j the a posteriori probability is O. Thus,
P(Hzl £1) = 0
P(Hzl £z) = 1.
28
PATRICK SUPPES
Note that the results are independent of the a priori probabilities P (HI) and P (Hz) as long as these probabilities are in the open interval (0,1). In the present case, then, how is the application of a Bayesian approach related to the two psychological models already sketched for the learning process? The answer I think is obvious. The one-element stimulus-association model yields exactly the same predictions as the Bayes model, ifit is assumed that the conditioning parameter c has the value 1, that is, if it is assumed that the subject always learns in one trial a correct association between nonsense syllable and response key. There are too many experiments to need detailed citations here to show that the assumption that c = 1 is not a reasonable one for paired-associate experiments. There is no doubt that the Bayesian model does not provide a good account of actual behavior in these experiments. Its generalization in the form of the one-element model is much more satisfactory. Advocates of a Bayesian approach as the first approximation to actual behavior will be quick to retort that they have in mind the application of the model to situations in which the subject can utilize his full resources of memory and reasoning. Some may wish to point out that indeed c would equal 1 if the subject were permitted to use pencil and paper in the course of the experiment, and simply to write down the correct association between the stimulus and response once it has been shown to him. However, any serious consideration of the general purpose and intention of such a pairedassociate experiment quickly shows that this defense of the Bayesian approach as an explanatory model of actual behavior is not really satisfactory at all. The paired-associate experiment is defined and set up in the manner that it is in order to provide an extremely simple paradigm of learning. The simplicity of that paradigm is destroyed once a subject is permitted such recording devices as a pencil and paper. A scientific hope of such experiments is that an adequate fundamental theory of the learning process can be developed for learning stripped of complicated processes of memory, association and reasoning that are utilized in everyday decisions. If the fundamental theory is genuinely correct, then it will lead ultimately to extensions to more complicated situations including the sort in which the learning problem confronting the subject is not one that he can trivialize by the use of some additional simple devices. Some of my subsequent examples will in fact be instances of this kind of experiment. A second kind of objection that might be offered by Bayesians to a comparison of the three models is that the real purpose of the Bayesian approach is to prescribe a normative course of behavior and not describe actual
CONCEPT FORMA nON AND BAYESIAN DECISIONS
29
behavior. In spite of the persuasiveness of the important distinction between normative and descriptive theories, this argument is too facile by half. The kind of situations which decision-makers are continually confronted with is precisely the kind of situation in which we place the subject. The subject faced with the paired-associate problem could, if he were given paper and pencil, readily and quickly solve the problem, but the point is that he is not given these additional aids. The decision-maker, whether it be an executive faced with a major policy decision, a logistics expert deciding on the next quarter's inventory, or a legislator deciding on how to present a crucial and controversial bill, is in a situation analogous to that of our subject, for the complexities of the one correspond to the simple restrictions of the other. In certain cases, given unlimited budget for computing purposes and unlimited staff to furnish scientific information, it might be possible for the decisionmaker to act rather completely like a Bayesian strategist. When this is not possible, as it usually is not, the decision-maker must make a large number of rough and ready judgments that do not easily fit within the frame of a detailed normative theory. Indeed, a primary aim of this paper is to show by the consideration of several simple examples that the attempt to structure the decision-making process entirely within the Bayesian framework will lead to serious miscalculations about actual performance and, in certain cases, to bad advice on normative performance. I now turn to a first simple example of concept identification. Let us suppose that a subject is to be shown triangles of various sizes, and let us also suppose that the instructions are meant to bias him in the direction of paying attention to size only. We tell him that he is to classify the triangles primarily according to size into Class A or Class B. In actual fact, in addition to having triangles of three different areas, each triangle will have the property of having an angle less than 15° or no angles less than 221°. Let us call the three sizes a, band c and the two angle properties sand t. Suppose we fix that Class A will consist of the triangles with the property a-s and b-t. Class B will then consist of the complement of Class A, that is, of the combinations a-t, b-s, c-s and c-t. I have picked the angle property because it is a feature of triangles that does not have much saliency for untrained subjects, whereas size is ordinarily a highly salient property. With these instructions and in this situation it is quite probable that many subjects would have a Bayesian distribution of prior probabilities that are non-zero only in terms of hypotheses about size. It does not really matter what specific prior probabilities we assume on hypotheses about size. Eliminating the hypothesis that all sizes belong in Class A and also the hypothesis of no sizes in Class A, there are six
30
PATRICK SUPPES
size hypotheses remaining, which we may write in terms of sizes assigned to Class A, as Ha, Hb, Hab, Hc, Hac and Hbc" We may assume a positive probability for each of the six. It is also a condition of the experiment that the six possible types of triangles are presented by the experimenter on an equally likely basis. It is easy to see that the hypotheses Ha, Hb and Hab will each have a probability of two-thirds of being correct. The explicit situation is shown in the following table, where the entry l indicates a correct classification, and 0 an incorrect classification, under each of the six hypotheses for the six types of figures.
Type of figure
A B B A B B
a-s a-t h~s
h-t ('-s c-t
Hypotheses
Ha
Hb
n;
n,
Hac
1 0
1 0 0 1
0 1
L 0
0
0 L 0 1
I 1
1
1
t
t
t
t
0 0 0
t
0 0 0
n.; 0
t
0 1 0 0
In the table the four cases for which each of the three hypotheses HQ' Hb and Hab is correct are indicated. The remaining three hypotheses, namely, H n H"c and H bc , will asymptotically each have a probability O. Within the framework of the six hypotheses the subject can do no better than indifferently select among HQ' Hab and Hb . It is clear from this analysis that from the Bayesian standpoint any subject who begins with his entire prior distribution weighted on the six hypotheses concerned with size alone will not be able to solve the problem completely. It may of course be objected that the assumption made about prior probabilities is not a reasonable one. The issue is complicated and I do not mean to suggest that I think definitive arguments can be given in support of the kind of assumption made. There is, however, a certain amount of evidence both from the behavior of subjects, and interrogation ofthem about their behavior, to show that in experiments in which an ultimately relevant concept has a very small degree of saliency, the subject begins initially by completely ignoring this concept or property. For such situations there would seem to be
CONCEPT FORMATION AND BAYESIAN DECISIONS
31
only a Pick wickean sense in which a strictly positive distribution over hypotheses involving this concept can be postulated. Let us now consider how we would approach the analysis of the subject mastering the problem in terms of some of the ideas of concept formation that have been developed in the last couple of years. The theoretical account I shall give will be somewhat more elaborate than the models much tested in recent experiments on concept formation (as for example, Bourne and Restle [1959], Bower and Trabasso [1964] and Suppes and Ginsberg [1962a], [1962b], [1963]). As the first stage of learning, let us assume that the subject, following the verbal cue given him by the experimenter, samples only the three size stimuli, which we have designated as a, band c. Initially he does not know how each of these stimuli should be connected or associated with Class A or Class B. We may thus postulate that they are in the unconditioned state. When a stimulus presentation is given which permits a sampling of one of the three stimuli, then in terms of the correction procedure given, that is, the statement as to whether or not the figure shown belongs to Class A or Class B, we may postulate a probability c that the size stimulus sampled will become conditioned to one of the two classes, that is, to one of the responses A or B. Notice that we are postulating initially that the subject samples with probability 1 whichever one of the size stimuli is available on a given trial. On this basis, the learning for stimulus c is particularly simple. We may just apply the one-element model described above for paired-associate learning. This stimulus starts in the unconditioned state and with probability c on each occasion on which it is sampled it enters the conditioned state, in this case, conditioning to Class B or Response B. When stimulus c is conditioned to Response B, on every occasion in which Response B is made on the presentation of this stimulus, the classification is proved to be correct, and therefore there are no grounds for the subject's changing or modifying this conditioning. In a complete sense, the conditioning of stimulus c exemplifies, as postulated here, the one-element model for paired-associate learning described above. (The use of 'c' to refer both to the stimulus and its probability of conditioning should not be a source of confusion, as the specific reference intended is always clear from the context.) The situation is considerably more complicated for stimuli a and b. Half the time that stimulus a or b is sampled the presented figure that has size property a or b can be classified in Class A, and the other half of the time, on a random basis, Class B. Intuitively when the subject finds that he cannot use stimulus a or b to make a correct classification, he will be led to sample other
32
PATRICK SUPPES
properties or aspects of the figure. Before he is led to make this additional sampling, there is often one strategy he will try. He may judge that his initial response connection for one of the stimuli was incorrect and he will reverse the association. For example, if on the first occasion that stimulus b is sampled, it turns out that the figure is classified as Class A, but on the second occasion that stimulus b is sampled, the figure is put in Class B by the experimenter, he may reverse the association and not yet be led to sample other stimuli. Let us designate the probability of such a reversal of the association by r, and let us postulate that he will sample a new property with probability s, when it turns out that the association that he has established is wrong. Extending the kind of assumptions that went into the development of the one-element model for paired-associate learning, we may postulate a fourstate Markov process describing this stage of the subject's learning. He begins in state U representing the fact that stimulus a, let us say, is unconditioned. We may pass from state U to either state A or B representing two possible responses to which stimulus a may be conditioned. After reaching state A or B, he will on each trial that a is sampled be incorrect with probability t. The matrix is then constructed so as to postulate that with probability -}s he enters state N, the state in which he samples a new property, and with probability tr he reverses the stimulus association from response A or response B or vice versa as the case may be. It is of course a constraint of the model that r + s;;:; t. The complete matrix then is as follows:
N
B A U
N
B
A
U
t }s
0
0
0 0 0
ts 0
1- 1S - tr tr 1c
tr 1- ts - 1r tc
---_.-
-
l-c
Note that state N is the absorbing state of this chain, because we are postulating that the subject will always be led, on the basis of his failure with the association established for the size stimuli a and b, to the sampling of a new property. It should also be remarked that this matrix represents the situation for stimulus b as well as for stimulus a. We are, as in the fashion of pairedassociate learning, postulating that the process of being led to state N when stimulus a is sampled is statistically independent of the process of being led to state N when stimulus b is sampled. No doubt in actual practice this assumption is probably slightly violated but it makes the quantitative treatment of the concept identification considerably more simple, and is therefore,
CONCEPT FORMATION AND BAYESIAN DECISIONS
33
a desirable feature of a first approximation. For fast learners we may postulate that s = 1 and c = 1. The matrix then assumes the following simple form: N B A U
N
B
A
1
0 2
0 0
1
Z .i.
11-
0
1-
0 Z
1
2
U
-"------
0 0 0 0
In a given experiment more detailed knowledge may be obtained by looking at the actual sequence of presentations of figures and observing their classification. If, for example, the subject was always wrong in classifying a figure with stimulus a in initial trials, he could be in state N on the third trial that a figure with size property a is presented. It may also be noted that we have postulated that in this concept-identification task the subject is learning only on trials on which he makes an error. Bower and Trabasso [1964] present impressive evidence that for the kind of experiment described here this is roughly the situation. I shall have more to say on this point later. Upon entering state N the subject is now in a position to sample a new property. Note the difference from the Bayesian formulation. Up to this point the probability of sampling any property other than a size stimulus has been O. It is only due to the failure of the size stimuli to lead to the correct solution that the subject has been forced to change his initial distribution and sample other properties. Suppose for instance that the subject now samples stimuli connected with the orientation of the base of the triangle. We may suppose that the base of the triangle varies from the horizontal in three different angles, namely 0°, 15° and 30° (these numerical values are taken for purposes of illustration only). We also shall suppose that the occurrence of figures with these respective orientations is randomly assigned independent of other characteristics and therefore any particular orientation will occur in Class A figures approximately half of the time and in Class B figures the other half. Various things can be postulated at this point. We can assume that the subject disregards size stimuli entirely and samples only orientation stimuli, or we can also postulate that he samples a size-orientation pattern combining stimuli exhibiting both properties. For many situations this latter pattern assumption has been shown to be a sound one. But whichever sampling procedure he adopts at this point, that is, concentration only on orientation or pattern sampling of orientation and size together, he will be led to the same results as before and will once again enter state N, and be required to select a new property for sampling.
34
PA TRICK SUPPES
Parenthetically it may be remarked that for those readers who fear that the process of concept identification as described here is too slow to describe what actually takes place, it may be said that it is not difficult to cite experiments in which a large number of trials is required by subjects to master what may appear to an experimenter or an observer with full knowledge of the situation as absurdly simple identification problems. When the number of trials to complete mastery of the problem is on the order of a hundred, many opportunities are presented for sampling different properties of the stimulus display presented. There are two important factors I have ignored in this analysis but which would in all likelihood enhance the rate of learning or the rate of concept identification. One is the factor of memory. When a new property is sampled, in many cases it is sampled and rejected simply on the basis of its ability to account for correct classification of items already seen and whose classification is remembered. On the other hand, it is not an unrealistic assumption to suppose that the transition matrix described above is essentially the sort of one that is used in testing from memory new sampled properties. The experimental difficulty of course is that itis not a simple or direct matter to elicit behavioral data giving evidence on this point. The second related phenomenon that I have ignored is the undoubted fact that when a given property is being sampled and used as a basis of classification it is often the case that simultaneously other properties are being sampled and silently rehearsed, meaning by this that their ability correctly to classify is being noticed even though they are not the properties used by the subject in making his classification on the given trial. Again it is not unreasonable to suppose that the process of rehearsal may be represented by a transition matrix very similar to the one given above. There is considerable indirect experimental evidence of the efficacy of rehearsal from the standpoint of learning. Several experimental studies have shown the positive effects of an increased amount of study time on the rate of learning of paired-associates. As I understand the matter, no simple Bayesian approach to information processing and decision making would take explicit account of these two aspects of concept formation and learning, namely, the effects of memory and rehearsal. It should be emphasized that in another sense memory may be taken account of in Bayesian procedures. Modern empirical Bayes procedures have in many cases been developed on the assumption of a finite memory, but the kind of use of memory suggested here is of a different sort, namely, memory of what happened on preceding trials is used in a new way on trial n to check out the efficacy of a property not considered or sampled prior to
CONCEPT FORMATION AND BAYESIAN DECISIONS
35
trial 11. From the Bayesian standpoint the use of this property on trial n in terms of items from memory would require the assumption that the property or concept had a positive prior distribution on earlier trials. The relatively simple concept-identification problem we have been considering is already beyond the resources of the standard systems of inductive logic, because the subject in the experiment is not told what are the relevant elementary properties. Although in principle inductive logics of the Carnapian variety have a method for handling questions of relevance, in practice they do not deal with the kind of thing that arises in any concept-identification experiment when the subject is not told what is relevant - to pick a very simple example, the relevant aspects might turn out to be relations rather than properties. Also, methods of constructing such logics as matters now stand do not provide any guidelines for enumerating large sets of properties among which the relevant ones are likely to lie. Focusing on concept-identification experiments makes it possible to draw an important distinction between Bayesian theory and the Carnapian sort of inductive logic. To make the standard inductive logic apply it is necessary to codify explicitly in the language of the logic all the evidence of past experience the subject considers pertinent to the experiment, but this I would claim is always a hopeless task. It is difficult enough to narrow the situation down to a manageable set of properties and relations, but it is humanly impossible to layout all the evidence that went into the selection of this set and the beliefs held about its members. To put it in simplest terms, it is at the least the problem of having a limited, finite memory. The Bayesian approach, on the other hand, is not bedeviled by this difficulty, because past experience can be encoded in the a priori distribution over the selected set of properties and relations. Once again, it is a question of a realistic conception of rationality. If we want to explicate the concept of rational human behavior, and not that of omniscient rational behavior, limitations on memory and computing power must be taken seriously. Taking such limitations seriously is of course imperative in attempting to apply an inductive logic. (The fact that these limitations are fundamental is why within the domain of deductive logic the theory of recursive functions is of quite restricted use in theorizing about or applying actual computers.) 3. Some common problems of Bayesian and stimulus-sampling models. The discussion of the last section to a certain extent overemphasizes the differences between Bayesian and stimulus-sampling models for decisions, particularly when the decisions involve concept identification. In the present section I
36
PATRICK SUPPES
want to emphasize some of the commonality between the two kinds of models and to point out some of the problems that beset them both. A particular point of this section is to show that many of the differences often emphasized in discussions of the cognitive or Bayesian approach as opposed to the stimulus-response approach is a difference primarily in terminology, and not so much in something that is sharply defined and empirically observable. A convenient place to begin is with the classical case of a two-choice problem with noncontingent reinforcement. The problem for the subject on each trial is to predict which one of two lights will flash. Using familiar notation, let us call £1 the reinforcing event corresponding to the flashing of the left light and £2 the reinforcing event corresponding to the flashing of the right light. The response that consists of predicting that the left light will flash is designated A 1 and the response that consists of predicting that the right light will flash is designated A 2 • The noncontingency of the situation is defined by making the probability of an £1 reinforcement on each trial equal to tt. and the probability of an £2 reinforcement I-n. It is understood that the events £1 and £2 are mutually exclusive and exhaustive, that is, on each trial exactly one of the two lights flashes, and the probability of which will flash is fixed by the parameter n, Everything that we have to say in what immediately follows applies, mutatis mutandis, to other more complicated reinforcement schedules, but the basic principles are precisely the same. Let us begin by considering some Bayesian models for this situation. In the first place these Bayesian models shall be defined in terms of several sets of hypotheses, and we shall call an exhaustive set of hypotheses, that is a set of hypotheses that covers every contingency, a strategy, It is understood that in the ordinary Bayesian terminology what] am now calling strategies would very often be called hypotheses, but the present language is suggestive of game-theoretic language, as well as of the kind of language that has been used by various people interested in cognitive models of the learning process. For simplicity let us begin with four hypotheses: h, : h i>: h2 : h2 , :
an an an an
£1 £1 £2 £2
reinforcement reinforcement reinforcement reinforcement
is followed is followed is followed is followed
by by by by
an an an an
£1; £2; £2; £1-
Given the above four hypotheses and the fact that hI and hI' (and h 2 and h2 ,) are incompatible, a strategy for the subject consists of believing, or acting as if he believed, one of the following four pairs of hypotheses: (hI' h 2 ) ,
CONCEPT FORMATION AND BAYESIAN DECISIONS
37
(hI' h 2 · ) , (hI" h 2 ) , (hI" h 2 · ) . Thus the strategy (hI' h 2 ) requires that an Al
response will be made if on the preceding trial an £1 reinforcement occurred, and an A 2 response will be made if an £2 reinforcement occurred on the preceding trial. As is apparent from what has already been said, the four strategies correspond to the four hypotheses relevant in the sense of a Bayesian model. Granted only a positive a priori probability for each of the four strategies, it is clear what is the asymptotic prediction of the Bayesian model, that is, what the asymptotic a posteriori probabilities of the strategies will be. Namely, P(h 1,h 2 ) = P(h t , h 2 . ) = P(h 1·,h 2 ) = P(h t.,h 2 ,) =
n(l-n) n2 (1- n? (1- n)n.
The Bayesian decision-maker with unlimited memory will then choose strategy (h t , h2 ,) with probability 1 for n>t. On the face of it this familiar Bayesian result leading to selection of event £1 with probability 1 seems very much in conflict with the standard theoretical results obtained in stimulus-sampling theory, which predicts that an At response will be made as a prediction of an £1 reinforcement with asymptotic probability tt, and we have the well-known matching law, first formulated by W. K. Estes. From what has been said thus far it is easy enough to formulate the stimulus-sampling model with N stimulus elements in the noncontingent situation. On each trial the organism is sampling one stimulus. It becomes conditioned to the response that is reinforced with probability c, and with probability 1- c its conditioning does not change. Among the N stimuli exactly one is sampled on each trial. This sampling takes place on a random basis, that is, there is a probability 1/N of any particular stimulus' being sampled, independent of what else may have occurred on past trials. When a stimulus is sampled the response is made to which that stimulus is conditioned. In terms of these theoretical assumptions, the behavior of the subject may be defined in terms of the parameters c, Nand n. This description of stimulus-sampling theory seems quite different from the Bayesian approach. I now want to show how closely related they actually are, and how easily a formal isomorphism between models of the two theories may be set up. To bring the two together let us first examine the Bayesian model under a highly restricted memory assumption. In particular,
38
PATRICK SUPPES
let us suppose that the subject, although he is a Bayesian, is only able to remember what happened the last time a test of any particular hypothesis in his strategy was made. Whenever the outcome of this test is negative, he immediately changes his strategy by replacing the incorrect hypothesis by the correct one. For example, if his strategy is (hi' hl ) and he finds on trial n that the £1 reinforcement occurring on trial n -1 is followed by £1' he then immediately changes his strategy to (hi" hl ) . In other words, he is making no use of any evidence concerning trial pairs for which an £1 reinforcement is followed by an £1 before trial n. His memory is of minimal length to use any of the evidence relevant to the hypotheses at all. The transition matrix for this Bayesian model with memory of length one is then the following: (hi' hl') (hl,h l,)
(hi' h l) (hl"h z,) (hl"h z)
ti
n(1 - n) n
Z
0
(hi' hl)
(l---;?
1 - 2n(1 - n) 0 n
Z
--
(hl"h l,) n(1 - n)
(hi" h l ) 0
---'--', .... " ..._,,-_.
1 - (n
l
0
+ (1 -
n(1 - n)
n?)
n(1 - n) (1 - n)Z 1-n
It is a simple matter to show that the asymptotic probability of an Al response with this transition matrix and the states as defined above is that of the familiar matching law, namely, tt: Let us now look at a formulation of the analog of the Bayesian model in terms of stimulus-sampling theory. As has already been indicated, its theoretical assumptions are formulated along the following lines. There is available a set of N stimuli which the subject has conditioned or associated with various possible responses. At the beginning of a trial the organism is in a certain state of conditioning. A set, possibly a proper subset of the N stimuli, is presented to him and he samples on a random basis, that is, with a uniform distribution. Exactly one of the stimuli is sampled and then the organism responds in terms of the association bond that stimulus has with one of the possible responses (in case the stimulus sampled is not conditioned to any response a guessing response is made of the kind already described in discussing paired-associate learning). After the response is made, the reinforcement is given and the stimulus sampled changes its conditioning with probability c to the response reinforced, in case the response made was incorrect. To apply these postulates and develop a model corresponding to the Bayesian model we shall assume there are exactly two stimuli. One of them is the £1 reinforcing event occurring on the preceding trial and the other is the
39
CONCEPT FORMATION AND BAYESIAN DECISIONS
£2 event occurring on the preceding trial. Thus, on each trial the subject has available exactly one of the two stimuli to sample, and the whole sampling process is thereby trivialized. Quite apart from this identification of the two stimuli, on the assumption of two stimuli (i.e., N = 2), there are exactly four states of conditioning corresponding to the four possible subsets of stimuli conditioned to the A 1 response. The complement of each subset is the set of elements conditioned to the A 2 response (in the present analysis we shall assume on every trial each stimulus is conditioned to exactly one response and that therefore there are no unconditioned stimuli on any trials). Representing the states of conditioning by the subset of elements conditioned to A 1 we then have the following notation for the four states: {S1' sz}, {S1}' {sz}, 0, where 0 designates the empty set. To show how the assumptions of stimulus-sampling theory are used to derive a transition matrix in terms of these four states, we may draw the tree of possibilities with the probabilities for each branch shown when we begin in a typical state, let us say {sz}.
c
rc : £,
1-c
A2
~S2\
£,=5,
~52t
\ 152:£2=5
152 t
rt :
1-Tt
2
rt
:£,
A, 1-Tt : £2
0
c
1-c
40
PATRICK SUPPES
By drawing trees for the other three states of conditioning to show what possibilities may arise at the end of the trial for each of the three when we start from that state, we may upon completing the trees, collect terms and obtain the following transition matrix:
{SI,S2} {sd {S2}
0
{SI,S2} l-e(l-n) en(1 - n) , en-
--"._--------
0
{sd e (1-----;? 1-2en(l-n) 0
en 2
{S2} en(l - n)
- - - _ .. , - - - - -
0 0
en(1 - n) 1-e(n 2+(1-n)2) e(l - n)2 1- en en(1 - n) 0
Casual inspection shows that this transition matrix is not the same as the one derived for the Bayesian models. On the other hand, if we let c= 1, we obtain the following special case of the stimulus-sampling model:
{S2}_-_ .._ - - - -0- {SI,S2} {SI} - - - n - - - (1- ny----- n(l-n) 0 {SI,S2} n(l - n) 1 - 2n(1 - n) o n(1-n) {SI} 1 - (n 2 + (1 - n)2) (1 _ n)2 n2 0 {S2} o o n(1-n) 1-n - - - - - , ...
And it is immediately apparent that the entries in this matrix are the same as those for the Bayesian model. The identity between the two matrices also suggests, what should already have been apparent, that is, the formal isomorphism between the states of the Bayesian model and the states of the stimulus-sampling model. The following correspondence
{h1,h z·} --+ {SI'SZ} {h 1, h 2 } --+ {sd {h 1·, h 2·} --+ {S2} {h1·,hz}--+O may be used to establish the isomorphism with the restriction that c= 1. Without this restriction the stimulus-sampling model is a slight generalization of the Bayesian one. H may be remarked that the stimulus-sampling model, with c estimated from experimental data, does not appear to fit data very well (for some results in this connection, see Suppes and Atkinson [1960], Ch. 10). Of course, many Bayesians would almost be pleased that the stimulus-sampling model did not fit well for they could say "I would hardly expect the simple Bayesian model you have defined to provide any sort of decent fit to human
CONCEPT FORMATION AND BAYESIAN DECISIONS
41
prediction data". The reply to this is straightforward. This same kind of Bayesian model may easily be extended to memories of finite length greater than 1, but immediately a common problem of the most essential sort for either the Bayesian or stimulus-sampling models arises. The problem is that we very quickly find ourselves in a combinatorial jungle out of which it is not easy to find a path. Consider, for example, the Bayesian model with finite memory of length 4 and again let us concentrate only on the pattern of reinforcement ignoring, although it is unrealistic, the pattern of preceding responses. For a finite memory of length 4 there will be 16 patterns of preceding reinforcements and thus a strategy will consist of a 16-tuple telling the Bayesian what to do when each of the 16 patterns is realized in the preceding 4 4 trials. This means there are (2)2 or (2)16 strategies to consider and thus this many states in the associated Markov process. The stimulus-sampling model with the additional parameter c has the same sort of difficulty. It is tedious but not impossible to obtain some results for models with this number of states. As the number of states increases, it rapidly becomes more difficult. The generalized conditioning models applied to a variety of data in Suppes and Atkinson [1960], Suppes and Schlag-Rey [1962a] and Suppes and Schlag-Rey [1962b] may prove useful in examining in more detail the relationships between Bayesian and stimulus-sampling ideas. The essential idea of these models is to generalize the probability c of conditioning to let the probability of conditioning depend upon preceding responses and reinforcements. For the kind of application discussed particularly in Suppes and Schlag-Rey [1962a], one gets a formulation of conditioning models that is very similar to a kind of probabilistic Bayesian model with finite memory. It would seem to be primarily a choice of language and not of concepts as to how one prefers to describe these models. We described them as conditioning models but it is a simple matter to translate this description into Bayesian language. Some additional remarks about these models are made in the next section. 4. The structural problems besetting the theory of concept formation. In Section 2, I tried to make the point that any simple Bayesian approach to decisions, actions or choices, encounters considerable difficulty in explaining or predicting behavior of human subjects even in simple concept-identification experiments. I also tried to describe there some approaches that seemed promising from the standpoint of mathematical learning theory and, in particular, the version which originates with Estes and is ordinarily called stimulus-sampling theory. In order not to draw the distinction between
42
PATRICK SUPPES
Bayesian models and stimulus-sampling models in too absolute a fashion, in the third section 1 tried to work out some of the formal similarities between the two approaches and in this case I chose for discussion a familiar paradigm in experimental psychology of recent years, namely, the two-choice situation with a noncontingent probabilistic schedule of reinforcement. In discussing various Bayesian models and the formally similar stimulussampling models that may be used to analyze the noncontingent case, 1 tried to sketch some of the combinatorial problems that quickly arise when more complicated and subtle models are considered. To mention these combinatorial models first in connection with the noncontingent case is almost a mistake, for fairly simple stimulus-sampling models of a rather different sort than the kind considered in the preceding section give quite a good account of much data from noncontingent experiments. I have in mind the kind of pattern stimulus-sampling models first discussed in Estes [1959], and also discussed in Suppes and Atkinson [1960] Ch. 10, and Atkinson and Estes [1962]. By considering the standard pattern model of stimulus-sampling theory, it is possible to bypass some of the combinatorial problems [ mentioned that arise for Bayesian models and the particular stimulus-sampling models corresponding to these Bayesian models. The point of this section is to examine empirical situations, or simplified experimental situations roughly corresponding to the empirical situations in which it does not seem possible to avoid these combinatorial problems. It is a fundamental thesis of this paper that it is in dealing with situations in which new concepts must be formed that the standard formulations of the Bayesian approach are most inadequate. To give the discussion some definiteness and concreteness, I shall primarily restrict myself to description of a class of experiments in which the problem facing the subject is to learn the grammar of a set of strings. It is to be emphasized, however, that in dealing with this grammatical example, I think of the problem of concept formation as being of a quite general nature. The difficulties besetting this example, particularly those of a combinatorial nature, apply equally well to any attempt to understand how humans learn to play well a complicated game like chess or make decisions rapidly when confronted with an incredibly wide choice of alternatives. To fix our ideas quite specifically, let us consider initially the thirty-two strings of length five made up of l's and O's. From a formal standpoint we may define a grammar for this set of strings as a subset of the set of thirty-two strings. The number of such grammars is then 2 3 2_2, where we exclude the universal and the empty grammar. Let us suppose that the subject is shown
CONCEPT FORMATION AND BAYESIAN DECISIONS
43
the cards one at a time and is asked to classify them as codes or non-codes, where we think of a code as being a grammatical string and a non-code as being a non-grammatical string. The theoretical problem is now to describe how the subject proceeds to find the correct grammar. A simple Bayesian approach would be to attempt to describe a subject's a priori distribution on the 2 3 2 - 2 possible grammars, and then to change this distribution as information is given to the subject concerning the classification of strings. It is just possible that for strings of length five something can be made of this Bayesian approach. For strings of length seven or eight or for anything approaching the complexity of chess, we must turn to the imposition of a considerable structure on the set of all possible grammars. It is, I would take it, the central problem of a theory of concept formation to provide such a structure and to state the laws by which organisms use the structure to solve the problem confronting them. One way of approaching the problem of characterizing the structure of the space of all grammars is the following. The idea is to express any possible concepts for solving the problem of classification as a point in the space of properties associated with the stimulus material of the problems. A new concept is formed by moving to a new point in the property space. In these terms the theory of concept formation relevant to solving a given set of problems consists of two parts: first, characterizing the appropriate space of properties, and, secondly, characterizing the laws of motion in the space. In terms of the kind of formulation of stimulus-sampling theory considered in earlier sections, it may be thought that the phrase, "laws of motion", is too grandiose, and that what is described are simply the assumptions for sampling properties or stimuli. My reply to this possible objection has already been stated. The usual formulations of sampling assumptions neither assume nor impose any substantial structure on the set of stimuli (or concepts). The point of the present formulation is to impose such a structure. The space of properties is conceived as a multi-dimensional space with each dimension corresponding to a property. (Admittedly in many applications the space will consist of a finite set of points and thus will not satisfy the usual mathematical definition of a multi-dimensional space, but that is not a matter of serious concern here. I shall use the word "dimension" the way it is used in the psychological literature of concept formation and not in a mathematical sense.) It is only after a space is postulated (i.e., a set with a structure), that it is possible to talk about motion in the space. The concept of motion in an arbitrary set with no postulated structure is not well defined. On the other hand, it is precisely the imposition of structure that seems to be necessary to
44
PATRICK SUPPES
bring some order and constraints to the discouragingly large number of possible concepts that may be considered in solving even a relatively simple problem. Once such a structure is imposed, laws of motion for the space, particularly when formulated as laws governing random walks, can be formulated. To illustrate some of the possibilities for constructing the basic properties we may look at the problem when the set of strings is only of length two and, as before, at each position in the string there occurs either a 1 or a O. According to the computations already indicated above it is immediately apparent that there are then 14 possible grammars for the set of four possible strings, excluding as before the universal grammar and the empty grammar. A simple ideographic space for this problem is the four-dimensional one, with one dimension for each card. The value on a given dimension is 1 if that string belongs to the hypothesis, and 0 otherwise. It is then trivial to represent any hypothesis as a point in this four-dimensional space. Such an ideographic space is not too unwieldly when the number of possible strings is small, but as has already been remarked, when this is the case the whole apparatus of a property space and the imposition of structure on this space is scarcely necessary. We may just as well use a straightforward stimulussampling or Bayesian model. A more natural space of properties, which would generalize to longer strings, is the following. Dimension 1 characterizes the first position. The value 1 on the first dimension indicates that symbol 1 must occur in the first position of a string, and the value 2 that symbol 0 must occur in this position. The value 0 on this dimension indicates that either a 1 or a 0 may occur in the first position. The second dimension is defined similarly in terms of occurrence of symbols in the second position of a string. The third dimension is defined in terms of agreement or difference between occurrence of symbols in the two positions of the string. The value 1 in the third dimension is taken to indicate that the symbols occurring in the first and second positions of the string must be the same, the value 2 to indicate that the first and second positions of the string must be occupied by different symbols, and as before, the value 0 that the first and second positions may be occupied by the same or different symbols. We have selected the 0 value for all dimensions to indicate that this dimension is not intuitively relevant to the concepts, hypothesis, or grammar in question. The first thing of course to be noticed about this threedimensional space is that there are a number of points that can be occupied by no concept that is nontrivial. Thus the concept to be represented by the coordinates (1,2, 1) is the empty grammar, for it is not possible for a string to
CONCEPT FORMATION AND BAYESIAN DECISIONS
45
have a 1 in the first position, a 0 in the second position, and yet to have the first and second positions occupied by the same symbol. Property spaces for other grammars or concepts connected with strings of this character indicate that this phenomenon is not easily eliminated. There does not seem to be a natural and simple way of defining orthogonal dimensions, but this does not seem to be an immediately crucial problem. Still another way of looking at the space of properties is in terms of properties of a given string rather than of the grammar of the set of strings. In this case the grammar is represented by a certain subset in the space of properties rather than as a point. Corresponding to the space just constructed a space of this sort is easily described for the strings oflength two, but I shall not go into details, because the present stage of our analysis of these problems, reinforced by some preliminary experimental evidence, indicates that this latter method is not the most desirable theoretical approach. The space [ did describe above for the strings of length two is deceptively simple. The extension of this same kind of description to strings of lengths greater than two soon becomes rather awkward if sufficient dimensions are required to locate with precision any grammar (or concept) in the set of all grammars. From experiments now being undertaken with Madeleine SchlagRey and some related experiments being conducted with elementary-school children in conjunction with Irene Rosenthal, it appears that for purposes of initial simplification of analysis we may in the case of strings of length three reduce the dimensions of the property space to a fairly small number, and lump the remaining unusual and not-likely-to-be-thought-of properties together. To give some rough indications of what we are finding, let me describe briefly the situation. In dealing with strings of length three, with the strings being built up from two symbols, there are 2 8 - 2 = 254 possible nontrivial grammars. We have found, however, that about 80-85 percent of the grammars conjectured by subjects may be classified under six main property headings and so we have restricted the analysis to these six properties together with a catch-all seventh category in which we place the remainder. The main point of our investigation at the present time is to find out to what extent the behavior of subjects in selecting and rejecting grammars (or more generally concepts or hypotheses) may be accounted for in terms of the application of stimulus-sampling models to the seven properties. To use the physical language mentioned earlier we are attempting to characterize the motions of the subjects' changes in concepts or hypotheses in terms of random walks with respect to the most salient properties, as for example, the occurrence of a 0 or 1 in one of the three positions, or the occurrence of a
46
PATRICK SUPPES
matching pair in the first and second, the second and third, or first and third positions. It is too early yet to decide whether or not this particular approach to concept formation will prove to be a fruitful one. Before concluding, I do want to indicate how the generalized conditioning models studied in Suppes and Schlag-Rey [1962a], which were mentioned earlier, have a bearing on finding the properties which are most salient for subjects in structuring their approach to the solution of a problem. The experiment analyzed was one with a probabilistic reinforcement schedule in which the reinforcement in a two-choice situation, on a given trial, depended or was contingent upon the subject's own preceding two responses. We were particularly concerned to analyze the experimental data to find the nature of the patterns to which subjects seem most likely to condition their responses. Basic data examined in the experiment were the conditional probabilities of an Ai response given the reinforcements and responses of the two preceding trials. Ten different models, each postulating that the conditioning of the responses depended on a different pattern, were considered. In Class T of the models the sequential dependence or conditioning was defined in terms of the two physical sides 1 and 2 of the key and light apparatus (of course, for some subjects side 1 was on the left side and for some subjects on the right). The point is that the conditioning parameters in Class I were defined in terms of the side. The five special cases considered in this Class were defined by restricting the dependency of Al,n+l to: (a) the response and reinforcement that occurred on trial n; (b) the two preceding reinforcements; (c) the two preceding responses ; (d) the two preceding reinforcements and the immediately preceding response; (e) the two preceding responses and the immediately preceding reinforcement. In Class II the conditioning parameters were defined, not in terms of the sides 1 and 2, but in terms of successful and unsuccessful responses, rewarding and punishing reinforcements, repetition or alternation responses, etc. In particular the five special cases were defined by: (a) the reinforcement on trial n was punishing or rewarding; (b) the reinforcements on trials n-l and n were punishing or rewarding; (c) the reinforcement on trial n was punishing or rewarding, and the response of trial n indicated anticipation of a repeating or alternating reinforcing event; (d) the reinforcement on trial n was punishing or rewarding, and the response on trial n was a repetition or alternation of the response on trial n -1 ; (e) the reinforcements on trial n-l and n were punishing or rewarding, and the response on trial n was a repetition or alternation of the response on trial n - 1.
CONCEPT FORMATION AND BAYESIAN DECISIONS
47
From the standpoint of the present paper these ten models provided an opportunity for gathering information on the kind of structure subjects tend to impose in such a probabilistic situation. The observed transition probabilities and the goodness-of-fit tests for the ten models of Class I and Class II are given as Table 2 of Suppes and Schlag-Rey [1962a), and will not be reproduced here. The most important observation about the results of the goodness-of-fit tests, however, is that with the same net number of degrees of freedom the fits of Class II models were uniformly better than Class 1. In addition, the assumption that the conditioning can be explained in terms of the last reinforcement's being punishing or rewarding yields a better fit than did any of the Class I assumptions with four parameters. The uniformly better results of the Class II models in comparison with the Class I models supports the hypothesis that subjects are in many cases more likely to sample patterns of stimuli defined in terms of complex relational properties than in terms of relatively concrete single events. Detailed information about the relative saliency of such relation-defined patterns is one of the most important things needed to move ahead with an empirically adequate theory of concept formation. The ideas about concept formation set forth in this paper are meant to be suggestive rather than definitive. I do hope, however, that the various kinds of examples considered present adequate evidence for maintaining that any theory of complex problem solving cannot go far simply on the basis of Bayesian decision notions of information processing. The core ofthe problem is that of developing an adequate psychological theory to describe, analyze and predict the structure imposed by organisms on the bewildering complexities of possible alternatives facing them. The simple concept of an a priori distribution over these alternatives is by no means sufficient and does little toward offering a solution of any complex problem. Moreover, understanding the structures actually used is important not only for an adequate descriptive theory of behavior but also for any normative theory intended to be applicable to human beings with finite powers of memory and computation. As the standard literature of inductive logic comes to grips with more realistic problems the overwhelming combinatorial possibilities that arise in any complex problem will make the need for higherorder structural assumptions self-evident. References ATKINSON,
R. C. and W. K.
ESTES,
1963, Stimulus sampling theory, in: Handbook of
48
PATRICK SUPPES
Mathematical Psychology, vol. 2, eds. R. R. Bush, R. D. Luce and E. Galanter (John Wiley and Sons, Inc., New York) pp. 121-268 BOWER, G. H., 1961, Application of a model to paired-associate learning, Psychornetrika, vol. 26, pp. 255-280 BOWER, G. H. and T. TRABASSO, 1964, Concept identification, in: Studies in Mathematical Psychology, ed. R. C. Atkinson (Stanford University Press, Stanford, California) BOURNE, L. E. and R. RESTLE, 1959, Mathematical theory of concept identification, Psychological Review, vol. 66, pp, 278-296 ESTES, W. K., 1959, Component and pattern models with Markovian interpretations, in: Studies in Mathematical Learning Theory, eds, R. R. Bush and W. K. Estes (Stanford University Press, Stanford, California) pp. 9-52 SAVAGE, L. J., 1954, Foundations of statistics (John Wiley and Sons, Inc., New York) SUPPES, P. and R. C. ATKINSON, 1960, Markov models for multiperson interactions (Stanford University Press, Stanford, California) SUPPES, P. and R. GINSBERG, I 962a, Application ofa stimulus sampling model to children's concept formation with and without an overt correction response, Journal of Experimental Psychology, vol. 63, pp. 330-336 SUPPES, P. and R. GINSBERG, 1962b, Experimental studies ofmathematical concept formation in young children, Science Education, vol. 46, pp. 230-240 SUPPES, P. and R. GINSBERG, 1963, A fundamental property ofali-or-none models, binomial distribution of responses prior to conditioning, with application to concept formation in children, Psychological Review, vol. 70, pp. 139-161 SUPPES, P. and M. SCHLAG-REY, 1962a, Test of some learning models for double contingent reinforcements, Psychological Reports, vol. 10, pp. 259-268 SUPPES, P. and M. SCHLAG-REY, 1962b, Analysis ofsocial conformity in terms ofgeneralized conditioning models. in: Mathematical Methods in Small Group Processes, eds. J. Criswell, H. Solomon and P. Suppes (Stanford University Press, Stanford, California) pp. 334-361
PROBABILISTIC INFERENCE AND THE CONCEPT OF TOTAL EVIDENCE* PATRICK SUPPES Stanford University, Stanford, California
1. Introduction. My purpose is to examine a cluster of issues centering around the so-called statistical syllogism and the concept of total evidence. The kind of paradox that is alleged to arise from uninhibited use of the statistical syllogism is of the following sort. The probability that Jones will live at least fifteen years given that he is now between fifty and sixty years of age is r. Jones is (1) now between fifty and sixty years of age. Therefore, the probability that Jones will live at least fifteen years is r. On the other hand, we also have: The probability that Jones will live at least fifteen years given that he is now between fifty-five and sixty-five years of age is s. (2) Jones is now between fifty-five and sixty-five years of age. Therefore, the probability that Jones will live at least fifteen years is s. The paradox arises from the additional reasonable assertion that ri=s, or more particularly that r>s. The standard resolution of this paradox by Carnap [1950J p. 211, Barker [1957] pp. 76-77, Hempel [1965] p. 399 and others is to appeal to the concept of total evidence. The inferences in question are illegitimate because the total available evidence has not been used in making the inferences. Taking the premises of the two inferences together, we know more about Jones than either inference alleges, namely, that he is between fifty-five and sixty years of age. (Parenthetically I note that if Jones happens to be a personal acquaintance what else we know about him may be beyond imagining, and if we were asked to estimate the probability of his living at least fifteen years we might find it impossible to layout the total evidence that we should according to Carnap et al. use in making our estimation.)
* The writing of this paper has been supported by a grant from the Carnegie Corporation of New York.
50
PATRICK SUPPES
There are at least two good reasons for being suspicious of the appeal to the concept of total evidence. In the first place, we seem in ordinary practice continually to make practical estimates of probabilities, as in forecasting the weather, without explicitly listing the evidence on which the forecast is based. At a deeper often unconscious level the estimations of probabilities involved in most psychomotor tasks - from walking up a flight of stairs to catching a ball - do not seem to satisfy Carnap's injunction that any application of inductive logic must be based on the total evidence available. Or, at the other end of the scale, many actually used procedures for estimating parameters in stochastic processes do not use the total experimental evidence available, just because it is too unwieldy a task (see, e.g., the discussion ofpseudo-maximumlikelihood estimates in Suppes and Atkinson [1960] Ch 2). It might be argued that these differing sorts of practical examples have as a common feature just their deviation from the ideal of total evidence, but their robustness of range, if nothing else, suggests there is something wrong with the idealized applications of inductive logic with an explicit listing of the total evidence as envisioned by Carnap. Secondly, the requirement of total evidence is totally missing in deductive logic. If it is taken seriously, it means that a wholly new principle of a very general sort must be introduced as we pass from deductive to inductive logic. In view of the lack of a sharp distinction between deductive and inductive reasoning in ordinary talk, the introduction of such a wholly new principle should be greeted with considerable suspicion. I begin my critique of the role of the concept of total evidence with a discussion of probabilistic inference. 2. Probabilistic inference. As a point of departure, consider the following inference form:
peA I B) = r PCB) = p peA) ~ rp.
--------_.----------
(3)
In my own judgment (3) expresses the most natural and general rule of detachment in probabilistic inference. (As we shall see shortly, it is often useful to generalize (3) slightly and to express the premises also as inequalities, P(AIB)~r
PCB)_--------~ p peA) ~ rp.
--_
...
(3a)
PROBABILISTIC INFERENCE
51
The application of (3a) considered below is to take r = p = 1- s.) It is easy to show two things about (3); first, that this rule of probabilistic inference is derivable from elementary probability theory (and Carnap's theory of confirmation as well, because a confirmation function c(h, e) satisfies all the elementary properties of conditional probability), and secondly, no contradiction can be derived from two instances of (3) for distinct given events B and C, but they may, as in the case of deductive inference, be combined to yield a complex inference. The derivation of (3) is simple. By the theorem on total probability, or by an elementary direct argument
peA) = peA I B)P(B) + peA I li)P(B) ,
(4)
whence because probabilities are always non-negative, we have at once from the premises that P(AIB) = rand PCB) = p, peA) ~ rp. Secondly, from the four premises
p(AIB) = r PCB) = P p(AIC) = s P(C)=a,
we conclude at once that P (A)~max(rp, sa), and no contradiction results. Moreover, by considering the special case of P (B)=P (C) = 1, we move close to (1) and (2) and may prove that r = s. First we obtain, again by an application of the theorem on total probability and observation of the fact that P (li) = 0 if P (B)= 1, the following inference form as a special case of (3)
p(AIB) = r PCB) = 1 peA) = r.
(5)
The proof that r=s when P (B)=P (C)= 1 is then obvious: (1)
(2)
(3)
(4) (5) (6) (7)
p(AIB) = r PCB) = 1 p(AIC) = s p(c) = 1 peA) = r peA) = s r=s
Premise Premise Premise Premise 1,2 3,4 5,6.
(6)
The proof that r = s seems to fly in the face of statistical syllogisms (1) and (2)
52
PATRICK SUPPES
as differing predictions about Jones. This matter I want to leave aside for the moment and look more carefully at the rule of detachment (3), as well as the more general case of probabilistic inference. For a given probability measure P the validity of (3) is unimpeachable. In view of the completely elementary - indeed, obvious - character of the argument establishing (3) as a rule of detachment, it is in many ways hard to understand why there has been so much controversy over whether a rule of detachment holds in inductive logic. Undoubtedly the source of the controversy lies in the acceptance or rejection of the probability measure P. Without explicit relative frequency data, objectivists with respect to the theory of probability may deny the existence of P, and in similar fashion confirmation theorists may also if the language for describing evidence is not explicitly characterized. On the other hand, for Bayesians like myself, the existence of the measure P is beyond doubt. The measure P is a measure of partial belief, and it is a condition of coherence or rationality on my simultaneously held beliefs that P satisfy the axioms of probability theory (forceful arguments that coherence implies satisfaction of the axioms of probability are to be found in the literature, starting at least with de Finetti [1937]). It is not my aim here to make a general defense of the Bayesian viewpoint, but rather to show how it leads to a sensible and natural approach to the concept of total evidence. On the other hand, I emphasize that much of what I have to say can be accepted by those who are not full-fledged Bayesians. For example, what I have to say about probabilistic inference will be acceptable to anyone who is able to impose a common probability measure on the events or premises in question. F or the context of the present paper the most important thing to emphasize about the rule of detachment (3) is that its application in an argument requires no query as to whether or not the total evidence has been considered. In this respect it has exactly the same status as the rule of detachment in deductive logic. On the other hand it is natural from a logical standpoint to push for a still closer analogue to ordinary deductive logic by considering Boolean operations on events. It is possible to assign probabilities to at least three kinds of entities: sentences, propositions and events. To avoid going back and forth between the sentence-approach of confirmation theory and the event-approach of standard probability theory, I shall use event-language but standard sentential connectives to form terms denoting complex events. For those who do not like the event-language, the events may be thought of as propositions
PROBABILISTIC INFERENCE
53
or elements of an abstract Boolean algebra. In any case, I shan use the language of logical inference to talk about one event implying the other, and so forth. First of all, we define A -+ B, as A v B in terms of Boolean operations on the events A and B. And analogous to (3), we then have, as a second rule of detachment:
P(B-+ A) ~ r P(B) ~ P __ ----_.
-
.-
(7)
:.P(A)~r+p-l.
The proof of (7) uses the general addition law rather than the theorem on total probability:
P(B-+A) = P(B v A) = P(B) + P(A) - P(B &A)
whence, solving for
P(A),
~r,
P(A) ~ r - P(B) + P(B &A) ~r-(1-p) ~r+p-l,
as desired. The general form of (7) does not seem very enlightening, and we may get a better feeling for it if we take the special but important case that we want to claim both premises are known with near certainty, in particular, with probability equal to or greater than 1-E. We then have
P(B-+A) ~ 1 P(B) ~ 1 :.P(A)
E E
(8)
~ 1- 2E.
It is worth noting that the form of the rule of detachment in terms of conditional probabilities does not lead to as much degradation from certainty as does (8), for
p(AIB) ~ 1 - E P(B) ~ 1 - E .'. P(A) ~ (1 - E)2 ,
(9)
and for E > 0, (1- E)2 > 1- 2E. It is useful to have this wen-defined difference between the two forms of detachment, for it is easy, on casual inspection, to think that ordinary-language conditionals can be translated equivalently in terms of conditional probability or in terms of the Boolean operation
54
PATRICK SUPPES
corresponding to material implication. Which is the better choice I shall not pursue here, for application of either rule of inference does not require an auxiliary appeal to a court of total evidence. Consideration of probabilistic rules of inference is not restricted to detachment. What is of interest is that classical sentential rules of inference naturally fall into two classes, those for which the probability of the conclusion is less than that of the individual premises, and those for which this degradation in degree of certainty does not occur. Tollendo ponens, tollendo tollens, the rule of adjunction (forming the conjunction), and the hypothetical syllogism all lead to a lower bound of 1- 2e for the probability of the conclusion given that each of the two premises is assigned a probability of at least 1- e. The rules that use only one premise, e.g., the rule of addition (from A infer A v B), the rule of simplification, the commutative laws and De Morgan's laws assign a lower probability bound of 1-8 to the conclusion given that the premise has probability of at least I-e. We may generalize this last sort of example to the following theorem. THEOREM 1. IfP (A)~ l-e and A logically implies B then P (B}~l-e. PROOF: We observe at once that if A logically impliesB then AU B=X, the whole sample space, and therefore Ar;;B, but if Ar;;B, then P(A)~P(B), whence by hypothesis P (B)~ I-e. It is also clear that Theorem 1 can be immediately generalized to any finite set of premises. THEOREM 2. If each of the premises At, ... , An has probability of at least 1 - e and these premises logically imply B then P (B) ~ 1 -ne. Moreover, in general the lower bound of I-ne cannot be improved on, i.e., equality holds in some cases whenever 1- ne ~ O. PROOF: By hypothesis for i= 1, ... , n, P (Ai)~ I-e. We prove by induction that under this hypothesis P (At & '" & An) ~ 1- 118. The argument for n = 1 is immediate from the hypothesis. Suppose it holds for n. Then by an elementary computation P(A1& ... &An&A n+ 1 } = 1-(1-P(A 1& .. ·&A,,))-(I-P(A n+ I ) )
+ P((A 1 & ... &A n) & An+ t)
+
~ 1 - (t - P(A t & ... &A n)) - (I - P(A n+ 1 ) ) ~ 1 - ne - e
~1-(11+1)e,
as desired. (Details of how to handle quantifiers, which are not explicitly treated in the standard probability discussions of the algebra of events, may be found in Gaifman [1965] or in the article by Krauss and Scott in this
PROBABILISTIC INFERENCE
55
volume. The basic idea is to take as the obvious generalization of the finite case P((3x)Ax) = sup {P(Aa l v Aa z v ... v Aa,,)}, where the sup is taken over all finite sets of objects in the domain. Replacing sup by infwe obtain a corresponding expression for P((Vx)Ax). Apart from details it is evident that however quantifiers are handled, the assignment of probabilities must be such that Theorem 1 is satisfied, i.e., that if A logically implies B then the probability assigned to B must be at least as great as the probability assigned to A, and this is all that is required for the proof of Theorem 2.) The proof that the lower bound 1 - ne cannot in general be improved upon reduces to constructing a case for which each of the n premises has probability 1- s, but the conjunction, as a logical consequence of the premises taken jointly has probability 1 - ne, when 1- ne ~ O. The example I use is most naturally thought of as a temporal sequence of events A \, ... , An. Initially we assign P(A I )
=
1- e
P(A\) = e.
Then
1 -2e
and more generally
P(AzIA\)=-1- e P(A z I A\ ) = I ,
P(A"IA,,_\An_z···A\) =
1 - ne 1 - (n - 1)e
P(AnIAn-IA,,_z ... A I ) = 1
in other words for any combination of preceding events on trials 1 to n-l the conditional probability of A" is 1, except for the case A,,_IA,,_Z ... AI. The proof by induction that P (An) = l-e and P (AnA,,-t ... A 1)= l-ne is straightforward. The case for n = 1 is trivial. Suppose now the assertion holds for n. Then by inductive hypothesis P(A,,+ IAn ... AI) = P(A n+ I I An ... AI)P(A n ... AI) 1-(n+l)e
=------
1 - ne
(1 - ne)
=1-(n+l)e,
56
PATRICK SUPPES
and by the theorem on total probability P(A n+ I) = P(A n+ [I An ... A[)P(A n ... At) + + [P(A n+ [I An A1)P(A n At) + ... + P(A n+ 1 I An A[)P(An At)J.
+
By construction all the conditional probabilities referred to in the bracketed ex pression are 1, and the unconditional probabilities in this expression by inductive hypothesis simply sum to ne, i.e.. l-(l-ne), whence P(A n+ I ) =
1 - (n 1
+ l)e
I
---'~1-nlo)+ne=1-e,
- ne
which completes the proof. It is worth noting that in interesting special cases the lower bound of I-ne can be very much improved. For example, if the premises At, ... , An are statistically independent, then the bound is at least (1 - e The intuitive content of Theorem 2 reflects a common-sense suspicion of arguments that are complex and depend on many premises, even when the logic seems impeccable. Overly elaborate arguments about politics, personal motives or circumstantial evidence are dubious just because of the uncertainty of the premises taken jointly rather than individually. A natural question to ask about Theorem 2 is whether any non-deductive principles of inference that go beyond Theorem 2 arise from the imposition of the probability measure P on the algebra of events. Bayes' theorem provides an immediate example. To illustrate it with a simple artificial example, suppose we know that the composition of an urn of black (B) and white (W) balls may be exactly described by one of two hypotheses. According to hypothesis H" the proportion of white balls is r, and according to H., the proportion is s. Moreover, suppose we assign a priori probability p to H, and 1- p to H,. Our four premises may then be expressed so:
y.
P(WIHr ) = P(WIHs ) = P(Hr ) = P(H,) =
r s
P
1 - p.
Given that we now draw with replacement, let us say, two white balls, we have as the likelihood of this event as a consequence of the first two premises
P(WWIHr ) = r 2 P(WWIH,) = S2,
PROBABILISTIC INFERENCE
57
and thus by Bayes' theorem, we may infer P(H I WW) r
r2p
= -2p+s2(1_p)' ---. r
(10)
and this is clearly not a logical inference from the four premises. Logical purists may object to the designation of Bayes' theorem as a principle of inference, but there is little doubt that ordinary talk about inferring is very close to Bayesian ideas, as when we talk about predicting the weather or Jones' health, and such talk also has widespread currency among statisticians and the many kinds of people who use statistical methods to draw probabilistic inferences. The present context is not an appropriate one in which to engage upon a full-scale analysis of the relation between logical and statistical inference. I have only been concerned here to establish two main points about inference. First, in terms of standard probability theory there is a natural form of probabilistic inference, and inference from probabilistically given premises involves no appeal to the concept of total evidence. Second, all forms of such probabilistic inference are not subsumed within the forms oflogical inference, and two examples have been given to substantiate this claim, one being the rule of detachment as formulated for conditional probability and the other being Bayes' theorem. 3. The statistical syllogism re-examined. There is, however, a difficulty about the example of applying Bayes' theorem that is very similar to the earlier difficulty with the statistical syllogism. I have not stated as explicit premises the evidence WW that two white balls were drawn, and the reason I have not provides the key for re-analyzing the statistical syllogism and removing all air of paradox from it. The evidence WW has not been included in the statement of the premises of the Bayesian example because the probability measure P referred to in the premises is the measure that holds before any taking of evidence (by drawing a ball) occurs. The measure P does provide a means of expressing the a posteriori probability after the evidence is taken as a conditional probability, but the hypothetical or conditional nature of this assertion has been too little appreciated. Using just the measure P there is no way to express that infact two white balls were drawn, rather than, say, a white ball and then a black ball. Using conditional probabilities we can express the a posteriori probabilities of the two hypotheses under any possible outcomes of one, two or more drawings. What we cannot express in these terms is the actual
58
PATRICK SUPPES
evidence, and it is a mistake to try. (It should be apparent that these same remarks apply to Carnapian confirmation functions.) Commission of this mistake vitiates what appears to be the most natural symbolic formulation of the statistical syllogism - the inference form (5) as a special case of (3). We can symbolize statistical syllogism (1) as follows, where e(x) is the life expectancy of person x and a(x) is the age of person x, and let} =Jones:
P(e(j) ~ 15j50 < a(j) < 60) 50 < a (j) < 60 ----.. P(e(j) ~ 15) = r ,
=
r (11)
Now let us schematize this inference in terms of hypothesis and evidence as these notions occur in Bayes' theorem P (hypothesis I evidence) = r evidence
(12)
., P (hypothesis) = r ,
and the incorrect character of this inference is clear. From the standpoint of Bayes' theorem it asserts that once we know the evidence, the a posteriori probability P (HIE) is equal to the a priori probability P (H), and this is. patently false. The difficulty is that the measure P cannot be used to assert that P (50
a (x) < 60)
=
r,
PROBABILISTIC INFERENCE
59
and secondly, given as second premise the event A that
m (j) & 50 < a (j) < 60, we may write the conclusion in terms of a new probability measure PA conditionalized on A: PA (e(j) ~ 15) = r. Moreover, it is clear that no paradox arises from (2) because the evidence expressed in the second premise of (2) represents an event B distinct from A, and the conclusion PB(e(j)~15)=s, is consistent with the conclusion PAe(j) ~ 15)= r of (1). There is still another way of putting the matter which may provide additional insight into the inferential kernel inside the dubious statistical syllogism. We may think of the premises as all the a posteriori probabilities given all the different possible kinds of evidence. As an additional final premise, some evidence A is asserted. On this basis a new measure PA is generated and the probability of the hypothesis is then asserted in terms of this new measure PA , as the conclusion of the inference. At this point it might seem easy to insist that delicate questions of consistency or coherence about the probability measure P do indeed differentiate deductive and inductive logic, but this is not at all the case. The problem of temporal order of knowledge is as characteristic of deductive as of inductive logic. In discussing deductive canons of inference we tacitly assume the statements whose inferential relations are being considered are all asserted or denied at a given time or are timeless in character. It is not a paradox of deductive logic that the joint assertion of two statements true at different times leads to a paradox - for example, it rained yesterday, and it did not rain yesterday. The same thing, I have argued, is to be said about the statistical syllogism. The same probability measure does not apply to the first and second premise; the measure referred to in the first premise is temporally earlier than the one implicit in the second premise and the conclusion. In the next section I turn to these temporal problems and their relation to the complex task of defining rationality, but before doing this I want briefly to pull several strands together and summarize in slightly different fashion the place given to the concept of total evidence by the view of probability advocated here. According to this view it is automatic that if a person is asked for the probability of an event at a given time it will follow from the conditions of coherence on all his beliefs at that time that the probability he assigns to the event automatically takes into account the total evidence that he believes has relevance to the occurrence of the event. The way in which the total evidence is brought in is straightforward and simple through the theorem on total
60
PATRICK SUPPES
probability. To be quite clear how this theorem operates it may be useful to take a somewhat detailed look at the gradual expansion of the probability of an event A in terms of given evidence Band C. For purposes of generality we may assume that the probability of Band C is not precisely 1 and therefore deal with the general case. (In the interest of compactness of notation, I here let juxtaposition denote intersection or conjunction.) First we have We also have
P(A) = P(A I B)P(B) + P(A I B)P(B).
(13)
+ P(A I C)P(C).
(14)
P(A) = P(A I C)P(C)
And in terms of both Band C we have the more complex version:
P(A) = P(A I BC)P(BC) + P(A I BC)P(BC)
+ P(A IBC)P(BC) + + P(A I BC)P(BC).
(15)
In the special case that P(B)=P(C)=1, we then have
P(A) = P(A I B) = P(A I C) = P(A I BC). We have in the general case, as indications of the relations between "partial" and "total" evidence, and
P(A I B)P(B) = P(A i BC)P(BC)
+ P(A I BC)P(BC)
P(A I BC)P(BC)
+ P(A I BC)P(BC).
P(A IB)P(B)
=
The point of exhibiting these identities is to show that no separate concept of total evidence need be added to the concept of a probability measure on an individual's beliefs. It also may seem that these identities show that the notion of conditional probability is not even needed. The important point however is that the serious use contemplated here for the notion ofconditional probability is in terms of passing from the probability measure expressing partial beliefs at one time to a later time. It is just by conditionalizing in terms of the events that actually occurred that this passage is made a good deal of the time by most information-processing organisms. I conclude this paper with a more detailed look at the processes by which beliefs are changed. 4. Rational changes in belief. It seems important to recognize that the partial beliefs, or probability beliefs as we may term them, that an individual holds as a mature adult are not in any realistic way, even for an ideally
PROBABILISTIC INFERENCE
61
rational individual, to be obtained simply by conditionalization, that is, in terms of conditional probability, from an overall probability measure which the individual was born with or acquired early in life. The patent absurdity of this idea does not seem to have been adequately reflected upon in Bayesian discussions of these matters. The static adherence to a single probability measure independent of time is characteristic of de Finetti [1937] and Savage [1954], but even a superficial appraisal of the development of a child's information-processing capacities makes it evident that other processes than conditionalization are required to explain the beliefs held by a mature adult. Moreover, even an adult who does not live in a terribly static and simple world will need other processes than conditionalization to explain the course of development of his beliefs during his years as an adult. Acceptance of the view that a person's beliefs at time t are to be expressed by a probability measure special for that time raises certain problems that go beyond ordinary talk about probabilities. Under this view what probability are we to assign to events that a person knows have occurred? I can see no other course but to assert that such events have probability 1. Thus if I know that Jones is between 50 and 60 years of age then this event has probability 1 for me. We shall take as an axiom linking knowing and probabilistic beliefs as well as an axiom linking beliefs and probabilistic beliefs exactly this assertion, that is, if a person knows or believes that an event occurred then the probability of this event for that person is 1. For example, since I now believe with certainty in my own mind that Julius Caesar stayed several weeks in Gaul the probability of this event for me is 1. The kind of problem that is solved by this axiom, and that is troublesome in a more detailed look at the standard Bayesian viewpoint, is the following. It is customary to remark, as has already been indicated, that the probability of an event should be conditionalized on that which has occurred. But it is also natural to ask what currently is the probability to be attached to an event that has occurred. Thus if the event B has occurred what probability is to be assigned to it (here I engage in the standard simplification of talking about the event occurring and not using the more careful locution of talking about knowing or believing that the event has occurred). It is not sufficient to say that we may refer to the probability of B given B because in this vein we can talk about the conditional probability of any event given that event. It is necessary to assign probability 1 to all such conditional probabilities independent of whether or not the event B has actually occurred. To put in another wayan argument already given, once the continual adjustment of probability to the current state of affairs is supposed, and there
62
PATRICK SUPPES
is much in ordinary thinking and language that supports this idea, the problem of seeming to need to introduce a separate assumption about using all the evidence available simply disappears. As has been shown, it is an immediate consequence of the theorem on total probability that when we discuss the probability of an event A the relevance of any information we have about the event is immediately absorbed in a calculation of this probability, as a direct consequence of the theorem on total probability. Unfortunately, there is another problem of total evidence, related to but not the same as, the one we started with. This new problem would seem to occupy a central position in any analysis of how we change our beliefs from one time to another. The problem is that of characterizing what part of the welter of potential information impinging on an organism is to be accepted as evidence and how this evidence is to be used to change the organism's basic probability measure. From the standpoint of psychology and physiology, a satisfactory empirical answer seems a great distance away. The fact of our empirical ignorance about matters conceptually so close to central problems of epistemology is philosophically important, but ofstill greater philosophical importance is the fact that our general concept of rationality seems intimately involved in any answer, empirical or not, that we give. Until we can say how an organism of given capacities and powers should process the information impinging on it we cannot have a very deep running concept of rationality. The point that I want to make here needs some delineation. The problem is not one of giving many instances in which there is great agreement on what is the rational way to process information. If a man is in a house that is burning, he is irrational calmly to continue to listen to music, or if a man is driving down a highway at high speed, he is irrational if he becomes absorbed in the beauty of the landscape and looks for forty or fifty seconds at an angle nearly perpendicular to the road itself. These simple instances can be multiplied to any extent desired, but what is not easy to multiply is the formal characterization of the principles that should be applied and that govern the variety of cases in which we all have a clear intuitive judgment. To a large extent work in inductive logic and statistical inference tends to obscure the fundamental character of this problem of giving principles by which information is to be judged important and to be responded to. The reason for this neglect in inductive logic or theoretical statistics is that once the formal language or the random variables are selected, then the problem of information-processing is reduced to relatively simple proportions. The selection of the language or the selection of the random variables, as the case may be, is the largest single decision determining how information will be
PROBABILISTIC INFERENCE
63
processed, and to a very large extent the simple rule of conditionalizing, so that the measure held at a later time arises from an earlier probability measure as a conditional probability measure, then furnishes the appropriate way to proceed. It might almost be said that the rule of pure prudence is always to derive beliefs held at time t from beliefs held at time t' by conditionalization on the probability measure characterizing partial beliefs at the earlier time. Although such a principle of pure prudence may seem attractive at first glance, in my own judgment it is a piece of pure fantasy. A myriad of events are occurring at all times and are noticeable by a person's perceptual apparatus. What is not the least bit clear is what sort of filter should be imposed by the individual on this myriad of events in order to have a workable simplifying scheme of decision and action. The highly selective principles of attention that must necessarily be at work do not seem to be characterizable in any direct way from the concept of conditional probability. The two interrelated processes that any adequate theory of rationality must characterize are the process of information selection and the process of changing beliefs according to the significance of the information that is selected. In other words, to what should the rational man pay attention in his ever-changing environment and how should he use the evidence of that to which he does attend in order to change his beliefs and thereby modify his decisions and actions. What seems particularly difficult to describe are rational mechanisms of selection. In a first approximation the classical theory of maximizing expected utility as developed by Ramsey, de Finetti, Savage and others uses the mechanism of conditional probabilities to change beliefs once the information to be attended to has been selected. This classical mechanism is certainly inadequate for any process of concept formation, and thus for any very deep running change in belief, and as Jeffrey [1965] points out, it is not even adequate to the many cases in which changes in belief may be expressed simply as changes in probability but not explicitly in terms of changes in conditional probability, because the changes in probability are not completely analyzable in terms of the explicitly noticed occurrence of events. For example, the probability that I will assign to the event of rain tomorrow will change from morning to afternoon, even though I am not able to express in explicit form the evidence I have used in coming to this change. In another article in this volume I have attempted to show how inadequate the Bayesian approach of conditional probability is in terms of even fairly simple processes of concept formation, and I do not want to go over that ground again here, except to remark that, it is clear on the most casual inspection that all in-
64
PATRICK SUPPES
formation processing that an organism engages in cannot be conceived of in terms of conditional probabilities. It is even possible to question whether any changes can be so expressed. The measure P effectively expressing my beliefs at time t cannot be used to express what I actually observe immediately after time t, for P is already "used up" so to speak in expressing the a priori probability of each possible event that might occur, and cannot be used to express the unconditional occurrence of that which in fact did happen at time t. Pragmatically, the situation is clear. If an event A occurs and is noticed, the individual then changes his belief pattern from the measure P to the conditional measure PA' and our reformulated version of the statistical syllogism is exemplified. What has not been adequately commented upon in discussions of these matters by Bayesians is that the probability measure P held at time t cannot be used to express what actually happened immediately after t, but only to express, at the most, how P would change if so and so did happen. In any case, genuine changes in conceptual apparatus cannot in any obvious way be brought within this framework of conditional probability. As far as I can see, the introduction of no genuinely new concept in the history of scientific or ordinary experience can be adequately accounted for in terms of conditional probabilities. A fundamental change in the algebra of events itself, not just their probabilities, is required to account for such conceptual changes. Again, it is a problem and a responsibility of an adequate theory of normative information processing to give an account of how such changes should take place. What I would like to emphasize in conclusion is the difficulty of expressing in systematic form the mechanisms of attention a rationally-operating organism should use. It is also worth noting that any interesting concept of rationality for the behavior of men must be keyed to a realistic appraisal of the powers and limitation of the perceptual and mental apparatus of humans. These problems of attention and selection do not exist for an omniscient God or an information-processing machine whose inputs are already schematized into something extraordinarily simple and regular. The difficulty and subtlety of characterizing the mechanisms of information selection and at the same time a recognition of their importance in determining the actual behavior of men make me doubt that the rather simple Carnapian conception of inductive logic can be of much help in developing an adequate theory of rational behavior. Even the more powerful Bayesian approach provides a very thin and inadequate characterization of rationality, because only one simple method for changing beliefs is admitted. It is my own view that there
PROBABILISTIC INFERENCE
65
is little chance of defining an adequate concept of rationality until analytical tools are available to forge a sturdy link between the selection and use of evidence and processes of concept formation. References BARKER, S., 1957, Induction and hypothesis (Cornell University Press, Ithaca, New York) CARNAP, R., 1950, The logical foundations of probability (University of Chicago Press, Chicago; second edition, 1963) DE FINETTI, B., 1937, La prevision: ses lois logiques, ses sources subjectives, Annales de I'Institut Henri Poincare, vo!. 7, pp. 1-68. Engl. trans!. in: Studies in subjective probability, eds. H. E. Kyburg Jr. and H. E. SmokIer (John Wiley & Sons, Inc., New York, 1964) pp, 93-158 GAIFMAN, H., 1964, Concerning measures in first-order calculi, Israel Journal of Mathematics, vol. 2, pp. 1-18 HEMPEL, C. G., 1965, Aspects of scientific explanation (The Free Press, New York) JEFFREY, R. C., 1965, The logic of decision (McGraw-Hili, New York) SAVAGE, L. J., 1954, The foundations of statistics (John Wiley & Sons, Inc., New York) SUPPES, P. and R. C. ATKINSON, 1960, Markov learning models for multiperson interactions (Stanford University Press, Stanford, California)
SIMPLICITY, ENTROPY AND INDUCTIVE LOGIC KURT WALK IBM Laboratory, Vienna, Austria
Introduction. It seems to be generally agreed that there is a principle of simplicity that plays an important role as a guide of our inductive behaviour: in situations where a choice is to be made among a number of otherwise equally acceptable hypotheses, the principle tells us to compare the hypotheses with respect to their simplicity, and to choose the hypothesis which is the simplest one. There is disagreement, however, concerning the grounds of the acceptance of this principle, and concerning the (quantitative) explication of the notion of simplicity. In this paper I will concentrate on an explication of the notion of simplicity of state descriptions. As for the justification of the principle of simplicity I refer to the discussion of this topic by Kemeny [1953], who holds that the principle is not a presupposition concerning the simplicity of nature, but that it may be regarded as a consequence of requirements set up by methodological considerations. The formulation of the principle proposed later in this paper is essentially in line with this view. The purpose of this paper is to advocate an explication of simplicity which leads to an entropy-like measure, and which has turned out to be useful in settling problems of application of inductive logic. Examples of the use of this interpretation of simplicity are given in the second part of the paper; they pertain to Catnap's system of inductive logic and to an extension of it. Measures of simplicity, however, already come into the picture prior to the establishment of a quantitative system of inductive logic, as a means to handle choice situations, and as a guide for the construction of a quantitative system. Hermes, Kiesow and Oberschelp give quantitative explications of the simplicity of state descriptions, with a view to justifying certain axioms of Carnap's theory of probability. In the first part of the paper I shall follow this line of thought, and show that a natural extension of the principle underlying the construction of these measures, and also other approaches, gives rise to an entropy-like measure, and that the results obtained in this way are
SIMPLICITY, ENTROPY AND INDUCTIVE LOGIC
67
deeper than those obtained by Hermes. The aim is to offer reasons for the acceptance of the proposed measure. 1. The explication of simplicityby Hermes, Kiesowand0 berschelp. Carnap's quantitative system of inductive probabilities [1963] is based on a set of axioms, which includes Kolmogorov's axioms of probability theory. These axioms, which are not at all sufficient for determining numerical values of probability (except in trivial cases), may be taken to be explained in terms of the basic notions of betting theory. In fact, the requirement of the rationality of combined bets yields precisely these axioms; see Kemeny [1955], Lehman [1955] and Shimony [1955]. Hermes [1958] poses the question whether an explanation, other than the appeal to common sense, can be given also to the further axioms of Carnap's theory, and he investigates the principle of simplicity for this purpose. In order to give the principle a precise meaning, an exact explication of simplicity is needed. This problem becomes manageable if the simplicity of statements is considered rather than that of propositions. The simplicity of a hypothesis then is measured by the simplicity of the statement which in a certain language expresses it. It is the first intuitive idea to relate the simplicity of a statement to its length. Hermes does not object to the obvious consequence that the properties of the particular language will influence the resulting measure of simplicity, arguing that scientific hypotheses really do not come into existence unless they are formulated in a well-defined language, and that scientific languages therefore play in any case a decisive role in scientific thinking. A language used to carry out these ideas will preferably be a formal language. Hermes uses for this purpose the predicate calculus with identity, with a finite number of individual constants, an arbitrary number of individual variables, and the usual logical constants, connectives, and quantifiers. What is the length of expressions in this language, and how is the length related to simplicity? An answer to these questions has been proposed by Kiesow [1958]. He first restricts the range of statements taken into consideration to state descriptions, in order to compare only statements of equal descriptive strength. The length of a state description is defined as the number of atomic statements ofwhich it is composed. The simplicity of a state description is measured by the length of the shortest equivalent state description ~ this assures that equivalent state descriptions are given the same simplicity measure. A state description 1 is simpler than a state description 2, if the length of the shortest
68
KURT WALK
state description equivalent to 1 is smaller than the length of the shortest state description equivalent to 2. Let us consider a simple example. We speak about n individuals aj, a z, '" an and one primitive predicate P. The state description sd asserts that n j individuals ai' a z , ... a nl, have the predicate P and the remaining n-nj individuals have the predicate --, P, and we assume that n 1 >n-nj' The language under consideration allows the representation of this statement as sd:
(1)
The length of sd according to this convention is equal to n - n j + 1, which is obtained by counting the atomic statements x=ai (i=nj + 1, ... , n) and the atomic statement --, Px. It can be shown that there is no description which is shorter in this sense. The principle of description leading to the construction of this expression is something like the following: the statement asserts that, as a rule, the individuals have the predicate P; but that there are several exceptions, namely the individuals anI + I , . . . an> which have the predicate --, P. So the length of the expression is determined by the number of exceptional individuals which disturb the uniformity of the state described. Before considering the implications of this measure together with the principle of simplicity, I briefly present two other explications of simplicity. One observes that it is essential for the length of state descriptions in Kiesow's sense that the language used contains the equivalence symbol <--+. This indicates a rather unpleasant dependence on the properties of the language. Although simplicity has been introduced basically as a language concept, efforts have been made to reduce this dependence. To be independent of particular logical operators, Oberschelp [1960] uses languages with no restriction on the kind of logical operators, so that the languages contain all two-place operators, all three-place operators and so on. The problem is again to find the shortest possible description equivalent to any state description. The result is surprisingly simple, so that it can easily be reported. Let us consider languages with m exclusive predicates (Carnap's Q-predicates), then each state description induces a partitioning of individuals into classes having the same predicate.Tf' the number of individuals in the largest class is k l , then the length of the shortest description is (2) thus depending for given m (the number of predicates) and n (the number of individuals) only on the cardinality of this largest class.
SIMPLICITY, ENTROPY AND INDUCTIVE LOGIC
69
The objection to be raised against this measure and also to Kiesow's measure clearly is that the distribution of the individuals over the classes other than the largest class is completely ignored. It was this objection that made Oberschelp propose as an alternative measure of simplicity not a single number, but an r-tuple of numbers: (3)
where the k, are the cardinalities of the classes of individuals ordered with respect to their size. A state description sd ; in this sense has to be regarded as simpler than a state description Sdl' if the first number (from left to right) differing for the two descriptions, is larger for state description sd I. Hermes [1958] makes the following comments on such an explication of simplicity: first, we prefer a measure which is a single number; second, the considerations which lead to measure (2) are based on an arbitrarily complicated language (it contains arbitrarily complicated logical operations), so that the complexity of a situation to be described may only cause the use of complex language elements instead of causing a long descriptive expression. I will postpone the question of the explication of simplicity at this point and consider the significance of the principle of simplicity for probability theory. Hermes proposes the following formulation as an axiom: a state description sd, is more probable than a state description sd, if and only if the simplicity of sd, is greater than the simplicity of Sdl' and the two state descriptions are equally probable if they are equally simple. The following example illustrates the use of the principle. We consider the case of a singular prediction. The evidence is a state description of n individuals specifying the cardinal numbers k l , k l , ... , k; of the classes of individuals having the same predicate. We have the choice among a set of hypotheses hI' h l , ... , h m , where the hypothesis hi predicts that the (n + 1)st individual (not mentioned in the evidence) has the predicate Pi (i=l, 2, ... , m). We compute the length of the state description of all n + 1 individuals under an assumption that hi is true. Doing this for all hypotheses hi allows for the comparison of the hypotheses with respect to simplicity. The result is, for all measures of simplicity mentioned, that the simplest case is obtained when the new individual is associated with the largest class, i.e. that that hypothesis is the most probable which gives the new individual the predicate that occurs most frequently in the evidence. The result confirms exactly that axiom in Carnap's axiom system which indicates the direction of inductive behaviour: things that are experienced to happen most frequently, are most probable to be experienced in the future.
70
KURT WALK
2. The explication of simplicity by entropy. To begin with, I return to the measure proposed by Kiesow. I have already stated that the description principle leading to this measure is the description of the universe (i.e. the set of individuals) as being partitioned into a class of individuals following a rule and a class of individuals constituting exceptions to this rule. If the whole universe follows the rule (i.e. if all individuals are alike), we have the most pleasant case for description - it suffices to report the rule. We will extend the principle of description in a natural way. The approach is characterized by the following features: (I) In the originally proposed measures there was only one rule which could be stated and to which exceptions could be given. We reformulate the description principle by saying that we will first give a rough description of the universe (a higher order description), i.e. we assign the universe to a member of a set of possible classes of universes; and in a second step we specify the details necessary to give a full specification. The total effort of description is the sum of the efforts involved in the two steps. The second effort will be the smaller, the fewer details are left open by the higher order description. The classes of universes, which are the object of the higher order description, are to be chosen in a way which is most natural for the universes under consideration. In the example considered in this paper we propose to characterize a class by the set of numbers k i , which are the cardinalities of the subsets of individuals having the same predicates. For illustration purposes we look at an example of a different universe. Suppose we know a finite number of points of a curve in the plane and we have to select a hypothesis which gives a description of the whole curve. It seems to be quite natural to classify all possible curves according to their order, so that a higher order description specifies the order of a curve. The details needed to specify fully a curve of the nth order are, for example, the coordinates of n + 1 points. A total description consists of a description of the order and a description of the details. We will choose that hypothesis which is the simplest to describe (which for almost any language used will give the curve with the lowest order possible). (2) We will not consider a specific language in our explication, but we will use the amount of decision (the decision content in the sense of information theory) involved in a description as a measure for the length of the description. This is to mean that if a choice among k possibilities is to be made, then the amount of decision involved in the selection of one possibility by a specifying description is equal to logk. If the logarithm of base r is used, then, according to a theorem of information theory, the minimum average length
SIMPLICITY, ENTROPY AND INDUCTIVE LOGIC
71
of a description specifying the selection (averaged over all possible selections) is greater than or at most equal to log.e, if an alphabet of r letters is used. In addition, it has been shown in information theory that this is a good bound. Using the decision content as a measure of the effort of description retains the original intent of explaining simplicity as a language concept. But the resulting measure, specifying the minimum length of an expression, is independent of special language properties. Of course, if we consider the total number of possible state descriptions for n individuals and m predicates, we could compute the decision content of a state description as the logarithm of this number, which is thus the same for all state descriptions. This is a useless measure; we get reasonable results only if we compute the decision content in connection with the levels of descriptions required under (1). We thus replace the dependence on specific language properties by the dependence on a specific description principle. The classes of state descriptions specified by the numbers k, correspond to Carnap's structure descriptions. The number of different structure descriptions for given m and n is (n + m - I)! N = _ .._ -- -(4) S
n!(m -
I)!
and the number of state descriptions for a given structure description is n! Nd=~rr i
kj !
.
(5)
The decision content of a total description is D
= log N, + log N,
(6)
which for fixed m and n depends only on the second term Dz
= log N, = log
n! rr. k;!
(7)
(The base of the logarithms is left unspecified; it is only assumed that it is fixed.) We now assert that a description 1 is simpler than a state description 2, and is to be given the higher probability, if D z of description 1 is smaller than D z of description 2. The measure D z is a single number and depends on the distribution of the individuals over all classes. It is thus possible to give a sensible ordering of the state descriptions with respect to their probabilities which is based on the simplicity principle. The measure is an entropy-like measure: for the number n
72
KURT WALK
of individuals approaching infinity, D z approaches nH: Dz---+nH
(8)
where H is the entropy of the distribution of the individuals over the classes: H
=-
L
k. k, ----'-log-'. n n
(9)
Before discussing the use of our explication of simplicity for probability theory, I sketch another approach which leads to the same measure (8). We again consider a population of individuals and a set of predicates, defining equivalence classes of individuals. In the most uniform population, an the individuals belong to one such class, i.e. the individuals are not distinguishable with respect to their properties. Speaking about an individual implies, however, that the individuals can be uniquely identified. The individuals are given names for this purpose, and in order to make n individuals of a uniform population identifiable, we have to supply n different names. Again according to information theory, a name out of a set of n different names must contain at least log,» decisions between r possibilities, i.e. it consists of at least log,a symbols out of an r-symbol alphabet (if all names are of equal length, which however is an unnecessary restriction). If the population is not uniform, we have to supply different names only for individuals which are in one and the same class, in order to "individualize" the population. If we have the predicates red and black, we may speak of a "red Jack" and a "black Jack", using the name"Jack" for two different individuals. If the r h class contains k, individuals, then the total minimum length of names for this class is k, log k., and the minimum length for the total population is
I
i
k, log k, = n log n - nH.
(10)
The variable part in this expression is again the entropy H of the distribution. We now consider a population the simpler (or the more uniform), the less individuality its members have, i.e. the greater the effort is that is required for making them individuals. We now return to the example of the singular prediction from the preceding chapter and apply the measure we have defined. We compute the decision content of the over-an state description sd, under the assumption that the hypothesis hi is true:
(n + I)! D(sd.) = l o g - - - - - - - - I
k 1!k z!. .. (k i+l)!. .. k m !
(11)
SIMPLICITY, ENTROPY AND INDUCTIVE LOGIC
73
It is easily seen that
if This results in an ordering of the hypotheses with respect to their simplicity and by this with respect to their probability, so that we have the basis for a comparative concept of confirmation. The result is more far-reaching than the result obtained with the measures based on the predicate calculus, which does not give an ordering of the hypotheses but only the separation of the one hypothesis which is to be considered the most probable. A quantitative system of inductive logic should retain the ordering of state-descriptions according to their probabilities as defined by their simplicities. In other words, there must exist a monotonic function transforming the simplicities into the probabilities of state descriptions. For the case of Catnap's c*-function this transformation is especial1y simple. The measure function (a priori probability) of a state description specifying the set of numbers k ., k 2 , ••• , k m in the c*-system is explained in Carnap [1952]: ca(sd)
(m - I)! TIki! =
_i __ • J
(m + n - I)!
(12)
The decision content (6), which explains the simplicity of state descriptions is given by (m + n - I)! D = log-------(13) (m-1)! TIki! i
so that the relation to the measure function is D = -log ca(sd).
(14)
This simple relation il1ustrates the exceptional position of the c*-function. We have explicated simplicity as the effort of description and thus as the effort involved in reporting about a situation, so that there is something which may wel1 be cal1ed information. We can reformulate the principle of simplicity in the fol1owing way: if you have to explain a situation by a hypothesis, then do this by adding as little information as possible, i.e. choose the hypothesis which contains the minimum of what one could cal1 "semantic noise". There is the simple case where we have observed n individuals, ni of which have the predicate P, and where we have to select a hypothesis describing a set of new individuals. IfnI is larger than n-s-n., then we choose the hypothesis
74
KURT WALK
assigning P to all new individuals. This follows from the simplicity principle in aLI its interpretations and also from Carnap's theory, although we may expect (and in fact do expect) a rather different distribution of P and -, P among the new individuals. But if we accept this different distribution, we also have to make a decision concerning the arrangement of P- and -, Pindividuals, i.e. we have to add information we don't have, which could be given equally well by a random-output procedure, and which is thus "noise". An important thing to note is that in information theory the concept of probability is prior to the concept of information, and that the use of the notion of information in an inductive system is in danger to become circular. In the present approach, however, we have made no reference to probability when speaking about simplicity and information - we just referred to a principle of description. 3. Applications to inductive logic. Inductive logic referred to in the sequel will denote Carnap's system of inductive logic and an extension of it, whose essential features have been described in Walk [1964]. The extension concerns a world of individuals that are arranged in a linear order. A short description of this system is appropriate at this point. The system is based on a set of languages 2 N.m' If a specific language 2 N,m is used, then the description of a finite sequence of individuals specifies the properties of all the N-tuples of adjacent individuals in the sequence. One N-tuple is characterized by an ordered sequence of N predicates, where the predicates are from the set of m predicates contained in 2 N,m' There are, obviously, m" different N-tuples possible. The information used for induction is the knowledge of the frequencies of the various N-tuples in the sequence. An atomic sentence in Carnap's system, which associates an individual constant with a predicate, corresponds to the description of an N-tuple of individuals in 2,y.m' The N-tuples, however, do not occur independently of one another. For example, let N=3, m=2, and the two predicates PI and P2 ; then a 3-tuple of individuals in a sequence with the predicates PI P2P2 implies the existence of a 3-tuple (immediately adjacent to the right) with the predicates P 2P2 PI or P2P2P 2. The set of possible adjacent N-tuples at a certain point in a sequence makes up the "state" of this point. The above example illustrates that the state is determined by an (n ·-1 )-tuple of predicates (in the example the 2-tuple P2P2)' The structure of a sequence as induced by !i?N.m is representable by a directed graph, as shown in the figure below for 2 3 , 2 ' The nodes of the graph represent the states of the sequence, the arcs, labelled with the predicates, represent the transitions from individuals in the
SIMPLICITY, ENTROPY AND INDUCTIVE LOGIC
75
sequence to adjacent individuals having the respective predicates. Each path in the graph corresponds to a sequence of individuals, and vice versa. A singular predictive inference concerns the transition from the end state of a sequence described in the evidence to one of the possible successor states.
P, P,
The description of a sequence in !l'N,m specifies the frequencies of the m N different N-tuples in the sequence. Because of the interdependence of the N-tuples, only certain sets of frequencies are possible for a sequence of a given number of individuals. For convenience, we think of a set of relative frequencies as represented by a vectorfin an mN-dimensional space S, where each coordinate measures the frequency of a specific N-tuple. Each possible description of a sequence of N-individuals marks a vector in the space. For an infinite sequence, a certain continuum of relative frequencies of N-tuples is possible, which defines a subspace Soo in the frequency space. The estimation of the long-run frequencies, given the frequencies in a finite sequence, is a transition from the vector representing these frequencies in S to one of the vectors in Soo- A distinguished vector in Soo is the vector f' for which the difference vector tofis minimum;f' is obtained by projection offon Soo. The freq uency vector f' corresponds to the"direct estimation" in Carnap's system. The vector of inductive probabilities for the singular predictive inferences is obtained as a weighted mean of this vector f' and a vector of a priori probabilities of the N-tuples. In a way completely analoguous to the establishment of the inductive probability for the singular prediction in Carnap's system, the vector fis weighted by the amount of evidence, i.e. the number of N-tuples in the sequence described in the evidence, and the a priori probabilities are weighted by a factor A. This constitutes a continuum of inductive methods for !l'N,m' N = 1 yields Carnap's original system. To be able to be somewhat more general at some points of the following discussion, we mention an obvious extension based on languages !l' G' G represents a strongly connected finite-state graph whose arcs are labelled with predicates. This graph serves the functions of the graphs of !l'N,m in a
76
KURT WALK
more general way, allowing for the consideration of sequences with logical constraints in their construction. The method which we just sketched applies to this case, provided that the description of each sequence allows the identification of its states as states of the graph (this is always possible, for instance, when the initial state is given).
3.1. The parameter k The choice of the parameter A, corresponding to the choice of a specific inductive method out of the continuum of methods offered by Carnap's system, is determined by a priori considerations on the "uniformity" of the universe to which the method is to be applied. The necessity of this choice usually troubles the prospective user of inductive logic. A picture of the implications of the choice of Ais obtained by computation of the a priori probabilities of structure descriptions of a large population. A structure description is characterized by the set of relative frequencies of individuals with equal predicates. The choice A=m, m the number of exclusive predicates (Q-predicates), gives equal a priori probabilities to all structure descriptions. A large A assigns high a priori probabilities to structure descriptions characterized by relative frequencies centred around the value of 11m, i.e. to descriptions of highly inhomogeneous populations. A low A highly confirms structure descriptions of uniform populations, i.e. populations allowing statements like: "almost all individuals have predicate P", or "almost none of the individuals have predicate Q". The a priori considerations before the application of inductive logic include the matching of one's guess on the uniformity of the universe of discourse and the family of a priori probability functions that are at hand. Leaving "uniformity" unspecified as it is in Carnap's books, means leaving the user with the problem of matching two unspecified quantities. Even if he would be able to give an ordering of his possible universes according to some uniformity criterion, he would not necessarily be able to do so for structure descriptions associated with their a priori probabilities. We will set "uniformity" to be synonymous to "simplicity" and use the explication of simplicity by entropy. The entropy of a structure description specifying the set of relative frequenciesJ; of individuals with predicate Pi is given by (15) H = log j..
-II i
Weighting the entropy of every structure description by its a priori probability results in an expectation value of the entropy which is a function of A (see Walk [1963]): (16) E(H) = F(A).
SIMPLICITY, ENTROPY AND INDUCTIVE LOGIC
77
This functions F (A) is monotonous, so that each value of A corresponds to a unique value of the expectation of H, and vice versa. A = CfJ corresponds to E(H)=Hm ax and A=O to E(H)=O. The equation relates the parameter of the inductive method to a measure that is well established as measure of information: the entropy of a structure description is the average amount of information provided by the observation of an individual mentioned in the structure description, in the sense of information theory. Since Hmax is determined by the underlying language, it suffices to deal with relative entropy, or the quantity R
= 1-
HJHm a x
(17)
which is known as redundancy. To expect a universe to be redundant to a certain degree, gives good reason to choose the parameter ). of the inductive system to be applied in such a way that the expectation of redundancy provided by this system matches this expectation. There is a large variety of situations where a guess of redundancy is at hand. As an example, it is wellknown that the redundancy of written text in natural language (computed on the basis of letter-frequencies) is about 15%. The estimation function of inductive logic, if applied to the universe of written text, is reasonably chosen with the corresponding parameter A. (Experiments show that the efficiency of estimation is near optimum for this choice.) 3.2. The choice of a priori probabilities. A priori probabilities are usually chosen according to symmetry considerations. The a priori probabilities of the singular predictive inferences in Carnap's system are evenly distributed among all predicates - for there is no a priori reason for preferring one predicate to another, and in the absense of information there is no a posteriori reason, either. In a situation of highest ignorance we expect the highest information supplied by the event that removes the ignorance. So the set of possible events that may remove our complete ignorance, say the observation of the properties of the first individual, will be given probabilities that reflect this expectation. This is just another way of expressing the same consideration. Measuring information by entropy means that we have to maximize the entropy defined by the a priori probabilities. This gives, as an almost trivial result, equal a priori probabilities to the singular predictive inferences for all predicates in Carnap's system. The result is not quite trivial when the languages 2 G are considered: we choose the a priori probabilities for the transitions in graph G in such a way that the entropy of the resulting Markov
78
KURT WALK
source is maximized. This rule gives a unique solution for the a pnon probabilities. We have discussed in the preceding paragraph the choice of A on the basis of some a priori knowledge of the redundancy. There seems to be a contradiction now between the presupposition of a knowledge on redundancy a priori, and the rule that redundancy is minimized by choice of probabilities a priori. Let us assume that we have good reason to expect a certain amount of redundancy; then we can try to choose probabilities a priori in such a way that this redundancy is obtained. This choice, however, is not unique. A unique solution for the set of probabilities is obtained only for the single case of the entropy being maximum. To make a selection among the different solutions would mean to add information that is not available, in contradiction to the above formulation of the principle of simplicity. 3.3. The choice of the language. Faced with an inductive problem, the first step is the formulation in a suitable language. This step implies selecting the information which is used in induction, and of other information that is ignored. Using one of the languages underlying Carnap's system for example, implies that no relation between individuals is considered, that individuals are distinguished by a fixed set of predicates, that names of individuals have no inherent meaning. A language fits to a given problem if the relevant information is used and the irrelevant information is ignored. The choice of the language is not only a matter of a priori considerations. Let us consider the languages :£'Nsm for the description of sequences. We may describe a given sequence of n individuals by any language :£'N,m with N~n and pertinent number of predicates m. With N = 1 the description contains just the freq uencies of individuals with equal predicates, correlation properties of the sequence are ignored. The description with n = N is the specification of one single N-tuple; since induction is based on enumeration in Carnapian systems, this provides little evidence. Which language is the best possible one for the description of a given sequence when the order of the underlying Markov process is not known? There is an answer, at least for the case that there is some a priori knowledge sufficient to justify the choice of A for each language. Each inductive method, based on a certain language and a certain A then gives a different picture of the world, and we may compare these pictures with respect to their simplicity. We may ask for the expected simplicity of the universe, simplicity measured in terms of entropy, and get a definite answer. The expected simplicity will in general be the higher the more information is provided by the evidence and the more regularities are in the evidence. So it
SIMPLICITY, ENTROPY AND INDUCTIVE LOGIC
79
may be concluded that a description of the evidence which results in the expectation of a simpler universe, is providing more relevant information and shows up more regularities, under the proviso, of course, that the inductive methods used are reasonable. We will prefer the description which makes us expect the highest simplicity. Thinking of entropy as a measure of information, we prefer a description which, using a reasonable inductive method, minimizes the amount of information which we expect to gain by experiencing further evidence. Gain of new evidence will in general make a different language the optimum language for description. Step by step observation will increase N steadily up to a certain point (for ergodic sequences). This is what one would intuitively expect. The above considerations give quantitative support. 4. Conclusion. The notion of simplicity has been used in the preceding chapter in considering application problems of inductive logic. The first example was the use of the explication of simplicity by entropy, for the matching of a priori expectation of simplicity provided by inductive logic to the expected simplicity of the universe to which it is to be applied; the second example was a use of the principle of simplicity; and the third example the use of expected simplicity as a measure of success of an inductive method. The simplicity principle basically governs choice situations. The examples presented concern choice situations from a rather low level (the choice of a hypothesis stating a singular predictive inference) up to the level of the choice of an inductive system. Let us assume for the moment the existence of a general inductive method which would yield a quantitative solution to any inductive problem, as soon as it is formulated in some sufficiently rich language. In this hypothetical case there is no explicit application of the principle of simplicity. The above considerations are symptoms of the fact that there is only a modest range of problems that can be treated with inductive logic in a straightforward manner. Those who believe in the development of inductive logic may hope that these considerations eventually will be subsumed under a more general system of inductive logic. References CARNAP, R., 1950, The logical foundations oj probability (University of Chicago Press, Chicago; second edition, 1963) CARNAP, R., 1952, The continuum of inductive methods (University of Chicago Press, Chicago)
80
KURT WALK
CARNAP, R., 1963, An axiom system for inductive logic, in: The Philosophy of Rudolf Carnap (The Library of Living Philosophers), ed. P. A. Schilpp (Open Court, La Salle, Illinois) pp. 973-979 HERMES, H., 1958, Zum Einfachheitsprinzip in del' Wahrscheinlichkeitsrechnung, Dialectica, vol. 12, pp. 317-331 KEMENY, J. G., 1953, The use of simplicity in induction, Philosophical Review, vol. 62, pp. 391-408 KEMENY, J. G., 1955, Fair bets and inductive probabilities, J. Symbolic Logic, vol. 20, pp.263-273 KIESOW, H., 1958, Die Anwendung eines Einfachheitsprinzips auf die Wahrscheinlichkeitsrechnung, Archiv fur mathematische Logik und Grundlagenforschung, vol. 4, pp. 27--41 LEHMAN, R. S., 1955, On confirmation and rational betting, J. Symbolic Logic, vol. 20, pp.251-262 OBERSCHELP, W., 1960, Ilber Einfachheitsprinzipien in del' Wahrscheinlichkeitstheorie, Archiv fur mathematische Logik und Grundlagenforschung, vol. 5, pp. 3-25 SHIMONY, A., 1955, Coherence and the axioms of confirmation, J. Symbolic Logic, vol. 20, pp. 1-28 WALK, K., 1963, Kumulative Information, Nachrichtentechnische Zeitschrift, vol. 16, pp.523-528 WALK, K., 1965, Extension of inductive logic to ordered sequences, IBM Technical Report TR 25.053
TWO MEASURES OF EVIDENTIAL STRENGTH HAKAN TORNEBOHM University of Gothenburg. Gothenburg, Sweden
In this paper we will construct two measures of evidential strength and examine how these measures are interrelated. One of these measures is a probability, the other one is based on an absolute and a relative measure of the amount of information. In the first section we will be mainly interested in the informationtheoretical measure. We will give an interpretation of this measure and show how it can be used to rationalize certain common procedures of confirmation in scientific practice. In the second section we will apply both measures to arguments from a random sample to a population in order to examine how they are related in a special case. 1. Construction of two measures of evidential strength. It would be desirable to have a measure of evidential strength or degree of confirmation De satisfying the following conditions: De 1. If E L-implies H, then Dc(HIE)=max. Dc 2. If E L-implies fl, then Dc(HIE) = min. Dc 3. Dc(HEjE)=Dc(HlE). De 4. If E and H are independent of each other, then Dc(HIE) =0. Unfortunately it does not seem to be possible to construct a reasonable measure satisfying all these conditions. In view of this situation two measures will be construed. One of them satisfies the conditions De 1, De 2 and De 3, and the other one the conditions De 1, De 2 and De 4. A logical probability P (HIE) is a measure of the first kind. In order to construct a measure of the second kind we will make use of an additional condition, viz. Dc 5. Dc(HlE)=F(C(H), C(HIE)), where F (C(H), C (HIE)) is a function of C (H), i.e. of the amount of information contained in H, and of C (HIE), i.e. of the amount of information that H adds to the information contained in E.
82
HAKAN TORNEBOHM
We impose the following reasonable conditions on the absolute amount of information: C I. If His L-true, then C (H)=O. C 2. If K L-implies H, then C (K)?: C (H). C 3. If Hand K are independent of each other, then C (HK) = C (H)+
C(K).
The relative amount of information is defined by Defl. C(HIK)= C(HK)- C (K). The following theorem is readily proved from the conditions Cl , C 2, C 3 and Def 1. Th 1. (a) C (H)?:O for every H. (b) If Hand K are L-equivalent, then C (H)= C (K). (c) If His L-false, then C (H)?: C (K) for every K. (d) C (HIK)?:O. (e) If K L-implies H, then C (HIK)=O. (f) If K L-implies H, and H is consistent then C (HIK)?: C (H). (g) C (HKIK)= C (HIK). We shall make use of Dc 5 and this theorem in order to construct one measure satisfying De I, De 2 and De 3, and another one satisfying De 1, De 2 and De 4. De 3 is satisfied, if F(.,.) is independent of C(H). F(C(HIE)) satisfies De 1 and De 2, if it is a monotonic decreasing function of C (HIE). We choose the function a- CU fi E ) with a>l, because this function is like a probability. In fact it follows from Th 1 that a-C(HIE) has the following properties: Th 2. (a) O:S;a-C(HIE):s; I. (b) If E L-implies H, then a-C(HIE)=1. (c) If Hand E are independent, then a-C(HjE)=a-C(H). Th 2 and also the fact that a logical probability P (HIE) satisfies conditions Dc 1, Dc 2 and Dc 3 makes it tempting to identify a-C(HIE) with P (HIE). This identification enables us to base a semantic information theory on a theory of logical probability by means of the following definitions: Def 2. C (H)= -log P (H). Def 3. C (HIE)= -log P (HIE). From these definitions we may deduce the following theorem: Th 3. (a) If H is L-true, then C (H)=O. (b) If K L-implies H, then C (K)?: C (H). *(d) If His L-false then C (H)= 00. (e) If K L-implies H, then C (H1K)=O.
TWO MEASURES OF EVIDENTIAL STRENGTH
83
*(f) If K L-implies H, then C (HIK)= 00. (g) If K and H are independent, then C (HIK)= C (H). (h) C(HKIK)=C(HIK). The starred clauses (d) and (f) are stronger than the corresponding clauses of Th 1. Concerning the probabilistic measure P (HIE) of evidential strength we remark that De 4 is not satisfied, for if Hand E are independent of each other, then P (HIE)=P(H)( #0). We shall now make use of De 5 in order to construct a measure of degree of confirmation satisfying the conditions De 1, De 2 and De 4. De 4 is satisfied, if
F(C(H), C(HIE)=G(C(H), C(HIE))(C(H)-C(HIE)). Dc 2 is satisfied, if G (C (H), C(HlE)) >0. IfG(C(H), C(HIE)>O,then G (C (H), C (HIE) (C (H)- C (HIE)::;; G (C (H), C (HIE) C (H).
The equality sign holds, if C (HIE)=O. If E L-implies H, then C(HIE)=O and G(. , .) C (H) is a maximum according to De l . Let the maximum value of Dc(HIE) be 1. Then
1 G(C(H), C(H I E) = C(H)' We are led by this construction to adopt the following informationtheoretical measure of evidential strength:
C(H) - C(H I E) Def 4. Dc (H I E) =--qH)- -- . It is readily verified that this measure satisfies the conditions De 1, De 2 and De 4.
Under what conditions is De 3 also satisfied?
C(H) C(HE) (Dc(HE I E) - De(H I E) = = C(H)(C(HE) - C(HE I E) -C(HE)(C(H) - C(H I E) = C(HIE)C(EIH). (1) De 3 is thus satisfied, if either C (HIE)=O, i.e. if the hypothesis does not add information to the evidence, or else if C(EIH)=O.
84
HAKAN TORNEBOHM
The first condition is trivial but the second one is not. We have C (EIH) = 0, if the evidence is L-implied by the hypothesis to be tested. In this case we may simplify the expression for Dc(HJE):
C(H)Dc(H I E) = C(H) - C(H I E) = C(H) + C(E) - C(HE) = = C (E) - C (E I H) = C (E) . C (E) Hence Dc(H I E) = - - . C(H) De (HIE) thus measures the degree to which the amount of information carried by H is shared by the evidence E. This interpretation of Dc(HIE) applies also to the more general case that
Dc(HIE)"2:.0:
C(H) = C(H I E) + (C(H) - C(H I E»).
The first term on the right side is the amount of information that H adds to the information carried by E. The second term, C(H)-C (HIE), when it is "2:. 0, is therefore a measure of the information that is common to Hand E.
C(H) - C(H I E) Dc(H I E) = - - - C(H) (if it is "2:.0) is thus the degree to which the information carried by the evidence overlaps with or covers the information that is contained in H. We may read De as "degree of covering", whenever Dc(HIE)"2:.0. This interpretation fails, if Dc(HIE)
p(fi) < P(fil E). C(fi) - C(iil E) > 0, Dc(RI E) > O.
We conclude that the information carried by E overlaps with the information that is contained in ii. It is plausible to regard E as positive evidence for H, if E and H share information. If E and ii share information, E is thus positive evidence for ii, and consequently negative evidence for H.
85
TWO MEASURES OF EVIDENTIAL STRENGTH
We conclude that (a) E is positive evidence for H, if Dc(HIE»O, (b) E is neutral or indifferent for H, if Dc(HIE)=O, and (c) E is negative evidence for H, if Dc(HIE)
Dc(H\E'),
(b) H is better confirmed than H' on the basis of the evidence E, if Dc(HIE) > Dc(H'IE).
The notion of confirmation embodied in the above interpretation and in the principle of covering agrees well with the practice of historians in their confirmation of narratives. To test a narrative an historian compares the information carried by the narrative with the information that is contained in authentic documents. The narrative is judged to be the better confirmed the larger the part of its information covered by the information in the documents. In confirming an hypothesis H it is customary to make use both of already accepted hypotheses and of fresh evidence of an observational kind. Let K be an accepted hypothesis or a conjunction of accepted hypotheses. Let E be fresh observational evidence. Let H be the hypothesis to be tested. We have
C(H)(Dc(H IKE) - Dc(H I K» = = C(H) - C(HKE) + C(KE) -C(H) + C(HK) - C(K) = C(E) - ((C(E) + C(K) - C(EK» - C(E I HK) = C(E)(l - Dc(EI K» - C(EI HK). Hence
C(E) C(EIHK) Dc(HIKE)=Dc(HIK)+C(H)(l-Dc(Ei K»- C(H).
(2)
Formula (2) shows that the following procedure is reasonable: 1. Choose among already accepted hypotheses an hypothesis or conjunction of hypotheses K, which shares a substantial part of the information carried by the hypothesis H to be tested, i.e. make Dc(H1K) large.
86
HAKAN TORNEBOHM
2. Deduce fresh evidence Efrom Hand K(then of course C(EIHK)=O), such that 3. E contains much information, i.e. C(E) is large, and that 4. E shares only a small portion of information with K i.e. Dc(EIK) is small. 2. Application of the probabilistic and information-theoretical measures to arguments from a random sample to a population. The hypotheses of interest to us in this section are of the form: "Np objects in a population of N objects have the property A", and the evidence is of the form: "ns objects in a random selection of n objects from the population have the property A". These sentences will be abbreviated in the following way: Hypotheses: (N, Np) or shorter p. Evidence: (n, ns) or shorter s. We will assume (a) that all atomic propositions of the form "Aa" carry the same amount of information, and (b) that different atomic propositions are independent of each other. Let tt: be the logical probability of an atomic proposition and n' that of a negation of an atomic proposition. We prove that
P(m,mq) =
(~)nmqn,mq"
where
q' = I - q.
Let Sm be the proposition (3). (a) It is readily verified that Sl is true.
(m, mq)=Aam(m-l, mq-l)v Aam(m-l, mq). Hence by the rules of logical probability calculus:
P (m, mq)=nP (m-I, mq-l)+n'P (m-I, mq). Suppose that Sm-1 is true. It follows from the above formula that
P ( m, mq ) = ((m mq
1) + (mqm - I))
We have just demonstrated that (b) Sm-C--+Sm'
1
tt mq tt rmq'
= ( m) tt mq tt rmq' . mq
(3)
87
TWO MEASURES OF EVIDENTIAL STRENGTH
(3) follows from (a) and (b) by induction. We have to find an expression for P UjJ). sjJ is L-equivalent to sf, where f=(N-n, Np-ns) describes the rest class obtained from the population jJ after removal of the sample s. sand f have no atomic proposition in common and are therefore independent of each other. We conclude that (4) PUjJ) = PU)P(f). It follows from (3) and (4) that
" .) PU)P(,') P (sip = P(jJ) P(jJls) = P(f) =
and
(5)
(N - n )nNP-nSn'NP.-ns', Np - ns
We proceed now to interpret the probability measures.
(:s)(~=:s) = ----------------------.-n!(N-n)!Np!Np'! =
". P(slp) = --
(:p)
ns! ns'! (Np - ns)! (Np' - ns')! N!
Np! Np'! N! ~~!(Np .=--~;)! .;;;7! (N p' ~ ns')! . n! (N - n)!
(:n (:~')
--------
(~)
(6)
(~~) is the number of ways in which ns A-objects can be selected from Np A-objects.
(~:') is the number of ways
in which ns' A-objects can be selected from
Np' A-objects. We conclude that the denumerator in (6) designates the number of subsets with the structure (n, ns) which are contained in a set with the structure (N, Np). P UljJ) may therefore be interpreted as a measure of the commonness of a subset of the structure (n, ns) in a population of the structure (N, Np).
88
HAKAN TORNEBOHM
Let
N-H/J
in the expression for P (SIp). Then
log P(SIN, Np) = log (:s)
+ log
(~~ :s) -log (~)
= log (:s) + log (N - n)! -log (Np - ns)! + -log (Np'- ns')! -log N!
+ log Np! + log Np'!
+ (N - n) log N - (N p - ns) log Np + ns - (Np'- ns') log Np' - N log N + Np log Np +
--> N-co
log (n )
+ Np' =
log Np'
log (:s) pn' p'n".
(7)
We infer from (7) that
p(S)=(n)nn'n m"= IimP(sIN,Nn). ns N-co
(8)
P (S) may thus be interpreted as a measure of the commonness of a sample with the structure (n, ns) in a very large population (N, Nn), in which the relative frequency of the property A is ti,
If a sample is obtained from a given population by a random selection, then it is most likely that the sample belongs to the most common kind of subsets in the population. On the other hand if a random sample ofsize n from a population ofsize N is given, then it is most likely that the sample comes from a population such that those subsets which are like the sample are the most common kind of subsets. We may therefore adopt the following principle:
Principle of maximum commonness. An hypothesis p about a population for which the degree of commonness P (sIp) is maximal is best supported by a given random sample described by s. How is the measure of commonness, which is a logical measure function, related to the notion of likelihood in statistics as applied to arguments from a random sample to a population? If several (say k) random samples of size n and relative frequencies s, are taken from a very large population of relative frequency p, then the likelihood function is
L(SbSZ, ... ,Sk;
p)=
TI k
i= 1
(
n ) ns, Ins" p 'p '. ns,
89
TWO MEASURES OF EVIDENTIAL STRENGTH
On the right hand side we have a product of measures, each of which can be interpreted as a measure of the commonness of a random sample such as 05; from a very large population with relative frequency p. The maximum likelihood principle requiring that p be chosen in such a way that L takes a maximum is a natural generalization of the principle of maximum commonness. It may accordingly be claimed that a justification given to the principle of maximum commonness lends support to the principle ofmaximum likelihood. We wish to employ the principle of maximum commonness to determine the parameter tt, n should be chosen in such a way that P (SIN, Nn) has a maximum for N-HX). As P (SIN, Nn) ~ P (S), tt must satisfy the following conditions: :n[IOg
02
- 2 log
on
(:s) + ns log
tt
+
ns' log n'J = 0, i ,.
P(S) < O.
The first condition leads to the estimate n =s. The second condition is automatically satisfied. Inserting s for tt in the expression for P (pis), we obtain
' I') = (N-n) P( ps N sNp-ns s ,Np'-ns' . p-ns
(9)
We wish to show that the measure (9) takes its maximum for that hypothesis for which the degree of commonness P (sip) is maximal. Let P (SIP) take its maximum for Po, so that P (SIPo)~ P (sIP) for all P, and let P (pis) take its maximum for Pt, so that P (Pds) ~ P (pis) for all p .
We will show (a) that p, =Po and (b) that the favored population described Po is the one which is most like the sample. Po and Pt must satisfy the conditions:
by
PCS I N,Npo
- 1)<
PCS I N,Npo»
pes I N,Npo+ 1)
peN, NPt - 11 05) < peN, NPll 05) > Pt N, Np ;
+ 1105).
(10)
90
HAKAN TORNEBOHM
Making use of (5.1) and (9), we obtain according to (10):
)( N) (N-I1)(N) ( NpoN-11 - I1S - 1 : Npo - 1 < Npo - ns : Npo > N-n ) ( N ) ( Npo-/1s+1 : Npo+l ,
and (10')
N-n ) Npl-ns+1 ,Npl'-ns'-I n . ( N' PI - /1S + 1 t: By division we obtain the simpler inequalities:
Npo - ns
Npo
~~~
N p~ - /1S' + 1 N p~ +1
< 1>
Np~
- ns'
Np~
:---,
N Po - ns + 1 N Po + 1
(10")
and Np , - ns
s'
- /1S'
S
Np~
+1
Np'l - ns' s < 1 >-------Npl - ns + 1 s'
The inequalities (l0") are readily transformed into
and
N Pos' < N p~s Np~s < Npos'
+ s ~ N p~s + 1 + s' ~ Npos' + 1
Np,s' < Np~s < Npis'
+ s ~ Np~s + 1 ~ + S' ~ NPls' + 1. ~
Np~s
) ~
(11)
(12)
Add Npos to both of the inequalities in (11), and Np 1S to both of the inequalities in (12). We then obtain
and
We conclude that
Npo < Ns Ns < Npo
+1 +1
(11')
Npl < Ns Ns < Np,
+1 + 1.
(12')
1 IPo-sl
and
Ipl- s l
1
(13)
TWO MEASURES OF EVIDENTIAL STRENGTH
91
(13) shows that P (Pis) and P (SIp) with n = s assume their maximal values for the same hypothesis Po and that this favored hypothesis assigns the same relative frequency to the population as the one which has been observed in the random sample. We sum up this result in the form: 1
If !Po - s] < -, then N
P(S Ipo)
~
P(S Ip) for all p, and
P(Po I 05) ~ P(p! 05) for all p.
(14)
(15)
The measure of commonness P (SIp) is not a measure of evidential strength according to the criteria laid down for such a measure in the first general section of this paper. The measure P (pis) satisfies the formal requirements of a probabilistic measure of evidential strength. But this fact alone is not a sufficient ground to adopt P (pis) as such a measure. P(pls) is related to P (SIp) by the Bayesian relation
If we take the population p and the sample 05 from p to be random samples from a very large "hyper"-population n, then we have shown that P (05) may be interpreted as the commonness of a random sample such as 05 in the "hyper"-population. A similar interpretation is of course possible for P (P). These interpretations give the same sense to all probabilities on the righthand side of the Bayesian equation. Our interpretation of P (S) (formula (8) made it possible to justify a choice of the parameter n which appears in the Bayesian formula by means of the maximum commonness principle. Why apply this principle to P (S) but not to P (p)? An obvious reason is that 05 is given but p is not. We have found that P (pis) after t: has been estimated is maximized by the same hypothesis p which is selected by the principle of maximum commonness. This fact is a strong reason (a) to adopt P (pis) with tt =s as a probabilistic measure of evidential strength for arguments from a random sample to a population and (b) to adopt the following principle of inference: Principle of maximum probability. That hypothesis about a population is best supported by a given random sample 05, for which the probabilistic evidential strength P (pis) with n = s is maximal.
92
HAKAN TORNEBOHM
This principle is not autonomous. It is supported by the principle of maximum commonness in conjunction with the interpretation of the prior probabilities as measures of commonness. We wish now to apply information-theoretical measures to arguments from a random sample to a population. From formula (15) we infer that
C(Po 105):0:; C(p I 05) for all p,
(16)
(16) shows that the favored hypothesis Po adds the least amount of information to the evidence. Starting from (14) we make the deduction:
C(Slpo):O:; C(Slp),
C(S) - C(Slpo) Z C(05) - C(05lp),
(17)
C(S) - C(S Ip) = C(S) + C(p) - C(po5) = C(p) - C(p Is) = C(p)Dc(p I 05).
(18)
Hence from (17) and (18) we obtain
C (Po) Dc(Po I 05) = C (Po) - C (Po Is) z C (p) - C (p I.~) = C (p) Dc(p I05). (19) (19) shows that the favored hypothesis Po shares the largest amount of information with the evidence. We will next show that the degree of information overlap is also maximal for the hypothesis Po. The difference C (Po)Dc(Polo5)- C (p) Dc(plo5) can be written in two different forms as follows
0:0:; C (Po) Dc(Po Is) - C (p) Dc(p I 05) = = C (p) (Dc (Po I 05) - Dc(p I 05)) + (C (Po) - C (p)) De(Po I 05) =
=C(Po)-C(Polo5)-C(p)+C(plo5)zO.
(20)
Hence
C(p)(Dc(po I 05) - Dc(p I S)) = = (C(Po) - C(p))(l- DC(Polo5)) - C(Polo5)
+ C(plo5).
(21)
From (20) it follows that
C (Po) - C (p) Z C (Po
Is) - C (p I05).
(22)
TWO MEASURES OF EVIDENTIAL STRENGTH
93
As 1- DC(Pols) ~ 0, we obtain from (21) and (22) the inequality
C(p)(Dc(po Is) - Dc(p Is)) ~ (C(Po Is) - C (p Is))(1 - Dc(po Is)) - C (Po I s) + C (p Is) = Dc(Po Is)( C (P I s) - C (Po Is)).
(23)
Furthermore, Moreover
C (p Is) - C (Po I s) ~ 0 (according to (16»).
(24)
Dc(Pols»O.
(25)
(25) is proved in the following way: Suppose that (25) is false. Then C (P)Dc(pls):::;O for all p in virtue of (20). Hence C (P):::; C (pis) for all p, so that P (p)~ P (Pis) for all p. As the equality sign does not hold for every P, we conclude that N
I
Np=O
P (P) >
N
I
Np=O
P (P I s),
but this is impossible, for N
I
Np=O
P(P)=l
and
N
I
Np=O
P(pls)=1.
We conclude from this impossibility that (25) is true. From (23), (24) and (25) we infer that
Dc(Po I s) ~ Dc(p I s).
(26)
(26) shows that both measures P (PIs) and Dc(p/s) are maxima for the same hypothesis Po which is favored by the principle of maximum commonness.
Dc(pls) differs from P (pis) in one important respect. Dc(pls) divides the set of possible hypotheses into hypotheses for which s is positive and hypotheses for which s is negative evidence. We have shown that DC(Pls) must take at least one positive value. We may show in a completely analogous manner that DC(Pls) must also be capable of taking at least one negative value in the interval 0:::; Np s: N. Fig. 1 is a qualitative plot of the two measures of evidential strength. We have found that the one hypothesis which is favored by the principle of maximum commonness also enjoys a privileged position with respect to the information-theoretical measures. They measure, respectively, added infor-
94
HAKAN TORNEBOHM
I p(pIS) I
\
I------+------'-----'~-__+_----
i !
Dc(
pi s)
r--~f----'----~-_+__---
.I'
NP
Np
Fig. I. Dc(Pi.Y) increases (decreases) when P (fil,\') increases (decreases). is negative evidence for the hypotheses for which Np <. NPI and Np> NP2 and positive evidence for all hypotheses for which NPI <, Np < Np2.
marion, i.e. C (pis); amount of shared information, i.e. C (P)Dc(Pls); and degree of information overlap or degree of covering i.e. Dc(Pls). We may accordingly formulate the following three principles: 1. Principle of minimum added information. That hypothesis IS most favored by given evidence which adds the least amount of information to the evidence. The more we assert, the larger is the risk of being wrong. Using the above principle we run the least risk of making a mistake. We might therefore also call it a principle of caution. 2. Principle of maximum information overlap, That hypothesis is best supported by given evidence which shares the largest amount of information with the evidence. 3. Principle ofmaximal degree af covering. That hypothesis is best favored by given evidence for which the degree of covering is maximal. The manner in which these three principles have been established shows how they are related to the two principles in the probabilistic approach to arguments from a random sample to a population. The second principle is mathematically equivalent to the principle of maximum commonness.
TWO MEASURES OF EVIDENTIAL STRENGTH
95
The first principle is equivalent to the principle of maximum probability, which (as we have seen) is not autonomous but is supported by the principle of maximum commonness. None of the measures used in the first two information-theoretical principles of inference are degrees of confirmation or evidential support, but the third principle refers to such a measure. This principle is supported by the first two information-theoretical principles. What is the point of proposing so many partly equivalent principles of inference? If one were mainly interested in the kind of simple statistical inferences, to which we have devoted the longest but not the most important part of this study, one would perhaps feel that not much is gained by this multiplicity of concepts and principles. However, our choice of examples of inductive arguments is motivated by more general interests. The main reason why the present author has applied the probabilistic and the informationtheoretical measures to simple statistical inferences is the desire, (1) to be confronted with problems of interpretation which are bound to arise in a more or less complex form whenever the general apparatus of concepts in the first section are applied to special kinds of arguments including more exciting ones; and (2) to show the interrelations between the two sets of measures and the two sets of principles of inference.
SEMANTIC INFORMATION AND INDUCTIVE LOGIC* JAAKKO HINTIKKA University of Helsinki, Helsinki, Finland and Stanford University, Stanford, California and JUHANI PIETARINEN Finnish State Council for Humanities (Valtion humanistinen toimikunta), Helsinki, Finland
I. In recent discussion, the hope has sometimes been expressed that the basic ideas of modern decision theory might be used in understanding the adoption and rejection of scientific hypotheses and theories 1. Are not these as good decisions as those that are typically studied in decision theory? If this theory has any pretentions to generality, should it not cover these scientific decisions also? More specifically, it has been suggested that the adoption of scientific hypotheses and theories (call it for convenience induction) can be understood as the process of maximization of certain "epistemic utilities" 2. Among them, there are such concepts as the simplicity and the explanatory power of a hypothesis". Ifthis should turn out to be possible, it may be hoped that some of the methods of decision theory and related disciplines might indeed be applied to the study of the philosophical problems of induction. It is far from obvious, however, that a program of this kind can be carried out. So far, attempts in this direction have failed to achieve any significant positive results 4. In this just a consequence of the particular assumptions
* This study was facilitated by a grant by the Finnish State Council for Humanities (Valtion humanistinen toimikuntai, While engaged in it, the senior author held a Finnish State Fellowship (Valtion apuraha varttuneille tieteenharjoittajille). 1 See e.g. Hempel [1962], especially pp. 153-156, and [1960], reprinted in Hempel [1965] pp. 53-79, especially pp. 73-78. 2 Hempel [1962]. :l On simplicity, see the articles in the third part of Foster and Martin [1966] and the papers listed in their bibliography. On the various measures of explanatory power that have been suggested, see Kyburg [1964], especially PP. 256-258, and the papers Kyburg refers to there. 4 See e.g. Isaac Levi's criticism of Hempel's proposals in Levi [1961]. Besides Hempel's suggestions, there are not many serious constructive ones in literature.
SEMANTIC INFORMATION AND INDUCTIVE LOGIC
97
underlying these attempts, or are there deeper reasons why this is not a useful point of view on the aims and procedures of empirical science? It might seem that a decision-theoretic approach is bound to falsify the character of scientific enterprise. In decision theory, the adoption of an alternative depends on its value or utility. Now in science, it might be maintained, we do not base our decision to adopt or to reject a hypothesis on the utility of doing so, but solely on the question as to whether it is likely to be true or not. The aim of science is to find the general laws by means of which phenomena could be explained, and it is a presupposition of all the explanations we offer that the laws which underlie these explanations are true. Thus the relevant considerations are not questions of utility, but rather questions of truth and vraisemblance or probability. In order to understand scientific methods we should study conclusions and not decisions, it has been maintained 5. From this point of view, the theory of induction will essentially be confirmation theory, perhaps in the first place theory of inductive probability. This purely theoretical search of truth or of approximations to truth seems to be something a decision-theoretic approach cannot accommodate. Or, if we take a more charitable view of it, we might say that this approach presupposes something different from the deductive-nomological model of science which we just adumbrated (Hempel [1962]). It seems to presuppose, rather, something like what Hempel (Hempel [1962] p. 50) calls "the acceptedinformation model of scientific knowledge". This model, according to Hempel, "does not require ... that the statements representing scientific knowledge at a given time be true; rather, it construes scientific knowledge as the totality of beliefs that are accepted at a given time as warranted by appropriate scientific procedures". Nevertheless, it is far from obvious that the two points of view are really incompatible. Even if we are looking for truth and nothing but the truth, we are likely to appraise two independent truths higher than either of them singly, and a logically stronger truth higher than a weaker one. For instance, from a stronger (more comprehensive) law more phenomena can usually be deduced, and it serves therefore better the purposes of scientific explanation than a weaker one. In general, we seem to value truths the higher the more informative they are. Thus a sort of utility, to wit, information, seems to be indispensable even for a hard-core adherent of the view that scientists are 5 For suggestions as to how "conclusions" might be distinguished from "decisions," see Tukey [1960].
98
J. HINTIKKA AND J. PIETARINEN
looking for truth and nothing but the truth. Indeed, those philosophers who have favored the use of "epistemic utilities" have emphasized strongly that they are to be "purely scientific" or "theoretical" utilities. Among these, information will in any case loom especially large. Furthermore, a sort of respect of truth or at least of probability is built into some of the most common decision-theoretic principles. What we are asked to maximize is usually not, and cannot be, the utility of anyone particular outcome, but the expected utility of one's decision, i.e. the average utility of all the different outcomes that one's decision leaves open, each of them weighted with the appropriate probability. Hence there is a sense in which even a decision-theorist is striving as close to truth as possible. Simple considerations of this kind suggest very strongly that in one direction the ideas of scientific enterprise as the maximization of "epistemic utilities" and as the search for truth come closer to each other than one perhaps first realizes. However foreign the general idea of utility is to a scientist, there is a special kind of utility whose expected value he is striving to maximize, viz. information. In this paper, an attempt will be made to show that, although this simpleminded idea has earlier failed to give satisfactory results, it can do so at least in certain simple cases, provided only that the concepts of information (semantic information) and (inductive) probability are understood in an appropriate way. A large part of our task accordingly consists in defining the relevant concept(s) of information 6. In order to keep our discussion within manageable limits, we shall confine our attention to generalizations (closed statements without any free singular terms) that can be formulated in a firstorder language by means of a finite number of monadic (one-place) predicates. 2. The generalizations which one can form from a fixed finite number of such monadic predicates can be divided into two kinds, strong generalizations and weak generalizations. (Inconsistent generalizations are disregarded here and in the sequel.) ,; Measures of semantic information have been proposed earlier by Carnap and BarHillel [1952] as well as Kemeny [1953], especially p. 297. The difficulty of defining the concept of information for the purpose of understanding scientific method from the decision-theoretical point of view is forcefully expressed by Luce and Raiffa [1957], especially p. 324: "Finally, if information is what is desired, then this requirement should be formalized and attempts should be made to introduce the appropriate information measures as a part of the loss structure. This hardly ends the controversy, for decision theorists are only too aware that such a program is easier suggested than executed!"
SEMANTIC INFORMATION AND INDUCTIVE LOGIC
99
I. Each strong generalization (C G ) is expressed by a sentence called a constituent:
C G = (Ex) Ct., (x) &(EX)Ct i2 (x) & ... &(Ex) Ctiw(x) & (x)(Ct;Jx) V Cti2(X) v ... V Ctiw(x)).
(1)
This sentence asserts that there exist exactly w different kinds of individuals in our universe. The complex predicates Cti/x) which characterize these kinds of individuals are called attributive constituents or Ct-predicates. They are identical with the Q-predicates of Carnap's. The number of attributive constituents is 2 k = K if our language system contains k primitive predicates. We can form 2K different constituents, which may be considered as descriptions of possible worlds, since each of our attributive constituents may be either exemplified or empty 7. Thus a strong generalization CGr(r= l , 2, , 2K ) states that each member of our universe instantiates one of w = 0, 1, 2, , K admissible kinds of individuals. An arbitrary strong generalization in which the number of admissible Q-predicates is w, e.g. (1), is denoted by Cwo [fwe have observed a number of individuals and these are found to satisfy c different attributive constituents, then there is a unique simplest constituent compatible with our evidence for which w = c. This constituent asserts that those and only those kinds of individuals are exemplified in the world at large as are already exemplified in experience. This constituent we shall call simply CeIl. Each weak generalization (C g ) can be represented as a disjunction of two or more strong generalizations (constituents): (2) Since each consistent closed sentence of first-order monadic predicate calculus has a complete distributive normal form (Hintikka [1965a] pp. 55-56), every universal hypothesis can be expressed as a disjunction of one or more constituents. In the special case where a hypothesis has only one member in its normal form it is called a strong generalization. 3. In order to define the amount of information which a sentence s contains two principal measures have been put forward (Carnap and BarHillel [1952]). One of them is the content of a sentence cont(s): cont (s) = p( ~ s) = 1 - pes),
(3)
For a more extensive discussion ofconstituents and attributive constituents, see Hintikka [1965a].
7
100
1. HINTIKKA AND J. PIETARINEN
where peS) is the logical probability of s. The informative content is thus the greater the more possibilities it excludes. Information and probability are in this case inversely related (in the sense shown by (3))8. The same basic idea underlies the other main measure of information inf(s): or, equivalently,
inf(s)
= -log2P(s), 1
inf (s) = log, ) 1 - cont (s
(4) (5)
Thus defined, both the functions cont and inf satisfy the requirements which are usually imposed on a measure of the amount of information (here designated by"!"). These requirements are the following: (i) if (SI =>S2) is logically true, then ! (sl)~1 (S2); (ii) l(s)~O; (iii) if s is logically true, then l(s)=O. Our two measures of information differ with respect to the conditions on which they are additive. It is easily seen that cont satisfies the following requirement: (iv.a) 1 (SI &S2)=! (s 1)+! (S2) if and only if (SI v S2) is logically true (that is to say, if SI and S2 have no content in common). On the other hand, the logarithmic measure of information inf is additive as follows: (iv. b) I (SI &S2) = 1 (s 1)+ 1 (S2) if and only if P(SI &S2)= peSt)P(S2) (that is to say, if SI and S2 are inductively independent). If information is measured by the content cont of a sentence, it can be shown on the basis of definition (3) and of condition (iv.a) that
However, definition (4) and condition (iv.b) allow the possibility that
H This general idea seems to have been insinuated into recent discussion through Wittgenstein's remarks in his Tractatus, although it was never explicitly stated by Wittgenstein. It was clearly formulated (independently of Wittgenstein) for the first time by Popper [1935], pp.73-74. More detailed discussions of the concept of information and of its relation to probability are found in Carnap and Bar-Hillel [1952] pp. 237-241, and in Hempel and Oppenheim [1948] especially pp. 170-173, reprinted in Hempel [1965] pp. 245-290, especially 282-288.
SEMANTIC INFORMATION AND INDUCTIVE LOGIC
101
Conditions (i)-~iv) deal with the amount of absolute information of a sentence. The relative information of a sentence Sl with respect to a sentence Sz is defined as follows:
I(Sllsz)=I(Sl&SZ)-I(sz).
(6)
An important special case of(6) is obtained when s. is an empirical hypothesis hand Sz is the available evidence e. The degree of confirmation of the hypothesis given the evidence is
c(h,e) = p(hle) =
p(h &e) pee) ,
(7)
so that, according to (3) and (6),
cont(hle) = cont(h &e) - cont(e) = p(e)(1 - c(h, e)).
(8)
Correspondingly, from definitions (4) and (7), inf(hle) = logzp(e) -logzp(h&e) = -logzc(h, e).
(9)
It can be seen from (9) that the logarithmic measure of relative information is inversely related to the corresponding conditional probability. It is interesting to note from (8) that this is not necessarily the case with the relative content of a sentence.
4. We shall next consider some special measures of semantic information. The first of them are based on a regular and symmetrical measure function m i' which gives to each constituent an equal a priori probability. It defines also the a priori probability of each state-description. This probability is obtained by dividing the weight of each constituent evenly among the state-descriptions that make this constituent true. This is the underlying measure function of Hintikka's system of inductive logic outlined in an earlier paper (Hintikka [196Sb]). Let us suppose now that our domain contains N different individuals. We have observed n of them and found these to represent c different kinds of individuals. We can then set up a strong generalization C w according to which all of the individuals in the whole universe represent w different kinds of individuals. Among these there must of course be all those that are already instantiated in experience, whence w ~ c. What is the information that C w adds to our evidence? In what follows we shall limit ourselves to the case N = 00, so that our formulae can be used as approximations in cases where N is finite but
102
J. H1NTIKKA AND 1. PIETARINEN
sufficiently large. From the expression for the degree of confirmation 9 and from (9) we obtain infj(Cwle) = log,
-i C)(~+i)nJ . i~l [( cw)n+ K-C(K W
(10)
From this one can verify that infA Cwle)~ a: when n grows without limit, except when w=c; then (10) approaches zero. The last-mentioned fact is of special interest. According to the similar measure inf* proposed by Carnap and Bar-Hillel (it differs from our measure inf in that it is based on Carnap's measure function m*) every universal hypothesis (which is not logically true) has as its information inf*(hle)= CD in an infinite domain. That is to say, inf*( ~hle)=O as soon as N = CD. In other words, the information-value of each consistent existential sentence becomes smaller and smaller as the size of universe increases. This is strongly counter-intuitive. This counter-intuitive result is avoided here. According to our measure inf'., the observation of an individual of some kind that has not been exemplified earlier is the more surprising the more individuals of the other kinds have been exemplified in experience. This is exactly what is suggested by our intuitive concept of information. The measure function m j yields also the following formulas: infj(C w ) = -log z(1/2 K ) = K;
infj(e&C w )
=
K
+ n log ,
infj(e)=K-logz.L [K-C(K-. 1=0
I
(11)
w;
c)( -+1 -)"J. ' C
I
(12) (13)
Some features of the behavior of inf', can be seen from the following example. It is seen that for relative information the values of both inf', and contj are obviously unrealistically small. 5. The defect just mentioned can be corrected by assigning a priori probabilities to constituents somewhat differently. " The expression for the degree of confirmation is the formula (13) in Hintikka [l965b]. It can be put in the following form c( C w , e)
~~ I
IK-c (K - C) ( W+ )n. ,~o
i
C
i
103
SEMANTIC INFORMATION AND INDUCTIVE LOGIC
EXAMPLE 1. The language contains two atomic predicates R(x) ("x is raven") and B(x) ("x is black"). The number of observed individuals is n, and these have been found to include black ravens, black non-ravens, and non-black non-ravens, which implies that c = 3. Let the generalization C3 say that all individuals in the universe belong to the observed kinds. In this case C3 entails (x)(R(x) :=> B (x)). Let N = 00 • The following values are then obtained:
.---------------
n
Cj(C3, e)
inf j (C 3Ie)
contjt Cs]«)
infj(e)
cont.t«)
8.247 9.943 11.613 13.273 14.915 16.542 18.157 19.771
0.99670862 0.99898426 0.99968176 0.99989901 0.99996761 0.99998952 0.99999659 0.99999888
---"-----
3 4 5 6 7 8 9 10
0.703 0.760 0.808 0.849 0.882 0.909 0.930 0.947
0.507 0.397 0.306 0.237 0.180 0.138 0.110 0.079
0.00097656 0.00024414 0.00006104 0.00001526 0.00000381 0.00000095 0.00000024 0.00000006
----
Let us suppose a world which contains all possible kinds of individuals. The description of this kind of world is obviously given by the (unique) constituent C K • We can ask: What is the probability of obtaining from this world a sample of (X individuals compatible with C w (i.e. exemplifying only such Ct-predicates as are allowed by C w to be exemplified)? The answer is easily seen to be
If each constituent C; is weighted by this factor we have (14)
Assuming these a priori probabilities is roughly speaking tantamount to assuming that one's world is as unpredictable (as far as generalizations are concerned) as a random world of a individuals. The a priori probability of each state-description is again obtained by dividing the probability of each constituent evenly among the state-descriptions which make the constituent true. Let the resulting measure function be called rna' From (14) we get a new measure of information for strong generalizations: (15)
104
J. HINTIKKA AND J. PIETARINEN
It can be seen from (15) that the absolute information of a generalization is the higher the fewer kinds of individuals it admits of (a being constant). Ifwe have some evidence, the generalization which receives the maximal information-value is C; (since the possibility w
infac(Cwle) = Iog ,
ac K-C(K - c)( w )n-1 w )n + .L . -. . [( -C
1=1
I
C
+I
(16)
Fig. I shows the behavior of (16). We see that the redundancy (lack of information) of Cc increases with accumulating evidence, while the information of the other constituents steadily grows. The highest informationvalue is possessed by the trivial hypothesis C K always when a-cn ; if «>n,
n>a
Fig.1.
n-
The function inf(Cw[e) when w=K, c -c.w-c.K, w=c, and O
the most informative hypothesis is CC ' If a=O, we have equation (10), while the value a=OCJ gives us inf(CKle)=O. This same generalization also has the absolute infac-value 0 when a=oo (see (15»). One more observation can be made here. The greater a is, the higher values of information does the hypothesis C, receive, while the information-values of Cw(w>c) become smaller. The parameter a thus reflects our expectations
105
SEMANTIC INFORMATION AND INDUCTIVE LOGIC
concerning the occurrence of different kinds of individuals: the greater IX, the more kinds are expected to be instantiated. The choice of this parameter determines how rapidly experience affects our a priori expectations. EXAMPLE 2. Take the situation described in Example 1, and let a = K = 4. Then the absolute values of hypothesis C w for the different values of ware
w
o 1 2 3
4
0.0000 0.0015 0.0235 0.1191 0.3765
9.411 5.427 3.070 1.403
1.0000 0.9985 0.9765 0.8809 0.6235
Let again w = c = 3; then the hypothesis C3 receives the following relative values: "--"--"--
n
C(C3, e)
inLx(C3Ie)
cont,,(C3Ie)
inLx(e)
cont., (e) -_.
3 4 5 6 7 8 9 10
0.428 0.500 0.571 0.640 0.703 0.760 0.808 0.849
1.220 1.000 0.807 0.643 0.507 0.397 0.306 0.237
0.00588241 0.00147060 0.00036765 0.00009191 0.00002298 0.00000574 0.00000144 0.00000036
6.226 8.420 10.088 11.930 13.661 15.351 17.032 18.688
__
...•..
---
0.98970579 0.99705879 0.99914215 0.99974469 0.99992256 0.99997610 0.99999251 0.99999762
We observe that an information receiver characterized by a = K cannot be said to be as ready to learn from experience as one with a = 0 (cf. Example 1). Obviously, most people's inductive behavior will correspond to values larger than a = K.
6. As was indicated in the first section of this paper, perhaps the most interesting question that comes up in the theory of semantic information concerns the role of the notion of information in the rational choice of hypotheses and theories. In recent literature the importance of this question has been emphasized especially vigorously by Sir Karl Popper [1935]. Popper has asserted that science does not aim at highly probable hypotheses and theories but rather "at a high informative content, well backed by experience". One can ask what light our concepts of information can throw on this claim. It is not entirely clear what exactly is implied by Popper's thesis. He has sometimes explained himself by saying that one ought to prefer the least
106
J. HINTIKKA AND J. PIETARINEN
probable hypothesis, and indicated that the notion of probability involved here is logical probability a priori. This suggests that he presupposes a concept of information not unlike our concept of absolute information. In Popper's view a statement is the more informative the more possibilities it excludes, i.e. the less probable it is. We agree with his view in the case of a priori probability which corresponds to absolute information, but not in the case of a posteriori probability and the corresponding notion of relative information, for it is perfectly possible that the degree of confirmation of a hypothesis should be inversely related to its relative information. There are hypotheses with a high a posteriori probability which add more information to our knowledge expressed by the evidence than hypotheses with a lower degree of confirmation 10. Here Popper's thesis of the inverse relation of probability and information seems to be inapplicable. There is another point, too. In so far as Popper wants to formulate his point in terms of a priori probability, what his notion of information amounts to is absolute information. Now there is no need for the absolute information and for the a posteriori probability to be related inversely in the same way as absolute information and a priori probability are inversely related. In fact, we can show that in certain important cases the quest of high absolute information leads to the same results as the quest of high a posteriori probability. This holds e.g. for strong generalizations, as can be shown as follows: From (15) and from the corresponding formula for the content measure conta(C w )
=
1-
1 -K
(
L ~J
j=O
)
-_.
(jjwt
(17)
it is seen that a strong generalization is the more informative the fewer kinds of individuals it admits of. ln other words, the most informative (absolutely speaking) strong generalization compatible with evidence is the one which says that as few kinds of individuals as possible are instantiated. This is obviously the constituent C; which says that only those kinds of individuals are instantiated in the world at large as already have been instantiated in experience. When the evidence is extensive enough and (X
SEMANTIC INFORMATION AND INDUCTIVE LOGIC
107
a priori probability, Popperian acceptability and a high degree of confir-
mation go together when one's evidence is large enough. This also shows that Popper's emphasis on the inverse relation of probability and information is not representative of all the interesting situations that can arise here. This leads us to ask what exactly Popper's claim amounts to. He does not of course advocate judging hypotheses on the basis of their (absolute) information alone, but requires also experiential backing. Both these factors are intended to be included in his notion of corroboration, which does not measure probability or informativeness alone but some sort of balanced result of both (plus possibly other factors as well). In choosing a hypothesis, one is according to Popper's lights supposed to maximize corroboration. One might expect that corroboration is the expected value of some utility which includes information. Isaac Levi has shown, however, that on the basis of the definitions which Popper himself has put forward his notion of corroboration cannot be interpreted in this way, except on pain of the nonsensical result that a scientist's utilities vary with his evidence U, This suggests that Popper's own way ofcharacterizing the concept of corroboration is somewhat ad hoc; although he tells us to value information, information cannot on his own view be understood as a utility in the sense of decision theory. Levi's negative results reinforce the larger question whether any approach to induction in terms of "epistemic utilities" has much hope of success. We touched this question above in the beginning of this paper. Here we are confining our attention to the possibility of using information as the only epistemic utility. In order to qualify as a utility, a measure of information must of course satisfy the usual Von Neumann-Morgenstern utility axioms. Among other things, it must not vary with evidence; hence the only kind of information which we can use here is absolute information. Does it yield natural results when used as the sole utility? Let us consider a theoretician (i.e. a person whose only relevant utility is information) who decides to accept a hypothesis h. This hypothesis has a certain information-content cont(h). If h is true, the utility of his decision is the valid information he has gained, i.e, cont(h). If h is false, it is natural to say that his disutility or loss is measured by the information he lost because of his wrong choice between h and < h, i.e. by the information he would have 11 As Levi puts it: "The fact that utilities vary withp(h, e) amounts to saying that scientists change their preferences for the various possible correct answers and mistakes with changes in the evidence" (Levi [19611 p. 623).
108
J. HINTIKKA AND J. PIETARINEN
gained if he had accepted ~h instead of h. This is measured by cont( ~h), whence the utility in this case is -cont( ~h)12. In order to take into account whatever backing experience has given to one's hypotheses we have to consider expected utilities, and strive to maximize them. If we adopt the utilities just explained, we can see what happens in our inductive logic. For h, the expected utility on the basis of the evidence e or U (h, e) is (18) p(hle)·cont(h) - p(~ hle)'cont(~ h). By substituting the definitions of cont(h) and cont( ~h) here and by simplifying we obtain (19) p(hle) - p(h), which is the increase in the probability of h that is brought about by the evidence e 1 :3. Let us see what this is in the different cases, and consider strong generalizations first. The expected utility of the acceptance of a strong generalization C w is in our terms U (C w' e) = c( C w ' e)' cont a ( C w ) - c ( ~ C w ' e)·cont a ( ~ C w ) ' (20) This yields the following function to be maximized: (21) where the a posteriori and a priori probabilities of the generalization are determined by the measure function rna (m, is obtained from this by putting a=O). It is clear now that the rule "maximize your expected utility" recommends the acceptance of the generalization Cc , since in formula (21) of the expected utility the a posteriori probability has its highest and the a priori 12 In his uses of the idea of maximizing expected information, Hempel in effect takes (apart from the role of evidence) the disutility of a wrong guess that h to be cont(h) instead of conttv h). This seems to be unnatural, for if ~ h is the case there is no true statement which would have this content. If ~ h is the case, the only relevant available "utility" is the information contained in ~ h. 13 Expression (19) has in fact been proposed by Levi for the explanatory power of h; see Levi [1963]. Finch has also put forward an expression which behaves essentially in the say way; see Finch [1960]. Expression (19) nevertheless yields unnatural values if the relevant a posteriori probabilities are taken to be those given by Carnap's confirmation function c*(h, e). For further suggestions for functions to be maximized, see the summary in Kyburg [1964] p. 257. It can be shown that none of the measures Kyburg lists except those mentioned above can be interpreted as expected values of an epistemic utility which would be independent of evidence.
SEMANTIC INFORMATION AND INDUCTIVE LOGIC
109
probability its lowest value in the case of this generalization (presupposing that O
The probabilities of both j- and a-system can easily be transformed in the usual way into the corresponding values of information. It is evident from (22) that the a posteriori probability of a generalization can always be raised by adding new disjuncts to its normal form. Here we have to admit, indeed, that Popper is right and that this is not the route to good generalizations. What shall we do here? Popper would presumably say again that we have to look for a hypothesis with a high informative content and a low probability. This means that we should choose a hypothesis which has a high degree of corroboration. But, as we have seen, there are good reasons for trying to find other measures for this concept than those suggested by Popper. One possibility is to take as the degree of corroboration of a generalization corr( c; e) = min [c( CGj' e)],
(23)
j
where the mII1ImUm is taken over all those constituents CG j for which c(CG j , e)#O. (If there are no such constituents in the normal form of Cg , corr(Cg , e)=O.) (23) has been proposed by Hintikka [1967] as a partial explication of corroboration. This measure satisfies some of Popper's requirements, e.g. itis usually the higher the more informative a generalization is. This can be easily seen, if you think a case in whichC; :::J Cg Z is logically true (i.e. one in which all the constituents of Cg , are included in the constituents of Cg ,) but Cg , is nevertheless compatible with our evidence. If the evidence remains the same in both cases, we obtain by definition (23) (24)
and by (9) (25)
110
J. HINTIKKA AND J. PIETARINEN
while (22) gives us
(26)
We cannot regard the degree of corroboration defined by (23) as the average (estimate) of any epistemic utility, however, since the utility in question would again vary with our evidence, as one can verify from (8). Instead of using this measure, we shall use as our sole epistemic utility the obvious information-theoretical measure cont(C,,) = 1 - p(C,,) = 1 -
L p(C
j= 1
G)
(27)
,
which remains constant with changing evidence. Equation (18) then gives us the expected value of this utility:
U(Cy,e) = p(C"le) - p(C g ) =
=
L c(CGj,e) - I
j=l
L
j=l
j=l
p(C G)
[c(CG}'e)- p(C G)]·
(28)
It may be noticed from (28) that the expected utility grows when a new constituent is added if and only if the degree of confirmation of this new constituent is higher than its a priori probability. For small values of n there can be several such constituents, but when n is large enough, there is only one, namely C c When n is large enough, it is likewise the case that for all the other constituents C"', c(C"" e)-p(Cw ) is negative; then omitting such a constituent increases the value of (28). From this it follows that the generalization which contains as its member the constituent C; receives the highest expected utility, provided that the number of observed individuals is sufficiently great. Here again the suggested measure of rational acceptability of scientific hypotheses recommends the generalization Cn this time as the best choice among all generalizations and not just among strong generalizations. AU this is of course true only for a large amount of evidence; if the available evidence is relatively small, we have to be, according to our rule, less daring in making generalizations. Since C; is for a sufficiently large body of evidence the best generalization we can make, we can thus say that for generalizations (closed sentences) and for evidence consisting of observations of individuals Hempel's notion of epistemic utility works very well when this utility is identified with one of our information measures cont. The rule "maximize expected utility" which in this case means "maximize expected information" leads us to accept, when the evidence is large enough, exactly those generalizations which a rational
SEMANTIC INFORMATION AND INDUCTIVE LOGIC
111
theoretician obviously would accept (Hintikka [1965b] pp. 277-279). Another, slightly more complicated argument shows that asymptotically similar results are obtained when we use inf instead of cont. This bears out to a considerable extent Hempel's conjecture that the difficulties that had encountered earlier attempts to apply the concept of epistemic utility are due to the particular ways of defining this concept which had been used 14. However, what we have to be careful about is not so much the definition of information in terms of inductive probability, but rather the choice of the underlying concept of inductive probability. At the same time, our observations show that expected utility has in the systems of inductive logic considered here somewhat the same properties as are ascribed by Popper to his notion of corroboration. Instead of adopting Popper's own characterization of this notion, one might perhaps adopt (28) as a tentative definition of corroboration and see to what extent Popper's claims can be interpreted and justified on this basis. At any rate, we could in this way make the discussion of this question a part of a larger framework of utility theory and decision theory-e. References CARNAP, R., 1950, Logicalfoundations ofprobability (University of Chicago Press, Chicago; second edition, 1963) CARNAP, R. and Y. BAR-HILLEL, 1952, An outline of a theory of semantic information, Techn. Report No. 247, Massachusetts Institute of Technology, Research Laboratory of Electronics; reprinted in Y. Bar-Hillel, 1964, Language and information (AddisonWesley, Reading, Mass.) pp. 221-274 FINCH, H. A., 1960, Confirming power of observations for decisions among hypotheses (I), Philosophy of Science, vol. 27, pp. 293-307 FOSTER, M. H. and M. L. Martin (eds.), 1966, Probability, confirmation and simplicity (The Odyssey Press, New York) HEMPEL, C. G., 1960, Inductive inconsistencies, Synthese, vol. 12, pp. 439--469; reprinted in Hempel [19651 HEMPEL, C. G., 1962, Deductive-nomological versus statistical explanation, Minnesota Studies in the Philosophy of Science, vol. 3, eds. H. Feigl and G. Maxwell (University of Minnesota Press, Minneapolis) pp. 98-169 HEMPEL, C. G., 1965, Aspects of scientific explanation and other essays in the philosophy of science (The Free Press, New York) 14 See Hempel [19601 p. 77: "Quite likely, our crude definition of epistemic utility is at fault". 15 Most of this study has been carried out by the junior author on the basis of suggestions by the senior author.
112
J. HINTIKKA AND J. PIETARINEN
HEMPEL, C. G. and P. OPPENHEIM, 1948, Studies in the logic 0/ explanation, Philosophy of Science, vol. 15, pp. 135-175 HINTIKKA, J., 1965a, Distributive normal forms in first-order logic, in: Formal Systems and Recursive Functions, eds. J. N. Crossley and M. A. E. Dummett, Proc. Eighth Logic Colloquium, Oxford, 1963 (North-Holland Pub!. Co., Amsterdam) pp. 47-90 HINTIKKA, J., I 965b, Towards a theory 0/ inductive generalization, in: Proc. 1964 Intern. Congress for Logic, Methodology and Philosophy of Science, ed. Y. Bar-Hillel (NorthHolland Pub!. Co., Amsterdam) pp. 274-288 HINTIKKA, J., 1967, Induction by enumeration and induction by elimination, in: Proc. Intern. Colloquium in Logic, Methodology and Philosophy of Science, London, 1965, ed. 1. Lakatos (North-Holland Pub!. Co., Amsterdam) KEMENY, J. G., 1953, A logical measure/unction, J. Symbolic Logic, vo!. 18, pp. 289-308 KYBURG, H. E., 1964, Recent work in inductive logic, American Philosophical Quarterly, vol. 1, pp. 249-287 LEVI, I., 1961, Decision theory and confirmation, Journal of Philosophy, vo!. 58, pp. 614-624 LEVI, I., 1963, Corroboration and rules 0/ acceptance, British Journal for the Philosophy of Science, vo!. 13, pp. 307-313 LUCE, R. D. and H. RAIFFA, 1957, Games and decisions (John Wiley, New York) POPPER, K. R., 1935, Logik der Forschung (Julius Springer, Wien). Eng!. trans!. with notes and appendixes with the title: The logic 0/ scientific discovery (Hutchinson, London, 1959) TUKEY, J. W., Conclusions versus decisions, Technometrics, vo!. 2, pp. 423-433
A TWO-DIMENSIONAL CONTINUUM OF INDUCTIVE METHODS * JAAKKO HINTIKKA University of Helsinki, Helsinki, Finland and Stanford University, Stanford, California
1. In this paper, an attempt will be made to compare with each other systematically a number of inductive methods. They include Carnap's A-continuum of inductive methods, explained in Carnap [1952], in Carnap and Stegmiiller [1959], and in Kemeny [1963]1, as well as the systems of quantitative inductive logic which have been outlined in Hintikka [1965b] and [1965c]. All these different systems will be fitted into one and the same continuum of inductive methods. The members of this continuum are characterized by two parameters which we shall call A and a. Of these, A is essentially the same as its namesake in Carnap's A-continuum. The main novelty here, as compared with Carnap's systems, is that in most cases generalizations (closed sentences) which are not logically true will receive asymptotically well-behaved degrees of confirmation (different from zero) also in an infinite universe. Carnap's A-continuum, in which this is not the case, will become a rather special case, obtainable by giving the other parameter a a certain special value. In fact, the basic ideas of the new continuum of inductive methods are seen most clearly in the case of an infinite universe.
2. First, we have to give certain preliminary explanations. The same sort of languages will be considered as are considered e.g. in Carnap [1950] and in Carnap and Stegrnilller [1959] except that our discussion will be entirely restricted to the monadic case (i.e. only one-place predicates are assumed to
* While working on this paper the author has held a Finnish State Fellowship (Valtion apuraha varttuneille tieteenharjoittajillei. 1 I. J. Good seems to be the only recent writer who has pointed out that some of the most salient features of Carnap's A-system were anticipated by W. E. Johnson, including the derivation of the characteristic function of the A-system from the assumption that it depends only on the total number of observed individuals and on the number of observed positive instances. See Good [1965] p, 26, and Johnson [1932].
114
JAAKKO HINTIKKA
be present). We shall use the terminology and notation of earlier works, such as Carnap [1950] and [1963], Hintikka [1965b] and [1965c] whenever applicable. As in the author's earlier papers, it is normally assumed that we are given k monadic predicates (1) PI (X), P2(X), ... .P, (x). Out of these we can build K = 2k Q-predicates (as Carnap calls them); we shall call them, taken in some suitable order, Ct;(x) (i= 1,2, ... , K). A Q-predicate says that certain specified predicates among (1) belong to X and that the rest do not belong to x. Different selections among predicates (1) give us different Q-predicates. From Q-predicates we can build constituents; a constituent says that certain specified Q-predicates are instantiated and that they are the only instantiated Q-predicates. Different selections of Q-predicates give us different constituents. An arbitrary constituent which says that exactly w Q-predicates (which it specifies) are instantiated will normally be called Cwo For example, if we have two primitive predicates (1), say B(x) ("x is black") and R(x) ("x is a raven"), each of the Q-predicates says which of these belong to an unspecified individual x. In other words, the Q-predicates are the following:
B(x)& R(x) B(x) & '" R(x) "-'B(x)& R(x) "-' B(x) & ~ R(x)
("x ("x ("x ("x
is is is is
a a a a
black raven") black non-raven") non-black raven") non-black non-raven").
Let us call these Ct ; (x), Ct 2 (x), Ct 3 (x) and Ct 4 (x), respectively. Constituents are formed from them by specifying which of them are exemplified and which ones are not. Here the second specification can also be given by saying that each individual in the world must belong to one of the exemplified Q-predicates. For instance, if there were only black ravens and non-black nonravens in existence, the following constituent would be true:
If Ct 3(x) is not exemplified, i.e. if all ravens are black, but if all the other Q-predicates are exemplified in the world, the following constituent is true:
If there are also non-black ravens in existence, i.e. if all the different Q-predi-
A TWO-DIMENSIONAL CONTINUUM OF INDUCTIVE METHODS
115
cates are instantiated, the following constituent is true:
(Ex) Ct 1 (x) & (Ex) Ct2 (x) & (Ex) Ct3 (x) & (Ex) Ct 4 (x) & (x)[Ct 1(x) V Ct2(x) V Ct 3 (x) V Ct 4 (x)]. Here the disjunction happens to be redundant, as it usually is not. This illustrates the structure and the meaning of constituents. Fuller discussion of Q-predicates and constituents as well as of their most important properties is found in Hintikka [1965a]; cf. also Hintikka [1965b]. 3. The motivation behind the new two-parameter system may be seen by means of the well-known formula of Bayes, applied to a constituent: (2)
The notation used here is self-explanatory. In the sequel, it will be assumed that our evidence e is of a certain special kind. It will be assumed that it consists of the observation of exactly n individuals which exemplify a total of c Q-predicates, which are
Cti,(X),Cti2(X), ... ,Ctic(X).
(3)
It will also be assumed that the number of observed individuals instantiating Ctij(x) is n i: thus c
The value of (2) is determined as soon as the different values of p (C w ) and of p(eICw ) are fixed. How are they to be determined? We shall first consider the a priori probabilities p( C w ) ' 4. In Carnap's A-continuum, we have in an infinite universe p(CK)=l, p(Cw)=O whenever w#-K. As a result, all factual generalizations have their probability (a priori and a posteriori) identically equal to zero. In order to avoid this consequence, we have to assign different non-zero a priori probabilities p( Cw) to the different constituents Cwo Obvious symmetry considerations argue for an assignment which depends on IV only. Since we want to keep as close as possible to Carnap's A-system, let us see first how these probabilities are determined in his system. Given a fixed constituent C w , the probability that a individuals are compatible with it (in the sense of exempli-
116
JAAKKO HINTIKKA
fying Q-predicates which are allowed to be instantiated by C w ) is
(wA/K) 1 + (wA/K) 2 + (wA/K) A 1+A. 2+A. --------
- - - - - - - _ ...
a-I + (wA/K) ----a-l+A
----
(4)
In the sequel, we shall designate the product of the numerators and of the denominators here by n(a,wA./K) and by n(a, A), respectively. The obvious way of extending (4) to values of the arguments a, w, A and K for which it is not automatically defined is to identify it with
rea + wAjK)·1(A.) I't« + A)·r(wAjK)
(5)
where rex) is the usual gamma-function. When 11. is large, (4) is close to the probability p (Cw ) in a universe of 11. individuals. For smaller values of 11., this probability is easily seen to be
i: (_ ly(~)n(!J.'(w -
i=O
\l
i)A/K). n(!J.,A)
(6)
We want to choose the a priori probabilities p( C w ) in such a way that we can see as clearly as possible the consequences of our choice. One simple way of doing so is to rely on a comparison between the universe we are considering (for the time being it is assumed to be infinite) and a finite Carnapian universe with 11. members. The larger this number 11. is, the less likely are constituents to hold in the latter universe, as can be seen from (6). Since all the other consistent generalizations (closed sentences) are representable as disjunctions of constituents, the same holds for generalizations at large. Hence it lies close at hand to specify the probabilities p( C w ) so as to make them essentially tantamount to the same probabilities in a finite Carnapian universe with 11. individuals. Then generalizations are as likely, or as unlikely, to hold in our infinite universe as in this Carnapian universe. Here the parameter 11. gives us a fairly good intuitive idea of what the probabilities in question are, for the effects of the size of the universe on the chances of the truth of generalizations are on the whole easy to guess. This enables us to appreciate the effects of our choice of the probabilities p( C w ) . It may be seen, however, that the most natural choice here is not to identify p(C w ) with (6) but to make it proportional to (4) (or (5). As we already pointed out, this does not make much difference when a is large. The advantages of (4) over (6) when 11. is small are best shown by means of an example. When 11.= 1, the only constituents for which (6) is different from
A TWO-DIMENSIONAL CONTINUUM OF INDUCTIVE METHODS
117
zero are the ones which say that only one Q-predicate is instantiated. Such an a priori distribution of probabilities to the constituents is not of great use in other universes. Thus we are led to make the a priori probabilities of constituents proportional to (4), i.e. to put (7) Here A may in principle depend on K (and perhaps also on arguments which are not explicitly indicated in (7)). This dependence on K may be brought out by writing (7) as follows: (7)* This will be our assignment in an infinite universe. We shall return to the finite case later. The meaning of (7) may also be explained by saying that according to (7) a constituent has the same probability as there is of obtaining in what we shall call an atomistic universe a sample of C( individuals which is compatible with this constituent, when the individual probabilities are calculated in terms of Carnap's A-system. By an atomistic universe we shall mean a universe in which the probability that a generalization holds is determined entirely on the basis of probabilities that the individuals falling within the scope of the generalization satisfy it, these individual probabilities being determined independently of the probability of all generalizations. 5. It is of interest to see as clearly as possible the role of the parameter C( in (7). One way of looking at it is as follows: In order for a constituent C w (w
U8
JAAKKO HlNTIKKA
In this respect, our new parameter o: is comparable with Carnap's old A which is also an index of the weight of the a priori (logical) factor in induction: the greater ), is, the closer our singular predictions are to their a priori values which are dictated by the considerations of symmetry. Similarly, the larger a is, the closer p( C w ) with w-c K is to its a priori value 0, we might say. The only difference between the two parameters is simply the fact that they pertain to different kinds of inductive inferences: o: is relevant to inductive generalization while A pertains to the singular predictive inference, more generally, to an inference from particulars to particulars. This basic difference implies the other ones: While symmetry considerations suggest nice finite values for singular inductive predictions, they argue for negligible a priori probabilities for constituents (and therefore for other generalizations) with the sole exception of C K when the universe is very large. 6. So far, we have determined one of the two main ingredients of Bayes' formula (2), viz. the probabilities p( C w ) . How are the likelihoods p( e]C w ) to be determined which constitute the second main ingredient of (2)? This is tantamount to deciding, after the a priori probability of a constituent has been fixed, how it is to be divided among all the state-descriptions that make this constituent true. In deciding this, we shall again keep as close to Carnap's A-continuum as possible. We cannot use Carnap's characteristic function (8) directly, for that would be inconsistent with the a priori probabilities which we have assigned to constituents. (In (8), n j and n have the meaning explained above; it is supposed to give the probability, on the kind of evidence we are presupposing, that the next individual has the Q-predicate Ctij(x).) However, a very small modification serves our purposes. Instead of (8), we can use as a kind of conditional characteristic function (9) where w is of course the number of Q-predicates occurring in the constituent Cwo What this means is that we consider a world in which C w is true and in which exactly w Q-predicates are therefore instantiated as if it were a Carnapian universe in which w Q-predicates can be defined. In other words, we consider the w Q-predicates which C w requires to be instantiated as if they
A TWO-DIMENSIONAL CONTINUUM OF INDUCTIVE METHODS
119
constituted a family in the sense of Carnap [1950] pp. 76-78. It is easily seen that this gives us (as characteristic functions do in general) a distribution of a priori probabilities to state-descriptions which in the case of an infinite universe is consonant with our earlier assignment of a priori probabilities to constituents. The use of (9) as distinguished from (8) may be argued for in terms of a kind of objectivism. If our characteristic function is to have any objective significance, it must not depend on the total number K of the Q-predicates which we can distinguish from each other. This is to some extent a conventional factor, which can e.g. be changed by introducing new meaning postulates. Compared with K, the number w of the Q-predicates that are in fact instantiated is a much more objective factor, which e.g. cannot be affected by new linguistic conventions. In any case, (9) is as close to the spirit, and the letter, of Carnap's A-system as we can get after having decided to give all constituents finite a priori probabilities along the lines of (7). On the basis of (9) we obtain, in the case of an infinite universe, the following expression for the likelihoodp(eIC,J:
Afw 1 + AjW 2 + AjW A 1+..1. 2+..1.
n1
1 + Afw
-
n1
-
1 + A.
A.jw n1
nz - 1 + Ajw - -A./w -------- - ... = n1
+ nz -
1 + A. nl
+ nz + A.
+ A.
[Cn
1 + A.jw
-- _ - - - - - - - - ..
n1 t:
j= 1
+1+A
(A.)J/ . n n(n,A). W j,
(10)
7. On the basis of (7)* and (10), we can calculate the a posteriori probability (degree of confirmation) p( Cwle) of a constituent C w, the a priori probability pee) of the kind of evidence we have presupposed, as well as the a posteriori probability (on this evidence) that an unknown individual has the Q-predicate Ct., (x), which we shall call p(hle). We obtain in the general case in which A. may be a function A. (w) of w, the following results: p (Cwle) is
W'A.(K»)/ _} { tt ( rx'K-- n(n,A.(w»
}Jl tt. C
(
A. (w») nv--;-
.
K-C[(K -. c){-(rx, (c------+ i).X(K»)/---------}--C (--- f(~+ i»)-J' .L n(n,A.(c + i») n n ,=o!
IT
K
)=1
tt
j , ---.--
c+!
(11)
120
JAAKKO HlNTlKKA
KiC[(K .=0
~ c)n(a, (C~~)'A(K»).n n(n j, A(C_-J-.i))!n(n,A(C + i»)J K
(
f (~) .
'-'-
,=0
I
J= [
C
------
tt
+ I -'----_
--
(IX, iA (K»)
... _._._._._-
K
(12)
while p(hle) is p(h & e)/p(e), where p(h &e) is like (12) except that n1 has been replaced by n [ + I and n by n + 1. (This presupposes of course that n 1> 0.) We have assumed in (11)-(12), as is natural to do, that in the expressions for a priori probabilities A is the same function of K as it is elsewhere of w. If it is assumed that A is a numerical constant independent of wand i, we obtain the following simpler expressions:
n(lX, wAlK)
p(Cwle)=K_c
j~O
p(e)
(h Ie)
p
K [(
c
Il n(nj,A/w)
~ c) n(IX,(c + i)A/K) }J n(n j, A/(C + i)] c
j=l
....-
(13)
...
1
~t: (K ~ c) n(IX,(c + i)A/K) )}[ n(nj,A/(c + i»)
=---K
._ _
---
i~O [(~)n(lX,iA/K)] 'n(n,A)
=
(14)
.
KiC[(K~C)n(IX,(c+i)AjK).6 n(nj,A/(c+i»)(n[ + A/(C+i»)] 1-0
~t] (K ~c) n(IX,(c+ i)A/K) jD n(nj,A/(C+ i))] (n+A) J- [
....
1
...
(15)
The formulas (10)-(15) establish the character of our two-dimensional system of inductive methods in the case of an infinite universe. Before examining the consequences of these basic formulas, we have to consider what we can say of the finite case. 8. Of the formulas just mentioned, (12) and (14)-(15) are applicable also in a finite universe in the sense that they give consistent results. What we propose to do is simply to take these formulas and continue to apply them in the finite case in the same way we did in the infinite case. The rest of one's inductive logic is thereby determined. In fact, (12) determines the probabilities p(hle) (for the kind of h which we have decided to consider) which in turn determine the characteristic function of one's system. As we know from
A TWO-DIMENSIONAL CONTINUUM OF INDUCTIVE METHODS
121
Carnap's work, e.g. from Carnap and Stegmiiller [1959] pp. 210-215, this determines the whole system. What we cannot do in the finite case is to use (11) and (13) to calculate the a posteriori probabilities of constituents. These probabilities p( Cwle) are determined by the characteristic function, however, and can be calculated by means of more complicated methods. Another thing which we cannot do in the finite case is to motivate the choice of (12) in the same way we did in the infinite case. However, a closely similar explanation can be given. We may even use the same formal expressions for the purpose, but with a new interpretation. In these formulas, we may consider C w ' not as a statement that the constituent in question is "actually" true, but as a statement to the effect that it is true "in principle". What this means can be explained as follows: What C w gave us in the infinite case was a certain method of assigning probabilities to singular predictions, in the way illustrated by the transition from (7) and (10) to (12). This is comparable to the choice of an urn; it gives us what Good calls a type I distribution 2. The probabilities p( Cw ) mean that the choice of different methods of this kind are themselves governed by a probability distribution, which now is (in Good's terminology) of type II. (Instead of a single urn, we are given a collection of urns, as it were.) There is no reason we cannot use this two-stage procedure of determining the distribution of probabilities to state-descriptions also in the finite case. The only new thing that happens here as compared with the infinite case is that certain state-descriptions st receive non-zero probabilities p(st\C w ) even though they do not make C w true. In such a case, it lies close at hand to say that C; is "in principle" true, that is to say, we have distributed the different Q-predicates to individuals in accordance with the type I distribution which corresponds to C w , but that it is not "in fact" true because it so happens that there are no individuals exemplifying certain Q-predicates that C w requires to be instantiated. This failure of C w to be "actually" true is then due to the "atomistic" aspect of the distribution of probabilities to state-descriptions; it is due to the accidents of the assignment of properties to individuals, quite apart from the assignment of probabilities to the truth of the different constituents "in principle". Our procedure thus presupposes a kind of distinction between accidental and nomic generality. (A constituent is nomically true if it is true both "in principle" and "in actual fact", accidentally if it is "actually" true but not 2 See Good [1965] pp, 8-10. This sense of expressions like "type I" is of course different from Pearson's.
122
JAAKKO HINTIKKA
true "in principle". This is easily generalized for other generalizations.) We can even calculate (roughly) the probability that some constituent which is "in principle" true should fail to be "actually" true. As might be expected, this probability is normally the higher the greater A is, but the smaller the greater a. is. The presupposition is not needed in the case of an infinite universe, where it might perhaps be felt to be more problematic than in finite domains of individuals. 9. One of the main things we are interested in is to see how some of the different systems of quantitative inductive logic which are known from the literature assume their place in the two-dimensional continuum. For this purpose, consider first what happens when a.-HfJ. In order to see this, let us divide both the numerator and the denominator of(15) by n(a., A). We obtain an expression whose behavior when a.~ 00 is determined essentially by what happens to the ratio
n(ex,(c + i)A/K) . _._,.,---n(a.,A)
(c + i)A/K 1 + (c A
+ i)A/K +A
------
I
ex-I _.+(c+ i)A/K __ .., , - Ci-I+A
(16)
when Ci~oo. This is easily seen by means of Stirling's formula which gives us, when o: is very large in relation to other parameters, the following approximation to (16):
r (A) ---------Ci r {(c + i)A/K}
((c+i)ljK) _).
.
(17)
When a.~ 00, (17) approaches 0, except when c + i = K; then it approaches 1. Then the sums of (15) contain only one term different from zero, and (15) reduces to the form
n1
+ (A/K)
----------
n+A
(18)
which gives us exactly Carnap's characteristic function for his A-system. The same result is established in the same way without making use of the assumption, which is implicit in (15), that A does not depend on w. Since the characteristic function determines the whole of one's inductive logic, this means that we obtain Carnap's A-system as a special case of our two-dimensional continuum of inductive methods when Ci~ 00. In view of the meaning of «, we may thus say that Carnap's A-continuum is related to our two-dimensional continuum in the same way as the particular system c+
A TWO-DIMENSIONAL CONTINUUM OF INDUCTIVE METHODS
123
which Carnap considers is related to his A-continuum, in the following sense: In the same way as singular inductive predictions are in a system based on c+ judged completely on a priori grounds, in the same way inductive generalizations are in Carnap's A-system judged on a priori grounds only. As a consequence, no true learning from experience takes place in so far as generalizations are concerned. As special cases we obtain all the systems considered by Carnap. For instance, the system based on c + is obtained from our continuum when a=oo, A=oo. The system based on c* is obtained by putting cx=oo, A=K. 10. Another interesting special case of our two-dimensional continuum obtained when A--+O. In view of the meaning of A, this amounts to judging singular inductive predictions on completely a posteriori grounds. From (15) it is seen that in this case the characteristic function assumes the form (19)
which is known as the "straight rule". (We assume here that A does not depend on w.) It is interesting to see that this result is obtained no matter what the value of the other parameter a is, provided it is finite. It is also readily seen that we then havep(Ccle)=l for that particular constituent C, which says that only those Q-predicates are instantiated in the world at large as are already exemplified in the evidence e, andp(Cwle)=O for all the other constituents. This obviously means judging inductive generalizations completely on a posteriori grounds. The significance of our observations may thus be expressed by saying that if singular inductive predictions are judged completely on a posteriori grounds, the same has to be done to inductive generalizations, too. 11. If we put a = 0, we are casting our lot completely on a posteriori considerations in so far as inductive generalizations are concerned. What happens in this case is seen from (5): all the a priori probabilities p( C w ) turn out to be proportional to the same constant, and therefore all equal to 1/2 K • This may seem to be a highly a priori method of determining the probabilities p( C w ) ' However, upon a closer examination one discovers that the degrees of confirmation of constituents p( Cwle) approach in this case their asymptotic values more rapidly than at other values of o: > O. Informally speaking, we assign high degrees of confirmation at a relative slight provocation. This brings out the sense in which the use of the value rx=O amounts to a predominant reliance on a posteriori factors.
124
JAAKKO HINTIKKA
In fact, in this case (13) becomes c
Il n(nj,Afw)
j=l
~t~(K ~ c) j1]
ln(n
or, generalized,
KiC(K - c) i~O
i
1
j,AI(c +
(20)
i))
fI
[r(~~)r Ijni-tj~/(c_~i)}) [r(AI(c + i))YJ=l r(n j + (AjW))
and (15) becomes
~t: [(K ~ c) j1]l {r(nj + (A/(c + i)))/r(AI(c + i))} (n -
------
-------------
1
+ {A/(c +
~t: [(K ~ c) j1]l r(n j + (AI(c + i)))/r(AI(c + i))] (n + A)
(20)*
i)n] . (21)
We have presupposed here that A is a numerical constant. The behavior of (20)* at large values of n is seen fairly easily. It depends essentially on the behavior of the product in (20)*. In virtue of Stirling's formula it may be written in the following approximate form
Il nj C
j=l
.
({l/(c+i)}-l/w)~
(22)
When w> C, there is at least one value of i such that the exponent is positive. When n grows without limit, there must be at least one n j which also goes to infinity. For these values of i andj (22) grows without a limit, which implies that (20)* decreases to zero. When w = c, the exponent of (22) is either negative or zero at all values of i, and it is negative at all values but i=O. For all the other values (22) approaches zero when n grows indefinitely, since there must then be at least one nj which also grows without a limit. This means that (20)* approaches one when n goes to infinity (while C remains constant). This shows that the degrees of confirmation of generalizations behave asymptotically in this case (a=O, A a numerical constant) exactly as we might hope them to behave. Since the only thing that is affected when a changes is the distribution of a priori probabilities to constituents and since this does not affect the asymptotic behavior of (13), the same holds for other (finite) values of a. 3 This generalizes what was found in Hintikka [1965b] for one particular system. It is to be noted, however, that although the asymptotic behavior of the degrees of confirmation of
3
A TWO-DIMENSIONAL CONTINUUM OF INDUCTIVE METHODS
125
A further observation can be made on the basis of (22) if we rewrite it into the form ]«l/(C+i n-l /W)}.
C
[
Il n
j=l
(22)*
j
Since the n/s are subject to the condition l.,nj=n, it is seen from (22)* what for a constant n is the best sort ofevidence for that particular constituent with w=c for whichp(Cwle) approaches one when n~oo. The best evidence is the one which maximizes Iln j, and this happens when the numbers nj are all as close to each other and to n] c as possible. In others words, the best situation we can hope for in making inductive generalizations is one in which the n observed individuals are distributed evenly among the c Q-predicates which are exemplified in experience. Then we have as much evidence for the generalization which says that no other Q-predicates are exemplified in the world as we can hope to have on the basis of having observed no more than n individuals. All this is very natural, for if the observed individuals were unevenly distributed, this fact would lead us (intuitively speaking) to wonder whether there might be similar unobserved irregularities which will cause one of the previously unobserved Q-predicates to be instantiated. It means that a form of the requirement of the (quantitative) variety of instances is enforced in our systems. All this of course holds only asymptotically, i.e. for sufficiently large values of the numbers of observed individuals. Furthermore, our argument holds only if our present assumptions are satisfied (A a finite numerical constant greater than zero). 12. We asked earlier what happens when IX~ 00, i.e. when generalizations are judged on a priori grounds only. It may be asked likewise what happens when A~OO, i.e, when singular inductive predictions are judged solely on a priori grounds. In order to see this, we may consider (7)* which may be written as follows:
~(i +l)(~+D···(~+ y)
K
i~O
(K) Ki(iK +~1)(K +12) ... (K +-A-1)· i
i
i
(23)
IX -
generalizations is quite natural in all the different systems one obtains by giving ex different values, the preasymptotic behavior of the degrees of confirmation need not be equally natural.
126
J AAKKO HINTIKKA
When A-+OO, this approaches (w/Kt
itO( ~)(~~)~.
(24)
In the same way we can see that (10) approaches (l/w)" when },-+oo. Substituting these val ues into (13) and simplifying we obtain for the degree of confirrnation of C", the following expression (25) This is a direct generalization of the expression for the degree of confirmation of a constituent which was obtained in Hintikka [1965b]. It reduces to the old result when IX=O. It is also seen without much difficulty that the characteristic function one obtains here is in the case of an infinite universe the same as in the system presented in Hintikka [1965b]. As I pointed out in the earlier paper, in the case IX =0 we obtain an expression for the degree of confirmation of a constituent which corresponds to wildly overoptimistic inductive behavior, i.e. yields too large values when n is small. Now we can see that this is only natural, for putting IX=O means that we let outselves be guided very strongly by what we find in experience. Hence the results we obtain in this case are not disconcerting at all but rather to be expected. They can be easily corrected now that we have the parameter IX at our disposal. The larger this parameter is, the more suspicious we are of prospective inductive generalizations. The role ofz in (25) is seen more vividly by considering an example. Let us recall the situation sketched in section 1 above. There we had but two primitive predicates, R(x) and B(x) (you may read them "raven" and "black"). If we assume that all the Q-predicates that one can form here have been instantiated in our experience except for the Q-predicate R(x) & ~ B(x), we have k = 2, K = 4, C = 3. The only question which is then left open by our evidence as far generalizations involving Rand B are concerned is whether the missing Q-predicate also fails to be instantiated outside our experience or not, i.e. whether all ravens are black or not. Corresponding to the positive and negative answers to this question we have two constituents C 3 and C4 which are the last two constituents written out in section 1. From (25) it is seen that the degree of confirmation of C 3 is then (26)
A TWO-DIMENSIONAL CONTINUUM OF INDUCTIVE METHODS
127
From (26) it is seen that ex is in this case the number of individuals we must observe before we are willing to make an even bet that all the remaining ravens will also be black. Observations of this kind may make it possible to relate our parameter ex to the actual inductive behavior of different people in different circumstances 4. It was also pointed out in Hintikka [1965b] that one does not obtain very natural results in this case for singular inductive predictions. In fact, it is easily seen that the hypothesis which says that an unobserved individual instantiates a certain Q-predicate which is already instantiated in experience has the asymptotic value lie; see Hintikka [1965b] p. 287. This is not surprising any more in view of the fact that the present special case was obtained by putting A= 00, i.e. by deciding to judge singular hypotheses on as completely a priori grounds as possible. In fact, the value lie is just the natural consequence of this decision. We cannot obtain the value 11K, for this would be incompatible with the degrees of confirmation which our constituents receive in this case. Hence lie is as close to a priori predictions as we can get, and hence just what we might expect as a consequence of putting A= O. 13. All the special cases of our two-dimensional system so far considered are obtained by giving suitable numerical values to ex and to A. The value of the latter may depend on K, however. We obtain further systems by making). dependent on w, A=A(W). Here we consider only the simplest system obtained in this way, which is clearly the one obtainable by putting A= w. This generalizes in a sense the choice A= K which gives rise to Carnap's c*. (In determining the a priori probabilities of constituents, by the same token we then have to put A=A(K)=K.) 4 An illustration of this kind is closely related to what Good calls "the device of imaginary results", see Good [1950] pp. 35, 70, and Good [1965] pp. 19,29. Some of Good's uses of this device (e.g. the last-mentioned one) seem to be based on an implicit atomicity assumption: the possibility that our evidence might serve to establish a strict generalization is disregarded (as may be entirely appropriate for the purposes Good has in mind). One is apt to view imaginary results of the kind Good envisages in a different light if this possibility is taken seriously. Kemeny's criticism of Carnap is an instructive case in point: "If the evidence, e, states that we drew balls from an urn, always replacing them, and the first hundred balls were all small and white; and if our hypothesis, h, predicts that the next hundred balls will also be small and white; then c*(h, e) [i.e. the degree of confirmation of the hypothesis in Catnap's c*-system] is approximately t. The reviewer would nevertheless refuse to bet 7: 1 against it, because he would strongly suspect that all the balls in the urn are small and white" (Kemeny reviewing Carnap [1950] in the Journal of Symbolic Logic vol. 16 (1951) p. 207; the italics are mine). The last clause brings out the reason why Kemeny's intuitions differ from Good's, cf. Good [1965] p. 29.
128
JAAKKO HINTIKKA
In this simple case we obtain from (11) after obvious simplifications
I e) = (a + W -l)!jK£C(K - c) (a +c_~~_n!
p(C w
i~O
(n+w-l)!
i
(n+c+i-l)"
(27)
This is a straightforward generalization of the system presented in Hintikka [1965c], to which it reduces when a=O. The fact that in this "combined system" one is again expected to jump to generalizations rather easily is predictable on the basis of the choice a = 0 on which it is based. This overoptimism is again eliminated by choosing the value of a sufficiently large. In the same way one obtains
p(h
I e) =
n1+1
KiC(K - c)(a + c + i - l)!(n + c)! i~O i (n+c+i)!
- ,------
n + c Ki c i~O
-(K -c) (a+_c +_i i
-- --
l)!(n + c - I)! (n+c+i-l)!
(28)
where h is the hypothesis that an unobserved individual satisfies Cl i l (x). This is again exactly the same as was found in the combined system (in an infinite universe) except for the presence of the parameter a. Our results may be compared with Carnap's results in the same way as in Hintikka [1965c]. In Hintikka [1965c] it was also shown for a=O that this system is obtained (in the limiting case of an infinite universe) by a three-stage procedure: First, equal a priori probabilities are given to the constituents; then the a priori probability of a constituent is divided evenly among the structure-descriptions that make it true; and finally the a priori probability of a structure-description is divided evenly among all the state-descriptions that make it true. In fact, we can obtain the more general system in the same way except that the a priori probability of each constituent C w has to be chosen to be proportional to (a+w-l)!/(w-l)!, i.e. proportional to (7) with A=K. This illustrates the naturalness of the system, and also its close relation to Carnap's old favorite system based on c*. The same relation is illustrated by the fact that when a-HX), our system approaches Carnap's c*-system. In fact, by the same limit argument as we have used earlier, it can be shown that when a-HI), p(hle) approaches (111 + 1)/(11 + K), which is exactly the characteristic function of a Carnapian system based on c*. Other choices of A give rise to other systems, most of which remain to be investigated. It might be especially interesting to see what would happen if A were made dependent on 111, 11z, ... , nco
129
A TWO-DIMENSIONAL CONTINUUM OF INDUCTIVE METHODS
14. By way of summary, we can exhibit the relation of some clear-cut systems which are members of our two-dimensional continuum in the form of the following table:
system
Catnap's A-continuum a system based on Carnap's c+ a system based on Carnap's c* straight rule Hintikka's Jerusalem system [1965b) the same generalized a system based on (20)-(21) Hintikka's "combined system" [l965c) the same generalized
parameter et
).
CD
A
CD
CD
CD
K 0
et
(does not matter) 0
CD
et
w
0 0 a
w w
).
The occurrence of et in the first column or A in the second column means that they are free parameters in the (kind of) system in question.
15. In Hintikka [1965b] it was conjectured that one can make a distinction between two essentially different tasks, namely, between singular inductive inferences and inductive generalizations. What has been found in the present paper bears out to a certain extent this conjecture. We have been able to arrange into a two-dimensional continuum several interesting inductive procedures in such a way that one of the parameters describes the character of one's assumptions about singular inductive predictions and the other describes one's assumptions about inductive generalization. In a sense we have thus separated the two from each other. In Hintikka [1965b] it was likewise suggested that this distinction is relevant to some aspects of the Carnap-Popper controversy in that Popper's claims seem to be best justified when applied to inductive generalizations while most of Carnap's fully developed work deals with singular inductive inference. This suggestion also derives some support from what we have found. Indeed, it was found that Carnap's whole A.-continuum deals with the case IX = 00 in which inductive generalizations are dealt with on a priori grounds only, and is hence informative about singular inductive predictions only.
130
JAAKKO HINTIKKA
In contrast to this, some of our remarks on inductive generalizations can be related to certain ideas Popper [1959] has put forward. One way of seeing this is to ask what sense one is to make of Popper's remarks about the ease or difficulty that there might be about falsifying a hypothesis or theory. What is meant by these remarks is obviously different from what is probably their most literal sense. If a hypothesis is true, it is simply impossible to falsify it. When Popper says that we ought to prefer, ceteris paribus, theories that are easily falsifiable he therefore does not mean theories that are easily falsifiable in our actual world. What he means might perhaps be explained by saying that the preferable theories would be easy to falsify in a completely atomistic world. On the other hand Popper also makes remarks to the effect that the preferable theories have a low absolute logical probability. What we have just seen implies that this absolute logical probability has to be understood as a probability in an atomistic universe. This suggests a way of measuring the absolute logical probability, which is essentially our a priori probability, of different generalizations in a Popperian spirit. This task reduces at once to the task of assigning a priori probabilities to the different constituents. In a Popperian view, these probabilities should be indicative of the probability of the different generalizations in an atomistic universe. Now if this universe is infinite, these a priori probabilities are all zero, except in the case of a logically true generalization. Hence we have to use as a measure some finite atomistic universe - a universe, say, with IX members - if we want to have any a posteriori probabilities that can be applied in large universes. In this way we are led to something very much like our assignment of a priori probabilities to generalizations in terms of (7). I n fact, we are led exactly to (7), I believe, if we decide that the probabilities that underlie our completely atomistic universe are to be Carnapian. Thus that part of our approach which turns on the use of the new parameter IX is related very closely to Popper's ideas of the inverse relation of a priori probability on one hand and on the other hand the acceptability of theories and the ease in falsifying them. It may not be entirely clear what I have meant here and in certain earlier remarks by a completely atomistic universe. An explicit explanation can now be given after we have first formulated a theory by using this heuristic idea in an informal way. A completely atomistic universe may in any view be characterized as one in which general laws hold only by virtue of chance - in which the only regularities, statistical or otherwise, that obtain, pertain to the interrelations of individuals. From the point of view which we have reached this means that an atomistic universe is one in which the best inductive
A TWO-DIMENSIONAL CONTINUUM OF INDUCTIVE METHODS
131
policy is obtained by putting IX = 00. This definition illustrates the possibilities of conceptualization that the use of the new parameter IX offers us. In a way we can perhaps also partially justify certain Popperian ideas in those systems in which IX is not infinite. These systems favor, it has been found, that constituent which has the smallest number of Q-predicates and yet is compatible with the evidence. They favor it in the sense of giving it the highest degree of confirmation when the number n of observed individuals is large enough. It is easily seen that this constituent also has, when IX>], the lowest a priori probability among all those that are compatible with the evidence we have and which is therefore in the Popperian sense the easiest to falsify among all of them. It is true that Popper does not think of this preferability as taking the form of a high a posteriori probability; and I agree that it does so only in the case of constituents, not in the case of other generalizations. But if it should take this form, even in the case of constituents, itis not surprising that it should do so only when IX is finite. 1n fact, the smaller IX is, the faster we switch our allegiance (measured by the degree of confirmation) from initially more probable to initially less probable constituents. Thus a small IX is in a sense an indication of one aspect of that intellectual boldness Sir Karl has persuasively advocated.
References CARNAP, R., 1950, The logical foundations of probability (University of Chicago Press, Chicago; second edition, 1963) CARNAP, R., 1952, The continuum of inductive methods (University of Chicago Press, Chicago) CARNAP, R., The philosopher replies: V. Probability and induction, in: The Philosophy of Rudolf Carnap (The Library of Living Philosophers), ed. P. A. Schilpp (Open Court, La Salle, Illinois) pp. 966-995 CARNAP, R. and W. STEGMULLER, 1959, Induktive Logik und Wahrscheinlichkeit (SpringerVerlag, Wien) GOOD, I. J., 1950, Probability and the weighing of evidence (Hafner, New York) GOOD, 1. J., 1965, The estimation ofprobabilities, An Essay on Modern Bayesian Methods, Research Monograph no. 30 (The MIT Press, Cambridge, Mass.) HINTIKKA, J., 1965a, Distributive normal forms in first-order logic, in: Formal Systems and Recursive Functions, Proc. Eighth Logic Colloquium, Oxford, 1963, eds. J. N. Crossley and M. A. E. Durnmett (North-Holland Pub!. Co., Amsterdam) pp. 47-90 HINTIKKA, J., 1965b, Towards a theory of inductive generalization, in: Proc. 1964 Intern. Congress for Logic, Methodology, and Philosophy of Science, cd. Y. Bar-Hillel (North-Holland Pub!. Co., Amsterdam) pp. 274-288 HINTIKKA, J., 1965c, On a combined system of inductive logic, in: Studia Logico-Mathematica
132
JAAKKO HINTIKKA
et Philosophica in Honorem Rolf Nevanlinna, Acta Philosophica Fennica, vol. 18, pp.21-30 JOHNSON, W. E., 1932, Appendix (edited by R. B. Braithwaite) to Probability: deductive and inductive problems, Mind N. S., vol. 41, pp. 421-423 KEMENY, J. G., 1963, Carnap's theory of probability and induction, The Philosophy of Rudolf Carnap (The Library of Living Philosophers), ed. P. A. Schilpp (Open Court, La Salle, Illinois) pp. 711-738 POPPER, K. R., 1959, The logic of scientific discovery (Hutchinson, London)
ON INDUCTIVE GENERALIZATION IN MONADIC FIRST-ORDER LOGIC WITH IDENTITY RISTO HILPINEN University of Jyvdskylii, Finland
1. Jaakko Hintikka has in a recent article [1965b] sketched a new system of inductive logic. When compared with the system of inductive logic constructed by Carnap [1950] the main advantage of Hintikka's system is that it gives fairly natural degrees of confirmation to inductive generalizations, whereas Carnap's confirmation function c* enables one to deal satisfactorily with singular inductive inference only. According to Carnap's system, general sentences which are not logically true receive non-negligible degrees of confirmation only if our evidence contains a large part of the individuals in the whole universe. In particular, in infinite domains of individuals a system of inductive logic based on c* gives all general sentences which are not logically true a zero probability independently of the amount of evidence. Hintikka's system is in principle applicable to all first-order languages. However, in the paper mentioned above degrees of confirmation of generalizations are actually calculated only in the case in which all our predicates are monadic. In the present paper we shall inquire what kinds of results one obtains by means of the system in question when the relation of identity is used in addition to monadic predicates. 2. Let us assume that our language contains k primitive monadic predicates Pi (i=l, 2, ... , k). By means of these predicates and propositional connectives it is possible to define K = 2k different kinds of individuals. These kinds of individuals are specified by certain complex predicates Ct j (j= 1,2, ... , K) which we shall call attributive constituents. In the present case in which we have in our language only monadic predicates, attributive constituents are simply Carnap's Q-predicates in a new guise". By specifying ofeach attributive constituent Ct, whether it is instantiated or not it is possible to define constituents. Constituents describe all the different kinds of "possible worlds" 1
For Q-predicates, see Carnap [1950] pp. 124--126.
134
RISTO HILPINEN
that can be specified by means of our monadic predicates, quantifiers, and propositional connectives". If we have in our language, in addition to propositional connectives, quantifiers, and k monadic predicates, also the sign of identity, it is possible to specify of each attributive constituent Ct.; not only whether it is instantiated or not, but also how many individuals exemplify Ct j in the universe of which we are speaking. If the maximal number of layers of quantifiers in the sentences of our language is q, it is possible to specify of each attributive constituent whether it is exem pIified by 0, 1, ... , q - I or ~ q individuals. When at most q layers of quantifiers are used, the number of constituents in our language is (q+ l)K. Constituents correspond to different partitions W=(Wq, Wq-l> ... , Wi> Wo) of Ct-predicates, where each subclass Wj , j=O, I, ... , q-I, is the class of those attributive constituents that are exemplified by exactly j different individuals and where the class Wq is the class of those attributive constituents that are exemplified by at least q individuals in our universe. The maximal number of layers of quantifiers in a sentence is also called its depth. We shall first consider a simple case in which q = 2. In this case it is possible to define 3 K different constituents. These constituents correspond to partitions W=(W2 , WI' Wo) of attributive constituents, where (1) W2 is the class of those attributive constituents that are exemplified by at least 2 individuals, (2) WI is the class of those attributive constituents that are exemplified by exactly one individual each, and (3) Wo is the class of such attributive constituents as are not instantiated at all. Let the numbers of Ct-predicates in the subclasses W2 , WI and Wo be respectively w 2 , WI and woo Because W is a partition of K Ct-predicates, \V 2
+ WI + W o = K.
(4)
For simplicity, the constituent corresponding to the partition W will also be called W. If attributive constituents are numbered in a suitable way, the constituent W can be written as follows:
W = (Ex) (Ct., (x) & (Ey)(y =I- x Sc Ct.; (y») & ... & :l Constituents and attributive constituents have been characterized in greater detail in a number of papers by Hintikka. See e.g. Hintikka [1965a].
ON INDUCTIVE GENERALIZATION
(Ex) (Ct;w,(x) & (Ey)(y #- x &Ctiw,(y»))& (EX)(Ct iw2+ 1 (x) &(y) (Ct iw2+ 1 (Y)::::J y = x)) & ... & (Ex) (Ct;w 2+wI (x) &(y)(Ct;w2 +w, (y) ::::J Y = x)) & (x)(Ct il (x) v ... V CtiW2+ W, (x)).
135
(5)
Let us assume that the whole domain of individuals of which we are speaking contains N individuals, and that we have observed n individuals sampled from the universe in question. These n observed individuals constitute our evidence. Our evidence can be represented as a partition of Ctpredicates, viz. as partition C=(C z, C l , Co), where (6) C z is the class of those attributive constituents that are exemplified in the evidence by at least 2 different individuals, (7) C 1 is the class of such Ct-predicates as are exemplified by exactly one individual in the evidence, and (8) Co is the class of those attributive constituents that are not instantiated in our evidence. In the sequel we shall use the expression "C" to refer both to the evidence and to the partition of Ct-predicates corresponding to it. Suppose again that the numbers of attributive constituents in the classes C z , C l and Co are respectively Cz, C l and Co. Because C is a partition of K Ct-predicates, (9) C z + C l + Co = K. An arbitrary constituent compatible with C will be called U. It corresponds to a partition U =(Uz , Vi> V o) of Ct-predicates, where the subclasses V z , VI and Uo are defined in the same way as the subclasses Wz , W1 and Wo of the partition W. A constituent V is compatible with C if and only if the class U z contains all Ct-predicates in the class C z and possibly other attributive constituents as well, and V 1 contains all Ct-predicates that belong to the class C 1 but not to Vz, and possibly other Ct-predicates. In other words, the partitions U have to satisfy the following requirements in order to be compatible with C: (10) Cz n V 1 = 0 C znVo=0
(11)
C l n Uo=0.
(12)
where "0" refers to the empty class. Which constituent W has the highest degree of confirmation with respect to C? This question may also be formulated as follows: How are the par-
136
RISTO HILPINEN
titions Wand C related to each other when the degree of confirmation of W with respect to C assumes its greatest value? To answer this question we shall consider the cross-partition D=Cn W; D=(D 2,2, D1,2' D O,2, D 2,l' D1. 1 ' Do, I. D 2 , 0 , Dl,o, Do,o), where C2
n
W2 = D 2,2
(13)
C1
n
W2 = D 1 , 2
(14)
Co
n
W 2 = D o, 2 , etc.
(15)
Co
n
Wo
=
(16)
Do,o·
For instance, in the evidence C each of the Ct-predicates in D l , 2' is exemplified by only one individual, but according to Wthere nevertheless are in the whole domain of individuals other individuals that exemplify the attributive constituents in question. It is obvious that the degree of confirmation of W with respect to C is different from zero only if W is compatible with C. We are here not interested in constituents incompatible with our evidence. W is compatible with C if and only if it is one of the constituents U, i.e. if D 2, 1
= 0,
(17)
D 2 ,0
= 0,
(18)
Dl,o
= 0.
(19)
Suppose now that the numbers of Ct-predicates in the classes D 2,2, ... , Do,o are respectively d 2,2, ... , do,o. Because of (17)-(19), and because D is a partition of K Ct-predicates, d 2 ,2
+ d l,2 + dl,1 + d O, 2 + d O, 1 + do,o =
(13)-(19) obviously imply
+ d l , 2 + d o, 2
K.
(20) (21)
11'2
= d 2 ,2
11'1
= dl,l + dO, 1
(22)
11'0
= do,o
(23)
C2
= d2 ,2
(24)
C1
= dl,l + d 1 , 2
(25)
Co =
do,o
+ do,1 + d O,2'
(26)
ON INDUCTIVE GENERALIZATION
137
Given the evidence C, the respective numbers of unobserved individuals that exemplify each Ct-predicate in each subclass of D are according to the constituent W subject to the following restrictions: D 2 , 2 : no restrictions
(27)
D 1 , 2 : at least 1
(28)
D1,1: 0
(29)
D O,2 : at least 2
(30)
DO,l: 1
(31)
Do,o: O.
(32)
The restrictions (28)-(32) are obviously implied by the definitions of the classes D 1 • 2 , ••• , Do,o. 3. In Hintikka's system of inductive logic, a priori probabilities are first distributed among constituents. The probability of each constituent is then divided evenly among the state-descriptions that make the constituent in question true. In the simplest case, we may assume that all constituents have received an equal a priori probability. In the sequel we shall mainly restrict our considerations to this simple case. Because the number of different constituents in our language with q=2 is 3\ each constituent has in this case 1j3 K for its a priori probability. A posteriori probabilities, or degrees of confirmation, are computed for the constituents according to Bayes' formula
P(W)P(CI W) P(W I C) = fp(U)p(CI u)'
(33)
u
where the sum in the denominator is taken over all constituents U compatible with C. Because we assumed that all constituents have an equal a priori probability, (33) reduces to
P(W I C) =
{;1~~Z)'
(34)
u
If the number of those state-descriptions which make the constituent W true, given the evidence C, is expressed by m(W), and the corresponding number in the absence of any evidence is expressed by M (W), and if the
138
RISTO HILPINEN
corresponding numbers for a constituent Dare m( D) and M (D), we have
and
P(CI W) = m (W) M(W)
(35)
P(CI D) = m (V). M(V)
(36)
According to (34), (35) and (36), the degree of confirmation of W with respect to C is thus
m(W)jM(W)
r
(37)
= Lm(U)/M(U)'
peW I C)
u
According to the elementary principles of combinatorial analysis, we obtain (N - n)' meW) = ,-_ . _.-- [W/- n - do. , + ()(I] (38) (N - n - dO,l)l and (39) and ()(2 in formulas (38) and (39) represent correction terms that are necessary because of the restrictions (28) and (30) and the definition of the class Wz . The exact values of these correction terms are
()(l
()(I
=
d
l,2+f ,2-
1
1=1
X
()(z
=
(-
tY
±
(,d 1,2,)
k=O
l-k
( N - n - d 0,1' )'
(N
--;]----:-)1
- n - o.i
r
t)'
(d
(do z) k
1I L (- IY (w.2)W2-L- (WZ. I
j=O
1 (_
j
i)
1)j
j=O
+ d I,Z + d O,Z -
2,Z
W2-)
1=1
i-
o 2
d .
(N - w)'
'
I -
j
(d
k)
O' 2 . -
j
X
.)N-n-do t- j
1_', ,(Wz - i (N-wI-I).
,
(40)
irr:'. (41)
They are correction terms in the sense that they do not affect the limit of (35) when N-HXJ, as one can check. The case in which we are speaking of an infinite universe is the easiest to deal with, and it also seems the most interesting from the point of view of inductive logic. Therefore we shall assume in the sequel that we are considering an infinite (or very large) universe. When N grows without limit,
ON INDUCTIVE GENERALIZATION
139
m ( W)/M (W) approaches the value m (W) ( 1 )(W~I'I) M(W) = N d 1. ;
wf'
(42)
The terms in the sum in the denominator of (37) correspond to all the constituents U compatible with the evidence C. For each constituent U we may again define the cross-partition E= C n U, where the subclasses E 2,2' ... , E o.o are defined in the same way as the subclasses D 2,2' ... , Do,o in the case of the cross-partition D. If the numbers of attributive constituents in the classes E 2,2' ... , Eo,o are respectively e 2.2, ... , eo,o, we obtain in analogy to (21)-(26) U2 = e 2,2 + e 1,2 + eO,2
(43)
U1
= e 1 , 1 + eO, l
(44)
Uo
=
eo,o
(45)
C2
= e2 ,2
(46)
C1 =
e1 , 1
+ el,2
(47)
Co =
eo,o
+ eO,l + eO,2'
(48)
The unobserved individuals have to satisfy conditions corresponding to (27)-(32) also for all cross-partitions E. Accordingly, we obtain corresponding to (42)
m (U) ( 1 )(U~I'I) M(U) = Ne", -~; .
(49)
(37), (42) and (49) together entail
P(W I C) = i.e.
/ 1 )(Wg1'I)j\( 1 )(U;I.I) (Ndl" .~f L Ne", . u; ,
(50)
u
II(
e" N d l")(w)n ~ (u ,) N 1.1
P (W I C) = 1 - e-U
U2
_2. W~I,l
(51)
Because the sum in the denominator of (51) contains terms corresponding to all constituents U compatible with C, there occurs also a term corresponding to such a constituent U in which e 1 1 =0. When N grows without limit, p(WIC) approaches therefore a value different from zero only if (52)
140
RISTO HILPINEN
When (52) holds, all terms with el,1 >0 in the denominator of (51) approach zero when N grows without limit. (51) thus becomes (53) According to (21), (24), (25), (43), (46), (47), and because el,1 =0 in all the terms of the sum in the denominator of (53), (53) can be written as follows:
P(W I C) = 1
/I( u
C
I
+ C2 _ +_ d 0;2 )n
c 1 + c 2+eO,2
(54)
P (WIC) assumes its largest value when
d O , 2 = 0,
(54) thus becomes P(W[C)=
1/\( L u
C
c1
(55) 1
+c 2
~)n
+ c 2 + eO,2
(56)
The terms in the sum in the denominator of (56) correspond to all the constituents U with e 1,1 = 0, e O,2;:;: 0 and eO,l ;:;: O. (60) may thus be expressed as follows:
P(WI C) =
1/(2CO+ i~1 (C;)2CO-iCl~-:2C: J)-
(57)
When n grows without limit, (57) approaches P(WIC)= I/Yo.
(58)
The principal results of our considerations so far are seen from formulas (52), (55) and (58). According to (52), C gives the highest degree of confirmation to a constituent W in which the class DI,1 is empty, i.e, in which all the attributive constituents in C 1 belong to W2 • According to this constituent, we should assume that although there are in our evidence Ct-predicates exemplified by one individual only, there are in our universe also other individuals that exemplify each of these attributive constituents. A case in which a certain attributive constituent is exemplified by exactly one individual, or by certain fixed finite number of individuals could be called a singularity. According to (52), it is not reasonable to assume that the singularities in our evidence are real singularities in the whole universe. Formula (55) says that it is not advisable to expect on the basis of the evidence C that there exist in our universe an arbitrary number ;:;: q of
141
ON INDUCTIVE GENERALIZATION
individuals that exemplify Ct-predicates not instantiated in our evidence. This result is similar to that obtained in Hintikka's (Hintikka [1965b] pp. 284-285) system of inductive logic when the sign of identity is not used. However, the situation is different with respect to possible singularities. According to (57) and (58), the degree of confirmation of W is independent of the value of do,t. The degree of confirmation of a constituent which says that some of the Ct-predicates not instantiated in C are exemplified by exactly one unobserved individual is equal to the probability of a constituent which denies this. No amount of evidence can therefore distinguish all constituents of our language as far as their a posteriori probabilities are concerned. Given any value of n, no matter how great, there always remain 2c O constituents which have an equal degree of confirmation with respect to C. The probabilities of all these constituents approach the values P (WjC) = 1/2 c O when n grows without limit. The constituents in question are those with W2=C2+Cl and O~do,t~co' We shall discuss these results in greater detail later. 4. In the preceding section we have considered the probabilities of such constituents as can be defined by means of at most two layers of quantifiers, i.e. constituents for which q = 2. Our results can be easily extended to the general case in which the maximal number of layers of quantifiers is any finite number q. In this case our constituents W correspond to partitions W=(Wq , Wq - t, ... , Wt, Wo) of attributive constituents and our evidence C can be described as a partition C=(Cq , C q - t , ... , C t , Co), Each subclass Wj' j = 0, 1, ... , q -1 of the partition W is the class of such attributive constituents as are exemplified by exactly jindividuals in our whole domain of individuals, and the class Wq is the class of such Ct-predicates as are exemplified by at least q individuals. Correspondingly, each subclass Cj,j= 1,2, ... , q-l, is the class of those Ct-predicates which are exemplified by exactly j observed individuals, and the class Cq is the class of such attributive constituents as are exemplified by at least q observed individuals. The numbers of Ct-predicates in the subclasses Wq , ... , Wo and Cq , ... , Co are respectively wq , ..• , Wo and cq , •.. , Co' We define again the cross-partition D = C n W, where each subclass Di,j (i=0, 1, ... , q;j=O, 1, ... , q) is defined by
o., = c, n W
(59)
j ,
and where the number of Ct-predicates in each subclass
Di,j
is di,j'
142
RISTO HILPINEN
An arbitrary constituent compatible with C is again called U, and it corresponds to a partition U =(Uq , Uq - 1 , .•. , U 1 , Uo) with respective numbers uq , uq - 1 , _.. , u 1 , U o of Ct-predicates in the subclasses. We define the crosspartition E = C n U in the same way as we defined D above. To be compatible with C the partitions U have to satisfy the condition All subclasses
Ei,j
= C, n
U, with j < i are empty.
(60)
(60) is a generalization of the requirements (10)-(12). Now P (WIC»O only if W is one of the constituents U, i.e. All subclasses
with j < i are empty.
Di,j
(61)
(59) and (61) together imply the generalizations of the formulas (21)-(26): Wq
q
I di,q i=O
=
q-l
wq -
I
1
=
W1
=
di,q-l
1
L: el
i,l
i=O
= elo,o
Wo
and
i=O
I
=
C1
Co
j=q-l
=
i.e. in short
q
(63.2) (63.q)
dO,j'
(63.q+ 1)
L: s.,
(64)
I
j=O j
i=O
I,]
q
j
+ 1)
,j
q
w-= j
c, =
d q- 1,j
I ell' j=l
=
(62.q)
(63.1)
q
Cq-l
(62.2)
(62.q
= dq •q
Cq
(62.1)
I=i d i •j •
(65)
In the same way we obtain as generalizations of (43)-(48) j
u j. = '\' L. e,',j. i=O
(66)
143
ON INDUCTIVE GENERALIZATION
and q
c,
I
=
j=i
(67)
ei,j'
Given the evidence C, the remaining N - n individuals may exemplify Ct-predicates in different subclasses of D according to following restrictions: (68) Every attributive constituent in each class Di,q with i=O, 1, ... , q-l, q, is exemplified by at least q - i individuals. (69) Every attributive constituent in each class Di,i with i=O, 1, ... , q-1, is exemplified by none of the unobserved individuals, and (70) Every attributive constituent in each class Di,j with i <j (i =0,1, ... , q- 2; j= 1,2, ... , q-1) is exemplified by exactly j-i unobserved individuals. The conditions (27)-(32) are a special case of the general conditions (68)-(70). Corresponding restrictions hold of course also for the classes Ei,j" Now we are ready to calculate the a posteriori probabilities of constituents. We shall again consider only the case in which our universe is infinite or very large. In the latter case our formulas give approximate values only, because our calculations concern the case in which N grows without limit. Now we have instead of (38) and (39)
[ ( l m(W)=
q-
2
q-
3
0)
])
(N - n) ... N - n - i~O d i•i+ l + 2 i~O d i,i+2 + ... + (q -l)i~/i,i+q-1 + 1 { ----. - - .q-3 - - - - -q-4 - ...------- . - - - - - - --------~x O di,i+231,1: di,i+3 ••• (q _ 2)1.1: d i,i+ _ l),d ' -1: 2 01-0 ot-O 01-0 • wq
X
I
Q'2('q
q-2 q-3 [N-n-(,~ di,i+l+2,~ di.i+2+, 1=0
1=0
,Q "
t
.. +(q-2j,~ di,i+q-2+(q-ljdO,q-tll 1=0
+ I.Jl..l N
(71)
and
M(W)=
~N(N -1) ... [N - - -- -
(
-- - -
2
(to a., + 2
x
1
Wq
2
[N-(,~ di.l+2~ 1=0
s., + ...
+(q
-l)~t~
q-2
3
i,231.1: d',3 21.1: 01-0 d 01-0
,
itO
di,q'l)
+ 1] (
---------------,---,- ~ X
1=0
•••
i,q-2(q (q _ 2)',1: , , - 0d q-2
d,,2+ .. ·+(q-2j,~
t=O
q-l
_ q-l
d',q-2+(q-l)~
1=0
1)',1: di,q-I ol-Q
d"q·tll
+
N
1hZ
(72)
where !Xl and !X2 again represent correction terms which do not affect the limit of m(W)/M (W) when N-->oo. When N grows without limit, we obtain
144
RISTO HILPINEN
according to (71) and (72)
(W) M(W)
m
q~l
q-l
q-l
q~l
. ~ d"i+2~ d2,i+,··+(q-l)dq-"q-,)'W-n.W (.~ d"i+2.~ d 2,i+ .. ·+(q-l)dq-1,q-,) N - ( 1""1 l='2 q q 1=1 1=2 2
q-3
q-4
3
'2fl.~ di,2-.~ d,,'+2)31l.~ d,,3-.~ d,,'+3) .1=0
1-0
.,-0
1-0
•••
q-l
(q _
1)1(.~ di.q-,-do,q) • 1-0
(73)
which is a generalization of (42). The corresponding generalization for (49) is obtained from (73) by substituting e for d and u for w everywhere in the formula. According to (37), (73), and the corresponding formula for m(U)jM (U) the degree of confirmation of W with respect to C is P(W I C)
= q-lq-l
N
L
-( ~ j
~
q-I
[
N
dj,')
i=j
)=1
-( ~ j
(
'w-n.w
q-Iq-l
~
j
j=J
~
dj,') q-I
i=j
•
i
q-' ~
---
ej,i)
j=li=j
.-_._---.-1
q-'
~
---------_.
( ~ j ~ ej,') q-l 'Uq~n'Uqj=li=j
l .~ e",+2~ 1 .~
(~ei,j-
----
q-j-'
.TIj!i=:O
(74) can be written also as follows:
u':' q-'
j
dl,,+2.~
,=2 .-1
~
lil
d2,,+ .. ,+(q-l)dq-l,q-'
q-t
q_1
Nt=-1
1=2
e2,,+· .. +(q-l)eq-1,q-1
] ei,i+j)
(74)
i=O
w; v:
= f, i..J - - - - - - - - n U
di,i+j)
i=D
j=2
j=2
1 -
q-j-l
(~d',j-
jO' j;:o;O
q q '
U
P(W I C)
TI
x
The terms in the sum in (75) again correspond to all constituents U compatible with C. Therefore there is obviously also a term corresponding to a partition U in which q-l
q-l
i= 1
i==2
L e1,i + 2 L e
2 ,i
+ ... + (q -1)eq -
l,q- l
= O.
(76)
145
ON INDUCTIVE GENERALIZATION
Now P (WIC) approaches a value different from zero when N grows without limit only if q-I
I
i= 1
dl,i
q-I
+2I
i=2
d 2,i
+ ... + (q -1)dq_ l,q_1 = 0
(77)
i.e., because the numbers di,j are always non-negative integers, dl,1
= 0;
d l ,2
= 0;
d2 ,2
= 0;
d2 ,3
= 0;
d1,q-1 =
... ,
d 2 ,q- l
(78.1)
0,
=0
(78.2) (78.q - 1)
dq-1,q-l = O. (78.1)-(78.q-1) can be expressed in a concise form by
di,j
= 0 for i = 1,2, ... ,q
= 1,2, ... ,q - 1 where
- 1 andj
i ~j.
(79)
(79) is obviously the generalization of the condition (52). (52) is obtained from (79) by substituting the number 2 for q. Because of (79), (75) reduces to
P(WjC)
=
1j'(C + ~ + u
I
C2
CI
C2
(80) assumes its greatest value when dO,q
+ + cq + dO,q)n + + cq + eO,q
= O.
(80)
(81)
(55) is obviously a special case of (81). Because of (81), we obtain
(82)
It is easy to see from (82) that the value of P (WI C) is independent of all classes DO,j;j
wq and
q
I
c,
(83)
dO,i ~ 0
(84)
=
i= I
q-l
I
i=O
have an equal degree of confirmation with respect to C.
146
RISTO HILPINEN
5. Our principal results are now seen from formulas (52), (55), (58), (79), (81), (83) and (84). They can be interpreted as follows: (i) It is not reasonable to assume that such singularities as are found in our evidence represent real singularities of our universe. (ii) All the constituents according to which in our universe there are unobserved singularities have an equal degree of confirmation with respect to any kind of evidence. Moreover, these constituents are exactly as probable as that constituent which denies the existence of any unobserved singularities. Our result (i) seems quite natural and reasonable from the point of view of intuitive judgment as soon as we pay attention to the fact that we have been dealing with an infinite universe, i.e. we have considered the case in which N---'> co, It is obvious that even if there were an attributive constituent exemplified by exactly one individual in our infinite universe, our chances to find the individual in question are minimal. This is clearly seen also from the formula (42): If we assume that the singularities found in our evidence represent real singularities of our universe, P (C1 W)---'>O when N---'> co. The probability that we should have in our evidence such a Ct-predicate instantiated as is exemplified by only one individual in the whole universe approaches zero when the total number of individuals in our universe grows without limit. What we just said holds of course also for any finite number of individuals. as is seen from (79). On the other hand, it is of course possible that our finite evidence contains singularities although in the universe there are no such singularities. Our second main result (ii) seems somewhat problematic. According to (ii), all constituents that say that in our universe there are unobserved singularities are equally probable. In addition, they are as probable as such a constituent as denies the existence of singularities. It is clear that in the case discussed above a reasonable man would choose such a generalization as says that the singularities in question do not exist. This generalization would be the simplest one, and in choosing it one would not postulate the existence of any unobserved kinds of individuals. Even ifit were not true in the sense that in our universe there in fact are singularities denied by the generalization in question, the risk that our future experiences would contradict it is minimal. However, such a choice cannot be defended in terms of our inductive logic. What is the reason for this discrepancy between our intuitions and our formal results? One could say that the generalizations we have considered are somehow unnatural. It seems rather strange to specify in one's generalizations numbers of individuals up to a given point, as we have done. In fact, these numerical assumptions are absolutely unverifiable when an infinite universe
ON INDUCTIVE GENERALIZATION
147
is concerned, and, as we have seen, they are not confirmable either. It is not possible to explain anything by postulating such singularities. Moreover, even if it were possible to express in our language the existence of singularities, our result (ii) suggests that we perhaps should not distribute a priori probabilities among the constituents according to the method used above. According to this method, each constituent with a fixed depth q received an equal a priori probability. Some of these constituents differed from each other only because of unconfirmable numerical assumptions, and the equality of their a priori probabilities was reflected again in the equality of their a posteriori probabilities. It would perhaps be advisable to give a relatively high a priori probability to such constituents as deny the existence of singularities, and lower a priori probability to other constituents. In Section 8 we shall inquire whether this can be done in some simple and natural way. To some extent, both the results (i) and (ii) may be taken to reflect the limitations of the language systems we are here studying, rather than the limitations of the basic ideas of our inductive logic. It does not seem reasonable to expect that the exact numbers of the different kinds of individuals we can distinguish from each other can in any case be accounted for in terms of purely qualitative concepts, i.e. monadic predicates. Our results may perhaps be taken to justify this pessimism. 6. If N is a finite number, i.e. if we are speaking of a finite domain of individuals, we obtain results different from those obtained in the previous case. In particular, if n is not negligible in comparison to N, (i) and (ii) do not hold any more. Our formulas in the sequel are rough approximations and presuppose that both Nand n are large in comparison with K. For the sake of simplicity, we shall restrict our remarks to the case in which q = 2. Formulas (38) and (39) hold for the finite case as well as for the infinite one. If n is not negligible in comparison with N, and both are large in comparison with K, P( CI W) is instead of (47) approximately
m(W) ~ (N - n)d
M(W)
D
N
•J
(_1)(W~I~) . w;
Because the value of the denominator
Ndl.l
I
(85)
P( CI U) of the formula for
P (WIC) is independent of the choice of W, we shall in the sequel inquire which constituent W has the highest degree of confirmation with respect to C
by considering formula (85) only. When (85) assumes its greatest value,
P (WIC) ass~mes its greatest value, too.
148
RISTO H1LPINEN
It is easy to see from (85) that the value of P (C1 W) with dO,l
(86)
=0
is larger than any of the values which it assumes when do,1 >0. When (86) holds, (85) reduces to
1P(CI W) = ( - d N 1.1
,) )(Wd'.
_2_. W~
(87)
Because of (21), (24) and (25), (87) can be written as follows:
(N~1'[)(~:2++C~1~~~~~1,~++d~:~;:'1
(88)
Because we assumed that n > K and thus n > d 1,1' (88) assumes its greatest value when (89) d O,2 = 0 (88) thus becomes 1 P(CI W) = N d , . , (c2 + C 1 - dl,l)dt.,-n. (90) How should we now choose dl,l so as to make (90) and therefore also P(WIC) as large as possible? The required choice of d1,1 depends on how the numbers n, N, C 1 and C 2 are related to each other. Let us consider two different possibilities. We may choose (91) and obtain (92) or alternatively (93) whence (94) By comparing formulas (92) and (94) one can see that if n>log~ cz
+Ct
C2
(
-
N
)C I,
(95)
a constituent with (93) has a higher probability than one with (91). Con-
ON INDUCTIVE GENERALIZATION
versely, if n
CZ+Ct
(
C2
N
)CI
149
(96)
a constituent W with d l l =0 has a higer degree of confirmation with respect to C than one with (93) and is thus preferable to it. If n=log~ ( cz+c,
c
2
N
)Cl ,
(97)
the two constituents in question have an equal a posteriori probability. As we mentioned earlier, our argument is not quite strict, because we have used formulas that are only rough approximations. Our results (95)-(97) cannot therefore be taken to give exact quantitative conditions for the preferability of our alternative constituents to each other. However, they can be interpreted qualitatively as follows: In our evidence C contains a relatively large part of the individuals in the whole universe, it is reasonable to assume that the singularities we have found in our evidence are genuine singularities of our universe. On the other hand, if the number n of observed individuals is small in comparison with N, it is not advisable to draw such a conclusion. In the latter case we ought to expect that in the whole domain of individuals there are unobserved individuals that exemplify attributive constituents in C i This result can be extended to apply to any number of layers of quantifiers. It is different from the result (i) obtained in the infinite case. When our universe is finite, (i) must be replaced by (i') It is not reasonable to expect that the singularities of our evidence are genuine singularities of our universe unless we have examined a large part of the individuals in the whole universe. (86) shows that our result (ii) does not hold if the number of individuals in our evidence C is not negligible in comparison with the total number of individuals in the whole domain of individuals. If N is a finite number, it is not reasonable to assume that there exist unobserved singularities in our universe, and the unreasonableness of such assumptions is brought out by our inductive logic. The smaller the ratio n] N, the smaller is the difference P (W:dO , 1 =OIC)- P (W:dO , 1 >OIC). When n/N-+O, we obtain the result (ii) as a limit case. 7. One can notice certain analogies between the results obtained in the present paper and those obtained by using Carnap's confirmation function c *.
150
RISTO HILPINEN
In fact, our system of inductive logic becomes more and more like Carnap's
system when q grows. As we mentioned earlier, constituents correspond to different partitions W = (Wq , Wq _ I ' ... , WI' Wo) of the class of all attributive constituents. Each constituent specifies a statistical distribution over the attributive constituents Ct.; Ct 2 , .•. , Ct K • Because of the restriction imposed on the number oflayers of quantifiers, the distributions are not defined completely: the class Wq contains all Ct-predicates exemplified by at least q individuals. If we do not restrict the numbers of layers of quantifiers in the way mentioned above, but instead let our formulas contain any number of layers of quantifiers, it is possible to define all the possible statistical distributions in our universe. If N is the total number of individuals in our universe, we need also N layers of quantifiers to specify completely all the possible statistical distributions. In this case our constituents become Carnap's structure-descriptions 3. (This procedure is of course unavailable in an infinite universe.) When we are speaking of an infinite universe, our result (ii) is analogous to that obtained in Carnap's system. If we let the maximal number of layers of quantifiers grow, the number of such constituents as we are not able to distinguish from each other in terms of their respective degrees of confirmation becomes larger, too. When the number q grows, the highest a posteriori probability given by e to any constituent W becomes smaller. By letting q become large enough the probability of any constituent with respect to any evidence whatsoever can be pushed arbitrarily close to zero. Carnap's system of inductive logic gives somewhat unsatisfactory and strange results in the case of generalizations, especially in an infinite universe, and so does Hintikka's theory, if applied in the particular way in which we have applied it so far (and which need not be the only way of applying it). This similarity is not accidental, but is due to the connection which there obtains between Carnap's system and the situations studied above. In a sense, our results bring out the underlying reasons for the failure of Carnap's system to deal with inductive generalizations. This perhaps is the greatest interest of applying Hintikka's ideas to the case at hand in the way we have done so far. We shall see, however, that if we look away from Carnap's system; there are somewhat more natural ways of dealing with inductive generalizations in a monadic first-order logic with identity. 8. In the preceding considerations we have assumed that as soon as the :l
For structure-descriptions, see Carnap [1950] pp. 114-117.
ON INDUCTIVE GENERALIZATION
151
maximal number q of layers of quantifiers is fixed, a priori probabilities are distributed evenly among all constituents that can be specified by using at most q layers of quantifiers. In Section 5 this method of assigning a priori probabilities to constituents was found to lead to result (ii) which does not seem to be in accordance with our inductive common sense. From the point of view of intuitive judgment it is clearly reasonable to prefer a constituent which denies the existence of unobserved singularities to any alternative constituent according to which such singularities do exist. However, according to result (ii), this preference cannot be defended in terms of our inductive logic. Can this shortcoming of our system be removed by assigning a priori probabilities to the constituents of depth «> 2 unevenly in some suitable and natural way? As we suggested earlier, it is perhaps advisable to assign a relatively high a priori probability to a constituent which denies the existence of singularities in our universe, and attach correspondingly lower probabilities to other constituents. This seems very natural indeed, because we are more willing to accept a constituent of the former kind than other constituents. Is there some simple way of doing this? Constituents of different depth are related to each other in certain simple ways which suggest a natural solution to this problem. Let us consider first constituents with q = 1. Constituents of this kind can be expressed without using the sign of identity. Each constituent of depth q= I is equivalent to a disjunction of constituents with q = 2. Constituents which can be specified by using at most two layers of quantifiers are equivalent to disjunctions of constituents of depth q = 3, and so on. In general, a constituent of depth q is always equivalent to a disjunction of constituents of depth q+ r (r = 1, 2, ... ), which are called by Hintikka ([l965a] p. 56) constituents subordinate to the first-mentioned constituent. An especially natural method of assigning a priori probabilities to constituents is obviously as follows: We distribute first a priori probabilities evenly among all constituents with q = 1. The probability of each constituent with q = 1 is then distributed evenly among all such constituents with q = 2 as are subordinate to it. This procedure can be continued and thus extended to constituents of any depth q.4 All the constituents with q = 2 will not have an equal a priori probability because the number of constituents subordinate to a constituent of depth q= I is different for different constituents of this depth. 4
This method, as well as the methods used earlier, has been suggested by Hintikka
[1965bl p. 282.
152
RISTO HILPINEN
It is easy to see that a constituent of depth q=2 has the larger a priori
probability the larger is the number of such attributive constituents as are not instantiated according to the constituent in question. When a priori probabilities are distributed among the constituents in the way described above the probability of a constituent with a given depth is independent of the maximal number oflayers of quantifiers that the sentences of our language are allowed to contain. This is of course a very natural feature of our new a priori probabilities. How do the a posteriori probabilities of constituents behave if a priori probabilities are distributed according to this new method? We shall restrict our remarks to the case in which q=2. The probability of each constituent of this depth is again distributed evenly among all state-descriptions which make the constituent in question true. It is assumed again that the universe we are dealing with is infinite or very large. Our evidence is supposed to be described in the same way as earlier. The a priori probability of each constituent with q= 1 is 1/2K • A constituent with q=2, according to which W o specified kinds of individuals are empty, has now obviously for its a priori probability
1 P(W) = 2~Kc-.-2KCC---W~o '
(98)
r.e.
(99)
According to Bayes' formula, the degree of confirmation of W with respect to C is thus (100)
where U represents an arbitrary constituent compatible with C. The values of P (C I W) and P (C IU) are given by formulas (42) and (49). Which constituent W has the highest degree of confirmation with respect to C? Because the denominator of (100) is identical for all constituents, we can answer this question by asking when the numerator of (100) assumes its greatest value. When N grows without limit, the numerator of (100) becomes according to (42) 2 W O W~I.I (101) Nd",' w~ .
ON INDUCTIVE GENERALIZATION
153
When N-+Cf), (101) approaches a value different from zero only if d!,!
= O.
(102)
When (102) holds, (101) becomes (103) (103) assumes obviously its greatest value when W o is as great as possible. In constituents compatible with C, 0 ~ W o ~ Co' The evidence C thus gives the highest degree of confirmation to a constituent W in which Wo
= Co
(104)
and therefore also (because of (23) and (26») and
««, = 0
(105)
o.
(106)
d o,2
=
In other words, C gives the highest degree of confirmation to a constituent W according to which (iii) Singularities of our evidence do not represent genuine singularities of the universe (formula (102», and (iv) In the whole universe there are only such kinds of individuals as are already instantiated in experience; i.e. unobserved singularities do not exist (formulas (105) and (106»). The result (i) obtained in Section 5 thus remains valid; it is identical with (iii). On the other hand, (ii) no longer holds: C gives the highest degree of confirmation to such a constituent as denies the existence of unobserved singularities. Accordingly, we should prefer this generalization; and our choice can now be defended in terms of inductive probabilities. Obviously the difference between (iv) and (it) is due to a different assignment of a priori probabilities. The method used in Sections 3-5 represents a priori indifference as far as the choice of a constituent with a fixed maximal depth q was concerned. This a priori indifference implies a posteriori indifference as regards constituents containing unverifiable numerical assumptions. On the other hand, according to the present method we prefer a priori a constituent which denies the existence of singularities, and this preference is reflected again in a posteriori probabilities. Both methods are, however, based on an even distribution of a priori probabilities to constituents. According to the former
154
RISTO HlLPINEN
method, a priori probabilities are distributed first evenly among the deepest constituents that can be specified without using more than q layers of quantifiers. According to the latter method, a priori probabilities are distributed first evenly among such constituents as contain only one layer of quantifiers. When the latter method is used, it is possible to consider constituents of different depths at one and the same time, which seems to be in accordance with our inductive habits. It might be suggested that the defects of Carnap's confirmation function c* are due to the fact that it is in effect based on the former and not on the latter method 5. References CARNAP, R., 1950, The logical foundations of probability (University of Chicago Press, Chicago; second ed., 1963) HINTIKKA, J., 1965a, Distributive normal forms in first-order logic, Formal Systems and Recursive Functions, Proc. Eighth Logic Colloquium, Oxford, 1963, eds. J. N. Crossley and M. A. E. Dummett (North-Holland Pub!. Co., Amsterdam) pp. 47-90 HINTIKKA, J., 1965b, Towards a theory of inductive generalization. Proc. 1964 Intern. Congress for Logic, Methodology, and Philosophy of Science, ed. Y. Bar-Hillel (North-Holland Publ. Co., Amsterdam) pp. 274-288
5 This study has been facilitated by a grant by the Finnish State Council of Humanities (Valtion humanistinen toimikuntai to a team led by Prof. Jaakko Hintikka.
INDUCTIVE GENERALIZATION IN AN ORDERED UNIVERSE RAIMO TUOMELA University oj Helsinki, Helsinki, Finland
1. Most of the existing work in inductive logic is concerned with monadic first-order logic, while few studies concerning polyadic cases have been made. In this paper an attempt is made to extend Hintikka's system of inductive logic(see Hintikka [1965aDfrom monadic first-order logic to a special polyadic case, viz. to certain ordered universes. We are mainly interested in universal generalizations describing the structure of these universes. For instance, "every third individual in the ordering has the property P" is a typical and frequently cited example of such a generalization. Only some brief comments will be made on singular sentences at the end of the paper. One reason for beginning the construction of an inductive logic from generalizations is that greater difficulties due to the logical interdependencies of individuals may be expected to arise in connection with singular sentences. Carnap discusses the problem of ordered universes briefly (Carnap [1950], § 15, B). He indicates that a solution may be looked for either by defining the ordering in the usual first-order logic or by using a richer language capable of expressing elementary number theory. Let us first consider the latter approach. Carnap suggests that two ways to a solution may be tried here. In the first we simply ignore the order and transfer the old definitions of degree of confirmation to the richer language. But this leads to results that may seem quite counter-intuitive: (1) As in unordered universes, the degree of confirmation (d.c.) of a universal generalization approaches zero as the number of individuals in the universe increases. It receives non-negligible values only when a relatively large part of the universe has been examined. (2) Carnap claims that in science we do not strive for generalizations that are highly confirmed themselves, but for those that have a high degree of instance confirmation. This kind of degree of confirmation is just the d.c. of the hypothesis that the next unexamined individual satisfies the generalization. But it does not take into account the order in which the individuals occur in the sequence. For instance, if we have examined a sequence of 100 individuals
156
RAIMO TUOMELA
such that every third individual has the property P and that the last two have the property ~P, the d.c. of the hypothesis that the Wist individual has the property P ought to be very high. However, in Carnap's system it is approximately t. Thus the first kind of solution in the language of elementary arithmetic seems unsatisfactory. In Carnap's second proposal for a solution the order of individuals is taken into account in defining the degree of confirmation of the sentences of the language. This can be done by using positional predicates (e.g. P(x;}=i'th individual has the property p). However, Carnap's critics (Putnam, Achinstein) have shown that there are serious difficulties also in this kind of approach. Their argumentation proceeds roughly as follows. First they adopt some conditions of adequacy which they try to make intuitively plausible. Then they show that no computable measure function defined in this kind of language can fulfil these conditions simultaneously. So they conclude that they have shown it impossible to construct an inductive logic for this kind of language. Carnap's answers to his critics amount to little more than to saying that the only thing they have proved is that their own conditions of adequacy are logically incompatible, and nothing else. We will not enter the details of this discussion here because in our approach we use first-order logic, which Carnap did not consider promising for the solution of the problem in question. 2. In this paper a first-order language system similar to Carnap's L is used. Only a slight modification concerning quantifiers will be made. The order among the individuals is defined axiomatically by using a dyadic predicate "immediate successor". To each individual belongs one property from a family of (complex) properties. The "strong" generalizations describing the structure of the universe specify what kinds of individuals there exist. The kind of an individual is determined by the properties belonging to it and to its predecessors and successors in the sequence. How shall the descriptions of possible kinds of worlds be formed? Carnap does this by means of his structure descriptions. The main drawback in Carnap's procedure seems to be its dependence on the domain of individuals in question. It is not very natural to start one's inductive logic from the assumption that one is always familiar with the whole of one's world, for the logic is largely designed to reconstruct some of the procedures we use in coming to know it. Hence it is not surprising that Carnap's theory leads to difficulties in connection with generalizations, as was mentioned above. Hence we may also conclude that we cannot use structure-descriptions in describing
INDUCTIVE GENERALIZATION
157
a possible kind of world but descriptions that are independent of one's list of individuals. They must be described by general and not by singular sentences. By a general sentence we mean here a sentence of first-order logic without individual constants and without free individual variables. A sentence which is not general is called singular.) This can be achieved by using in description the distributive normal forms of general sentences proved to exist e.g. in Hintikka [1965b]. According to Hintikka's results every consistent general sentence of first-order logic may be transformed into its distributive normal form which is a disjunction of certain conjunctions called constituents which are of a fixed depth d (= maximum number of layers of quantifiers occurring in it). As our language system contains a dyadic predicate and hence dependent argument places, a certain number of the constituents will turn out to be incompatible with the axioms for our universe. In the general polyadic case finding an effective method of eliminating inconsistent constituents is equivalent to finding an effective decision procedure for first-order logic, which is known to be nonexisting. However, ordered universes like ours constitute a special case in which this elimination can be accomplished. An inductive logic will be constructed by defining a regular symmetric measure function for the set of consistent constituents. The a posteriori probabilities of constituents are calculated by Bayes' theorem. Finally, the d.c. of a sentence expressible as a disjunction of constituents is of course obtained as a sum of the d.c. 's of the constituents in the disjunction. 3. The order in our universe will be defined by a dyadic predicate Rxy = "y is an immediate successor of x". Each individual in the ordering has exactly one predicate from a family of predicates. For the sake of simplicity this family will here consist of only two members P and ~ P. In defining the order we would obviously need three quantifiers. For besides saying that some y is an immediate successor of x we must add that every z that also is an immediate successor of x must be identical with y - and similarly with respect to the converse relation. Because of certain practical difficulties (to avoid laborious computations) we restrict the number of layers of quantifiers to two by using numerical quantifiers instead of the usual ones. This is not an essential restriction because it only reduces the number of attributive constituents and hence also the number of constituents while the method of eliminating inconsistent constituents remains essentially the same. The numerical quantifiers needed here are defined as follows by taking the
158
RAIMO TUOMELA
ordinary quantifiers as primitives:
(Exo)F(x) =df ~ (Ex)F(x), (Ex1)F(x) =df(Ex) [F(x) &(y)(F(y):::> y = x)], (Exz)F(x) =df(Ex)F(x)& ~ (ExdF(x).
(1)
The axioms for our ordered universe can now be given: AI. (X)(EY1)(Rxy) , A2. (X)(EY1)(Ryx) , A3. (x)(y)(Rxy:::> ~ Ryx).
(2)
Intuitively speaking these axioms say that every individual has exactly one immediate successor and predecessor. In addition the succession relation is asymmetric. What kinds of orderings are compatible with these axioms? Obviously finite cycles without initial and terminal members and infinite open orderings plus any combination (relation-theoretic sum) of these. Again, for the sake of simplicity, the relation-theoretic sums qualifying as models for our universe will be disregarded in developing the inductive logic as their effect on the degrees of confirmation of our constituents can be shown to be negligible. A proof of this is sketched in appendix 4. We could equally well have laid down other axioms for order, however, keeping in mind that the universes must be such as to permit the generalizations to be satisfiable in a finite domain. For instance, a linear order with an initial and a terminal member would also qualify as such. What will a generalization concerning our universe look like? The first task in answering this question is to define the notion of a possible kind of an individual which is called an attributive constituent in our language. The notion ofa possible kind ofa world is defined by listing all kinds of individuals that are exemplified and adding that they are the only ones exemplified. The result is called a constituent. We begin by defining the attributive constituents Ct1(x) which are complex attributes of the individual x. The index d gives the depth of the sentences in question. Consider some arbitrary individual x to be constituted. We specify its relation to all possible reference point individuals y in the ordering. The individual y is either (1) an immediate successor or (2) an immediate predecessor of x or (3) a distant individual (= neither a successor nor a predecessor). Of these (1) and (2) always exist in virtue of A1 and A2, and are unique. Hence the only question concerning them is whether they have the predicate P or not. Excluding very small cycles, there are also
159
INDUCTIVE GENERALIZATION
distant individuals. What we have to specify is whether there are distant individuals with the predicate P, and whether there are those without it. The result of these specifications is a Ct{ predicate attributed to x. To make this more concrete we give the beginning of the list of all these predicates:
__ successor distant individuals ---_
Ct;l-predicate
Cto;o(x) says CtO;1 (x) says CtO;2 (x) says Ct O;3 (x) says
Ct1;0(x) says etc. ---------------
------
predecessor
x itself
P P P P P
P P P P P -
----
_._._---
.. -.
._-_.~--
P P P P P
EP EP EP EP EP
EP EP EP EP EP
~----
(EP = a distant individual with the property P exists, etc.)
Now a purely formal description of the attributive constituents can be given. Taking into account axioms AI-A3 reduces the number of terms we need to write explicitly in the attributive constituents. For instance, terms like Rxx are excluded on grounds of asymmetry of (immediate) succession. This leads to
CtL(x) = (±)P(x) &(EYl) [Rxy &(±)P(y)] & &(EYl) [Ryx &(±)P(y)] & &(±)(Eyo)[ ~ Rxy & ~ Ryx &(±)P(y)] & & (Eyo) ~ {[Rxy &(±)P(y)] v [Ryx &(±)P(y)] v v [~Rxy & ~ Ryx &(±)P(y)]}
(3)
where i=O, ... , 7;j=0, ... ,3. The upper index (1) shows that these attributive constituents are of depth one. The number of attributive constituents is 2 s = 32 and hence the number of constituents is 2 3 2 • However, it will be seen that the number of those consistent constituents which we have to consider turns out to be very small and - with two obvious exceptions - the only attributive constituents occurring in them are of the same type Cti;o(x) which means that all kinds of distant individuals exist relative to x. Accordingly, as the attributive constituents differ only with respect to x, its immediate predecessor and successor, there is in the case of each constituent one-to-one correspondence between the attributive constituents of the consistent constituents and the following
160
RAIMO TUOMELA
triples S, which say what kind of property belongs to the immediate predecessor of x, x itself, and its immediate successor: So Sl Sz S3 S4
s,
S6 S7
P P P
P P P p P P
P j5 P j5 P P
PP P
PPP P j5 j5 .
The two exceptions as to the kind of distant individuals occurring in the attributive constituents of the consistent constituents are the two constituents containing only the attributive constituent Ct O ; 1 (x) or Ct 7 ;z(x) which allow only distant individuals with the property P or ~ P, respectively, to be exemplified. But this does not affect the one-to-one correspondence between the triples S, and our attributive constituents in each constituent. Thus we will use these triples as representatives of the attributive constituents when makingouranalysis. The next task is to define the notion of a constituent (a description of a possible kind of world) in our system. This is done by listing all kinds of individuals (= the triples S) and specifying whether they are exemplified or not. This is equivalent to saying which kinds of individuals are exemplified, and adding that these are the only kinds of individuals exemplified in our world. Formally speaking the constituent C~ is defined by (4) where tt. is the conjunction operator, and (J is the disjunction operator. But the axioms for our universe greatly reduce the number of sequences of triples which otherwise would qualify as constituents. We need an effective method for eliminating the inconsistent sequences of triples SiC = constituents). It will be given in graph-theoretical terms. We say that the triples S, and Sj are immediately adjacent (in that order) if two rightmost members of S, coincide with two leftmost members of Sj. In this case a move from the triple S, to Sj is called a transition of one step. Obviously, there are only two logically possible transitions of one step from a given triple Si' namely those to the two triples obtained by adding an individual with either the property P or P to the two rightmost members of S; Thus it is possible, for instance, to make a transition of one step from So to Sj but not to Sz. Let us now form the maximally great graph GM by
INDUCTIVE GENERALIZATION
161
taking all the triples S; as vertices and all the possible transitions of one step
(S;, Sj) as arcs. (See appendix 1 for the definition of the graph-theoretical
terms used.)
Constituents can now be represented by certain subgraphs of GM • These subgraphs (other than So and S7) are characterized by the fact that every vertex in them must occur as an initial and as a terminal vertex of some arc that does not return to the same vertex. For only then can an ordered finite universe satisfy the axioms At-A3. (Our reason for disregarding infinite universes here will be given later.) Subgraphs satisfying this condition are strongly connected or sums of strongly connected graphs. (We have already decided to disregard the latter; cf. appendix 4A.) A method of eliminating inconsistent constituents can now be given: A (putative) constituent given by (4) is finitely satisfiable if and only if it is representable by a strongly connected graph that is a subgraph of GM (or by a sum of such graphs). The list of such consistent constituents, which cannot be decomposed as sums, is given in appendix 2. It can be seen that the restrictions we have imposed on the set of all initially possible constituents are very strong. Out of 2 3 2 = 4294967296 constituents only 24 remain to be considered. An examination of our constituents (cf. appendix 2) shows that there is, with some exceptions, a one-to-one correspondence between these constituents and the graphs formed by using the triples S; as vertices. An example of an exceptional case with two-to-one correspondence is provided by the
162
RAIMO TUOMELA
graph SOS\S2S4' It corresponds to two constituents which have the attributive constituents Ct6;0(x), Ctto(x), Ct~;o(x) in common and are different only with respect to the distant individuals corresponding to S1' i.e. they include either Ct:;o or Ct:; I ' In the latter case it is denied that distant individuals with the property P exist. It can be shown that this kind of exception among the constituents is an uninteresting rarity as to its d.c. because it allows the problematic kind of triple to be exemplified by only a single evidence instance. Thus we may omit these few cases from our discussion as singularities. The proof that justifies this omission is given in appendix 4B. The one-zero matrix associated with a strongly connected graph (cf. appendix I) and hence with a finitely satisfiable consistent constituent must satisfy the following condition: in every row and every column of the matrix there must be at least one nonzero element (and at most two of them) which is not a diagonal element, except for the constituents So and S7 which are represented by an 1 x 1 matrix with 1 as the only element. An example of a consistent constituent is e.g. SoS \ S3S6S4' Its graph and matrix are presented in appendix 1. Thus all possible submatrices of the maximal matrix AM corresponding to G M which satisfy this condition will qualify as matrices corresponding to consistent constituents. Before attempting to construct an inductive logic it is, perhaps, worth while considering the possible effects of the simplifications made in constructing the set of consistent constituents. So far we have made two main simplifications whose effects have not yet been discussed in the text or in the appendixes. The first concerns the number of logically different individuals to be treated simultaneously (=the depth of the sentences in question). This number depends on the maximum number k of argument places in the predicates of the language used. In our case the most complex predicate is dyadic and the number of layers of (usual) quantifiers needed is three. But in defining our ordered universe we "quasireduced" this three-quantifier case to a two-quantifier case by using numerical quantifiers. - The other simplification was made in selecting the axioms for the universe. The axioms AI-A3 require that the graphs representing finitely satisfiable consistent constituents be strongly connected. However, if the universe is assumed to contain an initial and a terminal member these graphs need no longer be strongly connected. For instance, SOS1S2S5 would then qualify as a finitely satisfiable consistent constituent with the initial triple So' But after a transition to the second triple in the ordering (S1) has occurred So can no more be exemplified in future experience. Generalizations of this kind with "reflexive" elements (So or S7) in them are in a sense very
INDUCTIVE GENERALIZATION
163
interesting as far as their predictiveness is concerned. This is due to the fact that such cases are now allowed as have either no arc coming to the reflexive triple or no arc leaving it, and this makes it possible to predict accurately either the non-occurrence or the continual occurrence of the triple So or S7' respectively. These generalizations are also pathological as to their degree of confirmation, however, as will be indicated later. 4. The next task is to define a measure-function for constituents (= strong generalizations). The following features of Hintikka's inductive logic for the monadic case will be taken as conditions of adequacy for this measure: (1) Among possible strong generalizations compatible with the evidence there will be one approaching 1 as limit when the positive evidence increases. (2) The simplest constituent compatible with evidence will have the greatest a posteriori probability. As to the justification of these conditions both of them sound intuitively natural when considered separately. We want our strong generalizations to be as reliable as possible in a fair-betting sense. We also regard informativeness (roughly synonymous to simplicity in its presystematic sense) as a desirable feature of our strong generalizations. It will be seen that these conditions may be satisfied together in the system to be developed. The simplest way to assign a priori probabilities to generalizations is to give equal a priori probabilities to all the consistent constituents (j = 1..., w, ... ). (Of course the a priori probabilities could be assigned according to some other principle than that of indifference, too.) As a constituent is regarded as the set of its models its probability may be divided evenly among these models. The a posteriori probabilities b( C;le) are then given by Bayes' theorem: 2 b(C~)'b(eIC~) b(eIC~)
CJ
b(Cwle) =
Lb(~lc-~)J(C~) = Ii(~lcJ)" j
(5)
j
The evidence statement is given by a general sentence telling what kinds of triples of individuals have been exemplified in a sequence of n individuals. Due to the nature of our universe we can say only that these n individuals are adjacent individuals without being able to speak of their location with respect to all the other N - n individ uals of the universe. The degrees of confirmation for general sentences are obtained as sums of the d.c.'s of the constituents occurring in their normal forms. The main task is, of course, to evaluate the conditional probabilities b(eICJ). In other words we want the probability that just this kind of
164
RAIMO TUOMELA
evidence is observed provided that CJ is true. This can be evaluated as the relative frequency of models compatible with both the evidence and with CJ among all the models of A model of is a path oflength -1 in which n - 1 transitions coincide with the sequence oftransitions oftriples exemplified in the evidence. Let rjk Il be the number of different ways we can take k 1 =N -1 adjacent steps in the graph of CJ where any vertices of the graph may occur as possible initial and terminal vertices of the paths. This is just the total number of models of As our graphs are strongly connected, the evidence does not excl ude anyone-step transition possibilities from the remaining N - n individuals. Accordingly, rjk2l, where k 2 = N - n - l , represents the number of models compatible both with the evidence and with The conditional probability of the evidence is then obtained as a quotient between the number of favorable models and the num ber of all possible models 1: r<.k Il J(eICD=r:.k~l' (6)
CJ.
CJ
N
CJ.
CJ.
J
This probability measure can be shown to satisfy the usual probability axioms. The measures rjkl may be evaluated from the graphs by combinatorial means. But a more elegant way to do this is to use matrix theoretical devices and operate with the matrix associated with the graph of According to a well-known theorem, given e.g. in Berge [l964], p. 131, the element a~Y) of matrix A k (obtained by taking the product of A with itself k times) is equal to the number of distinct paths of length k which go from S, to Sj' The sum of the elements of A k says in how many ways we can take k steps in the graph of without regard to the initial and the terminal vertex of the path. This is identical with the measure r
CJ.
CJ
(7) where e is the unit row-vector, and e' is its transpose. I The only constituents consistent with the axioms AI-A3 that do not have strongly connected graphs are constituents satisfiable only in an infinite domain (e.g. 50515255). Thus their conditional probabilities of evidence are equal to zero in a finite domain, and hence their d.c.'s must also equal zero according to Bayes' theorem. Taking the limit, it is seen that this holds also in an infinite domain.
INDUCTIVE GENERALIZATION
165
We still want to know (a) what kind of function of N the measure r(k) is in order to evaluate the behavior of c:5(eIC}); (b) how it depends on the characteristics of the constituents C}. The first problem is solved by transforming A into its Jordan canonical form J by pre- and postmuitiplying it by certain nonsingular matrices P and r >. A=P-1JP (cf, appendix 3 for a more detailed explanation). Taking kth power of A is known not to affect P; thus A k =p- 1 rp (Browne [1958], p. 181). Hence (8) Here F (Ai k » is a linear combination of certain powers (~k) of the latent roots Az(l= 1, ... , r) of A. - By using theorems of the theory of nonnegative matrices some interesting results can be obtained as to the behavior of F ()t» in our case. A real matrix is nonnegative if it does not include any negative elements. It is irreducible if and only if the graph corresponding to it is strongly connected. Finally, the matrix is primitive if in the graph corresponding to it the greatest common divisor of all the lengths of its closed paths is = 1. A primitive matrix is necessarily both nonnegative and irreducible. All the matrices A j corresponding to the constituents C} are nonnegative and irreducible. They are also primitive except for those few which are permutation matrices. As to these exceptional matrices, the sums of their elements do not change when the matrices are raised to higher powers. Obviously, for such matrices c:5(eICn=l. - Thus we need consider only the theory of primitive nonnegative irreducible matrices. Some theorems concerning these matrices are stated below without proof (for proofs, see Varga [1962], ch. 2). Let A be a primitive nonnegative irreducible matrix. Then A has a dominant positive latent root t» such that: (a) w is simple, (b) w has an absolute value greater than that of any other latent root, (c) min
n
n
j=l
j=l
L aij ~ w ~ max L aij,
(d) w increases when any entry of A increases. As these results are applied to our case it is seen that the matrices A j associated with the finitely satisfiable consistent constituents have a unique dominant latent root Wi such that 1 ~ ta j ~ 2. In F (A\k» co j is the only latent root with the highest power k. Thus it is seen that the measure r(k) grows
166
RAIMO TUOMELA
exponentially towards infinity as k grows without limit. With sufficiently large values of k it can be approximated by aw~ where a is a small positive constant due to the matrices P and P - I (cf. above). Hence the expression for the conditional probability of evidence (7) (with k 2 - k 1 = n) can be approximated by (9) The complexity of a constituent Cf may be measured by the amount of one step transition possibilities in its graph. This is equal to the sum of the elements of the associated matrix A i: And by (d) this can be measured by Wj. Hence the smaller w j is the greater is the simplicity of C], Finally, the a posteriori probability of C~ is given by Bayes' theorem
() (e 2Ie) = IL~: = .. w
I
j
l/w~
I
j
1
__ .
(ww/wJn
(10)
Thus the a posteriori probability is independent of the number of individuals in the universe. It depends only on the simplicity of the constituents and the number of positive evidence instances (=n). The behavior of (10) is the most clear-cut in the asymptotic case, i.e. when n is allowed to increase without limit. Among the constituents compatible with given evidence there is (at most) one whose d.c. approaches 1 as a limit with increasing positive evidence. This is the constituent with the smallest dominant latent root and therefore the simplest in the sense defined above. The degree of confirmation of all the other constituents of course approaches zero. The qualification 'at most' is occasioned by the case where there are several constituents with equal minimal dominant latent roots. According to our previous results the matrices of these constituents cannot be primitive nonnegative irreducible ones. Therefore, only in cases where two or more permutation matrices occur among the matrices of the constituents compatible with the evidence the asymptotic behavior of (10) is exceptional. For in these cases all the constituents representable by permutation matrices have an equal share of the asymptotic degree of confirmation (= 1). - As one can see (cf. appendix 2) in our particular universe this exceptional case (= many permutation matrices compatible with given evidence) is not even possible. More generally, the degree of confirmation of a generalization consisting of a disjunction of constituents is obtained as the sum of the d.c.'s of the constituents occurring in the disjunction. Thus the d.c. of a generalization approaches I as a limit if and only if the constituent with the smallest latent
INDUCTIVE GENERALIZATION
167
root (of constituents compatible with given evidence) occurs in the distributive normal form of the generalization. Above the simplicity of a constituent was measured by the dominant latent root of the corresponding matrix. According to our previous results OJ is the greater, the more non-zero elements the matrix includes, and hence the more vertices and arcs the corresponding graph includes. In particular, the more vertices with two transition possibilities ( = arcs) there are, the more subpaths there are in the graph, hence the greater is the number of possibilities to take k steps in the graph. - All this amounts to saying that a constituent with the following properties is that with the highest d.c. both in preasymptotic and in asymptotic cases: (a) it allows the minimum number of kinds of individuals compatible with evidence to be exemplified; and (b) within these limits it allows as few transition possibilities as possible (and it is, if compatible with the evidence, a permutation matrix allowing at each point only one transition possibility). Thus our conditions of adequacy are satisfied. Let us take an example to illustrate our results. We assume that the following string ofindividuals has been observed in our universe: . .. PPPPPPP .... This amounts to saying that the triples attributive constituents 5051535654 have been exemplified in this order. The consistent constituents compatible with this evidence are: Ci=5051535654' C~=505153575654' C~ = 5 05 1525s535654,and the maxirnalconstituent C'[» 5 05 1525s535756S~. The dominant latent roots of the matrices corresponding to these constituk 2k ents can be evaluated by e.g. the power method: OJj = (trace A ) 1 / 2 . Thus we find that Cj has the smallest value of OJ( = 1.87)while C~ has the maximum OJ = 2. The other OJ-values lie between these limits. The substitution of these values in formula (10) yields the highest degree of confirmation for the simplest constituent ci. Finally a few words about confirmation of generalizations in a (linearly) ordered universe with an initial and a terminal member. As indicated previously, this situation is in general similar to ours. We have only to deal with more constituents as attributive constituents taking into account the first and the last individual must be added to the constituents. Another change would be the addition of finitely satisfiable constituents with reflexive elements (loops) i.e. constituents whose graphs are not strongly connected, e.g. 5 05 1525s· The measure-function r(k) for these constituents need not be an exponentially increasing function of N but only a monotonically increasing one, and
168
RAIMO TUOMELA
as clear-cut as here concerning their d.c.'s cannot be given. The constituents must therefore be studied one by one. For instance, the d.c. of SOSlSZSS can then be shown to approach t asymptotically. 5. Thus far the problem of evaluating the degree of confirmation a posteriori of constituents and hence of all general sentences has been solved in a way which satisfies our criteria of adequacy. We can still ask for the degree of confirmation of the hypothesis h that a certain attributive constituent compatible with CJ will be instantiated by a certain hitherto unexamined individual. This will be a singular prediction as the sentence h contains free individual symbols. We have
b(h&e) b(hle) = - bee) -, where in the denominator b(e)= Ij b(eICj ) 15 (Cj ) = Ij (1/wj)'b(CJ Here 15(11 & e) is obtained from bee) by replacing n by n+ 1. Then c5(hle)--+1 as 11--+ UJ. If we want to calculate the probability that a certain attributive constituent will be exemplified, we cannot assign equal probabilities to all the Ct;-predicates. As there are independent circuits in the graph we might first assign probabilities to them - equal or with weights proportional to the number of vertices (attributive constituents) they contain. But these probabilities do not change with increasing evidence. Perhaps we would want some learning from experience to take place even in the case of singular predictions and not only in the case of generalizations, for different circuits in the graph may not be exemplified equally often. This can be achieved by generalizing this system by adding a parameter which enables such learning from experience to take place (cf. Hintikka [1966]). But that does not belong in this paper. Anyway the main problem - the construction of a measure-function for generalizations in an ordered universe - has been solved in a way that satisfies the assumed conditions of adequacy.
ApPENDIX
1
Some basic concepts in the theory of graphs
(For reference, see Berge [1962] pp. 5-8.) We say that we have a graph whenever we have: 1. a set X, 2. a function r mapping X into X.
169
INDUCTIVE GENERALIZATION
A graph G can now be defined as the pair G=(X, r). Fig. 1 presents two graphs G1 and Gz corresponding to constituents SOSlS3S6S4 and SoS1 SZSSS3S6·
Gz
CO
So
G1 SI
S3
c~<1
"'0
S4
56
o ..
j
SI
I O\/OS, j
$3
0
S2
.. 0
"0
0
$6
Fig. 1
The set X consists here of the points or vertices Sj. (If S, and Sj are joined by a continuous line with an arrowhead pointing from S, to Sj this is denoted by SjErSi.) The pair (Sj, S) with SjErSi is called an arc of G. The set of arcs of a graph will be designated by U = {(Sj, Sj)}. Obviously G=(X, r)=(X, U). For an arc u=(Sj, Sj)' the vertex S, is called its initial vertex and Sj its terminal vertex. A path is a sequence (u1 , Uz, ... ) of arcs of a graph (X, U) such that the terminal vertex of each arc coincides with the initial vertex of the succeeding arc. A path is simple if it does not use the same arc twice, and composite otherwise. The length of a path j1=(u 1 , U z, ... , Uk) is the number of arcs in the sequence. A loop is a circuit of length I, which consists of a single arc. A graph is said to be strongly connected if there is a path joining any pair of arbitrary distinct vertices. For instance, G1 is a strongly connected graph whereas Gz is not. Finally, we need the concept of the matrix associated with a graph. If G has n points we form a n x n square matrix A. To the ith row of A we associate the point SiEX; to thejth column the point SjEX. Within the cells the elements of A are denoted by Aij; we put aij= 1 if (Sj, SJE U and aij=O if
170
RAIMO TUOMELA
(Sj, Sj)r/=U. The matrix At associated with the graph G, is given below:
At =
S0
Sl
So
1
1
St
0
0
S3
0
0
0
S6
0
0
0
0
1
0
0
0
S4
!
S3
S6
S4
0
0
0
0
0
--
0
._._-------
ApPENDIX
2
List of finitely satisfiable consistent constituents The constituents that are representable by strongly connected graphs (or by sums ofsuch graphs) are the only finitely satisfiable constituents compatible with the axioms AI-A3 given in the text. Those that are not sums of smaller graphs are listed below: (i) including So:
So SOSlSZ S4 SOSlSZ SSS3 S6S4 SOSlSZSSS3S7S6S4 SOSlS3 S6S4 SOSt S3 S7 S6S4 SZS4S0S1S3S6 SZ S4S0 S 1 S3S7S6 , (ii) including St but not So: St SZS4 SlSZSS S3S6S4 SlSZSSS3 S7 S6S4 SlS3S6S4 St S3S7 S6S4 StS3S6SSSZS4 SlS3S7S6SSSZS4 SZS4StS3S7S6 S3 S6S4St SZ'
INDUCTIVE GENERALIZATION
171
(iii) including Sz but neither So nor S\ (hence not including S4 either): SzSs SZ SSS3 S 6 SZSS S3 S 6 SZSS S3 S 7 S 6 , (iv) including S3 but none of So, S\, Sz (hence not including S4 either): S3 S 6 SS S3 S 7 S 6 SS, (v) including S4' Ss, S6 but none of So, St, Sz, S3: none, (vi) including S7 but none of SOSlSZS3S4SSS6: S7· ApPENDIX
3
The Jordan form of a matrix F or every complex n x n matrix A there exists a nonsingular n x n matrix which reduces the matrix A to its Jordan normal form, i.e. PAP-\
=
J
=
where each of the n, x n, submatrices J, has the form
172
RAIMO TUOMELA
rr,
The kth power of A is given by Ak=P -1 thus r(k) = eAke' =ep-1 Jkpe' = Ld~Y)(/)=F(,W)). F(A}k)) is a linear combination of the terms of eP -) J Pe' as explained in the text.
Li
ApPENDIX
4
A. Degree of confirmation and the partitioning of the population of individuals into subpopulations As models of our order axioms there occur finite cycles without initial and terminal members and as limit cases infinite open linear orderings as well as any combinations (relation-theoretic sums) of these. It was claimed in the text that the relation-theoretic sums may be omitted from discussion as their effect on the degree of confirmation of a constituent can be shown to be negligible in comparison to the degree of confirmation obtained when no partitioning of the population is allowed. The proof of this is easy. We count the measures r(k) both for the simple uniform universe consisting of only one ordering and for the complex case allowing the relation-theoretic sums to occur, and show that the degrees of confirmation in both cases are asymptotically eq ual. In the complex case, let the total population of N individuals be partitioned into [J subpopulations such that N l + N, ... + N p = N. Then the value of our measure r(k) becomes
In the simple case treated in the text the measure r(k) was defined as
As shown before the measures r(k) are exponentially growing functions of the number of possible one step transitions of the cycles. It can now be seen that the quotient r~)/r\k)-+ 1 as N-+oo. Denote the conditional probabilities of the evidence in the complex and the simple case by <5(eIC~)o=r~d/r6k2) and ()(eIC~)l =r\k tl / r\k2 ) , respectively, where z , and z , have the same meanings as in the text. Thus it is seen that <5(eIC~)o/<5(eIC~)l=(r~kJir\k2))/(r6k2)r\kI))-+1. Then it follows that <5(C~le)o/<5(C~le)l-+l,which completes our proof.
INDUCTIVE GENERALIZATION
173
B. Graphs corresponding to more than one constituent
As mentioned in the text there exist strongly connected graphs corresponding to two, and not only one, finitely satisfiable consistent constituents of our particular universe. There are four such constituents: SOSlSZS4' SOSlS3S6S4 and their mirror constituents (with respect to P) S7S6SSS3, S7S6S4S1S3' The problematic triples are Sz in the first two constituents and Ss in their mirror images. To each of these triples correspond two attributive constituents Ct.. 0 and Ct.. 1 (i = 1,6) with both kinds or only one kind of distant individuals exemplified. The constituents with the latter kind of attributive constituents are descriptions of an unusual world in which a certain kind ofindividual ta; 1 (x)) is allowed to be exemplified by only one evidence instance. We will show here that such exceptional constituent C; will always have a vanishingly small a posteriori probability, and thus the other constituent corresponding to the same graph can always be preferred to The d.c. of C; can be evaluated in the usual manner by (6). In graph theoretical terms r(k) is now the number of ways to take k transitions of one step such that the paths corresponding to the models are allowed to go only once through the vertex representing the problematic triple with only one kind of distant individuals. When this combinatorical task is solved it is seen that J(e\C;)= f(k 1 ) / f(k z) where f(k 1 ) and f(k z) are linear combinations of k , and k z , respectively, with the same parameters. The d.c. of C; is now obtained by (10) as usual. It is seen that when we let n-s N and Ns-s o», the denominator of (10) increases without limit and hence J( C;\e)~O in all cases.
C;.
References ACHlNSTEIN, P., 1963, Confirmation theory, order, and periodicity, Philosophy of Science, vol. 30, pp. 17-35 BERGE, C., 1964, The theory ofgraphs (Methuen & Co., London) BROWNE, E. T., 1958, Introduction to the theory ofdeterminants and matrices (University of North Carolina Press, Richmond, Virginia) CARNAP, R., 1950, The logical foundations of probability (University of Chicago Press, Chicago; second ed., 1963) CARNAP, R., 1963, Variety, analogy, and periodicity in inductive logic, Philosophy of Science, vo!' 30, pp. 222-227 HINTIKKA, J., 1965a, Towards a theory of inductive generalization, in: Proc. 1964 Intern. Congress for Logic, Methodology, and Philosophy of Science, ed. Y. Bar-Hillel (NorthHolland Pub!. Co., Amsterdam) pp. 274-288 HINTIKKA, J., 1965b, Distributive normal forms in first-order logic, in: Formal Systems and Recursive Functions, Proc. Eighth Logic Colloquium, Oxford, July 1963, ed.
174
RAIMO TUOMELA
J. N. Crossley and M. A. E. Dummett (North-Holland Pub!. Co., Amsterdam) pp. 4790 HINTIKKA, J., 1966, A two-dimensional continuum of inductive methods, this volume, pp. 113-132 PUTNAM, H., 1963, Degree of confirmation and inductive logic, The Philosophy of Rudolf Carnap (The Library of Living Philosophers), ed. P. A. Schilpp (Open Court Publishing Company, La Salle, Illinois) pp. 761-783 ROSENBLATT, D., 1957, On the graphs and asymptotic forms of finite Boolean relation matrices and stochastic matrices, Naval Research Logistics Quarterly, vol. 4, pp. 151-161 SCHNEIDER, H., editor, 1964, Recent advances in matrix theory (University of Wisconsin Press, Madison, Wisconsin) VARGA, R., 1962, Matrix iterative analysis (Prentice-Hall, Inc., Englewood Cliffs, New Jersey) WALK, K., Extension of inductive logic to ordered sequences, IBM Technical Report TR 25.053
NOTES ON THE "PARADOXES OF CONFIRMATION" MAX BLACK Cornell University, Ithaca, New York
1. Formulation of the paradoxes. The label, "paradoxes of confirmation", was assigned by Professor C. G. Hempel, in his well-known paper (Hempel [1945]), to the following sheaf of arguments. Consider the proposition, All ravens are black (which I shall call "the raven hypothesis"). Common sense is inclined to say the following three things about it: (i) The hypothesis is, or would be, shown to be false by the existence of a single non-black raven. (ii) The existence of a black raven supports, or would support, the hypothesis - is, or would be, empirical evidence favoring its truth. (iii) Not all objects bear upon the hypothesis, negatively or positively, in these ways: for instance, the existence of Halley's comet neither falsifies nor supports the raven hypothesis. In short, the common sense position is that the existence of some, but not all, things is relevant to the raven hypothesis. This might be called the principle of limited relevance. Hempel now offers the following arguments against this principle: The raven hypothesis can be symbolically represented as
(x)(Rx::> Bx),
(1)
and a positive instance of it will accordingly have the form Ra·Ba.
(2)
Now (i) is logically equivalent to its "contrapositive":
(3) a positive instance of which has the form ~
Bb-
~
Rb,
(4)
176
MAX BLACK
and is also logically equivalent to what I shall call its "comprehensive",
(x)[(Rx v
~
Rx)::::>
(~ Rx v
Bx)] ,
(5)
a positive instance of which has the form
(Rc v
~
Rc).( ~ Rc v Be).
(6)
On the assumption that logically equivalent hypotheses are "confirmed" (Hempel's term, which I shall discuss later) by the same instances, we are led to the following concl usions: (iv) Any non-black non-raven confirms the raven hypothesis (cf. formula (4) above). (v) Any non-raven also confirms the same hypothesis (because any object, c, for which ~ Rc is the case, will satisfy formula (6) above). (vi) Any black thing will also confirm the hypothesis (for Be will logically imply (6) above). Thus it would seem that the raven hypothesis is, or would be, "confirmed", for instance, by the existence of a white handkerchief (cf. (ivj], by the existence of a stone (cf. (v») and by the existence of a black pearl (cf. (vij). These conclusions are certainly startling. The foregoing arguments purport to show that any object, 0, without exception, is relevant to the raven hypothesis, in the sense of either confirming it or falsifying it. For if
NOTES ON THE "PARADOXES OF CONFIRMATION"
177
conferred upon formula (1) above by formulas (2) or (4) or (6) could be considered in abstraction from any meanings that might be attached to 'R' or to 'B' and hence without reference to any "background knowledge" which we might normally regard as relevant. Now it is argued that common sense's reluctance to accept some of the paradoxical instances produced in the last section can be explained by our possession of relevant "background knowledge": if we think the existence of a stone does nothing to strengthen the empirical evidence supporting the raven hypothesis, that is because we already know that no stones are ravens and are therefore receiving no additional information that is relevant>, It is extremely difficult, however, to suppress, ignore or "bracket" the knowledge we in fact have about ravens, colored things - and, more generally, about birds, other physical objects and other relevant broad-scale features of the universe 3. One might, however, deliberately construct an artificial context in which covert appeal to "background knowledge" would be impossible. Suppose somebody invites me to consider a proposition, H, say, of the form, All A are B, where 'A' and 'B' stand for definite characters, known to my interlocutor, but deliberately concealed from me. Let it be further supposed that my interrogator now tells me that, to his knowledge, a certain object, not further described or identified, is in fact both A and B: I am asked 2 Cf. Hempel's discussion of an attempt to "support" the assertion, All sodium salts burn yellow by burning a piece of ice: ..... we happen to 'know anyhow' that ice contains no sodium salt; this has the consequence that the outcome of the flame-colour test becomes entirely irrelevant for the confirmation of the hypothesis and thus can yield no new evidence for us" (Hempel [1945] p. 19). So far as I can see, this kind of defense could be used only in connection with "contrapositive instances" (cf. (4) above) and would leave the still more puzzling cases of "comprehensive instances" (stones, lumps of coal) to be explained. However, all three types of instances are made to seem relevant by Hempel's original argument and in the end the relevance of all three types of cases must be explained or explained away. 3 I am strongly inclined to think that nobody who lacked such knowledge could qualify as understanding the raven hypothesis or as knowing the language in which it is expressed. Could somebody who had never seen or heard of animals understand our intention in saying "All ravens are black"? And would not an effort to teach him the intended meaning naturally begin with a recital of certain/acts, familiar to us, but surprising to him - "There are creatures, like us in certain respects, but unlike in others, who do such and such, etc. etc."? If we did not need to understand the raven hypothesis in order to assess its confirmation by given empirical facts (as an advocate of a formal concept of confirmation expects), what I have just said would be irrelevant. But I shall be arguing that reference to meaning is necessary if apparently "paradoxical" consequences are to be generated.
178
MAX BLACK
to say whether, in my opinion, the likelihood of H being true has been increased by this information. The correct answer seems to me that I am in no position to have any rational opinion about the matter. For one thing, my enforced ignorance concerning the identity of the hypothesis in question prevents me from any reasonable opinion concerning the initial likelihood of H. (To have any opinion about this would be as absurd as to have an opinion about the height of some object, known to the questioner, but completely unknown to me.) Given this condition of almost total nescience on my part, it would seem absurd for me to say that I now have more empirical support for H than I had a moment ago. Of course, the report of a confirming instance does tend, in general, to favor a hypothesis, but it may be a long step from "tends to favor" to "positively supports" 4. I am inclined to say that if the presupposition of total ignorance of relevant background were taken seriously, a question concerning the empirical support for a given hypothesis could not yet arise. 'Empirical support' may well be a threshold concept whose application requires appeal to some background knowledge and hence one that fails to apply if such background knowledge is lacking. 3. Some pecularities of the stock example. I have been following the practice of previous writers in using Hempel's illustration - for it is intended to be no more than that - of the "raven hypothesis". Attention to some specific illustration is indeed essential, if what some philosophers like to call our "preanalytical intuitions" are to be consulted and, if necessary, rectified. For common sense speaks with an uncertain voice at best about abstract relations of "confirmation" between propositions about unidentified A's and B's (as I have just been arguing). However, the stock example may be held to have been un unfortunate choice, considering certain of its features which I shall now enumerate. (These oddities might plausibly be held responsible for at least some of the air of "paradox" that clings to Hempel's conclusions.) (a) Asymmetry between the raven hypothesis and its contrapositive. There The following analogy may help: Suppose I am told about some chess game, not otherwise identified, that White has just captured a pawn: should I conclude that White has a better chance of winning than he had before making that move? Well, other things being equal, capturing a pawn tends to help a chessplayer to win. But then so much depends upon the state of the game! Finding empirical support for an hypothesis may well resemble playing well at chess, rather than conducting a valid argument. (We can sometimes prove that a move is bad, but whether a move is good is usually debatable.) 4
NOTES ON THE "PARADOXES OF CONFIRMATION"
179
is a noticeable artificiality about the contrapositive, All non-black things are non-ravens and a still greater artificiality about what I have called the "comprehensive", Everything that is either a raven or a non-raven is either not a raven or black or both. If it is hard to imagine anybody seriously undertaking to put the contrapositive to empirical test, one reason may be that non-black is a non-individuating predicate, that omits any specification of the logical type of the things to which it is intended to apply. (Is a rainbow to count as a non-black thing? And what about an electron - or even a prime number?) Here may be one source of our resistance to the claim that any non-black thing, without exception, "confirms" the raven hypothesis 5. If we replace the original example by one in which predicate and subject are both individuating terms, we may get such a proposition as All vertebrate animals are warm-blooded animals, which might be symbolised as (x)(Ax. Vx
::::>
Ax· Wx).
(7)
The corresponding "restricted contrapositive", as it might be called, would be All cold-blooded animals are invertebrate animals, symbolised as (x)(Ax. ~ Wx::::> Ax· ~ Vx).
(8)
Here it is by no means implausible, or shocking to common sense, that the existence of a cold-blooded invertebrate should support the hypothesis. We might therefore conjecture that one source of the "paradoxes" is the choice, as a paradigm, of a proposition of the form (1) rather than one of the form (8). Let us call objects that are A's (i.e. animals, in our example) members of the range of the hypothesis: we would still have to explain why members inside the range seem eligible, while objects outside the range do not, to be treated as relevant instances of the hypothesis. For it is easy to transform (7) or (8) into a "comprehensive" form like (5) 6. And when this has been done, the old perplexities about apparently paradoxical instances (about unlimited relevance) will reappear. 5 Consider instead the statement, Anything whatever is either a non-raven or black or both (k, say). If my own "intuitions" can be trusted, no corresponding resistance is evoked: it seems reasonable enough to say that k is "confirmed" by a rainbow, by an electron, by the number 5 - or, more generally, by anything that is not black. The soundness of Hempel's conclusions accordingly turns on whether the raven hypothesis can properly be regarded as identical with, or at least logically equivalent to, the statement k. If the answer is Yes, Hempel is quite right. 6 Thus (7) is logically equivalent to (x) [(Ax V ~ Ax) ::> « ~ Ax V ~ Vx) v (Ax·Wx»].
180
MAX BLACK
(b) Uncertainty about the meaning of the raven hypothesis. It is natural to understand the assertion that all ravens are black as intending to attribute to all ravens some typical species-identifying character (like some identifying shape of beak). But if so, the hypothesis would express a "lawlike" association of attributes, applicable to possible as well as to actual specimens of the class of animals in question. And then any serious investigation of the truth of the hypothesis, thus construed, would be far more complicated than the symbolic formulas so far used would suggest. (If falsification of the hypothesis required only the existence of some non-black non-raven, why not "falsify" the hypothesis by painting some raven pink all over?) It seems clear, however, that neither Hempel nor other writers on this topic have understood the raven hypothesis in this natural way. It is not unfair to them, I think, to say that they conceive of the blackness of any given object as determinable by immediate inspection. What is more important, however, is their conception of the hypothesis in question as an indefinite accidental association of attributes - as the use of the sign for material implication in such formulas as (1) above shows 7. Now, if the allegedly universal blackness of ravens is conceived as a tremendous cosmic accident, with each raven merely happening to be black (if it is black), as a die might happen to throw a six, independently of what happens to any other raven, we are dealing with a proposition so unlikely, on general principles, to be true that it is hard to take the notion of "confirming" it seriously. This brings me to a final source of dissatisfaction with the stock example. (c) Uncertainty about the truth-value of the hypothesis. One is inclined to think that the truth of the raven hypothesis is a commonplace - to assume that "everybody knows" that ravens are black, just as everybody is supposed to know that robins are red. (This "background knowledge", if we genuinely possess it, will interfere massively with our efforts to scrutinise the allegedly abstract logical relationship of "confirmation".) But the hypothesis is a commonplace - if indeed it really is true - only when construed in the "lawlike" way. Taken however, as intended, as an enormously, perhaps infinitely, extended conjunction of independent singular accidental assertions, it seems almost certainly false. (If there is any non-zero probability against the "accidental" coincidence of two attributes in a given instance, the chance of these attributes coinciding indefinitely often becomes in the long run vanish7 Formal logic, for all its sophistication, is still notoriously embarrassed by the task of distinguishing a "lawlike" from an "accidental" generalization.
NOTES ON THE "PARADOXES OF CONFIRMATION"
181
ingly small- the blackness of all ravens would eventually become as unlikely as an uninterrupted run of sixes from a fair die.) It is worth noticing that, on the view of "confirmation" advocated by Hempel (and also, with some variations, by Carnap) the increment in the degree of "confirmation" contributed to the raven hypothesis by even a straightforward positive instance of a black raven is zero. For in this type of confirmation theory, every generalization has an a priori probability of zero, that cannot be raised by any finite number of positive instances s. Now, if this were correct, it would be hard to evoke the "intuitions" to which we are supposed to appeal. If the raven hypothesis remains infinitely unlikely to be true after any amount of empirical evidence has been found, it is perhaps not so paradoxical after all that a white shoe might make the same negligible contribution as anything else 9. (d) Conclusions. I have been objecting - pedantically, as some will think 10, - to my predecessors' preoccupation with the stock illustration of the ravens because it does not lend itself to contraposition and other relevant logical transformations, because its intended meaning is unclear (with a dangerous oscillation between a "lawlike" and an "accidental" interpretation) and because its presumed truth-value is uncertain (so that we don't know whether to treat it as a tiresome commonplace or a wildly implausible prediction). Can we find another simple example, free of these blemishes? Perhaps the following might serve: All American taxpayers turned on the radio at least once in the course of 1965 11 . Considering, in connection with this hypothesis This is plausible enough if a generalization is viewed as an extended logical product of indefinitely many singular statements about the independently existing individuals in the universe. Roughly speaking, the "width" of the singular confirming proposition is always negligibly small when compared with the "logical width" of the generalization under test. 9 Yet one is still inclined to think the white shoe irrelevant in a way in which the black raven is not! This is what still needs to be explained - or explained away. 10 But if empirical support is not a wholly formal notion - contrary to the hopes of the architects of inductive logic - such pedantry is quite in order. If the appropriate methods for an empirical test of a given hypothesis depend on the meaning of that hypothesis, clarification of such meaning is essential if paralogism is to be avoided. 11 In order to stay as close to Hempel's program as possible, I have deliberately chosen an hypothesis in which there can be no serious suggestion of a "lawlike" connection - one could hardly suppose that being an American taxpayer is a reason for listening to the radio. It seems to me an open question whether the taxpayer hypothesis is true: a statistical test, by sampling American taxpayers, would therefore have some point - if anybody were interested in the answer. 8
182
MAX BLACK
the three types of primajacie "paradoxical" instances produced by Hempel, we can perhaps say the following: restricted "contrapositive instances" (Americans who did not use a radio in 1965 and paid no taxes) still seem somewhat odd 12 - though perhaps less so than in the case of the raven hypothesis; on the other hand, the remaining types (e.g. non-Americans, also those who use their radios, whether Americans or not) still seem - with whatever justice - to be plainly irrelevant. I therefore conclude that the puzzle will not be resolved by introducing better illustrations: its roots must be deeper. 4. The technical sense of "confirmation" and the common notion of "empirical evidence". "Confirmation" is, of course, a term of art l 3 , however firmly established in the writings of English-speaking philosophers of science. It is plainly intended to be a technical surrogate for the common notion that [ have been referring to as "constituting empirical support" - or, at least, some component of that notion 14. Now, that common notion is closely connected with the notion of what might be called a deliberate investigation of a given hypothesis. Anybody who becomes seriously interested in determining the truth-value of some hypothesis, H, will normally initiate a search for relevant empirical data that may reasonably be expected to establish H, to refute it - or, failing either of these results, to alter the initial plausibility of H. (I shall ignore, for the purpose in hand, any questions about the coherence of H with provisionally accepted generalizations and theories.) A full-blooded, authentic, empirical inquiry of this sort (as, for instance, in contemporary investigations of the suspected causal links between cigarettesmoking and cancer) has the following characteristic features: IC
1:1
Some reasons for this lingering oddity will be suggested later.
It is good English to speak of "confirming a rumour" or "confirming a statement" -
approximately in the sense of citing an independent statement by a trustworthy source - but in non-technical contexts "confirming an hypothesis" would sound so odd as to be barely intelligible. II In the 1945 article, Hempel uses a number of ordinary, non-technical expressions, intended to indicate, at least roughly, the "preanalytical" concept he hopes to "reconstruct". He speaks of the raven hypothesis as "being tested by confrontation with experimental findings" (p. J), of its receiving "favourable evidence" (p. 2), of data being "in accord with" the hypothesis (ibid.) and so on. He also speaks of "relevant evidence" (p. 3) and asks how a fact could "affect ... [the] probability of a given hypothesis" (p. 9). It is doubtful whether these various expressions, that Hempel treats as synonyms, really do have the same meanings and the same presuppositions: for instance, some writers would refuse to assign probability to hypotheses or other general statements, while readily admitting that hypotheses can be empirically tested.
NOTES ON THE "PARADOXES OF CONFIRMATION"
183
(i) It is conducted against a background of at least provisionally accepted relevant information, notably including a large number of generalizations: the empirical investigator never begins with a clean slate 15. (ii) The investigation is typically selective or comparative: the merits of H are weighed, in the light of acquired empirical findings, against those of a finite number of rival hypotheses, H', H", .. .16 • (iii) The inquiry may be viewed as a finite series of investigation-episodes, each of which is conducted according to a predetermined plan of operations and prearranged understandings (rules of valuation) as to how and to what degree the possible outcomes of the episode are to count in H's favor. An empirical investigation, as here conceived, is not a fishing expedition conducted at random, but an orderly series of consecutive operations, performed according to a well-defined modus operandi, that may not be changed, without good reason, during the course of the investigation in question. When an hypothesis, H, has successfully withstood a rigorous examination of this kind it may be said to have been empirically supported, in a strong sense of that expression. Alternatively it may be said to have acquired (a certain amount of) empirical evidence. The notion of empirical support, as I have depicted it, is of course thoroughly "pragmatic": whether a given hypothesis has been supported (and in what degree) depends upon historical facts concerning the identity and competence of the investigators in question and the soundness of the procedures employed by them, as well as upon the character of the facts uncovered. It is particularly important to stress that the amount of support that acquired data yield normally depends strongly upon the method employed in finding those data 17: If I merely hear by accident of a hundred black ravens in Jerusalem, that cannot appreciably strengthen my belief in the raven hypothesis; the case would be different if the batch had been selected from scattered regions in accordance with a rigorous sampling technique. My sketch of the main features of the empirical investigation of hypotheses is somewhat idealized: important scientific discoveries have been made by 15 Try to imagine Sherlock Holmes operating with no experience of human nature, no preconceptions about criminal behavior, and so on! 16 In current statistical procedure, a choice is usually made between two hypotheses, antecedently designated. (The choice of the hypotheses partially determines the character of the subsequent inquiry.) 17 This point was repeatedly and emphatically made by Peirce. In this respect, he saw further than his commentators who have, for instance, tended to pooh-pooh Peirce's insistence upon the importance of "predesignation".
184
MAX BLACK
accident and we often do, in practice, admit, as empirical evidence, data stumbled upon in the absence of prior commitment to a rigorous searching procedure. (We can sometimes accommodate such uncovenanted findings retroactively, by treating them as ifwe had set out to find them - we might then, perhaps, speak of "virtual empirical support".) There are, therefore, weaker senses of "empirically supports" and "constitutes empirical evidence for" which do not imply the deliberate contrivance, adherence to explicit rules of procedure, and so on, that are characteristic of what I have above called the "stronger" senses of these expressions. Yet something of the stronger senses still clings to the weaker senses: if we count the fact F, however it came to our knowledge, as supporting H, that is normally so - in view of the nature of empirical support here advocated because we can, after the event, concoct a defensible and appropriate procedure by which F might have been discovered in such a way as to have rendered H more credible than its designated rivals. When this is not the case, there will be something distinctly misleading about using such expressions as "empirical support" or "empirical evidence". Let us see how these ideas apply to Hempel's discussion. He says that the existence of a black raven "confirms" the raven hypothesis. If one agrees, that is - I suggest - because we substitute "empirically support" for "confirms" and naturally think (retroactively!) of the admissible procedure of selecting a raven at random in order to determine whether or not it is black. But even this example suffices to highlight the gap between Hempel's designedly formal concept of "confirmation" and the pragmatic concept of empirical support outlined above. For it is one procedure to select a raven at random in order to determine whether it is black, and a different procedure to select a black thing at random (assuming that makes sense, which seems very, questionable) in order to see whether it is a raven. In general, an hypothesis, All A are B, will receive different amounts of empirical support from A random A has been found to be B and from A random B has been found to be A, respectively, while the hypothesis of course receives the same "confirmation" from Aa-Ba as from Ba-Aa i8. The gap between the two concepts becomes still more glaring when we consider the paradoxical but allegedly confirming instance of a non-raven. What would be the corresponding modus operandi of an "investigation epiJ' Cf. G. H. von Wright's discussion of the paradoxes in Von Wright [1957] pp. 125-127, in which he relies upon the difference made by the order in which an investigator comes to observe an A or a B.
NOTES ON THE "PARADOXES OF CONFIRMATION"
185
sode" to inquire into the raven hypothesis? Could it be: to look at anything one pleases, with the prior understanding that if it turns out to be not a raven that will count as supporting the hypothesis? Apart from the intrinsic absurdity of such a rule of operation, there is a conclusive objection to it, viz. that the outcome would not discriminate between the raven hypothesis and its contradictory, No ravens are black 19. Hence, finding a non-raven would not count as a test of the hypothesis 20, would not count as empirical support for it. To sum up, there seems to be an appreciable and important gap between the formal and artificially constructed concept of confirmation and certain familiar if somewhat ill-defined concepts in common use ("empirical support", "empirical evidence") of which it is intended to be a technical "reconstruction". Whatever empirically supports an hypothesis must at least be in logical agreement with it - but the converse is untrue. Now one explanation for the "paradoxes" may well be that we naturally think of them in terms of the pragmatic concept, rather than in terms of the technical concept of "confirmation". Of course, Hempel might retort that he can hardly be held responsible for any such confusion between the two concepts: let us remember what he means by "confirmation" and all risk of confusion will be obviated 21. But the issue is neither so simple, nor so clear-cut. The whole interest of Hempel's studies in confirmation (and of similar essays by those who hope to develop a formal inductive logic) depends upon the "adequacy" of his technical surrogate for the common notions of support and empirical evidence. Any striking disparity between the two notions is a prima facie ground for distrusting the "logical reconstruction" offered. (This, as I have already said, is what makes the "paradoxes of confirmation" philosophically significant - something more than substitutes for the daily crossword puzzle.) 5. Possible ways out. It may be useful at this point to survey the various strategies that have been used - or that might plausibly be used - for dealing with the paradoxes. 19 On the customary symbolization, (x) (Rx ::::> Bx) is compatible with (x) (Rx ::::> ~ Bx). But cf. the discussion below of "ordinary" uses of conditionals. 20 This point has been well made by Nelsen Goodman and, following him, by Scheffler [1963]. 21 One may recall the disputes between Spearman and those psychologists who rejected his definition of "intelligence" as inadequate. Spearman and his defenders used to say that by "g" they meant "g" (as technically defined, e.g. in connection with factor analysis). If anybody wanted to identify "g" with "intelligence" that was his affair!
186
MAX BLACK
We might (a) reject Hempel's original argument as unsound in some identified respect; or, we might (b) accept the soundness of his argument. If the latter course seems right, we shall need to explain why the argument's conclusions should seem "paradoxical". In so doing, we might rely (b i ) upon the temptation to invoke "background knowledge"; or (b 2 ) upon disparities between the technical notion of "material implication", as used in the paradox-generating arguments, with "ordinary" uses of "if-then"; or (bj) upon confusion between low (or negligible) confirmation and irrelevance; or, finally, (b 4 ) upon some other significant differences between "confirmation" and other notions with which it might easily be confused. (The last four are obviously compatible strategies.) I shall now make brief comments upon these possible solutions, reserving (b 2 ) and (b 3 ) for more extended treatment later. (a) Rejecting the paradox-engendering argument. Given the relative simplicity and perspicuity of Hempel's original exposition, there seem to be only three ways in which this might plausibly be done. (a.) We might try to reject the assumption that logically equivalent propositions receive exactly the same confirmation from given data. This is a decidedly uninviting stratagem. If "logically equivalent propositions" are understood to be such as are, of logical necessity, rendered true or false by precisely the same states-of-affairs, the conclusion seems inescapable that they cannot be confirmed in different degrees by the same evidence. (This verdict remains correct if we substitute "empirical support" for "confirmation" .) (a 2 ) We might try to reject the remaining assumption that an instance of a simple generalization always confirms it 22 . This is somewhat more plausible (given what we know, at the back of our minds, about the lurking notion of "empirical support"). However, I find it hard to see any reasonable way in which this assumption could be denied, once a formal definition of confirmation had been adopted. (a 3 ) We might try to question the alleged logical equivalence between the original raven hypothesis and its "contrapositive" (formula (3) above) or between it and its "comprehensive" (formula (5) above). There can be no question, however, that the three propositions, as expressed, are indeed logically equivalent. (Any lingering doubts about the equivalence of these That is to say, that a truth of the form Aa-Ba always confirms the corresponding hypothesis (x) (Ax =:> Bx).
col
NOTES ON THE "PARADOXES OF CONFIRMATION"
187
expressions - or, rather, the ordinary-language expressions corresponding to them - might better be dismissed under one of the subheadings that immediately follow.) On the whole, then, it seems to me that Hempel's argument, taken as intended, must be regarded as perfectly sound. There is no prospect of finding an internal flaw in it: if we are startled by its conclusions, the fault must lie in some stubborn confusion or prejudice. (b) Accepting the paradox-engendering argument. Hempel has made a strong case for the view that the common sense principle of limited relevance, which I mentioned earlier, arises only from a "misleading intuition": he claims that "the impression of a paradoxical situation is not objectively sound; it is a psychological illusion" (Hempel [1945] p. 18). We tend, he suggests, to confuse "practical" with "logical" considerations: for the very form of the raven hypothesis reveals a practical interest in the application of the hypothesis to ravens - and a relative lack of interest in its bearing upon non-ravens, non-black things, etc. 23 Yet, in this case, as in others like it, " ... the hypothesis nevertheless asserts something about, and indeed imposes restrictions upon, all objects (within the logical type of the variable occurring in the hypothesis ... )" (Hempel [1945]). The raven hypothesis, we need to see, is indeed an assertion about 24 every physical object in the world, claiming of each such thing that it is either a non-raven or a black thing; and, for similar reasons, any generalization whatever is about each and every thing of the appropriate logical type in the universe. Once we grasp this point, it should no longer appear startling or paradoxical that every physical object, without exception, bears one way or the other upon the truth of the raven hypothesis. On the whole, Hempel's analysis impresses me as attractively straightforward and persuasive, by contrast with some of the more elaborate explanations of the "psychological illusion" that some subsequent writers have proposed. If further explanation seems needed in order to account for our proneness to overlook the simple logical point that Hempel mainly relies upon (the logical equivalence of formulas (1), (3) and (5) above), we have a choice be23 Is it always true - or even generally true - that the subject of an assertion "interests" us more than the predicate, and that what is explicitly mentioned interests us more than what is implicit? How would "interest", in the relevant sense, be defined or detected? 24 There is surely a covert extension here of the common notion of about? Even philosophers don't talk "about" everything whenever they utter generalizations.
188
MAX BLACK
tween a number of plausible options (see (b 1)-(b4 ) above). Since enough has already been said about the possible influence of "background knowledge" (i.e. (b 1 ) ) , I shall proceed at once to discuss the possible influence of the logical gap between material implication and ordinary uses of "if-then" (option (b z)). 6. Some relevant pecularities of material implication. (a) Ordinary uses of singular conditionals. I wish to recall some familiar features of ordinary uses of sentences of the form If A then C, or of similar sentences obtainable from such sentences by changes of mood. The "logic" of such ordinary singular conditionals is closely related to the "logic" of general statements of the form All A are B and may be expected to throw some useful light on the latter. We shall find it convenient to distinguish between indicative singular conditionals, such as "If the temperature falls, we shall have snow", subjunctive singular conditionals, such as "If you were to touch that plate you would get burned", and counterfactual singular conditionals, such as "If [ had betted on Excelsior, I would have won". I can think of no other types that are relevant to the present discussion. It has often been observed that when a speaker asserts an indicative singular conditional, he normally implies some connection between the antecedent and the consequent. Suppose [ say, "If you interrupt Robinson now he will be angry". [f you do proceed to interrupt Robinson and he does become angry, that will not necessarily show that my original assertion was true: for if Robinson became angry because somebody entered the room at the moment you interrupted him, we should have to say that the truth of my original assertion remained unsettled. Thus the force of my original remark was approximately the same as that of "If you interrupt Robinson he will become angry because you interrupt him". In this more explicit form, the word "because" expresses the intended presence of some reason (often, though not always of a causal sort) why the antecedent and consequent should have the same truth-value. The character of the imputed connection between antecedent and consequent varies from case to case: antecedent and consequent may be intended to be both true in virtue of some common cause, or the implied link may be supplied by sornebody's promise, decision, and so on. The general formula seems to be that the truth of the antecedent A is such as to provide a reason, of some sort, for the truth of B. (Hence, somebody who in ordinary life says "If A then B" can always properly be asked, in the kind of case I here have
NOTES ON THE "PARADOXES OF CONFIRMATION"
189
in mind, why the truth of A should make B also true.) When a singular sentence is used in this familiar way, I shall speak of the statement as a connected singular conditional. For the reasons I have explained, a connected singular conditional statement is a stronger statement than the corresponding material conditional, symbolized as "A::::J B". Although if-then sentences are normally used in the way I have described, there are I believe, special and exceptional occasions when a speaker wishes to be understood as making only the weaker statement. When I say "If that penny comes down heads when tossed now, so will that other penny", I cannot mean that the truth of the antecedent will constitute a reason for the truth of the consequent: my intended meaning is simply that if A is made true, B will as a matter of fact, and not for any specifiable reason, also be true - just that and nothing more. In such a case one might speak of an unconnected or, perhaps an accidental singular conditional. Of course, a connected singular conditional logically implies the corresponding accidental singular conditional - but not vice versa. I do not wish to argue here that "if-then" has different meanings or senses in the types of cases I have called "unconnected" and "accidental". If I had to choose, I would say that the same meaning was involved each time. (b) Truth-conditions and direct verification of accidental conditionals. It is obvious that an accidental singular conditional is directly verified by A·B and is directly falsified by A· ~ B. If you toss both pennies and both show heads, my original assertion was true (since I made no further claim about there being any connection between the two states of affairs); if you toss them and the first shows heads, but the second tails, my original assertion was false. But suppose you throw the first penny into the fire as soon as I have made the prediction, so that the antecedent A remains unfulfilled. Then it seems that the truth-value of the conditional remains open; and the original assertion has received and can henceforward receive no direct test. (If this is correct, there is a sharp contrast with the truth conditions for a material implication of the form A::::JB.) We need not abrogate the law of excluded middle in such a case: even if you refuse to make the test, I can still sensibly maintain, "What I said was true - if you had tossed both pennies, the second would have come down heads if the first did". But in the absence of direct test, any further argument about the conditional's truth-value will have to rest upon indirect evidence. (If I had some hidden device that would allow me to produce heads at will, ( might have a good reason for reaffirming the
190
MAX BLACK
truth of the original accidental conditional, in the absence of direct verification.) Let us now compare these results with the corresponding results for the contrapositive, If not-B then not-A. It is seen at once that while this proposition is directly verified by ~ B· ~ A and directly falsified by ~ B· A, it has no direct verification or falsification if B is true. Thus we see that whereas the original proposition and its contrapositive are both falsified by the same complex state-of-affairs, A· ~ B, A· B, which directly verifies the original proposition, leaves its contrapositive's truth-value open, while ~ B· ~ A, which directly verifies the contrapositive, leaves the truth-value of the original proposition open. In the case of ~ A • B, neither of the two propositions receives direct verification. If we write P for the original proposition and Q for its contrapositive, we shall obtain the following summary: P is verified by A· B, falsified by A· ~ B, left open by ~ A • B and by ~A·~B.
Q is verified by ~ A . ~ B, falsified by A • ~ B, left open by ~ A • B and by A·B. Thus P and Q have different ranges of direct verification: if two men betted on P and Q respectively, then if one lost so would the other, but one might win while the other neither lost nor won. It seems, therefore, that in ordinary uses an accidental singular conditional and its contrapositive are not logically equivalent. This point can be clinched by showing that situations can arise in which one of the two propositions is directly verified while the other is actually false. Suppose P is "If you now press the switch, the light will go on" and Q is the corresponding contra positive, "If the light does not now go on, you will not in fact have pressed the switch", where both are intended to be taken "accidentally". Then if you do not press the switch and the light does not go on (i.e. if .~ A • ~ B is the case) Q will be directly verified. But this result is compatible with the falsity of P: we might know, for instance, that the lamp was broken and therefore be in a position to assert, in retrospect, "If you had pressed the switch the light would not have gone on", and hence to derive the falsity of P. Such results as these are so unlike the corresponding results for material conditionals, that conclusions based, as in Hempel's arguments, upon theorems of the standard propositional calculus must be interpreted with great caution. Before leaving this topic, we may notice the following simple way of representing "accidental singular conditionals" in terms of the familiar symbolism of the propositional calculus. Using the technique of Carnap's "reduction
NOTES ON THE "PARADOXES OF CONFIRMATION"
191
sentences" we may write the following formulas:
== B) B ::J (Q == ~ A)
A ~
::J
(P
which highlight the indeterminacy of direct verification previously noticed. These expressions can, in turn, be "solved" for P and Q, respectively yielding: P==A·Bv
~A·X
Q==~A·~BvB.Y
where X and Yare to be taken as indeterminate parameters - propositions of unspecified truth-values 25. As I have already suggested, values of X or Y, respectively, can sometimes be supplied by indirect reasoning from similar cases or, what comes almost to the same thing, by indirect reasoning from relevant generalizations. (c) The verification of restricted accidental generalizations. Let us now apply the results already obtained to the verification of the general statement All the white balls in this urn are solid (P', say). I have chosen a statement that, to common sense at least, seems to be about a finite, although unknown, number of objects, viz., the white balls contained in the urn in question. The corresponding "restricted" contrapositive may be taken to be All the nonsolid balls in this urn are non-white (Q', say). Our generalization, p', may reasonably be construed as a finite conjunction of an indefinite number of accidental singular conditionals: it says of each ball in the urn that if it is white, then, as a matter of fact (and not for any special "connection" or reason) that ball is also solid. The asymmetry between the conditions for direct test of an accidental singular conditional and its contrapositive will obviously reappear in the present case. When P' is directly tested by examining each ball in the urn separately, a given ball may be found to agree with P' by being both white and solid (W.S) or it may disagree with it by being white and not solid (W. ~ S), whereupon the testing process will terminate; but if it should be non-white, the instance will be dismissed as irrelevant. If, however, the restricted contrapositive, Q', is 2& It is easy to see that these parameters must be functions of A and B. Suppose we have Pi == A·B v ~ A,Xl and P2 == A·~ Bi V ~ A,X2. We want Pi. and P2 to be contradictories, which will require ~ A :::J ~ (Xl,X2) to be the case. Thus the "parameters" cannot be chosen altogether freely, if the ordinary conventions for "if-then" are to be respected. I shall not pursue this topic here.
192
MAX BLACK
being directly tested, different judgments will be in point (except in the case of falsification). Let a be the class of white-and-solid balls in the urn; b the class of whiteand-non-solid balls in the urn; c the class of non-white-and-solid balls in the urn; and, finally, d the class of non-white-and-non-solid balls in the urn. Then P' is partially verified by each member of a, is falsified by each member of b, and is unaffected by each member of c and by each member of d; while the restricted contrapositive, Q', is unaffected by each member of a, is falsified by each member of b, is unaffected by each member of c, and is partially verified by each member of d. We have, once again, the pattern previously observed of different though overlapping verification conditions, but identical falsification conditions. There is, however, the following new point. In order to establish P' directly, we must eventually examine each ball in the urn, in order to record it as partially verifying the hypothesis, or as irrelevant to it. If P' is in fact true (and not void on account of the absence of any white balls) completion of the entire process of direct testing will also, thereby, supply all the data we need for direct verification of the contrapositive, Q'. It is easily seen that if we have found by direct test that P' is true, then Q', if not vacuous (through the absence of non-solid balls in the urn) must likewise be true. We may therefore say, without doing any violence to our "intuitions", that any direct test of P' will be an indirect test of Q', and vice versa. This will sometimes make it plausible to examine cases of non-solid balls, even if we are setting out to test the direct generalization, P'. For example, if we knew that we could locate three balls that were known to be the only non-solid ones in the urn, examination of each of them for their color would provide us with a rapid way of indirectly testing P', without the lengthy and tedious routine of successively examining each ball in the urn. This might be regarded as a scrutiny of all the possible negative instances - those that might be of the form ~ S· W. On the other hand, it would never be appropriate to consider cases that are ~ W· S - whether we were testing P' or Q'. [I' we knew that we could extract a certain subset of the balls, each known to be both non-white and solid ( ~ W· S), we should at once discard them as being irrelevant to the testing of either P' or Q'. If we now apply these results to a modified form of Hempel's example, such as the hypothesis, All ravens in the New York Zoo are black (R, say), we shall want to say that each black raven in the Zoo directly confirms R, each non-black raven in the Zoo falsifies it, each white or other non-black object is irrelevant (so far as direct test goes) and each black thing in the Zoo
NOTES ON THE "PARADOXES OF CONFIRMATION"
193
either confirms it or is irrelevant, depending on whether or not it is a raven. Furthermore, each thing outside the New York Zoo must count as irrelevant. But suppose Hempel, or someone who agrees with his approach, asks us to consider instead the original raven hypothesis, All ravens are black? If this is intended to be understood as an unrestricted accidental generalization, to the effect that each raven (past, present and future) is in fact black, it is doubtful whether the notion of "direct verification" with its implication of successive scrutiny of every physical object, without exception, continues to make sense 26 . At any rate, the chance of such a tremendous cosmic "accident" occurring is so small, on general grounds, that there would be something odd about saying that observation of a single black raven should count as "partial" verification. Here, perhaps, "verification" assumes the negative sense of "absence of falsification" and with this understanding the sting is removed from the paradoxes. There is certainly nothing paradoxical about saying that both a white shoe and a black cat fail to falsify the raven hypothesis - although even here we might wish to make a distinction between an object, like the first, that might have falsified the hypothesis, and one like the second that could not have done so. Paradoxical suggestions would be conveyed by this manner of description only if we were led to suppose that an appropriate method of testing an unrestricted accidental generalization might reasonably consist of an unsystematic and exhaustive scrutiny of every object of the appropriate logical type in the entire universe. (d) Transition to the case of connected conditionals. I have been arguing that an "accidental" singular conditional of the form If A then B, does not have the same direct truth-conditions as its contrapositive, Ifnot- B then not-A, and may therefore be treated as a distinct proposition. I now wish to consider whether a similar point can be made with respect to "connected" singular conditionals. Take, as an example, If I press the switch the light will go on (If A then Bor P) with the intended implication that fulfillment of the antecedent will be a reason for the fulfillment of the consequent. The possibilities for direct testing of this strong conditional are even more restricted than in the case of the corresponding weak, accidental, conditional. For even if I do press the ~6 In the case of the balls in the urn, discussed above, the examination of a single ball is one step forward in a procedure which I know, in advance, will terminate at a known instant. But to undertake to scrutinise "everything in the universe" would be to start something which could never be known to have been accomplished.
194
MAX BLACK
switch and the light does go on (A. B), that might still have been just a coincidence, and more evidence of an indirect sort (e.g. concerning the mechanism of the switch) will be needed before the assertion can be regarded as established. Similar remarks apply to the contrapositive, If the light will not go on then I will not have pressed the switch (Q). SO, with respect to both P and Q we now have the situation that each of them is falsified by A· ~ Band neither is directly verified by any of the three remaining possibilities, A .B, ~ A • B, and ~ A • ~ B. It looks as if the asymmetry of the truth-conditions upon which I previously relied has vanished. Common sense, however, will still wish to make a distinction between the bearing upon P of the case A . B and the bearing upon it of the cases ~ A •B or ~. A • ~ B. Consider the first: If I press the switch, then for alII know the light may not go on, in which case P would be false; if, therefore, the light does go on, I have, to be sure, obtained no conclusive evidence of P's truth, but I have obtained some relevant information. A natural way to describe this would be to say that I have obtained some partial verification of P. (This would be all the more natural if we were to think of P as a conjunction of an accidental conditional and an assertion about the imputed "reason" or "connection"; the observation of A· B then directly verifies the first conjunct, while leaving the second open.) On the other hand, ifI do not press the switch, then there is no chance of my falsifying P, so nothing that subsequently happens, whether the light goes on or not, can give me any direct information. These cases, one is inclined to say, are irrelevant. If we apply similar considerations to Q, we get the following patterns of truth conditions for the original assertion and its contrapositive: P is falsified by A· ~ B, is partially verified by A· B and is unaffected by ~ A . B and by ~ A • ~ B, Q is falsified by A • ~ B, is partially verified by ~ A • ~ B, and is unaffected by ~A·B and by A·B. Thus we get a modified asymmetry, somewhat resembling what we found in the case of accidental conditionals, and can proceed as before. We must notice, however, that the logical relations between "strong" conditionals such as P and Q differ from those of the corresponding "weak" conditionals. It is easy to see, indeed, that if P is true, Q must also be true; and if P is false, Q cannot be true. Thus we must admit a relation of logical equivalence between the two propositions. It follows that any case that partially verifies P by direct test (A. C) will also indirectly and partially verify its contrapositive Q. Our analysis must accordingly be modified as follows:
NOTES ON THE "PARADOXES OF CONFIRMATION"
195
TABLE OF TRUTH CONDITIONS
A· ~ B directly falsifies P, directly falsifies Q. A·B partially and directly verifies P; and hence partially and indirectly verifies Q. ~ A· ~ B partially and directly verifies Q; and hence partially and indirectly verifies P. ~ A· B leaves P unaffected; leaves Q unaffected. (e) Conclusions. I believe that a prima facie case has now been made for thinking that the discomfort produced by the paradoxical cases of confirmation is partly due to the logical gap between material implication and "ordinary" implication. However, it is hard to be sure of this in the absence of any thorough and comprehensive examination of the discrepancies between the two concepts 27. 7. Bayesian approaches. I have left for the last a type of "solution", involving considerations of "inverse probability", that has been astonishingly popular P', considering the notorious difficulties that have generally discredited the classical "Bayesian" approach to the confirmation of empirical generalizations 29. The argument fastens upon the circumstances that the number of ravens in the universe is very much smaller than the number of non-black things. This being admitted, an attempt is made to show that the "prior" or antecedent likelihood of finding a raven to be black is smaller than the likelihood of finding a non-black thing to be a non-raven. Indeed, if the class of ravens is much smaller than the class of non-black things, the first likelihood is much smaller than the second. (Call this last contention Step one.) It is now urged that, on the basis of Step one, the increase in degree of confirmation produced by finding a raven to be black is much greater than the increase produced by A valuable contribution to this neglected task is a recent paper by Adams [1965]. See, for instance, Hosiasson-Lindenbaum [1940]. This contains one of the best attempts to deal with the paradoxes in the way now to be explained. For critical comments, see Hempel [1945] p. 21, footnote 2. A recent attempt of this sort is Mackie [1963], which contains references to other essays in the same vein. 29 There is a useful summary of such difficulties in Von Wright [1957] pp. 112-117. Von Wright says that" ... such uses of inverse probability as those of determining the probability that the sun will rise to-morrow or that the next raven will be black are illegitimate" (p. 115). Anybody who agrees will reject the "Bayesian approach" to the paradoxes at the very outset. 27
2B
196
MAX BLACK
finding a non-black thing to be a non-raven (call this last claim Step two). We now explain our initial reluctance to see that a white shoe, or any other non-black non-raven, is an authentically confirming instance of the raven hypothesis, as arising from a confusion between low confirmation and irrelevance: intuitively grasping, as we are supposed to do, the negligible contribution to the confirmation of the hypothesis made by a "paradoxical" instance, we mistakenly suppose that it makes no contribution at all. Once this error has been detected, there is no further reason to reject such an instance as irrelevant, even though it makes a trifling contribution in practice to the support of the hypothesis that interests us. J believe I can here dispense with a detailed examination of the ingenious arguments by which this approach has been supported, since the following considerations seem sufficient to show its inadequacy. (i) The defense of what I have called Step two (the crucial link in the argument offered) is admittedly intricate and problematic 30. (ii) Hence, even if Step two is correct (which I doubt), common sense - if it does rely upon the supposedly different contributions made by the two sets of instances - must be using a fallacious argument. (iii) No reason has been given to believe that "common sense" does in fact believe in Step tH'O - indeed the empirical evidence (and this is an empirical question l) suggests that common sense simply holds the paradoxical cases to be irrelevant. (iv) Consider the contrapositive, H', of the raven hypothesis, i.e. All non30 One might be inclined to think it obvious that an instance antecedently less likely to arise supports the hypothesis (H, say) more strongly than does an instance that is antecedently more likely to arise. (Roughly speaking, the less surprising an observed consequence of a law under empirical test, the less support such an observation gives to the law.) But any careful attempt to calculate the relevant degrees of "confirmation" quickly reveals the implicit fallacy. Call the positive instance (a black raven) a and the contrapositive instance (e.g. a white shoe) b. Let the probability of a being observed if H is true, P(aIH), be PI and similarly, let P(bl H) = qi; let P(a] ~ H) = P2; and, finally P(bl ~ H) = qz. Then, the observation of a raises the antecedent odds in favor of H in the ratio pl!P2; and the observation of b raises those same odds in the ratio of ql!q2. Whether the positive instance, a, supports H more strongly than the contrapositive instance, b, therefore depends on whether p ilp« is greater than QJ/q2. Clearly, more is involved than the sizes of the classes of ravens and non-black things respectively. What is at stake may, with some simplification, be said to be whether a predominance of contrapositive instances over positive instances is more likely on the supposition that H is true than upon the supposition that H is false. It is hard to see how this question could possibly be answered. As Peirce said, universes arc not as plentiful as blackberries; and hence speculation about the number of white shoes to be expected if not all ravens are black is bound to be idle.
NOTES ON THE "PARADOXES OF CONFIRMATION"
]97
black things are non-ravens (or Nothing that is not black is a raven). On the solution proposed, "common sense" ought to treat direct instances of H' (non-black things that are non-ravens) as irrelevant'". (For the subject class of H', the non-black things, is supposed to have enormously more members than the complement of its predicate class.) This is as paradoxical as what is supposed to be explained. On the whole, the Bayesian approach seems to me wrong in principle and ineffective in practice.
8. Postscript. Considering the amount of sophisticated discussion that the paradoxes have received, the lack of some generally acceptable solution is disappointing. The preceding remarks have no claim to serve as a satisfactory basis for such a solution. If they have any merit, it may be that of drawing attention to some subtleties that have been overlooked in the past. I have been concerned throughout to stress the gap between the syntactical notion of confirmation and the common notion of "empirical evidence" (see especially section 4). But the latter notion is still shrouded in unnecessary obscurity. References ADAMS, E. W., 1965, The logic of conditionals, Inquiry, vol. 8, pp. 166-197 HEMPEL, C. G., 1945, Studies in the logic of confirmation, Mind, vol. 54, pp. 1-26, 91-121 HOSIASSON-LINDENBAUM, Janina, 1940, On confirmation, J. Symbolic Logic, vol. 5, pp. 138148 MACKIE, J. L., 1963, The paradox ofconfirmation, British Journal for Philosophy of Science, vol. 13, pp. 265-277 SCHEFFLER, I., 1963, The anatomy of inquiry (Alfred A. Knopf, New York) VON WRIGHT, G. H., 1957, The logical problem of induction (Macmillan, New York)
31
This point has been well made by Professor I. Scheffler.
A BAYESIAN APPROACH TO THE PARADOXES
OF CONFIRMATION * PATRICK SUPPES
Stanford University, Stanford, Calif
1. Introduction. What I have to say about the paradoxes of confirmation from a Bayesian standpoint is rather simple. The ideas have been implicitly expressed several times, probably first by Hosiasson-Lindenbaum [1940]. Perhaps the only virtue of the present paper is to make the Bayesian ideas very explicit. The remarks in the last section on the different probabilistic forms of causal and noncausal laws are very likely the most original aspect of the analysis. The paradoxes arise from two "facts". First the sentence
(\7' x)(Ax -> Ex)
(1)
is logically equivalent to its contrapositive:
(\7' x)(i3x
->
Ax),
(2)
where "-,, is the symbol of negation (and later of set complementation). Second, the singular sentence Aa&Ba (3) seems to confirm (1) in a way that the singular sentence
Aa&Ba
(4)
does not, but with respect to (2) the roles of (3) and (4) are reversed, even though (1) and (2) are logically equivalent.
2. Bayesian approach. On a Bayesian approach, we first look at the four classes and assign each a prior probability in the universe of objects - exactly how this universe is to be characterized I leave open for the moment.
* r am indebted to Ernest W. Adams and Paul Holland for several helpful comments on an earlier draft of this paper. The writing of this paper has been partly supported by the Carnegie Corporation of New York.
THE PARADOXES OF CONFIRMATION
J99
Using the familiar notation '{x :Ax}' for describing the set of objects x that have property A, we then have in terms of four mutually exclusive and exhaustive classes p({x: Ax &Bx}) = PI P({x: Ax&Bx}) = pz p({x: Ax &Bx}) = P3 P({x: Ax&Bx}) = P4 and LPi = 1. Also for simplicity I assume throughout that Pi=FO, for i= J, 2, 3, 4. If we take the familiar example and let' Ax' be 'x is a raven' and' Bx' be 'x is black', then P4 should be much larger than PI' Pz and P3 for any very broadly construed universe. The central question is why we are right in our intuitive assumption that we should look at randomly selected ravens and not randomly selected nonblack things in testing the generalization that all ravens are black. We may consider the general case, representing classes by 'A' and 'B' in the obvious way: A = {x :Ax}, etc. First of all, we note that
peA) = PI + Pz,
(5)
PCB) = PI + P3'
(6)
and thus in terms of conditional probability
PCB I A) = __PJ __
(7)
P4--. P ( A- I B-) = ---pz + P4
(8)
PI
+ pz
Now we want to justify the sampling rule that we look at A's rather than
non-B's if P(BIA)
PI--- < ----P4 . - -PI
+ pz
pz
+ P4
(9)
It is an immediate arithmetical truth that (9) is true if and only if
(10) I believe the argument is straightforward for holding that the decision to look at A's rather than non-B's hinges upon the simple inequality (10). In sampling objects to confirm or disconfirm the generallaw '(if x) (Ax --> Bx)', we want to test the law. This, I take it, means that we want to sample items
200
PATRICK SUPPES
with a higher prior probability of disconfirming the law. This point is made clear by noting
and
-A ) =----pz P ( BI PI + pz peA I B)
= __ fJ2. P:
+ P4
(11)
,
(12)
and selection of an A has a higher prior probability of disconfirming the law than does selection of a non-B just when
P (A I B) < P (B I A) , that is, just when once again
PI < P4' It should be made clear that the adoption of a rational rule for what to observe or sample does not follow from the prior probabilities alone. Some other ingredient must be added, but the rule that tells us to select an A rather than a non-B when PI
THE PARADOXES OF CONFIRMATION
Then
1
201
PCB I A) = -- = 0.9090909 1.1
(13)
_ _ 0.9998989 peA I B) =0.999899- = 0.999999899.
(14)
The probability of (14) is so close to 1 that it seems ridiculous to try to change it by additional observation. This is far less true for the probability of (13). Consequently we sample ravens rather than nonblack things. On the other hand, it would be a mistake to think that P4 will always turn out to be close to 1, and therefore that we should always look at A's rather than non-B's. Consider the following example. Suppose that in a certain election district we want to test the generalization that all voters in this district are literate. The universe X of objects we define to be the adult population of the district. Let V be the subset of voters and L be the subset of literate people of X. A quite reasonable a priori probability for each of the four classes could be something like the following: PI = P2 = P3 = P4 =
P(V &L) = 0.75 P(V &L) = 0.05 P(P &L) = 0.15 P (P & L) = 0.05,
and because P4
202
PATRICK SUPPES
limited, the probability P4 becomes extremely close to 1. But when irrelevant objects like non black thoughts are excluded from sampling by the probability argument given above, the sample space represents a radically reduced population that does not have P4 absurdly close to 1. There are several features of the Bayesian viewpoint that stand in sharp contrast to the position of Carnap and others who work mainly in the framework of inductive logic rather than mathematical statistics. On the one hand, the logicians are scandalized at the vague and subjective character of the prior probabilities used by Bayesians. On the other hand, Bayesians are scandalized at the artificiality and simplistic character of nearly all examples considered by the logicians. I don't pretend to be able to offer arguments that will resolve a conflict of this depth, but I would like to say some things that have perhaps already been said elsewhere but that can perhaps be said in a somewhat new way to defend the Bayesian viewpoint. I would claim to do this without too much bias because in two other articles in this volume I am concerned to criticize what I take to be fundamental limitations of the Bayesian approach to induction and problems of rational information processing. First, the Bayesian is quick to remark that in most systems of inductive logic, the probabilities PI and P4 in the raven example are assigned the same, or nearly the same value, if the example is analyzed from scratch in terms of the classes or properties A and B. (For the Bayesian this pure a priori assignment reflects wanton waste of evidence already available.) Now the inductive logician may reply that if he were seriously pursuing this example, he would begin much further back and introduce a number of fundamental elementary predicates into his language, and use them to express explicitly the known evidence about ravens, A's and B's; or what have you. And this reply leads to the second Bayesian retort. The inductive logicians, to the Bayesian at least, are the heirs to an intellectually distinguished but misguided tradition of logical atomism that begins at least with Hume. The constraints imposed by the atomism of Wittgenstein's Tractatus are weak compared to the assumptions of statistical independence built into the Carnapian measures imposed on state or structure descriptions. It is the virtue of Carnap to have pursued the atomistic tradition to its more complete probabilistic version, for what the approach comes to in terms of specific questions of confirmation or observation can be settled in a way that is not possible for the relatively vague and non-categorical doctrines of Hume's Treatise or Wittgenstein's Tractatus. But for Bayesians it is an approach that is bound to fail for fundamental reasons. First, it is impossible to express in explicit
THE PARADOXES OF CONFIRMATION
203
form all the evidence relevant to even our simplest beliefs. There is no canonical set of elementary propositions to be approached as an ideal for expressing exactly what evidence supports a given belief, whether it be a belief about ravens, gods, electrons or patches of red. Arguments in support of this last assertion are numerous, and it is worth examining the most important ones because of the fundamental nature of this particular issue for any theory of induction or rational behavior. The simple memory of a computer does provide an example of evidence organized in terms of canonical propositions, but this is because all beliefs or propositions for the computer are categorical. Every issue or belief is an utterly black or white affair. There is no place for partial beliefs, tentative evidence or vague but perceptive hunches. And the lack of these characteristics is perhaps the most salient feature of contemporary computers. At the present time partial beliefs or intuitive hunches can probably be analyzed more thoroughly in terms of probability distributions than in any other fashion. It is not unreasonable to say that the propensity for generating Bayesian distributions is the human facility computers most sorely need. However, there is no need to bring computers into the argument. Analysis of ordinary beliefs, with appropriate contrast of Bayesian and Carnapian views, will stand on its own feet. If I ask a person who follows politics with any serious interest, what the Republican chances are of winning the 1968 presidential election, he would probably not hesitate to give some sort of qualitative answer, like "Not so good", "Unlikely", or "Very small in my judgment". And if I go on to ask him for the basis of his estimate of the chances, he will probably go on to offer a packet of heterogeneous facts and the reasons for thinking some of them are particularly significant. All this is very standard in political conversation and also very Bayesian. Most political beliefs are not quite pinned down; the evidence is assembled higgledy-piggledy from all kinds of sources that vary widely in reliability and relevance. Now it is not uncommon for an inductive logician to be willing to admit all this, but he may go on to say that while the consideration of hunches or badly formulated beliefs may be an essential part of the discovery, it is not essential in the validation and assessment of hypotheses. The Bayesian is not content with this thin bone of discovery. He will go on to reply that the vague and subjective prior distribution is of importance primarily in summarizing all the information about the experiment or proposed test which lies outside the narrow framework of the experiment itself, but which is still relevant in varying degrees. The assumption of a prior distribution is a systematic way of summarizing a great deal of heterogeneous information.
204
PA TRICK SUPPES
And here another point arises. The Bayesian is more modest than the inductive logician in what he hopes to express by means of a prior distribution. It is of fundamental importance to any deep appreciation of the Bayesian viewpoint to realize that the particular form of the prior distribution expressing beliefs held before the experiment is conducted is not a crucial matter. If a moderate number of observations is taken in the experiment, the conclusions drawn will be relatively robust, that is, relatively indifferent to moderate variations in the prior distribution; and, the more the number of systematic observations the more robust the conclusion. There is a very general theorem that can be stated here, but I shall not digress to formulate it precisely. It is to the effect that given any two prior distributions drawn from a large class of possible distributions, there is, for a broad class of experiments, a sufficiently large number of observations to bring the two posterior distributions as close together as is desired. For the Bayesian, concerned as he is to deal with the rea! world of ordinary and scientific experience, the existence of a systematic method for reaching agreement is important. To him it is hopeless to strive for an atomistic expression of the total relevant evidence in terms of elementary observation sentences. The welldesigned experiment is one that will swamp divergent prior distributions with the clarity and sharpness of its results, and thereby render insignificant the diversity of prior opinion. The Bayesian does not believe that we can find ways to express these diverse prior opinions in logically tight, explicit form. The task of the theory of rationality, for the Bayesian, is to understand how to conceive and design experiments that will eliminate or reduce diversity of opinion about serious questions, and part of the task of this theory is being clear about puzzling matters like the paradoxes of confirmation. I hope that these more general remarks will have defined more sharply the framework within which I have proposed to resolve the paradoxes. 3. Causal versus noncausallaws. I have some additional specific remarks to make about the paradoxes. From a standard statistical viewpoint the analysis already given goes only part of the way. A well-defined sample space for a well-defined experiment was not constructed for either the raven or voting example. What bothers me is that the construction of the appropriate sample space does not seem at all natural. Tn trying to determine why this is so, what has struck me most is the complete artificiality of the problem. The experimental literature of biology, psychology and to some extent even physics is full of meaningful experiments testing meaningful hypotheses in a statistical
THE PARADOXES OF CONFIRMATION
205
fashion, but essentially none of those hypotheses is of the form "All ravens are black". The main point seems to be that no one applies systematic statistical procedures for making inductive inferences or testing hypotheses when the hypothesis in question asserts a nonprobabilistic implication about a discrete classification. In the case of physics particularly, statistical procedures are used to test deterministic hypotheses, but the hypotheses are about continuous quantities, and statistical questions enter mainly in discussing errors of measurement. The reason for this attitude toward deterministic hypotheses seems clear. The assessment of evidence for or against such a hypothesis is a quite simpleminded affair. A single observation will falsify the hypothesis; all positive instances will confirm it. We have no serious or systematic statistical problem of assessing evidence. LaPlace, I am sure, would have considered the raven sort of problem rather silly, because he thought the apparatus of probability theory was to be applied to the determination of the complex causes of phenomena when no simple or deterministic scheme would work in practice. More can be said on this point, but let us see how the main thrust of these remarks bears on the paradoxes of confirmation moved to a more realistic setting. The paradigmsort of hypothesis is now: Smoking tends to cause cancer. Put in terms of classes A and R, we move from For all x, if x is A then x is B to: or:
P(RIA) > P(RIA)
(15)
P( cancer Ismoking) > P(cancer Inonsmoking).
(16)
The first thing to note is that the obvious form of the paradox of confirmation disappears for in general
P(RIA)
=1=
P(AIB),
i.e., the direct analogue of contraposition is not valid in terms of conditional probability. On the other hand, it reappears in another form, which is innocuous in many applications. We need the usual 2 x 2 contingency table to bring out the point. The distribution of the population (or sample) is shown by the numbers nij' B -------
A nl1
n 12
A
n 22
n21
(17)
206
PATRICK SUPPES
We may use this table to show that (15) holds if and only if
P(AIE) > P(AIB),
(18)
and (18) is a sort of probabilistic contrapositive of (15). Using (17), we have
P(BIA) > P(BIA) nil 111 1
+ 1112
>
11 1 11122 >
11111122
+ 11211122> 1122 11 1Z
+ 112 2
>
if and only if
1121 n21
+ 1122
if and only if
11121121 11121121
+ 11211122
112 1 1111
if and only if
+ 112 1
if and only if if and only if
P(AIE) > P(AIB),
which establishes the desired equivalence. Tn terms of smoking and cancer, we have:
P( cancer Ismoking) > P( cancer Inonsmoking) if and only if P( nonsmoking I noncancer) > P( nonsmoking Icancer), and not only does this seem reasonable, but it also seems reasonable to sample either the causes (smoking) or the effects (cancer) and their absences in establishing a probabilistic causal law. We may sample by looking at smokers and nonsmokers, or by looking at persons with cancer and those without cancer. (For detailed design of an experiment, the question of precisely what class seems a priori most appropriate to sample or, more realistically, in what proportions classes of individuals should be sampled, would follow the same line of analysis pursued earlier in discussing the raven example, and will not be considered in detail again.) However, a subtle point has been illegitimately smuggled in, and the situation changes when we consider something closer to the raven case, i.e., a noncausal law. We may entertain the noncausal probabilistic law: Most ravens are black.
(19)
The natural probability expression of this hypothesis is not the analogue of (15): (20) P(BIR) > P(BJR), but rather (21) P(BIR) > P(EIR),
THE PARADOXES OF CONFIRMATION
207
and without further assumption the apparent "contrapositive" probability analogue of (21) is not necessarily equivalent to it. To be explicit, (21) is not necessarily equivalent to
P(RIB) > P(RIB),
(22)
as may be seen from using table (17) as before, and with this observation, the paradoxes of confirmation vanish for (19). (It may be argued that the bare inequality of (21) does not reflect the exact meaning of most and that a stronger form of inequality should be used, but meeting this criticism is not crucial for the present discussion.) As far as I know, the relevance for the paradoxes of confirmation of the sharp distinction between causal and noncausal laws, particularly the relevance of the different probabilistic forms of such laws, has not been previously noticed. It should be apparent that the kind of causal law pertinent to this discussion is probabilistic rather than deterministic in character, and is of the sort ordinarily tested in biological, medical and psychological experiments and reported in contingency-table data. A certain lack of clarity in the distinction between causal and noncausal laws is also to be found in the terminology used in the statistical literature. Statisticians have developed measures of association for contingency-table data and the probabilistic causal laws tested by the tables. It would seem more natural to reserve the term association for testing the noncausal laws, but such tests are not ordinarily discussed in the same detailed fashion, undoubtedly because of the greater importance of causal laws from both a practical and conceptual standpoint. I do not mean to suggest that inequality (15) offers a very profound analysis of the probabilistic notion of cause. My limited objective in this paper has been to point out the conceptually sharp distinction between causal and noncausallaws when they are expressed in a probabilistic form. The ideas used here go no deeper than what I would call the level of naive causes. The identification of genuine causes, which to me seems necessarily relative to a particular conceptual scheme, requires a more elaborate probabilistic structure than I have introduced here. But the introduction of additional structure would not change what I have said about the non-existence of the paradoxes of confirmation for either causal or noncausallaws of a probabilistic sort. References HOSIASSON-LINDENBAUM, 1940, On confirmation, J. of Symbolic Logic, vol. 5, pp.133-l48
THE PARADOXES OF CONFIRMATION* G. H. VON WRIGHT The Academy of Finland, Helsinki, Finland
1. We consider generalizations of the form "All A are B". An example could be "All ravens are black". We divide the things, of which A (e.g. ravenhood) and B (e.g. blackness) can be significantly (meaningfully) predicated into four mutually exclusive and jointly exhaustive classes. The first consists of all things which are A and B. The second consists of all things which are A but not B. The third consists of all things which are B but not A. The fourth, finally, consists of all things which are neither A nor B. Things of the second category or class, and such things only, afford disconfirming (falsifying) instances of the generalization that all A are B. Since things of the first and third and fourth category do not afford disconfirming instances one may, on that ground alone, say that they afford confirming instances of the generalization. If we accept this definition of the notion of a confirming instance, it follows that any thing which is not A ipso facto affords a confirming instance of the generalization that all A are B. This would entail, for example, that a table, since notoriously it is not a raven, affords a confirmation of the generalization that all ravens are black. A consequence like this may strike one as highly "paradoxical". it may now be thought that a way of avoiding the paradox would be to give to the notion of a confirming instance a more restricted definition. One suggestion would be that only things of the first of the four categories, i.e. only things which are both A and B, afford confirmations of the generalization that all A are B. This definition of the notion of a confirming instance is sometimes referred to under the name "Nicod's Criterion". According to this criterion, only propositions to the effect that a certain thing is a raven and is
* The treatment of the Paradoxes of Confirmation which is suggested in this paper is
substantially the same as the one given in my essay, in Theoria, vol. 31 (1965), pp. 254-274. The non-formal parts of the discussion in the two papers are largely identical. The formal argument, as presented here, is more condensed and also, I hope, more perspicuous than in the Theoria paper.
THE PARADOXES OF CONFIRMATION
209
black can rightly be said to confirm the generalization that all ravens are black. But if we adopt Nicod's Criterion as our definition of the notion of a confirming instance we at once run into a new difficulty. Consider the generalization that all not-B are not-A. According to the proposed criterion we should have to say that only things which are not-B and not-A afford confirmations of this generalization. The things which are not-B and not-A are the things of the fourth of the four categories which we distinguished above. But, it is argued, the generalization that all A are B is the same as the generalization that all not-B are not-A. To say "all A are B" and to say "all not-B are not-A" appear to be but two ways of saying the same thing. It is highly reasonable, not to say absolutely necessary, to think that what constitutes a confirming or disconfirming instance of a generalization should be independent of the way the generalization is formulated, expressed in words. Thus any thing which affords a confirmation or disconfirmation of the generalization g must also afford a confirmation and disconfirmation respectively of the generalization h, if "g" and "h" are logically equivalent expressions. This requirement on the notion of a confirming instance is usually called "The Equivalence Condition". To accept Nicod's Criterion thus seems to lead to conflict with the Equivalence Condition. This conflict constitutes another Paradox of Confirmation. 2. Before we proceed to a "treatment" of the paradoxes which we have mentioned, the following question must be asked and answered: Are confirmations of the generalization that all A are B through things which are not-A always and necessarily to be labelled "paradoxical", and never "genuine"? Simple considerations will show, I think, that the answer is negative. Let us imagine a box or urn which contains a huge number of balls (spheres) and of cubes, but no other objects. Let us further think that every object in the urn is either black or white (all over). We put our hand in the urn and draw an object "at random". We note whether the drawn object is a ball or a cube and whether it is black or white. We repeat this procedurewithout replacing the drawn objects - a number of times. We find that some of the cubes which we have drawn are black and some white. But all the balls which we have drawn are, let us assume, black. We now frame the generalization or hypothesis that all spherical objects in the box are black. In order to confirm or refute it we continue our drawings. The drawn object would disconfirm (refute) the generalization if it turned out to be a white ball. If it is a black ball or a white cube or a black
210
G. H. VON WRIGHT
cube, it confirms the generalization. Is any of these types of confirming instance to be pronounced worthless? It seems to me "intuitively" clear that all the three types of confirming instance are of value here and that no type of confirmation is not a "genuine" but only a "paradoxical" confirmation. (Whether confirmations of all three types are of equal value for the purpose of confirming the generalization may, however, be debated.) I would support this opinion by the following ("primitive") argument: What we are anxious to establish in this case is that no object in the box is white and spherical. Not knowing, whether there are or are not any white balls in the box, we run a risk each time when we draw an object from the box of drawing an object of the fatal sort, i.e, a white ball. Each time when the risk is successfully stood, we have been "lucky". We have been this, if the object which our hand happened to touch was a cube (and, since we could feel it was a cube, need not be examined for colour at all); and we have been lucky, if the object was a ball which upon examination was found to be black. To touch a ball, one might say, is exciting, since our tension (fear of finding a white ball) is not removed until we have examined its colour. To touch a cube is not exciting at all, since it ipso facto removes the tension we might have felt. But to draw from the box is in any case exciting, since we do not know beforehand, whether we shall, to our relief, touch a cube, or touch a ball and, to our relief, find that it is black, or touch a ball and, to our disappointment, find that it is white. Let "S" be short for "spherical object in the box", "C" for "cubical object in the box", "B" for "black", and" W" for "white". All things in the box can be divided into the four mutually exclusive and jointly exhaustive categories of things which are Sand B, Sand W, C and B, and C and W. It is not connected with any air of paradoxality to regard things of all the four types as relevant (positively or negatively) to the generalization that all S are B. All things in the world can be divided into the four mutually exclusive and jointly exhaustive categories of things which are Sand B, S but not B, B but not S, and neither S nor B. Things of the first category obviously bear positively and things of the second category negatively on the generalization. But of the things of the third and fourth category some, we "intuitively" feel, do not bear at all on the generalization, have nothing to do with its content - and therefore "confirm" it only in a "paradoxical" sense. The categories of things C & Band S & W differ from the categories of things ~ S & B and ~ S & ~ B in this feature: All things of the first two cate-
THE PARADOXES OF CONFIRMATION
211
gories are things in the box, but some things (in fact the overwhelming majority of things) of the last two categories are things outside the box. The things which we "intuitively" regard as affording "paradoxical" confirmations of the generalization that all S are B are those things of the 3rd and 4th category which are not things in the box. I shall here introduce the term range ofrelevance of a generalization. And I shall say that the range of relevance of our generalization above that all spherical things in the box are black is the class of all things in the box. I now put forward the following thesis: All things in the range of relevance of a generalization may constitute genuine confirmations or disconfirmations of the generalization. The things outside the range are irrelevant to the generalization. They cannot confirm it genuinely. Since, however, they do not disconfirm it either, we may "by courtesy" say that they confirm it, though only "paradoxically". In order to vindicate my thesis I shall try to show, by means of a formal argument, that the irrelevance of the "paradoxical" confirmations consists in the fact that they are unable to affect the probability of the generalization. Showing this is one way, and a rather good one it seems to me, of dispelling the air of paradoxality attaching to these confirmations. 3. It is important to state explicitly the logico-mathematical frame of probability within which we are going to conduct our formal argument concerning the confirmation paradoxes. The probability concept of the confirmation theories of Carnap and Hintikka is a two-place functor which takes propositions (or, on an alternative conception, sentences) as its arguments. The probability concept used by us is a functor the arguments of which are characteristics (attributes, properties). Let "qJ" and "1/1" stand for arbitrary characteristics of the same logical type (order). The expression "P(cp/l/J)" may be read "the probability that a random individual is tp, given that it is l/J". Instead of "is" we can also say "has the characteristic", and instead of "given" we can say "on the datum" or "relative to". We stipulate axiomatically that, for any pair of characteristics which are of the same logical type and such that the second member of the pair is not empty, the functor "P( I )" has a unique, non-negative numerical value. Furthermore, the functor obeys the following three axioms: AI. (Ex)l/Jx &(x)(l/JX-4cpx)-4P(cp!l/J) = 1, A2. (Ex)l/Jx-4P(cpll/J)+P( --cpll/J)= 1, A3. (Ex)(xx &cpx)-4P(cp!X)' P(l/Jlx &cp)=P(cp&l/Jlx)·
212
G. H. VON WRIGHT
It is a rule of inference of the calculus that logically equivalent (names of) characteristics are intersubstitutable in the functor "P( I )" ("Principle of Extensionality"). The application of probabilities, which are primarily associated with characteristics, to individuals is connected with notorious difficulties. The application is sometimes even said to be meaningless. This, however, is an unnecessarily restricted view of the matter. If x is an individual in the range of significance of cp and t/J, and if it is true that P( cp!t/J) =p, then we may, in a secondary sense, say that, as a bearer of the characteristic t/J, the individual x has a probability p of being a bearer also of the characteristic ip,
4. If R is the range of relevance of the generalization that all A are B, and if this generalization holds true in that range, then it will also be true that (x)( Rx-» (Ax~ Bx)). - This may be regarded as a "partial definition" of the notion of a range of relevance. For the sake of convenience, I shall introduce the abbreviation" Fx" for "Ax-s Bx", "F", we can also say, denotes the property which a thing has by virtue of the fact that it satisfies the propositional function "Ax~ Bx", [ define a second-order property q/ R by laying down the following truthcondition: The (first-order) property X has the (second-order) property 1ftR, if and only if, it is universally implied by the (first-order) property R. That X is universally implied by R means that it is true that (x)(Rx~Xx). The property w R, in other words, is the property which a property has by virtue of belonging to all things in the range R. A property which belongs to all things in a range can also be said to be universal in that range. Assume we can order all things of which A, Band R can be significantly predicated into a sequence XI' x 2 , ••• , x n , •••• Then we can define a sequence of second-order properties .9"'\, :F 2 , ..• , .9'" n- ... as follows: The (first-order) property X has the (second-order) property :Fn , if and only if, it is true that RXn~Xxn' The property :Fn, in other words, is the property which a property has (solely) by virtue of belonging to a certain individual thing, if this thing is in the range R. ("If" here means material implication.) For the sake of convenience, I introduce the abbreviation "l[Jn" for the logical product of the first n properties in the sequence :F\,:F 2, ... , :Fn , ." • "1[Jn", we can also say, denotes the property which a property has by virtue of the fact that it is not missing from any of those of the first n things in the world which also are things in the range R.
THE PARADOXES OF CONFIRMATION
213
Finally, let "e" denote a tautological second-order property, i.e. a property which any first order property tautologically possesses - for example, the property of either having or not having the second-order property qt R (or ff n)· 5. We prove the following theorem of probability: T. P(o/IRIO»O ~(P( 'PiRliPn+ 1)> P( 'PiRliPn)~P(ffn+ lliP n ) < 1). The first-order property R trivially has the second-order property 0 & 4>no For, that OCR) may become equated with the tautology that epn(R) v ~ iPn(R), and "epn(R)" is an abbreviation for "(Rx 1~RX1)&'" &(Rxn~RxS'. Consequently, it is logically true (for all values of n) that (EX)(O(X)&iPn(X». It follows immediately that it is logically true, too, that (EX)O(X) and (EX)epn(X). From A3 we derive, by substitution and detachment, that P( epn & o/tRIO) =P(iPnIO)' P('PiRIO &iPn). "epn&'Pi/' is logically equivalent with "ollR" alone. This follows from the way the second-order properties were defined. That o/tR(X) means that (x)(Rx~Xx) and that iPn(X) means that (Rx 1~XX1)&'" &(Rxn~Xxn)' Similarly, "0 & epn" is logically equivalent with "4>n" alone. Substituting the simpler equivalents, the equality above reduces to that P( 'PiRIO) =P(epnIO)· P( 'PiRliP n). By an exactly analogous argument we derive the equality that P(qtRIO) =P(iPn+ 110)· P( 'PiRliPn+ 1)' Combining the two equalities we get thatP(iPnIO)'P(o/tRlepn)=P(epn+lI0)' .P('PiRlepn+ 1)' Now assume that P(°11 RIO) > O. Since probabilities are non-negative, it follows that P( 'PiRliPn+ 1)> P( 0/1Rlepn)~P( epnl O) > P( epn+ 110). By repeated application of A3 we detach the equalities that P( epnlO) =P(ff 110) · ... ·P(ffnI0&4>n_l) and that P(epn+lle)=p('~110)"'" P(ffnI0&iPn-l)·P(ffn+lI0&4>n). The assumption that PCOllRIO»O guarantees that all the factors of the products are different from O. Hence, after cancellation, we get that P(iP nI8)P('PiRliPn)~P(ffn+llepn)< 1. This completes the proof of T. Let us now see what this theorem amounts to in plain words. "P('PiRIO»O" says that the probability that a random property in the universe of properties is true of all things in the range R is greater than O. "P('PiRlepn+ l»P('PiRlepS' says that the probability that a random property is true of all things in the range R, is greater, given that it is true of
214
G. H. VON WRIGHT
those of the first n + 1 things in the world which fall in this range, than given (only) that it is true of those of the first n things which fall in this range. "P(J n + IlcP n ) < 1", finally, says that the probability that a random property is true of the (n + I yt thing in the world, if this thing belongs to the range R, is smaller than 1, given that this property is true of those of the first n things in the world which fall in that range. The theorem as a whole thus says the following: lfthe probability that a random property in the universe of properties is true of all things in the range R is not minimal (0), then the probability that this property is true of all things in the range is greater, given that it is true of those of the first n + 1 things which fall in the range, than given (only) that it is true of those of the first n things which fall in the range, if and only if, the probability that it is true of the(n+ 1)slthing, if this belongs to the range, is not maximal (1), given that it is true of those of the first n things which belong to the range R. Now apply the theorem to the individual property F. To say that F is true of all things in the range R is tantamount to saying that the generalization that all A are B is true in the range R. To say that F is true of those of the first n (or n+ 1) things in the world which are also things in the range amounts to saying that the first n (or n + 1) things afford confirming instances of the generalization that all A are B in the range R. To say that F is true of the (n + I yl thing, if this thing belongs to the range, finally, comes to saying that this thing affords a confirming instance of the generalization that all A are B in the range R. When applied to the individual property F, the theorem as a whole thus says the following: If, on tautological data ("a priori"), the probability that all A are B in the range R is not minimal, then the probability of this generalization is greater on the datum that the first n + I things in the world afford confirming instances of it than on the datum that the first n things afford confirming instances, if and only if, the probability that the (n + l.)" thing affords a confirming instance is not maximal on the datum that the first n things afford confirming instances. It follows by contraposition that, if this last probability is maximal (l), then the new confirmation of the generalization in the (n+ It instance does not increase its probability. The new confirmation is, in this sense, irrelevant to the generalization. 6. Now assume that the thing x n + 1 actually does not belong to the range
THE PARADOXES OF CONFIRMATION
215
of relevance R of the generalization that all A are B. In other words, assume that ",Rx n+ 1 . It is a truth of logic (tautology) that "'RXn+1~(Rxn+1~Exn+1)'Since "E" does not occur in the first antecedent, we can generalize the first consequent in "E". It is a truth oflogic, too, that", Rxn+ 1 ~(X)(Rxn+ 1 ~XXn+ 1)' By definition, ff n+1(X) can replace RXn+1~XXn+1' Thus it is a truth of logic that", Rxn+ 1 ~(X)ffn+ 1 (X). From this it follows trivially that", Rxn+ 1-+ (X)( 4in(X)-+'~n+1 (X)). According to axiom Al of probability (X) ( 4in (X ) -+ff n+ 1 (X)) entails that P(ffn+ 114in) = 1 - provided that at least one property has the (second-order) property 4in- The existential condition is satisfied, since the property R trivially has the property 4in • 4in(R) means by definition the same as (RX1 ~RX1) & ... &(Rxn-+Rx n) which is tautology. Herewith has been proved that, if it is the case that", Rxn+ l' i.e. if the (n + l)'t thing in the world does not belong to the range R, then it is also the case thatP(ffn+ ll4i n)=I, i.e. then the probability that this thing will afford a confirmation of any generalization to the effect that something or other is true of all things in this range, is maximal. This probability being maximal, the confirmation which is trivially afforded by the thing in question is irrelevant to any such generalization in the sense that it cannot contribute to an increase in its probability. And this constitutes a good ground for saying that a thing which falls outside the range of relevance of a generalization can be said to afford only a "vacuous" or "spurious" or "paradoxical", and not a "genuine", confirmation of the generalization in question. 7. After all these formal considerations we are in a position to answer such questions as this: Is it possible to confirm genuinely the generalization that all ravens are black through the observation, e.g., of black shoes or white swans? The answer is that this is possible or not, depending upon which is the range of relevance of the generalization, upon what the generalization "is about". If, say, shoes are not within the range of relevance of the generalization that all ravens are black, then shoes cannot afford genuine confirmations of this generalization. This is so, because no truth about shoes can then affect the probability of the generalization that, in the range ofrelevance in question, all things which are ravens are black. So what is then the range of relevance of the generalization that all ravens are black? Here it should be noted that it is not clear by itself which is the range of relevance of a given generalization such as, e.g., that all ravens are black. Therefore it is not clear either which things will afford genuine and
216
G. H. VON WRIGHT
which only paradoxical confirmations. In order to tell this we shall have to specify the range. Different specifications of the range lead to so many different generalizations, one could say. The generalization that all ravens are black is a different generalization, when it is about ravens and ravens only, and when it is about birds and birds only, and when it is - ifit ever is - about all things in the world unrestrictedly. As a generalization about ravens, only ravens are relevant to it, and not, e.g., swans. As a generalization about birds, swans are relevant to it, but not, e.g., shoes. And as a generalization about all things, all things are relevant - and this means: of no thing can it then be proved that the confirmation which it affords is maximally probable relative to the bulk of previous confirmations and therefore incapable of increasing the probability of the generalization. When the range of relevance of a generalization of the type that all A are B is not specified, then the range is, I think, usually understood to be the class of things which fall under the antecedent term A. The generalization that all ravens are black, range being unspecified, would normally be understood to be a generalization about ravens - and not about birds or about animals or about everything there is. I shall call the class of things which are A the natural range ofrelevance of the generalization that all A are B. It would be a mistake to think, when the range of relevance of a generalization is unspecified, it must be identified with the natural range. If it strikes one as odd or unplausible to regard the genus bird, rather than the species raven, as the range of relevance of the generalization that all ravens are black, this is probably due to the fact that the identification of birds as belonging to this or that species is comparatively easy. But imagine the case that species of birds were in fact very difficult to distinguish, that it would require careful examination to determine whether an individual bird was a raven or a swan or an eagle. Then the generalization that all birds which are (upon examination turned out to be) ravens are black might be an interesting hypothesis about hirds. Perhaps we can imagine circumstances too under which all things, blankets and shoes and what not, would be considered relevant to the generalization that all ravens are black. But these circumstances would be rather extraordinary. (We should have to think of ourselves as beings who quasi put their hands into the universe and draw an object at random.) Only in rare cases, if ever, do we therefore intuitively identify the unspecified range with the whole logical universe of things. It would also be a mistake to think that the range of a generalization must become specified at all. But even when the range is left unspecified we may
THE PARADOXES OF CONFIRMATION
217
have a rough notion of what belongs to it and what does not - and therefore also a rough idea about which things are relevant to testing (confirming or disconfirming) the generalization. No ornithologist would ever dream of examining shoes in order to test the hypothesis that all ravens are black. But he may think it necessary to examine some birds which look very like ravens, although they turn out actually to belong to some other species. 8. In conclusion I shall say a few words about the alleged conflict between the so-called Nicod Criterion and the Equivalence Condition (cf. above, section 1). The Nicod Criterion, when applied to the generalization that all A are B, says that only things which are both A and B afford genuine confirmations of the generalization. Assume now that the range of relevance of the generalization in question is A, i.e. assume that we are considering this generalization relative to what we have here called its natural range. Then, by virtue of what we have proved (sections 4-6), anything which is not-A cannot afford a genuine confirmation of the generalization. In other words: Within the natural range ofrelevance ofa generalization, the class ofgenuinely confirming instances is determined by Nicod's Criterion. But is this not in conflict with the Equivalence Condition? This condition, as will be remembered, says that what shall count as a confirming (or disconfirming) instance of a generalization cannot depend upon any particular way of formulating the generalization (of a number of logically equivalent formulations). Do we wish to deny then that the generalization that all A are B is the same generalization as that all not- Bare not-A? We do not wish to deny that "all A are B" as a generalization about things which are A expresses the very same proposition as "all not-B are not-A" as a generalization about things which are A. Generally speaking: when taken relative to the same range of relevance, the generalization that all A are B and the generalization that all not-B are not-A are the same generalization. But the generalization that all A are B with range of relevance A is a different generalization from the one that all not-B are not-A with range of relevance not-B. If we agree that, range of relevance not being specified, a generalization is normally taken relative to its "natural range", then we should also have to agree that, the ranges not being specified, the forms of words "all A are B" and "all not-B are not-A" normally express different generalizations. The generalizations are different, because their "natural" ranges of relevance are different. This agrees, I believe, with how we naturally tend to understand the two formulations.
218
G. H. VON WRIGHT
Speaking in terms of ravens: The generalization that all ravens are black as a generalization about ravens, is different from the generalization that all things which are not black are things which are not ravens as a generalization about all not-black things. But the generalization that all ravens are black as a generalization about, say, birds is the very same as the generalization that all things which are not black are not ravens as a generalization about birds. (For then "thing which is not black" means "bird which is not black".) Within its natural range of relevance, the generalization that all A are B can become genuinely confirmed only through things which are both A and B and is "paradoxically" confirmed through things which are B but not A, or neither A nor B. Within its natural range of relevance the generalization that all not-B are not-A can become genuinely confirmed only through things which are neither A nor B and is "paradoxically" confirmed through things which are both A and B, or B but not A. Within the natural range of relevance, Nicod's Criterion of confirmation is necessary and sufficient. Within another specified range of relevance R, the generalization that all A are B may become genuinely confirmed also through things which are B but not A, or neither A nor B. And within the same range of relevance R, the class of things which afford genuine confirmations of the generalization that all A are B is identical with the class of things which afford genuine confirmations of the generalization that all not-B are not-A. Thus, in particular, if the range of relevance of both generalizations are all things whatsoever, i.e. the whole logical universe of things of which A and B can be significantly predicated, then everything which affords a confirming instance of the one generalization also affords a confirming instance of the other generalization, and vice versa, all confirmations being "genuine" and none "paradoxical".
ASSIGNING PROBABILITIES TO LOGICAL FORMULAS DANA SCOTT Stanford University, Stanford, Calif," and PETER KRAUSS University of California, Berkeley, Calif:
1. Introduction. Probability concepts nowadays are usually presented in the standard framework of the Kolmogorov axioms. A sample space is given together with a a-field of subsets - the events - and a a-additive probability measure defined on this a-field. When the study turns to such topics as stochastic processes, however, the sample space all but disappears from view. Everyone says "consider the probability that X2 0", where X is a random variable, and only the pedant insists on replacing this phrase by "consider the measure ofthe set {WEQ:X(W)20}". Indeed, when a process is specified, only the distribution is of interest, not a particular underlying sample space. In other words, practice shows that it is more natural in many situations to assign probabilities to statements rather than sets. Now it may be mathematically useful to translate everything into a set-theoretical formulation, but the step is not always necessary or even helpful. In this paper we wish to investigate how probabilities behave on statements, where to be definite we take the word "statement" to mean "formula of a suitable formalized logical calculus". It would be fair to say that our position is midway between that of Carnap and that of Kolmogorov. In fact, we hope that this investigation can eventually make clear the relationships between the two approaches. The study is not at all complete, however. For example, Carnap wishes to emphasize the notion of the degree of confirmation which is like a conditional probability function. Unfortunately the mathematical theory of general conditional probabilities is not yet in a very good state. We hope in future papers to comment on this problem. Another question concerns the formulation of
* This work was partially supported by grants from the National Science Foundation and the Sloan Foundation.
220
DANA SCOTT AND PETER KRAUSS
interesting problems. So many current probability theorems involve expectations and limits that it is not really clear whether consideration of probabilities of formulas alone really goes to the heart of the subject. We do make one important step in this direction, however, by having our probabilities defined on infinitary formulas involving countable conjunctions and disjunctions. In other words, our theory is o-additive. The main task we have set ourselves in this paper is to carryover the standard concepts from ordinary logic to what might be called probability logic. Indeed ordinary logic is a special case: the assignment of truth values to formulas can be viewed as assigning probabilities that are either 0 (for false) or 1 (for true). Tn carrying out this program, we were directly inspired by the work of Gaifman [1964] who developed the theory for finitary formulas. Aside from extending Gaifman's work to the infinitary language, we have simplified certain of his proofs making use of a suggestion of C. RyllNardzewski. Further we have introduced a notion of a probability theory, in analogy with theories formalized in ordinary logic, which we think deserves further study. In Section 2 the logical languages are introduced along with certain syntactical notions. In Section 3 we define probability systems which generalize relational systems as pointed out by Gaifman. In Section 4 we show how given a probability system the probabilities of arbitrary formulas are determined. Tn Section 5 we discuss model-theoretic constructs involving probability systems. In Section 6 the notion of a probability assertion is defined which leads to the generalization of the notion of a theory to probability logic. In Section 7 we specialize and strengthen results for the case of finitary formulas. In Section 8 examples are given. An appendix (by Peter Krauss) is devoted to the mathematical details of a proof of a measure-theoretic lemma needed in the body of the paper. 2. The languages of probability logic. Throughout this paper we will consider two different first-order languages, a finitary language !E(w) and an infinitary language !E. To simplify the presentation both languages have an identity symbol = and just one non-logical constant, a binary predicate R. Most definitions and results carryover with rather obvious modifications to the corresponding languages with other non-logical constants, and we will occasionally make use of this observation when we give specific examples. The language !E(w) has a denumerable supply of distinct individual variables Vn' for each n < W, and !E has distinct individual variables v~, for each ~ <WI' where WI is the first uncountable ordinal. Both languages have logical
ASSIGNING PROBABILITIES TO LOGICAL FORMULAS
221
constants r., v, ---', V, 3, and = standing for (finite) conjunction, disjunction, negation, universal and existential quantification, and identity as mentioned before. In addition the infinitary language se has logical constants 1\ and V standing for denumerable conjunction and disjunction respectively. The expressions of se are defined as transfinite concatenations of symbols oflength less than W 1, and the formulas of secw) and se are built from atomic formulas of the forms Rv~v~ and v~=v~ in the normal way by means of the sentential connectives and the quantifiers. Free and bound occurrences of variables in formulas are defined in the well-known way. (For a more explicit description of infinitary languages see the monograph Karp [1964].) A sentence is a formula without free variables. We will augment the non-logical vocabulary of our languages with various sets T of new individual constants tET and denote the resulting languages by secw)(T) and seCT) respectively. It is then clear what the formulas and sentences of secW)(T) and seCT) are. For any set T of new individual constants let Y and yeT) be the set of sentences of se and seCT) respectively, and let oCT) be the set of quantifier-free sentences of seCT). We adopt analogous definitions for the language secw). If L is a set of sentences and cp is a sentence, then cp is a consequence of L if cp holds in all models in which all sentences of L hold, and we write L 1= cpo cp is valid if it is a consequence of the empty set, and we write 1= tp, For both languages 2 Cw) and :e we choose standard systems of deduction, and we write L f- cp if cp is derivable from L. cp is a theorem if cp is derivable from the empty set, and we write f- tp, (For details concerning the infinitary language we again refer the reader to Karp [1964].) By the well-known Completeness Theorem of finitary first order logic we have for every L <;;ycw) and every cpE.
222
DANA SCOTT AND PETER KRAUSS
two sentences
3. Probability systems. We start with the definition of a concept which corresponds to the notion of a relational system in ordinary logic. Recall that if .d is a Boolean algebra then a probability on .9/ is a o-additive probability measure on .#. A finitely additive probability on .d is a finitely additive probability measure on .#. For a detailed discussion of these concepts see Halmos [1963] and Sikorski [1964]. DEFINITION. A probability system (or sometimes, a probability model) is a quintuple
<.4.
If m=c
m
ASSIGNING PROBABILITIES TO LOGICAL FORMULAS
223
model. Most concrete examples of probability systems we have encountered have strict identity. However some intuitively very suggestive model constructions, such as the ultraproduct construction and symmetric probability systems which will be discussed in Section 5, lead beyond the realm of probability systems with strict identity. For this reason we thought it advisable to introduce the more general notion of a probability system. If 1ll=
a ~ b iff Id(a,b) = 1. The substitution property of Id implies that
~
is a congruence relation on
Ill. The cardinality of Ill, denoted by IIll/ is defined to be the cardinality of the
set of equivalence classes with respect to ~. Ifill is a probability system with strict identity then IIIII is the cardinality of the set A. More generally, for any subset A' c;; A the system-cardinality of A, denoted by IA'I~I' is defined to be the cardinality of the set of equivalence classes with respect to ~ which have a non-empty intersection with A'. 4. Probability interpretations. We now interpret the language !l' in probability systems and give a definition of the concept "a sentence cp holds in a probability system III with probability a", where O~ a~ 1 is a real number. The definition could be given in the traditional way using an analogue of Tarski's concept of satisfaction; however, in the context of probability logic it seems to be more appropriate to use the equally well-known device of new individual constants. Let 1ll=
=
1 ~ h(cp),
(iv) h(V CPi) = V h(CPi)' i<~
i<~
i<~
i<~
(v) h(/\ cp;) = /\ h(cp;), (vi) h(3vcp) = V h(cp(ta) )
,
aeA
(vii) h (V vcp) = /\ h (cp (t a) ) aeA
.
The following lemma is well-known and easy to prove:
224
DANA SCOTT AND PETER KRAUSS
4.1. (i) For all tp, l/JE,C/'(T'll)' ijf-CP+-+l/J, then h(cp)=h(l/J); (ii) h induces a a-homomorphism from .C/'(T~()/f- into d. Proof: By the definition of h it suffices to prove that for all cpE9'(T'l1)' if cpf- then h(cp)=l. This can be done by considering a standard system of deduction for 2'(T~(), and showing that h maps all axioms into 1 and that the property of being mapped into 1 is preserved under all rules of deduction. For a more detailed presentation see Karp [1964]. (ii) is an immediate conseq uence of (i). LEMMA
We identify h with the induced homomorphism and define Jl'll(cp/f-)= m(h(cp)) for all cpE.C/'(T~a. Then Jl~( is a probability on 9'(T'll)/f-. (This is a well-known fact in measure theory, and a proof can be found in Halmos [1963] p. 66.) Since hardly any confusion could arise we write Jl21(CP) for Jl~(cp/f-), and we read "Jl~(cp)=a" as "cp holds in the probability system m: wi th probability a". If m: is a probability system with strict identity and m is two valued, then for every CPE2'(TtI)' Jlm(cp)=1 iff cp holds in the model
Ilm(3vcp)= sup 1l21(V cp(tJ);
(G)
FeA«(J)
aeF
where A(w) is the set of all finite subsets of A. Proof: ,u~,(3vcp)=m(h(3vcp))=m( V h(cp(ta))). .,,1 is a measure algebra and aEA
therefore satisfies the countable chain condition (see Halmos [1963] p. 67). Thus there exists a countable subset A'c;;A such that
V h«(p(t a )) = V h(cp(t a ) )
Thus Since
aeA
aeA'
1l~1(3vcp) = m( V h(cp(t a ))) aeA'
J1~1
is a-additive,
J1~I( V (p(t a ) ) (leA'
.
= m(h( V cp(ta ))) = JIm ( V cp(ta) ) . {leA'
= sup
FeA'(UJ)
aeA'
,u~I( V cp(ta))· aeF
Condition (G) now follows from the choice of A'. We now present the preceding ideas from a slightly different point of view
ASSIGNING PROBABILITIES TO LOGICAL FORMULAS
225
to see that the probability system ~ may be identified with the restriction of the probability Il~( to {}(~()/f-. We first observe that from the definition of h and the countable chain condition in d it follows that for the purpose of our probability interpretation we may assume that d is the a-algebra generated by the union of the images of A x A under Id and R respectively. If the definition of h is restricted to clauses (i)-(v), then obviously Lemma 4.1 holds with ,5f'(Tm) replaced by {}(Tm). Since m is strictly positive on ,# we have Thus the quotient algebra of {}(Tm)jf- modulo the a-ideal {q>jf-Eo(Tm)jf-: J1m(q»=O} is isomorphic to d, and it is a well-known fact that the probability m on d may be uniquely recaptured from the probability Il~( on o(Tm)jf-. (See, e.g., Halmos [1963] pp. 64ff.) Thus the probability system ~ is, up to the obvious isomorphism, determined by the ordered pair (Tm, 11m), where Il'll is restricted to {}(T~l)jf-. In general any ordered pair (T, Il), where T is a set of new individual constants and Il is a probability on {}(T)jf-, uniquely determines a probability system ~. Indeed let A=T, let d be the quotient algebra of;;(T)jf- modulo the a-ideal {q>jf-:q>E{}(T), 1l(q»=O}, let m be the probability on d induced by u, and let Id(t, t') and R(t, 1') be the image of t= t'jf- and R(t, t')jf- under the canonical homomorphism of;;(T)jf- onto d. Then ~=(A, R, Id, .9/, m) clearly is a probability system; it is easy to check that the valuation homomorphism h is the canonical homomorphism, and Il is the restriction of Ilm to {}(T)jf-. Moreover, if ll(t=t')=O for all t, t'ETwhere Ii:.t', then ~ has strict identity. Thus we may also regard a probability system as an ordered pair (T, m), where T is a set of new individual constants, and m is a probability on {}(T)jf-. The probability systems with strict identity are then characterized by the condition m(t=t')=O for all t, t'ETwhere ti=t'. This is the form in which Gaifman [1964] introduces the concept of a probability model and, whenever convenient, we will also adopt this terminology. From this new point of view we have the following extension theorem: 4.3. Let (T, m) be a probability system. Then there exists a unique
THEOREM
probability m* on ,5f'( T)jf- which extends m and satisfies the Gaifman Condition: whenever 3Vq>EY' (T), then
(G)
m*(3 Vip) =
sup FET(W)
m*( V
where T(w) is the set of all finite subsets of T.
tEF
q>(t));
226
DANA SCOTT AND PETER KRAUSS
Proof: The existence of m* is clear from our considerations above. The uniqueness of the extension will be proved by transfinite induction. During the course of our proof we will make use of analogues of Lemma 7.9 which will be established separately, and of course independently, for the finitary language ::t'lwj (T) in Section 7 of this paper. For every ordinal ~ < WI' we shall define sets of sentences j~(T) S; Y' ~(T) S; YeT) by recursion: First let Jo(T)=J(T). Then if ~>O, let j~(T) be the closure of U .'/'~(T) under denumerable propositional combinations. For t1<~
every ~ < W j , let Y~(T) be the closure of ,)~(T) under quantification and finite propositional combinations. Then obviously whenever IJ < ~ < WI' and
Y'(T)=
U
';<(1)1
.J~(T)=
U
.'/'~(T).
~<(J)l
Now suppose III and I1 z are both o-additive probability measures on (/'(T)jf- which extend m and satisfy condition (G). We shall prove by transfinite induction that for every ~<Wl and every O, first observe that (U Y'~(T))jf- is a subalgebra '1<~
of Y(T)jf- that c-generates ,)~(T)jf-. By way of an induction hypothesis, we assume that 111 (
o-additive measures, we conclude by a well-known extension theorem of . measure theory (see Halmos [1950] p. 54), that 111 (
ASSIGNING PROBABILITIES TO LOGICAL FORMULAS
227
paper, which renders the existence part of Theorem 4.3 almost trivial, was suggested to us by Professor C. Ryll-Nardzewski. It is now clear how to define various probability-model-theoretic concepts in analogy to the standard concepts of ordinary model theory. We will discuss a few examples in the next section. 5. Model-theoretic concepts in the theory of probability systems. If
<
Remark: The concepts defined in (i), (ii) and (iii) correspond to the concepts of subsystem, !I!-subsystem and !I!-equivalence respectively in ordinary model theory. Not many interesting results concerning these concepts are known for the in finitary language !I!, a phenomenon which the probability-model theory of !I! seems to share with the ordinary model theory of !I!. In many cases the authors have been able to establish for probability logic analogies of major results known from ordinary logic; this is particularly true for the finitary language !I!(wl, for which several results have already been published by Gaifman [1964]. We present next a few standard constructions for probability systems. Independent Unions. Let I be an index set. For each iEI, let !l!i be the infinitary language whose only non-logical constant is the binary predicate R;, and let .'1"; be its set of sentences. Let T be a set of new individual constants. For each iEI, let
m=
iel
TI m, is the product measure on)(T)/f- induced by the family {mi : iEI}.
i€I
228
DANA SCOTT AND PETER KRAUSS
Then we define the independent union of the family of probability systems {
We note two corollaries of the construction given with this definition: 5.1. For every iEI and qJEY'i(T), m*(qJ)=mi(qJ). Proof: We argue by transfinite induction along the same lines as the uniqueness part of the proof of Theorem 4.3. Let iEI, and for every ~<Wl define sets of sentences di~(T)~ Y'i~(T) ~ Y'i(T), as in the proof of Theorem 4.3. If qJEdiO(T) then m* (qJ)=mi (qJ) by the definition of m. The rest of the induction is carried out as in the proof of Theorem 4.3. COROLLARY
We state a simple fact about product measures: Let X, Y be sets, let d, :!B be fields of subsets of X, Y respectively; let d, PJ be the IT-fields generated by s¥, :!B respectively; and let u, v be probabilities on d, PJ respectively. Let .;;f x PJ be the product IT-field of.91 and PJ. Then we have: LEMMA 5.2. If A is a probability on .;;f x PJ such that A(A x B)=/l(A)' v(B) for all Auy, BE!!IJ, then A(A x B)=/l(A)'v(B)for all AEd, BEPJ. Proof: Let .97 x!!IJ be the field of subsets of X x Y generated by rectangles A x B, where AE.W', BE.OJ. Then the condition A(A x B)=/l(A)' v(B) determines the probability A on d x .OJ. .;;f x PJ is o-generated by d x:!B. Thus this condition determines A on d x PJ. The product measure /l x v on d x PJ agrees with A on d x:!B. Thus A=/l x v, which proves the assertion. COROLLARY
5.3. For every n « co, let inEI and let qJnEY';jT). Then
m*( 1\ qJn)= fl m*(qJn)' n<w
n<w
Proof: The assertion follows from the continuity ofm* if we can establish: If n-c co, i o, " " i n - 1 EI and qJkEY'iJT)for all k «;n, then m*( 1\ qJk) = fl m*(qJk)' k
k
As in the proof of Theorem 4.3, for every ~<Wl' k «:n we define sets of If qJkE.'l'ik(T) for all k-en, then there exists ~<(()l such that qJkEY'ik~(T) for all k «:n. Accordingly we prove by transfinite induction: For every ~<(()l' if qJkE·Y'ik~(T) for all k «:n, then m* ( 1\ qJk) = flm* (qJk)' First if qJkE {}ik O(T) for all k < n, then the assertion sentenCeSJik~(T)~Y'ik~(T)~.Y'dT).
k
k
holds by the definition of m*. If qJkE.'l'ikO(T) for all k «:n, then for every k
229
ASSIGNING PROBABILITIES TO LOGICAL FORMULAS
string of quantifiers and Mk is an 0ik O(T)-matrix of CfJk> which means every substitution instance of Mk belongs to 0ik O(T). It is now easy to see that by an analogue of Lemma 7.10, straightforward computations with sup's and inf's, and the fact that we have established the assertion for CfJkEOikO(T), k «:n, we obtain m*( /\ QkMk)= m*(QkM k). We omit the cumbersome
n
k
k
details and illustrate the idea with a simple example. By an analogue of Lemma 7.10,
m*(3voMo(vo) A VvIMI(VI»= sup infm*[( V Mo(t o» Fo
toeEo
Fl
A (
1\ MI(tI»] tl EF I
= sup inf[m*( V Mo(to»'m*( /\ MI(tI»] Fo
FI
toeEo
II
eF I
= sup m*( V Mo(to»'inf m*( /\ MI(tt» Fo
toeEo
F1
IIEF t
because V Mo(to)EoioO(T), /\ MI (tl)EOitO(T), and those are formulas for toeFo
flEFt
which the assertion has already been established. Next assume ~ > and that the assertion holds for all ordinals smaller than ~. First suppose CfJkEOik~(T) for all k-en. Remember that for every k:«:n, (U Y ik" (T»/f- is a subalgebra of Yik(T) and rr-generates Oik~(T). Now
°
,,<~
suppose tfikEU Yik,,(T) for all k-en. Then for some 11«, tfikEYik,,(T) for ,,<~
all k-en. Thus, by inductive hypothesis, m*( /\ tfik)= k
dimensional version of Lemma 5.2, m*( /\ CfJk)= k
nm*(tfik)' By an n-
n m*(CfJk)' In the general k
k
case where CfJkEYik~(T) for all k-en, we proceed in the familiar fashion using prenex normal forms and Lemma 7.10, as in the second part of the case ~=o. This completes the proof of Corollary 5.3. The construction of independent unions is particularly valuable for the introduction into a given probability system of "a priori conditions" such as ordinary relational structures. For example, we may consider a probability system '!(I=
230
DANA SCOTT AND PETER KRAUSS
It is natural to ask for the definition of an analogue of the direct product of ordinary relational systems; however, a reasonable, natural generalization of this construction for probability systems does not seem to exist. On the other hand we are able to give an intuitively very suggestive definition of an analogue of the ultraproduct construction of relational systems. Ultraproducts. Consider again our language if with one binary predicate R. Let! be an index set. For each ie J, let T, be a set of new individual constants, and let j , m;) be a probability system, where m, is a probability on J(~)/f-. Let T= T, be the Cartesian product of the family of sets {~: iEI}.
n
iEJ
For qJEY'(T) and iEJ, let qJli be the projection of qJ onto the r h coordinate; that is, replace in tp every tETby tjE~. Then for every qJEy;J(T) and iEJ, f- qJ implies f- qJli. Finally let A be a probability on the power set of 1. Define for all (pEJ(T) a function m by the equation
m(qJ) =
J
mj(qJli)dA(i).
1
5.4. (i) For every qJ, ljIEJ(T), if f- qJ+->ljI, then m(qJ)=m(ljI). if r ip ; then m(cp)=1. (iii) m, regarded as afunction on J(T)/f-, is a probability. Proof: (i) and (ii) are trivial. Thus m may indeed be regarded as a function on .J (T)/f-, and it suffices to prove o-additivety. Suppose qJnEa(T) for all n < W, and f-, (qJlIl;\ (Pn) for all m iol1. Then f-, (qJlIlli;\ qJnli) for all m ion and all iEI. Thus for all iEJ LEMMA
(ii) For every qJEJ(T),
I1l j( V qJnli)= Il1l i(qJnl i). n<w
n<w
J
Therefore by the Dominated Convergence Theorem
m( V (Pn) = n<w
m j ( V CPn I i) dA(i)
JI
n<w
1
=
I
n
I1lj(qJn I i) dA(i)
n~roJ 11l;(qJnl i)dA(i) =
I
n<w
1
m(qJ,,).
We define the ultraproduct with respect to A of the family of probability
ASSIGNING PROBABILITIES TO LOGICAL FORMULAS
231
systems {
COROLLARY
5.5. For all lpEY(T),
m*(lp) =
J
m7(lpji)dA(i).
I
Proof: Define ]l(lp)=J1m7(lpli)dA(i), for all lpEY(T). By the same argument as in Lemma 5.4, ]l is a probability on Y(T)/!-. Clearly ]l extends m. By Theorem 4.3 it suffices to prove that ]l satisfies the condition (G). Let 3VlpEY(T). For every iEI, m7 (3 Vlp i i) =
sup m7 (V (lp I i)(t»).
FeTi(W)
teF
Therefore for every iEI and n
m7 (3vlp I i) = lim m7 (V (lp I i)(tik»). k
n-e co
For n
n~ 00
Thus, by the Monotone Convergence Theorem, J1 (3vlp) = J
m~ (3vlp I i) dA(i) =
I
=
J lim I
limJm~(V
nr- co
I
k
n-too
m~ (V
k
lp (Sk) I i) dA(i)
lp(sk)li)dA(i) = limJ1(V (P(Sk») n-.. .«co
k
=J1(V lp(Sk») = sup J1(V lp(t»). k<w
FeT(w)
teT
Remark: Let t, t'ETand let J={iEI:ti=t'}. If for every iEI,
232
DANA SCOTT AND PETER KRAUSS
function nETT is a permutation of T if tc is one-to-one and onto. A permutation tt is finite if n(t)=t for all but a finite number of lET. Following a suggestion of Gaifman [1964] we call
5.6. Let
permutation n ofT and every (pEJ(T), m*(
properties of sup's and inf's that we obtain m*(QM)=m*(QM"). We illustrate the argument with a simple example. By an analogue of Lemma 7.9,
m*(3v OVv1M(v o,v 1 )) = sup
inf
FoeT{W) F1ET(W)
Now V
rn e Fo
m*(V
/\
toeF01{EFI
M(to,t 1 ) )
.
/\ M(to, t l)EJO (T), and the assertion has already been establishtl
eF l
ed for these formulas, so we have for every F 0' F 1 ET(W)
m*( V
/\
10eFo tlEF l
Finally, since
tt.
M(tO,t 1 ) ) = m*( V
/\
toeEo llEFI
M"(n(tO),n(t 1 ) ) )
.
is onto,
m*(3v OVv1M(v o,v t )) =
e
sup
inf
FoeT(W) FIET(W)
m*( V
/\ M"(tO,t 1 ) )
10eFo fiEF.
Next, assume > 0 and that the assertion holds for all ordinals smaller than Recall that (U Y'q(T))/f- is a subalgebra of /7 (T)/f- that a-generates
e.
q<~
233
ASSIGNING PROBABILITIES TO LOGICAL FORMULAS
6~(T)/f-.
By the inductive hypothesis m*(qJ)=m*(qJ") for every
qJE
U 9"~(T).
~<~
Let E be the set of qJE;J~(T) for which the assertion holds. If qJnEE and f-qJn~qJn+l for all n «:co, then
m* (V qJn) n<w
=
lim m* (qJn)
n-e co
= lim m*(qJ;) =
m*( V qJ;). n<w
A similar argument for decreasing sequences proves that E is monotone, and therefore by a well-known fact about monotone classes (see Halmos [1950], p. 27), E=;J~(T). If qJEYJ~(T), we proceed as in the second part of the case of ~=O. In a sense the symmetric probability systems are diametrically opposite to ordinary relational systems. In ordinary relational systems the probability is as concentrated as possible; in symmetric probability systems it is completely dispersed. The condition of symmetry is a severe restriction on a probability system, as an example in Section 6 will demonstrate. This completes our discussion of probability-model-theoretic concepts, and we now turn to the analogue in probability logic of theories in ordinary logic. 6. Probability assertions. In ordinary logic a theory of i f is any subset of 9" closed under deduction. In probability logic we first have to define the concept of probability assertions which play the role of sentences (or, better axioms and theorems) in ordinary logic. For this purpose we introduce a new language Jt, the first-order language of real algebra. Jt has denumerably many distinct individual variables Am n < w. The non-logical constants of Jt are a binary predicate ~, binary function symbols + and ., and individual constants 0, + 1 and -1. The logical constants of vlt are /\, v, -', V and 3, standing for (finite) conjunction, disjunction, negation, universal and existential quantification respectively. Formulas and sentences of Jt are defined as usual. Jt is to be interpreted in the real numbers in the standard way with the obvious meaning being given to the symbols. Let Re denote the set of real numbers, and say that Jt is interpreted in the relational system 9t=(Re, ~, +, ',0, + 1, -I).
234
DANA SCOTT AND PETER KRAUSS
The set of sentences of ull true in 9t is called the set of theorems of real algebra. An algebraic formula is a quantifier free formula of uti. Every algebraic formula is equivalent in real algebra to a disjunction (conjunction) of conjunctions (disjunctions) of polynomial inequalities of the form p;;::O or p>O, where p is a polynomial with integral coefficients. We call an algebraic formula closed (open) if it is equivalent to a disjunction of conjunctions of polynomical inequations of the form p;;::O(p>O). It is obvious that in this definition we could have used the conjunctive instead of the disjunctive normal form. We now make several definitions. A probability assertion of se is an (n + 1)tuple <$, ({Jo, ... , ({In-l)' where n-c co, $ is an algebraic formula with exactly n free variables and ({Jo, ... , (Pn -1 E .C/. A probability assertion is called closed (open) if the algebraic formula is closed (open). A probability system , ({Jo, ... , ({In -1) if the n-tuple of real numbers <m«({Jo), ... , In«({Jn-l) satisfies $ in 9t. If L; is a set of probability assertions and IJI is a probability assertion, then IJI is a probability consequence of L; iff every probability model of all assertions in L; is also a probability model of IJI. IJI is a probability law of se if IJI is a probability consequence of the empty set of assertions. Immediately the familiar questions arise: Is there a method of deductively generating the probability consequences from a given set of probability assertions? Is there a method of deductively generating all probability laws? Under which conditions does a set of probability assertions have a probability model? Is there an analogue of the concept of consistency in ordinary logic? Obviously these questions are interrelated. Before we enter into their discussion, we insert some remarks concerning our definition of probability assertions. The definition of probability assertions depends both on the language se and the language jt, and it is clear how this definition could be generalized by considering languages uti' with stronger means of expression. The underlying idea of our approach is that we want to investigate polynomial inequalities in "variables" fl( ({Jo), ... , fl( ({In-l) with real coefficients, where (Po,· .. , ({In- 1 E Y' and fl is interpreted as a probability on .'/'/1-. Our definition does not quite realize this idea. For this purpose it would have been appropriate to introduce a language j{' which is like j{ but has individual constants for every real number. We easily see that by the continuity of addition and multiplication every closed probability assertion of uti' is equivalent to a denumerable set of probability assertions of uti' with rational coefficients and, after clearing denominators, to a denumerable set of proba-
<
ASSIGNING PROBABILITIES TO LOGICAL FORMULAS
235
bility assertions of JI. This, however, is not true for open probability assertions of JI'. We thus fall somewhat short of our objectives. Nevertheless since not very much work has yet been done towards the investigation of probability assertions, we chose the present formulation for its simplicity. If L:s; .'7, then L: determines a set of probability assertions {o.o -1 ~ 0, 1p):lpEL:}. Accordingly we say that
and
Let L:",
= {< qnAo - Pn ~ 0,
U
«-
q~Ao
+ P~ ~ 0, tp »: n < w}
and let L:= U L:",. Accordingly we say that
fl if m(Ip)=fl(
complete and consistent theory, then L1 uniquely determines a two valued o-additive probability measure fl on Yj't-. In this case
{V
n<w
P~n: ~
< wd
u {--.
[P~n
1\
P~'n]: ~
< ~' < w 1 , n < w}.
It is easy to see that L: is consistent. Suppose then there exists au-additive probability measure fl on Yj't- such that fl( O. Thus there are uncountably many p's such that fl(P) > O. Thus for some n < ca there are uncountably many
236
DANA SCOTT AND PETER KRAUSS
~ < WI such that f.1(P~n) > O. Since f.1([P~n /\ P~'n])=O for ~ # C this is a contradiction. Obviously every complete and consistent set of sentences of an infinitary proposi/ionallanguage has a model. In infinitary propositional logic the trouble therefore arises from the fact that the Prime Ideal Theorem fails for Boolean c-algebras. Naturally the question arises: Does every complete and consistent set 1:<;; Y' have a model? The answer is again no, and a counter-example is due to Professor C. Ryll-Nardzewski. Interestingly enough the counter-example produces a probability model of the complete consistent set of sentences under consideration. The question of whether every complete consistent set 1:<;; Y' has a probability model can, however, be settled by a similar counterexample, and we shall discuss both of these examples in a form slightly modified from Ryll-Nardzewski's original suggestion. Let 2 be an infinitary language with countably many one-place predicates P, for eachj<w, and define a probability model 1ll=(A, R j, d, m)j<", as follows: Let A =w, and let ,91 be the Borel sets of the product space (2"')"'; that is, the o-field of subsets of (2"')'" generated by all sets of the form gE(2"')"': ~(i) (j)=l}, where i,j<w. Let m be the product measure on d determined by m({~E(2"')"':~(i) (j)=l})=! for all i,j<w. Finally, forj<w, define Rj(i)=gE(2"')"':~(i)(j)=l} for all iEA. (Note: strictly speaking III is not a probability model since (s~, m) is not a measure algebra. Thus we would have to consider the quotient algebra s?iII of d modulo the o-ideal I={xEd:m(x)=O}, and lift m up to a strictly positive probability on diI. In this example, however, all sup's and inf's in ,91' that have to be taken into consideration are countable; clauses (vi) and (vii) of the definition of the valuation function h make sense; and everything comes out just the same. We can omit the tedious details.) Then let ~1={/;:jEA} be a set of new individ ual constants such that Ii # Ii" if i # 1'. Now we observe that for every IfJE S, the element h( IfJ )Es?i is invariant under all finite permutations of the second coordinate in (2"')"'. By the well-known 0-1 Law (Hewitt and Savage [1955] p. 496) m is two-valued on h(IfJ). Thus the set 1:={IfJE,'/}: m(h(1fJ ))=1} is a complete and consistent theory of 2. We wish to show that 1: has no model. Indeed, suppose IB=(B, Sj)j<", is a model of 1:. Since B must be non-empty, let bEB. For j<w define formulas Qj(v)=Piv), if bES j; while Qj(v)=iPiv), if brjSj' Then 3v[ A Qj(v)] holds in lB. However, as a j<w
straightforward computation shows, m(3v[ A Qj(v)])=O, which is a contral « o»
diction. On the other hand, by its very construction of 1:; that is, f.1~1(1fJ)=1 for allIfJE1:.
mis a probability model
ASSIGNING PROBABILITIES TO LOGICAL FORMULAS
237
For our second example we let m:'=(A, R~, d')i<<» be that Boolean-algebraic model where A=w, where d'=d/J, the ideal J being the a-ideal of all first-category sets in the Borel algebra d, and where R~(i)=Rj(i)/J for all i.] «co. Since A is countable, we note that the valuation h' for m:' is such that h'(
Boolean algebra has a strictly positive a-additive probability measure iff it is weakly distributive and has the Kelley property (see Kelley [1959]). In Karp [1964] we find the theorem that every countable consistent set E r:;;. Y' has a countable model in the ordinary sense. The exact analogue of that result also holds for probability logic. We can also treat the case of theories with identity. To help formulate the result we define the formula On for O
each probability system (T, m) with strict identity we have m(On)E{O, I} for O
238
DANA SCOTT AND PETER KRAUSS
THEOREM 6.2. (i) Let J1 be a probability on .'7'If-, and let E ~:7 be a countable set. Then there exists a countable probability model (T, m) such that for every cpEE, iii(cp)=J1(CP)' (ii) Iffor every O
We will give a detailed proof of part (ii) and leave the proof of part (i) to the reader. To prove this result we require a few lemmas. The first is a measure theoretic generalization ofthe well-known Rasiowa-Sikorski Lemma (see Rasiowa and Sikorski [1950]). The proof is given in full in the Appendix. 6.3. Let.iJ?J be a Boolean a-algebra and let d ~I:JB be a a-subalgebra. Let J1 be a probability on d, and for every m, n < w let bmnEI:JB. Then there exists a finitely additive probability v on I:JB such that LEMMA
(i) vex) = J1(x),for all
XE
d;
(ii) v( 1\ bmn) = lim v(l\ bmi),for all m < W. n-c rc
tf----J.r£·
i
For every cpE.J(T) we recursively define an ordinal number A(cp)<Wl' called the length of (P, by these equations:
(i) if (P is atomic, A(cp) = 1; (ii) J.( -, (p) = l(cp)
+ 1;
(iii) A«(P, v (P2) = A(cp,
A
(P2) = A(cp,) + A(CP2)
(iv) A( V (Pn) = l( 1\ CPn) = n<w
n<(J)
I
n<w
+ 1;
l(CPn)'
LEMMA 6.4. If cpE;J(T), A(cp)2 co, and -, occurs in cP only infront of atomic formulas, then there exists a sequence ljJiEJ(T) such that A(ljJ;) < A(cp) for all i < ill, and either hp<-+ V ljJ i or hp<-+ 1\ ljJ i : i<w
i « s»
Proof: By transfinite induction on A(cp), If A(cp)<W, the assertion holds trivially. Thus assume }.(cp)2W and that the assertion holds for all cP' such that }'«(P')
n<w
Thus for some n<w, either cP= V (Pi or cP= 1\ CPi' Consider the first case. i
i
Since A(CP)2W, we see that ;~«(Pi)2ill for some i«:n. Let m be the largest such integer i-:n. Then A(CPm) <),«(p) and A(CPi)<W for all m-c.i-cn. By inductive hypothesis there exists a sequence ljJjEJ(T) such that A(ljJj)
239
ASSIGNING PROBABILITIES TO LOGICAL FORMULAS
j
f-qJm~
Then
V t/Jj or
j<w
f-qJ~
f-qJm~
V [V qJi
j<w
t-c m
V
A t/Jj' Again consider the first case.
j<w
t/Jj
V
V qJJ.
m
It now follows from well-known laws of ordinal addition that for every j < OJ,
2( V qJi i<m
V
t/Jj
V
V qJi) = 2( V qJJ + 2(t/JJ + 2( \I
qJJ
<2(\/ qJi)+2(qJm)+2( \I
qJi)
m
i<m
m
i<m
m
= 2(qJ),
which proves the assertion. All other cases are treated analogously. If f! is a finitely additive probability on Yjf- and I £.9" is a set of infinite conjunctions and disjunctions, we say f! preserves I if for every A qJm V t/Jn EI, n<w
a-c ro
f! (A qJn)
=
f!( \I t/Jn)
= lim
n<w
and
n<w
lim f! (1\ qJi)
n-e co
n-r co
t« n
f!( \I t/Ji)' i
LEMMA 6.5. For every qJEa(T) there exists a denumerable set I£a(T) of infinite conjunctions and disjunctions such that for all finitely additive probabilities m i and m2 on o(T)jf- , if they agree on the finitary sentences and preserve I, then ml(qJ)=m2(qJ). Proof: By transfinite induction on 2(qJ). If 2(qJ)
or
i<w
f-qJ~
A t/Ji' Consider the first case. Then for every n < OJ, we note that
i<w
2(\/ t/Ji)<2(qJ). By the inductive hypothesis, for every n
denumerable set In such that for all finitely additive probabilities m l , mZ, if they agree on the finitary sentences and preserve In' then m i (\I t/Ji)=
m 2( \I t/J;). Let I= i
i
U In U{ V t/JJ Then clearly mi (qJ)=m 2(qJ) for all finite-
n-c cc
i<w
ly additive probabilities m., m2 which agree on the finitary sentences and preserve I. The other case is completely analogous.
240
DANA SCOTT AND PETER KRAUSS
Now we begin with the proof of part (ii) of Theorem 6.2. Let J1 be a probability on Y'/'r such that for every 0 ,,(ViJ, n-c co, be an enumeration of all existential sentences in f. Choose a sequence (JEW'" such that (i) (J(n)+ 1 <(J(n+ 1),jor all n<w; (ii) every individual constant in q>" has index < u(n). Consider the set of sentences r={tji:-tj:i<j<w}
u {3v inq>,,(v;,,)---+ V
;,;,,(,,)
q>,,(tj):n<w}.
The following lemma is essentially due to Ehrenfeucht and Mostowski [1961]. LEMMA 6.6. Iflll=
cursion:
(1) Iffor some n « co, m=(J(n) and if there exists an xEA~{ai:i<m} that satisfies q>,,(v;,,) in
Let e={IO,,:n<w}, let ,1=rU e, and define a mappingffrom y/e'r into Y(T}/,1 'r by f«(p/e 'r )=q>/,1 'r. Since e s,1, the mapping is well defined. Clearlyfis a (J-homomorphism. Moreover,jis an isomorphism into. Indeed, let q> E //' and suppose ,1 'r I q>. If e U {q>} has a model then, by the Lowenheim-Skolem Theorem, it has a denumerably infinite model. Thus by Lemma 6.6, ,1 U {q>} has a model, contrary to the assumption. Thus, e U {q>} has no model; therefore, by the "weak" Completeness Theorom, e'r I ip, which proves the assertion. Now let g be the canonical o-homomorphism of Y(T)/'r onto Y(T)/Ll'r; that is, g(q>/'r) = q>/Ll'r. For orientaiton we draw a
ASSIGNING PROBABILITIES TO LOGICAL FORMULAS
diagram
241
Y/I- ~ y/e I- ~ Y(T)/iJ I- ~ Y(T)/I-.
As is well-known, since /l(, Bn)= 1 for all Oep'. Since (j-IOg) t « co
(epjl-)=(j-IOg) (ep'/I-), we have /l(ep)=/l(ep'). By Lemma 6.5, for each ep' there exists a denumerable set E' £ d(T) of infinite conjunctions and disjunctions such that for all probability measures m l and m z onJ(T)/1- if they agree on the finitary sentences and preserve E', then ml (ep')=mz(ep'). Finally,
let };' be the union of all these sets E'. };' is denumerable. By Lemma 6.3, there exists a finitely additive probability v on Y(T)/I- which extends /l and preserves };'. Let n be the restriction of v to the finitary quantifier-free sentences. Then n is o-additive. (This is a consequence of Lemma 7.1 which will be established in Section 7.) As is well-known, n may be extended to a probability m on d(T)/I-. We claim that
r
{ti:l= tj:i <j < N}, e = {, 0n:n < N} U {On:N < n < w}, iJ=FUe. =
As before, define a mapping f from y/e I- into Y(T)/iJ I- such that f(ep/el-)=ep/iJl-. Againfis an isomorphism into. Indeed, let epEY' and suppose iJ I-,ep. Then
el-( /\
i<j
ti:l=tj)~'ep.
242
DANA SCOTT AND PETER KRAUSS
Thus
E> 1-:3 V o ... :3
However,
Vn - \ (
1\
i<j
E>1-3v O ••• 3v n _
t (
V i:# Vj) -+ ,
/\ i<j
q>.
Vi:#V j ) .
Thus E>I-'q>. Finally we observe that if q>' is obtained from q>EI: by eliminating quantifiers; that is, by successively replacing all existential subformulas 3vt/J(v) by V t/J (f i ) , then T 'r q><->q>'. Thus we can complete the proof as i-:»
before. We conclude the proof of Theorem 6.2 with a remark concerning the proof of part (i). In this case we need only replace the refined method of the Ehrenfeucht-Mostowski Theorem (Lemma 6.6) by the somewhat cruder method of the original Henkin Completeness Proof (Henkin [1949]). The steps are quite similar though simpler than those just given. In this case, however, we clearly cannot expect the probability model
We now give an example to show that there are probabilities on :7/1which have a probability model but do not have a symmetric probability model. Indeed, for every q>E.(/' let J1(q»=1 iff q> holds in <WI' <), the system of the countable ordinals with their natural ordering. Every WI is definable in 2:; that is, there exists a formula q>~ of 2: with exactly one free variable v such that for every I1<W I , 11 satisfies oin <Wt, <) iff 11=(. Thus whenever «WI' we have J1(3vq>~)=1; and whenever 11«<W I , we have J1(3v[q>,r /\ q>~])=O. Now suppose J1 has a symmetric probability model ~)=1 for every «WI' we obtain for every «WI some tET such that m*(q>~(t))>O. Since ~(t)).Thus for every «WI and every fET, we find m*(q>~(t))>O. Now consider a fixed tET. Then for some e>O, m*(q>~(t))~f. for infinitely many «WI' However, since In(3v[q>~/\q>~])=0 whenever 11«<W I • we find that m*(qJ~(t)/\q>~(f))=Owhenever 11«<W t, which cannot be the case. There is a positive result concerning symmetric probability models for the
«
ASSIGNING PROBABILITIES TO LOGICAL FORMULAS
243
finitary language 2(w) which is due to Gaifman [1964]. We will give a simplified version of his proof in Section 7, Theorem 7.14. The questions of whether there is a method of deductively generating the probability consequences from a given set of probability assertions, or of deductively generating all probability laws are clouded by the fact that it is difficult to give the concept of deductively generating a workable meaning for infinitary languages. Nevertheless we can make a few remarks. Our discussion thus far certainly shows that we have not much reason to expect a positive answer to the first question. On the other hand the second problem has in a sense a positive solution which we will present now. THEOREM 6.7. Let
<
i
2 iff the sentence VAo ... V An _ I
[[
i\ Ai = 0
ieI
1\
i\ Ai ~ 0
i-c n
1\ ..1. 0
+ ... + An _ I = 1] --+ cP]
is a theorem of real algebra.
Proof: Suppose
244
DANA SCOTT AND PETER KRAUSS
therefore yields a method of generating all probability laws of Sf. Whether there is a more useful way of generating the laws remains to be seen. Theorem 6.7 also provides suggestions for the definition of an analogue of the concept of consistency in ordinary logic. The general problem of conditions under which a set of probability assertions has a probability model is completely open for the infinitary language Sf, and it is apparently quite difficult. This completes our general discussion of the probability logic of the infinitary language Sf, and we now turn to the finitary language Sf(w) where the situation is somewhat more satisfactory. 7. The finitary case. The Boolean algebras Y(W)/f-, y(w)(T)/f- and )w) (T)/f- are subalgebras of the IT-algebras //' /f-, Y(T)/f- and o(T)/f- respectively. Our definitions and results concerning the infinitary language Sf therefore have rather obvious applications to the finitary language Sf(w), and in many cases they can be considerably strengthened. This is due to two important facts which we state for comment and reference. 7.1. Every finitely additive probability J1 on :7(w)/f- is IT-additive. Proof: For every ];<;;,,/)(W) and every
The following lemma is well-known and has an easy proof by means of elementary methods of functional analysis. For a purely algebraic proof we refer the reader to Horn and Tarski [1948]. LEMMA 7.2. Let fJ?J be a Boolean algebra and let d <;; fJ?J be a subalgebra. Every finitely additive probability on d can be extended to a finitely additive probability on fJ?J. As pointed out before, Gaifman [1964] gives a proof of the next theorem. Our proof of Theorem 6.2 can be essentially simplified to yield this result by replacing the role of Lemma 6.3 by Lemmas 7.1 and 7.2. Indeed Lemma 6.3 has been designed to patch up the difficulties arising from the fact that Lemma 7.2 fails for IT-additive probabilities. This in turn corresponds to the fact that the Prime Ideal Theorem for IT-ideals fails for Boolean IT-algebras. THEOREM
bility model.
7.3. (i) Every probability J1 on //,(w)/f- has a denumerable proba-
ASSIGNING PROBABILITIES TO LOGICAL FORMULAS
245
(ii) Iffor every O
Remarks: (1) As can readily be verified by an analysis of Lemma 6.6, our proof of part (ii) does not go through for a language 2(w) which either has infinitely many individual constants or non-denumerably many non-logical constants to begin with. This fact is substantiated by two counter-examples of Gaifman [1964]. Nevertheless, Theorem 7.3 still holds for these languages, as the proof in Gaifman [1964] shows. We will not discuss the question of adapting our method of Lemma 6.6 to this situation. (2) The cardinality statements of Theorem 7.3 obviously depend on our assumption that 2(ro) has only denumerably many non-logical constants. If we allow non-denumerably many non-logical constants then the well-known adjustments have to be made. The same remark applies to all other theorems of this part which contain statements about the cardinality of probability systems. Let T be the set of complete consistent theories in .P(w), that is, the set of prime ideals in y(ro)/f-. As is well-known from the Representation Theorem for Boolean Algebras (see, e.g., Halmos [1963] p. 77), T is a compact Hausdorff space with a basis of closed-open (clopen) sets of the form {LET: qJEL}, where qJEY(w), and y(ro)/f- is isomorphic to the field of clopen subsets ofT under an isomorphism which maps qJ/f- into {LET: qJEL}. Every model determines a complete consistent theory in 2(ro), that is, a point in T. By the ordinary Completeness Theorem, every complete consistent theory
246
DANA SCOTT AND PETER KRAUSS
of
2(10) has a model; thus T may be identified with the space of models. Many important results in the ordinary logic of 2(w) may conveniently be established through topological considerations in the space T. This topological construction can be generalized in the strictest sense of the word; thus the space M of probability models of 2(w) can be defined as a compact Hausdorff space such that the space T can be homeomorphically embedded into M. The construction makes use of well-known definitions and methods of functional analysis. For the details we have to refer the reader to Dunford and Schwartz [1958]. Let C (T) be the linear space of all continuous real functions on T. Since the characteristic functions of clopen subsets of T are continuous, we may regard Y(W)/'r- as a subset of C (T). Let L be the linear subspace of C (T) generated by .,/,(w)/'r- in C (T). As is well-known, C (T) is a Banach space Any finitely under the sup-norm where for XEC (T) we have [x] =sup x( ~ET
n
additive probability p on .,/,(w)/'r- uniquely extends to a linear functional p on C (T) such that (i) p(x):c;. [x] for all XEC (T); (ii) p(l)= 1. Conversely, any linear functional on C (T) satisfying (i) and (ii) uniquely determines a finitely additive probability on ,,/,(w)/'r-. Let C (T)* be the linear space of all continuous linear functionals on C (T). As is well-known, C (T)* is also a Banach space with its own norm such that for every pE C (T)*, Ilpll:::;; 1 iff p(x):::;; [x] for all XEC (T). Let M={pE C (T)*: Ilpll :::;; 1, p(l)=l}. By our remark above, the set of all finitely additive probabilities on y(w)/f- may be identified with M, and by Lemma 7.1 this set agrees with the set of probabilities on .,/,(W)/'r-. Every probability model determines a probability on Y(W)/f-, and by Theorem 7.3 every probability on .'/~(w)/'r- has a probability model, thus M may be identified with the space of probability models. We now consider C (T)* with the so-called weak star topology. The basic open neighborhoods are sets of the form
where n-c co, pEC(T)*, xo, ..., XII_1EC(T), and s c-O is a real number. It is easy to see that M is a closed subset of the unit-sphere {pE C (T)*: li!lll:::;; I}; therefore, by the Alaoglu Theorem M is a compact Hausdorff space with the relativized weak star topology. Every LET uniquely determines a two-valued probability p on y(w)/'r-,
ASSIGNING PROBABILITIES TO LOGICAL FORMULAS
247
and conversely. Thus there exists a natural embedding of T into M. Finally we observe that M is a convex set; that is, for every fl-l, ItzEM, and every real number 0< oc< 1, OCfl-l +(1- oc)fl-z EM. For any subset K of a linear space, the convex hull of K is the smallest convex set containing K. The closed convex hull of a subset K of a linear topological space is the closure of the convex hull of K. We say fl-EM is an extreme point of M iff for every fl-l, fl-z EM and every O
To prove Theorem 7.4 we first establish a useful lemma. LEMMA 7.5. Sets of the form M n N(fl-; xo, ... , x n - 1 ; e), where xo, ... , x n _ 1E.C/'(w)/f-, constitute a basis for the weak star topology ofM. Proof: Let fl-EC(T)*,xo, ... ,Xn _ 1 EC (T ), and e>O. Consider VE M n N(fl-; x o, ... , x n - 1 ; e). Let c:5 =e- max Iv(x;)- fl-(x;)l. By the Stone-Weieri
strass Theorem, there exist Yo, ... , Yn-l EL such that Ilxi - Yill < tc:5 for all i <no We show M n N(v;yo, ... ,Yn-l ;tc:5) s M n N(fl-;x o, ... 'X n- 1 ;e). Indeed, let AEM n N(v; Yo, ... , Yn-1; tc:5) and i«:n. Then since A, vEM
IA(xi) - fl-(xi)1 ::; IA(xi) - A(Yi)1 + IA(Yi) - v(yJI + Iv(y;) - V (x;)j + Iv(x i )
< [x, < e.
~
p(x;)1 )';11 IIAII + tc:5 + IIYi - x;llllvll + e - c:5 -
This proves that sets of the form MnN(p;xo, ... ,xn _ 1;e), where Xo,···, Xn - 1EL, form a basis. Now let xEL. Then x=ocoxo+···+OC'_IX,_I' where X o, ... , X,_ 1 E Y(w) If- and (J(O' ••• ' OC'-1 are real numbers. Consider vEMn N(p; x; e), and let c:5=e-lv(x)-fl-(x)i. Then a straightforward computation yields M n N (v; x;; c:5jr-loci J) s M n N(fl-; x ; e). This proves the
n
lemma.
i
Now we proceed with the proof of Theorem 7.4. For part (i) we regard T as a subset of M and show that the topology of T is the topology of M
248
DANA SCOTT AND PETER KRAUSS
reIativised to T. Let n «.co, flET and x o, ... , Xn_1E::/'(w)/'r-. Then {vET: Iv(x;) - fl(X;)1 < e for all i < n} =
n {vET: Iv(x i) -
i
fl(x;)1 < s}.
For every i «:n, {vET:lv(xi)-fl(X;)I<E} will be either the empty set, the whole set, or the set of all prime ideals of y(w)/'r- containing Xi' depending on the choice of fleX;) and s ; in any case it is a clopen subset of T. Since a similar argument shows that every clopen set of T is the restriction of an open set of M to T, Lemma 7.5 proves assertion (i). To prove part (ii) we let flET and consider fll, flzEM and O
(ii) L has a probability model with strict identity iff every finite subset of L has a probability model with strict identity. Proof: It is clear that the set of probability models of a closed probability assertion of stew) is a closed subset of M. Part (i) therefore foIlows from the compactness of M. To prove part (ii), let L be a set of closed probability assertions of se(w) such that every finite subset of L has a probability model with strict identity. Consider the set {On:OOn+d. For every O
where }.o = ik is short for - Ao ~ 0 if ik = 0, and Ao -1 ~ 0 if i k = 1. Clearly Llni is a set of closed assertions. Then for every 0 < n < w, there exists i EIn such that L U LIn; has a probability model. Indeed, suppose that this is not the case.
249
ASSIGNING PROBABILITIES TO LOGICAL FORMULAS
Then, by part (i), there exists O
i e Is;
a probability model with strict identity; however, for some iE/II this probability model has to be a model of Li ni, which is a contradiction. If for some n there exists ie I; which is not identically zero, and I: U Li ni has a probability model, then, by virtue of our observation above, there exists 0
7.7. IjVvcpEy(w)(T), then f.l(Vvcp)=inf f.l(/\ cp(t»). F
tEF
If cp, l/JEy(w)(T), then an occurrence of
250
DANA SCOTT AND PETER KRAUSS
7.8. (i) Let ljJ' be obtained from ljJ by replacing a simple occurrence of3vqJ in ljJ by V qJ(t). Then p(ljJ)=sup p(ljJ'). LEMMA
F
rEF
(ii) Let ljJ' be obtainedfrom ljJ by replacing a simple occurrence ofVvqJ in ljJ hy 1\ qJ(t). Then p(ljJ)=inf p(ljJ'). tEF
F
Let qJE//?(lU)(T) be of the form Qov o ... Qn-l vn- 1 M (v o, ... , Vn-I)' where for each i
(aeEo
Fn~1
(1l-\EF n- 1
M(to,···,t n- 1 ») .
Proof: By induction on n. For n = 1 the assertion holds by condition (G) and Lemma 7.7. For n> 1 we have by inductive hypothesis, p (qJ) = bd., ... bd n- 1 P( #0 ... Eo
Fn-l
taeFo
QnDn M (to, ... , tn-l> Dn») .
TI IFiI, where IFil is the number of elements We enumerate the Cartesian product set F= TI F say F= {.!ic:k
Let F 0' ... ,Fn-l be fixed. Let N = in Fi •
# tll-lEFn~t
i
i
j,
If
#0··· #n-I QnDnM(to,···,tn-l,Dn),
taeEo
tn-IEF II
-
J
where f=
the monotonicity of disjunction and conjunction and the elementary properties of sup's and inf's we have p( # QnvnM(J,vn») = bd; ... bd; p( # fEF
GfO
GfN-.
= bdnP( #
Thus
En
#n M(J,s»)
».
fEFsEGf
#n M(J,t n
feFtnEF n
p(qJ)=bdo···bdnP(#o ... #n M(to,···,t n»). Fa
En
(aeE o
{nEFn
251
ASSIGNING PROBABILITIES TO LOGICAL FORMULAS
The last lemma of this series is an easy consequence of Lemma 7.9 and the distributive laws. For a convenient formulation we adopt the notation of the proof of Lemma 7.9 and write
bd/l(# M(J))=bdo··.bdn-1/l(#o .. , feE
F
Fo
Fn~l
toeFo
#n-l n-
tn - t E F
l
M(t o, ... ,tn- 1 ) )
·
For every le-er, let qJiESI'(w)(T) be of the form QkMk, where Qk is a string of quantifiers and Mk is a formula. For every k < r, let bd, and #k be the associated boundary and distribution prefixes respectively. LEMMA
7.10.
(i) /l( 1\ qJk) = bd.; ... bdc.., /l( 1\ #k Mk) k
(ii) /l (V qJk) k
Fa
Fr-t
= bd., ... bd r Fo
Fr-t
k
l
tkEFk
/l (\j #k Mk )
·
k
We are now in a position to prove analogues of many important results about the ordinary logic of 2(wJ. As a matter of fact, if we regard closed probability assertions of 2(w) as the analogues of sentences in ordinary logic, then the analogy in many respects seems to be complete. There are exceptions, however: we mentioned before our failure to define an analogue of direct products. We state next a few of the positive results and comment on their proof. THEOREM 7.11 (DOWNWARD LOWENHEIM-SKOLEM THEOREM). Let be a probability system of cardinality Kz ;:0: W o, and let K1 be a cardinal number such that Wo~Kl ~Kz. Then there exists a probability system ml>~. Moreover, for any subset T 3 c:; Tz with systemcardinality ~ Kl' we may choose T 3 c:; T l . Proof: The proof uses standard methods and utilizes the fact that, by the Gaifman Condition (G), whenever 3vqJESI'(wJ(T), there exists a denumerable subset T' c:; T such that
>
m*(3vqJ)= sup m*(V qJ(t)), FeT'(W)
teF
where (T, m> is the given probability system. THEOREM 7.12 (DIRECTED UNION THEOREM). Let {(T;, m;>:iEI} be a ~ directed family of probability systems; that is,for all i,jEI there exists k e I such that both ~ m k>. Let T= UTi' and define,for iE I
252
DANA SCOTT AND PETER KRAUSS
every CPE,-/W)(T), m(cp)=mi(cp), where cpEo(w)(T;). Then
We now present a result concerning symmetric probability systems which obviously has no analogue in ordinary logic. This result is due to Gaifman [1964] whose proof we have simplified by using ideas from the ultraproduct construction of probability models. THEOREM 7.14. Let 1: be a set of probability assertions of 2(w). Then 1: has a denumerable probability model iff 1: has a denumerable symmetric probability model. Proof: Suppose 1: has a denumerable probability model
ASSIGNING PROBABILITIES TO LOGICAL FORMULAS
253
yields X(Q)=l. We define for all
p,,(
=
f m* (
By the same argument as in the proof of Lemma 5.4 we show that 11", regarded as a function on y(w) (T)jf-, is a finitely additive probability. By Lemma 7.1, 11" is a-additive. 11" satisfies the condition (G). Indeed, let 3V
Q
tt
is onto,
m*(3v
tEF
= sup m*(V
teF
Since T is denumerable by the Dominated Convergence Theorem,
11,,(3v
FeT(W)
teF
= sup fm*(V
teF
n
= sup 11,,(V
teF
If
Proof: By Theorems 7.3 and 7.14. Remarks: (l) It is clear that our proof of Theorem 7.14 depends on the assumption that L has a countable probability model. Consequently Corollary 7.15 depends on the assumption that 2(w) has only countably many non-logical constants; that is, in our standard case one binary predicate R.
254
DANA SCOTT AND PETER KRAUSS
Indeed, the counter-example concerning symmetric probability models given in Section 6 can be constructed in 2("1) if we allow non-denumerably many unary predicates in 2«"). (2) The probability system
{nEQ:n(t m ) = n(t"n = U({nEQ:n(l m ) = i<}'
Thus
i
Consequently tIJ;.(t m
tJ n {nEQ:n(t,,) = tJ).
= (II) =
j' fl
In
(n(l m ) = n(/,,)) dX(rr)
~ .I
1<;
A({/;W > O.
If
We conclude our discussion of the finitary language 2("1) with an immediate consequence of Theorem 6.7. 7.16. The set ofprobability laws of 2'("1) is recursively enumerable. Proof: In the remarks following the proof of Theorem 6.7 we can, for the finitary language 2("1), replace "effectively generate" everywhere by "recursively enumerate". This yields a proof of Theorem 7.16. THEOREM
8. Examples. We have reason to hope that the results of probability logic may have useful applications to deductive logic, inductive logic and to probability theory. The first point was illustrated by Ryll-Nardzewski's example of a complete theory without models. The second point is rather obvious, as a matter of fact our work originally started with a study of Carnap's inductive logic. We will illustrate the third point by considering
ASSIGNrNG PROBABrLrTIES TO LOGICAL FORMULAS
255
some well-known measurability problems in the theory of stochastic processes with non-denumerable index sets. Let ~ be the two point-compactification of the set of real numbers; that is, the set of real numbers together with the points - 00, 00. Let (\) be the set of rational numbers. Let f!jj be the a-field of Borel-sets of ~. Let T be an index set, and we will choose Ts; ~. Let f!jjT be the product a-field of subsets of the Cartesian product space ~T induced by f!jj. As is well-known, a stochastic process with index set T may be identified with a probability space <~T, f!jjT, m), where m is a probability on f!jjT. (For further details see e.g., Loeve [1960] p. 497 ff.) Let ~ be the a-field of Borel sets of ~T; that is, the a-field of subsets of ~T generated by the closed sets of the product topology on ~T. Then ,oljT r;;;;.~, and f!jjT =I=~ if T is non-denumerable. During the investigation of stochastic processes one frequently would like to assign probabilities to sets Xr;;;;. ~T which are not m-measurable; that is, X ¢f!jjT. It is well-known that this can always be done with finitely many sets at a time. Thus if X o, ... , X n - 1 r;;;;. ~T, then m can always be extended to a probability on the a-field generated by f!jjT U {X o, " " Xn-d (see, e.g., Halmos [1950] p. 71). The extension is not unique, however. In general, given a a-field d2f!jjT, the question arises of whether there exists a probability n on d which extends m. Moreover, one attempts to specify convenient conditions which render such an extension unique. Nelson [1959] investigates this question for ~ and gives a sufficient condition for the existence of a uniquely determined extension. He also shows that many interesting sets belong to~. Another extension result is Doob's Separability Theorem (Loeve [1960] p. 507). Let .# be the a-field generated by f!jjT and sets of the form {XE~T:X(t)EC}, where Ir;;;;.~ is an open interval and Cr;;;;.~ is closed.
n
tElnT
Then Doob's Theorem says that every probability m on f!jjT has an extension to .Pi'. Moreover, the extension n may be assumed to be separable; that is, there exists a denumerable subset ss: T such that for all open intervals I and closed sets C, n(
n
tEInT
{XE~T:x(t)EC})=n(
n
tEIns
{XE~T:X(t)EC}).
S is called a separating set. The separability condition makes the extension unique. Upon closer inspection the separability condition turns out to be an instance of a "Gaifman Condition." This will appear more clearly during the later development of our example. Indeed, it seems that from the earliest investigations of the extension problem conditions for the "reasonableness"
256
DANA SCOTT AND PETER KRAUSS
of extensions of probabilities on B?JT have been proposed which strikingly resemble particular instances of a "Gaifman Condition" (see, e.g., Doob [1947]). Thus one might be tempted to put down a Gaifman Condition on extensions of probabilities on B?JT to a certain rr-field d~B?JT which renders such extensions unique, provided they exist, and then to investigate the problem of the existence of probabilities satisfying this condition. Our example will point in this direction. Of course, a Gaifman Condition is most conveniently stated in terms of a language rather than in terms of certain representations of sets. For our example we use the infinitary language if. As non-logical constants of if we provide a binary predicate <, and for every qe (j) a unary predicate P q • Moreover we augment if by a set of individual constants which, for convenience, we choose to be the index set T~ IR. Let S be the set of relational systems of the similarity type of if(T). We embed IR T pointwise into S. For xEIR T define 'llxES as follows: 'llx=
8.1. For all t , t'ET, qEQ and (S;W,
.
(I) M(t
= t) il ,
IR
T
T
~ IR if t
= ) (1\;/'
= t'
\l'Iljt=l=t
'
T
< 1') n IR T = , IR if t < t' (0 if' t-c t' T (iii) M(Pq(t)) n IR = {XE IRT:x(t) S; q} (iv) M(II.{J) il IR T = IR T ~ M(I.{J) (ii)
M(t
(v) M( V (Pi) i<~
n IR T = U M(l.{Ji) n IR T
(vi) M(!\ l.{Ji) n IR = T
i<~
(vii) M(:! VI.{J) n IR T
n M(I.{J;) n IR
i<~
T
i<~
=
(viii) M(V VI.{J) n IR T =
U M(I.{J(t)) n
rET
n M(I.{J(t)) il
tET
IR
T
IR
T
.
ASSIGNING PROBABILITIES TO LOGICAL FORMULAS
257
The operation of restriction to a subset is a complete homomorphism of fields of sets. By Lemma 8.1, (i)-(vi), this homomorphism maps {M(qJ):qJEd(T)} onto f18T. Accordingly, we define for all qJEJ(T),
11m (M(qJ)) = m (M(qJ) n lR T) . We obtain a probability 11m on {M(qJ):qJEd(t)} and thus also on d(T)/f-. We write /lm(qJ) for Ilm(M (qJ)) and obtain (T, 11m) as a probability system. We are interested in the a-field d = {M(qJ) n [RT: qJEY(T)}. By our remark above f18T S; d, and we will see later that d contains a vast assortment of interesting sets. First we must complete our series of definitions. Suppose n is a probability on d which extends m. Define for all qJEY(T),
vn(qJ) = n(M(qJ) n
[RT).
We say that n satisfies the Gaifman Condition if Vn satisfies (G). We thus have as an immediate corollary of Theorem 4.3: THEOREM 8.2. For every probability m on f18T there exists at most one probability on d which extends m and satisfies the Gaifman Condition.
This settles the uniqueness part of the extension problem, the existence part is of course much more difficult. Consider the probability system (T, 11m) induced by m. It is well-known that the equation n(M(qJ)n lR T )= 11= (M(qJ)) for all qJEY(T) defines a probability non d iff whenever qJE yeT) and [RT S; M (qJ), then Il~ (qJ ) = 1. Indeed, just in this case n is a well-defined set function on d (see, e.g., Halmos [1963] p. 65). This leads to: THEOREM 8.3. Let m be a probability on f18T. Then there exists a probability on d which extends m and satisfies the Gaifman Condition iff whenever qJEY(T) and [RT S;M(qJ), then 1l~(qJ)= 1.
The authors have been able to show that not every probability m on f18T has a Gaifman extension to d. A counter-example can already be produced with the case of dependent Bronoulli trials. In this case the stochastic process is two-valued, the space [RT collapses to 2 T , and in our language !E we only need one unary predicate P (together with -c , of course). Further M(P(t)) n 2 T = {xE2 T : x(t)= 1} so that pet) means "success at time t", For qJ we choose a finitary sentence which says "P has a least upper bound" as follows:
3 Va [\I Vt [P(v t ) -
V t S;
va] /\ \I Vt [\I Vz [P(v z) --+ Vz S; Vt]
--+ Va S; VI]].
258
DANA SCOTT AND PETER KRAUSS
Then 2Ts;M«p), but a measure m may be defined so that tl~(
nc
eI
t
T {XEIR T :x(t)sq}=M ( \lvo[t[Pq(vo)J ) n fR,
and the separability condition amounts to the Gaifman Condition (G) for sentences of the form \lvo [II Pq(V o)], where qEtD. (Note that the uniformity of the separating set S is really no strengthening of the Gaifman Condition since both the open intervals and the closed subsets of fR have countable bases.) Doob's Separability Theorem now provides a positive answer to our second question. If we let .'71' be the o-field generated by :J?JT and sets of the form M(\lvO[t l P,/vo)])n fRT, where qEtD, and 1(, 1z E T, then every probability on .OJ T has a Gaifman extension to ss", Indeed the authors have been able to prove the corresponding condition of Theorem 8.3, which yields a rather unorthodox proof of the Separability Theorem. Moreover, it can be shown that .'7/' is in a certain sense maximal; that is, Doob's Theorem is about the best result we can expect. A detailed discussion of these results would take us too far afield here. On the other hand it is necessary to point out that the o-field .'7/ is highly unorthodox in terms of the traditional notions of probability theory. S'~ is described by means of the infinitary language 2, and the nested application of Boolean operations combined with quantifiers has no clearly discernible analogue in usual classical methods of generating o-fields. However, we can show that many interesting subsets of [RT which have traditionally been considered belong to c'7i. The striking feature of our approach is that we can show this by just writing down the ordinary definitions of these sets in the language 2. We give a few examples by first describing the set M(
ASSIGNING PROBABILITIES TO LOGICAL FORMULAS
259
(1) The set of non-decreasing functions:
VVoVVI [vo S VI
-+
1\ [Pq(vI)-+Pq(V o)]].
qeQ
(2) The set offunctions assuming a maximum:
3v,Vvo 1\ [Pq(v,)-+Pq(v o)]. qeQ
(3) The set offunctions assuming at most n different values:
3vo···3vn_,Vvn V [/\ Pq(Vi)~Pq(Vn)]. i
qEQ
In the following examples the sentence
ApPENDIX by Peter Krauss
A measure-theoretic generalization of the Rasiowa-Sikorski Lemma In Rasiowa and Sikorski [1950] the following theorem is proved, which now is generally known as the RASIOWA-SIKORSKI LEMMA:
260
DANA SCOTT AND PETER KRAUSS
Let <:YJI, /\, v, ,....,) be a Boolean algebra, let bE!!J such that b # 1, and for every m-eso let 'Cm~!!J be a subset such that 1\ 'Cm exists in !!J. Then there exists a prime ideal jl in !!J such that (i) bEjl; (ii) for every m-c.co, 1\ 'CmEjl ifffor some CE'Cm, CEjl.
An immediate consequence is the following relativised version: THEOREM 1. Let
(iii) If ~ ft· Proof: Apply the Rasiowa-Sikorski Lemma to the quotient algebra B/['!]. We prove the following measure theoretic generalization of Theorem 1: 2. Let <.oJJ, r., v, ~) be a Boolean a-algebra, let d~!!J be a a-subalgebra, and let 11 be a a-additive probability measure on d. Let v' be a finitely additive probability measure on !!J such that v' (x) = 11(x) for all XEd, let XO, ... ,xn_tE/!IJ and e>O. Finally, for every m, n-c co let bmnE!!J. Then there exists a finitely additive probability measure v on !!J such that THEOREM
(i) Iv(x;) - v'(x;)1 < e for all i < n;
(ii) for every m < co, v( 1\ b mn) = lim v( 1\ b m;); n<w
(iii) vex)
= 11 (x)for all XEd.
n-e co
i
Throughout the rest of this appendix we assume that
3. Let bEfJIJ~,r#. Then [dU {b}]={(xn b)U (y~b):x,YEd}.
261
ASSIGNING PROBABILITIES TO LOGICAL FORMULAS
4. Let
and let a, cEd such that aszbs;« and fl(a)= sup {fl(X):x<;;;.b, xEd}, fl(c)=inf {fl(X):b<;;;.x, XEd}. Let d=c~a, and let rx, B:» 0 be real numbers such that a + jJ= 1. Define for x, y Ed: LEMMA
v((X
n b)
bE:!lJ~d
U (y ~ b)) = fl((X
n a)
U
(y
~
c)) + rxfl(X n d) + Pfl(Y n d).
Then v is a a-additive probability measure on [dU {b}] such that V(X)=fl(X) for all XEd.
The next lemma is also known: LEMMA 5. Let bEIJi ~ d and let v be a finitely additive probability measure on [dU {b}] such that v(x)=j1(x)for all XEd. Then v is a-additive. Proof: Let xnEd, n-cto be a decreasing sequence. Then V(( 1\ x n) n<w
Suppose
n b):S;
v(( 1\ x n) n b) < lim v(x n n b). n<w
Then
lim v(x n n b).
n-->oo
n--> 00
fl( 1\ x n) = v( 1\ x n) = v(( 1\ x n ) n<w
n<w
n<w
n b) + vH 1\
< lim v(x n n b) n-e co
+ lim
n<w
x n ) ~ b)
v(x n ~ b)
n-e co
contradicting the hypothesis that j1 is (J-additive. Lemma 3 now proves Lemma 5. LEMMA
6. Let bnEIJi, n<w be a decreasing sequence such that 1\ bn=O. n<w
Then there exists a finitely additive probability measure v on IJi such that
(i) lim V (bn ) = 0; n--> 00
(ii)
V (x)
=
fl(x)jor all XE d.
Proof: Define by recursion: do=d, d n+ 1 = [dnU {b n}], for n «:co. By Lemma 4 we see that without loss of generality we may assume that bnrfdm for n-cco, For every n « co choose anEd such that a.s:b; and fl(an)=sup {fl(X):x<;;;.b m xEd}. Since b; is decreasing we may assume that an is de-
creasing. By recursion we wish to define a o-additive probability measure Vn on d n such that Vo=fl and for n-c co, Vn+ 1 (X) = Vn(X) for every XEdno Suppose vn has been defined on d no To define vn+ 1 on slfn+ 1, let a~, c~ Edn be
262 such that
DANA SCOTT AND PETER KRAUSS a~<;;;bn<;;;c~ and
vn(a~) = sup{vn(x):x <;;; bn,xEdn} vn(c~)
= inf{vn(x):b n <;;;
x,xEsi'n}'
Since an
We prove by induction on n: For every n-c co, xEd",
By definition, vn+l(xnbn)=vn(xna~). In case n=O, let xEd o. Then n a;) E .91 and by the definition of a~,
x
Now suppose for every
XES~n'
vn+l(xn bn)=sup{,u(z):z<;;;xn b",zEd}. We first show: vn+2(bn+I)=Vn+2(an+I)' In fact, by definition, vn+2(bn+I)= Vn+l(a~+I)' By the induction hypothesis, Vn+ 1 (bn)=,u(an)=v n+ 1 (an)' Since a;,+ I <;;;bn+l <;;;b", Vn+ 1(a~+ 1)= vn+ 1(a~+ 1 n an). Since a~+ 1Ed n+ 1 and a~+ 1 <;;; b", a~+ 1= X n b; for some XES'i'n" Thus vn+ 1 (a~+ 1)= Vn+ 1((X nan) n bn), where x n anEs¥n" By these induction hypothesis,
Furthermore an+ 1 <;;;a;, + 1 <;;;bn+ 1 and an+ 1<;;;an" Thus an+l<;;;a~+lnan= (x nan) n b; <;;; b; + I' Therefore, by the definition of an+ I ' Vn+ 1 ((x nan) n bn) = ,u(a/!+I)' This proves vn+2(bn+I)=Vn+2(an+l)' Now let
XE'#n+I'
Then
In fact, I'/! + 2 (b/! +1) = Vn+ 2 (a n+ I) and an+ 1 <;;; bn+ I- Therefore vn+ 2 (x n b/!+ I) = vn+2(x n a/!+ I)' Since x n a n+ 1 E'#n+1 and x n an+ 1 <;;;b n+ 1
xnbn+I,Vn+2(xnbn+I)'::;;sup {,u(z):z<;;;xnb n+ l, ZEd}. Clearly Vn+2(xn bn+1)2SUp {Jl(z):z<;;;xn bn+l> ZE.r:i}, which completes the inductive proof. As a direct corollary we obtain: For every n « co, vn+l(bn)=,u(an). Now
define a finitely additive probability measure v on the subalgebra
U
n<w
d
n
263
ASSIGNING PROBABILITIES TO LOGICAL FORMULAS
by v(x)=vn(x) for xEdn, and extend v to a finitely additive probability measure on BO. Then lim v (bn)= lim jl(an)=O since /\ an£ /\ bn=O. This n-e co
proves Lemma 6.
n<w
n-s co
n<w
Now we proceed to prove Theorem 2. C (X) with the sup-norm is a Banach space. Any finitely additive probability measure v on BO uniquely extends to a linear functional on C (X) such that (i) v (x) S Ilxll for all XE C (X); (ii) v(l)=1. And, conversely, any linear functional on C (X) satisfying (i) and (ii) uniquely determines a finitely additive probability measure on BO. Let C (X)* be the conjugate space of C (X) and consider the weak star topology for C (X)*. For q:>EC(X)*, x o, ... , Xn+1EC(X), and e>O, let N(q:>;x o, ""Xn-l ;e) = {t/tEC(X)*: 1t/t(xJ - q:>(xJI < ef or i < n}
M .... = {q:>EC(X)*: 11q:>11 S I,q:>(x) = jl(x)for all xEd}.
By the Alaoglu Theorem, M .... with the weak star topology is a compact Hausdorff space. LEMMA
7. Let bnEBO, n<w be a decreasing sequence such that /\ bn=O. n
For every r> 1, let Pr={q:>EM.... :III's lim q:>(bn)}. Then P, is nowhere dense in
M .....
Proof: Since Pr=
some q:>EC (X)*,
n {q:>EM.... : I/rsq:>(bn)}, P, is closed in M ..... Suppose for
n
X O, ... ,
Xn- 1 EC (X) and e>O, we have
By the Stone-Weierstrass Theorem there exist vo, ... , vn - 1 EL such that Ilxi- Viii ' be the restriction of q:> to M. By Lemma 5, tp' is c-additive on [d U{co,···, c.; -I}]' By Lemma 6, tp' has an extension t/t to C (X) such that t/tEM .... and lim t/t(bn)=O. Let i-c.n, Then q:>(V i) = q:>'(Vi) = t/t(v;). Thus Iq:>(x i )
-
t/t(xi)1
s s
+ )t/t(Vi) - t/t(Xj)J + 11t/tIIIIX i - Viii < e.
1q:>(xJ - q:>(vJI II q:> II
II Xi
Thus t/t E P, which is a contradiction.
-
viii
264
DANA SCOTT AND PETER KRAUSS
Now consider the hypothesis of Theorem 2. It clearly suffices to assume that for every m < ill, o.: is a decreasing sequence for n < ill such that /\ bmn=O. For every m « co, r211et
n<w
By Lemma 7, p=
U U
m<
W
r2:: 1
P, is of first category in M ...... Since the set
N(v'; x o, ... , x n - I ; e) n M.<J1 is a non-empty open set and M ..... is a compact Hausdorff space, Theorem 2 follows by the Baire-Category Theorem. References DOOB, J. L., 1947, Probability in function space, Bull. Am. Math. Soc., vol. 53, pp. 15-30 DUNFORD, N. and J. T. SCHWARTZ, 1958, Linear operators I (Interscience, New York) EHREN FEUCHT, A. and A. MOSTOWSKI, 1961, A compact space of models of axiomatic theories, Bul!. Acad, Pol on. Sci. Sec. Sci. Math., Astron. Phys., vol. 9, pp. 369-373 FENSTAD, J. E., A limit theorem in polyadic probabilities, Proc. Logic Colloquium in Leicester, August 1965 (forthcoming) GAIFMAN, H., 1964, Concerning measures on first-order calculi, Israel J. Math., vol. 2, pp.I-18 HALMOS, P. R., 1950, Measure theory (Van Nostrand, New York) HALMOS, P. R., 1963, Lectures on Boolean algebras (Van Nostrand, New York) HENKIN, L., 1949, The completeness of the first-order functional calculus, J. Symbolic Logic, vo!. 14, pp. 159-166 HEWITT, E. and L. SAVAGE, 1955, Symmetric measures on Cartesian products, Trans. Am. Math. Soc., vo!. 80, pp. 470-501 HORN, A. and A. TARSKI, 1948, Measures in Boolean algebras, Trans. Am. Math. Soc., vo!. 64, pp. 467-497 KARP, C. R., 1964, Languages with expressions of infinite length (North-Holland Pub!. Comp., Amsterdam) KELLEY, J. L., 1959, Measures in Boolean algebras, Pacific J. Math., vol. 9, pp. 1165-1177 LOEVE, M., 1960, Probability theory (Van Nostrand, New York) Los, J., 1962, Remarks on foundations of probability, Proc. Intern. Congress of Mathematicians, Stockholm, 1962, pp, 225-229 Los, J. and E. MARCZEWSKI, 1949, Extension of measure, Fundamenta Mathematicae, vol. 36, pp. 267-276 NELSON, E., 1959, Regular probability measures on function space, Ann. Math., vol. 69, pp.630-643 RASIOWA, H. and R. SIKORSKI, 1950, A proofof the completeness theorem ofGiidel, Fundamenta Mathematicae, vol. 37, pp. 193-200 SIKORSKI, R., 1964, Boolean algebras (second ed., Springer Verlag, Berlin)
PROBABILITY AND THE LOGIC OF CONDITIONALS* ERNEST W. ADAMS University of California, Berkeley, California
1. Introduction. The purpose of this paper is to give a rigorous mathematical foundation for a theory of the logic of conditionals based on probabilistic concepts, which has been informally presented in Adams [1965]. This theory involves a formal calculus analogous to the propositional calculus, for symbolizing conditional statements and their truth-functional components, and gives rules for determining the formal validity of inferences symbolized within the calculus. The objective of setting up this calculus is to give a more adequate representation of inferences involving conditionals than does the propositional calculus. Thus, many writers, e.g. Lewis [1932], Belnap [1960], Belnap and Anderson [1962], and Angell [1962] have felt that the fact that the propositional calculus allows the 'fallacies' of material implication (from 'not p' to infer 'if p then q', or from 'q' to infer 'if p then q') as valid inferences shows that this calculus does not adequately represent the laws of the ordinary English 'if then', and they have proposed alternative calculi aiming to better represent the logic of conditionals. The calculus I will develop here has the same objectives as those of these other writers - only I hope to show that it has several advantages over previously given calculi for representing the logic of English conditionals. Before going into the details of our calculus, or discussing its advantages, I shall briefly describe the main theses upon which the formal theory rests. The main philosophical ideas upon which our theory is based will be stated here without much discussion or defense, for which the reader may refer to Adams [1965]. My first contention is that an adequate understanding of inferences involving conditionals must take into consideration other things besides their truth conditions. This is connected with the fact that in ordinary parlance the words 'true' and 'false' have no unambiguous sense as applied to
* This work was supported in part by National Science Foundation Grant GS-824, and was carried out during the tenure of a Fellowship at the Center for Advanced Study in the Behavioral Sciences.
266
ERNEST W. ADAMS
conditional statements whose antecedents prove to be false. Without getting involved with the tricky question as to whether one can repair the lack of definition of the terms 'true' and 'false' by supplying a satisfactory general criterion of truth which would apply to conditionals, the lack of a clear ordinary definition suggests that it may be more profitable not to attempt to characterize validity of inferences involving conditionals by the requirement that it should be impossible for the premises of the inference to be true but the conclusion to be false. My second and more positive contention is that in analyzing the validity of inferences involving conditionals, one must consider what [ vaguely characterize as conditions ofjustified assertability for the statements involved. Some points of contrast between justified assertability and truth are the following. First, 'truth' suggests verification or tests to determine truth, and tests must be performed at some time and, usually, on some things. Normally, the things upon which verification tests are performed are things referred to in the statement under test, and the test is performed at a time indicated by the tense of the statement, or explicitly referred to in the statement (thus, to verify a statement like 'John will get married tomorrow', one must wait until tomorrow, and then observe John). In contrast, the conditions of justified assertability are ones which prevail at the time of the making ofthe statement, and depend on such things as the authority of the speaker to say what he says at the time he says it. Though there is certainly a connection between justified assertability and truth, one is usually neither a necessary nor a sufficient condition for the other. In particular, one may be well justified in making a statement which ultimately proves to be false, and one may also be completely unjustified in making a statement which ultimately proves to be true. The foregoing observation leads immediately to my third main thesis. The extent to which one is justified in making a statement on an occasion is a matter of degree. And, if it is the case that one may be very well justified in making a statement even when there is non-negligible likelihood that the statement may eventually prove false, this suggests that it may be fruitful to bring probabilistic ideas explicitly into consideration in developing a theory of the logic of conditionals. That is, I propose to replace the vague and unquantified notion of 'justified assertability' by that of 'high probability' (i.e. probability very close to 1). This leads in turn to formulating an alternative criterion of 'reasonableness' for inferences, in place of the criterion of validity stated in terms of truth-conditions, as follows: an inference is reasonable just in case it is impossible for its premises to have high probability while the conclusion has low probability. Thus, we arrive at the criterion for reason-
PROBABILITY AND THE LOGIC OF CONDITIONALS
267
ableness of inferences simply by substituting 'high probability' for 'true' in the classical definition of 'validity'. As it turns out, this substitution makes no difference in those inferences of the propositional calculus which do not involve conditionals, the intuitive reason being that, where 'true' is clearly defined, 'high probability' is the same as 'high probability of being true'. The close connection between high probability and truth which exists for ordinary unconditional statements does not prevail with conditionals. This is a consequence of my fourth basic thesis, which is that the probabilities of conditional statements are to be construed as conditional probabilities. The conditional probability of the statement 'if p then q' can easily be seen to be high just in case it is much more likely that p and q are both true than that p is true and q is false. But this condition is quite different from one requiring that the probability that 'if p then q' is true (where truth is connected with tests) is high. The fact that the close connection between probability and truth which exists for unconditional statements fails to hold for conditionals partly explains, I think, why it is that the propositional calculus gives a more adequate representation of logical relations among nonconditional statements than it does among conditionals. The main elements of the present theory may be summed up as follows: I propose to replace the criterion of 'validity' stated in terms of truth conditions by a criterion of 'reasonableness'. This criterion requires that the high probability of the premises of a reasonable inference must guarantee that the conclusion also has a high probability. And, finally, the probabilities of conditional statements are taken to be conditional probabilities. With this informal exposition, we are in a position to proceed to the development of a precise mathematical formulation and analysis of these ideas. Before doing that, though, it may help to motivate what follows to describe some of the evidence which can be brought in support of our theory, showing how it helps to explain many facts concerning the logic of conditionals, and even predicts some heretofore unsuspected phenomena. First, it is easy to explain in the light of the theory why it is that the 'fallacies' of material implication do strike people as fallacious. Why should we be reluctant to infer from 'It will not rain tomorrow' the conclusion 'If it does rain tomorrow then the game will be played'? One good reason, at least, for our reluctance is surely this: we can only assert 'It will rain tomorrow' with a high probability, but this high probability does not by any means entail a high probability for 'If it does rain tomorrow then the game will be played'. At any rate, this conclusion would certainly not be a safe one to draw from the assertion, and our intuitive recognition of this fact explains, I
268
ERNEST W. ADAMS
think, why we regard the inference as questionable. Note that in arguing in the foregoing way we have taken explicitly into account the situation of the persons making the statements in question, and of their hearers, and not just the things the statements are about (tomorrow's weather, and the game). A second bit of evidence supporting the theory is that it predicts that some of the rules of inference involving conditionals which have so far been accepted as valid are not in fact universally valid, and that it should be possible to find instances of premises and conclusions conforming to these rules such that it would not be reasonable to infer the conclusions from the premises. Several examples of 'exceptions' to usually valid rules are given in Adams [1965], and I select one for illustration - a counter-example to show that the law of Hypothetical Syllogism does not always yield reasonable inferences. The counter-example to this rule of inference (to infer 'if p then r' from 'if q then r' and 'if p then q') is the following: If Smith wins the election then Brown will retire to private life. If Brown dies before the election then Smith will win the election. Therefore, if Brown dies before the election then he will retire to private life. Of course, it may be argued that this example and ones like it represent the kinds of logical oddities which formal logic may properly ignore, or which may be handled in ad hoc ways and don't require to be taken into account in a general theory. The point in favor of the present theory, however, is that it predicts the existence of such examples, and yields considerable information to help in the search for them (by telling what kinds of probability relations must subsist between 'p', "q' and 'r' and their compounds if it is to be the case that 'if q then r' and 'if p then q' have high probabilities, but 'if P then r' has a low probability). The third kind of evidence supporting the theory is that it yields an explanation as to why certain rules of inference involving conditionals are usually accepted without question, even though they 'have exceptions' (if the theory is correct), whereas other rules of inference (e.g., those associated with the fallacies of material implication) are immediately seen to be fallacious. One example is the logical law connecting disjunctions with conditionals (to infer 'if p then q' from 'either not p or q' and vice versa), which the theory shows to have exceptions, though it is usually unchallenged. Indeed, if this law were universally valid, then this would justify treating the English 'if then' as a material conditional for the purposes of logical analysis. What the present theory shows is that inferring 'if p then q' from 'either not p or q' is not always reasonable, but that the only situation under which 'either not p or q' has a high probability but 'if p then q' has a low one is the situation in
PROBABILITY AND THE LOGIC OF CONDITIONALS
269
which 'not p' has a high probability. Assuming this, we have an immediate explanation of why we are ordinarily willing to infer 'if p then q' from 'either not p or q': the reason is that people do not ordinarily assert a disjunction when they are in a position to assert one of its members outright (in fact, it is misleading to do so, and therefore doing it probably runs against strong conventions for the proper use of language). Thus, if one heard it said that 'either the game will not be played tomorrow, or the Dodgers will win' he would be well justified in inferring 'if the game is played tomorrow, then: the Dodgers will win', and what would justify the inference would be the knowledge that the person asserting 'either the game will not be played or the Dodgers will win' did not do so simply on the grounds of information he had to the effect that the game would not be played. Similar analysis explains why the laws of Contraposition and Hypothetical Syllogism, both of which have exceptions, nevertheless are reasonable in the loose sense that if conventions of proper language use (e.g., don't assert a disjunction when you are in a position to assert one of its members outright) are adhered to, then on any occasion on which the premises are asserted with high probabilities the conclusions must also have high probabilities (whereas no such conventions justify the 'fallacious' inferences of material implication). We now turn to the formal development of the theory. This formal theory is not here being offered as a serious rival to standard formal logic. As will be seen, the formalism involves many questionable and arbitrary features, and it certainly falls far short of current formal logic in scope and power (in spite of its advantages in handling conditionals). On the other hand, it seems to me desirable to attempt to develop a precise theory even though it is highly provisional, since in this way we may hope to bring to light difficulties which would otherwise go unnoticed, and to get some indications which could lead to improvements. 2. Basic concepts. Three formal concepts are basic to the present theory: those of a formula (and that of a language), of a probability function for a language, and of reasonable consequence which is a relation holding between sets of formulas and formulas. The formulas we work with are intended to symbolize ordinary truth-functionally analyzable propositions, and conditionals whose antecedents and consequents are truth-functional propositions. Formal truth-functional compounds are constructed just using the operations for conjunction, disjunction and negation (the symbols for which are' &', ' v " and' - " respectively), and conditionals are constructed using the arrow, ' ..... '. As a technical convenience I also include two distinguished
270
ERNEST W. ADAMS
formulas, 'T' and' F', among the atomic formulas, which are interpreted as a logical truth and a contradiction, respectively. One important limitation of this formalism, distinguishing it for the ordinary propositional calculus, is that the arrow can occur only as the main connective of a formula, and therefore truth-functional compounds of conditionals, as well as iterated conditionals, cannot be symbolized within the formalism. The general reason for imposing this limitation is to insure that all formulas of the language can have probabilities assigned to them which obey the axioms of the standard calcul us of conditional probability, which does not apply in its usual form to compound expressions with conditional components. That this restriction is somewhat more severe than necessary is probably obvious: in fact, formal denials and 'quasi-conjunctions' of conditionals will be introduced later as meta-linguistic abbreviations, and, as Patrick Suppes has pointed out, conditionals with conditional consequents might be introduced with the assumption that they obey a probabilistic version of the law of Importation. Not all 'truth-functional' compounds of conditionals are so easily handled, however (see Adams [1965] for informal discussion of some problems arising with disjunctions of conditionals, 'only if' statements, and conditionals with conditional antecedents) and so it seems somewhat easier at the outset to impose a blanket restriction excluding all compounds with conditional components, rather than attempt to formulate complicated rules specifying just which compounds containing conditionals can have probabilities assigned, and which cannot. 1 1.1. A formula is any expression of one of the following three kinds: i) atomic - 'T', 'F', and all lower case Latin letters, with or without numeral subscripts; ii) truth-functional - all atomic formulas and expressions constructable from them using the binary connectives' &' and' v " and the unary connective' - ' ; iii) conditional - all formulas of the form qJ-'>t{J, where qJ and t{J are truth-functional formulas. 1.2. Let o: be a set of atomic formulas including 'T' and' F'. The language of ct is the set of all formulas which include only atomic formulas in ct. A language is a set of formulas which is the language of some set, ct, of atomic formulas including 'T' and 'P'. In what follows, I will ordinarily use lower case Greek letters as variables ranging over truth-functional formulas, use capital Latin letters from 'A' DEFINITION
PROBABILITY AND THE LOGIC OF CONDITIONALS
271
through 'D' as variables ranging over conditional formulas, and use capital Latin letters'S' - '2' as variables ranging over sets of formulas. I will speak freely of a formula or set of formulas, S, tautologically implying a formula A, even where S and A may include conditional formulas. In applying the concepts of tautology and tautological implication to conditional formulas, it will be understood that they are to be regarded as material conditionals for the purpose of the application. Thus, I will say that 'p' tautologically implies '-p--+q,' though the inference is not reasonable. An abbreviation which we will use informally is to write 'cp -l/J', in place of 'cp & -l/J'. It is convenient to introduce at this point some auxilliary concepts and notation applying to formulas and sets of formulas, even though most of it will only be employed in later sections. Some metalinguistic notation is defined first. DEFINITION 2. Let A be a formula. 2.1. If A=l/J then ~A= -l/J; if A=cp--+l/J, then -A =cp--+ -l/J. 2.2. If A=l/J then Ant(A)=T, Cons (A) = l/J and Cond(A)=T--+l/J; if A=cp--+l/J then Ant(A)=cp, Cons(A)=l/J, and Cond(A)=cp--+l/J. What Definition 2.1 does is to introduce a kind of metalinguistic negation operation, which is the same as the ordinary negation as applied to truthfunctional formulas, and which, as applied to a conditional cp--+l/J, yields the 'contrary conditional', cp--+ -l/J. Denials of conditionals could have been included directly in the object language, but putting them into the metalanguage emphasizes the fact that these are to be regarded as abbreviations, and are not to be interpreted as symbolizing English statements having the form of denials of conditionals. Definition 2.2 introduces formal symbols for the antecedents and consequents of conditionals. It also introduces a kind of formal conditional T--+l/J associated with a truth-functional statement, ljJ. It will be seen that the formulas l/J and T --+ljJ are interchangeable in all of their logical relations (including ones of reasonable inference), and it is convenient to be able to reduce everything to just the consideration of relations among conditionals, by replacing any truth-functionalljJ by T --+l/J. The next series of notions to be defined apply only to finite languages: i.e., to languages, 2, generated by finite sets of atomic formulas. As is well known, these languages are 'atomic', in the following sense. There is a set, SD2, of 'state descriptions for 2', having the following properties: (1) SD2 is a finite set of consistent truth-functional formulas of 2, (2) any two distinct members ()( and f3 of SD2 are tautologically inconsistent, and (3)
272
ERNEST W. ADAMS
every truth-functional formula cp of 2 which is not tautologically equivalent to F is tautologically equivalent to a (unique) disjunction offormulas of SD2. These SD-sets for finite languages are constructable in well known ways, and [ will assume without giving the explicit construction that each finite language, 2, has associated with it a uniquely specified SD-set which will be called 'the SD-set of 2'. This assignment of SD-sets to finite languages is assumed in the following definition. DEFINITION 3. Let 2 be a finite language, let SD be the SD-set of 2, let (X be a member of SD, let S be a finite set of formulas of 2, let A be a formula of 2, and let Ant(A)=cp and Cons(A)=t/J. 3.1. (X belongs to tp if and only if (X tautologically implies tp, 3.2. SD (A) is the set of all (X in SD belonging to cp; SD (A) is the set of all (X in SD belonging to cp-t/J. 3.3. SD (S) is the union of all SD (B) for Bin S; SD (S) is the union of all SD(B) for B in S. 3.4. S is null if and only if SD(S)=SD(S). The significance of functions SD(S) and SD(S) will only become apparent later. For the moment we may note the following. Strictly speaking, the concepts introduced in Definition 3 should all be relativized in the particular language 2. We are being somewhat informal in not making this relativization explicit in the notation. In future developments, ambiguity will be avoided by always specifying which language is under consideration. In the case of the concept 'null', the dependence on the language is only apparent, since a set S of formulas is null relative to one language if and only if it is null relative to all (finite) languages. Probability functions for languages are defined next, using essentially the Kolmogorov axioms for conditional probability, with one important modification. The modification consists in assigning probability 1 to all conditional formulas whose antecedents have probability O. The justification for this procedure is the following. Assignment of probability 1 to a statement can be justified if it can be shown that the statement is perfectly 'safe', and in Adams [1965] 'safe' is interpreted to mean 'safe in a betting sense'. Now, any bet on a conditional is itself conditional, and is called off in case the antecedent 'condition' fails to be satisfied. If the antecedent condition has probability 0, then the bet is certain to be called off, and therefore the bet (and, by implication, the statement) is perfectly safe in the sense that it is practically sure not to be lost (though it is equally sure not to be won). The foregoing argument to justify assigning probability 1 to conditionals whose antecedents
273
PROBABILITY AND THE LOGIC OF CONDITIONALS
have probability zero is certainly not conclusive: for the present it is being used as a heuristic justification for making some disposition of these troublesome cases. The arbitrariness of this disposition shows the need to consider the consequences of alternative solutions. One such alternative is in fact discussed in Section 9, where it is shown that the alternative does lead to somewhat different rules for 'reasonable inference', but the two systems of rules are related in a very simple way. DEFINITION 4. Let se be a language. 4.1. A probabilityfunction for se is a real-valued function P with domain se such that: for all truth-functional formulas
°
274
ERNEST
W.
ADAMS
namely, that it should be possible to guarantee arbitrarily high probability for A by assigning sufficiently high (but less than 1) probabilities to the formulas of S. We shall see later that this requirement is equivalent to the following even weaker requirement: that if A is reasonably inferable from S, it should not be the case that arbitrarily high probabilities for formulas in S are compatible with arbitrarily low probability for A. In addition to the notion of 'reasonable consequence', a concept of 'strict consequence' is also defined, according to which A is a strict consequence of S if and only if the fact that all formulas of S have probability 1 guarantees that A has probability 1. As might be expected, strict consequence proves to be equivalent to tautological consequence, and hence, in fact, is a less stringent requirement than reasonable consequence. DEFINITION 5. Let 2' be a language, and let S and A be, respectively, a set of formulas of 2', and a formula of .!e. 5.1. A is a reasonable consequence of S (in symbols, SIf-A) if and only if for all s > 0 there exists b > 0 such that for all probability functions P for 2', if P(B»1-b for all B in S, then P(A»1-e. 5.2. A is a strict consequence of S (in symbols, SI- A) if and only if for all probability functions P of .!e, if PCB) = 1 for all Bin S, thenP(A) = 1. Three things to be noted about the concepts just defined are the following. First, formal strictness would require that the dependence of the concepts of reasonable and strict consequence on the language be made explicit in the notation. In fact, however, the dependence is only apparent, since it is easy to show that A is a reasonable or strict consequence of S, 'relative to .!e' if and only if it is a reasonable or strict consequence of S relative to any other language. Second, it is trivial to see that, if PI' Pb .. ' is a uniform sequence of probability functions and S is a finite set, and lim n --> 00 (Pn(B)) = 1 for all Bin S, but lim n --> 00 (Pn ( A)) < 1, then A is not a reasonable con seq uence of S. The converse is also true (i.e., if the fact that lim n --> 00 (Pn(B)) = 1 for all B in S guarantees that lim n --> 00 (Pn(A)) = 1, then A is a reasonable consequence of S), but this is not so obvious. Finally, I have deliberately used the tautological consequence symbol '}' here as the symbol for strict consequence. The justification has already been noted: namely that strict and tautological consequence are equivalent. This is shown in Theorem 1, below, which establishes some other elementary consequences of the basic definitions.
THEOREM 1. Let.!e be a language, and let S and A be, respectively, a set of formulas and a formula of .!e.
PROBABILITY AND THE LOGIC OF CONDITIONALS
275
1.1. SI- A if and only if A is a tautological consequence of S. 1.2. If SII- A then SI- A. 1.3. II- is a deduction relation, when restricted to finite sets: i.e., if Sand S' are finite sets of formulas, then i) if A is in S, then SII- A, ii) if S'II- B for all B in S, and SII- A, then S'II- A, iii) if S' and A' result from S and A, respectively, by substituting a truth-functional formula A' is a tautology, and therefore it has probability 1, for any probability function. And, it follows from this that any probability function, P, such that P(B;)=1 for i=l, ... , n also assigns P(A') = 1. But, it follows directly from the definition of a probability function that if C is any formula and C' is its 'associated' material conditional (or C'=C, if C is truth-functional), then P(C)=1 if and only if P(C')=1. Hence, if P(Bi) = 1, for i = 1, ... , n, then P(A)= 1, and therefore A is a strict consequence of Sf, and hence of S. This concludes the proof of 1.1. Proof of 1.2. This follows immediately from Theorem 1.1 just proven. For, if SII- A, but not SI- A, then A is not a tautological consequence of S, and therefore there is a truth-assignment to the formulas of the language under which all formulas of S have the value 'true', but A has the value 'false'. Again, a probability function can be defined as in the proof of 1.1,
276
ERNEST W. ADAMS
such that P(B)= 1 for all Bin S, but peA) =0, from which it follows trivially that A cannot be a reasonable consequence of S. This proves 1.2. Proof of 1.3. That if A is in S, then SIf- A is obvious. To prove part (ii), suppose that Sand S' are both finite, that S'II- B for all Bin S, and SII- A. For any e> 0 there exists b >0 such that if PCB»~ I-b for all Bin S, then peA»~ I-e. Since S'If- B for all Bin S, there exists bB for all Bin S such that if P(C» 1-b B for all C in S', then PCB»~ I-b. Since Sis finite, there exists a minimum b B for all B in S, which is positive; let b o be this minimum. Clearly, then, if P(C» I-b o for all C in S', then PCB»~ I-b for all Bin S, and therefore peA»~ I-e. Hence S'If- A, as was to be shown. Part (iii) follows directly from the fact that, for any probability function P' of!£ it is possible to construct another probability function P of se such that P(B)=P'(B') for all formulas Band B ' of se, where B' results from B by replacing all occurrences of a in B by qJ. The construction of P is elementary and will not be described here. Assuming this construction, it follows directly that if not S'If- A', then not SIf- A. For, if there were some e>O such that for all b>O there existed a probability function P' such that P'(B'» I-b for all B' in S', but P' (A')::;; 1 -e, then it would also be the case that PCB»~ I-b for all Bin S but peA)::;; 1-e, and hence not SIf- A. This concludes the proof. Theorem 1.1 shows that the notion of strict consequence is of no formal interest, since it is equivalent to tautological consequence. The intuitive significance of Theorem 1.1 is that it suggests that we should not 'get in trouble' in analyzing logical relations among conditional statements by treating them as material conditionals, so long as the premises of our arguments can be asserted with logical certainty. That is, where we may expect trouble in applications of standard logic is in situations in which we are reasoning from premises which are not known with certainty. Theorem 1.3 is significant in showing that the reasonable consequence relation has at least some minimal properties of deduction relations, and therefore justifies calling this a 'consequence' relation, at least as applied to finite sets of premises. That the probabilistic consequence relation is not a deduction relation where its domain is extended to include infinite sets of formulas is seen from the fact that it fails to satisfy the compactness condition: i.e. there are infinite sets of formulas, S, and formulas A, such that SII- A, but not S/If- A for any finite subset, S', of S. An example of a set Sand formula A having this property is as follows. Let S be the set of all formulas B;= 'a, v a. ; 1 --->ai + 1 & - a;' for i = 1, 2, ... (where the 'a;' are distinct atomic formulas), and let A = 'a1--->F'. Now it is a trivial consequence of the axioms of probability that if P(BJ>f for all i = 1,2, ... , then P(aJ::;;-tP(ai+ 1) for all i,
PROBABILITY AND THE LOGIC OF CONDITIONALS
277
from which it follows that P(a 1) must be 0, hence P(ar-+ F) = I. Clearly, therefore, SII-A, since an arbitrarily high probability for A can be guaranteed by requiring that all formulas of S have probability of at least t. On the other hand, the same argument shows that for any finite subset S' of S, an assignment P(a1»0, and therefore P(a 1---+F) =0 is consistent with assigning arbitrarily high probabilities to all formulas of S', so it is not the case that S'II-A.
In what follows we shall be concerned exclusively with the reasonable consequence relation restricted to finite sets of premises. It will prove that the reasonable consequence relation restricted to finite sets is equivalent to several other conditions with intuitive significance, and in fact it is possible to give a system of rules of inference within a natural deduction system such that a conclusion, A, follows from a finite set, S, of premises if and only if A is derivable from S by those rules. These rules will be given in the following section, in the definition of the relation of 'probabilistic consequence', and it will be shown that derivability in accordance with these rules is a sufficient condition for a conclusion to be a reasonable consequence of premises. The proof that probabilitistic consequence is also a necessary condition for reasonable consequence (the completeness proof) is more difficult, and requires further preliminaries. 3. Probabilistic consequence. We now give a set of rules for deriving 'probabilitistic consequences' from sets of formulas, S, and show that if a formula, A, is a probabilistic consequence of S, then A is a reasonable consequence of S. The rules for deriving probabilistic consequences form the clauses of Definition 6, below. DEFINITION 6. Let S be a set of formulas. Then the set of probabilistic consequences (abbreviated 'p.c.s.') of S is the smallest set S' having S as a subset such that for all truth-functional formulas ip, lJ', and y: PCl. if
lJ'---+y;
if either
278
ERNEST W. ADAMS
In practice, In considering the probabilistic consequences of a set S of formulas of a language 2, we may restrict the application of the rules of inference PCI-PC8 to just formulas of 2. That is, the class of all p.c.s. of S that are formulas of f£/ is the same as the class of formulas derivable from S by rules PCI-PC8 restricted to formulas of 2. Observe concerning rules PCI-PC8 that all of these are tautologically valid. This is to be expected, of course, if probabilistic consequence is to be equivalent to reasonable consequence, since all reasonable consequences are tautological consequences (Theorem 1.2). Two of these rules are, however, conspicuously weaker than well known corresponding rules of tautological inference. Thus, the tautologically valid rule to deduce q>~y from q> v 'P~y is weakened in PCS so that q>~y can only be infered from q> v 'P~y together with 'Jf~ - y. Likewise, PC8 is a weakened version of the Hypothetical Syllogism. As noted in Section 1, the Hypothetical Syllogism is not reasonable in complete generality; what PC8 asserts is that the weaker law that q>~y can be infered from q>~ 'P and q> & 'P~y is reasonable in complete generality. Something similar also holds in the case of PCS; the stronger version of that rule (that q>~y can be derived from q> v P~y) is not reasonable in complete generality. In fact, we shall eventually prove the following: if either PCS or PC8 is replaced by their 'stronger' versions, in the definition of probabilistic consequence, then all tautological consequences are derivable by the modified rules. The next theorem asserts what was taken for granted above: namely that all probabilistic consequences are reasonable consequences. All that is required to prove this is to show that any conclusion derivable by a single application of any of PCI-PC8 is a reasonable consequence of the formulas from which it is immediately derived, since Theorem 1.3 guarantees that reasonable conseq uences of reasonable conseq uences are themselves reasonable consequences. THEOREM 2. Let S be a set of formulas, and let A be a formula. If A is a probabilistic consequence of S then A is a reasonable consequence of S. Proof As noted above, all that is required is to show that all immediate inferences in accordance with rules PCI-PC8 are reasonable consequences of the formulas from which they are derived. This will be shown in only two cases - Rules PCI and PC4, since the proofs of the other rules proceed in entirely similar fashion. PCI is trivial. If q> is tautologically equivalent to 'P, then for any probability function P, P(q» = P(P), and thereforeP(q>--+y) = P(lp--+y). Hence 'Jf--+y is clearly a reasonable consequence of q>--+y.
PROBABILITY AND THE LOGIC OF CONDITIONALS
279
The proof of rule PC4, as well as the other 'nontrivial' rules PCS-PC8, is most easily obtained using the following simple inequality (which will prove important in other developments): for any probability function P and truthfunctional formulas tp, ':?, y and f.t such that peep) and P(y) are both positive,
P(P v f.t) PcP) P(f.t) ----_.. < - - +-P(ep v y) - P (ep) P(y)" This follows as a matter of simple algebra. For, if we set P(':?-f.t)=a, P(':?&f.t)=b, P(f.t- ':?)= c, P(ep -y) =d, P(ep &y)=e, and P(y- ep)= f, then P(':?v f.t)=a+b+c, P(ep v y)=d+e+ f, P(':?)=a+b, P(f.t)=b+c, P(ep)= d-i-e, and P(y)=e+ f, then what must be shown is that:
a+b+c a+b b+c ----<--+-d
+ e + f-
d
+e
e
+ J'
where all the numbers involved are non-negative, and all of the denominators are positive. Verifying this inequality is trivial, though tedious, since the result of cross-multiplying to clear of fractions yields an inequality in which all the terms on the left are cancelled by ones on the right; hence the ineq uality reduces to an inequality of the form 0::; t, where t is a sum of products of terms a - f Carrying out of this verification is left to the reader. Now it can be shown that it is possible to guarantee a probability at least 1-e for ep v ':?-"y by requiring that both P(ep-"y) and P('P-"y) be greater than 1 _.te. Assume first that both probabilities P (ep) and P (':?) are positive, and consider the probabilities P(ep-" -y) and P('P-" -y). We have then: P (qJ -" - y) =
and
P (lp -" - y) =
P(ep - y) = 1 - P (ep -" y) < !e peep) P(p - y)
P(tf) = 1 -
P ('1' -" y) < !e .
By the inequality just proven, then,
P«ep - y) v (lp - y)) ... - - < P(ep v '1')
.it; 2
+ .i e = 2
e.
But, it is an elementary consequence of the axioms of probability that, if f(ep v 0/) is not zero,
P(ep v lp -"y) = 1- P«ep - y) v (lp_=-~)) P(ep vp) hence P(ep v I/I-"y) > i-e.
280
ERNEST W. ADAMS
In the case in which one or both P(qJ) or pel{!) is zero, the proof is even simpler, for it follows directly from the axioms in that case that if, say, P( qJ) =0 then P (qJ v l{!--+y) = P(l{!--+y), and therefore a value at least 1- a for P(({} v l{!--+y) is assured by requiring that the probability of P(l{!->y) be at least I-a. This concludes the proof that qJ v l{!--+y is a reasonable consequence of qJ--+y and l{!--+y, and therefore finishes the argument. An interesting sidelight on the proof of Theorem 2 is the following: it is possible to guarantee a probability at least I-a for an immediate inference from a single premise by requiring that the premise have probability at least I - s, and it is possible to guarantee a probability at least 1- s for an immediate inference from two premises by requiring that both premises have probabilities at least I -1e. It will be shown later that this result generalizes to remote inferences as well: that is, if an inference of a conclusion from n premises is reasonable, then it is possible to guarantee a probability at least I-e for the conclusion by requiring the premises to have probabilities at least I-eln. Thus, one may establish conclusively that an inference of a conclusion from n premises is not reasonable, by finding some a such that the probabilities of all the premises is greater than I-eln but the probability of the conclusion is less than I -a. We conclude this section by listing a number of probabilistic consequences which follow from rules PCI-PC8. These consequences will be used in proving the completeness of the rules. THEOREM 3. Let 'I and qJI, ... , qJn and 'PI' ... , 'Pn be truth functional formulas. 3.1. If Y is a tautological consequence of 'P[ then qJ[--+y is a p.c. of qJI --+ 'P[. 3.2. qJ[ -> 'P[ and qJI --+qJj & 'P[ are p.c.s. of one another. 3.3. qJt v qJl--+ - «PI - 'Pd is a p.c. of qJI--+ 'Pl' 3.4. qJt v ... vqJn--+-(qJ[-'PI)& ... &-«Pn-'Pn) is a p.c. ofqJt--+'Pt, ... , qJn --+ 'Pn ' 3.5. If qJl & 'P1 is a tautological consequence of qJt & 'PI and qJt - 'P t is a tautological consequence of qJl- 'P1 then qJl--+ 1Jf1 is a p.c. of qJt--+ 'Pt. Proof The proofs of four parts of this theorem are most conveniently presented in the form of schemata of natural deduction derivations of the conclusions of the inferences from their premises. The proof of 1.1 goes as follows: 1. qJ[--+1Jf1 given. 2. 'PI f--y ('P j tautologically implies 'I) given.
PROBABILITY AND THE LOGIC OF CONDITIONALS
2S1
3. CPl & 'Pi ~y 2, PC3. 4. CPl ~y 1, 3, PCS. The derivation of CPl ~CPl & 'Pi from CPl ~ 'Pi goes as follows: 1. CPl ~ 'Pi given. 2. CPl tautologically implies CPl' 3. CPl ~CPl 2, PC3. 4. CPl ~CPl & 'Pi 1, 3, PC7. The derivation of CPl ~ 'Pi from CPl ~CP1 & 'P1 is also simple. 1. CP1 ~CPl & v, given. 2. CPl & 'Pi tautologically implies 0/1 &CP1' 3. CPl ~ 'P1 &CPl 1, 2, Theorem 3.1. 4. CP1 ~ 'Pi 1, 3, PC 6. The proof of 3.3 goes: 1. CP1 ~ 'Pi given. 2. CP1&'Plf--(CP1-'P1) tautology. 3. CP1 & 'P1~ - (CP1 - 'Pi) 2, PC3. 4. CPl ~ - (CP1 - 'P1) 1, 3, PCS. 5. CPz-CP lf--(CP1-'P1) tautology. 6. CPZ-CP1~-(q)1-'Pl) 5,PC3. 7. CP1V(CPZ-CP1)~-(CP1-'P1) 4, 6, PC4. 8. CP1 v CPz~ -(CPl - 'Pi) 7, PCl. Theorem 3.4 is obtained by simple iteration of applications of Theorem 3.3, plus use of rule PC7. Thus, Theorem 3.3 entails that CP1 v ... V CPn-+ -(CPi- 'Pi) is a p.c. of lfJi~ 'Pi for each i= 1, ... , n. And applying PC7 n-I times yields a derivation of the desired formula as a p.c. of the formulas CPl v ... V CPn~ -(CPi- 'Pi)' for i= 1, ... , n. Theorem 3.5 requires a somewhat longer derivation. 1. CPl ~ 'Pi given. 2. (CPl &CPZ)V(CP1-CPZ)~'Pl 1, PCl. 3. CP1 & 'Pi f- cpz & -r, given. 4. CP1 - cpz f- - 'Pi 3. 5. CP1 -cpz-+ - 'P1 4, PC3. 6. CPl &cpz-+'P1 2, 5, PC5. 7. CP1 &cpz & 'Pi f- 'Pz 3. S. CPl &cpz & 'P1~ 'Pz 7, PC3. 9. CPl &CPz~ v, 6, 8, PC8. 10. cpz - 'Pz f- CP1 - 'P1 given. 11. CPz-cplf-'PZ 10. 11, PC3. 12. cpz - CPl ~ 'Pz
282
ERNEST W. ADAMS
13. (
9,12, PC4.
This concludes the proof of the theorem. 4. P-orderings and their associated uniform sequences of probability functions. We have now obtained sufficient conditions for a formula A to be a reasonable consequence of a set of formulas, S: namely that A be a probabilistic consequence of S. In this and the next section it will be shown that these conditions are also necessary. In order to do this we need to consider systematically the construction of probability functions and sequences of probability functions with respect to which the probabilities of the formulas in S approach 1 as a limit, but the limit approached by the probability of A is less than 1. One rather convenient way of generating such sequences of probability functions is based on what will be called a 'P-ordering' of the truth-functional formulas of a language. Intuitively, a P-ordering is a weak ordering relation of the truth functional formulas of a language which can be interpreted such that if (P and tp are truth-functional formulas and o s: tp holds, then the probability ratio P(P)jP(
r-: F);
iii)
PROBABILITY AND THE LOGIC OF CONDITIONALS
283
doing this, though, we prove that P-orderings of finite languages are generated in a rather simple way from weak orderings of the SD-sets of the language (augmented by F). THEOREM 4. Let 2 be a finite language, let SD be the SD-set of 2, and let s; be a binary relation over the truth-functional formulas of 2. Then s; is a P-ordering of 2 if and only if there exists a weak ordering, s; 0, of the set SD+ =SDu{F}, such that: i) FS;olX for all IX in SD+, and F
284
ERNEST
W.
ADAMS
if and only if ex::; p. The first equivalence follows from conditions (i) and (iii) of Definition 7.1. For, by condition (iii), tp::; 'P (where tp is assumed to be a disjunction of elements of SD +) if and only if ex' ::; 'P, for all ex' in SD + belonging to tp, and by condition (i), ex'::; 'P, for all such elements ex' if and only if ex::; 'P, where ex is a maximal element of SD+ belonging to tp. The second equivalence follows from conditions (i) and (iv) of Definition 7.1. Thus, supposing that IfI is a disjunction of members of SD+, then it follows from condition (iv) that ex::; 'P if and only if ex::; f3' for some disjunct P' in 'P, and by condition (i) ex::; P' for some such disjunct, if and only if ex::; P, where P is a maximal element of SD + belonging to lJ'. This proves condition (ii) of the theorem, and therefore that if ::; is a P-ordering of :E, then there is a weak ordering ::; 0 of SD + satisfying conditions (i) and (ii). The proof of the theorem is completed by showing that if ::; 0 is a weak ordering of SD +, and s, 0 and s; satisfy conditions (i) and (ii) of the theorem, then v; satisfies conditions (i)-(iv) of Definition 7.1, and therefore x; is a P-ordering of !E. Each of conditions (i)-(iv) of Definition 7.1 follows trivially from the fact that s; 0 is a weak ordering of SD + , and that s; and s; 0 satisfy conditions (i) and (ii) of the theorem. Thus, to show that s; is a weak-ordering, suppose that ex, P and (j are, respectively, maximal members of SD+ in the ordering s; 0 belonging to tp, 'P and y, respectively, where tp, 'P and yare truth-functional formulas of :E. By condition (ii) of the theorem, tp::; 'P holds if and only if ex::; oP, and 1fI::; tp holds if p::; oex; and, since s; 0 is a weak ordering either ex::; oP or fJ::; orx, and therefore either tp::; IfI or 'P::; tp hold. Again, if tp::; 'P and 'P::; y, then, by condition (ii) of the theorem, ex::; of3 and f3::; o(), and since ::; 0 is a weak ordering, rx::; o(j, and therefore o s: y. Hence the relation of ::; is strongly connected and transitive, and so is a weak ordering. Conditions (ii)-(iv) of Definition 7.1 follow in similar fashion, and we omit the arguments. This concludes the proof. Now we come to what is more important about P-orderings: namely, their connection with uniform sequences of probability functions. It was asserted that the relation (p::; 'P could be interpreted to mean that the limit approached by the probability ratios P( 1fI)/P( tp) is positive. This idea is made precise in Definition 8, below, which introduces the idea of a uniform sequence Pt, Pl , ... associated with a P-ordering s . DEFINITION 8. Let:E be a language, let ::; be a P-ordering of :E, and let ..• be a uniform sequence of probability functions of :E. Then P l , Pl , ... is a uniform sequence associated with s, if and only if for all truth
PI' Pl ,
PROBABILITY AND THE LOGIC OF CONDITIONALS
285
ir;
functional formulas q> and lfJ of 2, q> ~ lfJifand only if lim., 00 (q> v 'P-+ 'P)) >0. We will not go through the argument to show that, provided the limits involved exist, then lim n --> 00 (Pn (q> v 'P-+ lfJ») is positive if and only if the limit of P; ('P)/P; (q» is positive (or possibly equal to plus infinity), and therefore the intuitive interpretation of q> ~ 'P is justified, since we will not use this fact in what follows. The proof, however, is elementary. Likewise, it follows trivially from Definition 8 that if PI' P l , ... is associated with ~, then q> < 'P holds if and only iflim n --> 00 (p" (q> v 'P-+q») = 0, and (provided the limit exists), this is equivalent to the condition that lim,., 00 (Pn (q»/P; ('P)) = O. What are important for present purposes are the facts asserted in the next theorem. THEOREM 5. Let 2 be a finite language, and let A and S be, respectively, a formula and a finite set of formulas of 2. 5.1. If PI' P l , ... is a uniform sequence of probability functions for 2, then there is a unique P-ordering ~ of 2 such that PI> P l , ... is associated with ~. 5.2. If ~ is a P-ordering of 2 then there is a uniform sequence of probability functions of 2 associated with ~. 5.3. If ~ is a P-ordering of 2, and PI' P l , ... is a uniform sequence associated with ~, then A holds in ~ if and only if limc., 00 (Pn (A») = 1. 5.4. If SII-A then A holds in all P-orderings of 2 which all formulas of S hold in. Proof of 5.1. This proof proceeds by showing that the binary relation defined by the condition: for all q> and 'P, q> ~ 'P if and only if limj., oo(Pn (q> v lfJ-+ lfJ») > 0 satisfies conditions (i)-(iv) of Definition 7.1, and is therefore a P-ordering of 2, and is, furthermore one such that PI' P l , ... , is a uniform sequence associated with it, according to Definition 8. The proof of each of conditions (i)-(iv) of Definition 7.1 is routine, and we shall actually carry out only the proof of (i) - that ~ is a weak ordering of the truth functional formulas of 2. That either q> ~ 'P or 'P ~ q> must hold follows, since lim (Pn(q> V
n--> 00
'P -+ q> v 'P») = 1
Hence at least one of the two limits on the right above must be positive, and therefore either q> ~ lfJ or 'P ~ q>. The transitivity of the relation ~ follows from the following inequality
286
ERNEST W. ADAMS
of the pure calculus of probability: for any probability function P" whatever, P" (q> v y ...... y) :::": P"(q> v p ...... P)' P" (P v y ...... y) . This inequality follows by simple algebra from the axioms of probability as given in Definition 4.1. Assuming this inequality, transitivity follows immediately, since if both q> ~ P' and 'P ~ y, then both of the limits of P" (q> v 'P...... P) and p,,(P'vy ...... y) must be positive, hence the limit of their product must be positive, and therefore, by the inequality, the limit of P" (q> v y...... y) is also positive, hence q> ~ y holds. This concludes the proof of 5.l. Proof of 5.2. Let ~ be a P-ordering for 2, and let SD+ =SDu{F}, where SD is the SD-set for 2. Assume that the elements of SD + are ordered as follows: i.e., the elements of SD+ are ordered in increasing 'blocks', CX"i_,+I' ••. , CX"i where the elements within each block are all equivalent to one another. Now, for each i = I. ... , k , let Pi be the disjunction of the elements in the ith block: I.e .•
We first define the values of P,,({3i)' for n=2, 3, ... and for i= 1, ... , k. If k=2, then everything is trivial; we set P" ({31) = 0 and P" ({32)= 1 for all n = 2, 3, .... If k > 2 then the probabilities are defined as follows:
P,,(fJJ=
1
n
fori=I .... ,k-1
2",-i'
1/1"'-2-1
P,,(fJJ = 1- -2",-2 /1
/1-
1
for n = 2, 3, .... Now. it follows by simple algebra from the foregoing equation that:
I
and
i= 1
P" (fJJ
1
P,,(fJi) = 1,
----
for
/1
= 2,3, ...
for i = I, .. _, k - 1,
/1
= 2,3 ....
Next the probabilities of the individual elements of SD + are defined by setting the probabilities of the elements of anyone Pi equal: for j = 1, ... , m;
PROBABILITY AND THE LOGIC OF CONDITIONALS
287
if Cl. j is a disjunct of Pi' then
where r, is the number of disjuncts of Pi' Clearly, the probabilities Pn(Cl. j) form a distribution over the set SD, and therefore define a unique probability function P; for the language 2. It is clear also that Pz, P3 , •.. form a uniform sequence of probability functions for 2, and all that remains to be shown is that this is a uniform sequence associated with :<:;;. Assuming the result of Theorem 5.1, all that is necessary to prove that P 2 , P3 , ••• is a sequence associated with :<:;; is to show that for any two Cl. h and Cl. j in SD+, Cl.h:<:;;Cl. j if and only if limn~w (Pn(Cl. h V Cl.r->Cl. j ) ) is positive. For, by Theorem 5.1, the sequence Pz, P3 , •.• is associated with some unique P-ordering of 2, and according to Theorem 4, this ordering is itself uniquely determined by its restriction just to the set SD +. But now what remains to be proved is trivial. If IXh:<:;;Cl. j, then either IXh~Ct.j or IXh< rJ. j. In the first case, P; (Ct. h)= P;(Ct. j), from which it follows immediately that P; (Ct. h V Ct.r-+Cl.j)=-h and therefore the limit of these probabilities is 1, which is greater than O. If Ct.h < Ct. j ' it follows that Ct.h and Ct. j belong to disjunctions Pp and Pq such that p -qJ) )=0. But Pn(qJ v -qJ-> -qJ)= 1 - Pn(qJ), and therefore limn~w(Pn(qJ))=I-O=1. The argument can be inverted to show that if limn~w (Pn (qJ)) = 1, then A holds in :<:;;. If A is conditional, say A = qJ-> 'I', and A holds in :<:;;, then either qJ - tp < qJ & tp or qJ:<:;; F. In the first case, it is not the case that qJ & tp:<:;; qJ - 'I', and therefore limn~w (P'((qJ & tp) v (qJ - tp)->qJ - tp)) =limn~w (Pn(qJ-> - tp)) =0. Now, only a finite number of values of Pn(qJ) can be zero, for if an infinite number were zero, then an infinite number of values of P; (qJ-> - tp) would be equal to 1, hence the limit could not be zero. We can assume without loss of generality that all values of P;(qJ) are positive, in which case, P;(qJ-> - tp) = 1- P;(qJ-> tp), from which it follows that the limit approached by P;(qJ-> tp)
288
ERNEST W. ADAMS
must be 1, since the limit of Pn ( qJ~ - P) is zero. The argument is reversible, so if the limit of P,,(qJ~P) is 1, then qJ~P holds in :S;. In case qJ:s;F, the limit approached by Pn(qJvF~F) must be 1. Since Pn(qJv F-"F) can only be 0 or 1, and 1 only if Pn(qJ)=O, it can only be that Pn(qJ) is 0 for all but a finite number of values of Pn(qJ). But, in this case, P; (qJ~ P) can differ from 1 for only a finite number of values of n, and therefore limn~(X) (Pn (lp-" P)) = 1. Conversely, if limn~(X) (Pn (lp~ P)) = 1 and qJ:S; F, clearly lp-" P holds in :S::. This concludes the proof of 5.3. Proof of 5.4. This follows trivially from 5.2 and 5.3. Suppose that there is a P-ordering :s; such that all members of S hold in :S;, but A does not hold in it. Then by 5.2 there is a uniform sequence PI' Pz , ... associated with :s;, and by 5.3, Iimn~(X)(Pn(B))=1 for all Bin S, but limll~(X)(Pn(A))
PROBABILITY AND THE LOGIC OF CONDITIONALS
289
9.3. The ordered partition of SD generated by S is the sequence SD l ' ... , SD p+ 1 of subsets of SD such that SD 1 =SD~SD(Sl)' SD i + 1 =SD(Si)~SD(Si+l) for i=l, ... ,p-l, and SD p+ t =SD(Sp), where S1> ... , Sp is the reduction sequence of S. 9.4. If it is not the case that SD (S) = SD, and if the ordered partition of SD generated by Sis SD t , ... , SD p+ b then the standard Pi-ordering of!l' associated with S is the P-ordering :s; such that for all a and fJ in SD, if a is in SD i and fJ is in SD j then a:S; F if and only if i = P + 1, and a:S; fJ if and only if j:S;i, Definition 9.4 presupposes what has not been shown: namely, that the ordered partition of SD generated by S is a partition of SD. This and other important properties of these partitions and their associated P-orderings will be derived below. First, however, it may help to make the intuitive bases of the concepts introduced in Definition I-~ 9 clearer to illustrate them as they apply in a simple example. Consider the a language generated from just the two ! atomic sentences 'p' and 'q' (plus 'T' and 'F'). The SD-set of !l' may be bed taken to consist of the four formulas a='-p& -q', b='p& -q', c='p&q', and d='-p&q', whose relations may be most easily visualized with the aid of the Venn Diagram to the right. Now, let S be the set containing just the four formulas p ..... q, - q-'> - p, q-'> - p, and p v q-'>p. To determine the immediate reduction of S, we must determine which formulas A in S have the property that SD(A)~SD(S). These two concepts are characterized in Definitions 3.2 and 3.3. SD (A) is the set of SDs belonging to Ant(A) (the antecedent of A), SD(A) is the set of SDs belonging to Ant(A)& -Cons(A), and SD(S) is the union of all SD(A) for A in S. The formulas A of S, together with the sets SD(A) and SD(A) are conveniently represented in a table, as below: formula A l.p-+q 2. - q --+ - P 3. q --+ - p 4. p V q --+ p
The formulas A of S such that
SD(A) {b, e} {a, b} {e, d} {b, e, d}
SD(A)
{b} {b}
{e} {el}
SD(S) SD(A)~SD(S)are
=
{b, c, d}.
clearly the first, third and
290
ERNEST W. ADAMS
fourth in the above table: i.e., the formulas p--*q, q--*-p, and pvq--*p. These, then, comprise the set Red (S). To construct the reduction sequence of S, we simply iterate the process described above. Thus, we set St =S, and set Sz= Red(St)={p--*q, q--* -p, p v q--*p}. To construct S3, we should find Red(Sz), which again is easily determined from the above table by finding all A in Sz such that SD(A) <;;SD(Sz). But, here it is clear that SD(Sz)= {b, c, d}, since Sz consists of formulas 1,3 and 4, and the union ofSD(A) for A in that set is {b, c, d}. It is the case then that SD(A)<;;SD(Sz) for all A in Sz - i.e. Red (Sz)=Sz. This being the case, the construction of the reduction sequence terminates with Sz: i.e., the reduction sequence isjust the sequence St, Sz, where St =S and Sz consists of formulas 1, 3 and 4. Having constructed the reduction sequence of S, the determination of the ordered partition of SD generated by S goes as follows. We take SD t to be the set of all SDs outside ofSD(St). Since SD(Sd=SD(S)={b, c, d}, then SD t = {a}. Next, SD z is defined as the set difference SD(St)~SD(Sz). We have already observed that SD(St)=SD(Sz)= {h, c, d}, and therefore SD z is the empty set, A. Finally, SD 3 is defined to be SD(Sz)={b, c, d}. The ordered partition of SD generated by S is then just the sequence of sets SD 1 ={a}, SDz=A, SD 3={h, c, d}. Given the foregoing ordered partition, the associated standard P-ordering of iF is constructed essentially by reversing the ordering of the SDs in the partition, and setting all elements of SD 3 equivalent to F; i.e., the associated ordering is determined by the following ordering of SDv{F}:
The manner in which the ordering of two truth-functional formulas q> and lfI in /1" is determined by the above ordering of the SDs is spelled out in Theorem 4: essentially, the ordering of q> and lfI is the same as the ordering of the maximal SDs in the ordering belonging to each of them. Thus, to determine the ordering of p and q, we need only to note that all of the SDs belonging to both of them are equivalent, and therefore p and q are equivalent. On the other hand, the maximal SDs belonging to p are strictly less than the maximal SD belonging to - p (which is a), and therefore p is strictly less than -p in the ordering. Note in particular that the following hold in the standard ordering: p::::;F, -q& -( -p)< -q& -p, q si F, and p v q s.F, from which it follows that the four formulas of S all hold (in the sense of Definition 7.2) in the standard ordering. The foregoing is not accidental: one of the most
PROBABILITY AND THE LOGIC OF CONDITIONALS
291
essential facts about standard orderings associated with sets of formulas, S, is that all formulas of S hold in them. This is proved in the next theorem. THEOREM 6. Let 2 be a finite language, let SD be its SD-set, let S be a finite set of formulas of 2, and let SD t, ... , SD p+ 1 be the ordered partition of SD generated by S. 6.1. SDt> ... , SD p+ t is a partition of SD. 6.2. IfSD (S) is not equal to SD then all formulas of S hold in the standard P-ordering of 2 associated with S. Proof of 6.1. This follows immediately from the fact that in the reduction sequence St, ... , Sp of S, each later element in the sequence is a subset of the earlier elements, and therefore the subset relations SD(Sp)~SD(Sp_t) ~ .,. ~SD(St) hold. Elementary set theory then entails that the sets SD t =SD-SD(St), ... , SD p+ t =SD(Sp) are mutually exclusive, and their union is SD. Proof of 6.2. Let A be a member of S, and suppose that SD",SD(S) is non-empty, hence the standard P-ordering, :C:;, associated with S is defined. We can assume without loss of generality that A is conditional, since if A is unconditional, A holds in :c:; if and only if the conditional formula Ant(A)--+Cons(A)=Cond(A) holds. Suppose that A =cp--+lJI: it will be shown that A holds in :C:;. Note first that it follows immediately from the definition of the ordered partition SDt> ... , SD p+ t that for i=l, ... ,p, SD(S;) = SD i+ t u ... u SD p+ t, where St, ... , Sp is the reduction sequence of S. Since AES=St, A is in at least one S, in the reduction sequence, and if A is in S, then SD(A)~SD(Si) =SDi+1U ... uSD p+ 1' Suppose first that A is in Sp. Then SD(A)~SDp+l' and the latter set is the set of all SDs which are set equivalent to F in the ordering :C:;. But, SD(A) is the set of all SDs belonging to cp, and therefore all SDs belonging to cp are equivalent to F, and so cp:C:; F. This in turn entails that cp--+ lJ' holds in :C:;. Suppose that A is not in Sp' Then there is some i such that A is in S, but not in Si+l' As before, SD(A) is a subset of SD(Si)=SDi+tu ... uSD p+t' On the other hand, since A is not in Si+ t = Red (S;), it is not the case that SD(A)~SD(Si)' so SD(A) must contain an SD in the union SD t u ... uSD i . Now, the members ofSD(A) are all SDs belonging to cp-lJI, and
292
ERNEST W. ADAMS
therefore all SDs belonging to qJ- 'Pare in SD i+ 1 u ... uSD p+ j - The members of SD(A) are all SDs belonging to tp, so there must be some SD in SD 1 u ... u S D, belonging to qJ, and therefore to qJ & 'P, since that element cannot belong to qJ - 'P. It follows immediately from this that qJ - 'P < qJ & 'P, since the maximum elements in the ordering belonging to qJ - 'P are in SD i + 1 u ... u SD p + I' and these are strictly less than the maximum elements in qJ & 'P, which are in the union SD 1 u ... uSD i . But, qJ - 'P < tp & 'P entails that qJ-+ 'P holds in s. Therefore, the assumption that A is in S entails that A holds in s, so Theorem 6.2 is proved. 6. Completeness and a decision procedure. Let A be a formula and S be a finite set of formulas. We are now ready to establish the equivalence of the following three conditions: (I) A is a probabilistic consequence of S (in the sense of Definition 6), (2) A is a reasonable consequence of S, and (3) A holds in all P-orderings in which all members of S hold. Actually, we have already shown that condition (1) entails condition (2) (Theorem 2), and that condition (2) entails condition (3) (Theorem 5.4), so what we have to do now is 'close the ring' by showing that condition (3) entails condition (1). It turns out to be easier to do this if three more links are added to the chain: conditions (4), (5) and (6) such that condition (3) entails (4), (4) entails (5), (5) entails (6) and (6) entails (I). Adding these links actually simplifies the proof and, moreover, yields an immediate decision procedure for determining whether a conclusion is a reasonable consequence of premises. THEOREM 7. Let 2 be a finite language, let SD be its SD-set, let A be a formula and S be a finite set of formulas of 2; let Sf be the set Su{ ~ A}, let f S;, ... , S; be the reduction sequence for Sf, let SD 1 , ••. , SD;+ 1 be the ordered partition of SD generated by Sf, and let So be the set S;~{ ~A} (i.e., So is the set resulting from the deletion of ~ A from S;). Then the following conditions are equivalent: (I) A is a probabilistic consequence of S, (2) A is a reasonable consequence of S, (3) A holds in all P-orderings of 2 in which all members of S hold,
(4) SD(A)~SD(S;)
(5) SD(A)~SD(So) and SD(So)~SD(So)~SD(A)~SD(A), (6) for some subset S" of S, SD(A)~SD(S") and SD(S")~SD(S") ~SD(A)~SD(A).
Proof. That condition (1) entails condition (2) and condition (2) entails
condition (3) were proven in Theorems 2 and 5.4, and that (5) entails (6) is
PROBABILITY AND THE LOGIC OF CONDITIONALS
293
trivial. What will now be shown is that (3) entails (4), (4) entails (5) and (6) entails (1). Assume first that condition (3) is satisfied: i.e., A holds in all P-orderings of !l? in which all members of S hold. We can assume here and later that A is a conditional formula q>-+ 'P, since if A is some unconditional formula 'P, then A can be replaced by T -+ 'P throughout without altering any of the conditions under consideration. Consider now the augmented set S' =Su{ ~A}, where ~A was defined (Definition 2.1) to be the formula q>-+ - 'P. Either SD(S')=SD, in which case the standard ordering associated with S' is undefined, or SD(Sl;6 SD, in which case the standard ordering associated with S' is defined. Consider the case in which SD(S')= SD first. In this case condition (4) holds trivially, since SD(A) is clearly a subset of the set of all SDs of 2:'. Now, suppose that SD(Sl;6 SD, and therefore that the standard ordering, ::s;, associated with S' is determined. Since ~ A is in S', and all formulas of S' hold in S (by Theorem 6.2), ~ A holds in ::S;. Likewise, since S is a subset of S', all formulas of S hold in s, and therefore A must hold in ::s;, by the assumption that condition (3) holds. Now, it follows directly from the definition of 'holding' that two 'contrary' formulas A = q>-+ 'P and ~ A = q>-+ - 'P can both hold at once in a P-ordering S only if q> - P < q> & 'P or q> S F, and ~ A holds if and only if either q> & 'P < q> - 'P or q>::S; F, so both can hold at once only if q>::S; F. But condition (4) follows directly from the fact that q>sF, since q>sF can hold in the standard ordering S if and only if all SDs belonging to q> are in the last member of the ordered partition SD~, ... , SD~+l' And, SD~+l is by definition equal to SD(S;) (Definition 9.3), and SD(A) is the same as the set of all SDs belonging to tp, hence SD(A) ~ SD( S;). Therefore we have shown that condition (3) entails condition (4). Suppose now that condition (4) holds. We consider separately the two cases in which ~ A = q>-+ - 'P is not a member of S;, and in which ~ A is a member of S;. In the first case, the derivation of condition (5) from condition (4) is trivial. For, if ~A is not a member of S; then So=S;. Assuming that condition (4) is satisfied, SD(A)~SD(S;), and therefore SD(A)~SD(S;), since SD(A) is a subset of SD(A). By the definition of the reduction sequence (Definition 9.2), however, SD(S;)=SD(S;), so SD(A)~SD(S;)=SD(So)' And, that SD(So)~SD(So)is a subset of SD(A)~SD(A) follows because So=S;, and SD(S;)=SD(S;), and therefore SD(So)~SD(So)is empty. Now assume that condition (4) holds, and that ~ A = q>-+ - P is a member of To prove that SD(A)~SD(So),suppose that tx is a member of SD(A):
S;.
294
ERNEST W. ADAMS
i.e., a is an SD belonging to CfJ - 'P (see Definition 3.2). SD (A) is a subset of SD(A), hence, by condition (4), a, which is a member of SD(A), is a member of SD(S~)=SD(S~). Also, since S~=Sou {CfJ~ - 'P}, SD(S~)=SD(So)u SD (CfJ ~ - 'P): hence a is a member of SD (So)uSD (CfJ ~ - 'P). But the elements of SD (CfJ~ - 'P) are just the SDs belonging to CfJ & 'P, and so a does not belong to SD (CfJ~ - 'P). Therefore a is a member ofSD (So), and we have shown that SD(A) is a subset of SD(So)' To show that SD(So)~SD(So) is a subset of SD(A)~SD(A), suppose that a is a member of SD(So)~SD(So)'Since So is a subset of S~, a must be an element of SD(S~)=SD(S~)=SD(So)uSD(CfJ~- 'P). By hypothesis, though, a, is not in SD(So), so it must be a member of SD(CfJ~ - 'P)= SD( ~ A). But, the members ofSD (CfJ~ - 'P) are all of the SDs belonging to CfJ & 'P, and these are the SDs composing the set SD (A)~SD(A). Hence, a is a member of SD(A)~SD(A), and we have shown that condition (4) entails condition (5). Now suppose that condition (6) holds: i.e., SD(A)s;SD(S") and SD(S") ~SD(S")s;SD(A)~SD(A)for some subset S" of S. We will show that A is a probabilistic consequence of S", and hence of S, since S" is a subset of S. Let us assume that S" is the set of conditional formulas CfJt ~ 'P l, ... , CfJn~ 'Pn - if any unconditional formula 'Pi occurs in S, then we can replace it by T - 'Ph since the latter is a probabilistic consequence of 'P, (Rule PC2 of Definition 6). Now construct the formula:
According to Theorem 3.4, B is a probabilistic consequence of S". Moreover, it is the case that SD(B)=SD(S") and SD(B)=SD(S"), and therefore SD(B)~SD(B)=SD(S")~SD(S"). That SD(B)=SD(S") follows from the fact that the antecedent of B is the disjunction of the antecedents of the , CfJn~ 'P/I in S". Therefore, SD(B) is the set of all SDs formulas (Pl--+ Ill l , belonging to CfJt v V CfJn' which is the union of the SDs belonging to CfJl' CfJz, etc., which is equal to SD(S"). That SD(B)=SD(S") follows from the fact that Ant (B) & - Cons (B) is tautologically equivalent to (CfJl - 'P l) V ... V (CfJ/I- 'Pn)· SD(B) is the set of all SDs belonging to the formula Ant(B)& - Cons (B), and is therefore the same as the set of SDs belonging to the union of the sets of SDs belonging to CfJl - 'Pi> ... , CfJn - 'P". But, SD (CfJi~ 'P) is the set of all SDs belonging to ip, - 'Pi' so SD (S") is equal to the union of all these sets, and is therefore equal to SD (B). Now, given any two formulas A and B, it can only be the case that
PROBABILITY AND THE LOGIC OF CONDITIONALS
295
SD(A)s;SD(B) if Ant(B)-Cons(B) is a tautological consequence of Ant(A)-Cons(A). For SD(A) and SD(B) are, respectively, the SDs belonging to Ant(A)-Cons(A) and Ant(B)-Cons(B), and the set of SDs belonging to one formula are a subset of the set of SDs belonging to a second if and only if the second is a tautological consequence of the first. Therefore, Ant(B)-Cons(B) is a tautological consequence of Ant(A)-Cons(A), since SD(A)s;SD(S")=SD(B). The same kind of argument can be used to show that if SD (B),....,SD (B)S; SD (A),...., SD (A), then theformulaAnt(A)&Cons(A) is a tautological consequence of Ant(B)&Cons(B). Therefore Ant(A)& Cons(A) is a tautological consequence of Ant(B)&Cons(B), since SD(B) ,....,SD (B) = SD (S,,),...., SD (S"):s; SD (A),....,SD(A). We have now shown both that Ant(B)-Cons(B) is a tautological consequence of Ant(A)-Cons(A), and that Ant (A) &Cons (A) is a tautological consequence of Ant (B) & Cons (B). It follows from Theorem 3.5 in this case that A is a probabilistic consequence of B, and hence A is a probabilistic consequence of S. This completes the proof. Theorem 7 presents the key mathematical results of this paper. In the following section we will derive some relatively easy correlates of Theorem 7, some of which have, perhaps, a more immediate intuitive significance than does the main theorem. In concluding this section, note that Theorem 7 provides the basis for a fairly direct decision procedure for determining whether a formula A is a reasonable consequence of a finite set, S, offormulas. Probably the simplest procedure for determining whether A is a reasonable consequence of S is to construct the reduction sequence of the augmented set S'=Su{,....,A}, and determine whether SD(A) is a subset of SD(S~), where S; is the final term in the reduction sequence of S'. According to condition (4) of the theorem, A is a reasonable consequence of S if and only if SD(A) is a subset of SD(S;). The following illustration will help to make clear how this decision procedure works. Suppose that the problem is to determine whether the tautologically valid inference of - p-+q from p v q is reasonable. In this case Sis the singleton set {p v q}, and A = - p-+q. As always, we begin by replacing all non-conditional formulas by their corresponding conditionals in the standard way. In this case, p v q is replaced by T -+p v q, the justification being that the two formulas are each reasonable consequences of the other. The next step is to form the augmented set S' = Su {,...., A}, where >- A in this case is -p-+ -q. The next step is to form the reduction sequence, S;, ..., S;, of S'. One simple way is to use the tabular representation of formulas B, and the sets SD(B) and SD(B). These are exhibited here, letting a, b, c and d be the
296
ERNEST W. ADAMS
formulas' -p & -q', 'p & -q', 'p &q' and '-p&q', respectively (see diagram on p. 289). Then we have: formula B T-+pVq -p-,.-q
SD (B)
SD(B)
(a, b, c, d} {a, d} SD(SD
{a} {d}
= SD(S') = {a, d}.
The second term of the reduction sequence, S~, is the set of all B in S; = S' such that SD(B)s;SD(S;): so in this case, S~ is the singleton set {-p~ -q}. S~ is determined in similar fashion, by finding all B in S; such that SD(B)s;SD(SD= {d}. In this case there are no members of B of S~ with this property, so S3 = A. And, since SD(A) =SD(A), S~ terminates the sequence: i.e., the reduction sequence of S' is the sequence S;, S;, S~ where: S'! S~
= {T ~ p v q, - p ~ - q} = {- p ~ - q}
S~ = II
SD(S;) = {a,d} SD(S~)
= {d}
SD(S~)
= SD(S;) = A.
Now, SD(A)=SD( -p~q)={a, d}, and this is not a subset of SD(S;)=A, and so it is not the case that A is a reasonable consequence of S. The procedure outlined above also leads immediately to the construction of a P-ordering in which all members of S hold, but A does not hold, if it proves to be the case that A is not a reasonable consequence of S. In particular, if s; is the standard P-ordering associated with the augmented set S' = Su {-A}, but SD(A) is nota subset of SD(S;), then all formulas of Shold in ~, but A does not. Thus, in the foregoing example, the ordered partition of SD generated by S' is the sequence SD 1 , ... , SD 4 , where SD t =SD~SD(S;)= {b, c}, SD z =SD(S;)~SD(S~)= {a}, SD 3 =SD(Sn~SD(S~)= {d}, and SD 4 = SD (S~) = A. The standard P-ordering is thus determined from the following ordering of the set SDu{F}:Foo(Pn(A))=O. Something stronger than just that A is not a reasonable consequence of S has been shown: namely that an arbitrarily high probability for all of the formulas of S is consistent with an arbitrarily low probability for A. This fact holds in general, and this is one
PROBABILITY AND THE LOGIC OF CONDITIONALS
297
among a number ofcorollaries to Theorem 6 which are derived in the following section. 7. Consequences of the completeness theorem. One important corollary of the completeness theorem is still another necessary and sufficient condition for a formula A to be a reasonable consequence of a finite set of formulas S. This corollary is stated in terms of another 'consequence' relation which has some significance in its own right. There is an intuitive sense in which we may speak of a conditional statement of the form 'if p then q' as being verified in a situation in which p and q are both found to be true, as being falsified in a situation in which p is found to be true and q is found to be false, and as being neither verified nor falsified in case p is found to be false. If 'verified' and 'falsified' are taken in the senses described above, then a conditional formula is verified just in case the corresponding conditional bet wins, is falsified if the bet loses, and is neither verified nor falsified in case the 'condition' of the bet is not fulfilled, so that the bet is not in force. The concepts of verification and falsification under a truth assignment are next defined formally, and in terms of those the relation of strong entailment is defined such that a set S of formulas strongly entails a formula A in case: (1) any truth assignment falsifying A must falsify at least one member of S, and (2) any truth assignment not falsifying any member of S and verifying at least one member of S must verify A. DEFINITION 10. Let 2 be a language, let S and A be respectively a set of formulas and a formula of 2, and letfbe a function mapping the class of truth-functional formulas of 2 into {0,1}. 10.1. f is a truth-assignment for 2 if and only if for all truth-functional formulas qJ and lJT in 2,/( -qJ)=l- f(qJ),/(qJ& '1')= f(qJ)'f('1') andj(qJ v '1')= f(qJ) + f('1')- f(qJ & '1'), andf(F)=O andf(T)= 1. 10.2. Iffis a truth-assignment for 2 then A is verified underfif and only if either A is truth-functional and f (A) = 1 or A = qJ-+ lJT for some qJ and '1', andj(qJ)= j('1') = 1; A isjalsified underjif and only if either A is truth-functional andj(A)=O, or A =qJ-+ '1' for some qJ and '1', and j'{o) = 1 andj('1')=O. 10.3. S strongly entails A if and only if for all truth assignments f of 2: (i) if no Bin S is falsified under f, then A is not falsified under f, and (ii) if no B in S is falsified under f, and at least one B is verified, then A is verified under f. Another way of describing the relation of strong entailment between a
298
ERNEST
W.
ADAMS
set S and a formula A is to say that not losing on any bet on a formula of S guarantees not losing on A, and not losing on any bet on a member B ofSand winning on at least one of them guarantees winning on A (and not just not losing). It should perhaps be noted that, despite any intuitive significance the concept of strong entailment may have, the relation formally defined here is not a deduction relation in the usual sense. It is not always the case, for instance, that if a formula A is strongly entailed by a subset of S, then A is strongly entailed by S (even if S is finite). The connection between strong entailment and reasonable consequence is shown in the next theorem. THEOREM 8. Let!l' be a finite language, and let A and S be, respectively, a formula and a finite set of formulas of !l'. Then A is a reasonable consequence of S if and only if there is a subset, S', of S such that S' strongly entails A. Proof By Theorem 7, a necessary and sufficient condition for A to be a reasonable consequence of S is that there exists a subset S' of S such that: SD(A)~SD(S') and SD(S')~SD(S')~SD(A)~SD(A).It will now be shown that S' satisfies the two conditions above if and only if S' strongly entails A in the sense of Definition 10.3. It will then follow directly that A is a reasonable consequence of S if and only if there exists a subset S' of S such that S' strongly entails A. If SD is the SD-set of !l', then every element a of SD defines a unique truth-assignrnent j; for !f! such that for all truth-functional qJ in !f!,
/, (p) = 1 if and only if a belongs to
qJ.
Furthermore, the correspondence between members of SD and truth function is one-one since for every truth function f there is a unique element I'J. of SD such thatf=.f~. Now, consider any formula A and set of formulas S of il'. The following facts are easily established: (1) A is falsified underz, if and only if a is in SD(A), (2) A is verified underr, if and only if a is in SD(A)~SD(A), (3) at least one member of S is falsified underr, if and only if a is in SD (S), (4) no member of S is falsified and at least one is verified underr, if and only if a is in SD(S)~SD(S). In proving (1)-(4) we need only consider the case in which A and S consist of conditional formulas, since the strategy of replacing any unconditional formula IJf by the conditional T ~ IJf works here as elsewhere. Now, if A = (p~ 1Jf, thenr, falsifies A if and only if/,(qJ) = 1 and/, (1Jf)=0, hence if and only if a belongs to qJ and not to 1Jf. But, these are precisely the conditions
PROBABILITY AND THE LOGIC OF CONDITIONALS
299
for a to belong to SD (A) (see Definition 3.2). Similarly, A is verified under fa ifand only iffa(qJ)=fa('P)= 1, hence ifand only if a belongs to both qJ and 'P, which is again trivially equivalent to the requirement that a belongs to SD(A)~SD(A). At least one member of Sis falsified under j', ifand only if for some B in S, fa falsifies B, hence if and only if for some B is S, a is in SD (B). And this is equivalent to the condition that a be in the union of all SD (B) for Bin S, which is by definition equal to SD (S). Finally, a necessary and sufficient condition for fa not to falsify any B in S, and to verify at least one is that a not be a member of any SD (B) for Bin S, and be a member of SD(B)~SD(B)for some B in S. The latter is equivalent to the requirement that a be a member of the set:
U (SD(B) ~ SD(B)) ~ U SD(B).
BES
BES
By elementary set theory, though, the above set is equal to
U SD(B) ~ U SD(B),
BES
BES
and this is by definition the same as SD(S)~SD(S). Having established equivalences (1)-(4) above, the rest of the proof is trivial. Thus, ifSD (A) is a subset ofSD (S'), then every truth assignment that falsifies A must falsify at least one member of S' (by equivalences (1) and (3», hence every truth assignment not falsifying any member of S' must not falsify A. And, if SD(S')~SD(S') is a subset of SD(A)~SD(A)then every truth assignment not falsifying any member of S' and verifying at least one must verify A (by equivalences (2) and (4». Hence, S' must strongly entail A. All of these equivalences are reversible, so that if S' strongly entails A then SD(A) is a subset of SD(S) and SD(S')~SD(S') is a subset of SD(A) ~SD (A), and therefore SIf-- A. This completes the proof. Theorem 8, besides possibly helping to illuminate the intuitive significance of the reasonable consequence relation (which is now seen to be closely connected with strong entailment), affords a rather quick way of checking inferences which are tautologically valid, to see if they are also reasonable. If an inference is tautologically valid, then no truth assignment which fails to falsify all of the premises can falsify the conclusion; and what is required in addition to guarantee that the strong entailment relation holds, is to show that every truth assignment failing to falsify any of the premises and verifying at least one of them must also verify the conclusion. Often this second condition is quite easy to check. For example, consider contraposition as a
300
ERNEST W. ADAMS
rule of inference for conditionals: to infer - q_ - p from p_q. In this case there is only one premise, p-q, and so what must be the case in order for p-q to strongly entail -q- -p is that verifying p_q entails verifying - q- - p. But, this is clearly not the case, since verifying p-q requires assigning both 'p' and 'q' the value 'true', whereas verifying -q--p requires assigning 'p' and 'q' both the value 'false'. Hypothetical Syllogism is also seen not to satisfy the condition of strong entailment, for by assigning the values 'false', 'true', 'true' to 'p', 'q' and 'r', respectively, we can fail to falsify p-q, verify q-r, but fail to verify p-r, which shows that p-q and q-r do not strongly entail p-r. The reader can, incidentally, easily check rules PCI-PCS for probabilistic consequence, to verify that in each case the premises do strongly entail the conclusion. Thus, it is clear that p-q &r does strongly entail p_q, since p-q is a tautological consequence of p-s q Scr, and, furthermore, verifying p-q &r requires assigning 'true' to each of 'p', 'q' and 'r", which in turn verifies p-q. Checking to determine whether a conclusion is strongly entailed by premises has two drawbacks as a method for determining whether the conclusion is a reasonable consequence of premises. One of these is obvious: a conclusion may be a reasonable consequence of premises without being strongly entailed by them, provided that a subset of the premises strongly entail the conclusion (i.e., some of the premises are redundant). Thus, p-p is a reasonable consequence of - p, though - p does not strongly entail p-p. The reason is that p_p is strongly entailed by the empty set (in fact the necessary and sufficient condition for a formula to be strongly entailed by the empty set is simply that it be a tautology). Hence, showing that a conclusion is not strongly entailed by a set of premises is sufficient to show that the concl usion is not a reasonable con seq uence of the premises only if the premises are in fact not redundant. The second drawback of testing for strong entailment as a decision procedure for reasonable consequence is that the method provides less information about the relation between the premises and conclusion than does the one outlined at the end of Section 6. In particular, the method there outlined showed how to construct a P-ordering (and associated uniform sequence of probability functions) in which the premises hold but the conclusion does not, in case the conclusion is not a reasonable consequence of the premises. This is important in applications of the formal calculus where, once having shown theoretically that a conclusion is not a reasonable consequence of premises, one wants to find instances of the premises and conclusion among sentences of English, in which it is obvious that the premises have high
PROBABILITY AND THE LOGIC OF CONDITIONALS
301
probabilities but the conclusion has a low one. The P-ordering gives information as to what the probability relations among the atomic propositions belonging to the premises and conclusion must be in order that the premises should have high probabilities, but the conclusions have a low probability, and this information is extremely useful in the search for counter-examples in English. On the other hand, showing that, say, verifying n-vs does not entail verifying - q~ - p, does not provide any direct means for finding out what the probability relations among 'p' and 'q' and their compounds must be, in order that p~q should have high probability, but - q~ - p have a low one. Aside from its intuitive significance, the connection between reasonable consequence and strong entailment established in Theorem 8 provides a useful tool in the derivation of many general 'metatheorems' concerning reasonable consequence. A number of these are given in Theorem 9, below. THEOREM 9. Let A and B be formulas and S be a finite set of formulas. 9.1. SIl-A if and only if S, ~AII-Ant(A)~F. 9.2. If S, BII- A and S, ~ BII- A then SII- A. 9.3. If either S is empty or A is truth functional, then SII- A if and only if S tautologically implies A. 9.4. If all formulas of S are truth-functional, then SII- A if and only if either A is a tautology, or S tautologically implies Ant (A) &Cons (A). 9.5. If S= {CPl ~ If'l' ... , CPn~ If'n} and SII- A but not S'II- A for any proper subset S' of S, then A is a reasonable consequence of C(S) = (CPi v ... V CPn)~ - (CPi - If'l)&··· & - (CPn - If'n)· 9.6. If A = p~ If', where p is an atomic formula not occurring in S, then SII- A if and only if either p~ If' is a tautology, or SII- -If'~F.l Proofof9.1. Assume that A=cp~1jJ and therefore ~A=cp~-IjJ. It follows directly from Definition 6 that cp~F is a probabilitistic consequence of {cp~ljJ, cP~ -1jJ}. Hence, if SII- A, then cp~F=Ant(A)~F must be a probabilistic and therefore a reasonable consequence of S, and ~ A. To show the converse, suppose that cp~Fis a reasonable consequence of S and ~ A = cP ~ -1jJ. By Theorem 8, there exists a subset S' of Su { ~ A} such that S' strongly entails cp~F. Either ~ A is in S' or it is not. In the second case, S' must be a subset of S, and therefore there exists a subset of S which strongly entails cp~F. Hence cp~F is a reasonable consequence of S, according to Theorem 8. But, it is easy to show that A = cP~1jJ is itself a 1 An incorrect statement of the conditions under which S II- p in S, was originally presented in Adams [1965).
--+
.p, where p doesnot occur
302
ERNEST W. ADAMS
reasonable consequence of (p-+F, and therefore in this case, A must be a reasonable consequence of S. In case ~A=(p-+-t/! is a member of S', let So=S'~{cp-+-t/!}: i.e., S' = Sou { ~ A}. It will be shown that So, which is a subset of S, strongly entails A. Suppose first that a truth assignment rfails to falsify any member of So, but falsifies A = cp-+ t/!..f therefore verifies ~ A = cp-+-t/!, hence fails to falsify any member of S' = Sou { ~ A}, and verifies at least one. But S' strongly entails cp-+ F, and therefore f would have to verify cp-+F, which is impossible. Therefore if/fails to falsify any member of So, it cannot falsify A. Now suppose thatz'fails to falsify any member of So and verifies at least one member of it. In this case f must actually verify A, for if it failed to do so, fwould fail to falsify ~A, and therefore would fail to falsify any member of S'=Sou{ ",A}, and verify at least one, and so would have to verify cp-+F, which is impossible. Therefore, iff fails to falsify any member of So and verifies at least one member, it verifies A. This shows that So strongly entails A, and therefore A is a reasonable consequence of S, by Theorem 8. This concludes the proof of 9.1. Proof of 9.2. Suppose that S, Blf- A and S, '" Blf- A. According to Theorem 9.1 then, S, B, ~AIf-Ant(A)-+F and S, "'B, ~AIf-Ant(A)-+F. It will be shown that S, ~AIf-Ant(A)-+F, from which it will follow by Theorem 9.1 again that SIf- A. Suppose that A = Cp-+t/!, and therefore >> A = cp-+ - t/!, and Ant(A)=(p. Further, let S+ be the set Su{"'A}: by hypothesis then S+, Blf-cp-+F and S+, '" Bif-cp-+F, and what must be shown is that S+If-cp-+F. From the fact that S+, Blf-cp-+F, it follows that there exists a subset S' of 5 +, B such that 5' strongly entails cp-+F. In case B does not belong to 5', then S' is a subset of S +, and therefore there is a subset of S + strongly entailing cp-+F, so S+If-cp-+F, as was to be shown. Suppose on the other hand that B does belong to 5'. Now let 5 0=5' ~ {B} and therefore S'=Sou{B} strongly entails (p-+F. The immediate consequence of this is that So strongly entails ~B. For, suppose thatfis a truth assignment failing to falsify any member of So, but falsifying ~ B; thenfwould actually verify B, so fwould fail to falsify any member of 5' = Sou {B}, and would verify at least one member (namely, B), and hence should verify cp-+F, since the latter is strongly entailed by S'. But it is impossible to verify tp-s F, and so no truth assignment failing to falsify any member of So can falsify", B. If a truth assignmentf fails to falsify any member of So and verifies at least one member, then it must actually falsify B and therefore verify ~ B, since if it failed to falsify B, then no member of S' would be falsified, and at least one would be verified, which would again entail that cp-+F was verified, which is impossible.
PROBABILITY AND THE LOGIC OF CONDITIONALS
303
It has been shown, then, that S + has a subset, So, which strongly entails '" B, from which it foIlows that S + If- '" B. But, it foIlows immediately from this together with the assumption that S+, '" BIf-
to be shown. This concludes the proof of 9.2. Proof of 9.3. According to Theorem 1.2, if SIf- A then S tautologically implies A. Hence to prove Theorem 9.3, what we must do is to show that if either S is empty of A is truth functional, and S tautologically implies A, then SIf- A. Suppose first that S is empty, and S tautologicaIly implies A - i.e., A is a tautology. Then S strongly entails A, since no truth assignment falsifies A, and no truth assignment can verify at least one member of S. Hence, in this case SIf- A, by Theorem 8. Now suppose that A is truth functional and S tautologically implies A. Again, it cannot be the case that any truth assignment which fails to falsify any member of S falsifies A. But this entails that any truth function failing to falsify any member of S must actually verify A, since in the case of truth functional formulas A, any truth function failing to falsify them must verify them. Therefore, again Smust strongly entail A, hence SIf- A. This proves 9.3. Proof of 9.4. Suppose that all formulas of S are truth functional. If SIf- A, then there is a subset Sf of S such that Sf strongly entails A. Two cases must be considered: (1) Sf is empty, and (2) S/ is not empty. In case Sf is empty, then it strongly entails A if and only if A is a tautology. In case (2), it wiIl be shown that S' tautologically implies Ant(A)&Cons(A). Again, we can assume without loss of generality that A is conditional, say A =
304
ERNEST W. ADAMS
entailed by Ant(A) and Cons(A), hence Ant(A), Cons (A)II- A, and so, by transitivity of II- , SII-A. This concludes the proof of Theorem 9.4. Proofof9.5. Suppose that S= {fPc> 'PI, ... , fPn--+'P,.} and SII-A, but not S'II- A for any proper subset S' of S. This means that S must itself strongly entail A. But it is not hard to see that S strongly entails A if and only if the formula
strongly entails A. The reason is this: C (S) is so constructed that any truth assignment failing to falsify any member of S also fails to falsify C(S), and any truth assignment failing to falsify any member of S and verifying at least one of them must verify C (S). Thus, if a truth assignment, j, failed to falsify any fPi--+ 'Pi for i = 1, ... , n, but falsified C (S), it would have to assign 'false' to one of the formulas fPi - 'Pi in the consequent of C (S). But this would then falsify the formula fPi--+ 'Pi' contrary to hypothesis. Iff failed to falsify any fPi--+ 'Pi for i= 1, ... , n, and verified at least one of them, it would have to assign the value 'true' to at least one of the fPi' for i = 1, ... , n, and to all of the - (fPi- 'PJ for l = 1, ... , fl. But this would make both the antecedent and consequent of C(S) true, and would verify C(S). The converse of the above also holds: any truth assignment failing to falsify C (S) must fail to falsify any member of S, and any assignment which verifies C(S) verifies at least one member of S. From what has been shown, then, C(S) strongly entails A, since S does, and therefore C(S)II- A. This concludes the proof of 9.5. Proof of 9.6. Suppose that A =p--+ 'P, where p is an atomic formula not occurring in S. Assume first that SII- A. Then there is a subset S' of S such that S' strongly entails A. If S' is empty, then A must be a tautology. If S' is not empty it will be shown that - 'P--+F is strongly entailed by S'. Since S' strongly entails A, A is a tautological consequence of S'. But, it is an easy theorem of the propositional calculus that if S' tautologically implies A = p--+'P, where p does not occur in S', then S' tautologically implies 'P, and hence S' tautological1y implies - 'P--+F. And, it follows directly from this that any truth assignment failing to falsify any member of S' must also fail to falsify - 'P --+ F. We show next that the condition that any truth assignment failing to falsify any member of S' and verifying at least one member of S' verifies - 'P--+Fholds vacuously, since no truth assignment can fail to falsify all members of S' and verify at least one. The reason is clear: such a truth assignment would have to verify p--+'P, since A =p--+ 'P is strongly entailed by S'. But since p does not occur in S', the value 'false' could be assigned
PROBABILITY AND THE LOGIC OF CONDITIONALS
305
to p independently, so that the truth assignment would fail to falsify any member of S', verify at least one member, but not verify p~ IJ', since p was assigned the value 'false'. Therefore, the subset S' strongly entails -IJ'~F, and hence SIf- - 'P ~ F. Conversely, suppose that either A =p~ IJ' is a tautology, or SIf- -IJ'~F. In the first case, A is a reasonable consequence of the empty set, and hence of S. In the second case, p~ 'P is a reasonable consequence of S because it is strongly entailed by - 'P~F. Thus, any truth assignment failing to falsify - 'l'~F must give the value 'true' to 'P, and hence fail to falsify p~ IJ'. And, no truth assignment verifies -IJ'~F, so the other requirement for strong entailment holds vacuously. Hence in either case that A is a tautology or SIf- -IJ'~F, SIf-A. This concludes the proof. Each of the parts of Theorem 9 has some intuitive significance, which will be discussed briefly. Theorems 9.1 and 9.2 are exhibited to show that at least some of the usual metatheorems of the propositional calculus carryover to the theory of reasonable inference. The rule that if Ant(A)~Fis a reasonable consequence of S and ~ A, then A is a reasonable consequence of S (Theorem 9.1) is a kind of generalization of the reductio ad absurdum principle, and it reduces to the familiar principle in case the conclusion, A, is truth functional. The generalization of the rule to the conditional case, A =
306
ERNEST W. ADAMS
steps in the reasoning may not be strictly in accordance with the rules for reasonable inference. Theorem 9.4 says, essentially, that no interesting conditional conclusions can be drawn from unconditional premises. A conditional conclusion from premises would not be practically interesting if it were in fact tautological, or if it could only be asserted on the grounds that its antecedent and consequent could be independently asserted, and Theorem 9.4 says that these are the only conditions under which the conditional can be reasonably inferred fom truth functional premises. Once again, one may look upon this result as casting doubt on the adequacy of the present theory. At any rate, most empiricists would probably argue that the data upon which all of our conclusions about the world are based should be represented as unconditional statements, and if these are the only 'premises' we have to go on, we must be able to justify infering certain conditional consequences from them. This shows that the traditional problems of induction may have their counterparts even in the very modest calculus here set forth. On the more technical side, Theorem 9.4 shows what we might have expected on intuitive grounds: that conditional statements can't be replaced by logically equivalent (in the sense of our calculus) truth functional statements. In particular, it follows immediately from the theorem that the formula p~q is not equivalent to any truth functional formula, for it could only be a reasonable consequence of a truth functional formula qJ if both p and q followed from qJ, and in that case, qJ would not be a reasonable consequence of p~q. The principle significance of Theorem 9.5 is to show that in deriving reasonable inferences from non-redundant sets of premises, one can replace the set ofpremises (here, as always, assumed to be finite) by a single conditional premise. In a sense, this suggests that one might introduce a kind of 'quasiconjunction' operation applying to conditionals, as follows: if A =qJI ~ PI and B=qJz~ 'Pz , then we would set This conjunction does have the property that any conclusion, C, which follows from A and B, but not from A alone or B alone, also follows from A &B. One can also show easily that quasi-conjunction reduces to ordinary conjunction in case both A and B are truth functional. On the other hand, this is not a true conjunction operation because in general neither A nor B is a reasonable consequence of A & B. In fact, Theorem 9.5 entails directly that no such 'true' conjunction operation is definable within this calculus. In particular, if
PROBABILITY AND THE LOGIC OF CONDITIONALS
307
we set A=p~r and B=q~r, then if C were any formula which was both a reasonable consequence of A and B, and which had both A and B as reasonable consequences, then C would have to be a reasonable consequence of the quasi-conjunction: A&B
= (p v q) ~ - (p - r) &
- (q - r),
which is equivalent to p v q~r. But, the latter formula has neither p~r nor q~r as a probabilistic consequence. It seems to me that this result somewhat confirms our intuitions about conditionals: namely that the joint assertion of two conditionals is not the same as, and cannot be paraphrased as the assertion of a single conditional. The intuitive significance of Theorem 9.6 is this: we cannot infer anything about what will happen if a certain situation occurs (represented by the atomic formula p) from premises which do not in any way refer to that situation, unless some very strong conditions are met. One might say that this shows that not only is the well known fallacy of material implication - to infer q~p from p - not valid in our calculus, but no fallacy of a similar kind can arise unless special conditions are met. Ones intuitive reaction to the inference of q~p from the premise p is that what is wrong here is the drawing of a conclusion about what would happen in situation q from a premise not involving q (e.g. to infer from 'it will rain tomorrow' the conclusion 'if John's car is a Ford then it will rain tomorrow'). Of the two special cases in which the inference of a conclusion p~ IfF, whose antecedent is not mentioned among the premises, is reasonable, one is familiar: namely, that in which the conclusion is a tautology, and therefore practically uninteresting. The other case is that in which it is also possible to deduce the formula - 'P~F from the premises. What is important to note here is that, in a sense, the statement - 'P ~ F is much 'stronger' than the statement 'P. That is, though - 'P~Fand 'Pareequivalenttruthfunctionally, IfF can have in general a probability which is arbitrarily close to, but different from 1, whereas - 'P~Fcan have a high probability only when its probability is equal to 1, and that probability is equal to 1 only if the probability of 'Pis 1. Thus, the second special case in which p-» 'P can be reasonably infered from premises not containing p is that in which a high probability of the premises actually insures the certainty of 'P (in the sense that the probability of 'P is 1). This intuitive explanation has a more precise counterpart in the result stated in Theorem to.3, to follow. Let us note here, by way of caution, that we are treading on dangerous ground in attaching significance to formulas of the form 'P~F, which occur
308
ERNEST W. ADAMS
in both Theorems 9.1 and 9.6. The reason is that in assigning any probability other than 0 to them, we are relying on our somewhat doubtful convention to assign any conditional the probability 1 if the probability of its antecedent is O. Remarks in Section 9 throw further light on this issue. In concluding this section, three further corollaries to Theorem 8 are derived, all of which deal explicitly with probabilities. THEOREM 10. Let 2? be a finite language, let A and S be, respectively, a formula and a finite set of formulas of 2?, and let ({J be a truth functional formula of 2? 10.1. Ifnot SII-- A then for all s >0 there exists a probability function P of 2? such that P(B» 1-<; for all Bin S, but P(A)<<;. 10.2. If SII-- A and S has n members, then for all <;>0, and probability functions P of 2?, if P(B) > 1 -<; for all Bin S, then P(A» I-n<:. 10.3. If SIt- - l[f---+ F and S has n members, then for all probability functions P of 2?, if P(B» I-lin for all Bin S, then P(l[f) = 1. Proof of fO.l. Suppose that A is not a reasonable consequence of S. We can assume, as usual, that A is conditional, say A =
309
PROBABILITY AND THE LOGIC OF CONDITIONALS
C(S') is seen to be equivalent to the iterated quasi-conjunction B 1 & ... &Bn in the sense that the antecedent and consequent of B 1 & ... ScB; are tautologically equivalent, respectively, to the antecedent and consequent of C (S'). And, the necessary and sufficient condition that C (S') strongly entail A = ep-+ 'P is just that ep- 'PtautologicaUy imply Ant( C(S')) -Cons( C(S')), and Ant(C(S'))&Cons(C(S')) tautologically imply ep& P. Now we will show that if C=ep1-+ P 1 and D=ep2-+ P 2, and we define C&D=(ep1 v e(2)-->-(ept- 'P1) & - (ep2 - lJI 2) , and if P(C» 1-10 and P(D» 1-8, then P(C &D» 1-10-8. Assume first that neither P(ept) nor P(ep2) is zero. Then:
P(ep1 - 1[11) P( ~ C) = P(ept -+ - 'P1) = -P(ept)--- = 1 - P(C) < 10, and
P( ~ D) = P(ep2
-+-
'P2) =
P(ep2 - 'P2) = 1- P(D) < 8. P(ep2)
By the inequality of general probability theory cited in the proof of Theorem 2,
P((ep1 - 'P 1) V (ep2 - 'P2)) ----- ----- --- s P(ep1
Ve(2)
P(ep1 - P 1) P(ep2 - P 2) _ + < 10 P(epd P(ep2)
+ 8.
But, by the elementary calculus of probability,
P((ep1 - P t ) v (ep2 - lJI 2)) P(C&D)=I-- --------->1-10-8. P(ept Ve(2) In case either ept or ep2 is zero, the proof is even simpler. For example, suppose that P(ept)=O. Then by elementary probability,
P(C&D) = P(ep1 v ep2 -+- (ept - P t) & - (ep2 - lJI 2)) = = P(ep2 -+ 1[12) = P(D), hence, clearly P(C &D» 1-10-8. Thus, we have shown that the 'probability defect' of the conjunction C &D is not greater than the sum of the 'defects' of the conjuncts. Iterating this, it follows directly that if
P(epi-+ PJ > 1- 10 for i = 1, ... .n, then P(C(S')) > 1- nco We have supposed that C(S') was so constructed that Ant(A)-Cons(A) tautologically implies Ant( C(S'))- Cons (C(S')) and Ant (C(S')) &Cons (C(S')) tautologically implies Ant(A)&Cons(A). To shorten writing, let us set A = ep-+ P and C(S')=y-+ fl. Then ep - P tautologically implies Y- fl and
310
ERNEST W. ADAMS
I' & Ji tautologically implies
P(y - Ji) ---,-~---,-
>
P(
--
P(Y&Ji)- P(
from which it follows immediately that: P(y &Jl) P(y -+ Ji) = - - P(y &Ji) + P(y -
P(
{l)
~-
- - - - - = P(
P(
If P(y &Ji) =0 but P(y)#O, then P(Y-+Ji) =0, so clearly P(Y-+Ji)~ P(
PROBABILITY AND THE LOGIC OF CONDITIONALS
311
A technical observation on Theorem 10.2 is that it reduces to a Theorem of Suppes (this volume) in the case in which all of the formulas involved are truth-functional. Comparison of the proofs illustrates strikingly how much simpler the 'inequality' calculus of unconditional probabilities is than the corresponding calculus of inequalities among conditional probabilities. 8. Analysis of inferences which are tautologically valid, but not reasonable. The general results derived in the previous sections provide a stock of mathematical tools with which to attack a number of questions having to do with the formal theory of reasonable inference and its applications. The final two sections deal briefly with two topics of some methodological interest connected with the present theory, which the earlier results shed some light on. In both cases the discussion will be informal. The nature of inferences which are tautologically valid but not reasonable in the sense of this theory warrents further consideration. In particular, we may ask: given an instance of an inference schema to infer a conclusion A from a set of premises S, where A is a tautological but not a reasonable consequence of S, what conditions must obtain in order that the probabilities of the premises be high, but that of the conclusion be low? Our results concerning P-orderings give some information on this question, as will be shown below. First, however, a few examples of inference schemata which are tautologically valid but not reasonable may help to show how pervasive this phenomenon is. None of the following schemata are reasonable: (1) to infer p~q from any of -pvq, -p, or q; (2) to infer - p~ F from p; (3) to infer p~r frompvq~r; (4) to infer p&q~r from p-s r ; (5) to infer -q~ - p from p~q; (6) to infer p~r fromp~q and q~r; (7) to infer p~ - q from p~ r and q~ - r. As an example of a standard metatheorem of inference which is not valid for reasonable inference, we may cite the deduction theorem (rule ofconditionalization). That is, the following rule does not hold for reasonable inference: if S is a set of formulas and
312
ERNEST W. ADAMS
premises S according to those rules if and only if A were a tautological consequence of S. The proof of this is trivial in the case of the first rule: to infer (p-+l/J from -qJvl/J. For, suppose that S tautologically implies a formula, A. If A is truth functional, then A is also a reasonable consequence of S (by Theorem 9.3), and hence A would also follow by the augmented rules. If A is conditional, say A=qJ-+l/J, then clearly it is the case that S tautologically implies the truth functional formula -qJ v l/J, so -qJ v l/J is also a reasonable consequence of S. But, with the additional rule to infer qJ-+l/J from - (P v l/J, A = qJ-+l/J could also be derived from S. The reader can easily verify in the cases of the other non-reasonable rules that adding them to those for reasonable inference would make the resulting system equivalent to the propositional calculus. Since all rules except (1) and (2) are highly plausible intuitively, this shows what very strong intuitively plausible reasons there are for representing 'if then' statements in English by material conditionals for the purposes of logical analysis. Thus, one may say, granted that one wants to give a truth condition analysis of conditionals in order to determine their logical relations with other statements, then one is practically forced to assign truth values to them which are the same as those for the material conditional. Now we return to the question raised earlier: what probability relations must the components of the premises and conclusion of a tautologically valid but not reasonable inference have in order for the premises to have high probability but the conclusion have a low probability? Answering this question may be the most interesting application of the formal theory, and it is here that the results concerning P-orderings prove useful. For, supposing that the premises and conclusion are S and A, respectively, we can deal with the problem most efficiently by asking the related question: that conditions must be satisfied by a P-ordering:::; in order that all the formulas of S hold in :::;, but A does not? To illustrate, consider the non-reasonable inference rule: to infer p-+q from - p v q. What sorts of P-orderings are such that - p v q holds in the ordering but p-+q does not? To formulate the conditions under which - p v q holds, but p-+q does not, we may use the Venn Diagram (see p. 289), letting a, b, c and dstand for - p& -q, p & -q,p &q and - p &qrespectively. Then: - p v q holds in :::; if and only if b
not hold in :::; if and only if:
c :::; band F < b v c.
PROBABILITY AND THE LOGIC OF CONDITIONALS
313
Now, it is a simple matter of inference from the axioms for P-orderings to show that the two conditions above hold if and only if: F < band c ~ band b < a v d.
Given the interpretation of qJ < ljJ as meaning that the probability of qJ must be small relative to that of ljJ, the above conditions lead to the following conclusions concerning the probabilities of a, b, c and d (and by implication, about the probabilities of combinations of them): (l) that P (p) > 0, (2) that P(b) is not small by comparison to P(c), and (3) thatP(b) is small with respect to P (a v d) = P ( - p). One inference in particular is significant: namely that, since P(c) is not large relative to PCb) and P(b) is small relative to pea v d), then P(p)=P(bvc) is small relative to P(-p)=P(avd). Hence, the formula - p must hold in the ordering ~. The foregoing facts help in the search for an actual counterexample to the rule to derive p~q from - p v q: i.e., such an example must be one in which pep -q)= P(b»O, pep -q) is not small relative to pep &q)= P(c), and it is small relative to P( - p) - and, therefore, P( - p) is close to I. Now, all we have to do is to find examples of English statements, p and q, whose probabilities have these relations. First, the probability of - p must be high, so that of p must be small: i.e., p must be a statement with low probability. Let us take any such statement to start with: e.g., let p = Mr. Jones will have an accident on his way to work.
Now q must be chosen in such a way that the probability of p - q is not small in comparison to p&q. In fact, one way to insure that this condition is satisfied is to require that pep &q) be small in comparison to P(p-q), which is the same as to require that, if the event p does occur (which is unlikely), then it will be much more likely that q does not occur than that it does. So, we may take q to assert something which would be highly improbable in the event of p's occurrence, for example: q=Mr. Jones will arrive on time for work.
All that remains, after having chosen p and q in the above way, is to verify that these are statements such that - p v q is highly probable, but p~q is not. In this case, - p v q = Either Mr. Jones will not have an accident on his way to work, or he will arrive on time for work, and p~q=If Mr. Jones has an accident on his way to work, then he will arrive on time for work.
314
ERNEST W. ADAMS
Now, - P v q is probable, simply because its first disjunct (that Mr. Jones won't have an accident on his way to work) is probable, but clearly p-+q is highly improbable, and so it would certainly be unreasonable to deduce p-+q from the premise - p v q in this case. The foregoing example also illustrates a point made in Section 1. The example is odd in one respect: namely that it suggests that the premise - p v q is being asserted in a situation in which the stronger assertion, - p could be made (in fact, both - p and q could be asserted here). I suggested earlier that to actually assert a disjunction under such circumstances is misleading (the hearer is likely to ask 'what is the connection between -p and q?'). And, if we are justified in assuming that speakers do not deliberately make misleading statements, the inference of p-+q from - p v q is 'reasonable' in the sense that it can be justified by this assumption. The above point can be made more precise in the following way. If, on some occasion, P( - P v q» l-e, but P(p-+q)-::;.-t, it is easy to show that P( - p» 1-2e, and P(p)<2e. More generally, we may assert: if S tautologically implies (p-+1jJ but not SIf-
9. A modified criterion of reasonableness. This section is concerned with the possible doubtfulness of those consequences of the basic assumptions of the present theory which depend on the arbitrary assignment of probability 1 to all conditionals whose antecedents have probability O. One way to avoid the problem of having to make some arbitrary assignment of probabilities to conditionals whose antecedents have zero probability is to work only with probability functions which assign values in the open interval (0,1) to all but logically false or logically true statements. Such probability functions have been called regular: they may be formally defined as probability functions in the sense of Definition 4.1, which satisfy the following additional axiom: for all truth functional 0 there exists <5 > 0 such that for all regular probability functions p, if P(B) > 1-(5 for all Bin S, then P(A»l-e. We will now consider briefly and in-
PROBABILITY AND THE LOGIC OF CONDITIONALS
315
formally the consequences of this modified criterion of reasonableness, confining attention exclusively to finite sets of premises. r-reasonable consequence is a relation intermediate in strength between reasonable consequence and tautological consequence: i.e., for premises S and conclusion A, if SII-A then Sll-rA, and if Sll-rA then SI- A. The relation II- r also satisfies two conditions for deduction relations stated in Theorem 1.3, but it fails to satisfy the substitutivity condition: if SII- A, and S' and A' result from S and A, respectively, by replacing their atomic formulas by other formulas, then S'II- A'. In particular, the relation p----* FII-, F holds, but F----*Fll-rF does not. Since SII-A entails Sll-rA, all of the basic rules of probabilistic consequence (Definition 6) are also valid for r-reasonable consequence. One rule not valid for reasonable consequence in general, but valid for r-reasonable consequence is the following "stop rule": PCr. If qJ does not tautologically imply F, then qJ----* Fll-rF. It can be proved that, in fact, if PCr is added to rules PCl-PC8 for probabilistic consequence, then any consequence A follows from a (finite) set of premises S according to the augmented rules if and only if Sll-rA. Along with the notion of r-reasonable consequence, it is possible to define the notion of an rP-ordering, which is a P-ordering (Definition 7.1) :s; such that for all truth functional qJ, qJ:S; F holds only if qJ is tautologically false. And, as would be expected, all rP-orderings are associated with uniform sequences of regular probability functions. The most important fact about the modified criterion of reasonableness is this: for any formula A and set offormulas S, if Sll-rA but not SII- A then (1) Sll-rF, (2) not SII-F, and (3) for some non-tautologically false formula tp, SII- qJ----* F. The significance of the foregoing is this: if a formula A is an rreasonable consequence of S, but not a reasonable consequence of S, then S must be 'r-reasonably inconsistent'. And, the reason for this must be that some formula of the form qJ ----* F is derivable from S, where qJ is not tautologically false. We have just seen that the only case in which r-reasonable consequence differs from reasonable consequence is that in which the set of premises, S, has a reasonable consequence of the form qJ----* F, where qJ is not tautologically false. Under those circumstances, Sll-rF, and therefore Sll-rA for all formulas A. What can be said about sets of premises S which do have formulas of the form qJ----* F as reasonable consequences? The following can be proven: that S entails such a formula if and only if it has a subset S' which is 'empirically inconsistent' in the sense next defined. A set S' of formulas is empirically in-
316
ERNEST W. ADAMS
consistent if and only if: (1) there is a truth assignment under which at least some members of S' are verified or falsified, and (2) every truth assignment which verifies some member of S' falsifies some other member of st. A pair of formulas which are empirically inconsistent are p---+q and p---+ - q, since it is possible to verify or falsify them, but any truth assignment verifying one falsifies the other. A single formula which is empirically inconsistent is p---+ - p, since it is possible to falsify it, but not to verify it. The intuitive motivation for calling situations like those described above 'empirically inconsistent' is that they represent conditional assertions such that, should any of the conditions of the assertions be 'realized', then at least one of the assertions would be falsified (and, it must be possible to realize the conditions). Thus, one may 'get away with' asserting both p---+q and p---+-q simply because it happens that p proves to be false, so neither assertion is 'contradicted by the facts'. Nevertheless, the assertions do 'conflict' in that they both give information about a contingency (that p will prove true), but that information is inconsistent - we certainly couldn't rely on these assertions to guide us in making provision for the contingency that p might be true. The foregoing argument constitutes further intuitive justification for adding the rule of inference PCr to the rules which are valid for reasonable consequence (for instance, rules PCI-PCS). What the arguments suggests is that any premises which do have a formula of the form
SUBJECT INDEX
acceptability of a generalization, 11, 10 - of singular hypotheses, 15, 18 acceptance of a generalization, rule for, 11, 13 accepted-information model of science, 97 adequate evidence, 2 atomistic universe, 130 attention, mechanism of, 64 - , principle of, 63 axioms of order, 158 background knowledge, 177, 180, 186 Bayes' theorem, 56 Bayesian approach, 198 - to the paradoxes of confirmation, 195, 198 model, 28, 37 - with memory of length one, 38 theory, 22, 27 viewpoint, 52, 61, 202 betting theory, 67 Boolean algebra, 244 - o-algcbra, 238 Carnapian inductive logic, 35 causal laws, 198, 204, 207 certainty, 307 chess, 23, 43 cognitive model, 36 commonness, measure of, 87 - , principle of maximum, 88, 94 compactness condition, 276 - theorem, 248 completeness, 292 - proof, 277 - theorem, 221, 297 comprehensive, 179, 186
- instances, 177 - , of a universal implication, 176 concept formation, 21 - identification, 22, 24, 29 conditionals, accidental, 189,190,191,193 - , connected, 189, 193 - , counterfactual, 188 - , indicative, 188 - , logic of, 265 - , subjunctive, 188 conditioning, 26 - model, generalized, 41, 46 confirmation, 176, 178, 180, 181, 182, 184, 185, 186, 196, 197 - , degree of, 219 function c* (Catnap), 73,102,123, 127, 128, 133, 149, 154 - c+ (Carnap), 123 - , paradoxes of, 175, 198, 208 - theory, 201 consistency and logical closure as conditions of acceptability, 4, 12 consistency and logical closure as epistemic principles, 3 constituent, 7, 99, 114, 133, 160 - , attributive, 7, 99, 133, 158 - , depth of a, 153 constituent, graph of a, 160, 173 - , subordinate, 151 constituents and structure-descriptions, I SO content, informative, 99, 100 contingency table, 205 contraposition, 299 - , law of, 269 contrapositive, 186, 190, 192, 194, 196, 198 - , asymmetry between it and its original, 178
318
SUBJECT INDEX
- instances, 177, 196 - instance, restricted, 182 -, probabilistic, 206 -,restricted, 179, 191, 192 - , of a universal implication, 175 corroboration, 107, 111 - , degree of, 109, 110 countable consistent set, 237 Ct-predicate, 7, 99, 133 decision procedure, 292 - theory, 21, 200 deduction relation, 298 - theorem, 311 deductive-nomological model of science, 97 degree of covering, 84, 94 - -, principle of maximal, 94 denumerably infinite model, 240 depth, 134 - of a constituent, 153 - of a sentence, 141, 158, 162 detachment, rule of, 52 - in probabilistic inference, 50 deterministic hypotheses, 205 direct product, 230 directed union theorem, 251 distributive normal form, 9, 99 Doob's separability theorem, 258 downward Lowenheim-Skolem theorem, 251 effort of description, 71 eliminating quantifiers, 241 empirical evidence, 182, 183, 184,185 - support, 178, 184, 185, 186 - -, as a pragmatic concept, 183 empirically inconsistent, 316 entropy, 66, 70, 76, 78 entropy, information and, 77 epistemic utility, 96, 98, 107 equivalence condition, 175,209,217 evidence, 58 - , concept of total, 49 evidential strength, 81 -, information-theoretical measure of, 83 --, probabilistic measure of, 83
expected utility, 98, 107, 108, 110 - - , classical theory of maximizing, 63 finitely additive probability, 222, 244 Gaifman condition, 224, 225, 256, 257 generalization, strong, 98, 99 -, weak, 98, 99 generalizations, degree of confirmation of, 6, 109, 115, 124, 126, 130, 150, 168 - and knowledge, 18 grammar, 42 graph, 168 - of a constituent, 160, 173 -, matrix associated with a, 162 - , strongly connected, 161, 162, 169 hypothesis, range of a, 179 hypothetical syllogism, 268 independent union, 227 inductive logic, 202 - - , Carnapian, 35 inference, non-deductive principle of, 56 - rule, non-reasonable, 312 infinitary formula, 220 information, 77 -, absolute, 82, 101, 106 - , amount of, 81 - and entropy, 77 - overlap, 84 - - or covering, principle of, 85 - -, principle of maximum, 94 -, principle of minimum added, 94 -, processing, 202 - -, problem of, 62 -, its relation to probability, 82, 100, 106, 107 -, relative, 82, 101, 106 -, semantic, 96, 98 - , - theoretical measure of evidential strength, 83 instance-confirmation, 7 inverse probability, 195 Kelley property, 237 knowledge, background, 177, 180, 186
SUBJECT INDEX
-, definition of, 1 - , generalizations and, 18 - and probability, 1, 5, 12 Kolmogorov axioms, 219, 272
:e, probability function for, 273 - , SD-set of, 272 -, state description for, 271 A-continuum of inductive methods (Carnap), 75, 113, 115, 119, 122, 129 language, 270 - , finitary, 220, 244 - , finite, 271, 283, 291, 308 - , infinitary, 236 lawlikeness, 180, 181 length of a state description, 67 limited relevance, principle of, 175, 176 logical closure as a condition of acceptability, 4, 12 lottery paradox, 4, 17 - , avoided, 17 Lowenheim-Bkolem theorem, downward, 251 - , upward, 252 Markov process, 78 material implication, 265 - - , fallacy of, 307 - - , its role in the paradoxes of confirmation, 180, 186, 188 mathematical learning theory, 41 matrix, 162 - associated with a graph, 162 - , Jordan form of, 165, 171 - , permutation, 165, 166 maximum likelihood principle, 89 memory, 34 - of a computer, 203 -, finite, 34 - of length one, Bayesian model with, 38 - of minimal length, 38 -, restricted, 37 Nicod's criterion, 208, 209, 217 noncontingent reinforcement, 36 non-deductive principle of inference, 56 non-reasonable inference rule, 312
319
numerical quantifier, 157 ordered universe, 155 orderings for sets of formulas, standard, 288 P-ordering, 282 paired-associate experiment, 24 paradox, lottery, 4, 17 - of universal relevance, 176 paradoxes of confirmation, 175, 198, 208 - - , Bayesian approaches to the, 195,198 - -, possible solutions to, 186, 195, 199 parameter a (Hintikka), 13, 103, 104, 113, 115, 116, 118, 130 - - as an index of caution, 117, 13l - - and inductive behavior, 127 - -, infinite, 122 - -,zero, 123, 126 - A (Carnap), 13,75,76,77,113,115,118, 122, 127 - - as a function of the number of instantiated Q-predicates, 119 parameters a and A, relation of, 118 permutation matrix, 165, 166 polynomial inequality, 249 prior distribution, 203 probabilistic consequence, 277, 288 - contrapositive, 206 - inference, 50 - -, rule of detachment in, 50 - measure of evidential strength, 83 probability assertion, 233 - , finitely additive, 222, 244 - function for 2, 273 - -, regular, 314, 315 -, inverse, 195 - and knowledge, 1, 5, 12 - law, 234 - logic, 220 - model, 235, 238 - -, denumerable, 244, 252 -, denumerable symmetric, 252 - -, with strict identity, 248 - - , symmetric, 242, 253 - - - theoretic concept, 227 probability, principle of maximum, 91
320
SUBJECT INDEX
- systems, symmetric, 231, 252 - theory, 220 product, direct, 230 -, ultra-, 230 property space, 44 propositional calculus, 265 psychological model, 24, 25 pure prudence, rule of, 63 Q-predicate, 76, 99, 114, 133 quantifiers, eliminating, 241 -, numerical, 157 Rasiowa-Sikorski lemma, measure-theoretic generalization of the, 259 range of relevance, 211, 215 -- -,natural, 216,217,218 rational behavior, 23, 35 - change in belief, 60 rationality, 23, 67 -, theory of, 63 reasonable consequence, 269, 274, 300, 302, 308 reasonableness, criterion of, 267 - , modified criterion of, 314 reduction sequence, 288 redundancy, 77, 78 requirement of the variety of instances, 125 simplicity, 66, 70, 76, 78, 96, 163, 167
- , principle of, 66, 73, 79 - of a state description, 67, 73 singularity, 140, 141, 146, 149 - , existence of, 147, 151, 153 -,observed, 146, 149 - , unobserved, 146, 149 space of properties, 44 state description for se, 271 - , length of a, 67 - , simplicity of a, 67, 73 statistical independence, assumption of, 202 - syllogism, 49,57 stimulus-association model, 24, 25 - - sampling model, 35 - - theory, 39 straight rule, 123 strict consequence, 274 strong entailment, 297, 300, 302 structure description, 71 substitutivity condition, 315 tautologically valid, but not reasonable, 311 truth condition, 265 ultraproduct, 230 uniform sequence, 284 uniformity, 76 upward Lowenheim-Skolem theorem, 252 utility, epistemic, 96, 98, 107 valuation function, 223
