Lecture Notes in Computer Science Commenced Publication in 1973 Founding and Former Series Editors: Gerhard Goos, Juris Hartmanis, and Jan van Leeuwen
Editorial Board Takeo Kanade Carnegie Mellon University, Pittsburgh, PA, USA Josef Kittler University of Surrey, Guildford, UK Jon M. Kleinberg Cornell University, Ithaca, NY, USA Friedemann Mattern ETH Zurich, Switzerland John C. Mitchell Stanford University, CA, USA Moni Naor Weizmann Institute of Science, Rehovot, Israel Oscar Nierstrasz University of Bern, Switzerland C. Pandu Rangan Indian Institute of Technology, Madras, India Bernhard Steffen University of Dortmund, Germany Madhu Sudan Massachusetts Institute of Technology, MA, USA Demetri Terzopoulos New York University, NY, USA Doug Tygar University of California, Berkeley, CA, USA Moshe Y. Vardi Rice University, Houston, TX, USA Gerhard Weikum Max-Planck Institute of Computer Science, Saarbruecken, Germany
3049
Maurice Bruynooghe Kung-Kiu Lau (Eds.)
Program Development in Computational Logic A Decade of Research Advances in Logic-Based Program Development
13
Volume Editors Maurice Bruynooghe Katholieke Universiteit Leuven Department of Computer Science Celestijnenlaan 200A, 3001 Heverlee, Belgium E-mail:
[email protected] Kung-Kiu Lau University of Manchester Department of Computer Science Manchester M13 9PL, United Kingdom E-mail:
[email protected]
Library of Congress Control Number: 2004107507 CR Subject Classification (1998): F.3.1, D.1.1, D.1.6, I.2.2, F.4.1, D.2, D.3 ISSN 0302-9743 ISBN 3-540-22152-2 Springer-Verlag Berlin Heidelberg New York This work is subject to copyright. All rights are reserved, whether the whole or part of the material is concerned, specifically the rights of translation, reprinting, re-use of illustrations, recitation, broadcasting, reproduction on microfilms or in any other way, and storage in data banks. Duplication of this publication or parts thereof is permitted only under the provisions of the German Copyright Law of September 9, 1965, in its current version, and permission for use must always be obtained from Springer-Verlag. Violations are liable to prosecution under the German Copyright Law. Springer-Verlag is a part of Springer Science+Business Media springeronline.com © Springer-Verlag Berlin Heidelberg 2004 Printed in Germany Typesetting: Camera-ready by author, data conversion by Boller Mediendesign Printed on acid-free paper SPIN: 11012061 06/3142 543210
Preface
The tenth anniversary of the LOPSTR1 symposium provided the incentive for this volume. LOPSTR started in 1991 as a workshop on logic program synthesis and transformation, but later it broadened its scope to logic-based program development in general, that is, program development in computational logic, and hence the title of this volume. The motivating force behind LOPSTR has been the belief that declarative paradigms such as logic programming are better suited to program development tasks than traditional non-declarative ones such as the imperative paradigm. Specification, synthesis, transformation or specialization, analysis, debugging and verification can all be given logical foundations, thus providing a unifying framework for the whole development process. In the past 10 years or so, such a theoretical framework has indeed begun to emerge. Even tools have been implemented for analysis, verification and specialization. However, it is fair to say that so far the focus has largely been on programmingin-the-small. So the future challenge is to apply or extend these techniques to programming-in-the-large, in order to tackle software engineering in the real world. Returning to this volume, our aim is to present a collection of papers that reflect significant research efforts over the past 10 years. These papers cover the whole development process: specification, synthesis, analysis, transformation and specialization, as well as semantics and systems. We would like to thank all the authors for their valuable contributions that made this volume possible. We also thank the reviewers for performing their arduous task meticulously and professionally: Annalisa Bossi, Nicoletta Cocco, Bart Demoen, Danny De Schreye, Yves Deville, Sandro Etalle, Pierre Flener, John Gallagher, Samir Genaim, Gopal Gupta, Ian Hayes, Patricia Hill, Andy King, Vitaly Lagoon, Michael Leuschel, Naomi Lindenstrauss, Nancy Mazur, Mario Ornaghi, Dino Pedreschi, Alberto Pettorossi, Maurizio Proietti, CR Ramakrishnan, Sabina Rossi, Abhik Roychoudhury, Salvatore Ruggieri, Tom Schrijvers, Alexander Serebrenik, Jan-Georg Smaus, Wim Vanhoof and Sofie Verbaeten.
April 2004
1
Maurice Bruynooghe and Kung-Kiu Lau
http://www.cs.man.ac.uk/∼kung-kiu/lopstr/
Table of Contents
Specification and Synthesis Specifying Compositional Units for Correct Program Development in Computational Logic . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Kung-Kiu Lau and Mario Ornaghi Synthesis of Programs in Computational Logic . . . . . . . . . . . . . . . . . . . . . . . . David Basin, Yves Deville, Pierre Flener, Andreas Hamfelt, and Jørgen Fischer Nilsson Developing Logic Programs from Specifications Using Stepwise Refinement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Robert Colvin, Lindsay Groves, Ian J. Hayes, David Hemer, Ray Nickson, and Paul Strooper
1
30
66
Semantics Declarative Semantics of Input Consuming Logic Programs . . . . . . . . . . . . . Annalisa Bossi, Nicoletta Cocco, Sandro Etalle, and Sabina Rossi
90
On the Semantics of Logic Program Composition . . . . . . . . . . . . . . . . . . . . . 115 Antonio Brogi
Analysis Analysing Logic Programs by Reasoning Backwards . . . . . . . . . . . . . . . . . . . 152 Jacob M. Howe, Andy King, and Lunjin Lu Binding Time Analysis for Mercury . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 189 Wim Vanhoof, Maurice Bruynooghe, and Michael Leuschel A Generic Framework for Context-Sensitive Analysis of Modular Programs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 233 Germ´ an Puebla, Jes´ us Correas, Manuel V. Hermenegildo, Francisco Bueno, Mar´ıa Garc´ıa de la Banda, Kim Marriott, and Peter J. Stuckey
Transformation and Specialisation Unfold/Fold Transformations for Automated Verification of Parameterized Concurrent Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 261 Abhik Roychoudhury and C.R. Ramakrishnan
VIII
Table of Contents
Transformation Rules for Locally Stratified Constraint Logic Programs . . 291 Fabio Fioravanti, Alberto Pettorossi, and Maurizio Proietti Specialising Interpreters Using Offline Partial Deduction . . . . . . . . . . . . . . . 340 Michael Leuschel, Stephen J. Craig, Maurice Bruynooghe, and Wim Vanhoof
Termination Characterisations of Termination in Logic Programming . . . . . . . . . . . . . . . 376 Dino Pedreschi, Salvatore Ruggieri, and Jan-Georg Smaus On the Inference of Natural Level Mappings . . . . . . . . . . . . . . . . . . . . . . . . . . 432 Jonathan C. Martin and Andy King Proving Termination for Logic Programs by the Query-Mapping Pairs Approach . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 453 Naomi Lindenstrauss, Yehoshua Sagiv, and Alexander Serebrenik
Systems Herbrand Constraint Solving in HAL . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 499 Bart Demoen, Mar´ıa Garc´ıa de la Banda, Warwick Harvey, Kim Marriott, David Overton, and Peter J. Stuckey
Author Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 539
Specifying Compositional Units for Correct Program Development in Computational Logic Kung-Kiu Lau1 and Mario Ornaghi2 1
2
Department of Computer Science, University of Manchester Manchester M13 9PL, United Kingdom
[email protected] Dipartimento di Scienze dell’Informazione, Universita’ degli studi di Milano Via Comelico 39/41, 20135 Milano, Italy
[email protected]
Abstract. In order to provide a formalism for defining program correctness and to reason about program development in Computational Logic, we believe that it is better to distinguish between specifications and programs. To this end, we have developed a general approach to specification that is based on a model-theoretic semantics. In our previous work, we have shown how to define specifications and program correctness for open logic programs. In particular we have defined a notion of correctness called steadfastness, that captures at once modularity, reusability and correctness. In this paper, we review our past work and we show how it can be used to define compositional units that can be correctly reused in modular or component-based software development.
1
Introduction
In software engineering, requirements analysis, design and implementation are distinctly separate phases of the development process [18], as they employ different methods and produce different artefacts. In requirements analysis and design, specifications play a central role, as a frame of reference capturing the requirements and the design decisions. By contrast, data and programs only appear in the implementation phase, towards the end of the development process. There is therefore a clear distinction between specifications and programs. In Computational Logic, however, this distinction is usually not maintained. This is because there is a widely held view that logic programs are executable specifications and therefore there is no need to produce specifications before the implementation phase of the development process. We believe that undervaluing specifications in this manner is not an ideal platform for program development. If programs are indistinguishable from specifications, then how do we define program correctness, and how do we reason about program development? We hold the view that the meaning of correctness must be defined in terms of something other than logic programs themselves. We are not alone in this, see e.g., [17, p. 410]. In our view, the specification should axiomatise all our relevant knowledge of the problem context and the necessary data types, whereas, M. Bruynooghe and K.-K. Lau (Eds.): Program Development in CL, LNCS 3049, pp. 1–29, 2004. c Springer-Verlag Berlin Heidelberg 2004
2
Kung-Kiu Lau and Mario Ornaghi
for complexity reasons, programs rightly capture only what is strictly necessary for computing. In the process of extracting programs from specifications, a lot of knowledge is lost, making programs much weaker axiomatisations. This suggests that specifying and programming are different activities, involving different methodological aspects. Thus, we take the view that specifications should be clearly distinguished from programs, especially for the purpose of program development. Indeed, we have shown (in [28,29]) that in Computational Logic, not only can we maintain this distinction, but we can also define various kinds of specifications for different purposes. Moreover, we can also define correctness with respect to these specifications. Our semantics for specification and correctness is model-theoretic. The declarative nature of such a semantics allows us to define steadfastness [34], a notion of correctness that captures at once modularity, reusability and correctness. Open programs are incomplete pieces of code that can be (re)used in many different admissible situations, by closing them (by adding the missing code) in many different ways. Steadfastness of an open program P is pre-proved correctness of the various closures of P , with respect to the different meanings that the specification of P assumes in the admissible situations. For correct reuse, we need to know when a situation is admissible. This knowledge is given by the problem context. We have formalised problem context as a specification framework [27], namely, a first-order theory that axiomatises the problem context, characterises the admissible situations as its (intended) models, and is used to write specifications and to reason about them. In this paper, we review our work in specification and correctness of logic programs, including steadfastness. Our purpose is to discuss the role of steadfastness for correct software development. In particular, we are interested in modularity and reuse, which are key aspects of software development. Our work is centred on the notion of a compositional unit. A compositional unit is a software component, which is commonly defined as a unit of composition with contractually specified interfaces and context dependencies only [46]. The interfaces declare the imported and exported operations, and the context dependencies specify the constraints that must be satisfied in order to correctly (re)use them. Throughout the paper, we will not refer to compositional units as software components, however, for the simple reason that as yet there is no standard definition for the latter (although the one we used above [46] is widely accepted). So we prefer to avoid any unnecessary confusion. In our compositional units, the interfaces and the context dependencies are declaratively specified in the context of the specification framework F axiomatising the problem context. F gives a precise semantics to specifications and allows us to reason about the correctness of programs, as well as their correct reuse. Thus, in our formalisation, a compositional unit has a three-tier structure, with separate levels for framework, specifications and programs. We introduce compositional units in Section 2, and consider the three levels separately. We focus on model-theoretic semantics of frameworks and specifications, and on steadfastness (i.e., open program correctness).
Specifying Compositional Units for Correct Program Development
3
In Section 3, we show how the proposed formalisation of compositional units can be used to support correct reuse. Our aim is to highlight the aspects related to specifications, so we consider only the aspects related to the framework and the specification levels, while assuming the possibility of deriving (synthesising) steadfast programs from specifications. At the end of each section we briefly discuss and compare our results with related work, and finally in the conclusion we comment on future developments.
2
Compositional Units
In our approach, compositional units represent correctly reusable units of specifications and correct open programs. Our view is that specifications and programs are not stand-alone entities, but are always to be considered in the light of a problem context. The latter plays a central role: it is the semantic context in which specifications and program correctness assume their appropriate meaning, and it contains the necessary knowledge for reasoning about correctness and correct reuse. This is reflected in the three-tier structure (with model-theoretic semantics) of a compositional unit, as illustrated in Figure 1.
Compositional Unit K Framework F (ΠF ⇒ ∆F ) Signature Σ Axioms X Theorems T Specifications Sp1 ; . . . ; Spn ; RD1 ; . . . RDk Programs Pid1 : Sπ1 ⇒ Sδ1 {C1 }; . . . ; Pidh : Sπh ⇒ Sδh {Ch }
Fig. 1. A three-tier formalism.
At the top level of a compositional unit K, we have a specification framework F , or framework for short, that embodies an axiomatisation of the problem context. F has a signature Σ, a set X of axioms, a set T of theorems, a list ΠF of open symbols, and a list ∆F of defined symbols. The syntax ΠF ⇒ ∆F indicates that the axioms of F fix (the meaning of) the symbols ∆F whenever F is composed with frameworks that fix ΠF . The defined and open symbols belong to the signature Σ, which may also contain closed symbols, namely symbols
4
Kung-Kiu Lau and Mario Ornaghi
defined completely by the axioms (i.e., independently from ΠF ). Frameworks are explained in Section 2.1, and framework composition is explained in Section 3.1. In the middle, we have the specification section. Its role is to bridge the gap between the framework F and the chosen programming language. So far, we have considered only logic programs, and the corresponding specification formalism is explained in Section 2.2. The specification section contains the specifications Sp1 , . . ., Spn of the program predicates occurring in the program section. It may also contain a set of specification reduction theorems theorems RD1 , . . . , RDk , that are useful to reason about correct reuse. Specification reduction is explained in Section 3.2. At the bottom, we have the program section. Programs are open logic (or constraint logic) programs. An open program Pidi : Sπi ⇒ Sδi {Ci } (1 ≤ i ≤ h) has an identifier idi , an interface specification Sπi ⇒ Sδi and a set {Ci } of implementation clauses. Sπi and Sδi are lists of specifications defined in the specification section. An interface specification contains all the information needed to correctly reuse a correct program. Programs and correctness are explained in Section 2.3. Correct reuse is explained in Section 3.3. 2.1
Specification Frameworks
A specification framework F is defined in the context of first-order logic, and contains the relevant knowledge of the necessary concepts and data types for building a model of the application at hand. We distinguish between closed and open frameworks. A closed framework F = Σ, X, T has a signature Σ, a set X of axioms, and a set T of theorems. It has no open and defined symbols, that is, all the symbols of Σ are closed. Example 1. An example of closed framework is first-order arithmetic NAT = ΣNat , XNat , TNat , introduced by the following syntax:3 Framework NAT ; decls: Nat : sort; 0 : [ ] → Nat; s : [Nat] → Nat; + , ∗ : [Nat, Nat] → Nat; axs: Nat : construct(0, s : Nat); + : i + 0 = i; i + s(j) = s(i + j); : i ∗ 0 = 0; i ∗ s(j) = i ∗ j + i; thms: i + j = j + i; ... 3
In all the examples, we will omit the outermost universal quantifiers, but their omnipresence should be implicitly understood.
Specifying Compositional Units for Correct Program Development
5
The signature ΣNat , introduced in the declaration section decls, is the signature of Peano’s arithmetic. The axioms XNat , introduced in the axs section, are the usual ones of first-order arithmetic. 0 and s are the constructors of Nat and their axioms, which we call the constructor axioms for Nat, are collectively indicated by construct(0, s : Nat). The latter contains Clark’s equality theory [35] for 0 and s, as well as all the instances of the first-order induction schema. NAT has been widely studied, and there are a lot of known theorems (in section thms), including for example the associative, commutative and distributive laws. Theorems are an important part of a framework. However, they are not relevant in the definitions that follow, so we will not refer to them explicitly here. For closed frameworks we adopt isoinitial semantics, that is, we choose the intended model of F = Σ, X to be a reachable isoinitial model, defined as follows: Definition 1 (Reachable Isoinitial Model [5]). Let X be a set of Σ-axioms. A Σ-structure i is an isoinitial model of X iff, for every model m of X, there is a unique isomorphic embedding i : i → m. A model i is reachable if its elements can be represented by ground terms. Definition 2 (Adequate Closed Frameworks [30]). A closed framework F = Σ, X is adequate iff there is a reachable isoinitial model i of X that we call ‘the’ intended model of F . In fact i is one of many intended models of F , all of which are isomorphic. So i is unique up to isomorphism, and hence our (ab)use of ‘the’. As shown in [5], adequacy entails the computability of the operations and predicates of the signature. Example 2. NAT is an adequate closed framework. Its intended model is the standard structure N of natural numbers (N is a reachable isoinitial model of XNat ). N interprets Nat as the set of natural numbers, and s, + and ∗ as the successor, sum and product function, respectively. The adequacy of a closed framework is not a decidable property. We have the following useful proof-theoretic characterisation, which can be seen as a “richness requirement” implicit in isoinitial semantics [31]: Definition 3 (Atomic Completeness). A framework F = Σ, X is atomically complete iff, for every ground atomic formula A, either X A or X ¬A. Theorem 1 (Adequacy Condition [38]). A closed framework F = Σ, X is adequate iff it has at least one reachable model and is atomically complete. Closed adequate frameworks can be built incrementally, starting from a closed adequate kernel, by means of adequate extensions.
6
Kung-Kiu Lau and Mario Ornaghi
Definition 4 (Adequate Extensions [30]). An adequate extension of an adequate closed framework F = Σ, X is an adequate closed framework Fδ = Σ ∪ δ, X ∪ Dδ such that: a) Dδ is a set of (Σ ∪ δ)-axioms, axiomatising a set of new (i.e., not in Σ) symbols δ; b) the Σ-reduct i|Σ of the intended model i of Fδ is the intended model of F . The notions of reduct and expansion are standard in logic [4]. The Σ-reduct i = i|Σ forgets the interpretation of the symbols not in Σ, in our case the new symbols δ. Conversely, i is said to be a (Σ ∪ δ)-expansion of i , that is, a (Σ ∪ δ)-expansion is a (Σ ∪ δ)-interpretation that preserves the meaning of the old Σ-symbols, and interprets the new δ arbitrarily. In Definition 4, by b), the intended model i of an adequate extension is an expansion of the old intended model, that is, adequacy entails that the meaning of the old symbols is preserved. If the axioms Dδ of an adequate extension are explicit definitions, we say that they are adequate explicit definitions. Since they are important in our approach, we briefly recall them. An explicit definition of a new relation r has the form ∀x • r(x) ↔ R(x), where x indicates a tuple of variables and (as usual) “• ” extends the scope of a quantifier to the longest subformula next to it. The explicit definition of a new function f has the form ∀x • F (x, f (x)), where R(x) and F (x, y) are formulas of the framework that contain free only the indicated variables. The explicit definition of f has the proof obligation X ∀x • ∃!y • F (x, y), where X are the framework axioms (as usual, ∃!y means unique existence). R(x) is called the definens (or defining formula) of r, and F (x, y) the definiens (or defining formula) of f . Explicit definitions have nice properties. They are purely declarative, in the following sense: they define the new symbols purely in terms of the old ones, that is, in a non-recursive way. This declarative character is reflected by the following eliminability property, where Σ is the signature of the framework and δ are the new explicitly defined symbols: the extension is conservative (i.e., no new Σ-theorem is added) and every formula of Σ + δ is provably equivalent to a corresponding formula of the old signature Σ. Moreover, if we start from a sufficiently expressive kernel, most of the relevant relations and functions can be explicitly defined. Finally, we can prove: Proposition 1. If the definiens of an explicit definition is quantifier-free, then the definition is adequate. If the definiens is not quantifier-free, adequacy must be checked. To state the adequacy of closed frameworks and of explicit definitions, we can apply proof methods based on logic program synthesis [26,27] or constructive logic [38]. Example 3. The kernel NAT of Example 1 is sufficiently expressive in the following sense. Every recursively enumerable relation r can be introduced by an
Specifying Compositional Units for Correct Program Development
7
explicit definition.4 For example, we can define the ordering relations ≤ and < by the explicit definitions: D≤ : i ≤ j ↔ ∃k • i + k = j; D< : i < j ↔ i ≤ j ∧ ¬i = j. Since the outermost universal quantifiers are implicitly present, D≤ is the closed formula ∀i, j • i ≤ j ↔ ∃k • i + k = j (similarly, D< is understood to be universally closed). Since the definiens ∃k • i + k = j of ≤ is quantified, adequacy of D≤ must be checked. It can be proved by logic program synthesis, as follows. (a) We derive the following clauses in NAT + D≤ : P≤ :
0≤i← s(i) ≤ s(j) ← i ≤ j.
(b) In NAT + D≤ we prove the only-if part of the completed definition [35] of ≤ in P≤ (the if part is guaranteed by a)). (c) Finally, we prove that P≤ existentially terminates, i.e., for every ground atom A, the goal ← A finitely fails or has at least one successful derivation (with program P≤ ). By (a), (b) and (c) we get ([27], Theorem 11) that the extension by D≤ is adequate. By the way, adequacy entails that the new predicate ≤ is computable. We do not have to check the adequacy of D< , because its definiens is quantifier free. D< uses ≤. However, an explicit definition of < and a proof of its adequacy can be given directly in NAT , by the eliminability of explicit definitions. Thus we could define < first, prove its adequacy, and then define ≤ on top of <. That is, the order of explicit definitions is not relevant. We can explicitly define functions, for example the integer square root sqrt: Dsqrt : sqrt(i) ∗ sqrt(i) ≤ i ∧ i < s(sqrt(i)) ∗ s(sqrt(i)). The proof obligation ∀i • ∃!j • j ∗ j ≤ i ∧ i < s(j) ∗ s(j) can be proved in NAT by induction. Adequacy follows from the fact that the definiens j ∗ j ≤ i ∧ i < s(j) ∗ s(j) is quantifier free. An open framework F (Π ⇒ ∆) = Σ, X represents an incomplete axiomatisation. It has a non-empty import list Π, containing the symbols left open by the axioms, and a (possibly empty) disjoint export list ∆, containing the symbols that are defined by the axioms, in terms of the open ones. The closed symbols are the symbols of the signature that are not in Π ∪ ∆, and their meaning is fixed in a unique way by the axioms. We distinguish three sets of axioms, where ΣK is the sub-signature of the closed symbols: 4
Every recursively enumerable relation is Diophantine (Matijacevic theorem [37]).
8
Kung-Kiu Lau and Mario Ornaghi
– the kernel axioms XK = X|ΣK (. . . |ΣK is the subset of the axioms with symbols from Σk ); the kernel axioms axiomatise the closed symbols, that is, FK = ΣK , XK must be an adequate closed framework, that we call the closed kernel ; – the constraints XC = (X|(ΣK ∪ Π)) \ XK , which constrain the possible interpretations of the open symbols Π; – and the definition axioms XD = X \ (XK ∪ XC ), which fix the meaning of the defined symbols ∆, in terms of the open and closed symbols. Example 4. The following open framework axiomatises lists with generic elements X and a generic total ordering on X. From now on, in the examples, the variables of sort X will begin with x, y, z, w, those of sort Nat with i, j, h, k, and those of sort ListX with l, m, n, o. Framework LIST (X, ⇒ ListX, nil, ., @, nocc); kernel: NAT ; decls: X : sort; ListX : sort; : [X, X]; nil : [ ] → ListX; . : [X, ListX] → ListX; @( , ) : [X, Nat, ListX]; nocc : [X, ListX] → Nat; ListX : construct(nil, . : ListX); @: x@(0, l) ↔ ∃y, m • l = y.m ∧ x = y; x@(s(i), l) ↔ ∃y, m • l = y.m ∧ x@(i, m); nocc : nocc(x, nil) = 0; x = y → nocc(x, y.l) = nocc(x, l) + 1; ¬x = y → nocc(x, y.l) = nocc(x, l); constrs: : TotalOrdering().
defaxs:
The signature ΣNat , the axioms XNat and the theorems TNat of the imported kernel NAT are automatically included. In the definition axioms defaxs, nil and “.” are the list constructors, as indicated by construct(nil, . : ListX), which contains Clark’s equality theory and structural induction on constructors; x@(i, l) means that the element x occurs at position i in the list l, where positions start from 0; nocc(x, l) is the number of occurrences of the element x in the list l; by the constraint axioms constrs, is a total ordering relation. To specify the basic operations on the ADT of lists, the closed kernel NAT is not necessary. We have imported it for specification and reasoning purposes. Indeed, by using natural numbers we can introduce @ and nocc. The resulting language and axiomatic system give a rich starting framework, which allows us to explicitly define the usual operations on lists and ordered lists, and to reason about them (see Example 6).
Specifying Compositional Units for Correct Program Development
9
An open framework F has a class of not necessarily isomorphic intended models, since XK ∪ XC allows many (ΣK ∪ Π)-interpretations, that we call premodels. The semantics considered here is a variant of the one presented in [30]. A pre-model is an expansion of the intended model of the kernel that satisfies the constraints XC . For every pre-model p, the axioms of F fix a corresponding intended p-model ip , defined as follows. A p-model of F is a Σ-model m of X such that m|(ΣK ∪ Π) = p, that is, m coincides with p over the closed and open symbols. Since Π may contain open sorts, we consider Π-reachable models, where Π-reachability is reachability in an expansion containing a new constant for each element of each open sort. Definition 5 (p-isoinitial Models). A p-model i is a p-isoinitial model of F iff, for every p-model m, there is a unique isomorphic embedding i : i → m such that i is the identity over the open sorts. Definition 6 (Adequate Open Frameworks and Intended Models). An open framework F is adequate iff, for every pre-model p of F , there is a Πreachable p-isoinitial model ip , that we call the intended p-model of F . m is an intended model of F iff there is a pre-model p of F such that m is the intended p-model of F . For every pre-model p, the intended p-model is unique up to isomorphism. Intended models with non-isomorphic pre-models are, of course, non-isomorphic. We consider closed frameworks as a limiting case, where the kernel coincides with the whole framework and the unique intended model coincides with the unique pre-model. Example 5. LIST is an adequate open framework. In it, a pre-model p coincides with N for the kernel signature ΣNat and interprets X as any set with a total ordering . The intended p-model of LIST interprets ListX as the set of the finite lists with elements from X, and the other defined symbols in the way already explained in Example 4. Adequate open frameworks can be built incrementally, by adequate extensions, where the intended models of an adequate extension F of a framework F are expansions of intended models of F . Definition 7 (Adequate Extensions). A framework F is an adequate extension of an adequate open framework F iff F is an adequate open or closed framework, the signature and the axioms of F contain those of F , the kernel signature of F contains the kernel signature of F , and for every intended model i of F , the reduct i |Σ is an intended model of F . In the limiting case, an adequate extension F of an open framework F may be a closed framework. In this case, we say that F is an instance of F , and the axioms that “instantiate” (i.e., close) the open symbols are called closure axioms. A set of closure axioms is called a closure. Closures will be considered in Section 3.1, together with other framework operations.
10
Kung-Kiu Lau and Mario Ornaghi
In general, the adequacy of an extension is not decidable, but we may have different kinds of extensions, with different adequacy conditions. In particular, we distinguish: – Parameter extensions. In this case new parameters and/or new constraints are added. Parameter extensions are adequate iff, adding new constraints, consistency is preserved. – Defined symbol extensions. In this case new defined symbols, together with the corresponding definition axioms, are added. Adequate explicit definitions are still useful for introducing new defined symbols, and adequacy can be stated in a way similar to those mentioned before for closed framework extensions (by program synthesis or constructive logic [27,38]). Proposition 1 still holds. – Kernel extensions. In this case the closed kernel is extended by new closed symbols, as already shown for closed frameworks. Example 6. The framework LIST (X, ⇒ ListX, nil, ., @, nocc) can be obtained by extending the framework LIST (X ⇒ ListX, nil, ., @, nocc), without and without constraint axioms, by the parameter : [X, X] constrained by TotalOrdering(). The kernel NAT can be extended by explicitly defining the most useful operations and predicates on natural numbers. The defined symbols can be extended by the relevant operations on lists, by means of explicit definitions. For example, the definitions of list membership, length, concatenation and permutation are:5 D∈ Dlen D| Dperm
: x ∈ l ↔ nocc(x, l) > 0 : ∀i • (∃x • x@(i, l)) ↔ i < len(l) : ∀i, x • (i < len(l) → (x@(i, l) ↔ x@(i, l|m))) ∧ (len(l) ≤ i → (x@(i, m) ↔ x@(i + len(l), l|m))) : perm(l, m) ↔ ∀x • nocc(x, l) = nocc(x, m)
D∈ gives rise to an adequate extension, because its definiens is quantifier free. The definiens of Dlen is ∀i • (∃x • x@(i, l)) ↔ i < k and the proof obligation requires a proof of ∀l • ∃!k • ∀i • (∃x • x@(i, l)) ↔ i < k. Since the definiens is quantified, adequacy must be checked (and can be proved), by constructive proofs or by program synthesis. Adequacy must be checked (and can be proved) also for D| and Dperm . Using : [X, X], we can also define operations on ordered lists, like l L m (lexicographic ordering on lists), ord(l) (l is an ordered list), and so on. Their properties can be proved using the total ordering constraints. For example, we can prove that the lexicographic ordering L is, in turn, a total ordering. 2.2
Specifications
In a compositional unit K, specifications assume their proper meaning only in the context of the framework F . In this section we define formally what we 5
In Dlen and D| , the universal quantifiers of the definiens have not been omitted.
Specifying Compositional Units for Correct Program Development
11
mean by specifications in F and we show some examples. We maintain a strict distinction between specification frameworks and (program) specifications and, to distinguish the function and relation symbols of the framework from those computed by programs, the latter will be called (program) operations. Definition 8 (Specifications and S-expansions). Let F (Π ⇒ ∆) = Σ, X be a framework. A specification Sω in (the context of) F is a set of closed (Σ + ω)-formulas, that define a set of operations ω in terms of F . An Sω -expansion of a model m of F is a (Σ + ω)-expansion m of m such that m |= Sω . That is, Sω can be interpreted as an expansion operator that associates with every intended model of F the corresponding Sω -expansions, namely the expansions that interpret the specified operations according to Sω . Definition 9 (Strict Specifications). A specification Sω is strict in a framework F , if, for every model m of F , there is only one Sω -expansion. It is nonstrict otherwise. Now we list different kinds of strict and non-strict specifications considered in [28], essentially based on explicit definitions. The specification formalism considered here is tailored to logic programs with definite clauses in a many-sorted signature. Program semantics is based on minimum Herbrand models, where program data (those used in programs) coincide with ground terms. We assume that the signature ΣD of program data is pre-defined by the framework F , and that, for every closed or defined sort s of ΣD , F contains the axioms construct(c1 , . . . , cn : s), where c1 , . . . , cn are the constructors of s. They are the unique operations of sort s that can be used in logic programs. This assumption concerns Herbrand models of standard logic programs, where construct(. . .) holds, but our treatment readily extends to the specification formalism for constraint logic programs by assuming that ΣD is the constraint signature, and is pre-defined by the framework. Since in logic programs only program predicates are not pre-defined, we have to specify only them. There are different forms of specifications. If-and-Only-if Specifications. An if-and-only-if specification in a framework F is an explicit definition of a new predicate r: Sr :
∀x • r(x) ↔ R(x)
By the well known properties of explicit definitions, for every model m of the framework F , there is only one Sr -expansion of m, that is, Sr is strict. Example 7. In NAT we can specify, for example, the following predicates: Sdiv : div(i, j, h, k) ↔ i = j ∗ h + k ∧ k < j; Sdivides : divides(i, j) ↔ ∃h • div(j, i, h, 0); Sprime : prime(i) ↔ ∀j • divides(j, i) → j = 1 ∨ j = i;
12
Kung-Kiu Lau and Mario Ornaghi
Super-and-sub Specifications. A super-and-sub specification in a framework F is of the form Sr :
∀x • (Rsub (x) → r(x)) ∧ (r(x) → Rsuper (x))
where Rsub (x) and Rsuper (x) are two formulas of F such that F ∀x • Rsub (x) → Rsuper (x). The implication ∀x • Rsub (x) → Rsuper (x) is satisfied by the models of F . Therefore, in every intended model i, the relation Rsub in i, i.e., the set of values x such that i |= Rsub (x), is a sub-relation of the relation Rsuper , and the specified relation r is any relation that is a super-relation of Rsub but is a sub-relation of Rsuper . Conditional Specifications. A conditional specification of a new relation r in a framework F has the form: ∀x, y • I C(x) → (r(x, y) ↔ R(x, y))
(1)
where I C(x) is the input condition, and R(x, y) is the input-output condition. Both I C(x) and R(x, y) are formulas of F . (1) specifies r(x, y) only when the input condition I C(x) is true, while nothing is required if the input condition is false. That is, I C(x) states that r(x, y) is to be called only in contexts that make it true. This fact allows us to assume I C(x) when reasoning about correct reuse, as shown in Section 3.2. (1) is equivalent to the following super-and-sub specification, which allows us to apply the results of [34] in correctness proofs: ∀x, y • (I C(x) ∧ R(x, y) → r(x, y)) ∧ (r(x, y) → ¬I C(x) ∨ R(x, y)) Example 8. In the open framework LIST (X, ⇒ ListX, nil, ., @, nocc), we have for example the following specification: Ssort : sort(l, m) ↔ perm(l, m) ∧ ord(m); Smerge : ord(l) ∧ ord(m) → (merge(l, m, o) ↔ ord(o) ∧ perm(l|m, o)); Ssplit : (len(l) > 1 ∧ split(l, m, n) → perm(l, m|n) ∧ len(m) < len(l)∧ len(n) < len(l)) ∧ (len(l) > 1 → ∃m, n • split(l, m, n)); Ssort is an if-and-only-if specification, Smerge is a conditional specification. By the input condition, merge(l, m) is to be called only in contexts where the input lists are ordered. If they are not, o is not required to be ordered. Ssplit is an example of another form of non-strict specification (called a selector specification) that we do not discuss here (see [28]). 2.3
Interface Specifications, Programs, and Correctness
Here we consider correctness of (logic) programs with respect to interface specifications. In Section 3.2, we will consider the role of interface specifications in correct reuse. We start by introducing some terminology.
Specifying Compositional Units for Correct Program Development
13
The signature ΣP of a program P contains the declarations of its predicate, constant and function symbols, and the sorts occurring in such declarations. The data signature of P is the subsignature of its sort, constant and function symbols. According to the previous section, the data signature belongs to the framework signature. We will distinguish open and closed programs, as follows. The defined predicates of a program P are those that occur in the head of at least one clause of P , while the (possible) open predicates of P are those that occur only in the body. A program P is open if its signature contains at least one open sort or predicate. It is closed if no open symbol belongs to its signature. A interface specification for an open program P is of the form Sπ ⇒ Sδ , where Sπ are specifications of a set π of predicates that includes all the open predicates of P , and Sδ are specifications of a set δ of predicates that are included in the defined predicates of P . We will write P : Sπ ⇒ Sδ to indicate that P has specification Sπ ⇒ Sδ . If P has no open predicates, then Sπ will be empty. In this case, we write P : ⇒ Sδ . Example 9. In a compositional unit with open framework LIST , we can declare the following open sorting program (where Ssplit , Smerge and Ssort are as shown in Example 8): Program Psort : Ssplit , Smerge ⇒ Ssort {
sort(nil, nil) ← sort(x.nil, x.nil) ← sort(x.y.l, o) ← split(x.y.l), m, n), sort(m, m1 ), sort(n, n1 ), merge(m1 , n1 , o). } Programs may be open independently from the framework, i.e., closed frameworks may contain open programs. For example, in the closed framework NAT , we can declare: Program Pprod : Ssum ⇒ Sprod { prod(i, 0, 0) ← prod(i, s(j), h) ← prod(i, j, k), sum(k, i, h). } { where:
Ssum : sum(x, y, z) ↔ z = x + y; Sprod : prod(x, y, z) ↔ z = x · y.
Now we can define program correctness. We will first explain the correctness of closed programs in closed frameworks, because it is simpler and more intuitive. Then we introduce correctness of open programs.
14
Kung-Kiu Lau and Mario Ornaghi
Correctness of Closed Programs. A closed program P has only defined predicates, and an interface specification of P is of the form ⇒ Sδ . For simplicity, we will consider the case of interface specifications ⇒ Sr with one defined predicate r (the extension to ⇒ Sr1 , . . . , Srk , with k > 1, is immediate). We define program correctness in a closed framework as follows: Definition 10 (Correctness of Closed Programs). Let F be a closed framework with intended model i. Let Sr be a specification of a predicate r, P be a program that computes r, and h be the minimum Herbrand of P . P is correct with respect to the interface specification ⇒ Sr iff the interpretation of r in h coincides with the interpretation of r in one of the Sr -expansions of i. For conciseness, we will say that P : ⇒ Sr is correct, to indicate that P is correct with respect to ⇒ Sr . For a strict specification Sr , there is only one Sr -expansion of i, that is, the new symbol r defined by Sr has a unique interpretation in i, and one in h. Correctness of P : ⇒ Sr means that the two interpretations of r coincide, or, at least, are isomorphic. This is illustrated in Figure 2. Framework F
interpretation of r in isoinitial model i of F
Sˆr Pr
interpretation of r in minimum Herbrand model h of Pr
Fig. 2. Strict specifications.
If Sr is not strict, then r has many interpretations with respect to i. Correctness of P : ⇒ Sr in this case means that the interpretation of r in h coincides with one of the interpretations of r with respect to i. This is illustrated in Figure 3. Framework F Sr Pr
...
interpretations of r in isoinitial model i of F interpretation of r in minimum Herbrand model h of Pr
Fig. 3. Non-strict specifications.
Specifying Compositional Units for Correct Program Development
15
Steadfastness: Correctness of Open Programs. Now we consider open programs in open frameworks, and we discuss the associated notion of correctness. An open program is correct if it behaves as expected in all the circumstances. We called this property steadfastness [34]. The correctness relation between an interface specification Sπ ⇒ Sδ and an open program P cannot be defined as in Definition 10, because open frameworks may have many intended models and we cannot use minimum Herbrand models as the semantics of open programs, since in the minimum Herbrand models, open relations are assumed to be empty, and therefore cannot play the role of parameters. So in [34] we introduced minimum j-models, together with the notion of steadfastness, to serve as the basis for a model and proof theory of the correctness of open programs. Here we first recall the definition of steadfastness informally, and then define correctness of open programs, and give its relevant properties. A pre-signature for an open program P is a signature Ω that contains the data signature and the open predicates of P , but not the defined predicates of P . A pre-interpretation in Ω is an Ω-interpretation. That is, symbols of Ω are considered to be open, and a pre-interpretation j interprets them arbitrarily. In contrast, the intended meaning of the defined predicates of P is stated by its clauses, in terms of j. Let P be a program with defined predicates δ. To define the intended meaning of δ in a pre-interpretation j, we introduce j-models. A j-model of P is a model m of P , such that m | Ω = j, i.e., m coincides with j over Ω. Since two distinct j-models m and n differ only for the interpretation of δ, we can compare them by looking at δ: we say that m is contained in n, written m ⊆δ n, iff the interpretation of (the predicates of) δ in m is contained in that of δ in n. We can show that a minimum j-model (with respect to ⊆δ ) exists. The minimum j-model of P will be indicated by jP , and it represents the interpretation of δ stated by the program P , in the pre-interpretation j. Using minimum j-models, steadfastness in an interpretation can be defined as follows. Definition 11 (Steadfastness). Let P be an open program, Ω be a pre-signature for P , and r be a predicate defined by P . P is steadfast for r in a (Ω + r)interpretation i if and only if the interpretation of r in its minimum i | Ω-model coincides with the interpretation of r in i. More intuitively, steadfastness in i for r means that the interpretation of r in i coincides with the interpretation of r stated by P , when the open symbols Ω are interpreted as in i (i.e., when the pre-interpretation is i | Ω). Consider for example the open program P in the context of NAT : r(x) ← p(z, x) where x and z are of sort N at. ΣNat ∪ {p : [N at, N at]} is a pre-signature for this program. Consider the interpretation i1 where ΣNat is interpreted as in NAT , r(x) means “x is even”, and p(z, x) means “z + z = x”. If we interpret p as in i1 , we can easily see that the interpretation of r(x) in the corresponding minimum
16
Kung-Kiu Lau and Mario Ornaghi
model of P coincides with the interpretation of r(x) in i1 , i.e., P is steadfast in i1 . Similarly, if we consider i2 that interprets p(z, x) as “z ∗ z = x”, to get steadfastness i2 has to interpret r(x) as “x is a perfect square”. Correctness is steadfastness in the expansions of the intended models stated by the interface specification: Definition 12 (Correctness of Open Programs). Let F be an open framework, and P : Sπ ⇒ Sδ be an open program. P : Sπ ⇒ Sδ is correct in F iff for every intended model i of F and every Sπ -expansion iπ of i, there is a Sδ -expansion iπ,δ of iπ , such that P is steadfast in iπ,δ for the predicate symbols of δ. The intuitive meaning of the previous definition is the following: Σ + π is a pre-signature for P , and iπ is a pre-interpretation that interprets the data signature according to F and the open symbols according to Sπ , i.e., iπ represents a legal parameter passing. Steadfastness of P in iπ,δ means that the interpretation of δ stated by P for the parameter passing iπ is correct with respect to Sδ . The following important properties of correct reusability hold (see [34]): Proposition 2 (Inheritance). Let F be an adequate extension of an adequate (open) framework F . If P : Sπ ⇒ Sδ is correct in F , then it is correct in F . As we will show in Section 3, framework composition can be treated in terms of extension. Therefore inheritance yields a first level of correct reusability, namely reusability of correct programs through framework composition, extension and instantiation. This level of correct reusability would not be important, however, if we could not guarantee the correctness of the composition of the inherited open programs. This second, important level of correct reusability will be called compositionality. In compositionality, interface specifications play a central role, as shown by the following theorems: Theorem 2 (Compositionality). If P : Sπ1 , Sδ2 ⇒ Sδ1 and Q : Sπ2 ⇒ Sδ2 are correct in a framework F and are not mutually recursive, then P ∪ Q : Sπ1 , Sπ2 ⇒ Sδ1 , Sδ2 is correct in F . As we can see, interface specifications indicate how programs can be composed to correctly interact. Theorem 2 can be extended to mutually recursive programs, but in this case we have to check that open termination [34] is preserved. By inheritance and compositionality we get reusability, as shown by the following example. Example 10. We can show that the open program P : Ssplit , Smerge ⇒ Ssort in Example 9 is correct in the open framework LIST . This means that, in every instance of LIST , P : Ssplit , Smerge ⇒ Ssort is always correct with respect to the specification ⇒ Ssort , provided that it is composed with closed correct programs Qmerge : ⇒ Smerge and Qsplit : ⇒ Ssplit . This example shows that compositionality corresponds to a priori correctness of open programs in a framework. It thus corresponds to correctness of open modules in a library. It is to be contrasted with a posteriori correctness, i.e., correctness established by verification after program composition.
Specifying Compositional Units for Correct Program Development
2.4
17
Related Work
Specification frameworks are similar to Abstract Data Types (ADT’s). ADT’s became popular in the 80’s and have been widely studied [47]. In general, they are based on the initial algebra approach, that is, intended models are initial models. Parametric ADT’s have also been studied. These are similar to our open frameworks, even though they are technically defined in a different way. A detailed treatment of algebraic ADT’s, including the parametric case, can be found, for example, in [13]. The initial algebra approach is adequate for ADT specification, where the purpose is to give the minimal signature and axioms that are needed to characterise the desired data and operations. Initial models generalise the idea of minimum Herbrand models, and always exist for algebraic ADT’s and consistent Horn theories [21]. The existence of an initial model allows us to axiomatise only positive knowledge and to use (consistently) negation as failure: a fact is false if we do not have evidence of its truth. This allows for very compact axiomatisations. In contrast, our purpose is “knowledge representation”, that is, we are looking for an expressive signature and a rich set of axioms, to obtain a framework that represents our overall knowledge of a problem domain and allows us to reason about it. Isoinitial semantics requires stronger axiomatisations, and better meets our “richness requirement”, compared to initial semantics. It was introduced in [5], with the purpose of giving a model-theoretic characterisation of computable ADT’s. Finally, our approach is different from the algebraic approach in the threelevel architecture of our compositional units, and in the role that frameworks play in it. In this regard, we are closer to the two-tiered specification style of Larch [20], where specifications have two components: the first one is written in the Larch Shared Language LSL, and the second one in a Larch Interface Language, which is oriented to the programming language and is used to specify the interfaces between program components, i.e., the way they communicate. We consider non-recursive definitions, like explicit or conditional definitions, to be an important tool for both extending frameworks and specifying programs. In this regard, our work is similar to [36]. At the program and specification levels, our approach is in the tradition of logic program synthesis and correctness. Our notion of correctness for closed programs is similar to the one introduced in [22]. Correctness of open programs with respect to specifications similar to our interface specifications is considered in [10]. A conditional specification is like a pre-post-condition style of specification as in VDM [24], Z [45], and B [1], except that it is declarative. Declarative conditional specifications for logic programs were introduced in [8].
3
Operations on Compositional Units
In this section we consider compositional units as building blocks for program development, that is, we focus on operations on compositional units that al-
18
Kung-Kiu Lau and Mario Ornaghi
low their correct reuse in the process of developing an application. This process starts from pre-existing compositional units, and iteratively extends them either directly, by inserting domain specific knowledge, or by reusing, i.e., incorporating, other compositional units. Reuse can, in turn, be factorised into composition and extension, which are the basic operations considered in this section. Such operations involve the framework, the specification and the program levels. We will consider the different levels separately. 3.1
Framework Reuse
To compose two units C1 and C2 , first we compose and/or extend their frameworks F1 and F2 into a common extension F , and then use the specifications and programs in this richer F . Thus, in general, the framework level is the first level to be involved in operations on compositional units. Here we consider the basic operations needed at this level. Framework Morphisms. Operations like renaming or identifying different symbols may be needed for framework reuse. This kind of operation is formalised by framework morphisms. Before introducing framework morphisms, we briefly recall signature and theory morphisms [19,13]. A signature morphism µ : Σ1 → Σ2 is a map from the symbols of Σ1 to those of Σ2 that preserves the declarations. Σ2 extends Σ1 , in the following sense: – Σ2 contains the µ-image of Σ1 ; – every Σ1 -formula F translates into a Σ2 -formula µ(F ); – instead of Σ1 -reducts we have µ-reducts: the µ-reduct of a Σ2 -interpretation m is the Σ1 -interpretation m|µ that interprets every symbol σ of Σ1 as m interprets the image µ(σ). A theory morphism µ : Σ1 , X1 → Σ2 , X2 is a signature morphism µ : Σ1 → Σ2 such that µ(X1 )∗ ⊆ X∗2 , where ∗ denotes the proof-theoretic closure. µ works as a generalised extension, in the sense that Σ2 contains (the µ-image of) Σ1 and the theorems of X2 contain (the µ-translation of) those of X1 . Framework morphisms are defined as follows: Definition 13 (Framework Morphism). Let F (Π ⇒ ∆) = Σ, X and F (Π ⇒ ∆ ) = Σ , X be two frameworks. A framework morphism : F → F is a theory morphism : Σ, X → Σ , X such that the kernel signature of F contains the -image of the kernel signature of F . F can be considered as a generalised extension of F . We say that it is the extension generated by the morphism : F → F . Let F be adequate. We say that F is an adequate extension of F if F is adequate and, for every intended model i of F , the -reduct i | is an intended model of F . Framework extensions considered in the previous section, that simply introduce new symbols and axioms, are a particular case. They correspond to inclusion morphisms that map each symbol σ into σ itself.
Specifying Compositional Units for Correct Program Development
19
Example 11. A (generalised) adequate extension of LIST is: Framework LIST 1 ( ⇒ ListNat, nil, ., @, nocc) extends LIST ; close: X by Nat; rename: ListX by ListNat; It is generated by the morphism defined by the clauses close and rename. maps the sort symbol X into Nat, the sort symbol ListX into ListNat, and leaves the other symbols unchanged. Of course, arities and sorts in relation, function and constant declarations are translated by replacing X and ListX by Nat and ListNat. For example, now we have nil : [ ] → ListNat and . : [Nat, ListNat] → ListNat. In LIST 1 , ListNat is closed (its intended meaning is the set of finite lists of natural numbers) and the only open symbol is : [Nat, Nat]. A closed adequate extension can be obtained by closing by D :
xy ↔x≤y
In this case, we have simply added the new axiom D , that is, we have an inclusion morphism of LIST 1 into a closed framework, that we will indicate by LISTNAT . The morphisms considered in this example are at the basis of the closure operations that we consider next. Closure. A closure is an extension that closes the meaning of some symbols. Here, we consider closure by internalisation, as defined in [30]. As shown in [30], internalisation can be used to implement constrained parameter passing, as well as to introduce objects as the closures of suitable open frameworks that represent classes. Let F (Π ⇒ ∆) = Σ, X be an open framework. An internalisation of an open symbol is one of the following operations: – Sort closure. The closure: close S by s renames the open sort S by a sort s of the signature ΣK of the closed kernel. No axioms are added. – Relation closure. The operation: close r by ∀x • r(x) ↔ R(x) closes r by the new closure axiom ∀x • r(x) ↔ R(x). The declaration of r may contain only sorts of ΣK , and the defining formula R(x) is a ΣK -formula. – Function closure. The operation: close f by ∀x • F (x, f (x)) closes f by the new closure axiom ∀x • F (x, f (x)). The declaration of f may contain only sorts of ΣK , and the defining formula F (x, y) is a ΣK -formula such that XK ∀x • ∃!y • F (x, y). Let F (Π ⇒ ∆) = Σ, X be an open framework. A closure by internalisation is an internalisation that closes all the open sorts by closed sorts and all the open
20
Kung-Kiu Lau and Mario Ornaghi
relation and function symbols by a set DΠ of closure axioms, and satisfies the following constraint satisfaction condition: XK ∪ DΠ XC
(2)
It produces the framework F = Σ , (X\XC )∪DΠ , T∪XC where Σ is obtained by replacing in Σ each open sort s with the sort s closing s, DΠ are the closure axioms of the open functions and relations, and, by (2), the constraints XC have been deleted from the axioms and added to the theorems. Example 12. LISTNAT of Example 11 is a closure of LIST . It has been obtained by a sort closure and a relation closure, by the definition D . Constraints are satisfied because XNat ∪ D TotalOrdering() Now the total ordering axioms TotalOrdering() are no longer constraints, but theorems. A different closure could be obtained, e.g., by closing by the reverse ordering x y ↔ y ≤ x. A closure of a framework F should be an adequate closed extension of F . We can prove: Theorem 3 (Closure). Let F (Π ⇒ ∆) = Σ, X be an adequate framework, and F = Σ , (X \ XC ) ∪ DΠ , T ∪ XC be the result of a closure. Then F is an adequate closed extension of F iff F is consistent and atomically complete. The relation and function closures preserve consistency because DΠ are explicit definitions in the kernel and, by the constraint satisfaction condition, XC become theorems. Thus consistency is preserved if sort closures preserve the consistency of X \ XC . A sufficient condition to preserve consistency is that no cardinality restrictions are imposed on the open sorts, as is commonly the case (like, e.g., the open sort X in generic lists). Concerning atomic completeness, let K be the extension of the closed kernel of F by the closure axioms DΠ . Atomic completeness may be not guaranteed for two reasons: (a) K is not atomically complete because DΠ are not adequate explicit definitions in the kernel, or (b) the atomic completeness of K is not sufficient to obtain the atomic completeness for the defined symbols, because stronger properties are required by the definition axioms. To avoid (a), DΠ must be adequate explicit definitions in the kernel. With quantifier free defining formulas, adequacy is guaranteed by Proposition 1. An example of (b) is the definition axiom r(x) ↔ ∃y • p(x, y), where p is a parameter; in this case, K should prove ∃y • p(x, y) or ¬∃y • p(x, y) for every ground x, i.e., atomic completeness of K does not suffice. However, in general it is reasonable to look for definition axioms that close the defined symbols whenever the open ones become closed, i.e., case (b) should be the exception. Thus, if we do not use quantified defining formulas, closures by internalisation are, in general, adequate.
Specifying Compositional Units for Correct Program Development
21
Closure may also be performed incrementally, step by step. A partial closure is called a specialisation, because it does not close all the open symbols. Besides partial closure, we may have other kinds of specialisation. For example: adding constraints, using open symbols in the defining formulas, mapping open sorts into non-closed sorts, and so on. All these operations can be formalised by extension morphisms, but we will omit relevant details. Example 13. The framework LIST 1 of Example 11 is a specialisation of LIST , obtained by the partial closure of X. Framework Composition. Framework composition essentially coincides with framework union. The simplest case is disjoint union. However, it may happen that we want to preserve a common part, for example natural numbers. Here we consider the composition of two frameworks F1 and F2 that have a common subframework G containing their closed kernel, and have disjoint signatures for the symbols not in G. In this case, composition preserving G can be defined as the operation +G that builds the composite F1 +G F2 simply by making the union of signatures, open and closed symbols, and axioms. If F1 and F2 share symbols not in G, then we rename such symbols, to make them different before performing the union. +G is syntactic composition. Its semantic counterpart is amalgamation. Two intended models i1 of F1 and i2 of F2 are amalgamable if they coincide over the common signature. Their amalgamation is the interpretation i1 + i2 that coincides with i1 over the signature of F1 and with i2 over the signature of F2 (the definition is consistent, because i1 and i2 coincide over the common signature). The intended models of F1 +G F2 are the amalgamations of the pairs of amalgamable intended models of F1 and F2 . This kind of composition has been formalised in ADT’s using pushouts (see e.g., [13]), and the pushout approach also works for frameworks, and it allows us to generalise the operation +G . We do not consider the general case here for conciseness. Example 14. Let BOOL be a framework defining booleans in the usual way. Lists of booleans with open ordering : [Bool, Bool] can be defined starting from the disjoint union LIST + BOOL, as follows: Framework LIST 2 ( ⇒ ListBool, nil, ., @, nocc) extends LIST + BOOL; close: X by Bool; rename: ListX by ListBool; We can compose lists of booleans LIST 2 and lists of natural numbers LIST 1 (see Example 11). To avoid duplicating the kernel of natural numbers, we perform the composition with common subframework NAT : LIST 1 +NAT
LIST 2
22
Kung-Kiu Lau and Mario Ornaghi
To distinguish the non-common symbols, the composition renames them. Since we allow overloading, only sort and constant renaming may be needed. For example, we have nil1 : [ ] → ListNat, ] nil2 : [ ] → ListBool, overloaded . : [Nat, ListNat] → ListNat and . : [Nat, ListBool] → ListBool, and so on. 3.2
Specification Reuse
Specifications are used in two ways. Before composition or extension, they are used as a guide to search for possible compositional units that specify a desired context and set of operations. For example, if we need list sorting, we look for compositional units that contain the framework for lists with totally ordered elements, a specification Ssort of a sort operation, and a program P : . . . ⇒ Ssort , . . .. After composition or extension, the compositional units guide program composition, according to Theorem 2. Reusability after composition or extension is enhanced by specification reduction, as considered in [15]. Indeed, after extension or composition, we have a richer framework, where new properties have been added. It may happen that a specification can be reduced to a new specification, that is, in the new context the new specification can replace the old one. Informally, an “old” specification S reduces to a “new” specification S if correctness with respect to the new S entails correctness with respect to the old S. Formally, we give the following definition: Definition 14 (Specification Reduction). Let F be a framework, and Sω , Sω be two sets of specifications in F . We say that Sω reduces to Sω iff ω ⊆ ω and F Sω → Sω . For two interface specifications Sπ1 ⇒ Sδ1 , Sπ2 ⇒ Sδ2 , we say that Sπ1 ⇒ Sδ1 reduces Sπ2 ⇒ Sδ2 iff Sπ2 reduces to Sπ1 and Sδ1 reduces to Sδ2 . Reduction is transitive and reflexive. Its meaning is made clear by Theorem 4: Theorem 4. Let F be a framework, and Sπ1 ⇒ Sδ1 and Sπ2 ⇒ Sδ2 be two interface specifications. If Sπ1 ⇒ Sδ1 reduces to Sπ2 ⇒ Sδ2 in F , then every program P that is correct with respect to Sπ2 ⇒ Sδ2 is also correct with respect to Sπ1 ⇒ Sδ1 (in F ). Example 15. Let K be a compositional unit with open framework LIST , and let Slhd be the strict specification Slhd : lhd(x, y) ↔ x y. In the extension LISTNAT of LIST , Slhd : lhd(x, y) ↔ x ≤ y reduces to Slhd (we prove Slhd → Slhd by the closure axiom x y ↔ x ≤ y). Thus Slhd ⇒ Smerge (where Smerge is defined in Example 8) reduces to Slhd ⇒ Smerge , and we can use Slhd when deriving correct merge programs. For example, we could write a correct program Pmerge : Slhd ⇒ Smerge which avoids comparisons with 0, since 0 is the minimum natural number; Pmerge would correctly override a (possibly) inherited Pmerge : Slhd ⇒ Smerge .
Specifying Compositional Units for Correct Program Development
23
In the reduction of conditional specifications [15], we can take into account the call context. This is shown in the following example. Example 16. In the open framework LIST we can give the following specifications: : l = x.nil ∧ ord(m) → (merge(l, m, o) ↔ ord(o) ∧ perm(x.m, o)); Smerge : split(x.l, m, n) ↔ m = x.nil ∧ n = l. Ssplit (Ssplit → Ssplit can be proved in LIST ). Ssplit of Example 8 reduces to Ssplit Smerge of Example 8 reduces to Smerge in a call context where the input con dition l = x.nil ∧ ord(m) of Smerge holds for the lists l and m to be merged. Indeed, in such a context, Smerge corresponds to merge(x.nil, m, o) ↔ ord(o) ∧ perm(x.m, o), Smerge to merge(x.nil, m, o) ↔ ord(o) ∧ perm(x.nil|m, o), and they are equivalent. We will say that Smerge contextually reduces to Smerge . Contextual reduction implies contextual reuse, that is, Smerge correctly re duces to Smerge only when the input condition of Smerge is true. As a conse quence, we cannot replace Smerge by Smerge in isolation, but we have to consider in isolation, because the call context. In contrast, we can replace Ssplit by Ssplit the corresponding reduction is not contextual. As we will see in Example 17, Smerge and Ssplit are tailored to the insertion sort algorithm. In a similar way, we can specialise Smerge and Ssplit to obtain specifications tailored to different sorting algorithms, like merge sort, quick sort, and so on.
In general, it is useful to list proven reduction theorems in the specification section of a compositional unit. Such a list would allow us to automatically search for families of program compositions, giving rise to families of implementations. It is for this reason that we have put RD1 , ...RDk in Fig. 1. 3.3
Program Reuse
Like specifications, programs in compositional units can be used before and after unit composition. We use programs before composition when we look for existing compositional units containing specific algorithms. Otherwise, reuse is after unit composition, when we use the inherited programs to solve the problem in question. The operation that allows us to reuse the inherited programs is program composition. It is strongly guided by specifications. Specification reduction is important for program reusability, since it allows us to use the richer knowledge obtained after framework composition and extension to solve the puzzle of composing the inherited open programs into a correct solution of the problem at hand. Example 17. Let K be a compositional unit with framework LIST , and let us assume that it already contains the correct program Pinsert : Slhd ⇒ Sinsert , where Pinsert implements the usual algorithm for inserting an element into its
24
Kung-Kiu Lau and Mario Ornaghi
correct position in an ordered list, Slhd is the specification shown in Example 15, and Sinsert is: Sinsert :
ord(l) → (insert(x, l, m) ↔ (ord(m) ∧ perm(x.l, m)).
We show how reductions of Example 16 can be used to solve the puzzle of obtaining a correct sorting program Qsort : Slhd ⇒ Ssort . If we compose Pinsert with the correct one-clause programs Psplit : ⇒ Ssplit Plink : Sinsert ⇒ Smerge
split(x.l, x.nil, l) ← merge(x.nil, l, o) ← insert(x, l, o)
, Smerge . By the specification rewe get a correct program Qaux : Slhd ⇒ Ssplit ductions of Example 16, the interface specification Ssplit , Smerge ⇒ Ssort contextually reduces to Ssplit , Smerge ⇒ Ssort . Thus, the program Psort : Ssplit , Smerge ⇒ Ssort of Example 9 is also correct with respect to Ssplit , Smerge ⇒ Ssort , be cause the input condition of Smerge is satisfied in the call context of merge, as required. By composing Psort and Qaux , we get a correct Qsort : Slhd ⇒ Ssort . Qsort can be closed in the instances that close lhd. For example, Slhd reduces in a compositional unit with framework LISTNAT , as shown in Example to Slhd 15. Suppose that our compositional unit already contains a correct program Pleq : ⇒ Sleq . If we compose it with the correct one-clause program Plhd : Sleq ⇒ Slhd
lhd(x, y) ← leq(x, y)
we get a closed correct program Qlhd :⇒ Slhd . By specification reduction we get that Qlhd : ⇒ Slhd is also correct. Then the closed program Qlhd ∪Qsort : ⇒ Ssort is correct in LISTNAT . 3.4
Related Work
At the framework level, our approach to modularity and reuse is in the tradition of algebraic ADT’s [2,13,47]. We can apply the techniques developed there, based on theory morphisms. Our specification frameworks should not be confused with the specification frames introduced in [25]. The latter, like institutions [19], are general frames for the composition and reuse of formal theories. With respect to modularity and compositionality, our frameworks with open symbols and defined symbols are similar, for example, to modules with import and export interfaces, as introduced in [14]. In [25], a distinction between parameterised specifications and parameterised data types is introduced, following [42]. In [42], programs and specifications are considered as different entities, involved in different phases and different methodological aspects of program development, and a distinction between parameterised specifications and specifications of parameterised programs is introduced. In this, [42] is very close to our general view, but our approach is different. Our three-level architecture of compositional units is closer to Larch [20]. Like Larch, our specifications state precisely how open programs interact,
Specifying Compositional Units for Correct Program Development
25
and allow us to compose them correctly. However, unlike Larch, we have a further specification level, which is intermediate between the framework level and the interface specification level. This yields a further level of correct reuse, through the specification reduction theorems. With regard to modularity in logic programming, there are approaches based on ideas similar to our j-models (see [7]), while the approach proposed in [39] relates to specification frames [25]. However, all these approaches do not distinguish between specifications and programs. A distinction between programs and specifications is made in [40], where modular Prolog programs (as proposed in [44]) are derived from first-order specifications (based on Extended ML [43]). However, in [40], the role of specifications is different from ours, and there is no counterpart of specification frameworks. Finally, in the area of object-oriented analysis and design, component-based development methods [12,3] have emerged, where components and reuse are two of the main aspects of the software development process. In this area, a software component is a unit of composition with contractually specified interfaces and context dependencies only [46]. Our compositional units broadly fit this characterisation, considering interface specifications Sπ ⇒ Sδ as interfaces, and specifications and their reducibility relation in the context of the framework as context dependencies.
4
Conclusion
In this paper we have essentially collected our previous work on program specification and synthesis, and we have organised it by introducing compositional units, which are a more complete and refined version of correct schemas [16]. Then we have illustrated the basic operations for extending and correctly reusing (composing) compositional units. A compositional unit is a unit of reuse that contains both a formalisation of the problem domain, at the framework level, and a collection of open programs, correct with respect to their specifications, at the specification and program levels. The framework level specifies, by the constraint axioms, when and how a compositional unit can be correctly reused. The specification and program levels support program reuse and development. The examples of Section 3 have been mainly devoted to illustrating the role of specifications in the correct reuse of compositional units for program development. In particular, specifications are a guide for program composition, and specification reduction allows us to deal with the problem of adapting the inherited open programs to the specific context of reuse. In this paper, we have not considered program synthesis, because we concentrated on specifications and their role in the reuse of compositional units and correct open programs. However, there is a strong relationship to logic program synthesis [11], and indeed our research started in this area. An interesting fact is the possibility of using logic program synthesis as a way for expanding frameworks in an adequate way [27].
26
Kung-Kiu Lau and Mario Ornaghi
The distinct levels for specifications and programs distinguish our approach. At these levels, we have integrated our research on steadfast open programs and specifications. As we have shown, specifications and steadfast programs yield a further level of reuse, through specification reduction and program composition. We believe that this is an important feature of our approach, especially in the context of so-called software components [46]. Our future work will be devoted to the study of the applicability of our approach to the development of correct component-based software. On the one hand, we want to develop the approach further, based on logic programs, along the following two lines: (a) We will extend our approach to other kinds of logic programs. For constraint logic programs and those normal programs that have one intended model, the extension of our results is almost immediate. (b) We will study methods for deriving steadfast programs from their interface specifications, based on our compositional units and on the results of [34] and the ideas exposed in [16]. To this end, tools would be necessary for developing an interactive environment where we can define and compose specification frameworks, specifications and programs, and use a proof assistant for developing the necessary proofs. We are looking at logical frameworks like Isabelle [23] as possible candidates. On the other hand, we want to consider the extension of our approach to different programming paradigms. This can be done in two ways. The first choice is to define, on top of specification frameworks, different specification formalisms, oriented to different program languages. Such formalisms would provide different interface specification languages, in a way similar to Larch [20]. The second choice is to use our compositional units as meta-level declarative specifications of systems implemented in possibly imperative programming languages. So far, we have considered only the second choice. We began a study of object-oriented systems, with the aim of testing the versatility of our model and, hopefully, of obtaining a formalisation of object-oriented compositional units that could be used as software components. In [32], we introduced a static model of object-oriented systems, suitable for formalising states and queries. Our static approach shares similarities with [6,36], and allows us to formalise UML class and object diagrams [41], queries and OCL constraints [9]. The introduction of time in our object-oriented systems is work in progress. Our final goal is to obtain a methodology for the specification and the development of correct component-based software, where programs are developed together with the formal proof of their correctness. This methodology should allow the development of correct compositional units to be used as software components, that is, units of composition that can be deployed independently and are subject to composition by third parties [46]. Acknowledgements We are very grateful to the referees for their valuable suggestions and comments. This paper has been radically improved as a result of their efforts.
Specifying Compositional Units for Correct Program Development
27
References 1. J.R. Abrial. The B-Book: Assigning Programs to Meanings. Cambridge University Press, 1996. 2. E. Astesiano, H.-J. Kreowski, and B. Krieg-Br¨ uckner, editors. Algebraic Foundations of Systems Specifications. Springer, 1999. 3. C. Atkinson et al. Component-based Product Line Engineering with UML. AddisonWesley, 2001. 4. J. Barwise, editor. Handbook of Mathematical Logic. North Holland, 1977. 5. A. Bertoni, G. Mauri, and P. Miglioli. On the power of model theory in specifying abstract data types and in capturing their recursiveness. Fundamenta Informaticae, VI(2):127–170, 1983. 6. R.H. Bourdeau and B. H.C. Cheng. A formal semantics for object model diagrams. IEEE Trans. Soft. Eng., 21(10):799–821, 1995. 7. M. Bugliesi, E. Lamma, and P. Mello. Modularity in logic programming. J. Logic Programming, 19,20:443–502, 1994. Special issue: Ten years of logic programming. 8. K.L. Clark. Predicate Logic as a Computational Formalism. Report 79/59, Imperial College of Science and Technology, University of London, 1979. 9. S. Cook, A. Kleppe, R. Mitchell, B. Rumpe, J. Warmer, and A. Wills. The Amsterdam manifesto on OCL. In T. Clark and J. Warmer, editors, Object Modeling with the OCL: The Rationale behind the Object Constraint Language, LNCS 2263, pages 115–149. Springer, 2002. 10. Y. Deville. Logic Programming. Systematic Program Development. AddisonWesley, 1990. 11. Y. Deville and K.-K. Lau. Logic program synthesis. J. Logic Programming, 19,20:321–350, 1994. Special Issue: Ten Years of Logic Programming. 12. D.F. D’Souza and A.C. Wills. Objects, Components, and Frameworks with UML: The Catalysis Approach. Addison-Wesley, 1999. 13. H. Ehrig and B. Mahr. Fundamentals of Algebraic Specification 1. Springer-Verlag, 1987. 14. H. Ehrig and B. Mahr. Fundamentals of Algebraic Specification 2. Springer-Verlag, 1989. 15. P. Flener, K.-K. Lau, and M. Ornaghi. On correct program schemas. In N. Fuchs, editor, Proc. LOPSTR 97, Lecture Notes in Computer Science 1463, pages 124– 143. Springer-Verlag, 1998. 16. P. Flener, K.-K. Lau, M. Ornaghi, and J. Richardson. An abstract formalisation of correct schemas for program synthesis. Journal of Symbolic Computation, 30(1):93– 127, July 2000. 17. J.H. Gallier. Logic for Computer Science: Foundations for Automatic Theorem Proving. Harper and Row, 1986. 18. C. Ghezzi, M. Jazayeri, and D.Mandrioli. Fundamentals of Software Engineering. Prentice Hall, second edition, 2003. 19. J.A. Goguen and R.M. Burstall. Institutions: Abstract model theory for specification and programming. J. ACM, 39(1):95–146, 1992. 20. J.V. Guttag and J.J. Horning. Larch: Languages and Tools for Formal Specification. Springer-Verlag, 1993. 21. W. Hodges. Logical Features of Horn Clauses. In D.M. Gabbay, C.J. Hogger, and J.A. Robinson, editors, Handbook of Logic in Artificial Intelligence and Logic Programming, Volume 1 :449–503, Oxford University Press, 1993. 22. C.J. Hogger. Derivation of Logic Programs. J. ACM, 28(2):372–392, 1981.
28
Kung-Kiu Lau and Mario Ornaghi
23. Isabelle: www.cl.cam.ac.uk/Research/HVG/Isabelle 24. C.B. Jones. Systematic Software Development Using VDM. Prentice Hall, second edition, 1990. 25. R.M. Jimenez, F. Orejas, and H. Ehrig. Compositionality and Compatibility of Parametrization and Parameter Passing in Specification Languages. Math. Struct. in Computer Science, 5:283–314, 1995. 26. C. Kreitz, K.-K. Lau, and M. Ornaghi. Formal reasoning about modules, reuse and their correctness. In D.M. Gabbay and H.J. Ohlbach, editors, Proc. Int. Conf. on Formal and Applied Practical Reasoning, Lecture Notes in Artificial Intelligence 1085, pages 384–399. Springer-Verlag, 1996. 27. K.-K. Lau and M. Ornaghi. On specification frameworks and deductive synthesis of logic programs. In L. Fribourg and F. Turini, editors, Proc. LOPSTR 94 and META 94, Lecture Notes in Computer Science 883, pages 104–121. Springer-Verlag, 1994. 28. K.-K. Lau and M. Ornaghi. Forms of logic specifications: A preliminary study. In J. Gallagher, editor, Proc. LOPSTR 96, Lecture Notes in Computer Science 1207, pages 295–312. Springer-Verlag, 1997. 29. K.-K. Lau and M. Ornaghi. The relationship between logic programs and specifications — the subset example revisited. J. Logic Programming, 30(3):239–257, March 1997. 30. K.-K. Lau and M. Ornaghi. OOD frameworks in component-based software development in computational logic. In P. Flener, editor, Proc. LOPSTR 98, Lecture Notes in Computer Science 1559, pages 101–123. Springer-Verlag, 1999. 31. K.-K. Lau and M. Ornaghi. Isoinitial semantics for logic programs. In J.W. Lloyd et al , editor, Proc. 1st Int. Conf. on Computational Logic, Lecture Notes in Artificial Intelligence 1861, pages 223–238. Springer-Verlag, 2000. 32. K.-K. Lau and M. Ornaghi. Correct object-oriented systems in computational logic. In A. Pettorossi, editor, Proc. LOPSTR 01, Lecture Notes in Computer Science 2372, pages 168–190. Springer-Verlag, 2002. 33. K.-K. Lau and M. Ornaghi. Specifying object-oriented systems in computational logic. In M. Bruynooghe, editor, Pre-Proceedings of LOPSTR 03, pages 49–64, 2003. Report CW 365, Dept. of Computer Science, Katholieke Universiteit Leuven, Belgium. 34. K.-K. Lau, M. Ornaghi, and S.-˚ A. T¨ arnlund. Steadfast logic programs. J. Logic Programming, 38(3):259–294, March 1999. 35. J.W. Lloyd. Foundations of Logic Programming. 2nd edn., Springer-Verlag, 1987. 36. T. Maibaum. Conservative extensions, interpretations between theories and all that. In M. Bidoit and M. Dauchet, editors, Proc. TAPSOFT ’97: Theory and Practice of Software Developement, pages 40–67. Springer-Verlag, 1997. LNCS 1214. 37. Yu.V. Matijacevic. Recursively enumerable sets are Diophantine. Dokl. Akad. Nauk SSSR, 191:279–282, 1970. 38. P. Miglioli, U.Moscato, and M. Ornaghi. Abstract parametric classes and abstract data types defined by classical and constructive logical methods. J. Symbolic Computation, 18:41–81, 1994. 39. P. Miglioli, U.Moscato, and M. Ornaghi. An algebraic framework for the definition of compositional semantics of normal logic programs. The Journal of Logic Programming, 40:89–123, 1999. 40. M.G. Read and E.A. Kazmierczak. Formal program development in modular Prolog: A case study. In T.P. Clement and K.-K. Lau, editors, Proc. LOPSTR 91, pages 69–93. Springer-Verlag, 1992.
Specifying Compositional Units for Correct Program Development
29
41. J. Rumbaugh, I. Jacobson, and G. Booch. The Unified Modeling Language Reference Manual. Addison-Wesley, 1999. 42. D. Sannella, S. Sokolowski, and A. Tarlecki. Toward formal development of programs from algebraic specifications: parametrisation revisited. Acta Informatica, 29(8):689–736, 1992. 43. D. Sannella and A. Tarlecki. Extended ML: past, present and future. In Proc. 7th workshop on specification of abstract data types, LNCS 534, pages 297–322. Springer-Verlag, 1991. 44. D. Sannella and L.A. Wallen. A calculus for the construction of mdular prolog programs. In IEEE 4th Symposium on Logic Programming, IEEE, 1987. 45. J.M. Spivey. The Z Notation: A Reference Manual. Prentice Hall, second edition, 1992. 46. C. Szyperski, D. Gruntz, and S. Murer. Component Software: Beyond ObjectOriented Programming. Addison-Wesley, second edition, 2002. 47. M. Wirsing. Algebraic specification. In J. Van Leeuwen, editor, Handbook of Theoretical Computer Science, pages 675–788. Elsevier, 1990.
Synthesis of Programs in Computational Logic David Basin1 , Yves Deville2 , Pierre Flener3 , Andreas Hamfelt4 , and Jørgen Fischer Nilsson5 1
Department of Computer Science ETH Zurich Z¨ urich Switzerland
[email protected] 2 Department of Computing Science and Engineering Universit´e catholique de Louvain, Pl. Ste Barbe 2, B-1348 Louvain-la-Neuve, Belgium
[email protected] 3 Computing Science Division, Department of Information Technology Uppsala University, Box 337, S-751 05 Uppsala, Sweden
[email protected] 4 Computer Science Division, Department of Information Science Uppsala University, Box 513, S-751 20 Uppsala, Sweden
[email protected] 5 Informatics and Mathematical Modelling Technical University of Denmark DK-2800 Lyngby, Denmark
[email protected]
Abstract. Since the early days of programming and automated reasoning, researchers have developed methods for systematically constructing programs from their specifications. Especially the last decade has seen a flurry of activities including the advent of specialized conferences, such as LOPSTR, covering the synthesis of programs in computational logic. In this paper we analyze and compare three state-of-the-art methods for synthesizing recursive programs in computational logic. The three approaches are constructive/deductive synthesis, schema-guided synthesis, and inductive synthesis. Our comparison is carried out in a systematic way where, for each approach, we describe the key ideas and synthesize a common running example. In doing so, we explore the synergies between the approaches, which we believe are necessary in order to achieve progress over the next decade in this field.
1
Introduction
Program synthesis is concerned with the following question: Given a not necessarily executable specification, how can an executable program satisfying the specification be developed? The notions of “specification” and “executable” are M. Bruynooghe and K.-K. Lau (Eds.): Program Development in CL, LNCS 3049, pp. 30–65, 2004. c Springer-Verlag Berlin Heidelberg 2004
Synthesis of Programs in Computational Logic
31
here interpreted broadly. The objective of program synthesis is to develop methods and tools to mechanize or automate (part of) this process. In the last 30 years, program synthesis has been an active research area; see e.g. [14,4,40,13,26,29] for a description of major achievements. The starting point of program synthesis is usually a formal specification, that is an expression in some formal language (a language having a syntax, a semantics, and usually a proof theory). Program synthesis thus has many relationships with formal specification [69]. As the end product is a verified correct program, program synthesis is also related to formal methods in the development of computer systems [22], and to automated software engineering. All of these disciplines share the goal of improving the quality of software. Program Synthesis in Computational Logic. It is generally recognized that a good starting point for program synthesis is to use declarative formalisms such as functional programming or computational logic, where one specifies what a program should do instead of how. We focus here on the synthesis of recursive programs in computational logic, which provides an expressive and uniform framework for program synthesis. On the one hand, the specification, the resulting program, and their relationship, can all be expressed in the same logic. On the other hand, logic specifications can describe complete specifications as well as incomplete ones, such as examples or properties of the relation that is to be computed. Since all this information can be expressed in the same language, it can be treated uniformly in a synthesis process. There exist many different approaches to program synthesis in computational logic and different ways of viewing and categorizing them. For example, one can distinguish constructive from deductive synthesis. In constructive synthesis, a conjecture based on the specification is constructively proved, and from this proof a program is extracted. In the deductive approach, a program is deduced directly from the specification by suitably transforming it. As will be shown in this paper, these two approaches can profitably be viewed together and expressed in a uniform framework. In a different approach, called schema-based synthesis, the idea is to use program schemas, that is some abstraction of a class of actual programs, to guide and enhance the synthesis process. Another approach is inductive synthesis, where a program is induced from an incomplete specification. Objectives. Our intent in this paper is to analyze and compare three state-ofthe-art methods for synthesizing recursive programs in computational logic. The chosen approaches are constructive/deductive synthesis, schema-guided synthesis, and inductive synthesis. We perform our comparison in a systematic way: we first identify common, generic features of all approaches and afterwards we use a common example to explain these features for each approach. This analysis forms the basis for an in-depth comparison. We show, for example, that from an appropriately abstract viewpoint, there are a number of synergies between the approaches that can be exploited. For example, by identifying rules with schemas, all three methods have a common, underlying synthesis mechanism
32
David Basin et al.
and it becomes easier to see how the methods can be fruitfully combined, or differentiated. Overall, we hope that our comparison will deepen the communities understanding of the approaches — their relationships, synergies, where they excel, and why — and thereby contribute to achieving progress in this field. We see this paper as complementary to surveys of program synthesis in computational logic (or more precisely in logic programming), in particular [26,29]. Rather than a making a broad survey, we focus on the analysis and in-depth comparison of the different approaches and we also consider schema-guided synthesis. Due to lack of space and to comply with our objectives, some technical details are omitted. Here, the reader may rely on his or her intuitive understanding of relevant concepts or follow pointers to references in the literature. Organization. Section 2 presents the different elements that will be used to present and compare the chosen synthesis approaches. These elements include general features of program synthesis approaches as well as the example that will be used for their comparison. Sections 3 through 5 describe the three chosen approaches: constructive/deductive synthesis, schema-guided synthesis, and inductive synthesis. To facilitate a systematic analysis and comparison of the methods, each section has a similar structure. Section 6 compares the three approaches. Finally, Section 7 draws conclusions and presents perspectives for future developments.
2
Elements of Comparison
In the subsequent sections, we will present three synthesis approaches. For each approach, one representative method is described. However, before describing them, we first present their general features. These features are developed in the context of each particular method and serve both to facilitate our analysis and systematize our comparison. We also introduce our example. 2.1
General Features
Specification. The starting point for program synthesis is a specification expressed in some language. For each synthesis method, we must fix the specification language and the form of the specification (e.g., a formula or a set of examples). Mechanism. Program synthesis methods are based on calculi and procedures prescribing how program are synthesized from specifications. Although the underlying mechanisms of the various systems differ, there are, in some cases, similar underlying concepts. Heuristics. Program synthesis is search intensive and heuristics are required in practice to guide the synthesis process. Are the heuristics specific to a synthesis method or are there common heuristics? How effective are the heuristics in the different methods and to what extent do different methods structure and restrict the search space?
Synthesis of Programs in Computational Logic
33
Background Knowledge. Usually, non-trivial specifications refer to background knowledge that formalizes information about the properties of objects used in the specification, e.g., theories about the relevant data types. Human Interaction. Human interaction involves two different issues. First, how much can a human be automatically assisted? Second, what is the nature of human-computer interaction in synthesis? How can the human step in and, for example, give key steps rather than leave the matter to blind search? Allowing input at critical points requires appropriate system support. Tool Support. What kind of tool support is needed for turning a synthesis method into a viable system? Scalability. Scalability is a major concern in program synthesis. Synthesis systems should not only be able to synthesize small simple programs, but they should also be able to tackle large or complex programs that solve real-life problems. 2.2
The Chosen Example
The same example will be used throughout the paper to facilitate a comparison of the different methods. We have chosen a problem simple enough to present in full, but complex enough to illustrate the main issues associated with each approach. Specification 21 Let L be a list, I a natural number, and E a term. The relation atpos(L, I, E) holds iff E is the element of L at position I. By convention, the first element of a list is at position 0. The atpos relation can be formally specified as follows: atpos(L, I, E) ↔ ∃P, S . append (P, E · S, L) ∧ length(P, I) where append and length have their usual meaning, and are assumed to be defined in the background theory. In the formula above, and in the rest of the paper, free variables are assumed to be universally quantified over the entire formula. As list notation, we use nil to represent the empty list, and H · T for the list with head H and tail T .
3
Constructive and Deductive Synthesis
We will now look at two approaches to synthesizing programs that are often grouped together: constructive and deductive synthesis. We shall highlight their similarities by viewing both from the same perspective: In both cases, deduction can be used to synthesize programs by solving for unknowns during the application of rules.
34
3.1
David Basin et al.
Background
For historical reasons, and because the ideas are simplest to present there, we begin by considering synthesis of functional programs in constructive type theory. Constructive type theories are logics used for reasoning about functional programs. The simplest example is the simply typed λ-calculus [5,48], which we briefly review here. Programs in the simply typed λ-calculus are terms in the λ-calculus, which are built from variables, application, and abstraction. Types are built from a set of base types, closed under the function space constructor →. One reasons about judgments that assert that a term t has a type T , relative to a sequence of bindings Γ , of the form x1 : A1 , . . . , xn : An , which associate variables to types. The valid judgments are inductively defined by the following rules: Γ, x : A M : B x:A∈Γ abst hyp Γ (λx. M ) : (A → B) Γ x:A Γ M :A→B
Γ N :A
Γ (M N ) : B
appl
These rules comprise a deduction system for proving that a program t has a type T . Under the propositions-as-types interpretation, this type may also be understood as a logical proposition (reading ‘→’ as intuitionistic implication) that specifies t’s properties. Of course, the specification language is quite weak, so it is difficult to specify many interesting properties. In stronger type theories, such as [24,56], types correspond to propositions in richer logics and one can, for example, specify sorting as t : (∀x : int list . ∃y : int list . perm(x, y) ∧ ord (y)) .
(1)
This asserts that the program t is a function that, on input x, returns an ordered permutation y. The given deduction system can be used for program verification: given a program t and a specification T , prove t : T . For example, for p and q types, we can verify that the program λx. λy. x satisfies the specification p → (q → p): x : p ∈ x : p, y : q x : p, y : q x : p
hyp
x : p λy. x : q → p
abst
λx. λy. x : p → (q → p)
(2)
abst
Perhaps less obviously, the same rules can be used for program synthesis: given a specification T , construct a program t such that t : T . This can be done by 1. Reversing the direction in which rules are applied and proofs are constructed. That is, build the proof in a goal-directed, “refinement style” way by starting with the goal and working towards the axioms.
Synthesis of Programs in Computational Logic
35
2. Leaving the program t as an unknown, or metavariable, which is solved during proof. Let’s try this out in the example above. Using capital letters to indicate metavariables, we begin with R : p → (q → p) . Resolving this with the (conclusion of the) abst rule yields the new goal x : p R1 (x) : (q → p) , where R is unified with λx. R1 (x). Applying abst again results in x : p, y : q R2 (x, y) : p , where R1 (x) = λy. R2 (x, y). Finally, applying hyp unifies the assumption x : p with R2 (x, y) : p, instantiating R2 (x, y) to x and completing the proof. Composing the substitutions yields the previously verified program t = λx. λy. x. The account above is complicated by the fact that the abstraction operator λ binds variables and, to work properly, higher-order unification is required when applying rules. The rules constitute clauses in a higher-order (meta-)language and proofs are constructed by higher-order resolution. A higher-order logic programming language or logical framework based on higher-order resolution like λ-Prolog [27], ELF [61], or Isabelle [59] would support this kind of proof. There are two conclusions we would like to draw. First, verification and synthesis are closely related activities. In fact, when rules are applied using (higher-order) resolution, they are essentially identical. The only difference is whether unification is between ground or non-ground terms, i.e., whether or not an answer substitution is built. This conclusion should not be surprising to those working in logic programming: the same sequence of resolution steps can be used to establish a ground query p(t) or a non-ground one p(X), generating the substitution X = t. Second, constructive synthesis is of a deductive nature and the line between the two can be fine. As the analogy with Prolog shows, proofs construct objects. In type theory, the objects are programs. Indeed, the idea of proofs synthesizing programs, sometimes called proofs-as-programs, can be decomposed into proofs-as-programs = proofs-as-objects + objects-as-programs. In our example, unification, not the constructivity of the logic, is responsible for constructing an object. Constructivity does not play a role in the synthesis of objects, but rather in their execution and meaning. That is, because the logic is constructive, the synthesized terms can be executed and their evaluation behavior agrees with the semantics of the type theory. In contrast, [49], for example, presents a classical type theory where programs correspond to (noncomputable) oracles that cannot be executed. There one might say that the line is crossed from constructive (and deductive) program synthesis to deductive object synthesis.
36
David Basin et al.
The use of unification is at the heart of deductive and constructive synthesis. Unification is driven by resolution, to synthesize, or solve for, programs during proofs. This idea goes back to work in the 1960s on using first-order resolution to construct terms that represent plans or, more generally, programs [19,42]. In the logical framework community, the use of higher-order metalogics to represent rules and the use of higher-order unification to apply them is now standard, e.g., [2,8,9,23]. For example, the Isabelle distribution [59] comes with encodings of a number of type theories, where programs can be synthesized as described here. The vast majority of approaches for synthesizing logic programs are based on first-order reasoning, e.g., equivalence preserving transformations. There have been many proposed methods and [26] contains a good survey. They differ in the form of their axioms (Horn clauses, iff -definitions, etc.), exact notion of equivalence used (and there are many, see e.g., [55]), and ease of automation. Many of these, for example unfold-fold based transformations [60], can be recast as synthesis by resolution using rules like those presented here [7,10]. 3.2
Overview
Specifications. In type theory, programs and specifications belong to different languages. When synthesizing logic programs, the specification language is typically the language of a first-order theory and the programming language is some suitable, executable subset thereof. By sharing the same language, logic programs are well suited for deductive synthesis where specifications are manipulated, using equivalence preserving transformations, until a formula with some desired form or property is reached. Mechanism. The mechanism for synthesizing logic programs during proofs is essentially the same as what we have just seen for type theory. However, what is proved (i.e., the form of the theorem to be proven), and the proof rules used to establish it, are of course different. Namely, we will prove theorems about equivalences between specifications and programs and we will prove these theorems using rules suitable for establishing such equivalences. For our example, we will employ the following rules: A↔A
↔−refl
A1 ↔ B1
A2 ↔ B2
(A1 ∨ A2 ) ↔ (B1 ∨ B2 )
∨−split
In addition, for building recursive programs that recurse over lists we employ the rule schema A1 A2 A3 ind , ∀L, X . P (L, X) ↔ Q(L, X) where L is a variable ranging over lists, X denotes sequences of zero or more variables of any type, and the assumptions Ai are: A1 ≡ ∀L, X . Q(L, X) ↔ (L = nil ∧ B(X)) ∨∃H, T . L = H · T ∧ S(H, T, X)
Synthesis of Programs in Computational Logic
37
A2 ≡ ∀X . P (nil, X) ↔ B(X) A3 ≡ ∀T . (∀X . P (T, X) ↔ Q(T, X)) → ∀H, X . P (H · T, X) ↔ S(H, T, X) This rule, which can be derived by induction on the list L, states the equivalence between predicates P and Q (which are metavariables). For the purpose of synthesis, we can take A1 as the definition of Q, and A2 and A3 constrain (and will be used to define) Q’s base and recursive cases. In A3 , we are allowed to use the existence of Q, when defining Q, but only on smaller arguments. We will show below how, by applying these rules (using higher-order resolution), we can construct R while proving its equivalence to atpos. Heuristics and Human Interaction. Proof rules, like those given above, can be applied interactively, semi-interactively, or even automatically. The use of a tactic based theorem prover [41], which allows users to write programs that construct proofs, leaves open the degree of automation. [50,51], for example, show how to completely automate the construction of such synthesis proofs in a tactic based setting. In this work, the most important tactic implements the rippling heuristic of [17,12]. This heuristic automates the application of rewrite or equivalence preserving transformation rules in a way that minimizes differences between terms or formulas. Rippling is typically used in inductive theorem proving to enable the use of the induction hypothesis in simplifying the induction conclusion and it can be used in a similar way during program synthesis where rules that introduce recursion (like ind ) produce induction-like proof obligations. Rippling has been used to automate completely the synthesis of a number of non-trivial logic programs. However, it should be noted that some interaction with the user is often desirable since the application of proof rules, in particular rules that build recursive programs, determines the efficiency of the synthesized program. Background Knowledge. The approach we present here for synthesizing logic programs involves two kinds of rules. The first kind are rules, like ↔−refl and ∨−split , which are derived rules of first-order logic. These derived rules are not, strictly speaking, necessary (provided we are working in a complete axiomatization of first-order logic), but their addition makes it easier to construct synthesis proofs by reasoning about equivalences. The second kind of rules are theory specific rules, e.g., rules about inductively defined data types like numbers and lists. The rule ind given above is an example of such a rule. It is derivable in a theory that axiomatizes lists and formalizes induction over lists. Tool Support. For synthesizing the atpos example, we have used the Isabelle system. Isabelle’s basic mechanism for proof construction is top-down proof by higher-order resolution, which is precisely what we require. Moreover, as a logical framework, Isabelle supports the derivation of new rules, so we can formally derive, and thus insure the correctness of, the specialized rules needed for synthesis; in our example, we derive the rules just presented in a standard first-order
38
David Basin et al.
theory of lists. Finally, tactics can be used to partially, or entirely, automate proof construction. The Isabelle distribution comes with simplifiers and decision procedures that we used to semi-automate synthesis. Scalability. The search space in most approaches to deductive synthesis is quite large. In practice, building non-trivial programs requires an environment that supports heuristics for automating simple proof steps, e.g., by the application of tactics. It is also important that the user can safely augment a synthesis system with derived rules. As we will later observe, schemas, for schema guided synthesis, can be seen as derived rules specialized for synthesizing programs of a particular form, and their integration with deductive synthesis approaches can help with large scale developments. Examples of this are provided in [1]. 3.3
Example
Let us illustrate our synthesis method on the atpos example. We wish to construct a logic program equivalent to the specification 21. As with synthesis in the type theory, we use a metavariable, R, to stand in for the desired program. Hence we start with ∀L, I, E . atpos(L, I, E) ↔ R(L, I, E) .
(3)
Working backwards, resolving (using higher-order unification) this conclusion with the conclusion of the ind rule yields the three subgoals ∀L, I, E . R(L, I, E) ↔ (L = nil ∧ B(I, E)) ∨∃H, T . L = H · T ∧ S(H, T, I, E) ∀I, E . atpos(nil, I, E) ↔ B(I, E) ∀T . (∀I, E . atpos(T, I, E) ↔ R(T, I, E) → ∀H, I, E . atpos(H · T, I, E) ↔ S(H, T, I, E) and Q is unified with R. The first subgoal constitutes a program template, which will later be filled out by solving the other subgoals. In the second subgoal, expanding the definition of atpos results in ∀I, E . (∃P , S . append (P , E · S , nil) ∧ length(P , I)) ↔ B(I, E) . Let I and E be arbitrary. To show (∃P , S . append (P , E · S , nil) ∧ length(P , I)) ↔ B(I, E) , observe that there are no values for P or S for which append (P , E · S , nil) is true. Hence this subgoal is equivalent to false ↔ B(I, E) . We can complete the proof with ↔−refl, which unifies B(I, E) with false.
Synthesis of Programs in Computational Logic
39
For the third subgoal, we assume the existence of an arbitrary list T and the antecedent of the implication (which amounts to an induction hypothesis) and must prove the consequent (the induction conclusion). Hence, expanding the definition of atpos, we assume ∀I, E . (∃P , S . append (P , E · S , T ) ∧ length(P , I)) ↔ R(T, I, E) and we must prove, for some arbitrary H, I, and E, (∃P , S . append (P , E · S , H · T ) ∧ length(P , I)) ↔ S(H, T, I, E) . Now, since P ranges over lists, for any formula φ(l), ∃P .φ(P ) is equivalent (by case analysis) to φ(nil) ∨ ∃H, T . φ(H · T ). Hence, the above is equivalent to ((∃S . append (nil, E · S , H · T ) ∧ length(nil, I)) ∨ (∃H , T , S .append (H · T , E · S , H · T ) ∧ length(H · T , I))) ↔ S(H, T, I, E) . We proceed by decomposing the disjunction on the left-hand side by resolving with ∨−split . Doing so builds a disjunction for S, by instantiating S(H, T, I, E) with S1 (H, T, I, E) ∨ S2 (H, T, I, E), and yields the two subgoals: ∃S . append (nil, E · S , H · T ) ∧ length(nil, I) ↔ S1 (H, T, I, E) ∃H , T , S . append (H · T , E · S , H · T ) ∧length(H · T , I) ↔ S2 (H, T, I, E) For the first, the left-hand side is true whenever ∃S . E = H ∧ S = T ∧ I = 0. Hence, setting S to T , this subgoal is equivalent to (E = H ∧ I = 0) ↔ S1 (H, T, I, E) . We can again discharge this using ↔−refl, which unifies S1 (H, T, I, E) with E = H ∧ I = 0. Now, under the standard definition of append and length, the second subgoal is equivalent to (∃I .s(I ) = I ∧ (∃T , S .append (T , E · S , T ) ∧ length(T , I ))) ↔ S2 (H, T, I, E) where s(I ) represents the successor of I . We can now simplify this using the antecedent (induction hypothesis), which yields (∃I .s(I ) = I ∧ R(T, I , E)) ↔ S2 (H, T, I, E) . We complete the proof with ↔−refl , unifying S2 (H, T, I, E) with ∃I .s(I ) = I ∧ R(T, I , E). We are done! If we apply the accumulated substitutions to the remaining assumption A1 we have ∀L, I, E . R(L, I, E) ↔ (L = nil ∧ false) ∨∃H, T . L = H · T ∧ ( (E = H ∧ I = 0) ∨ ∃I . s(I ) = I ∧ R(T, I , E)) .
40
David Basin et al.
and we have proved the equivalence of (3) under this definition, i.e., atpos(L, I, E) is equivalent to the synthesized instance of R(L, I, E). The alert reader may have wondered why we did not complete the proof earlier by resolving with ↔−refl. In this example, our goal was to transform atpos so that the result falls within a particular subset of first-order formulae, sometimes called pure logic programs [16] or logic descriptions [25], that define logic programs. These formulae can be easily translated to Horn clauses or run directly in a language like G¨ odel [47]. In this case, we get the clauses: atpos(nil, I, E) ← f alse atpos(H · T, I, E) ← E = H, I = 0 atpos(H · T, I, E) ← s(I ) = I, atpos(T, I , E)
which can be simplified to atpos(E · , 0, E) ← atpos( · T, s(I ), E) ← atpos(T, I , E) 3.4
Analysis
Overall, when cast in this way, the deductive synthesis of logic programs is quite similar to the previous constructive/deductive synthesis of functional programs. In both cases, we leave the program as an unknown, and solve for it, by unification, during proof. Of course, the metatheoretic properties of the programs produced are quite different. In the case of logic program synthesis, the rules, as they are given, do not enforce that the object constructed has any special syntactic properties (e.g., is a pure logic program); we only know that it is an equivalent formula. Moreover, we do not a priori know anything about its termination behavior (although it is not difficult to show that the induction rule builds predicates that terminate when the first argument is ground). This kind of development, as with most approaches to logic program synthesis, is best described as deductive synthesis. They are constructive only in the weak sense that, at the metalevel (or metalogic, if one is carrying out the proof in a logical framework), one is essentially proving a theorem of the form ∃R . ∀L, I, E .atpos(L, I, E) ↔ R(L, I, E) and building a witness (in this case, a predicate definition) for R. (For more on this notion of constructivity and the proof theory behind it, see [11].) Many proposed methods for the constructive synthesis of logic programs can also be explained in this way. For example, the Whelk Calculus of [71], which is motivated by experiments in synthesizing relations in a constructive type theory, can be recast as this kind of synthesis [6].
Synthesis of Programs in Computational Logic
4
41
Schema-Guided Synthesis
We here outline Flener, Lau, Ornaghi, and Richardson’s definition, representation, and semantics of program schemas: see [33] for details. 4.1
Background
Intuitively, a program schema is an abstraction of a class of actual programs, in the sense that it represents their data-flow and control-flow, but neither contains all their actual computations nor all their actual data structures. Program schemas have been shown to be useful in a variety of applications. In synthesis, the main idea is to simplify the proof obligations by taking the difficult ones offline, so that they are proven once and for all at schema design time. Also, the reuse of existing programs is made the main synthesis mechanism. A symbol occurring in a theory T is open [52] in T if it is neither defined in T , nor a predefined symbol. A non-open symbol in T is closed in T . A theory with at least one open symbol is an open theory; otherwise it is closed. This terminology applies to formal specifications and logic programs. An (open) program for a relation r is steadfast [25,53] with respect to its specification if it is correct with respect to its specification whenever composed with programs that are correct with respect to the specifications of its (open) relations other than r. Among the many possible forms of programs, there are the divide-and-conquer programs with one recursive call: if a distinguished formal parameter, called the induction parameter, say X, has a minimal value, then one can directly solve for a corresponding other formal parameter, called the result parameter, say Y ; otherwise, X is decomposed into a “smaller” value T (under some well-founded relation ≺) by splitting off a quantity H, so that a sub-result V corresponding to T can be computed by a recursive call, and an overall result Y can be composed from H and V . A third formal parameter, called the passive parameter, say Z, participates unchanged in these operations. Formally, this problem-independent dataflow and control-flow can be captured in the following open program for r: r(X, Y, Z) ← min(X, Z), solve(X, Y, Z) r(X, Y, Z) ← ¬min(X, Z), dec(X, Z, H, T ), r(T, V, Z), comp(H, Z, V, Y )
(DC)
The relations min, solve, dec, comp are open. When I is the induction parameter, L the result, and E the passive parameter, so that atpos(L, I, E) ↔ r(I, L, E), a closed program for atpos is the instance of DC under the program substitution min(X, Z) ← X = 0 dec(X, Z, H, T ) ← X = s(T )
solve(X, Y, Z) ← Y = Z · S comp(H, Z, V, Y ) ← Y = F · V
(φ1 )
This substitution captures the problem-dependent computations of that program. But programs by themselves are syntactic entities, hence some programs are undesired instances of open programs. For instance, the generate-and-test
42
David Basin et al.
program r(X, Y, Z) ← g(X, Y, Z), t(Y, Z) is an instance of DC under the substitution min(X, Z) ← true dec(X, Z, H, T ) ← true
solve(X, Y, Z) ← g(X, Y, Z), t(Y, Z) comp(H, Z, V, Y ) ← true
An open program such as DC thus has no fixed meaning. The knowledge captured by an open program is not completely formalized, and the domain knowledge and underlying language are still implicit. In order for such open programs to be useful for guiding synthesis, such undesired instances need to be prevented and some semantic considerations need to be explicitly added. A program schema [33] has a name, a set of formal sort and relation parameters, a signature with sorted relation and function declarations, a set of axioms defining the declared symbols, a set of constraints restricting the actual parameters, an open program T called the template, and specifications S of the relations in T , such that T is steadfast with respect to S in that axiomatization. The schema DC can be abduced, as in [32], from our informal account of how divide-and-conquer programs work. The parameters SX, SY, SZ, SH are sorts; they are used in the signatures of the other parameters, which are relations. There are no axioms because the signature declares no other symbols than the parameters. The template is the open program DC, which defines the relation r and has min, solve, dec, comp as open relations. The closed relation r is specified by Sr , and the open relations have Smin , Ssolve , Sdec , Scomp as specifications. The conditional specification Sr exhibits ir , or as the input/output conditions of r, while Sdec exhibits idec , odec as the input/output conditions of dec. The input/output conditions of the remaining open relations are also expressed in terms of the parameters ir , idec , or , odec . The constraints restrict dec to succeed at least once if its input condition holds, and then to yield a value that satisfies the input condition of r (so that a recursive call to r is “legal”) and that is smaller than X according to ≺, which must be a well-founded relation (so that recursion terminates). The open program DC is steadfast with respect to Sr , within the given axiomatization. In the schema REUSE, the parameters SX, SY, SZ are sorts; they are used in the signatures of the other parameters, which are relations. There are no axioms because the signature declares no other symbols than the parameters. The template is the open program {r(X, Y, Z) ← q(X, Y, Z)}, which defines the relation r and has q as the open relation. The relation r is specified by Sr , and the relation q has the same input/output conditions as r. There are no constraints on the parameters. This schema provides for the reuse of a program for q when starting from a specification for r. The open program Reuse is steadfast with respect to Sr , within the given axiomatization. 4.2
Overview
Let us now examine the specifications, mechanism, heuristics, background knowledge, human interaction, tool support, and scalability of schema-guided synthesis.
Synthesis of Programs in Computational Logic Schema DC(SX, SY, SZ, SH, ≺, ir , or , idec , odec ) sorts: SX, SY, SZ, SH ≺ : (SX, SX) relations: ir , idec : (SX, SZ) : (SX, SY, SZ) odec : (SX, SZ, SH, SX) or axioms: (none) constrs: idec (X, Z) → ∃H : SH . ∃T : SX . odec (X, Z, H, T ) idec (X, Z) ∧ odec (X, Z, H, T ) → ir (T, Z) ∧ T ≺ X wellF ounded(≺) specifs: ir (X, Z) → ( r(X, Y, Z) ↔ or (X, Y, Z) ) ir (X, Z) → ( min(X, Z) ↔ ¬idec (X, Z) ) ir (X, Z) ∧ ¬idec (X, Z) → ( solve(X, Y, Z) ↔ or (X, Y, Z) ) idec (X, Z) → ( dec(X, Z, H, T ) ↔ odec (X, Z, H, T ) ) odec (X, Z, H, T ) ∧ or (T, V, Z) → ( comp(H, Z, V, Y ) ↔ or (X, Y, Z) ) template: r(X, Y, Z) ← min(X, Z), solve(X, Y, Z) r(X, Y, Z) ← ¬min(X, Z), dec(X, Z, H, T ), r(T, V, Z), comp(H, Z, V, Y ) Schema RE USE(SX, SY, SZ, ir , or ) sorts: SX, SY, SZ or : (SX, SY, SZ) relations: ir : (SX, SZ) axioms: (none) constraints: (none) specifications: ir (X, Z) → ( r(X, Y, Z) ↔ or (X, Y, Z) ) ir (X, Z) → ( q(X, Y, Z) ↔ or (X, Y, Z) ) template: r(X, Y, Z) ← q(X, Y, Z)
43
(C1 ) (C2 ) (C3 ) (Sr ) (Smin ) (Ssolve ) (Sdec ) (Scomp ) (DC)
(Sr ) (Sq ) (Reuse)
Specifications. Among the many possible forms of specifications, there are the classical conditional specifications: under some input condition ir on inputs X, Z, a program for relation r succeeds iff some output condition or on X, Z and output Y holds. Formally, this gives rise to the following open specification of r: (Cond) ∀X : SX . ∀Y : SY . ∀Z : SZ . ir (X, Z) → ( r(X, Y, Z) ↔ or (X, Y, Z) ) The open symbols are the relations ir , or and the sorts SX, SY, SZ. Other forms of specification can also be handled. Mechanism. Schema-guided synthesis from a specification S0 is a tree construction process consisting of 5 steps, where the initial tree has just one node, namely S0 : 1. Choose a specification Si that has not been handled yet. 2. Choose a program schema with parameters P , axioms A, constraints C, template T , and specifications S.
44
David Basin et al.
3. Infer a substitution θ1 under which Si is an instance of the specification (available in S) of the defined relation in template T . This instantiates some (if not all) of the parameters P . 4. Choose a substitution θ2 that instantiates the remaining (if any) parameters in P , such that the constraints C hold (i.e., such that θ1 ∪ θ2 C) and such that one can reuse existing programs PQ for some (if not all) of the now fully instantiated specifications S ∪ θ1 ∪ θ2 of the open relations in template T . Simplify the remaining (if any) specifications in S ∪ θ1 ∪ θ2 , yielding SG . 5. Add T ∪ PQ — called the reused program — to the node with Si and add the elements of SG to the unhandled specifications, as children of Si . These steps are iterated until all specifications have been handled; the overall result program P0 for S0 is then assembled by conjoining, at each node, the reused programs. If any of these steps fails, synthesis backtracks to its last choice point. Schema-guided program synthesis is thus a recursive specification (problem) decomposition process followed by a recursive program (solution) composition process. The REUSE schema can be chosen at Step 2; it forces the reuse at Step 4 of a program for q, because q is its only open relation. Every schema leads to some reuse at Step 4; for instance, DC results in the reuse of a program for dec. Heuristics. Many choice points reside in schema-guided synthesis, so heuristics are needed to make good decisions, possibly by looking ahead into the synthesis. Some heuristics can be applied when designing a schema. For instance, a synthesis strategy is the choice at Step 4 of the open relations for which programs are reused. All templates envisaged by us so far have only a few meaningful strategies, hence it is best to hardwire these. For instance, template DC has only two interesting strategies: when starting with dec, the divide-and-conquer schema is as above; when starting with comp, it would have to be reexpressed in terms of the input/output conditions of r and comp, giving rise to another schema, with the same template. Other heuristics can be expressed as applicability conditions. For instance, the question arises of what program schema to apply at Step 2. An implicit heuristic can be achieved by ordering the schemas; putting REUSE first would enforce our emphasis on reuse. There also is the question of how to apply a chosen program schema at Step 3. For instance, with DC, one of the formal parameters in the given specification Sr has to be the induction parameter, and another the result parameter. This can be done based on the sort information in Sr : only a parameter of an inductively defined sort can be the induction parameter. One can also augment specifications with mode information, because parameters declared to be ground at call-time are particularly good induction parameters [25]. Background Knowledge. Step 2 assumes a base of program schemas, capturing a range of program classes. Also, Step 4 relies on a base of reusable programs. For instance, for the DC schema, a base of specifications and programs for dec programs and ≺ well-founded relations needs to be available.
Synthesis of Programs in Computational Logic
45
Human Interaction. Schema-guided synthesis can be fully automated, as demonstrated with Cypress [65], Kids [66], DesignWare [67], and PlanWare [15]. However, interactive synthesis is preferable, with the human programmer taking the creative, high-level, heuristic design decisions, and the synthesizer doing the more clerical work. The design issues are intelligible to humans because the very objective of program schemas is to capture recognized, useful, humandesigned programming strategies and program classes. Tool Support. An implementation of schema-guided synthesis can be made on top of any existing proof planner, exploiting the fact that program schemas can be seen as proof methods [35]. This provides support for the necessary higherorder matching and discharging of proof obligations. Scalability. The search space of schema-guided synthesis is much smaller than for deductive synthesis. First, schema-guided synthesis by definition bottoms out in reuse, both of the template itself and of existing programs. One can significantly reduce the number of reuse queries by applying heuristics detecting that an ad hoc program can be trivially built from the specification. Second, the proof obligations of Steps 3 and 4 are quite lightweight. Schema-guided synthesis thus scales up to real-life synthesis tasks, especially if coupled with a powerful program optimization workbench and sufficient domain knowledge. For instance, Smith [67] has successfully deployed his tools on real-life problems, such as transportation scheduling. 4.3
Example
Let us synthesize a program from the following specification, open in sort ST: ∀L : list(ST) . ∀I : nat . ∀E : ST . true → ( atpos(L, I, E) ↔ ∃P , S : list(ST) . append (P , E · S , L) ∧ length(P , I))
(Satpos )
The first iteration of synthesis proceeds as follows. At Step 1, the specification Satpos is chosen because it is the only unhandled specification. At Step 2, suppose schema DC is chosen, after a failed attempt to apply schema REUSE . At Step 3, the specification Satpos is inferred to be an instance of Sr , when atpos(L, I, E) is seen as r(I, L, E), under the substitution
SX, SY, SZ = nat, list(ST), ST ir (X, Z) ↔ true or (X, Y, Z) ↔ ∃P , S : list(ST) . append (P , Z · S , Y ) ∧length(P , X)
(φ2 )
So far, 5 of the 9 parameters of DC have been instantiated. At Step 4, suppose the following substitution is chosen: SH = nat idec (X, Z) ↔ ¬X = 0
A ≺ B ↔ B = s(A) odec (X, Z, H, T ) ↔ X = s(T )
46
David Basin et al.
This instantiates the remaining 4 parameters of DC in a way that the constraints C1 , C2 , C3 hold and that the program Pdec = {dec(X, Z, H, T ) ← X = s(T )} can be reused to meet the now fully instantiated specification Sdec . The specifications of the remaining open relations in template DC are now also fully instantiated: true → ( min(X, Z) ↔ ¬¬X = 0 ) (Smin ) true ∧ ¬¬X = 0 → (solve(X, Y, Z) ↔ ∃P , S . append (P , Z · S , Y ) ∧ length(P , X)) (Ssolve ) X = s(T ) ∧ ∃P , S . append (P , Z · S , V ) ∧ length(P , T ) → (Scomp ) ( comp(H, Z, V, Y ) ↔ ∃P , S . append (P , Z · S , Y ) ∧length(P , X) ) They can be simplified into the following specifications: ) min(X, Z) ↔ X = 0 (Smin X = 0 → ( solve(X, Y, Z) ↔ ∃S : list(ST) . Y = Z · S ) (Ssolve ) X = s(T ) ∧ ∃P , S . append(P , Z · S , V ) ∧ length(P , T ) → (Scomp ) (comp(H, Z, V, Y ) ↔ ∃F : ST . Y = F · V )
At Step 5, the program DC ∪ Pdec becomes the reused program for Satpos , , Ssolve , and Scomp are added to the now empty list of unhandled while Smin specifications. , Ssolve , and The next iterations of synthesis proceed as follows. When Smin Scomp are chosen, suppose applications of some suitable variants of REUSE succeed through the ad hoc building of the programs Pmin = {min(X, Z) ← X = 0}, Psolve = {solve(X, Y, Z) ← Y = Z · S}, and Pcomp = {comp(H, Z, V, Y ) ← Y = F · V }. Since no new specifications were created, the synthesis is completed and has discovered the substitution φ1 . For call-mode atpos(+, −, +), say, the corresponding logic program atpos(L, I, E) ← I = 0, L = E · S atpos(L, I, E) ← ¬I = 0, I = s(T ), atpos(V, T, E), L = F · V can be implemented [25], say by the Mercury compiler [68], into the following steadfast program: atpos(E · S, 0, E) ← atpos(F · V, s(T ), E) ← atpos(V, T, E) The comp operator had to be moved in front of the recursive call to achieve this. (Prolog cannot do this, so mode-specific implementation is left as a manual task to the Prolog programmer.) This example illustrated a relatively simple use of the DC schema. In [31], a quicksort program is synthesized, using a variant of the divide-and-conquer schema DC with two recursive calls.
Synthesis of Programs in Computational Logic
4.4
47
Analysis
Schema-guided synthesis captures recognized, useful, human-designed programming strategies and program classes in program schemas. In doing so, it takes the hardest proof obligations offline, preventing their repeated proof across various syntheses and making reuse of existing programs the central mechanism for synthesizing programs. In the presence of powerful program optimization tools and sufficient domain knowledge, it thus naturally scales up, without any limitations on specification forms or program forms, due to the modular nature of the various forms of background knowledge. Heuristic guidance issues are still best tackled by humans, so schema-guided synthesis is best carried out interactively. A unified view of schema-guided synthesis and proof planning has been proposed [35], revealing potential new aspects of program schemas, such as applicability conditions capturing heuristics, as well as the possibility of formulating program schemas as proof methods and thereby reusing an existing proof planner as a homogeneous implementation platform for both the schema applications and the proof obligations of schema-guided synthesis. Our future work includes redoing the constraint abduction process for more general divide-and-conquer templates, where some nonM inimal(X, Z) is not necessarily ¬min(X, Z), and crafting the corresponding strategies, in order to allow the synthesis of a larger class of programs. Other design methodologies need to be captured in logic programming schemas; for instance, a global search schema has been proposed for the synthesis of constraint logic programs [37].
5
Inductive Synthesis
Following a brief introduction to inductive generalization, we present a particular approach to induction of recursive logic program called compositional inductive synthesis, which is described in detail in [46]. 5.1
Background
The inductive approach to program synthesis originates in inductive logic. Inductive logic is concerned with the construction of logical theories T explaining available observations or events. This means that, given evidence in the form of atomic formulas a1 , a2 , . . . , as , the logical induction approach is to devise an appropriate logical theory T so that T a1 ∧ a2 ∧ . . . ∧ as . A major concern is to constrain T so as to rule out trivial solutions, such as T being inconsistent (thus supporting any evidence), or T being identical to the conjunction of available evidence. In the more traditional application of logical theories of induction in artificial intelligence, the quest is for a theory T taking the form of general rules, e.g., scientific rules, supporting the given evidence. In the context of induction of logic programs addressed here, the “observations” are
48
David Basin et al.
intended sample program input-output results in the form of atomic formulas, and the theory T is to be a definite clause logic program. Thus the consistency of T is guaranteed, but computational properties such as termination and computational tractability of the synthesized program have to be separately considered. So the goal of inductive logic programming (ILP) is to obtain a collection of clauses with universally quantified variables, which subsumes the given finite list of intended program results. The main approach to achieve this goal is syntactic generalization of the given examples. Consider atoms p(a, a·b·nil) and p(b, b·nil). These two unit clauses generalize to the clause program p(X, X ·Y ) ←. This rests on the existence of a dual of the most general unifier of two atoms known as the least general generalization (LGG) [63,62]. In this simple case, the LGG yields the intended program as a unit clause witness, p(X, X · Y ) p(a, a · b · nil) ∧ p(b, b · nil). The syntactical generalization of terms has been extended to a notion of generalized subsumption of clauses [18,63] and further to a method known as inverse resolution, see e.g., [58]. This method has proven useful for concept formation, deductive databases and data mining. However, it is too weak for induction of recursive logic programs. Consider examples of list concatenation, e.g., p(nil, a · nil, a · nil) and p(a · nil, b · nil, a · b · nil). The least general generalization yields the clause p(X, Y · nil, a · Z) ←, which fails to capture the recursive definition of concatenation. Providing more examples eventually leads to an overly general clause: the universal predicate p(X, Y, Z), which subsumes all concatenation examples though it blatantly fails to capture concatenation of lists. A general remedy for over-generalization is to include negative examples, which are understood as examples in the complement set of the intended result set of atoms. In general, the key problem in synthesizing such programs is the invention and introduction of appropriate recursive forms of clauses. Compositional inductive synthesis employs a compositional logical language for computing relations in analogy to functional programming languages intended for composing and computing functions. The method does not apply the above generalization mechanisms. A program takes the form of a variable-free predicate expression ϕ encompassing elementary predicates and operators for combining relations and producing new resulting relations. Let ϕ e mean that the tuple (of terms) e is deducible from the program predicate expression ϕ. The computational semantics of the language can then be explained by means of inference rules of the form ϕ1 e1
...
ϕn en
op(ϕ1 , . . . , ϕn ) e
,
where e depends on op and e1 , . . . , en , as explicated in the concrete rules below. Let ϕ e1 + . . . + en mean ϕ ei for i = 1..n, so that + combines result tuples. Thus, ϕ e1 +e2 +. . . expresses that the tuples ei of the term form t1 , t2 , . . . , tn are computable from the n-ary predicate expression ϕ.
Synthesis of Programs in Computational Logic
49
In the language Combilog employed here, the given elementary predicates are constant formation, identity and list construction defined by the inference rules: constc c
id t, t
cons h, t, h · t
In addition to the elementary predicates, there is a collection of operators, which map argument relations to relations. The three fundamental operators are here defined by: ϕ t1 , t2 , . . . , tn make µ1 ,µ2 ,...,µm (ϕ) tµ1 , tµ2 , . . . , tµm ϕ1 e + e
ϕ2 e + e
and (ϕ1 , ϕ2 ) e
(and )
(make)
ϕ1 e1
ϕ2 e2
or (ϕ1 , ϕ2 ) e1 + e2
(or )
The make operator is a generalized unary projection operator carrying an auxiliary vector of indices µ1 , . . . , µm serving to reorder arguments and introduce don’t cares. As described in [46], Combilog possesses a compositional semantics in which and is set intersection and or is set union, which motivates the inference rules for the and and or operators. These operators reflect, respectively, logical conjunctions in clause bodies and multiple defining clauses. This operator language becomes as expressive as ordinary clause programs if the language is extended with facilities for naming predicate expressions and using these names recursively in program predicate definitions. However, in the present form the language does not introduce predicate names in a program. Instead, the defined predicates are anonymous and in order to accommodate recursive formulations e.g., for list processing, the iteration operators foldr and foldl are introduced. These operators are akin to the fold operators in functional programming and with theoretical underpinning in the theory of primitive recursive functions as discussed in [45,46], The associated rules are: ψ t1 , t3 foldr (ϕ, ψ) t1 , nil, t3
(foldr 0)
foldr (ϕ, ψ) t1 , t2 , z ϕ h, z, t3 foldr (ϕ, ψ) t1 , h · t2 , t3 ψ t1 , t3 foldl (ϕ, ψ) t1 , nil, t3
(foldl 0)
ϕ h, t1 , z foldl (ϕ, ψ) z, t2 , t3 foldl (ϕ, ψ) t1 , h · t2 , t3
(foldr > 0)
(foldl > 0)
For instance, with foldr available, the well-known append concatenation predicate is make 2,1,3 (foldr (cons, id )), where the make operator swaps the two first arguments.
50
David Basin et al.
Below we illustrate the application of the rules using the append program, proving make 2,1,3 (foldr (cons, id )) a · nil, b · nil, a · b · nil: id b · nil, b · nil foldr (cons, id) b · nil, nil, b · nil
(foldr 0)
cons a, b · nil, a · b · nil
foldr (cons, id ) b · nil, a · nil, a · b · nil
(foldr > 0)
(make) make 2,1,3 (foldr (cons, id )) a · nil, b · nil, a · b · nil When the inference rules are used to compute result tuples, these tuples are unknown parameters to be determined in the course of the execution. In contrast, in the compositional inductive synthesis method, the result tuples are given initially, as a contribution to the result, whereas ϕ1 , . . . , ϕn are (partly) unknown program constituents to be determined recursively in the course of the synthesis. These inference rules are used in the way described in Section 3.1 for building proofs in a goal directed manner where the program constructs are unknowns, given as metavariables, and instantiated during proof. This facilitates understanding of the induction process as a stepwise, principled, program composition process. 5.2
Overview
Let us now present compositional inductive synthesis in terms of its generic features. Specifications. In inductive synthesis, specifications are partial extensional definitions of the programs to be induced, i.e., a set of atoms or tuples constituting sample program results. No other problem specific specifications need be employed. Mechanism. The operators are similar to schemas in the schema guided approach to synthesis. In the present method, the program is synthesized in a strict recursive divide-and-conquer process by tentatively selecting an operator and then recursively attempting synthesis of constituent parameter programs. Our synthesis takes advantage of the metainterpreter outlined below for compositional programs and does not rely on generalization mechanisms. The approach can be characterized as the top-down stepwise composition and specialization of a Combilog program intended as a solution in the sense that the program subsumes the program examples. The search involved in choosing between operators is taken care of by the back-tracking mechanism in the synthesizer. In principle, our synthesis proceeds by introducing meta-variables for the left operand predicate expressions of in the proof construction, and then successively instantiating these variables in the course of the goal-driven proof construction; in doing so, we also appeal to the rule ϕ e1
ϕ e2
ϕ e1 + e2
,
Synthesis of Programs in Computational Logic
51
which is used for goal splitting on the program examples. Thus the above proof may be conceived of as a trace of a sample inductive synthesis proof. In our metainterpreter system, the relationship ϕ e is realized as a binary predicate syn, which simultaneously serves as metainterpreter and synthesizer. The key principle of our synthesis method is the inverted use of our metainterpreter so that the first argument program predicate is to be instantiated in the course of synthesizing a program. Thus the heart of the synthesizer is clauses of the following, general, divideand-conquer form for the available operators: syn(comb(P1 , . . . , Pm ), Ex ) ← apply comb(Ex , Ex 1 , . . . , Ex m ) ∧ syn(P1 , Ex 1 ) ∧ . . . ∧ syn(Pm , Ex m ). Programs consisting of an elementary predicate are trivially synthesized without recursive invocation of syn. Let us consider the synthesis of a basic predicate expression for the head predicate yielding the head of a non-empty list, given say the two examples a · b · nil, a and a · nil, a. Synthesis of head is initiated with a goal clause ← syn(P, [[a, b], a]) ∧ syn(P, [[a], a]). A successful proof instantiates P with the synthesized expression make 3,1 (cons). Heuristics. A detailed description of the synthesizer is found in [46]. To prevent the synthesizer from running astray in the infinite space of possible program hypotheses, the search is conducted as an iterative deepening. To avoid unwanted trivial program solutions, further constraints are imposed on the synthesizer. Consider, for instance, synthesis of the append predicate. An overly general solution is obtained as the universal predicate, say, with the expression make 2,3,4 (constc ) corresponding to the clause p(X1 , X2 , X3 ). As mentioned, such unwanted solutions might be ruled out by the use of negative examples. However in our synthesizer we have chosen to enforce well-modedness constraints on the synthesized programs thus suppressing the above solution in favor of the recursive P = make 2,1,3 (foldr (cons, id )), which is obtained as the syntactically smallest solution given the two sample results nil, nil, nil and a · nil, b · nil, a · b · nil and the mode pattern [+, +, −], and complying with the usual clauses for append. The synthesis proceeds as a goal-driven proof construction of the sample proof shown in the above section. Background Knowledge. The elementary predicates and the operators determine the admissible forms of programs and thereby constitute a form of background knowledge. No problem-specific background knowledge is provided but a search bias may be imposed by providing additional auxiliary predicates.
52
David Basin et al.
Tool Support. For synthesizing the at pos program, a system called Combinduce was used, which is based on the method outlined above and described in detail in [46]. Human Interaction and Scalability. The current experimental system conducts the inductive synthesis automatically. The computational search costs limit the size of inducible programs to around 6 predicates and operators. However, we envisage integration of the CombiInduce principles into a semiautomatic compositional development system. In this system, the programmer can offer assistance by proposing appropriate auxiliary predicates within the pertinent data type. The imposition of data types will also serve to constrain further the search space of well-moded program candidates. Recursion (fold) over lists will be generalized to other data types later. 5.3
Example
Since at this stage, the synthesis system supports list as the only data type we represent the number n as a list of length n with constants i, where i can be any constant. Synthesis of the atpos program from the single sample a·b·nil, i·nil, b yields the solution atpos = foldl (make 4,3,2 (cons), make 3,1 (cons))) as illustrated by the following trace: make 3,1 (cons) b · nil, b make 4,3,2 (cons)
, a · b · nil, b · nil
foldl (make 4,3,2 (cons), make 3,1 (cons))
b · nil, nil, b
(foldl 0)
(foldl > 0) foldl (make 4,3,2 (cons), make 3,1 (cons)) a · b · nil, i · nil, b The synthesized program is the Combilog form of the definite clause program atpos(L, I, E) ← syn(foldl (tail , head ), [L, I, E]) syn(tail , [ , F · T, T ]) ← syn(head , [F · T, F ]) ← Synthesis with the foldr operator is not possible. However, swapping the two subgoals of foldr yields the operator foldrrev allowing the following variant program to be synthesized atpos = make 3,2,1 (foldrrev (cons, make 1,3 (cons))). The relationship between such a pair of variant programs is theoretically established by a duality theorem stated and proved in [44]. In order to facilitate the comparison of the synthesis approaches, let us transform the first Combilog form of the atpos definite clause program into a recursive atpos program. We first unfold the atpos clause: atpos(L, nil, E) ← head (L, E) atpos(L, X · T, E) ← tail (X, L, Z), syn(foldl (tail , head ), [Z, T, E])
Synthesis of Programs in Computational Logic
53
Now, unfolding head and tail, and folding back the second literal with atpos, we obtain the following logic program. atpos(L, nil, E) ← L = E · T atpos(L, X · T, E) ← L = F · Z, atpos(Z, T, E)
5.4
Analysis
Check that meaning is preserved! Designing a metainterpreter for Combilog is simplified by the variable-free form of Combilog programs, the separation of predicate expressions and terms in separate arguments, and the elimination of introduced predicate names. These simplifications substantially reduce search and allow us to effectively use the metainterpreter as the backbone of our ILP method by reversing the provability metalogic programming demo predicate as examined e.g., in [43] and in [21] for ordinary definite clauses. In [46] we compare with other inductive synthesis systems and report results on successful automatic synthesis of a number of textbook programs including non-naive as well as naive reversal of lists. The latter program makes calls for the auxiliary predicate append, which is recursively induced. This predicate invention, which is generally considered problematic in ILP, is handled smoothly in our compositional method since explicit predicate names are not introduced. The outlined compositional method facilitates a program development methodology where customized domain specific operators are added to the general purpose ones. Moreover, it seems that the compositional method surpasses more traditional ILP methods with respect to predicate invention and termination of induced programs within the considered class of primitive recursive relations delineated by the available recursive operators.
6
Comparison
In this section, the synthesis approaches are compared from different points of view. First, we compare the synthesized atpos programs. Afterwards, we contrast the general features of the different approaches. Finally, we conclude by analyzing how schemas are used, implicitly or explicitly, in program synthesis and we suggest that they play a central role in understanding different synthesis methods. In the following, we will refer to inductive synthesis, deductive synthesis, and schema-guided synthesis to denote the particular synthesis methods presented in this paper. 6.1
The atpos(L,I,E) Program
All three methods yielded the same program. This was the case even though they differ in which variable they choose as an induction parameter: both inductive synthesis and schema-guided synthesis choose I as the induction parameter, while
54
David Basin et al.
deductive synthesis chooses L. In the case of deductive synthesis, we could just as well have carried out induction on I. However, for schema-guided synthesis, switching would require a separate schema with a different template, namely with an additional non-recursive clause for the non-minimal case. The same holds for inductive synthesis where a fold combinator over numbers and an associated rule would be required. In general, the choice of the induction parameter will affect the form of the resulting program and even its complexity [25]. In this regard, deductive synthesis offers more flexibility, as one can perform induction over any wellfounded relation, and development (hence program construction) proceeds in smaller steps. Of course, in schema-guided synthesis and inductive synthesis, one can always introduce new schemas, respectively operators, corresponding to new ways of building programs, as the need arises. 6.2
Specification
The forms of the specifications in deductive synthesis and schema-guided synthesis are similar. Both are first-order formulas asserting a possibly conditional equivalence. In inductive synthesis, the specification is a finite set of examples (a subset of the extensional definition of the relation), which is by nature incomplete (when the extensional definition is infinite). Specifications in inductive synthesis may also include negative examples or properties [28,36], but in general they remain incomplete. This incompleteness is a significant difference and, as we will see, it has far-reaching consequences. Indeed, it will play a key role in differentiating inductive synthesis from the other two approaches with respect to the other generic features. For the deductive synthesis and schema-guided synthesis approaches, in contrast to inductive synthesis, it is important for non-trivial applications to be able to construct complex specifications and this requires ways of parameterizing and combining specifications. In our work on deductive synthesis, we achieve this, in practice, by using logical frameworks like Isabelle [59], which provide support for structured theory presentations. In schema-guided synthesis, [33] express program schemas as extensions of specification frameworks [52], which support parameterized specifications and their composition. Of course, the use of first-order logic as a specification language has its limitations. For example, in schema-guided synthesis, we needed the well-foundedness of a relation ≺ as a constraint in the DC schema. However, a formalization of well-foundedness generally falls outside of first-order logic, unless one formalizes, e.g., set-theory. A work-around is to assume that some fixed collection of relations is declared to be well-founded. The alternative is to use a stronger (higher-order) logic or theory [1] where concepts such as well-foundedness can be defined and well-founded relations can be constructed. Stronger logics, of course, have their own drawbacks; in particular it is more difficult to automate deduction.
Synthesis of Programs in Computational Logic
6.3
55
Mechanism
As presented, the mechanisms used in the three methods appear quite dissimilar. Deductive synthesis is oriented around derivations, schema-guided synthesis was described using an algorithm for applying schemas, and inductive synthesis uses a meta-interpreter to build programs. Yet it is possible to recast all three so that the central mechanism is the same: a top-down application of rules is used to incrementally construct a program, during a derivation, in a correctness-preserving way. In deductive synthesis, derived rules are applied topdown, using higher-order unification to build programs as a “side-effect” of proof construction. Although the mechanism for applying schemas has been presented in an algorithmic fashion, it is possible to recast schema-guided synthesis as the application of rules in a deductive system [1]; namely, a schema constitutes a (derivable) rule whose premises are given by the schema’s constraints and (the completion of its) template and the conclusion is given by the schema’s specifications. Viewed in this way, schema-guided synthesis, like deductive synthesis, constructs programs, during proofs, by the higher-order application of rules. The main distinction between the two methods boils down to the rules, granularity of steps, and heuristics/interaction for constructing proofs. Finally, in inductive synthesis, rules are also given for constructing Combilog programs. There, the rules are automatically applied by a Prolog meta-interpreter. Although they differ in form, the rules employed by the different methods have a similar nature. Not surprisingly, in all cases, mathematical induction plays a key role in program synthesis, as it is necessary for constructing iterative or recursive programs. In deductive synthesis, induction principles can be derived from induction principles for data types or even the inductive (leastfixedpoint) semantics of logic programs [1]. The induction principles (perhaps in a reformulated form, e.g., the ind rule of Section 3.2) are then explicitly applied and their application constructs a template for a recursive program. In schemaguided synthesis, the correctness of schemas for synthesizing recursive programs is also justified by inductive arguments. Indeed, complex schemas can be seen as kinds of complex macro-development steps that precompile many micro steps, including induction. One might say that induction is implicitly applied when using a schema to construct recursive programs. In inductive synthesis, programs are iterative, instead of recursive, and programs that iterate over lists (or, more generally, other inductively defined data types) are built using fold rules. Again, mathematical induction principles play a role, behind-the-scenes, in justifying the correctness of iteration rules, and rule application can be seen as an implicit use of induction. There is, of course, a tradeoff. By compiling induction into specialized rules, schema-guided synthesis and inductive synthesis can take larger steps than deductive synthesis; however, they are more specialized. In particular, by building only iterative programs, the inductive synthesis method presented can sharply reduce the search space, but at the price of limited expressibility. The underlying mechanisms are, in some respects, fundamentally different. Although all three methods are based on first-order logic, any system implementing deductive synthesis (respectively schema-guided synthesis) will require
56
David Basin et al.
higher-order unification (respectively higher-order matching). This is necessary to construct substitution instances for variables in rules and schemas that range over functions, relations, and more generally, contexts (terms with holes); the downside is that higher-order matching and unification are more difficult than their first-order counterparts, and the existence of multiple unifiers (respectively matchers) can lead to large branching points in the synthesis search space. The operator form of Combilog means that rules in inductive synthesis manipulate only first-order terms. Moreover, all complications concerning object language variables are eliminated. This simplifies the metainterpreter and reduces the synthesis to search in the space of operator combinations subjected to wellmodedness constraints. Finally, the differing nature of the specifications, in particular, complete versus incomplete information, makes a substantial difference in the underlying semantics of the different methods and the relationship of the synthesized program to its specification. As presented here, both deductive synthesis and schemaguided synthesis construct programs that are (possibly under conditions) equivalent to some initial specification. In the case of inductive synthesis, equivalence is weakened to implication or entailment. This changes, of course, the semantics of the rules. Moreover it has a significant impact on extra-logical considerations, i.e., considerations that are not formalized in the synthesis logic (e.g., the program synthesized should have a particular syntactic form or complexity). In inductive synthesis these considerations (in particular, having a syntactically small recursive program that entails the examples) become central to the synthesis process and it is important to use a well-specified strategy, embodied in a metainterpreter, to ensure them. 6.4
Heuristics
Each of the methods presented has an infinite search space. However, the spaces are differently structured and different heuristics may be employed in searching them. In deductive synthesis, one proceeds in a top-down fashion, employing induction and simplification. The search space has both infinite branching points associated with the application of higher-order unification (as there may be infinitely many unifiers) and branches of unbounded length (as induction may be applied infinitely often and simplification may not necessarily terminate). In practice, an effective heuristic is to follow an induction step by eager simplification; here, rippling can be used to control the simplification process and guarantee its termination. Moreover, with the exception of applying induction, unification problems are usually of a restricted form, involving “second-order patterns,” which can be easily solved [51]. Hence, it is possible, in some cases, to use heuristics to reduce the search space to the point where synthesis can be completely automated. Schema-guided synthesis uses a strict recursive divide-and-conquer strategy in the selection of operators and the synthesis of the parameter programs. It also employs a stepwise composition/specialization of programs where the objective is
Synthesis of Programs in Computational Logic
57
to reuse existing code. Analogous to deductive synthesis, critical branch-points include schema selection and selection of a substitution (higher-order matching is required as the same schema can be used in different ways). Search can be conducted as an iterative deepening search employing heuristics. Although schema-guided synthesis also has an infinite search space, it is fair to say that when a program is in the search space, one is likely to find it more quickly than with deductive synthesis since the steps in schema-guided synthesis are larger, and hence the program is at a shallower ply in the search tree. The search space in inductive synthesis is more difficult to navigate than in the other two methods because of the additional extra-logical concerns mentioned previously. Here a strict control (dictated by a metainterpreter) is required to generate candidate programs in a particular order. To make automated search practical, the search space is restricted, a priori, by restrictions in the method. For example, the programs synthesizable are restricted to those involving iteration, instead of general recursion, and the use of combinators ensures that firstorder (Prolog) unification suffices for program construction. In addition there is the well-modedness requirement and, to reduce explosive branching, the use of or is restricted. It is an interesting question as to whether any of these pruning measures could be profitably used in the other approaches. 6.5
Background Knowledge
The three approaches formalize background knowledge in different ways. For deductive synthesis, background knowledge about data types is given by a standard first-order theory augmented with appropriately reformulated (for synthesis) induction schemas (e.g., ind ). For schema-guided synthesis, background knowledge must be formalized in terms of a base of program schemas, capturing a range of program classes, which may (or may not) directly incorporate information about data types, as well as a database of reusable programs and information about well-founded relations (typically associated with data types). Here, more work is usually required to formalize background knowledge, but the payoff is that this work is done once and for all and the resulting schemas can be used to reduce search and guide development to specialized classes of programs. For inductive synthesis, the background knowledge is basically the elementary operators (const, id, cons, etc.), which encode knowledge about iterative programs operating over lists. As with the other approaches, this knowledge is domaindependent, and synthesizing programs operating over other data types would require additional rules. 6.6
Human Interaction and Scalability
The deductive synthesis proof presented was constructed interactively. There, within a first-order formalization of list theory, specialized rules for synthesis were derived, and interactively applied. However, proof search can also be automated using tactics and one can adjust the size of proof steps by deriving new proof rules (analogous to complex program schemas). This process of writing
58
David Basin et al.
tactics and deriving new rules is open, leads to a customizable approach, and can, at least in theory, scale arbitrarily. The use of tactics also makes it possible to arbitrarily mix automation with human interaction. Conversely, the schema-guided synthesis method was presented as fully automatable, although a human could be used to drive the selection of schemas and substitution instances. Indeed, as with deductive synthesis, this is often preferable, as it provides a way of influencing extra-logical concerns, such as the complexity of the synthesized program. The approach scales well as specialized schemas can be tuned to particular classes of problems (divide and conquer, global search, etc.). Moreover, there is a natural mechanism for the reuse of programs. For the moment, there is no human interaction in the presented method for inductive synthesis. It is not clear either how feasible this is, given the importance that extra-logical concerns play in the synthesis process. How would a human know, for example, that steps suggested will generate the simplest possible program? The reuse of existing programs also is not handled. It is not clear how the inductive synthesis approach can be scaled up to synthesize more complex programs with recursion or iteration. For complex examples, the incomplete nature of the input specification makes the program space so intractable that human interaction, heuristics, support for reuse, and “more complete” specification information, such as properties [30,28], appear necessary. But even with these extensions, the purely inductive approach to the synthesis of programs with recursion or iteration remains very hard, and it seems doubtful whether this approach will ever scale up to the synthesis of complex, real-life programs. When the synthesized program does not feature recursion or iteration (and methods for this are outside the scope of this paper) then the inductive synthesis approach can usefully scale. This is witnessed by recent progress in ILP, on problems in domains, such as face recognition [54], where only (large) sets of input/output examples are available as humans have difficulty writing a formal, complete specification [34]. 6.7
Tool Support
For deductive synthesis, we used Isabelle [59], a generic logical framework, for our implementation. For schema-guided synthesis, the higher-order proof planning system λClam can be used, upon reformulation of the program schemas as proof planning methods [35]; this has the nice side-effect that the proof obligations of schema-guided synthesis can also be discharged using the same theorem proving machinery. For inductive synthesis, a specialized Prolog implementation was used. It is interesting to speculate on whether generic logical frameworks, like Isabelle, could be effectively used for all three approaches. And could the approaches even be profitably combined? Our discussion at the top of Section 6.3 suggests that a generic logical framework can effectively be used for schema-guided synthesis. Of course, there are
Synthesis of Programs in Computational Logic
59
some potential drawbacks. First, a logical framework requires recasting any synthesis method as one based on theorem proving; for instance, schema-guided synthesis was not cast this way in Section 4. This may require some contortions; see [9] for an example of this. Second, the logical framework will impose its own discipline for presenting and structuring theories, and this may deviate from that desired by a particular synthesis method; e.g., specification frameworks [52] provide more structuring possibilities than those possible using the Isabelle system. Finally, a hand-coded synthesis system will probably be more efficient. Although it is easy to write a Prolog interpreter (to realize inductive synthesis) as a tactic in a logical framework, this involves a layer of metainterpretation and a corresponding slow-down in execution time. The price may be too high when substantial search is involved. As to the question whether the approaches could be profitably combined, the answer is a clear ‘yes’ for deductive synthesis and schema-guided synthesis, and we will develop this point in the next sub-section. Combining inductive synthesis with the other approaches raises the question of how to deal with the ensuing redundancy in the overall specification, as the incomplete part supposedly is a logical consequence of the complete one. To a human programmer, examples attached to a specification that is intended to be complete often facilitate the understanding of the task. But an automated synthesizer probably does not need such help. Should there be a contradiction between the complete specification and the examples, then the overall specification is almost certainly wrong. In the absence of such a contradiction, one knows nothing about the quality of the overall specification and thus has to forge ahead. The question then arises of how to exploit the redundancy. A convincing proposal was made by Minton [57]: to cope with the instance sensitivity of the heuristics used to efficiently solve ubiquitous, NP-hard, constraint satisfaction problems, industry-strength solver synthesizers should use training instances (i.e., the input parts of examples) in addition to the specification of the problem, so that the most suitable heuristics can be empirically determined during synthesis. As long as the actual runs of the synthesised program are on instances within the distribution of the training instances, a good performance can be guaranteed. 6.8
Implicit versus Explicit Use of Schema
A central part of our comparison has been that the boundaries between deductive synthesis, schema-guided synthesis, and inductive synthesis are somewhat fluid with respect to the use of schemas. In particular, from the appropriate viewpoint, the difference between deductive synthesis and schema-guided synthesis is vanishingly small. We would like to close the comparison by driving these points home. The derived rules in deductive synthesis for reasoning about equivalences are rule schemas, i.e., rules with metavariables ranging over predicates. These are metavariables from the view of a metalogic, but they also can be viewed as uninterpreted relations in the object logic and play the same role as the open relation symbols in schema-guided synthesis. Viewed this way, if the background
60
David Basin et al.
theory of deductive synthesis is formalized as a specification framework, then the inference rules are a variation of the program schemas in schema-guided synthesis. For example, the ind rule with its assumptions A1 –A3 presented here in deductive synthesis is similar (although not equivalent) to the DC schema developed in schema-guided synthesis. In particular: – ind commits to an induction parameter of type list, whereas DC has an open sort SX for the induction parameter; – ind commits to one-step, head-tail decomposition of the induction parameter, whereas DC has an open relation dec for this; – DC commits to always one recursive call in the step case, whereas ind is flexible (there can be any number of recursive calls); – the assumption A1 of ind plays the same role as the template DC in DC, but they differ in content; – the predicate variable B of ind plays the same role as the open relation solve in DC; – the assumption A2 of ind plays the same role as the specification Ssolve in DC; – the predicate variable S of ind does not play the same role as the open relation comp in DC; indeed, an instance of S may include recursive call(s), whereas recursion is dictated by the template DC and is thus not considered when instantiating comp; – the assumption A3 of ind plays the same role as the specification Scomp in DC, but they differ in content; – there is no explicit equivalent of the constraints C1 , C2 , and C3 and the specifications Smin and Sdec of DC in ind. The differences here are not due to the underlying synthesis mechanism, but are an artifact of the particular implicit schema used (for reasons of simplicity) in this presentation of deductive synthesis. More elaborate rules and schemas, neither committed to a particular type nor a well-founded relation, have been developed in deductive synthesis, as presented in, e.g., [1,3]. A similar comparison can be made between the foldr and foldl operators in inductive synthesis, and the DC schema in schema-guided synthesis. The foldr and foldl operators can also be seen as implicit program schemas. More elaborate rules could also be used to build Combilog programs in larger steps. Program schemas are thus used (implicitly or explicitly) in the different synthesis approaches. In the literature, program schemas are often reduced to templates, formalized as higher-order expressions, and applied using higherorder unification. As shown in schema-guided synthesis, such templates must be enhanced with semantic information, expressed for instance through axioms, constraints, and specifications. Viewing such schemas as derivation rules, and schema application as logical inference, the distinction vanishes between the schema-guided and deductive/constructive approaches. For instance, in [1] it is shown how schemas for transformational development can be formalized as derived rules and combined with other kinds of verification and synthesis. In [30,28], a DC-like schema is used in the context of inductive synthesis.
Synthesis of Programs in Computational Logic
7
61
Conclusion
In this paper, we have analyzed and compared representative methods of three approaches to program synthesis in computational logic. Despite their differences, we established strong similarities. In particular, program schemas are used (implicitly or explicitly) in each of the methods and are central in driving the synthesis process and exploiting synergies. We would therefore like to conclude by discussing some limitations of schemas and open issues. Despite their central role, schemas have their limitations. Schemas are usually expressed in some logical language, but any given language has syntactical restrictions that in turn restrict what can be expressed as a schema. For example, a first-order language fixes the arity of predicates and functions, their associated types, etc. There is no way to capture certain simple kinds of generalization or extra-logical annotations, for example to employ term or atom ellipses t1 , . . . , tn of variable length n. As an example of this limitation, consider the ind rule of Section 3.2. There we used X to denote a sequence of zero or more variables and hence the induction rule given cannot be captured by a single schema, but rather requires a family of schemas, one for each n. Extensions here are possible; [64,28,70,39,20] provide notions of schema patterns that describe such families and can be specialized as needed before, or during, synthesis. Schemas are here defined as abstractions of classes of programs. At the same time, they formalize particular design strategies, such as divide-and-conquer or global search; part of the associated strategy can also be specified by associated tactics, which choose induction parameters, find appropriate well-founded relations, and so on. However, in their present form, schemas cannot handle more sophisticated design strategies, namely strategies abstracting a class of programs that cannot be obtained by instantiation with formulae. Typical examples are so-called design patterns [38], which aim at the description of software design solutions and architectures (typically described by UML diagrams and text). How to extend schemas to handle such strategies is an open problem in program synthesis. Overall, by examining the relationships and differences between the chosen synthesis methods, we have sought to bring out synergies and possibilities for cross-fertilization, as well as limitations. The primary synergies involve a common mechanism: a notion of schematic rule and the use of unification to apply rules in a top-down way that incrementally construct a program, during a derivation that demonstrates its correctness. The primary differences concern the nature of the specifications, in particular the information present; this also manifests itself in different semantics and radically different search spaces for the different methods. As it is, the purely inductive approach to the synthesis of programs with recursion or iteration remains very hard, and it seems doubtful whether this approach will ever scale up to the synthesis of complex, real-life programs. Fortunately, fruitful combinations of these synthesis approaches exist. In the end, we believe that progress in this field will be based on exploiting the identified synergies and possibilities for cross-fertilization, as well as supporting an enhanced, flexible use of schemas. We hope, with this paper, to have made a
62
David Basin et al.
constructive analysis of the last decade of research, thereby showing a possible path for the next decade.
Acknowledgements We would like to thank the anonymous referees for their feedback and our coinvestigators on research related to this paper.
References 1. P. Anderson and D. Basin. Program development schemata as derived rules. Journal of Symbolic Computation, 30(1):5–36, 2000. 2. A. Ayari and D. Basin. Generic system support for deductive program development. In T. Margaria and B. Steffen, editors, Proc. of TACAS’96, volume 1055 of LNCS, pages 313–328. Springer-Verlag, 1996. 3. A. Ayari and D. Basin. A higher-order interpretation of deductive tableau. Journal of Symbolic Computation, 2002. To Appear. 4. R. Balzer. A 15 year perspective on automatic programming. IEEE Transactions on Software Engineering, 11(11):1257–1268, 1985. 5. H.P. Barendregt. The Lambda Calculus: Its Syntax and Semantics, volume 103 of Studies in Logic. North-Holland, second, revised edition, 1984. 6. D. Basin. IsaWhelk: Whelk interpreted in Isabelle. In P. Van Hentenryck, editor, Proc. of ICLP’94, page 741. The MIT Press, 1994. 7. D. Basin. Logic frameworks for logic programs. In L. Fribourg and F. Turini, editors, Proc. of LOPSTR’94 and META’94, volume 883 of LNCS, pages 1–16. Springer-Verlag, 1994. 8. D. Basin. Logical-framework-based program development. ACM Computing Surveys, 30(3es):1–4, 1998. 9. D. Basin and S. Friedrich. Modeling a hardware synthesis methodology in Isabelle. Formal Methods in Systems Design, 15(2):99–122, September 1999. 10. D. Basin and B. Krieg-Br¨ uckner. Formalization of the development process. In E. Astesiano, H.-J. Kreowski, and B. Krieg-Br¨ uckner, editors, Algebraic Foundations of System Specification, pages 521–562. Springer-Verlag, 1998. 11. D. Basin and S. Matthews. Adding metatheoretic facilities to first-order theories. Journal of Logic and Computation, 6(6):835–849, 1996. 12. D. Basin and T. Walsh. Annotated rewriting in inductive theorem proving. Journal of Automated Reasoning, 16(1–2):147–180, 1996. 13. A.W. Biermann. Automatic programming. In S.C. Shapiro, editor, Encyclopedia of Artificial Intelligence, pages 59–83. John Wiley, second, extended edition, 1992. 14. A.W. Biermann, G. Guiho, and Y. Kodratoff, editors. Automatic Program Construction Techniques. Macmillan, 1984. 15. L. Blaine, L. Gilham, J. Liu, D.R. Smith, and S. Westfold. PlanWare: Domainspecific synthesis of high-performance schedulers. In Proc. of ASE’98, pages 270– 279. IEEE Computer Society Press, 1998. 16. A. Bundy, A. Smaill, and G.A. Wiggins. The synthesis of logic programs from inductive proofs. In J.W. Lloyd, editor, Computational Logic, Esprit Basic Research Series, pages 135–149. Springer-Verlag, 1990.
Synthesis of Programs in Computational Logic
63
17. A. Bundy, A. Stevens, F. van Harmelen, A. Ireland, and A. Smaill. Rippling: A heuristic for guiding inductive proofs. Artificial Intelligence, 62(2):185–253, 1993. 18. W. Buntine. Generalized subsumption and its application to induction and redundancy. Artificial Intelligence, 36(2):375–399, 1988. 19. C.-L. Chang and R.C.-T. Lee. Symbolic Logic and Mechanical Theorem Proving. Academic Press, 1973. 20. E. Chasseur and Y. Deville. Logic program schemas, constraints and semiunification. In N.E. Fuchs, editor, Proc. of LOPSTR’97, volume 1463 of LNCS, pages 69–89. Springer-Verlag, 1998. 21. H. Christiansen. Implicit program synthesis by a reversible metainterpreter. In N.E. Fuchs, editor, Proc. of LOPSTR’97, volume 1463 of LNCS, pages 90–110. Springer-Verlag, 1998. 22. E.M. Clarke and J.M. Wing. Formal methods: State of the art and future directions. ACM Computing Surveys, 28(4):626–643, 1996. 23. M.D. Coen. Interactive program derivation. Technical Report 272, Cambridge University Computer Laboratory, UK, 1992. 24. T. Coquand and G. Huet. The calculus of constructions. Information and Computation, pages 95–120, 1988. 25. Y. Deville. Logic Programming: Systematic Program Development. International Series in Logic Programming. Addison-Wesley, 1990. 26. Y. Deville and K.-K. Lau. Logic program synthesis. Journal of Logic Programming, 19–20:321–350, 1994. 27. A. Felty and D. Miller. Specifying theorem provers in a higher-order logic programming language. In E.L. Lusk and R.A. Overbeek, editors, Proc. of CADE’88, volume 310 of LNCS, pages 61–80. Springer-Verlag, 1988. 28. P. Flener. Logic Program Synthesis from Incomplete Information. Kluwer Academic Publishers, 1995. 29. P. Flener. Achievements and prospects of program synthesis. In A.C. Kakas and F. Sadri, editors, Computational Logic: Logic Programming and Beyond; Essays in Honour of Robert A. Kowalski, volume 2407 of Lecture Notes in Artificial Intelligence, pages 310–346. Springer-Verlag, 2002. 30. P. Flener and Y. Deville. Logic program synthesis from incomplete specifications. Journal of Symbolic Computation, 15(5–6):775–805, 1993. 31. P. Flener, K.-K. Lau, and M. Ornaghi. Correct-schema-guided synthesis of steadfast programs. In Proc. of ASE’97, pages 153–160. IEEE Computer Society Press, 1997. 32. P. Flener, K.-K. Lau, and M. Ornaghi. On correct program schemas. In N.E. Fuchs, editor, Proc. of LOPSTR’97, volume 1463 of LNCS, pages 124–143. SpringerVerlag, 1998. 33. P. Flener, K.-K. Lau, M. Ornaghi, and J.D.C. Richardson. An abstract formalisation of correct schemas for program synthesis. Journal of Symbolic Computation, 30(1):93–127, 2000. 34. P. Flener and D. Partridge. Inductive programming. Automated Software Engineering, 8(2):131–137, 2001. 35. P. Flener and J.D.C. Richardson. A unified view of programming schemas and proof methods. In A. Bossi, editor, Proc. of LOPSTR’99, pages 75–82. Tech. rept. CS-99-16, Univ. of Venice, Italy, 1999. Also see Technical Report 2003-008 at the Department of Information Technology, Uppsala University, Sweden, 2003. 36. P. Flener and S. Yılmaz. Inductive synthesis of recursive logic programs: Achievements and prospects. Journal of Logic Programming, 41(2–3):141–195, 1999.
64
David Basin et al.
37. P. Flener, H. Zidoum, and B. Hnich. Schema-guided synthesis of CLP programs. In Proc. of ASE’98, pages 168–176. IEEE Computer Society Press, 1998. 38. E. Gamma, R. Helm, R. Johnson, and J. Vlissides. Design Patterns: Elements of Reusable Object-Oriented Software. Addison-Wesley, 1995. 39. T.S. Gegg-Harrison. Extensible logic program schemata. In J. Gallagher, editor, Proc. of LOPSTR’96, volume 1207 of LNCS, pages 256–274. Springer-Verlag, 1997. 40. A.T. Goldberg. Knowledge-based programming: A survey of program design and construction techniques. IEEE Transactions on Software Engineering, 12(7):752– 768, 1986. 41. M.J. Gordon, R. Milner, and C.P. Wadsworth. Edinburgh LCF: A Mechanized Logic of Computation, volume 78 of Lecture Notes in Computer Science. SpringerVerlag, 1979. 42. C. Green. Application of theorem proving to problem solving. In Proc. of IJCAI’69, pages 219–239. Morgan Kaufmann, 1969. 43. A. Hamfelt and J. Fischer Nilsson. Inductive metalogic programming. In S. Wrobel, editor, Proc. of ILP’94, volume 237 of GMD-Studien, pages 85–96, 1994. 44. A. Hamfelt and J. Fischer Nilsson. Declarative logic programming with primitive recursive relations on lists. In M.J. Maher, editor, Proc. of JICSLP’96, pages 230–243. The MIT Press, 1996. 45. A. Hamfelt and J. Fischer Nilsson. Towards a logic programming methodology based on higher-order predicates. New Generation Computing, 15(4):421–448, 1997. 46. A. Hamfelt, J. Fischer Nilsson, and N. Oldager. Logic program synthesis as problem reduction using combining forms. Automated Software Engineering, 8(2):167–193, 2001. 47. P. Hill and J.W. Lloyd. The G¨ odel Programming Language. The MIT Press, 1994. 48. J.R. Hindley and J.P. Seldin. Introduction to Combinators and the λ-Calculus. Cambridge University Press, 1986. 49. D.J. Howe. On computational open-endedness in Martin-L¨ of’s type theory. In Proc. of LICS’91, pages 162–172. IEEE Computer Society Press, 1991. 50. I. Kraan, D. Basin, and A. Bundy. Logic program synthesis via proof planning. In K.-K. Lau and T. Clement, editors, Proc. of LOPSTR’92, Workshops in Computing Series, pages 1–14. Springer-Verlag, 1993. 51. I. Kraan, D. Basin, and A. Bundy. Middle-out reasoning for synthesis and induction. Journal of Automated Reasoning, 16(1–2):113–145, 1996. 52. K.-K. Lau and M. Ornaghi. On specification frameworks and deductive synthesis of logic programs. In L. Fribourg and F. Turini, editors, Proc. of LOPSTR’94 and META’94, volume 883 of LNCS, pages 104–121. Springer-Verlag, 1994. 53. K.-K. Lau, M. Ornaghi, and S.-˚ A. T¨ arnlund. Steadfast logic programs. Journal of Logic Programming, 38(3):259–294, 1999. 54. C.L. Lisett and D.E Rumelhart. Facial recognition using a neural network. In Proc. of the 11th International Florida AI Research Symposium FLAIRS-98, pages 328– 332, 1998. 55. M.J. Maher. Equivalences of logic programs. In J. Minker, editor, Foundations of Deductive Databases and Logic Programming. Morgan Kaufmann, 1987. 56. P. Martin-L¨ of. Constructive mathematics and computer programming. In Proc. of the Sixth International Congress for Logic, Methodology, and Philosophy of Science, pages 153–175. North-Holland, 1982. 57. S. Minton. Automatically configuring constraint satisfaction programs: A case study. Constraints, 1(1–2):7–43, 1996.
Synthesis of Programs in Computational Logic
65
58. S. Muggleton. Inverse entailment and Progol. New Generation Computing, 13(3– 4):245–286, 1995. 59. L.C. Paulson. Isabelle: A Generic Theorem Prover, volume 828 of LNCS. SpringerVerlag, 1994. 60. A. Pettorossi and M. Proietti. Transformation of logic programs. In D.M. Gabbay, C.J. Hogger, and J.A. Robinson, editors, Handbook of Logic in Artificial Intelligence and Logic Programming. Clarendon Press, 1998. 61. F. Pfenning. Logic programming in the LF logical framework. In Logical Frameworks, pages 149–181. Cambridge University Press, 1991. 62. G.D. Plotkin. A note on inductive generalization. In B. Meltzer and D. Michie, editors, Machine Intelligence 5, pages 153–163. Edinburgh University Press, 1970. 63. J.C. Reynolds. Transformational systems and the algebraic structure of atomic formulas. In B. Meltzer and D. Michie, editors, Machine Intelligence 5, pages 135–151. Edinburgh University Press, 1970. 64. D.R. Smith. The structure of divide and conquer algorithms. Technical Report 52-83-002, Naval Postgraduate School, Monterey, California, USA, 1983. 65. D.R. Smith. Top-down synthesis of divide-and-conquer algorithms. Artificial Intelligence, 27(1):43–96, 1985. 66. D.R. Smith. KIDS: A semiautomatic program development system. IEEE Transactions on Software Engineering, 16(9):1024–1043, 1990. 67. D.R. Smith. Toward a classification approach to design. In M. Wirsing and M. Nivat, editors, Proc. of AMAST’96, volume 1101 of LNCS, pages 62–84. SpringerVerlag, 1996. 68. Z. Somogyi, F. Henderson, and T. Conway. The execution algorithm of Mercury: An efficient purely declarative logic programming language. Journal of Logic Programming, 29(1–3):17–64, 1996. 69. A. van Lamsweerde. Formal specification: A roadmap. In A. Finkelstein, editor, The Future of Software Engineering, pages 147–159. ACM Press, 2000. 70. W.W. Vasconcelos and N.E. Fuchs. An opportunistic approach for logic program analysis and optimisation using enhanced schema-based transformations. In M. Proietti, editor, Proc. of LOPSTR’95, volume 1048 of LNCS, pages 174–188. Springer-Verlag, 1996. 71. G.A. Wiggins. Synthesis and transformation of logic programs in the Whelk proof development system. In K.R. Apt, editor, Proc. of JICSLP’92, pages 351–365. The MIT Press, 1992.
Developing Logic Programs from Specifications Using Stepwise Refinement Robert Colvin2 , Lindsay Groves2, Ian J. Hayes1 , David Hemer1 , Ray Nickson2 , and Paul Strooper1 1
School of Information Technology and Electrical Engineering, University of Queensland, Brisbane, Australia
[email protected],
[email protected],
[email protected] 2 School of Mathematical and Computing Sciences, Victoria University of Wellington, New Zealand
[email protected],
[email protected],
[email protected]
Abstract. In this paper we demonstrate a refinement calculus for logic programs, which is a framework for developing logic programs from specifications. The paper is written in a tutorial-style, using a running example to illustrate how the refinement calculus is used to develop logic programs. The paper also presents an overview of some of the advanced features of the calculus, including the introduction of higher-order procedures and the refinement of abstract data types.
1
Introduction
The aim of this paper is to present an overview of a refinement calculus for logic programs. The calculus provides a framework for the stepwise refinement of logic programs from specifications. As with other refinement calculi, such as the imperative refinement calculus of Back [2], we make use of a wide-spectrum programming language that includes both specification constructs and a subset that corresponds to executable code. This allows one to transform a specification to code within a single notational framework. The specification constructs include a specification command that allows the effect of a program to be specified in terms of a general predicate, and an assumption command that defines the range of values for which a program is expected to work. A semantics for the refinement calculus has been given which models commands (both specifications and code) as partial functions from sets of bindings of program variables to subsets of those bindings [12]. A tool has been developed to support the refinement calculus [15], based on the Isabelle/HOL theorem prover. To enhance the expressive power of the language, it has been augmented with both higher-order procedures [7] and a module mechanism with local (abstract) data types [6]. Higher-order procedures allow generic procedures to be written that apply a parameter procedure in a systematic manner. For example, the procedure map relates two (equal-length) lists of values by relating their corresponding elements according to a procedure given as a parameter to map. M. Bruynooghe and K.-K. Lau (Eds.): Program Development in CL, LNCS 3049, pp. 66–89, 2004. c Springer-Verlag Berlin Heidelberg 2004
Developing Logic Programs from Specifications Using Stepwise Refinement
67
Modules allow an (abstract) data type to be associated with a set of procedures for manipulating values of that type. Programs can then be developed using higher-level data types that may be specified in terms of mathematical structures, such as (multi-)sets and relations, that may not be directly available in the implementation language. By suitably restricting the structure of programs using such a module, the module may be replaced by a module of a similar structure that uses an implementable or a more efficient representation of the data type. We give an overview of the refinement calculus by presenting the refinement of a program, applyInst , that applies instantiations to a meta-expression to give an expression. The example is derived from an algorithm for adapting reusable library components in a software development system [13]. In Sect. 2 we introduce the wide-spectrum language. In Sect. 3 we give a specification of the applyInst procedure. In Sect. 4 we introduce the notion of refinement, present some refinement laws, and begin the refinement of applyInst . In Sect. 5 we further refine applyInst by introducing higher-order procedure calls. The refinement is completed in Sect. 6, where we replace an abstract specification type with an implementation type. Sect. 7 discusses aspects of the refinement calculus project that distinguish it from other logic program derivations schemes.
2
The Wide-Spectrum Language
In our wide-spectrum language we can write both specifications and executable programs. This has the benefit of allowing stepwise refinement within a single notational framework. A program in our language is a collection of parameterised procedures. Each procedure has a body which is a command whose only free variables are the parameters of the procedure. As well as commands that correspond to programming language constructs, the wide-spectrum language contains two commands that are not necessarily executable: the specification command, that constrains its free variables; and the assumption command, that can be used to define the context in which a command is required to work correctly. Commands may also be formed by using disjunction, parallel conjunction, sequential conjunction, existential and universal quantification, and recursion. A specification command is of the form P , where P is formula of predicate logic. The specification X = 1 may be understood as binding X to 1 in states where X is unbound; it succeeds in states where X is bound to 1, and fails if X is bound to something other that 1. In our semantics we model the meaning of specification command, and any command in general, as a function from sets of bindings to sets of bindings, where a binding maps variables to values [12]. A binding represents an answer, providing a (single) value for every free variable. A variable X is unbound in a set of bindings, or state, if X is mapped to every possible value by the bindings. Using this functional meaning for commands, the behaviour of a command S is to constrain the set of answers to only those that satisfy S ; alternatively, S eliminates those answers that do not satisfy S .
68
Robert Colvin et al.
The meaning function, e, of any command in our language satisfies the property e(s) ⊆ s for a set of bindings s; this models the constraining nature of logic programs (command execution cannot decrease “groundedness”). We provide further examples below. The following is a definition of the procedure foo, that constrains its parameter X to be either 0 or 1, and Y to be one greater than X . foo = (λ X , Y : N • (X = 0 ∨ X = 1) ∧ Y = X + 1) The name foo is defined (=) as a procedure whose parameters are natural numbers X and Y , and whose body is a specification command that constrains the possible values for X and Y . During the refinement process high-level procedure specifications are broken down into components that can be executed directly in an implementation language such as Prolog or Mercury [24]. This typically involves turning logical connectives in the specification into corresponding connectives in the programming language. For example, foo may be implemented as the procedure: (λ X , Y : N • (X = 0 ∨ X = 1), Y = X + 1)
(2.1)
Here we have replaced the logical connectives ‘∧’ and ‘∨’ by the corresponding program connectives ‘,’ and ‘∨’. We use the symbols ‘∧’ and ‘∨’ for the logical operators conjunction and disjunction as well as the program operators for parallel conjunction and disjunction, respectively. Similarly we use the symbols ‘∃’ and ‘∀’ for both logical and program quantification. This does not lead to confusion within programs because the logical operators and quantifications can only appear inside specification and assumption commands. A summary of the operators and quantifiers of the language is shown in Fig. 1. We use S and T to stand for commands and X to stand for program variables.
∨ ∧ , ∃ ∀
disjunction parallel conj. sequential conj. existential quant. universal quant.
Example S ∨T S ∧T S, T (∃ X : Z • S ) (∀ X : Z • S )
Fig. 1. Summary of operators in the wide-spectrum language
The meaning function of a disjunction S ∨ T constrains the set of answers to those that satisfy either S or T ; similarly, the meaning function of a conjunction S ∧ T restricts the set of answers to those that satisfy both S and T . For instance, X = 0 ∨ X = 1 constrains the set of answers to those that either bind X to 0 or 1. The program X = 0 ∧ X = 1 constrains the set of answers to those that bind X to both 0 and 1, i.e., the empty set. A command
Developing Logic Programs from Specifications Using Stepwise Refinement
69
that returns an empty set of answers acts as Prolog’s fail command - it is equivalent to false. The meaning function of a sequential conjunction (S , T ) is more interesting. It imposes an ordering on the execution; semantically, the answers satisfying S are passed as the input to the meaning function of T . Hence, T may assume that S is satisfied before it executes. For instance, in (2.1), the program connective ‘,’ ensures that X is bound before the equality involving Y . This ordering allows the final equality to be implemented using the ‘is’ built-in of Prolog. Our wide-spectrum language has an executable subset we refer to as code. A procedure is code if it has a straightforward translation into a logic programming language such as Prolog or Mercury [24]. This means the procedure uses the operators sequential and parallel conjunction, disjunction, and existential quantification, and may be recursive. Furthermore, the specification commands must contain predicates that have counterparts in the implementation language, e.g., equality, and procedure calls are only allowed on procedures that have also been refined to code. The procedure foo (2.1) satisfies these constraints - the corresponding Prolog syntax for the procedure foo is: foo(X, Y) :- (X = 0; X = 1), Y is X + 1. When defining procedures we often require some properties of its parameters. For instance, the following procedure member has an assumption command that its second parameter, L, is bound to a list of natural numbers; this is represented by the assumption command {L ∈ list(N)}. It also has a specification command that constrains its first parameter, E , to be an element in the range of (set of elements in) L, i.e., in our notation E ∈ ran(L). member = (λ E : N, L : list(N) • {L ∈ list(N)}, E ∈ ran(L))
(2.2)
We make no assumption about whether E is bound or unbound. If E is bound, the procedure checks whether E is an element of L, failing if it is not. If E is unbound, it becomes bound to each element of L. In this paper we use “bound” to refer to a variable for which we have an assumption (command) that it is bound to a value of its type. Semantically, the meaning function of an assumption {A} is a partial function that is defined for only those states that satisfy A. An assumption does not constrain the set of answers. Hence, the behaviour of the command {L ∈ list(N)} is undefined for states in which L is not bound to a list of natural numbers, and does not restrict the set of answers if L is bound appropriately. The worst possible program in our language is {false}, which we call abort. Its behaviour is undefined for any input: it may do anything, including not terminating, halting abnormally, failing (returning an empty answer set), or succeeding with arbitrary answers. As another example of the use of assumption commands, consider the following procedure: divide = (λ X , Y , Z : N • {X ∈ N ∧ Y ∈ N ∧ Y = 0}, Z = X div Y )
70
Robert Colvin et al.
This procedure assumes that the variables X and Y are bound to natural numbers, and that Y is non-zero, and establishes the relation that Z is the integer quotient of the division of X by Y . It is not required to do anything when Y is zero. Assumptions are often needed to justify refinement steps, for example to ensure that the primitive predicates will be executed correctly when translated into the implementation language. We may implement the specification of member (2.2) as a recursive procedure. A natural number E is a member of a nonempty list [H | T ] if either E = H or E is a member of T . µ mem • (λ E : N, L : list(N) • {L ∈ list(N)}, (∃ H : N, T : list(N) • L = [H | T ], (E = H ∨ mem(E , T )))) The notation (µ mem • body) defines mem to be the least fixed point solution for mem of the (recursive) equation mem = body. A least fixed point always exists for our recursive programs [9], though for non-terminating recursions the fixed point is the worst possible program, abort. The refinement rule for introducing recursion prevents us from deriving non-terminating recursions in our refinements [12].
3
An Example: Applying Instantiations
In many computing applications it is necessary to define a mechanism for systematically replacing occurrences of certain syntactic constructs by other constructs. For example, in macro languages such as TEX, M4, and the language defined by the C preprocessor, parameterised macros are replaced by structures in the object language. A similar process may also be used to obtain partial evaluations of logic (and other) programs: schematic variables are consistently instantiated by other expressions. In this section we present a specification of a program that performs such a replacement. This particular example is based on an algorithm used for adapting reusable library components in the CARE language [13]. In CARE, library components can be parameterised over metavariables. Components are used by instantiating their metavariables by expressions. The CARE tool includes an algorithm, based on higher-order pattern matching, for finding an instantiation that maps the metavariables occurring in library components (the source) to their corresponding object expressions. To simplify the presentation, we have chosen the easier task of applying a given instantiation to a component (source) to obtain the object. Expressions (the results of applying instantiations) are constructed from variables (with names taken from the given set VName) and functors (with names in FName) applied to lists of expressions. Constants are viewed as nullary functors. E ∈ Expr ⇔ (∃ X : VName • E = var (X )) ∨ (∃ F : FName, L : list(Expr ) • E = fn(F , L))
(3.1)
Developing Logic Programs from Specifications Using Stepwise Refinement
71
Meta-expressions are a generalization of expressions which may contain metavariables applied to some parameters; we call such applications schemas. Metaexpressions are transformed by instantiating their metavariables to give an expression. The constructors var and fn are as for expressions (except that the arguments of a function are themselves meta-expressions); in addition, there is a constructor for schemas, whose names are drawn from the given set MVar . M ∈ MetaExpr ⇔ (∃ X : VName • M = var (X )) ∨ (∃ F : FName, L : list(MetaExpr ) • M = fn(F , L)) ∨ (∃ V : MVar , L : list(MetaExpr ) • M = schema(V , L))
(3.2)
Instantiations map occurrences of metavariables to patterns. Patterns may contain place holders of the form ph(i), where i is a natural number. Place holders give the position of the corresponding parameter in the schema arguments. Patterns are another generalization of expressions: as well as variables and functors, patterns may contain these placeholders. P ∈ Pattern ⇔ (∃ X : VName • P = var (X )) ∨ (∃ F : FName, L : list(Pattern) • P = fn(F , L)) ∨
(3.3)
(∃ N : N • P = ph(N )) Instantiations are thus partial functions (→) from metavariables to patterns: Inst = MVar → Pattern For example, let f be a binary function, p and q be metavariables, and g and h be nullary functions. Consider an instantiation I that maps p to g and q to h. I (p) = fn(g, [ ]) I (q) = fn(h, [ ]) Applying I to the meta-expression f (p, q) results in the expression f (g, h). For readability purposes we use conventional notation to write (meta-)expressions, though formally the meta-expression f (p, q) and the expression f (g, h) are represented by fn(f , [schema(p, [ ]), schema(q, [ ])]) and fn(f , [fn(g, [ ]), fn(h, [ ])]), respectively. Instantiations may also map metavariables to patterns involving placeholders. For example, the instantiation I below defines a metavariable p that accepts two parameters, denoted by place holders ph(1) and ph(2), and yields an expression which might be interpreted as the difference between the second and double the first: I (p) = fn( − , [ph(2), fn( ∗ , [ph(1), fn(2, [ ])])])
72
Robert Colvin et al.
Applying I to the meta-expression p(a, b) results in the expression b −a ∗2. The expanded representation of the two expressions are schema(p, [fn(a, [ ]), fn(b, [ ])]) and fn( − , [fn(b, [ ]), fn( × , [fn(a, [ ]), fn(2, [ ])])]), respectively. Elements of Inst are partial as they need not have a mapping for every metavariable. The range of Inst is restricted to patterns, which themselves contain no schemas, therefore we only need to consider one level of instantiation application. We now give three properties of a relation applyInst for applying an instantiation to a meta-expression. For all I ∈ Inst and Q ∈ Expr : (∀ X : VName • applyInst(I , var (X ), var (X ))) (∀ F : FName, L : list(MetaExpr ) • applyInst(I , fn(F , L), Q ) ⇔
(3.4)
(∃ L : list(Expr ) • #L = #L ∧ (∀ i : 1..#L • applyInst(I , L(i), L (i))) ∧ Q = fn(F , L ))) (∀ V : MVar , L : list(MetaExpr ) • applyInst(I , schema(V , L), Q ) ⇔
(3.5)
(∃ L : list(Expr ) • #L = #L ∧ (∀ i : 1..#L • applyInst(I , L(i), L (i))) ∧ V ∈ dom(I ) ∧ substph(L , I (V ), Q )))
(3.6)
Property (3.4) states that applying an instantiation to a variable has no effect. Property (3.5) states that the result of applying I to fn(F , L) is fn(F , L ), where the length of L, #L, is the same as the length of L , and L is the result of applying I to each element of L. To determine the result of applying I to schema(V , L), property (3.6), we again construct the list L which is the result of applying I to the elements of L. We extract the definition of V from I , I (V ), and use the relation substph to substitute place holders in I (V ) with expressions from the parameters list L . The result of this, Q , is the instantiation of schema(V , L) via I . Note that if V is not in the domain of I , applyInst(I , schema(V , L), Q ) is false. We define the result of substituting place holders with corresponding values from a list of expressions by introducing a relation substph. It is defined by the following three properties, one for each of the three forms of patterns. For all Params ∈ list(Expr ) and Out ∈ Expr : (∀ X : VName • substph(Params, var (X ), var (X )))
(3.7)
(∀ N : N • substph(Params, ph(N ), Out ) ⇔ N ∈ 1..#Params ∧ Out = Params(N ))
(3.8)
(∀ F : FName, L : list(Pattern) • substph(Params, fn(F , L), Out ) ⇔ (∃ L : list(Expr ) • #L = #L ∧ (∀ i : 1..#L • substph(Params, L(i), L (i))) ∧ Out = fn(F , L )))
(3.9)
Developing Logic Programs from Specifications Using Stepwise Refinement
73
Property (3.7) states that substituting place holders has no effect on a variable. Property (3.8) replaces a place holder ph(N ) with the N th element from the list Params, provided N is a valid index into Params. If the input is a functor (3.9), we recursively apply substph to each of its parameters to obtain a value for Out . We now specify our top-level program, applyInst , in terms of the relation applyInst(I , M , Q ). applyInst = (λ I : Inst , M : MetaExpr , Q : Expr • {I ∈ Inst ∧ M ∈ MetaExpr }, applyInst(I , M , Q )) Since we are applying an instantiation to a meta-expression, we make the assumption that the instantiation I and the meta-expression M are already bound to values of the appropriate types. Any program that calls applyInst must ensure that the assumption is satisfied. We refine applyInst in subsequent sections.
4
Refinement
Specifications are transformed into code via a sequence of correctness-preserving steps; this process is known as refinement. We say a command S is refined by a command T , written S T , if T terminates normally for all inputs for which S terminates normally (with respect to its assumptions) and T computes the same set of answers as S whenever S terminates. Each step in a refinement is justified by the use of a refinement law, which has been proved correct with respect to the underlying semantics. Below we present some refinement laws, and then illustrate their use by beginning the refinement of the procedure applyInst from Sect. 3. 4.1
Refinement Laws
We present a selection of refinement laws below. Where a law is divided into two parts by a horizontal line, the part above the line is the proof obligation that must be satisfied for the refinement below the line to be valid. A predicate equivalence, P ≡ Q , states that P and Q are equivalent for all possible values of their free variables. Similarly, P Q states that P implies Q for all possible values of their free variables. The symbols ⇔ and ⇒ are the usual equivalence and implication of predicates, which may or may not be true for given values of their free variables. We use A, P and Q for predicates, and S and T for commands. Law 1 Weaken assumption P Q {P } {Q }
Law 2 Equivalent specifications P ≡Q P Q
We can refine an assumption command by transforming its predicate under logical implication using Law 1. We can refine a specification command by trans-
74
Robert Colvin et al.
forming its predicate under logical equivalence using Law 2. These laws correspond to weakening assumptions and maintaining the effect on free variables, respectively. Law 3 Assumption in context
Law 4 Propagate assumption
A (P ⇔ Q ) {A}, P {A}, Q
P , S P , ({P }, S )
Law 3 generalises Law 2, in that we may make use of the assumption predicate A in proving the equivalence of predicates P and Q . Assumptions may be propagated through sequential conjunction using Law 4. We use this law to pass contextual information around a program. Law 5 Parallel to sequential
Law 6 Lift disjunction
S ∧ T S, T
P ∨ Q P ∨ Q
A parallel conjunction can be refined to a sequential conjunction using Law 5. The second component of a sequential conjunction, T , may assume properties established by the first component, S , using Law 4. Law 6 allows a predicate disjunction inside a specification command to be lifted to its corresponding widespectrum program operator. Similar laws hold for parallel conjunction and the quantifiers. Law 7 Monotonicity of parallel conjunction S S ∧ T T S ∧ T S ∧ T Monotonicity laws state that the result of replacing a component of a program by its refinement refines the entire program. In this case, if S refines S and T refines T then the parallel conjunction S ∧ T refines S ∧ T . Monotonicity holds for all the operators and both quantifiers in the wide-spectrum language. We use monotonicity laws implicitly in refinements. 4.2
Example: Initial Steps
In this section we begin the refinement of the procedure applyInst from Sect. 3. The initial stages of the refinement presented below follow the structure of applyInst. However, care needs to be exercised when introducing recursion to ensure the resulting procedures terminate. Some parts of the refinement require additional techniques which are introduced in later sections. We begin with the specification as given in Sect. 3: applyInst = (λ I : Inst , M : MetaExpr , Q : Expr • {I ∈ Inst ∧ M ∈ MetaExpr }, applyInst(I , M , Q ))
(4.1)
Developing Logic Programs from Specifications Using Stepwise Refinement
75
Since the definition of applyInst is recursive, we develop a recursive implementation of applyInst , using the principle of well-founded induction. Let S (X ) be a specification involving a parameter X of type σ, ≺ be a well-founded order on σ, and id be a fresh name. As is usual for a recursive procedure with parameter X , when developing the code for the procedure we may assume that the procedure satisfies its specification for values smaller than X . That is, we assume the inductive hypothesis S (Y ) id (Y ) for all Y ≺ X when refining S (X ). If under that assumption we can refine S (X ) to P , then S µ id • (λ X : σ • P ). For the applyInst example, the parameter X is the triple (I , M , Q ), whose type σ is Inst × MetaExpr × Expr . The well-founded ordering (I , M , Q ) ≺ (I , M , Q ) is satisfied when M is a subexpression of M . Finally, we choose the name apply as our id . The inductive hypothesis is that for all I : Inst , M : MetaExpr , Q : Expr : {I ∈ Inst ∧ M ∈ MetaExpr ∧ M ≺ M }, applyInst(I , M , Q ) apply(I , M , Q )
(4.2)
We can use the inductive hypothesis to introduce recursive calls to apply within procedure applyInst . We will then have refined applyInst to the recursive procedure µ apply • (λ I : Inst , M : MetaExpr , Q : Expr • . . . apply(. . .) . . .). We begin the refinement of the body applyInst (4.1). Initially our goal is to manipulate the body so that recursive calls may be introduced using (4.2). Using Law 3 (assumption in context ) with the assumption M ∈ MetaExpr allows us to refine applyInst(I , M , Q ) to the following. M ∈ MetaExpr ∧ applyInst(I , M , Q ) The following proof obligation was required to apply Law 3: I ∈ Inst ∧ M ∈ MetaExpr (applyInst(I , M , Q ) ⇔ (M ∈ MetaExpr ∧ applyInst(I , M , Q ))) Continuing with the refinement, we expand the predicate M ∈ MetaExpr using (3.2), and distribute the resulting disjunction over applyInst(I , M , Q ) using Law 2 (equivalent specifications). We lift the resulting disjuncts using Law 6 (lift disjunction) and expand the scope of the quantifications and then lift them. In addition, we lift the conjunctions and refine them by sequential conjunctions using Law 5 (parallel to sequential ). The body of applyInst is now: (∃ X : VName • M = var (X ), applyInst(I , M , Q )) ∨
(4.3)
(∃ F : FName, L : list(MetaExpr ) • M = fn(F , L), applyInst(I , M , Q )) ∨
(4.4)
(∃ V : MVar , L : list(MetaExpr ) • M = schema(V , L), applyInst(I , M , Q ))
(4.5)
76
Robert Colvin et al.
Note that for each of the three branches, the structural form of M , established by the first specification command in each branch, e.g., M = var (X ), can be assumed when refining the second specification command, applyInst(I , M , Q ). We now refine each branch in turn. The first branch (4.3), where M is a variable, may be refined by using Law 2 (equivalent specifications) with (3.4) on the second conjunct. The resulting code is: (∃ X : VName • M = var (X ), Q = var (X )) The second branch (4.4), where M is a function application, may be refined using (3.5). We lift the resulting conjunctions and quantifiers, giving: (∃ F : FName, L : list(MetaExpr ) • M = fn(F , L), (∃ L : list(Expr ) • #L = #L ∧ (∀ i : 1..#L • applyInst(I , L(i), L (i))) ∧ Q = fn(F , L ))) Noting the presence of the specification command applyInst(I , L(i), L (i)), we can introduce a recursive call using (4.2) provided we can establish the assumption {I ∈ Inst ∧ L(i) ∈ MetaExpr ∧ L(i) ≺ M }. We note that we are in a context in which I and M are assumed to be bound variables of type Inst and MetaExpr respectively. Since M is bound, and M = fn(F , L) is established earlier in a sequential conjunction, it follows that L must be bound also, and therefore each element of L is bound. Furthermore, L(i) ≺ M holds since L(i) is a subexpression of L, which is a subexpression of M . We introduce the assumption {I ∈ Inst ∧ L(i) ∈ MetaExpr ∧ L(i) ≺ M } and propagate it into the second branch to syntactically match (part of) our program with the left-hand side of the refinement in the inductive hypothesis (4.2). (∃ F : FName, L : list(MetaExpr ) • M = fn(F , L), (∃ L : list(Expr ) • #L = #L ∧ (∀ i : 1..#L • {I ∈ Inst ∧ L(i) ∈ MetaExpr ∧ L(i) ≺ M }, applyInst(I , L(i), L (i))) ∧ Q = fn(F , L ))) We can now refine lines four and five to a recursive call, using (4.2). (∃ F : FName, L : list(MetaExpr ) • M = fn(F , L), (∃ L : list(Expr ) • #L = #L ∧ (∀ i : 1..#L • apply(I , L(i), L (i))) ∧ Q = fn(F , L ))) The universal quantification will be eliminated in Sect. 5.1.
(4.6)
Developing Logic Programs from Specifications Using Stepwise Refinement
77
The third branch (4.5), where M is a schema, may be refined using (3.6). We lift the resulting conjunctions and quantifiers, giving: (∃ V : MVar , L : list(MetaExpr ) • M = schema(V , L), (∃ L : list(Expr ) • #L = #L ∧ (∀ i : 1..#L • applyInst(I , L(i), L (i))) ∧ V ∈ dom(I ) ∧ substph(L , I (V ), Q ))) We again introduce a recursive call within the universal quantification, after introducing the assumption {I ∈ Inst ∧ L(i) ∈ MetaExpr ∧ L(i) ≺ M } as in the refinement of (4.4). (∃ V : MVar , L : list(MetaExpr ) • M = schema(V , L), (∃ L : list(Expr ) • #L = #L ∧ (∀ i : 1..#L • apply(I , L(i), L (i))) ∧ V ∈ dom(I ) ∧ substph(L , I (V ), Q )))
(4.7)
The universal quantification will be eliminated in Sect. 5.1. The last line includes the expression I (V ), which is not directly executable. We show how to develop code for this situation in Sect. 6. First, in Sect. 4.3, we refine the last line to a call on a procedure that implements the relation substph.
4.3
Example: Substituting Parameters for Place Holders
We define a procedure that implements the relation substph under the assumption that its first two parameters are bound. Any program that calls substph, such as applyInst , must ensure the assumptions are satisfied. substph = (λ Params : list(Expr ), In : Pattern, Out : Expr • {Params ∈ list(Expr ) ∧ In ∈ Pattern}, substph(Params, In, Out )) To implement the specification command substph(L , I (V ), Q ) from (4.7) as a procedure call substph(L , I (V ), Q ), we must establish the assumption {L ∈ list(Expr ) ∧ I (V ) ∈ Pattern}. L ∈ list(Expr ) follows from the recursive calls apply(I , L(i), L (i)). We refine the parallel conjunction involving apply(I , L(i), L (i)) to sequential conjunction (Law 5), and then use Law 4 to establish L ∈ list(Expr ) as an assumption before substph(L , I (V ), Q ). I (V ) ∈ Pattern follows from I ∈ Inst and V ∈ dom(I ), therefore we similarly refine the parallel conjunction involving V ∈ dom(I ) to sequential conjunction and propagate the assumption I (V ) ∈ Pattern. The code for applyInst so far is:
78
Robert Colvin et al.
applyInst µ apply • (λ I : Inst , M : MetaExpr , Q : Expr • {I ∈ Inst ∧ M ∈ MetaExpr }, (∃ X : VName • M = var (X ), Q = var (X )) ∨ (∃ F : FName, L : list(MetaExpr ) • M = fn(F , L), (∃ L : list(Expr ) • #L = #L ∧ (∀ i : 1..#L • apply(I , L(i), L (i))) ∧ Q = fn(F , L ))) ∨ (∃ V : MVar , L : list(MetaExpr ) • M = schema(V , L), (∃ L : list(Expr ) • #L = #L ∧ (∀ i : 1..#L • apply(I , L(i), L (i))), V ∈ dom(I ), substph(L , I (V ), Q )))) We refine the body of substph following a similar pattern to that of applyInst . We introduce a case analysis on the type of In, apply the properties (3.7), (3.8) and (3.9) as appropriate, and lift the predicate operators to their wide-spectrum counterparts. As with applyInst , we refine the conjunctions occurring in the pattern In = . . . ∧ . . . by sequential conjunctions. This allows us to satisfy assumptions for the recursive calls that are introduced as part of the refinement. (∃ X : VName • In = var (X ), Out = var (X )) ∨ (∃ N : N • In = ph(N ), N ∈ 1..#Params ∧ Out = Params(N )) ∨ (∃ F : FName, L : list(Pattern) • In = fn(F , L), (∃ L : list(Expr ) • #L = #L ∧ (∀ i : 1..#L • substph(Params, L(i), L (i))) ∧ Out = fn(F , L ))) The first disjunct is already code. The second disjunct involves an array-like access of a list. We may refine this to a call on a recursive procedure that traverses the list and returns the N th element, or fails if N is not a valid index. For brevity we omit the refinement and assume procedure elemi(L, I , E ), that implements I ∈ 1..#L ∧ E = L(I ) exists in our target implementation language. The refinements of similar list processing procedures are presented in [9]. In the third disjunct we introduce a recursive call in a similar manner as for applyInst . The universal quantification is eliminated in Sect. 5.1. Collecting the refinement of substph gives: substph µ sub • (λ Params : list(Expr ), In : Pattern, Out : Expr • {Params ∈ list(Expr ) ∧ In ∈ Pattern}, (∃ X : VName • In = var (X ), Out = var (X )) ∨ (∃ N : N • In = ph(N ), elemi(Params, N , Out )) ∨ (∃ F : FName, L : list(Pattern) • In = fn(F , L), (∃ L : list(Expr ) • #L = #L ∧ (∀ i : 1..#L • sub(Params, L(i), L (i))) ∧ Out = fn(F , L )))) To refine this program to code, we eliminate the universal quantifications in Sect. 5 and refine the last line of applyInst in Sect. 6.5.
Developing Logic Programs from Specifications Using Stepwise Refinement
5
79
Higher-Order Procedures
In this section we continue the refinement of the applyInst example to illustrate the use of higher-order procedures. A higher-order procedure is one that takes a procedure as a parameter. For example, consider the following specification of the standard higher-order procedure map, which applies a procedure P to all the elements in a list L, returning the list L . map = λ P : σ → τ → Cmd , L : list(σ), L : list(τ ) • {L ∈ list(σ)}, #L = #L ∧ (∀ i : 1..#L • P (L(i), L (i)))
(5.1)
The higher-order parameter, P , is a procedure that takes two parameters, of (generic) types σ and τ , respectively, and provides a command (type Cmd ) that defines the relation between these parameters. The map procedure then relates two equal length lists, L and L , provided every element of L is related to the corresponding element of L by P . In [7] we show how map may be refined to recursive code. From the definition of map we may deduce the following refinement law. Law 8 Introduce map. For all L and L of type list (σ) and list (τ ), respectively, and all procedures P that take two parameters of type σ and τ , {L ∈ list(σ)}, #L = #L ∧ (∀ i : 1..#L • P (L(i), L (i))) 5.1
map(P , L, L )
Example: Introducing map
Recall the second case of applyInst , where the input pattern is a functor (4.6): (∃ F : FName, L : list(MetaExpr ) • M = fn(F , L), (∃ L : list(Expr ) • #L = #L ∧ (∀ i : 1..#L • apply(I , L(i), L (i))) ∧ Q = fn(F , L ))) Note that the second and third lines almost match the definition of map (5.1). From the assumption M ∈ MetaExpr in applyInst we can introduce the assumption L ∈ list(MetaExpr ), which implies L is bound. (∃ F : FName, L : list(MetaExpr ) • M = fn(F , L), (∃ L : list(Expr ) • {L ∈ list(MetaExpr )}, #L = #L ∧ (∀ i : 1..#L • apply(I , L(i), L (i))) ∧ Q = fn(F , L ))) The third and fourth lines now match (5.1), except that apply takes three parameters instead of the two expected by map. To match fully with the definition of map, we use a partial application of apply, apply(I ). In our language all
80
Robert Colvin et al.
procedures are curried (though for brevity of presentation we have not shown them as such). Hence, apply(I ) is a function which takes two parameters, as required by the signature of map, and we may write apply(I , L(i), L (i)) as apply(I )(L(i), L (i)). We use Law 8 (introduce map) with apply(I ) as the first parameter to map, giving: (∃ F : FName, L : list(MetaExpr ) • M = fn(F , L), (∃ L : list(Expr ) • map(apply(I ), L, L ) ∧ Q = fn(F , L ))) The procedure apply with instantiation I is applied to each element of L, resulting in the list L . Given a target language that implements a map function and supports partial application of procedure calls, such as Mercury [24], the above command can be translated to executable code. Using similar refinements to those above, we may replace the universal quantifications appearing elsewhere in applyInst and substph by calls to map. Collecting the refinement of applyInst and substph gives: applyInst µ apply • (λ I : Inst , M : MetaExpr , Q : Expr • {I ∈ Inst ∧ M ∈ MetaExpr }, (∃ X : VName • M = var (X ), Q = var (X )) ∨ (∃ F : FName, L : list(MetaExpr ) • M = fn(F , L), (∃ L : list(Expr ) • map(apply(I ), L, L ) ∧ Q = fn(F , L ))) ∨ (∃ V : MVar , L : list(MetaExpr ) • M = schema(V , L), (∃ L : list(Expr ) • map(apply(I ), L, L ), V ∈ dom(I ), substph(L , I (V ), Q )))) substph µ sub • (λ Params : list(Expr ), In : Pattern, Out : Expr • {Params ∈ list(Expr ) ∧ In ∈ Pattern}, (∃ X : VName • In = var (V ), Out = var (V )) ∨ (∃ N : N • In = ph(N ), elemi(Params, N , Out )) ∨ (∃ F : FName, L : list(Pattern) • In = fn(F , L), (∃ L : list(Expr ) • map(sub(Params), L, L ) ∧ Out = fn(F , L )))) Only the last line of applyInst is not code; we present the refinement of this line in Sect. 6.5.
6
Modular Logic Program Refinement
In this section we outline a technique for module data refinement [6], where a program is refined by changing the type of some of its variables. We assume a type and operations on that type are encapsulated in a module. By making some assumptions about the way in which such modules are used, we can develop efficient implementations of abstract modules. We use module refinement to complete the refinement of the applyInst procedure.
Developing Logic Programs from Specifications Using Stepwise Refinement
6.1
81
Modules
A module is a collection of procedures that operate on values of a given data type. We refer to variables of the given type as opaque. For instance, consider the module AbstractInst that operates on values of the abstract partial function type for Inst . Module AbstractInst Type Inst = MVar → Expr init = (λ I : Inst • I = ∅) lookup = (λ I : Inst , K : MVar , V : Expr • {I ∈ Inst ∧ K ∈ MVar }, (K , V ) ∈ I ) update = (λ I : Inst , K : MVar , V : Expr , I : Inst • . . .) End The procedure init establishes I as the empty function, ∅, while lookup establishes V as the value of I (K ), or fails if K is not in the domain of I . The procedure update, the details of which we omit for brevity, may be used to construct a nonempty value I of type Inst . The AbstractInst module could be generalised to implement a partial function with any types for the domain and range; for simplicity we use the above instance where the partial function is from MVar to Expr . For encapsulation purposes, a program that uses the abstract Inst type should make use of that type only through the procedures of the AbstractInst module. A program that uses AbstractInst must also respect its intended modes, which can be determined by looking at the assumptions for each procedure. If a parameter is assumed to be of the opaque type, that parameter is called an input to the procedure; if there is no such type assumption the parameter is called an output. Since partial functions are not directly implemented in most languages, we would like to replace all the references to the abstract module with references to a concrete module that faithfully implements the abstract procedures using an implementation language data type. 6.2
Module Refinement
In general we say a module M is module-refined by module M under the following condition: all programs P are refined by replacing calls to the procedures of the module M by calls to the corresponding procedures in the module M . While this definition is the most general, by restricting the class of programs P for which the module refinement must hold we can simplify some of the reasoning. Furthermore, by assuming that calls to a module occur in a certain order (imposed by sequential conjunction), we can allow efficient representations to be used that would not be possible in the more general case. Consider the following program that uses the procedures from AbstractInst : (∃ I : Inst • init (I ), . . . , (∃ I : Inst • update(I , X , Y , I ), . . . , lookup(I , K , V )))
82
Robert Colvin et al.
There is a strict order on the calls to init , update, and lookup, though within the . . . there may be arbitrary commands that do not use the module or variables of its type. Suppose we have a module ConcreteInst that is a module refinement of AbstractInst , providing procedures init + , lookup + and update + using an implementable type, Inst + , to represent the instantiation. Since ConcreteInst is a module refinement of AbstractInst , we may refine the above program to (∃ I + : Inst + • init + (I + ), . . . , (∃ I + : Inst + • update + (I + , X , Y , I + ), . . . , lookup + (I + , K , V )) Note, however, that it is not the case that init init + . Indeed, because they operate on different types (Inst and Inst + ), one could not possibly refine the other since they provide different sets of answers for their parameters. To prove that a module M is refined by a module M we use a coupling invariant, which is a relation between variables of the abstract and concrete type. Each pair of corresponding procedures from the modules are checked against the conditions for module refinement, given in [6], using the particular coupling invariant chosen. However, in many situations it is possible to automatically calculate a concrete module, given an abstract module and a coupling invariant. Using the calculation process to derive the concrete module guarantees that the conditions for module refinement will be met. In Sect. 6.3 we refine applyInst to use a call on lookup from the AbstractInst module. In Sect. 6.4 we introduce module calculation, and in Sect. 6.5 we show how it may be applied to the lookup procedure. 6.3
Example: Introducing lookup
In the third case of applyInst , where M is a schema, we need a refinement of the command: V ∈ dom(I ), substph(L , I (V ), Q ) This is the only part of the applyInst program that makes direct use of the instantiation I . However we are not able to refine this directly to code because the expression I (V ) is not directly implementable in most logic programming languages. Below we refine the above program fragment to make use of the procedure lookup from the AbstractInst module. We separate I (V ) from the use of its value (sometimes called flattening). We introduce an existential variable FDefn that has the value I (V ), using Law 2 (equivalent specifications). (∃ FDefn : Expr • V ∈ dom(I ) ∧ FDefn = I (V )), substph(L , I (V ), Q ) Treating the abstract partial function representation of an instantiation, I , as a set of pairs, we rewrite V ∈ dom(I ) ∧ FDefn = I (V ) as (V , FDefn) ∈ I . Now we lift the existential quantifier, expand its scope to encompass the call to substph, and replace I (V ) with FDefn in the call to substph. (∃ FDefn : Expr • (V , FDefn) ∈ I , substph(L , FDefn, Q ))
Developing Logic Programs from Specifications Using Stepwise Refinement
83
The command (V , FDefn) ∈ I is refined by a call to lookup, since I ∈ Inst and V ∈ MVar are guaranteed by the context. (∃ FDefn : Expr • lookup(I , V , FDefn), substph(L , FDefn, Q ))
(6.1)
Thus the only non-trivial reference to the instantiation I in the applyInst procedure occurs in a call on module AbstractInst . 6.4
Module Calculation
A technique for deriving, or calculating, a concrete module from an abstract module has been developed [9]. Consider an abstract procedure of the form (λ I : σ, V : τ • {A}, P ), having no opaque output parameters, and in which V is not of the opaque type (V is referred to as a regular parameter). Given a coupling invariant CI (I , L) relating a variable I of the abstract type σ with a variable L of the concrete type σ + , we may calculate the corresponding concrete procedure as: (λ L : σ + , V : τ • {(∃ I : σ • CI (I , L) ∧ A)},
(6.2)
(∀ I : σ • CI (I , L) ∧ A ⇒ P )) The assumption may be understood as a constraint on L that there exists some abstract instantiation I which satisfies the abstract assumption A and to which L is related via the coupling invariant. Similarly, the specification command can be understood as specifying that, for all abstract instantiations I related to L and satisfying the assumption A, the abstract specification P must hold. Once a procedure has been calculated in the above form, the developer then simplifies the assumption and specification to eliminate references to the abstract type σ. In many cases, depending on the form of the coupling invariant, this can be done via applications of the one-point laws. In the next section we use the above result to calculate the concrete procedure for lookup. The calculation technique may also be applied to abstract procedures with opaque output parameters, but for brevity we do not present the general form of the corresponding concrete procedure here (see [9] for details). 6.5
Example: Calculation
We can calculate the corresponding concrete procedure for lookup after choosing an appropriate concrete representation and relating it to the abstract type via a coupling invariant. We choose to concretely represent the partial function by a list whose elements are pairs of MVar s and Expr s. We relate a variable I of the abstract (partial function) type with a variable L of the concrete (list) type using the coupling invariant I = ran(L). This coupling invariant states that the abstract instantiation I contains all of the pairs in the list L. The relationship is straightforward since a partial function can be thought of as a set of pairs,
84
Robert Colvin et al.
the first elements of which form the domain of the function, with the second elements being the corresponding values for the members of the domain. Hence, the range of the list [(x , g), (y, h)] forms the set {(x , g), (y, h)}, which is a partial function which maps x to g and y to h. Using the general form of (6.2) with the coupling invariant I = ran(L), noting that the type σ is Inst and K and V are regular variables, generates the concrete procedure lookup + : lookup + = (λ L : list(MVar × Expr ), K : MVar , V : Expr • {(∃ I : Inst • I = ran(L) ∧ I ∈ Inst ∧ K ∈ MVar )}, (∀ I : Inst • I = ran(L) ∧ I ∈ Inst ∧ K ∈ MVar ⇒ (K , V ) ∈ I )) This is a valid module refinement of lookup using a list of pairs to represent the abstract partial function. However, it is rather complex and not directly executable at this stage, since it still uses the abstract type (though such references are scoped by quantifications). We refine the procedure body to code. The assumption and specification commands may be simplified using the one-point rules for existential and universal quantification, respectively, and the resulting redundant antecedent in the specification command may be removed using Law 3 (assumption in context ), giving: {ran(L) ∈ Inst ∧ K ∈ MVar }, (K , V ) ∈ ran(L) We simplify the assumption using Law 1 (weaken assumption) since ran(L) ∈ Inst L ∈ list(MVar × Expr ) However we must still refine the specification (K , V ) ∈ ran(L) to code. This is a membership test in the list L. We omit the details of the refinement for brevity, and assume that our target implementation language has an appropriate procedure member , similar to that presented in Sect. 2. After applying the calculation technique to the init and update procedures (each of which contains output parameters, and therefore require slightly different calculations to that of lookup [9]), we have the full concrete module. We use Inst + as the name of the concrete type list(MVar × Expr ). Module ConcreteInst Type Inst + = list(MVar × Expr ) + (λ L : Inst + • L = [ ]) init = lookup + = (λ L : Inst + , K : MVar , V : Expr • {L ∈ Inst + ∧ K ∈ MVar }, member ((K , V ), L)) ... update + = End Since we have followed the calculation process, we may refine a program that uses AbstractInst – provided the program satisfies the structural restrictions
Developing Logic Programs from Specifications Using Stepwise Refinement
85
discussed in Sect. 6.2 – by replacing each of its calls to procedures of module AbstractInst with calls to the corresponding procedures of module ConcreteInst. Collecting the refinements from each section gives us the complete program. It uses the type Inst + and procedure lookup + from the ConcreteInst module. We use the the symbol D to indicate that the refinement of applyInst is a data refinement, since we data refined the original instantiation type (partial function) to a list of pairs. applyInst D µ apply • (λ I : Inst + , M : MetaExpr , Q : Expr • {I ∈ Inst + ∧ M ∈ MetaExpr }, (∃ X : VName • M = var (X ), Q = var (X )) ∨ (∃ F : FName, L : list(MetaExpr ) • M = fn(F , L), (∃ L : list(Expr ) • map(apply(I ), L, L ) ∧ Q = fn(F , L ))) ∨ (∃ V : MVar , L : list(MetaExpr ) • M = schema(V , L), (∃ L : list(Expr ) • map(apply(I ), L, L ), (∃ FDefn : Expr • lookup + (I , V , FDefn), substph(L , FDefn, Q ))))) substph µ sub • (λ Params : list(Expr ), In : Pattern, Out : Expr • {Params ∈ list(Expr ) ∧ In ∈ Pattern}, (∃ X : VName • In = var (V ), Out = var (V )) ∨ (∃ N : N • In = ph(N ), elemi(Params, N , Out )) ∨ (∃ F : FName, L : list(Pattern) • In = fn(F , L), (∃ L : list(Expr ) • map(sub(Params), L, L ) ∧ Out = fn(F , L ))))
7
Conclusions
In this paper we have presented a refinement calculus for logic programming, and illustrated how it can be used by developing a small, but non-trivial, logic program from its specification. Our refinement calculus is similar in style to deductive logic program synthesis (surveys of which can be found in [3, 11]). At the most fundamental level, logic program development is the manipulation of predicates from general logic to a subset that corresponds to code, and developing a logic program in either the refinement calculus or synthesis style will require similar manipulation. We compare our approach to other logic program development schemes in the next section. A distinguishing feature of the refinement calculus approach is its rich specification language. In particular, a program (fragment) has an associated assumption component, similar to the precondition component of a program specification in an imperative programming formalism. This allows one to partially specify procedures, in the sense that their operation is not defined if the assumptions do not hold. We make use of this when developing recursive procedures by requiring that recursive calls satisfy an assumption that their arguments are bound to values that are strictly less than those of the enclosing call according to some well-founded relation. For instance, if the member procedure given in
86
Robert Colvin et al.
Sect. 2 is passed an unbound parameter for formal parameter L, then the tail of L, T , will also be unbound. This will result in infinite recursion. In the refinement calculus framework, the conditions for introducing recursion require a well-founded ordering to be maintained. To satisfy this condition, the recursive parameter must be bound. In this paper we have introduced recursion somewhat informally, but a more formal approach based the use of a refinement law for introducing recursion may be found in [12]. The translation to actual logic program code is not as straightforward as for an imperative language. The translation is not just a matter of turning conjunctions into commas and using defined language primitives – the order of conjuncts in a procedure goal must also be considered. At the logic level, conjunction is commutative, and therefore does not provide any guide to ordering its conjuncts. Knowledge of the execution mechanism of the implementation language is required to correctly order the conjuncts in a goal. In the calculus framework, assumptions and sequential conjunction partially bridge this gap. In Sect. 4.3 we saw that parallel conjunctions needed to be refined to sequential conjunctions so that the assumptions of the second operand of the conjunction (which was refined to a procedure call to substph) were established by the first operand (these assumptions are required to ensure the recursion of substph terminates). This ordering is precisely that required in a real (Prolog) implementation to ensure termination. The order of remaining parallel conjunctions is irrelevant to the satisfaction of procedure call assumptions, and termination of recursion (the order may be relevant, however, to performance issues – in this case, knowledge of the execution mechanism is required). Related to the issue of ordering conjuncts (goals) is the ordering of disjuncts (clauses). Given our total-correctness requirement, recursive procedures developed using the recursion introduction refinement law will terminate regardless of the ordering of disjuncts, assuming that assumptions are met. For this reason, the wide-spectrum language does not have a sequential disjunction operator. The refinement calculus approach as described in [12] has been extended in several directions. One of these is data refinement [8, 6], where the type of a program variable is refined to some other type, usually for implementation purposes, as illustrated in Sect. 6. The specification language has been extended to include higher-order constructs [7]. The introduction of higher-order constructs simplifies some refinements by the use of powerful higher-order procedures, as illustrated in Sect. 5. The specification language has also been extended to include demonic non-determinism [14], although we did not make use of it in this paper. Demonic, or “don’t care” non-determinism allows one to choose between sets of possible answers that a program must return; normally an implementation must return exactly the same set of answers as the specification. The set of answers associated with a demonic choice between two programs S and T , written S T , is either the set of answers that S returns or the set of answers T returns. This is in contrast to the set of answers associated with a disjunction S ∨ T , which is the union of the set of answers for S and T .
Developing Logic Programs from Specifications Using Stepwise Refinement
87
A tool has been developed to support the refinement calculus [15], based on the Isabelle/HOL theorem prover. By using facilities provided by the theorem prover it is possible to automatically discharge many proof obligations associated with refinement law applications, though the user of the tool guides the refinement by selecting which rules to apply. A code generation tool has also been developed [5]. It takes output from the refinement tool and generates executable code for the Mercury language [24]. This involves the deduction of intended (Mercury) mode information from the assumptions a procedure makes about its parameters. The full semantics of the calculus and more details on some of the above topics can be found in [9]. 7.1
Related Work
Traditionally, the refinement calculus has been used to develop imperative programs from specifications [1, 21, 22, 20]. The increase in expressive power of logic programming languages, when compared with imperative languages, leads to a reduced conceptual gap between a problem and its solution, which means that fewer development steps are required during refinement. An additional advantage of logic programming languages over procedural languages is their simpler, cleaner semantics, which leads to simpler proofs of the refinement steps. Finally, the higher expressive level of logic programming languages means that the individual refinement steps typically achieve more. There have been several proposals for the constructive development of logic programs, for example in Jacquet [17]. Much of this work has focused on program transformations or equivalence transformations from a first-order logic specification [4, 16]. Read and Kazmierczak [23] propose a stepwise development of modular logic programs from first-order specifications, based on three refinement steps that are much coarser than the refinement steps proposed in this paper. This leaves most of the work to be done in discharging the proof obligations for the refinement steps, for which they provide little guidance. Another approach to constructing logic programs is through schemata [19]. A logic program is designed through the application of common algorithmic structures. The designer chooses which program structure is most suitable to a task based on the data types in question. As such, the focus of this method is to aid the design of large programs. The refinement steps and corresponding verification proofs are therefore much larger. Deductive logic program synthesis [3, 11] is probably the most similar to the refinement calculus approach. In deductive synthesis, a specification is successively transformed using synthesis laws proven in an underlying framework (typically first-order logic). As mentioned earlier, the main difference between most deductive synthesis approaches and logic program refinement is the inclusion of assumptions in the wide-spectrum language, acting as preconditions. However, Lau and Ornaghi [18] have the concept of a conditional specification, which includes an input relation for a procedure (e.g., types, modes) with respect to which the synthesis of the procedure can take place. The refinement calculus generalises this by allowing an assumption (input relation) for any arbitrary
88
Robert Colvin et al.
program fragment. Another aspect of deductive synthesis is that the deduction rules are derived with the SLD computation rule in mind. Thus aspects of termination, completeness etc., have to be dealt with during the synthesis process. The refinement approach leaves clause ordering and computational termination as part of the translation from wide-spectrum language to code. Deville [10] introduces a systematic program development method for Prolog that incorporates assumptions and types similar to ours. The main difference is that Deville’s approach to program development is mostly informal, whereas our approach is fully formal. A second distinction is that Deville’s approach concentrates on the development of individual procedures. By using a wide-spectrum language, our approach blurs the distinction between a logic description and a logic program. For example, general predicates may appear anywhere within a program, and the refinement rules allow them to be transformed within that context. Similarly, programming language constructs may be used and transformed at any point.
References [1] R.-J. Back. Correctness preserving program refinements: Proof theory and applications. Tract 131, Mathematisch Centrum, Amsterdam, 1980. [2] R. J. R. Back. A calculus of refinements for program derivations. Acta Informatica, 25:593–624, 1988. [3] David Basin, Yves Deville, Pierre Flener, Andreas Hamfelt, and Jorgen Fischer Nilsson. Synthesis of programs in computational logic. Ten years of LOPSTR, 2004. [4] K. Clark. The synthesis and verification of logic programs. Research report, Imperial College, 1978. [5] R. Colvin, I. Hayes, D. Hemer, and P. Strooper. Translating refined logic programs to Mercury. In Michael Oudshoorn, editor, Proceedings of the Twenty-Fifth Australasian Computer Science Conference, volume 4 of Conferences in Research and Practice in Information Technology, pages 33–40. Australian Computer Society, January 2002. [6] R. Colvin, I. Hayes, and P. Strooper. A technique for modular logic program refinement. In K.-K. Lau, editor, Logic Based Program Synthesis and Transformation (LOPSTR 2000), Selected Papers, volume 2402 of LNCS, pages 38–56. Springer, 2001. [7] R. Colvin, I. J. Hayes, D. Hemer, and P.A. Strooper. Refinement of higher-order logic programs. In M. Leuschel, editor, Proceedings of the International Workshop on Logic-based Program Synthesis and Transformation (LOPSTR 2002), volume 2664 of Lecture Notes in Computer Science, pages 126–143. Springer, 2003. [8] R. Colvin, I. J. Hayes, and P. Strooper. Data refining logic programs. In Jim Grundy, Martin Schwenke, and Trevor Vickers, editors, International Refinement Workshop and Formal Methods Pacific 1998, Discrete Mathematics and Theoretical Computer Science, pages 100–116. Springer-Verlag, 1998. [9] Robert Colvin. Contextual and Data Refinement for the Refinement Calculus for Logic Programs. PhD thesis, School of Information Technology and Electrical Engineering, University of Queensland, August 2002.
Developing Logic Programs from Specifications Using Stepwise Refinement
89
[10] Y. Deville. Logic Programming: Systematic Program Development. AddisonWesley, 1990. [11] Y. Deville and K.-K. Lau. Logic program synthesis. Journal of Logic Programming, 19,20:321–350, 1994. Special Issue: Ten Years of Logic Programming. [12] I. J. Hayes, R. Colvin, D. Hemer, R. Nickson, and P. A. Strooper. A refinement calculus for logic programs. Theory and Practice of Logic Programming, 2(4– 5):425–460, July–September 2002. [13] D. Hemer. An algorithm for pattern-matching mathematical expressions. In Lindsay Groves and Steve Reeves, editors, Proc. Formal Methods Pacific (FMP’97), Discrete Mathematics and Theoretical Computer Science, pages 103–123. Springer Verlag, July 1997. [14] D. Hemer, R. Colvin, I. Hayes, and P. Strooper. Don’t care non-determinism in logic program refinement. In James Harland, editor, Proceeding of Computing: the Australasian Theory Symposium, volume 61 of Electronic Notes in Computer Science (ENTCS). Elsevier Science, January 2002. http://www.elsevier.nl/locate/entcs/volume61.html. [15] D. Hemer, I. Hayes, and P. Strooper. Refinement Calculus for Logic Programming in Isabelle/HOL. In R. Boulton and P. Jackson, editors, Theorem Proving in Higher Order Logics, 14th International Conference, TPHOLs 2001, volume 2152 of LNCS, pages 249–264. Springer, September 2001. [16] C.J. Hogger. Derivation of logic programs. Journal of the ACM, 28(2):372–392, 1981. [17] J.-M. Jacquet, editor. Constructing Logic Programs. Wiley Professional Computing, 1993. [18] K.-K. Lau and M. Ornaghi. The relationship between logic programs and specifications — the subset example revisited. J. Logic Programming, 30(3):239–257, March 1997. [19] Emmanouil I. Marakakis. Logic Program Development Based on Typed Moded Schemata and Data Types. PhD thesis, Department of Computer Science, University of Bristol, February 1997. [20] C. C. Morgan. Programming from Specifications. Prentice Hall, second edition, 1994. [21] C. C. Morgan and K.A. Robinson. Specification statements and refinement. IBM Journal of Research and Development, 31(5), September 1987. [22] J.M. Morris. A theoretical basis for stepwise refinement and the programming calculus. Science of Computer Programming, 9(3):287–306, December 1987. [23] M.G. Read and E.A. Kazmierczak. Formal program development in Modular Prolog: A case study. In T.P. Clement and K.-K. Lau, editors, Proc. of LOPSTR’91, Workshops in Computing, pages 69–93. Springer Verlag, 1991. [24] Z. Somogyi, F.J. Henderson, and T.C. Conway. Mercury, an efficient purely declarative logic programming language. In R. Kotagiri, editor, Proceedings of the Eighteenth Australasian Computer Science Conference, pages 499–512, Glenelg, South Australia, 1995. Australian Computer Science Communications.
Declarative Semantics of Input Consuming Logic Programs Annalisa Bossi1 , Nicoletta Cocco1 , Sandro Etalle2,3 , and Sabina Rossi1 1
Universit` a di Venezia, via Torino 155, 30172 Venezia, Italy {bossi,cocco,srossi}@dsi.unive.it 2 University of Twente, P.O. Box 616, 6200 MD Maastricht, The Netherlands
[email protected] 3 CWI, Kruislaan 413, P.O. Box 94079, 1090 GB Amsterdam, The Netherlands
Abstract. Most logic programming languages actually provide some kind of dynamic scheduling to increase the expressive power and to control execution. Input consuming derivations have been introduced to describe dynamic scheduling while abstracting from the technical details. In this paper we review and compare the different proposals given in [9], [10] and [12] for denotational semantics of programs with input consuming derivations. We also show how they can be applied to termination analysis.
1 1.1
Introduction Dynamic Scheduling in Logic Programming
In logic programming the selection rule determines which atom in a query is selected at each derivation step. The standard selection rule is the left-to-right one of Prolog, which is simple to implement, but which can cause problems both with termination and with negation when selected atoms are not fully instantiated. Moreover there are situations – like in the context of parallel executions or generate-and-test patterns – that require a more flexible control mechanism (dynamic scheduling) in which the atom to be selected is determined at runtime. Dynamic scheduling is achieved by using a dynamic selection rule and this increases the expressive power of the language and allows for a finer control of the execution. In practical systems, dynamic selection rules are implemented by means of constructs such as delay declarations (as in G¨ odel [26] and ECLiPSe [27]) or block declarations (as in SICStus Prolog [28] – block declarations are actually a special kind of delay declarations). Alternatively, in concurrent logic languages such as GHC [43], programs are augmented with guards controlling the selection of atoms dynamically. For example Moded Flat GHC [45] uses conditions based on modes and instantiation constraints imposed on individual clauses. Delay declarations, advocated by van Emden and de Lucena [46], were introduced explicitly in logic programming by Naish [37,34]. By associating conditions to predicate symbols, delay declarations indicate when an atom can be selected M. Bruynooghe and K.-K. Lau (Eds.): Program Development in CL, LNCS 3049, pp. 90–114, 2004. c Springer-Verlag Berlin Heidelberg 2004
Declarative Semantics of Input Consuming Logic Programs
91
for resolution. Such conditions are based on instantiation: typical delay declarations are ground(X) or nonvar(X) which specify that the associated atom can be selected for evaluation only if its argument X is respectively a ground term or a non-variable term. Delay declarations can be also combined together by means of logical operators, allowing for more complex control. To see how delay declarations can enforce dynamic scheduling, consider the following programs APPEND and IN ORDER: %
append(Xs,Ys,Zs) ← Zs is the concatenation of the lists Xs and Ys append([H|Xs],Ys,[H|Zs]) ← append(Xs,Ys,Zs). append([],Ys,Ys).
%
in order(Tree,List) ← List is an ordered list of the nodes of Tree in order(tree(Label,Left,Right),Xs) ← in order(Left,Ls), in order(Right,Rs), append(Ls,[Label|Rs],Xs). in order(void,[]).
together with the query Q : read tree(Tree), in order(Tree,List), write list(List). where read tree and write list are defined elsewhere. If read tree cannot read the whole tree at once – say, it receives the input from a stream – it would be nice to be able to run the “processes” in order and write list on the available input. This can be done properly only if one uses a dynamic selection rule. Prolog’s rule would call in order only after read tree has finished, while other fixed rules would immediately diverge. For instance, the fixed rule that selects always the second atom in a clause body, and that selects the first one only when the body contains only one atom can lead to nontermination, as the query in order(Tree,List) can easily diverge. The same applies to the rule that always selects the rightmost atom in a query, with the extra problem that write list(List) would be called with a non-instantiated argument: if write list is non-backtrackable (as many IO predicates are) this would imply that this selection rule yields a wrong output. In the above program, in order to avoid nontermination one can declare that predicates in order, append and write list can be selected only if their first argument is not just a variable. Formally, delay in order(T, ) until nonvar(T). delay append(Ls, , ) until nonvar(Ls). delay write list(Ls, ) until nonvar(Ls). These declarations prevent in order, append and write list from being selected “too early”, i.e., when their arguments are not “sufficiently instantiated”. Note that instead of having interleaving “processes”, one can also select several atoms in parallel, as long as the delay declarations are respected. This approach
92
Annalisa Bossi et al.
to parallelism has been first proposed by Naish [36] and – as observed by Apt and Luitjes [5] – “has an important advantage over the ones proposed in the literature in that it allows us to parallelize programs written in a large subset of Prolog by merely adding to them delay declarations, so without modifying the original program”. Compared to other mechanisms for user-defined control, e.g., using the cut operator in connection with built-in predicates that test for the instantiation of a variable (var or ground), delay declarations are more compatible with the declarative character of logic programming. Nevertheless, many important declarative properties that have been proven for logic programs do not apply to programs with delay declarations. This is mainly due to the fact that delay declarations can cause deadlock situations, in which no atom in the query respects its delay declaration and therefore no atom is selectable. Because of this the well-known equivalence between model-theoretic and operational semantics does not hold. As an example, consider the query append(X,Y,Z) with the execution mechanism described above: it does not succeed (it deadlocks) and this is in contrast with the fact that (infinitely many) instances of append(X,Y,Z) are contained in the least Herbrand model of APPEND. 1.2
Semantics of Logic Programs with Dynamic Scheduling
By introducing dynamic scheduling we obtain more powerful and flexible programs but we are faced with the problem of finding new techniques for ensuring correctness and termination of such programs and more generally for analyzing them. The standard semantics and properties are no longer valid when an atom can be delayed under some condition. In particular the standard semantics cannot capture the possibility of floundering when no atom in the goal can be selected. Hence it is not surprising that only relatively few proposals have been given for a semantics for logic programs with dynamic scheduling despite of their practical importance. The first proposal of an operational semantics for dynamic scheduling in the form of coroutining was given by Naish [35]. He defined SLDF resolution, which is a straightforward generalization of SLD resolution, where execution of atoms may be suspended indefinitely. He also considered termination of such programs and observed that if the set of callable atoms is closed under instantiation, the termination behaviour is more amenable to analysis. Moreover Naish stressed the importance of mode information for reasoning about termination of such programs. An operational semantics for constraint logic programs (CLP) with dynamic scheduling has been given also by Debray et al. [19]. Falaschi et al. [24,33,23] have defined a denotational semantics for CLP programs with dynamic scheduling where the semantics of a query is given by a set of closure operators (each operator corresponds to a sequence of rule choices). They start from an operational semantics for constraint logic programs with dynamic scheduling given in terms of derivations from the goals, which is similar to the one in [19] and in [32]. Then they give a semantics in terms of and-trees, which captures the structure of a derivation in a compositional way. An and-tree
Declarative Semantics of Input Consuming Logic Programs
93
can be seen as a function mapping an initial constraint to its answer. The denotation of a sequence of atoms is then a set of closure operators, corresponding to the and-trees which have this sequence as root. Their denotational semantics is the analogue of the bottom-up S-semantics [13] for usual logic programs, where atoms are mapped to their set of answers. Such a denotational semantics can be used as a basis for the analysis of logic programs with dynamic scheduling, since closure operators can be abstracted by descriptions that capture their behaviour. This idea was followed by Marriott et al. in [32] where a framework for global dataflow analysis for logic programming languages with dynamic scheduling is developed. Its main use is to give information on calling patterns. In [17] the analysis is further improved both in precision and in efficiency. From such proposals also optimization techniques for logic programs with dynamic scheduling have been derived, such as in [38]. A very elegant definition of an algebraic and logical semantics for constraint logic languages with dynamic scheduling has been given by Marriott in [31]. It corresponds to an operational semantics based on the one given by Naish in [35] generalized to arbitrary constraints. Delayed atoms are considered as constraints and then the soundness and completeness results for success and finite failure for CLP are extended to CLP with dynamic scheduling. In spite of these proposals some problems remained open. Dynamic scheduling is often introduced to ensure the termination of the program, preventing possible diverging derivations. Nevertheless, while for pure Prolog programs (i.e., logic programs employing the fixed leftmost selection rule) there exist results characterizing when a program is terminating such as in [7,18,14] no such a characterization was derived for programs with dynamic scheduling from these semantics. 1.3
Semantics of Input Consuming Derivations
In order to provide a characterization of dynamic scheduling that is reasonably abstract and amenable to termination analysis, Smaus introduced in [40] input consuming derivations. The definition of input consuming program relies on the concept of mode. A moded program is a program in which each atom’s arguments are partitioned into input and output ones. Output arguments are those produced by the atom during the computation process, while input arguments are consumed. Roughly speaking, in an input consuming program only atoms whose input arguments are not instantiated through the unification step are allowed to be selected. We believe that – in many cases – the adoption of “natural” delay declarations is equivalent to considering only input consuming derivations [11]. This is the case, for instance, of the programs mentioned in the example above together with their natural mode where the first position of in order is considered in input, while the second one is in output. In fact under normal circumstances, the adoption of the stated delay declarations enforces nothing but a restriction to input consuming derivations. Moreover also other control mechanisms, such
94
Annalisa Bossi et al.
as the one in Moded Flat GHC, are similar to requiring input consuming derivations: the resolution of an atom with a definition must not instantiate the input arguments of the resolved atom. Input consuming programs allow for simpler definitions of denotational semantics and have nice properties regarding termination. Henceforth they seem to be a resonable and safe approximation to programs with general dynamic scheduling. In this paper we review and compare the different proposals given for denotational semantics of programs with input consuming derivations. We also show how they can be applied to termination analysis. Our review is based on [9], [10] and [12]. 1.4
Structure of the Paper
The paper is organized as follows. Section 2 contains some preliminary notations and definitions including input consuming programs. Section 3 introduces a first denotational semantics capturing computed answer substitutions of successful derivations. This semantics applies to well and nicely moded input consuming programs. In Section 4 a second denotational semantics for simply moded input consuming programs is presented which is able to model also intermediate results of partial derivations. Section 5 shows how these semantics have been used to characterize termination properties of input consuming programs. Section 6 concludes the paper.
2
Preliminaries
The reader is assumed to be familiar with the terminology and the basic results of logic programs and their semantics [1,2,29]. In this section we introduce few notions that will be used in the sequel. 2.1
Terms and Substitutions
Let T be the set of terms built on a finite set of data constructors C and a denumerable set of variable symbols V. For any syntactic object o, we denote by Var(o) the set of variables occurring in o. A syntactic object is linear if every variable occurs in it at most once. A substitution θ is a mapping from V to T . Given a substitution σ = {x1 /t1 , . . . , xn /tn }, we say that {x1 , . . . , xn } is its domain (denoted by Dom(σ)), and Var ({t1 , . . . , tn }) is its range (denoted by Ran(σ)). Note that Var (σ) = Dom(σ) ∪ Ran(σ). We denote by the empty substitution: Dom() = Ran() = ∅. The result of the application of a substitution θ to a term t is said an instance of t and it is denoted by tθ. Given a substitution σ and a syntactic object E, we denote by σ|E the restriction of σ to the variables in Var(E), i.e., σ|E (x) = σ(x) if x ∈ Var (E), otherwise σ|E (x) = x. If t1 , . . . , tn is a permutation of x1 , . . . , xn then we say that σ is a renaming. The composition of substitutions is denoted by juxtaposition, i.e., xθσ. We say that t is a variant of t , written t ≈ t , if t and t are instances of each other. In this
Declarative Semantics of Input Consuming Logic Programs
95
case there exists a renaming θ such that t = tθ. A substitution θ is a unifier of terms t and t if tθ = t θ. We denote by mgu(t, t ) any most general unifier (mgu, in short) of t and t . 2.2
Programs and Derivations
Let P be a finite set of predicate symbols. An atom is an object of the form p(t1 , . . . , tn ) where p ∈ P is an n-ary predicate symbol and t1 , . . . , tn ∈ T . Given an atom A, we denote by Rel(A) the predicate symbol of A. A query is a finite, possibly empty, sequence of atoms A1 , . . . , Am . The empty query is denoted by 2. Following the convention adopted in [2], we use bold characters to denote sequences of objects: so, for instance, t denotes a sequence of terms, while B is a query (i.e., a possibly empty sequence of atoms). A (definite) clause is a formula H ← B where H is an atom (the head ) and B is a query (the body). When B is empty, H ← B is written H ← and is called a unit clause. A (definite) program is a finite set of clauses. We denote atoms by A, B, H, . . . , queries by Q, A, B, C, . . . , clauses by c, d, . . . , and programs by P . Computations are constructed as sequences of “basic” steps. Consider a nonempty query A, B, C and a clause c. Let H ← B be a variant of c variable disjoint from A, B, C and assume that B and H unify with mgu θ. The query (A, B, C)θ is called a resolvent of A, B, C and c with selected atom B and mgu θ
θ. A derivation step is denoted by A, B, C =⇒P,c (A, B, C)θ. The clause H ← B is called its input clause. The atom B is called the selected atom of A, B, C. If P is clear from the context or c is irrelevant then we drop the reference to them. A derivation is obtained by iterating derivation steps. A maximal sequence θ
θ
θn+1
1 2 δ : Q0 =⇒ P,c1 Q1 =⇒P,c2 · · · Qn =⇒ P,cn+1 Qn+1 · · ·
is called a derivation of P ∪ {Q0 } provided that for every step the standardization apart condition holds, i.e., the input clause employed is variable disjoint from the initial query Q0 and from the substitutions and the input clauses used at earlier steps. θn θ1 Derivations can be finite or infinite. If δ : Q0 =⇒ P,c1 · · · =⇒P,cn Qn is a θ
finite prefix of a derivation, also denoted by δ : Q0 −→ Qn with θ = θ1 · · · θn , we say that δ is a partial derivation and θ is a partial computed answer substitution of P ∪ {Q0 }. If δ is maximal and ends with the empty query, then θ is called computed answer substitution (c.a.s., for short). In this case we say that the derivation is successful. The length of a (partial) derivation δ, denoted by len(δ), is the number of derivation steps in δ. 2.3
Modes and Input Consuming Programs
Modes are a common tool for verification. A mode is a function that labels as input or output the positions of each predicate in order to indicate how the arguments of such a predicate should be used.
96
Annalisa Bossi et al.
Definition 1 (Mode). A mode for a predicate symbol p of arity n, is a function mp from {1, . . . , n} to {I , O }. We call moded atom (clause, program, query), any atom (clause, program, query) which has a mode associated to its predicate symbols. If mp (i) = I (resp. O ), we say that i is an input (resp. output ) position of p (with respect to mp ). In the examples, we often indicate the mode by writing the atom p(mp (1), . . . , mp (n)), e.g., append(I , I , O ). We assume that each predicate symbol has a unique mode associated to it; multiple modes may be obtained by simply renaming the predicates. We denote by In(Q) (resp. Out(Q)) the sequence of terms filling in the input (resp. output) positions of predicates in Q. Moreover, when writing an atom as p(s, t), we are indicating that s is the sequence of terms filling in its input positions and t is the sequence of terms filling in its output positions. The notion of input consuming derivation was introduced in [40] as a formalism for describing dynamic scheduling in an abstract way. Definition 2 (Input Consuming Derivation). θ
– A derivation step A, B, C =⇒ (A, B, C)θ is input consuming if In(B)θ = In(B). – A derivation is input consuming if all its derivation steps are input consuming. In the following sometimes we use ic-derivation for input consuming derivation and we call input consuming program (ic-program) a program when considered with respect to input consuming derivations only. Example 3. Consider the program REVERSE with accumulator and the following modes: reverse(I , O ) and reverse acc(I , O , I ). reverse(Xs,Ys) ← reverse acc(Xs,Ys,[]). reverse acc([],Ys,Ys). reverse acc([X|Xs],Ys,Zs) ← reverse acc(Xs,Ys,[X|Zs]). The following derivation δ of REVERSE ∪ {reverse([X1,X2],Zs)} is input consuming. δ: reverse([X1,X2],Zs) ⇒ reverse acc([X1,X2],Zs,[ ]) ⇒ reverse acc([X2],Zs,[X1]) ⇒ reverse acc([ ],Zs,[X2,X1]) ⇒ 2. Allowing only input consuming derivations is a form of dynamic scheduling, since whether or not an atom can be selected depends on its degree of instantiation at runtime. Given a non-empty query, if no atom is resolvable via an input consuming derivation step and no failure arises, then we say that the query deadlocks. Therefore, an ic-derivation can either be successful or finitely failing or infinite or deadlock. Each ic-derivation which is not a deadlock is also an SLD derivation.
Declarative Semantics of Input Consuming Logic Programs
2.4
97
Classes of Moded Programs
In the sequel we are going to refer to classes of programs that in some way behave well with respect to the given mode. In particular, we are going to use the concepts of well moded program (Dembinski and Maluszynski [20]), of nicely moded program (Chadha and Plaisted [15]) and of simply moded program (Apt and Etalle [4]). Definition 4 (Well, Nicely and Simply Moded Program). – Well Moded. A clause p(t0 , sn+1 ) ← p1 (s1 , t1 ), . . . , pn (sn , tn ) is well moded if for all i ∈ [1, n + 1] Var (si ) ⊆
i−1
Var (tj ).
j=0
If we call producing positions the input positions of the head and the output positions of the body and consuming positions the other ones, then we can intuitively say that a clause is well moded if every variable in a consuming position occurs also in an earlier (w.r.t. the indices, which have been deliberately chosen in this way) producing position. – Nicely Moded. A clause p(t0 , sn+1 ) ← p1 (s1 , t1 ), . . . , pn (sn , tn ) is nicely moded if t1 , . . . , tn is a linear sequence of terms, Var(t0 ) ∩ Var(t1 , . . . , tn ) = ∅, and for all i ∈ [1, n] Var (si ) ∩
n
Var (tj ) = ∅.
j=i
Intuitively a clause is nicely moded if there are no conflicts among producing positions, (a variable may appear in at most one producing position with one exception: a variable may appear twice in a producing position of the head), and a variable may not be consumed before it is produced. – Simply Moded. A clause p(t0 , sn+1 ) ← p1 (s1 , t1 ), . . . , pn (sn , tn ) is simply moded if it is nicely moded and t1 , . . . , tn is a linear sequence of variables. – A query Q is well (resp. nicely, simply) moded, if the clause q ← Q is well (resp. nicely, simply) moded, where q is a variable-free atom. Note that an atomic query p(s, t) is well moded if s is a sequence of ground terms and it is nicely moded if t is linear and Var (s) ∩ Var(t) = ∅. – A program is well (resp. nicely, simply) moded, if all of its clauses are well (resp. nicely, simply) moded. Hence the class of simply moded programs is a subclass of nicely moded ones and it includes both some well moded and some non-well moded programs. In [42] permutation well (nicely) moded programs and queries are also defined, i.e., programs and queries which would be well (nicely) moded after a permutation of the atoms respectively in the bodies and in the queries.
98
Annalisa Bossi et al.
Example 5. – The program APPEND of the introduction in the mode append(I , I , O ) is well nicely and simply moded. – REVERSE with accumulator of Example 3 is well and simply moded. – Furthermore, consider the following program PALINDROME palindrome(Xs) ← reverse(Xs,Xs). in the mode palindrome(I ), together with the program REVERSE with the modes reverse(I,O). This program is well moded but not nicely moded (since Xs occurs both in an input and in an output position of the same body atom). However, since the program REVERSE is used here for checking whether a list is a palindrome, its natural modes are reverse(I,I) and reverse acc(I,I,I). With these modes, the program PALINDROME is both well moded and simply moded. Most programs are simply moded (see the mini-survey at the end of [4]) and often programs that are not simply moded can naturally be transformed into simply moded ones (see [10]). The above notions of well, nicely and simply moded are “persistent” with respect to input consuming derivations. The following lemma is a straightforward extension of [5, Lemma 30]. Lemma 6. In a input consuming derivation, every resolvent of a well (resp. nicely, simply) moded query and a well (resp. nicely, simply) moded clause is well (resp. nicely, simply) moded. Notice that in the case of nicely and simply moded programs the above lemma depends on the fact that only input consuming derivations are considered. Indeed, when “normal” SLD derivations are considered persistence holds only when the leftmost selection rule is used. Otherwise, speculative bindings might destroy the property of being nicely moded. On the other hand, for well moded programs, any SLD resolvent of a well moded query with a well moded clause is well moded ([2]). Finally, it is worth reminding that, when considering nicely (respectively simply) moded, input consuming programs, half of the famous switching lemma still applies. The following Left-Switching Lemma that has been proven in [10]. Lemma 7. (Left-Switching) Let the program P and the query Q0 be nicely moded. Let δ be a (partial) input consuming derivation of P ∪ {Q0 } of the form θ
θn+1
θn+2
1 δ : Q0 =⇒ c1 Q1 · · · Qn =⇒ cn+1 Qn+1 =⇒ cn+2 Qn+2
where – Qn is a query of the form A, A, B, B, C, – Qn+1 is a resolvent of Qn and cn+1 w.r.t. B, – Qn+2 is a resolvent of Qn+1 and cn+2 w.r.t. Aθn+1 .
Declarative Semantics of Input Consuming Logic Programs
99
Then, there exist Qn+1 , θn+1 , θn+2 and a derivation δ such that θn+1 θn+2 = θn+1 θn+2
and θn+1
θ
θn+2
1 δ : Q0 =⇒ c1 Q1 · · · Qn =⇒ cn+2 Qn+1 =⇒ cn+1 Qn+2
where δ is input consuming and – – – – 2.5
δ and δ coincide up to the resolvent Qn , Qn+1 is a resolvent of Qn and cn+2 w.r.t. A, , Qn+2 is a resolvent of Qn+1 and cn+1 w.r.t. Bθn+1 δ and δ coincide after the resolvent Qn+2 . The S-semantics
The aim of the S-semantics approach (see [13]) is modeling the observable behaviors for a variety of logic programming languages. The observable we consider here is the computed answer substitutions. The semantics is defined as follows: S(P ) = { p(x1 , . . . , xn )θ | x1 , . . . , xn are distinct variables and θ
p(x1 , . . . , xn ) −→P 2 is an SLD derivation}. This semantics enjoys all the valuable properties of the least Herbrand model as summarized below in the following. To present the main results on the Ssemantics we need to introduce two further concepts: Let P be a program, and I be a set of atoms closed under variance. – The immediate consequence operator for the S-semantics is defined as: TPS (I) = { Hθ | ∃ H ← B variant of a clause of P ∃ C ∈ I, renamed apart4 w.r.t. H, B θ = mgu(B, C)}. – I is called an S-model of P if TPS (I) ⊆ I. Falaschi et al. [25] showed that TPS is continuous on the lattice of term interpretations, that is sets of possibly non-ground atoms, with the subset-ordering. Powers S of the operator TPS are defined in the standard way as follows: ∞ TPS ↑ 0(I) = I, S S S S TP ↑ (i + 1)(I) = TP (TP ↑ i(I)), and TP ↑ ω(I) = i=0 TP ↑ i(I). We abbreviate TPS ↑ ω(∅) to TPS ↑ ω. In [25] they proved the following: – S(P ) = least S-model of P = TPS ↑ ω. 4
Here and in the sequel, when we write “C ∈ I, renamed apart w.r.t. some expression e”, we naturally mean that I contains the atoms C1 , . . . , Cn , and that C is a renaming of C1 , . . . , Cn such that C shares no variable with e and that two distinct atoms of C share no variables with each other.
100
Annalisa Bossi et al.
Therefore, the S-semantics enjoys a declarative interpretation and a bottomup construction, just like the Herbrand one. In addition, we have that the Ssemantics reflects the observable behavior in terms of computed answer substitutions, as shown by the following well-known result. Theorem 8 ([25]). Let P be a program, A be a query. The following statements are equivalent: ϑ
– there exists an SLD derivation A−→P 2, – there exists A ∈ S(P ) (renamed apart w.r.t. A), such that σ = mgu(A, A ), where Aσ ≈ Aϑ. Example 9. Let us see this semantics applied to the programs APPEND and REVERSE so far encountered. – S(APPEND) = { append([],X,X), append([X1],X,[X1|X]), append([X1,X2],X,[X1,X2|X]), . . . }. – S(REVERSE) = { reverse([],[]), reverse([X1],[X1]), reverse([X1,X2],[X2,X1]), ... reverse acc([],X,X), reverse acc([X1],X,[X1|X]), reverse acc([X1,X2],X,[X2,X1|X]), . . . }. 2.6
Semantics of Input Consuming Programs
In Sections 3 and 4 we present two semantics for input consuming programs which are related to S-semantics. To define such semantics, the observables we focus on are the computed answer substitutions. First, we consider a semantics given by the computed answer substitutions of successful derivations. This corresponds to the S-semantics of logic programming [13] when restricted to a particular set of queries. Given a program P and a set of queries C, this semantics can be defined formally as θ
Osic (P, C) = {Aθ| A ∈ C and there exists an ic-derivation A −→P 2}. While this semantics appears very natural, it can be unsuitable for modelling the reactive nature of input consuming programs. In fact, as we mentioned in the introduction, input consuming derivations can be used to model dynamic scheduling and parallelism, and in this context it is very important to model the results of partial computations. Indeed, the standard semantics for concurrent logic languages such as ccp [39,22] and GHC [44] often capture such intermediate results, or in any case, also the results of non-successful computations [16]. In fact, the (partial) result of a computation may trigger another computation by
Declarative Semantics of Input Consuming Logic Programs
101
instantiating sufficiently the input positions of another atom so that it becomes resolvable. Because of this, when one wants to characterize for instance termination, the adoption of a semantics which is able to model intermediate results becomes essential, as shown in Section 5. Thus we also consider a semantics capturing the results of partial input consuming derivations. Given a program P and a set of queries C, this semantics can be defined formally as θ
Opic (P, C) = {Aθ| A ∈ C and there exists an ic-derivation A −→P B}. where B is any query.
3
Semantics of Well Moded Input Consuming Programs
To characterize our two semantics for ic-programs, we start from the simplest case: when one is interested only in the successful derivations. Then – if one does not restrict to ic-derivations – the observables (given by successful derivations) can be captured by the S-semantics of classical logic programs. In this section we show that the standard S-semantics is compositional and correct also for input consuming programs, provided that the programs are well and nicely moded and that only nicely moded queries are considered. The results reported in this section are proved in [9]. Proposition 10. Let P be a well and nicely moded program, A be a nicely moded atomic query. The following statements are equivalent: ϑ
(i) there exists an input consuming derivation A−→P 2, (ii) there exists A ∈ S(P ) (renamed apart w.r.t. A), and σ = mgu(A, A ) such that In(A)σ ≈ In(A), where Aσ ≈ Aϑ. To extend Proposition 10 to arbitrary (non-atomic) queries we need the following definition. Definition 11. Let A = p1 (s1 , t1 ), . . . , pn (sn , tn ) be a query. We define VIn ∗ (A) :=
n
{x| x ∈ Var (si ) and x ∈
i=1
i−1
Var (tj )}.
j=1
VIn ∗ (A) denotes the set of variables occurring in an input position of an atom of A but not occurring in an output position of an earlier atom. Note that if A is well moded then VIn ∗ (A) = ∅. Theorem 12. Let P be a well and nicely moded program, A be a nicely moded query and NM be the class of nicely moded queries. The following statements are equivalent:
102
Annalisa Bossi et al.
(i) there exists Aϑ ∈ Osic (P, NM ), (ii) there exists A ∈ S(P ) (renamed apart w.r.t. A), and σ = mgu(A, A ) such that Aσ|VIn ∗ (A) ≈ A, where Aσ ≈ Aϑ. The condition Aσ|VIn ∗ (A) ≈ A above says that the substitution σ just renames the variables occurring in an input position of A but not occurring in an output position of an earlier atom. In case of an atomic query A := A, we might substitute this condition with the somewhat more attractive condition In(A)σ ≈ In(A) of Proposition 10. Theorem 12 shows thus that S(P ) is compositional and correct for input consuming programs, provided that programs are well and nicely moded and that queries are nicely moded. In other words, given the restrictions on programs and queries, the S-semantics is correct with respect to the observables given by the computed answer substitutions of successful ic-derivations. Example 13. Consider the program APPEND of the introduction with the mode append(I,I,O). S(APPEND), reported in Example 9, allows us to draw a number of conclusions: – append([X,b],Y,Z) has an input consuming successful derivation. In particular, it has an input consuming derivation with c.a.s. {Z/[X, b|Y]}. This can be derived by just looking at S(APPEND), from the fact that A = append([X1,X2],X3,[X1,X2|X3]) ∈ S(P ) and that append([X,b],Y,Z) is - in its input positions - an instance of A. – append(Y,[X,b],Z) has no input consuming successful derivations. This is because there is no A ∈ S(P ) such that append(Y, [X, b], Z) is an instance of A in the input positions. – Observe that the query append(Y,[X,b],Z) has infinitely many successful SLD derivations and no failures. Therefore it does not fail also when we consider ic-derivations. Since, as noted above, the query has no input consuming successful derivations, this implies that – in presence of input consuming derivations – append(Y,[X,b],Z) will eventually either deadlock or run into an infinite derivation. The previous results hold also in case the programs are permutation well and nicely moded and queries are permutation nicely moded [42]. While in the context of SLD (not input consuming) derivations the S-semantics is also fully abstract, when considering input consuming program this is not so. Consider the following two trivial programs: P1 P2
= { c1: c2:
p(X). p(a).
}
={
p(X).
}
In both programs the mode is p(I). These two programs, despite being different, yield exactly the same computed answer substitutions for all queries when icderivations are considered. In fact the extra clause c2 in P1 can resolve an atom
Declarative Semantics of Input Consuming Logic Programs
103
A only if A contains the term a in its input position, but in this case c2 behaves exactly as c1 does5 . Nevertheless, the S(P1) = {p(X), p(a)} = {p(X)} = S(P2), demonstrating that the S-semantics is not fully abstract when considering icderivations. In the next section we present a more complex semantics, which is also fully abstract for ic-derivations.
4
Semantics of Simply Moded Input Consuming Programs
The semantics presented in the previous section applies only when we are interested in the computed answer substitutions of successful derivations. As we discussed before, there are many situations in which we also want to model the (intermediate) results of partial derivations. For instance, this will be the case when – in the next section – we study the termination of input consuming programs. In this section we define a somewhat more complex denotational semantics which has the advantage of modelling the observables given by both successful and partial derivations in a rather symmetric way. The two semantics we are going to introduce are compositional, correct and fully abstract with respect to the operational semantics of input consuming simply moded programs and queries, i.e., Osic (P, SM ) and Opic (P, SM ), where SM is the class of simply moded queries. As in the standard S-semantics, this is a denotational semantics that can be built by means of a bottom-up construction. 4.1
Simply Local Substitutions and Simply Local Models
When input consuming derivations are applied to simply moded programs and queries, important properties follow from the way clauses become instantiated along the derivations. The notion of simply local substitution is introduced in [12] to reflect this instantiation mechanism. A clause c = H ← B1 , . . . , Bn becomes instantiated by its “caller” (the atom that is resolved using c) and its “callees” (the clauses used to resolve the body atoms of c). Thus, a simply local substitution is defined as the composition of several substitutions, σ0 , σ1 . . . , σn , one for each atom in the given clause, such that σ0 binds the input variables of the head of the clause, and each σi (i > 0) creates a binding from the output variables to input terms of Bi σ0 , . . . , σi−1 . Definition 14 (Simply Local Substitution). Let θ be a substitution. We say that θ is simply local w.r.t. the clause H ← B1 , . . . , Bn if there exist substitutions σ0 , σ1 . . . , σn and disjoint sets of fresh (w.r.t. c) variables v0 , v1 , . . . , vn such that θ = σ0 σ1 · · · σn where 5
The only observable difference between P1 and P2 lies in the multiplicity of the answers: the query q(a) succeeds twice in P1 and only once in P2, but answer multiplicity is not an observable we consider here.
104
Annalisa Bossi et al.
– Dom(σ0 ) ⊆ Var(In(H)) and Ran(σ0 ) ⊆ v0 , – for i ∈ [1..n], Dom(σi ) ⊆ Var (Out(Bi )) and Ran(σi ) ⊆ Var (In(Bi )σ0 σ1 · · · σi−1 ) ∪ vi . The substitution θ is simply local w.r.t. a query B if θ is simply local w.r.t. the clause q ← B where q is any variable-free atom. Example 15. Consider the program APPEND together with the mode append(I, I,O) and its recursive clause c : append([H|Xs], Ys, [H|Zs]) ← append(Xs, Ys, Zs). The substitution θ = {Xs/[], Ys/W, Zs/W} is simply local w.r.t. c. In fact θ = σ0 σ1 where σ0 = {Xs/[], Ys/W} and σ1 = {Zs/W}. Consider now the query Q : append([a, X, c], Ys, Zs), append(Zs, [b], Ls). The substitution θ = {Zs/[a,X,c|Ys]} is simply local w.r.t. Q. In fact θ = σ1 σ2 where σ1 = {Zs/[a,X,c|Ys]} and σ2 is the empty substitution. The denotational semantics we are about to define is based on a restricted notion of model. Here and in the sequel interpretations are sets of moded atoms closed under variance. Definition 16 (Simply Local Model). Let M be a set of moded atoms. We say that M is a simply local model of a clause c : H ← B1 , . . . , Bn if for every substitution θ simply local w.r.t. c, if B1 θ, . . . , Bn θ ∈ M then Hθ ∈ M .
(1)
M is a simply local model of a program P if it is a simply local model of each clause of it. Clearly a simply local model is not necessarily a model in the classical sense, since the substitution θ in (1) is required to be simply local. For example, given the program {q(1)., p(X) ← q(X).} with modes q(I ), p(O ), a model must contain the atom p(1), whereas a simply local model does not necessarily contain p(1), since {X/1} is not simply local w.r.t. p(X) ← q(X). A minimal simply local model exists and it is bottom-up computable by applying the following operator [12]. Definition 17. Given a program P and a set of moded atoms I, we define TPSL (I) = I ∪ {Hθ | ∃ c : H ← B variant of a clause of P, θ is simply local w.r.t. c, Bθ ∈ I}
Declarative Semantics of Input Consuming Logic Programs
105
TPSL is monotonic and continuous on the lattice where sets of moded atoms are ordered by set inclusion. In the following we denote by SM P the set of all simply moded atoms of the extended Herbrand universe of P . In [12] it is proven that if P is simply moded and I ⊆ SM P then TPSL ↑ ω(I) is the least simply local model of P containing I
(2)
This allows us to define our models. Definition 18. Let P be a program. We define – MPSL is the least simply local model of P , – PM SL P is the least simply local model of P containing SM P . The existence of these models is guaranteed by (2), in fact (2) also shows how to construct them, as it implies that SL MPSL = TPSL ↑ ω(∅), and PM SL P = TP ↑ ω(SMP )
4.2
(3)
Relation among Denotational and Operational Semantics
To relate the MPSL and PMPSL to Osic (P, SM ) and Opic (P, SM ) we need to relate TPSL to the results of input consuming derivations; this is achieved in the following lemma, proved in [12]. Lemma 19. Let the program P and the query A be simply moded and I ⊆ SM P be a set of moded atoms. The following statements are equivalent: ϑ
(i) there exists an input consuming derivation A −→P C with C ⊆ I, (ii) there exists a substitution θ simply local w.r.t. A, such that Aθ ⊆ TPSL ↑ ω(I), where Aϑ ≈ Aθ. We can now prove that MPSL and PM SL P fully characterize the semantics of icderivations for simply moded programs and queries, namely they are equal to Osic (P, SM ) and Opic (P, SM ), respectively. Theorem 20. Let P be simply moded. Then (i) MPSL = Osic (P, SM ). (ii) PMPSL = Opic (P, SM ). Proof. Immediate by (3), Lemma 19 and the definitions of Osic (P, SM ) and Opic (P, SM ). An example follows.
106
Annalisa Bossi et al.
Example 21. Let us consider again the program APPEND. 1. First let us consider its successful ic-derivations. Hence we have to build SL MAPPEND SL MAPPEND = {append([t1 , . . . , tn ], s, [t1 , . . . , tn |s]) | n ∈ [0..∞], and t1 , . . . , tn , s are any terms}.
Notice that this model is different from S(APPEND), reported in Example 9. We are going to relate S(P ) and MPSL later in this section. 2. Now let us consider the results of partial derivations. Recall that PM SL APPEND is obtained by repeatedly applying TPSL to each simply moded atom. Simply moded atoms are append(s, t, x) where s and t are arbitrary terms but x is a variable not occurring in s or in t. We obtain SL PM SL APPEND = MAPPEND ∪ {append(s, t, x) | x is a fresh variable } ∪ {append([t1 , . . . , tm |s], t, [t1 , . . . , tm |x]) | x is a fresh variable}
where s, t, t1 , . . . , tm are arbitrary terms. Consider now the query append([a, b, c|X], Y, Z). It is straightforward to check that the substitution θ = {Z/[a, b|Z ]} is simply local w.r.t. it, and that SL append([a, b, c|X], Y, Z)θ ∈ PMAPPEND . Therefore, by using Theorem 20, we can conclude that there exists a partial derivation starting in append([a, b, c|X], Y, Z), with computed answer θ. Following the same reasoning, one can also conclude that the query has a partial derivation with computed answer θ = {Z/[a|Z ]}. 4.3
Relation between S-semantics and Denotational Semantics for IC-programs
In this section we compare the denotational semantics MPSL with the S-semantics S(P ) of simply moded programs. First, we need a new definition: let I be a set of moded atoms, the input closure of I is defined as: InCl(I) = {Aθ | A ∈ I and Var (A) ∩ Var (θ) ⊆ Var (In(A))} So the input closure of an atom is obtained by instantiating its input positions in all possible ways, provided that no new links are created between the input and the output positions. Theorem 22. Let P be a well and simply moded program, then MPSL = InCl (S(P ))
Declarative Semantics of Input Consuming Logic Programs
107
Proof. First observe that the class of simply moded programs is contained in the class of nicely moded programs, hence Theorem 12 is applicable also when we consider well and simply moded programs and simply moded queries. - MPSL ⊆ InCl (S(P )). Let A be simply moded and assume Aϑ ∈ MPSL . Then, by Theorem 20, Aϑ ∈ Osic (P, SM ). By Theorem 12 there exists A ∈ S(P ) (renamed apart w.r.t. A), and σ = mgu(A, A ) such that In(A)σ ≈ In(A) and Aσ ≈ Aϑ. Since A is simply moded, we can choose σ such that Dom(σ) ∩ Var (A ) ⊆ Var(In(A )). Therefore Aϑ ≈ Aσ = A σ ∈ InCl (S(P )). - MPSL ⊇ InCl(S(P )). Let Aθ ∈ InCl (S(P )) and A = p(s, t) ∈ S(P ). There exist a simply moded atom A = p(s , z), renamed apart w.r.t. A, and a substitution σ such that σ = mgu(A, A ), In(A )σ = In(A ) and A σ = Aσ ≈ Aθ. By Theorem 12 there exists ϑ such that A ϑ ∈ Osic (P, SM ) and A ϑ ≈ A σ ≈ Aθ. Hence, by Theorem 20, Aθ ∈ MPSL .
5
Semantic-Based Verification of Termination
There have been only few proposals which tackled the specific problem of verifying the termination of logic programs with dynamic scheduling, namely by Apt and Luitjes [5], Marchiori and Teusink [30] and Smaus. Input consuming derivations were indeed introduced by Smaus in [40] to simplify the study of program properties which depend on selection rules and in [41] he started to study in particular the problem of termination of input consuming derivations. In [10] and [12] we study two classes of programs terminating with respect to input consuming derivations and well-formed queries. The two classes differ in various aspects. First of all, two different classes of well-formed queries are considered: nicely moded queries in [10], simply moded queries in [12]. To give an uniform presentation, in [12] we consider a parametric class of programs in which all input consuming derivations terminate. The parameter is a given class C of queries. Definition 23 (Input Termination w.r.t. a class C of queries). Let C be a class of queries. A program is called input terminating with respect to C if all its input consuming derivations started in a query in C are finite. The second difference among the two classes of terminating programs in [10] and [12] is in the termination proof techniques. The first class follows the style of [3,8] and it uses a simple (syntactic) termination condition, but it is also a rather restrictive class. The second class follows the style of [6,7], that is based on a more complex model theoretic approach, and it uses the semantics introduced in Section 4; this is a significantly larger class of programs. Let us consider first the more restrictive and simple class introduced in [10]: The class of nicely moded quasi recurrent programs. Its definition is based on the notion of well moded level mapping, first introduced in [21]. Here we use well moded level mappings extended to all the terms on BPE as in [10]. BPE , the extended Herbrand base of P , is the set of equivalence classes of all (possibly non-ground) atoms, modulo renaming, whose predicate symbols appear in P .
108
Annalisa Bossi et al.
Definition 24 (Moded Level Mapping). Let P be a program and BPE be the extended Herbrand base for the language associated with P . A function | | is a moded level mapping for P if: – it is a function | | : BPE → N from atoms to natural numbers; – for any t and u, |p(s, t)| = |p(s, u)|. For A ∈ BPE , |A| is the level of A. Definition 25 (Quasi Recurrency). Let P be a program. – A clause of P is called quasi recurrent with respect to a moded level mapping | | if for every instance H ← A, B, C of it, if Rel(H) Rel (B) then |H| > |B|6 . – P is called quasi recurrent with respect to | | if all its clauses are. P is called quasi recurrent if it is quasi recurrent with respect to some moded level mapping | | : BPE → N. Theorem 26. Let P be a nicely moded program. If P is quasi recurrent then P is input terminating with respect to the class of nicely moded queries. The proof of this theorem can be found in [10]. Thus, the quasi recurrent condition is a sufficient condition for input termination of nicely moded programs and nicely moded queries. But it is not a necessary condition: there are nicely moded programs input terminating on all nicely moded queries which are not quasi recurrent as shown by the following simple example taken from [10]. Example 27. Consider the following program with moding p(I, O). p(X,a) ← p(X,b). p(X,b). This program is clearly input terminating, however it is not quasi recurrent. For the first clause to be quasi recurrent it would have to be the case that |p(X, a)| > |p(X, b)|, for some moded level mapping | |. On the other hand, since p(X, a) and p(X, b) differ only for the terms filling in their output positions, by definition of moded level mapping, |p(X, a)| = |p(X, b)|. Hence, we have a contradiction. A full characterization can be obtained only by further restricting the class of programs, passing from nicely moded to simply moded and input-recursive programs. 6
Given two predicate symbols defined in a program P we denote by p q the fact that the definitions of the two predicates are mutually recursive.
Declarative Semantics of Input Consuming Logic Programs
109
Definition 28 (Input-Recursive Program). Let P be a program. – A clause H ← A, B, C of P is called input-recursive if if Rel(H) Rel (B) then Var(In(B)) ⊆ Var (In(H)). – A program P is called input-recursive if all its clauses are. Input-recursive is a syntactic condition on a clause requiring that the set of variables occurring in the arguments filling in the input positions of each recursive call in the clause body is a subset of the set of variables occurring in the arguments filling in the input positions of the clause head. The class of inputrecursive programs has strong similarities with the class of primitive recursive functions and recurrent logic programs. It does not include programs whose termination depend on the so-called inter-argument relations such as quicksort. Quasi recurrency fully characterizes input termination of simply moded and input-recursive programs with respect to nicely moded queries. Theorem 29. Let P be a simply moded and input-recursive program. P is quasi recurrent if and only if P is input terminating with respect to the class of nicely moded queries. The proof of this theorem can be found in [10]. To consider a larger class of input terminating programs we can follow the same approach pursued by Apt and Pedreschi in defining acceptable programs and use a model to capture the inter-argument relations between the atoms in a query. Intuitively, the model represents all the possible contexts in which a specific atom in a query can be called. Standard models suffice when standard left-to-right derivations are considered, that is when the contexts depends only on the computed answers of the atoms occurring on the left of the considered one. When input consuming derivations are considered, the description of all the possible contexts is much more complex since there may be atoms in the query which are only partially computed when the considered atom is selected. Hence a computed answer semantics does not provide enough information, which is why we need to capture partial computed answers of input consuming derivations. The semantics defined in [12] and the concept of simply local model give us the right tools and allow us to identify a large class of input terminating programs which includes also programs employing a non-trivial recursion scheme such as quicksort, permute, transpose. In fact, based on the notion of simply local models, in [12] we introduced the notion of simply acceptable programs which corresponds to the notion of acceptable programs introduced in [6]. Definition 30 (Simply Acceptable Program). Let P be a program and M a simply local model of P containing SM P . – A clause c of P is simply acceptable with respect to a moded level mapping | | and M if for every variant H ← A, B, C of c and every substitution θ simply local with respect to c, if Aθ ∈ M and Rel (H) Rel(B) then |Hθ| > |Bθ|.
110
Annalisa Bossi et al.
– P is simply acceptable with respect to M if there exists a moded level mapping | | such that each clause of P is simply acceptable with respect to | | and M . We also say that P is simply acceptable if it is simply acceptable with respect to some M and moded level mapping | |. Simple acceptability fully characterizes input termination of simply moded programs with respect to simply moded queries. Theorem 31. Let P be a simply moded program. P is simply acceptable if and only if it is input terminating with respect to simply moded queries. The following example shows how we can use the above theorem to reason about termination of a program. Example 32. Consider the following PERMUTE program permute([X|Xs],Ys) ← insert(Zs,X,Ys), permute(Xs,Zs). permute([],[]). insert([],X,[X]). insert([U|Xs],X,[U|Zs]) ← insert(Xs,X,Zs). We consider it with two different modes. 1. First, consider the mode permute(O , I ), insert(O , O , I ). Notice that the program is not input terminating in this mode: by repeatedly selecting the rightmost atom, the query permute(Xs,Ys) generates an infinite input consuming derivation. By Theorem 31, we can prove it by showing that PERMUTE in this mode cannot be simply acceptable with respect to PM SL PERMUTE and a moded level mapping which is invariant under renaming. First note that PM SL PERMUTE contains every atom of the form insert(Us, U, t) where Us and U are disjoint from t, i.e., every simply moded atom whose predicate is insert. Therefore, in particular, insert(Us, U, Vs) ∈ PM SL PERMUTE . The substitution θ = {Ys/Vs, Zs/Us, X/U} is simply local w.r.t. the first clause. Therefore, for this clause to be simply acceptable, by Theorem 31, there would have to be a moded level mapping, invariant under renaming, such that |permute([U|Xs], Vs)| > |permute(Xs, Us)|. This is a contradiction since a moded level mapping depends only on the input arguments (the second argument of permute) and we are considering a level mapping invariant under renaming. Thus Theorem 31 can be used to diagnose a program, in that we can pinpoint why it does not input terminate. 2. Now consider the program PERMUTE together with the mode permute(I , O ), insert(I , I , O ). In this case, in order to make the program simply moded we have to permute the two body atoms of the first permute clause7 . I.e., permute is redefined as 7
Actually, everything we state applies to the class of permutation simply moded programs, i.e., those programs and queries that are simply moded possibly after a permutation of body atoms. For the sake of notation simplicity, we avoid to refer to this in a structural way.
Declarative Semantics of Input Consuming Logic Programs
111
permute([X|Xs],Ys) ← permute(Xs,Zs), insert(Zs,X,Ys). permute([],[]). Notice that the program is now input terminating with respect to simply moded queries. This is in fact the natural mode of the PERMUTE program. To demonstrate the termination one can apply Theorem 31 using any simply local model containing SM P together with the following moded level mapping: |permute(l, )| = len(l), |insert(l, , )| = len(l).
6
Conclusion
In this paper, we have illustrated two denotational semantics proposed in [9], [10] and in [12]