Objects, Components, Models and Patterns: 47th International Conference, TOOLS EUROPE 2009, Zurich, Switzerland, June 29-July 3, 2009, Proceedings (Lecture Notes in Business Information Processing)

Lecture Notes in Business Information Processing Series Editors Wil van der Aalst Eindhoven Technical University, The N...

Author: Manuel Oriol | Bertrand Meyer

102 downloads 824 Views 11MB Size Report

This content was uploaded by our users and we assume good faith they have the permission to share this book. If you own the copyright to this book and it is wrongfully on our website, we offer a simple DMCA procedure to remove your content from our site. Start by pressing the button below!

Report copyright / DMCA form

DOWNLOAD PDF

Lecture Notes in Business Information Processing Series Editors Wil van der Aalst Eindhoven Technical University, The Netherlands John Mylopoulos University of Trento, Italy Norman M. Sadeh Carnegie Mellon University, Pittsburgh, PA, USA Michael J. Shaw University of Illinois, Urbana-Champaign, IL, USA Clemens Szyperski Microsoft Research, Redmond, WA, USA

33

Manuel Oriol Bertrand Meyer (Eds.)

Objects, Components, Models and Patterns 47th International Conference, TOOLS EUROPE 2009 Zurich, Switzerland, June 29-July 3, 2009 Proceedings

13

Volume Editors Manuel Oriol University of York Department of Computer Science Heslington, York, YO10 5DD, UK E-mail: [email protected] Bertrand Meyer ETH Zurich Zurich, Switzerland E-mail: [email protected]

Library of Congress Control Number: 2009929321 ACM Computing Classification (1998): D.2, H.1, K.6 ISSN ISBN-10 ISBN-13

1865-1348 3-642-02570-6 Springer Berlin Heidelberg New York 978-3-642-02570-9 Springer Berlin Heidelberg New York

This work is subject to copyright. All rights are reserved, whether the whole or part of the material is concerned, specifically the rights of translation, reprinting, re-use of illustrations, recitation, broadcasting, reproduction on microfilms or in any other way, and storage in data banks. Duplication of this publication or parts thereof is permitted only under the provisions of the German Copyright Law of September 9, 1965, in its current version, and permission for use must always be obtained from Springer. Violations are liable to prosecution under the German Copyright Law. springer.com © Springer-Verlag Berlin Heidelberg 2009 Printed in Germany Typesetting: Camera-ready by author, data conversion by Scientific Publishing Services, Chennai, India Printed on acid-free paper SPIN: 12705689 06/3180 543210

Preface

For the past 20 years the TOOLS conferences have continuously spread new technologies in the world of object-orientation, component technology and software engineering in general. They constitute a particularly important forum for software researchers and practitioners to present work that relies on producing tools. This year’s TOOLS continued the tradition and presented a strong program that included original contributions in all major ﬁelds of the object-oriented paradigm. As in TOOLS 2008, the concept of model occupied a considerable place; but other topics such as reﬂection, aspects, modelling languages, debugging, and virtual machines design were also strongly represented. While most conferences had a decrease in the number of submitted papers, TOOLS made the choice of quality and decided to lower its acceptance rate to 25%, as a ﬁxed maximum announced in the Call for Papers. Out of 67 submitted contributions, the reviewing process led to the selection of 17 as full papers. For the ﬁrst time, two short papers were also accepted for presentation at the conference. The 47th edition of the conference could not have seen the light without the numerous helpers that made it happen. The quality of the work of the Program Committee (PC) deserves particular acknowledgement; not only did the PC members work hard to produce helpful reviews on time, sometimes with the help of external referees, but extensive discussions followed the initial recommendations, leading to an excellent conference program and extensive advice to the authors. The Organizing Committee worked hard to schedule the events and make the conference a pleasant event. We take the opportunity to thank everyone who made this event happen in a way or another. April 2009

Manuel Oriol Bertrand Meyer

Organization

TOOLS EUROPE 2009 was organized by the department of Computer Science, ETH Zurich.

Chairpersons Conference Chair Program Chair Workshops Chair

Bertrand Meyer (ETH Zurich, Switzerland) Manuel Oriol (University of York, UK) Alexandre Bergel (INRIA Lille, France) Johan Fabry (University of Chile, Chile) Philippe Lahire (Universit´e de Nice, France) Marcus Denker (University of Bern, Switzerland) Ilinca Ciupa, Claudia G¨ unthart, Marco Piccioni

Publicity Chair

Organizing Committee

Program Committee Patrick Albert Balbir S. Barn Mike Barnett Claude R. Baudoin Bernhard Beckert Alexandre Bergel Judith Bishop Phil Brooke Cristiano Calcagno Ana Cavalcanti Dave Clarke Bernard Coulette Jing Dong St´ephane Ducasse Gregor Engels Patrick Eugster Manuel Fahndrich Jos´e Luiz Fiadeiro Michael Franz Judit Nyekyne Gaizler Benoˆıt Garbinato Carlo Ghezzi Tudor Girba Martin Glinz

ILOG Fellow, France Middlesex University, UK Microsoft Research, USA Schlumberger, France University of Koblenz, Germany INRIA Lille, France University of Pretoria, South Africa University of Teesside, UK Imperial College, UK University of York, UK KU Leuven, Belgium IRIT/University of Toulouse, France University of Texas at Dallas, USA INRIA Lille, France University of Paderborn, Germany Purdue University, USA Microsoft Research, USA University of Leceister, UK University of California, Irvine, USA Pazmany University, Hungary University of Lausanne, Switzerland Politecnico di Milano, Italy University of Bern, Switzerland University of Zurich, Switzerland

VIII

Organization

Martin Gogolla Jeﬀ Gray Pedro Guerreiro Joseph Kiniry Ivan Kurtev Ralf Laemmel Philippe Lahire Mingshu Li Dragos Manolescu Erik Meijer Peter M¨ uller Jonathan Ostroﬀ Richard Paige Marc Pantel Frederic Peschanski Alfonso Pierantonio Alexander Pretschner Bran Selic Anatoly Shalyto Friedrich Steimann Perdita Stevens ´ Eric Tanter Dave Thomas Laurence Tratt Antonio Vallecillo Roel Wuyts Amiram Yehudai Andreas Zeller

University of Bremen, Germany University of Alabama at Birmingham, UK Universidade do Algarve, Portugal University College Dublin, Ireland University of Twente, The Netherlands University of Koblenz-Landau, Germany Universit´e de Nice, France Chinese Academy of Sciences, China Microsoft, USA Microsoft, USA ETH Zurich, Switzerland York University, Canada University of York, UK IRIT/University of Toulouse, France LIP6 - UPMC Paris Universitas, France Universit` a degli Studi dell’Aquila, Italy TU Kaiserslautern/Fraunhofer IESE, Germany IBM, USA St. Petersburg State University ITMO, Russia Fernuniversit¨ at in Hagen, Germany University of Edinburgh, UK University of Chile, Chile Bedara, Canada Bournemouth University, UK Universidad de M´ alaga, Spain IMEC, Belgium Tel Aviv University, Israel Saarland University, Germany

External Referees Nicolas Arni-Bloch Martin Assmann Robert Atkey Jan-Christopher Bals Stephanie Balzer Hakim Belhaouari Manuel F. Bertoa Joel-Alexis Bialkiewicz Domenico Bianculli Luca Cavallaro Antonio Cicchetti Ilinca Ciupa Marcio Cornelio

Zekai Demirezen Simon Denier Marcus Denker Mauro Luigi Drago Martin Ernst Baris G¨ uldali Simon Hudon Ethan Jackson Mikolas Janota R´obert Kitlei Tam´as Kozsik Jannik Laval David Makalsky

Alessandro Margara Marjan Mernik Dimiter Milushev Tu Peng Fiona Polack Damien Pollet Jose Proenca Lukas Renggli Leila Ribeiro Jose E. Rivera Markku Sakkinen Tibor Somogyi Cliﬀord Sule

Organization

Yu Sun Peter Swinburne Robert Tairas M´ at´e Tejfel

Faraz Torshizi Sarvani Vakkalanka Andreas W¨ ubbeke Ye Yang

Yajing Zhao

IX

Table of Contents

Invited Presentations On Realizing a Framework for Self-tuning Mappings . . . . . . . . . . . . . . . . . . Manuel Wimmer, Martina Seidl, Petra Brosch, Horst Kargl, and Gerti Kappel

1

Programming Models for Concurrency and Real-Time (Abstract) . . . . . . Jan Vitek

17

Reﬂection and Aspects CIF: A Framework for Managing Integrity in Aspect-Oriented Composition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Andrew Camilleri, Geoﬀrey Coulson, and Lynne Blair A Diagrammatic Formalisation of MOF-Based Modelling Languages . . . . Adrian Rutle, Alessandro Rossini, Yngve Lamo, and Uwe Wolter Designing Design Constraints in the UML Using Join Point Designation Diagrams . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Vanessa Stricker, Stefan Hanenberg, and Dominik Stein Stream-Based Dynamic Compilation for Object-Oriented Languages . . . . Michael Bebenita, Mason Chang, Andreas Gal, and Michael Franz

18

37

57

77

Models Algebraic Semantics of OCL-Constrained Metamodel Speciﬁcations . . . . Artur Boronat and Jos´e Meseguer Specifying and Composing Concerns Expressed in Domain-Speciﬁc Modeling Languages . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Aram Hovsepyan, Stefan Van Baelen, Yolande Berbers, and Wouter Joosen

96

116

Early Crosscutting Metrics as Predictors of Software Instability . . . . . . . . Jos´e M. Conejero, Eduardo Figueiredo, Alessandro Garcia, Juan Hern´ andez, and Elena Jurado

136

Extensibility in Model-Based Business Process Engines . . . . . . . . . . . . . . . Mario S´ anchez, Camilo Jim´enez, Jorge Villalobos, and Dirk Deridder

157

XII

Table of Contents

Theory Guaranteeing Syntactic Correctness for All Product Line Variants: A Language-Independent Approach . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Christian K¨ astner, Sven Apel, Salvador Trujillo, Martin Kuhlemann, and Don Batory A Sound and Complete Program Logic for Eiﬀel . . . . . . . . . . . . . . . . . . . . . Martin Nordio, Cristiano Calcagno, Peter M¨ uller, and Bertrand Meyer

175

195

Components A Coding Framework for Functional Adaptation of Coarse-Grained Components in Extensible EJB Servers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Olivier Caron, Bernard Carr´e, Alexis Muller, and Gilles Vanwormhoudt A Leasing Model to Deal with Partial Failures in Mobile Ad Hoc Networks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Elisa Gonzalez Boix, Tom Van Cutsem, Jorge Vallejos, Wolfgang De Meuter, and Theo D’Hondt

215

231

Monitoring Reusing and Composing Tests with Traits . . . . . . . . . . . . . . . . . . . . . . . . . . . St´ephane Ducasse, Damien Pollet, Alexandre Bergel, and Damien Cassou

252

Flow-Centric, Back-in-Time Debugging . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Adrian Lienhard, Julien Fierz, and Oscar Nierstrasz

272

A Classiﬁcation Framework for Pointcut Languages in Runtime Monitoring . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Karl Klose and Klaus Ostermann

289

Systems Generation Fast Simulation Techniques for Design Space Exploration . . . . . . . . . . . . . Daniel Knorreck, Ludovic Apvrille, and Renaud Pacalet PyGirl: Generating Whole-System VMs from High-Level Prototypes Using PyPy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Camillo Bruni and Toon Verwaest

308

328

Short Papers Using Grammarware Languages to Deﬁne Operational Semantics of Modelled Languages . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Daniel A. Sadilek and Guido Wachsmuth

348

Table of Contents

Automatic Generation of Integrated Formal Models Corresponding to UML System Models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Helen Treharne, Edward Turner, Richard F. Paige, and Dimitrios S. Kolovos Author Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

XIII

357

369

On Realizing a Framework for Self-tuning Mappings Manuel Wimmer, Martina Seidl, Petra Brosch , Horst Kargl, and Gerti Kappel Vienna University of Technology, Austria [email protected]

Abstract. Realizing information exchange is a frequently recurring challenge in nearly every domain of computer science. Although languages, formalisms, and storage formats may diﬀer in various engineering areas, the common task is bridging schema heterogeneities in order to transform their instances. Hence, a generic solution for realizing information exchange is needed. Conventional techniques often fail, because alignments found by matching tools cannot be executed automatically by transformation tools. In this paper we present the Smart Matching approach, a successful combination of matching techniques and transformation techniques, extended with self-tuning capabilities. With the Smart Matching approach, complete and correct executable mappings are found in a test-driven manner.

1

Introduction

In this paper we present a self-tuning approach for information integration. Our approach—the Smart Matching approach—allows the derivation of high quality executable mappings for diﬀerent kinds of schemas from small, simple examples deﬁned by the user. Seamless information exchange between diﬀerent sources is an important task in nearly every engineering area of computer science. This ubiquitous challenge recurs in various application domains starting from the exchange of data between databases over the exchange of ontology instances between semantic web services to the exchange of complete models between modeling tools. Although every engineering area has its own languages, formalisms, and storage formats, the problem is always the same. Two heterogeneous schemas have to be bridged in order to transform instances of the one schema to instances of the other. Thus, a generic information integration solution for the use in diﬀerent application domains is highly valuable.

This work has been partly funded by FFG under grant FIT-IT-819584 and FWF under grant P21374-N13. Funding for this research was provided by the fFORTE WIT - Women in Technology Program of the Vienna University of Technology, and the Austrian Federal Ministry of Science and Research.

M. Oriol and B. Meyer (Eds.): TOOLS EUROPE 2009, LNBIP 33, pp. 1–16, 2009. c Springer-Verlag Berlin Heidelberg 2009

2

M. Wimmer et al.

On the one hand, matching techniques for automatically ﬁnding semantic correspondences between schemas have been developed in the context of information integration. On the other hand, transformation techniques have been established for automatically transforming data conforming to a schema A to data conforming to a schema B in a semantic preserving way. However, the combination of matching approaches and transformation approaches is accompanied with several diﬃculties due to the huge gap between detected alignments, mappings, and executable transformation code. For bridging this gap, two problems have to be solved. First, state-of-the-art matching techniques do not produce executable transformation code. Second, for the automatic execution of the transformation, correctness and completeness of the found alignments are indispensable. A solution for the ﬁrst problem is presented in [7], where reusable mapping operators for the automatic transformation of models are introduced. A solution for the second problem is conceptually proposed in [8], where we discussed how to improve current integration techniques by the application of self-tuning methods in a test-driven manner resulting in the Smart Matching approach. Based on this work, we implemented the self-tuning matching framework Smart Matcher. In this paper we describe how the Smart Matcher increases the completeness and correctness of alignments based on predeﬁned examples. In particular, we present the realization of the Fitness Function for self-evaluation and the Mapping Engine for self-adaptation which are the basis of the self-tuning. This paper is organized as follows. In Section 2 we present the Smart Matching approach at a glance and the running example of this paper. Sections 3 and 4 elaborate on the two core components of the Smart Matcher namely the Fitness Function and the Mapping Engine, respectively. In Section 5 we present an evaluation of our approach. Related work is discussed in Section 6. In Section 7 we conclude and give an outlook to future work.

2

Smart Matching at a Glance

In this section, we present the Smart Matcher and its underlying conceptual architecture at a glance. The Smart Matching method combines existing integration techniques, i.e., matching and transformation approaches, by introducing a dedicated mapping layer for bridging the gap between alignments, mappings, and executable transformation code as described in [1]. Feedback-driven self-tuning improves the quality of the mappings in an iterative way [9]. For the implementation of the self-tuning capabilities, the quality of mappings has to be evaluated. Therefore, we adopt test-driven development methods from software engineering [3] for developing integration solutions. The Smart Matcher’s architecture is illustrated in Fig. 1. Basically, the application of the Smart Matcher comprises eight phases which are described in the following. 1. Develop example instances. Like in test-driven software development, we specify an expected outcome for a speciﬁc input, i.e., instances for input and output schemas representing the same real world example are developed. Those instances are necessary to verify the mappings identiﬁed in later phases in order

On Realizing a Framework for Self-tuning Mappings 2 InitialMatcher

MappingLanguage Repository

CARMappingOperators CARMappingOperators pp g Operators p CAR CARMappingOperators Mapping CARMappingOperators

e.g.,COMA++,FOAM,... COMA FOAM

3 Alignments

Mapping Engine MappingEngine load

3

7

6

feedback

load

MappingModel 4

Schema

read

TransformationEngine

SchemaB

SchemaA instanceOf

Instance

write

instanceOf

instanceOf

actual

target

Bankomat Bankomat ExampleX Example X 1

1

describe

cas cashdispens cash dispens´ Example ExampleY Y´

cashdispenr pY cashdispr cash dispr Example ExampleY describe

Real World Real World Real World Bankomat Example Bankomat

FitnessFunction B d BasedonDiffͲOperator Diff O

5

Fig. 1. The Smart Matching approach at a glance

to follow a feedback-driven integration method. The user deﬁned input instance is later called LHS (left-hand side) instance, the output instance is named target RHS (right-hand side) instance. This is the only time during the whole Smart Matching process where human input is necessary. 2. Generate initial alignments. Existing matching tools create initial alignments which may serve as input for the Smart Matcher. Every tool which supports the INRIA alignment format [4] may be used. This phase is optional, i.e., the Smart Matcher is also able to start from an empty set of alignments. 3. Interpret initial alignments. The alignments produced in Phase 2 are transformed into an initial mapping model. This task is performed by the Mapping Engine. 4. Transform instances. In this phase, the Transformation Engine executes the initial mapping model and thereby transforms LHS instances into actual RHS instances. 5. Calculate diﬀerences. The Fitness Function compares the actual and the target RHS instances by means of their contained objects, values, and links. The diﬀerences between the instances are collected in a Diff-Model, which expresses the quality of the mappings between the source schema and the target schema. 6. Propagate diﬀerences. The diﬀerences, i.e., missing and wrong objects, values, and links, identiﬁed by the Fitness Function are propagated back to the Mapping Engine. The feedback is based on the assumption that schema elements are not appropriately mapped if diﬀerences are calculated for their instances. 7. Interpret diﬀerences and adjust the mapping model. The Mapping Engine analyzes the feedback and adapts the mapping model between source and target

4

M. Wimmer et al.

schema by searching for and applying appropriate mapping operators. Depending on the types of schemas to be integrated and the used mapping language, diﬀerent kinds of mapping strategies may be used. 8. Iteration. Now Phase 4 and Phase 5 are repeated. If the comparison in Phase 5 does not detect any diﬀerences or if a certain threshold value is reached, the process stops. Otherwise, the Smart Matcher’s Mapping Engine adapts the mapping model based on the identiﬁed diﬀerences and starts a new iteration by returning to Phase 4. In the following, we elaborate on the Fitness Function necessary for selfevaluation in Phase 5 and the Mapping Engine necessary in particular for the self-adaptation in Phase 7. Therefore, the next sections comprise detailed descriptions on the design rationale and implementation of these two components based on the example depicted in Fig. 2 (a). The schema on the LHS assigns a discount with a height to each customer who is characterized by name and famName. The schema on the RHS describes similar information, namely the rebate height of a person. Only the family a person belongs to is modeled by its own class. A correct mapping between those schemas should detect that the attribute famName ought to be mapped to the class family what is not given by the mapping shown in Fig. 2 (b) which has been produced from the automatically computed alignments (cf. Fig. 2 (a)).

3

A Fitness Function for Mapping Models

The aim of the Fitness Function is to provide feedback on the quality of the mapping models to the Mapping Engine. In the transformation scenario, the quality of a mapping model indicates the correctness of the transformation of the input model to the corresponding output model. Using a test-driven approach where input and output pairs (e.g., cf. Fig. 2 (c)) are given, the generated output model is compared with a given target output model (cf. Fig. 2 (d)). The smaller the diﬀerences between the actual output model and the target output model are, the better the quality of the mapping model is. This raises the question how to compute expressive diﬀerences between actual and target instance models. When using object-oriented schemas which typically consist of classes, attributes, and references, diﬀerences on the instance level appear between objects, values, and links. In the following, we describe our heuristic-based comparison method which is suﬃcient to compute the necessary feedback for the mapping engine without applying expensive comparison algorithms. Comparing Objects. In general, objects have a unique ID within an instance model (cf. Fig. 2 (c), e.g., f1:Family). However, when we want to compare actual instance models with target instance models which are independently created, one may not assume that two objects with the same ID are equivalent. This is because often IDs are arbitrarily assigned, e.g., based on the order the objects are instantiated. Therefore, we cannot rely on exact ID-based

On Realizing a Framework for Self-tuning Mappings LHSSchema

RHSSchema

SchemaMatching Output

Discount height ggets

~0.7

height

~0.6

gets

Discount height

Rebate height

A2A

gets

gets R2R

Person

~1.0

Person

label

Customer

~0.45

LHSInstances

c1:Customer name =homer famName =simpson

Family label

name famName

(c)

p1:Person

((d)) RHSActual RHS Actual Instances

p1:Person f1:Family

label =simpson

RHS Target Instances RHSTargetInstances

p1:Person label =homer

label =simpson

p2:Person

name =marge famName =simpson

label =marge

c3:Customer

p3:Person

f2:Family

p3:Person

name =ned famName =flenders

label =ned

label =flenders

label =flenders

height =10%

Family label

c2:Customer

d1:Discount

belongsTo g A2A

(a) (b)

RHSTargetInstances g

label =homer

label

C2C

Customer

belongsTo

name famName

RHSSchema

InitialMappingModel C2C

Rebate

~0.9

LHSSchema

p2:Person label =simpson

r1:Rebate height =10%

5

r1:Rebate height =10%

p2:Person

f1:Family label =simpson

label =marge

p3:Person

f2:Family

label =ned ned

label =flenders = flenders

r1:Rebate height =10%

Fig. 2. Running example: (a) alignments, (b) initial mapping model (c) user-deﬁned test instances, (d) actual vs. target instances for initial mapping model.

comparison approaches, instead we have to apply a heuristic. The weakest condition, which is necessary but not suﬃcient, for verifying that two instance models are equivalent is to count the objects of a particular class in each instance model and compare the resulting numbers, i.e., compute the cardinalities for a given class. Consequently, having the same amount of objects for each class of the RHS schema (abbr. with rhsSchema) in the actual instance model (abbr. with actual) and in the target model (abbr. with target), is an indication that the applied mappings between the classes of the schemas ought to be correct and complete. This consideration leads to the following condition. Condition 1 := forall classes c in rhsSchema | actual.allObjects(c).size() = target.allObjects(c).size()

Because Condition 1 is not suﬃcient, it is possible that in certain cases the amount of objects is the same for a given class but the mappings are not correctly deﬁned. Assume that for example a LHS class has a mapping to a RHS class, which is instantiated twice, but actually the LHS class should be mapped to another RHS class which is also instantiated twice. However, due to the fact that the RHS classes have both the same amount of instances, no diﬀerence can be determined when considering Condition 1 only. Thus, the mapping between the classes are falsely interpreted as correct. To be sure that two objects are equal, a deep comparison is necessary, meaning that attribute values and links of the objects under consideration have to match. Using deep comparison, this kind of false mappings between classes can be detected.

6

M. Wimmer et al.

Example. Fig. 2 (d) illustrates on the LHS the actual instances generated by the interpretation of the automatically computed alignments and on the RHS the given target instances. When Condition 1 is evaluated on these two models, the cardinalities of the classes Rebate and Person are the same for the actual model and for the target model. However, the cardinality of the class Family is obviously not equivalent for both instance models. The Family objects are totally missing in the actual model. Comparing Values. Having the same cardinalities for a speciﬁc class is a prerequisite for verifying that actual and target instance models are the same. To be sure that two objects, one in the actual and the other in the target instance model, are equal, these objects must contain exactly the same attribute values. Therefore, for each attribute of the RHS schema two sets of values are computed. The ﬁrst one comprises all values of a speciﬁc attribute for the actual model and the second one does the same for the target model. In case these two sets comprise the same elements, the mapping for the RHS attribute seems to be correct. Otherwise the attribute mapping is assumed to be incorrect or missing. Condition 2 := forall attributes a in rhsSchema | actual.allValues(a) = target.allValues(a)

Example. When Condition 2 is evaluated for our running example, one can easily see that for the attribute Rebate.height Condition 2 holds. However, for the attribute Person.label the computed sets are totally diﬀerent, namely for the actual instances the resulting set is {simpson, simpson, flenders} which has no match with the computed set for the target instances {homer, marge, ned}. Furthermore, Condition 2 does not hold for Family.label, because in the actual model there are no Family objects that may hold such values. Comparing Links. The last indication that the actual and target instance models are equal, and consequently that the mapping model between the schemas is correct, is that the structure of the instance models is the same, i.e., the links between objects must be equivalent. In particular, equal objects must have the same amount of incoming and outgoing links, which is in general hard and expensive to prove. In order to provide a fast comparison method for links, we decided only to check if the references of the RHS schema have the same amount of instantiated links. Condition 3 := forall references r in rhsSchema | actual.allLinks(r) = target.allLinks(r)

Example. Considering our running example, Condition 3 holds for the reference Person gets Rebate. However, for the reference Person memberOf Family Condition 3 does not hold. In the actual model we have no instantiations of this reference, thus the result of the allLinks operation is the empty set, whereas in the target model the operation allLinks produces a set with three entries. Preparing the feedback for the mapping engine. After evaluating these three conditions for all RHS schema elements, the feedback for the Mapping

On Realizing a Framework for Self-tuning Mappings

7

Engine is deﬁned as the set of all RHS schema elements which do not fulﬁll the aforementioned conditions. For the adaptation of the mapping model done by the Mapping Engine, we decided that it is currently suﬃcient to know the schema elements for which diﬀerences on the instance level have been found. Therefore, these schema elements are collected for the feedback only, and not the actual diﬀerences, i.e., objects, values, and links. Concerning our running example, the following set is computed as feedback for the Mapping Engine {Family, Person.label, Family.label, Person partOf Family}.

4

A Feedback-Aware Mapping Engine

After the Fitness Function has computed the feedback, the Mapping Engine interprets it and adapts the current mapping model to improve its quality. In order to adapt mapping models in an appropriate and eﬃcient way, we make use of two kinds of strategies. First, local strategies are needed to ﬁnd out if a given mapping operator may be applied appropriately for a set of given schema elements. Second, a global strategy is needed to guide the mapping process in general and in particular for orchestrating the local strategies. 4.1

Local Strategies

As depicted in the Smart Matcher architecture (cf. Fig. 1), the LHS instance models are transformed into RHS actual instance models. Subsequently, the difference between the actual and target instance models is computed on the RHS which is then propagated back to the Mapping Engine. Consequently, the Smart Matcher always transforms form left to right but adapts the mapping models from right to left. For adaptation purposes, the Mapping Engine has to apply new mappings which can also replace existing ones. However, before a new mapping is applied, it has to be ensured that the selected mapping operator is applicable for a given set of schema elements, i.e., the mapping model should be executable and the execution should result in consistent RHS instances. In particular, each mapping operator needs a speciﬁc LHS structure and RHS structure as well as already applied mapping operators as context. Based on these constraints, local strategies can be derived for determining if a given mapping operator can be applied in a particular situation. Therefore, we have extended our mapping operators with a isApplicable method which checks for given LHS and RHS schema elements if a mapping can be applied in the current state of the mapping model. For determining the implementation of the isApplicable methods, information inherent in the mapping language is needed. This means, the local strategies can be derived from the abstract syntax and static semantic constraints of the mapping operators which have to be assured to generate correct and executable mappings. If the isApplicable method returns true for given LHS and RHS elements, the mapping operator can be applied. Thus, with the help of the local strategies, we are able to automatically build mappings between the schemas. However, until now, we have not deﬁned how a mapping model is actually build. Therefore, in addition to the local strategies, a global strategy is needed which guides the overall mapping process.

8

M. Wimmer et al.

4.2

Global Strategies

Before a global strategy can be derived, one ﬁrst has to think about how mapping models are manually developed. Generally, mapping models are developed in a three-step process. First, some prototypical mappings are established. Second, these mappings have to be interpreted, i.e., in the transformation scenario the RHS instances are generated from the LHS instances. Third, the output of the transformation has to be veriﬁed in order to be sure that the mappings were correctly developed. For automation purposes, these steps have to be covered by the global strategy. However, what aggravates the derivation of the global strategy is that there a several variations how to proceed in each step. For example, one user may prefer to build several mappings between classes in the ﬁrst place before mappings between attributes and references are created. Whereas, another user may prefer to ﬁnd only one mapping between two classes and then tries to ﬁnd all attribute mappings for these two classes. These examples already show that the derivation of a global strategy is not as predetermined as the derivation of the local strategies. In particular, the following variation points for global strategies exist. – Sequence of mapping operator applications. The ﬁrst decision that has to be made is the sequence in which the diﬀerent types of mapping operators are applied. One possibility may be ﬁrst ﬁnding structural commonalities between the schemas by applying symmetric operators, followed by bridging structural heterogeneities by applying asymmetric operators. – Depth-first vs. breadth-first search. Additionally to the application sequence of mapping operators, the global strategy may decide to ﬁnd all C2C mappings ﬁrst and afterwards A2A and R2R mappings are explored (breadth-ﬁrst ) or ﬁnd iteratively one C2C mapping and for this mapping all possible A2A and R2R mappings are searched (depth-ﬁrst ). – All-at-once vs. incremental evaluation. Finally, it has to be decided, when and how often mappings are evaluated. The global strategy may ﬁnd several possible mappings and evaluate them all together (all-at-once) or evaluate each found mapping individually (incremental ). The all-at-once approach is similar to the waterfall model in software engineering, with its disadvantage of spending much eﬀort when working in the wrong direction. This can be avoided with the incremental approach; however, much more evaluation steps are necessary. Because there are diﬀerent ways to deﬁne a global strategy, we decided to use a state-machine based implementation approach for global strategies. The major advantage of this approach is to separate basic functionality, such as transforming models, computing diﬀerences between models, as well as searching and applying mappings, from the global strategy guiding the overall mapping process. This allows us to reconﬁgure the global strategy faster than compared to reconﬁgurations on source code level, which is especially needed to empirically analyze various conﬁgurations of the aforementioned variation points. In the following, we present the most eﬃcient conﬁguration explored in our empirical

On Realizing a Framework for Self-tuning Mappings a

9

b [termination condition]

Init

[[not RHS_containClass && not RHS_containAtt && RHS_containRef]]

entry / computeRHS() entry / computeLHS()

[not RHS_containClass && RHS_containAtt]

[RHS_containClass]

Select RHS Class [inC2C_BL && not inA2C_BL] inA2C BL] [not inC2C_BL]

Search 4 LHS Class [not mapFound]

Search 4 LHS Attribute

do / isApplicable()

entry / isApplicable()

[mapFound] /apply()

[mapFound] /apply()

Evaluation entry / transform() entry / evaluate()

[cMapEval2True && cMapHasAtt]

Select RHS Att

[cMapEval2False || not cMapHasFeat]

a

[not inA2A_BL] inA2A BL]

[not mapFound]

Search 4 LHS Att [cMapHasAtt]

do / isApplicable() [[mapound] d] /apply()

Evaluation entry / transform() entry / evaluate() [not cMapIsApproved && not cMapHasFeat] /setCMap_eval2False a

[ not cMapHasAtt && cMapHasRef]

[cMapIsApproved &&

b not cMapHasFeat]

Select RHS Ref [not inR2R_BL]

[not mapFound]

Search 4 LHS Ref [cMapHasRef]

do / isApplicable() [mapFound] /apply

Evaluation entry / transform() entry / evaluate() [not cMapIsApproved && not cMapHasRef] /setCMap_eval2False() a

[cMapIsApproved &&

b not cMapHasRef]

Fig. 3. Incremental depth-ﬁrst global strategy implemented as UML state machine

experiments. Fig. 3 illustrates this global strategy as an UML state diagram, which is a depth-ﬁrst and incremental strategy regarding to the aforementioned variation points. The depth-ﬁrst search also determines the application sequence of the mapping operators. Please note that only those parts of the global strategy are illustrated which are relevant for our running example. The ﬁrst task of the global strategy is to calculate the not yet correctly mapped LHS and RHS schema elements based on the output of the Fitness Function. Then, it proceeds with the establishment of mappings able to act as context mappings for other mappings. More speciﬁcally, the idea is to examine one mapping between a RHS class and an arbitrary LHS element, and then attributes and references of this RHS class are used to approve the correctness of this context mapping. This means, if a mapping for a RHS class is found and this mapping fulﬁlls the cardinality constraint (cf. Condition 1 in Section 3), in the next steps, mappings for its features, i.e., attributes and references, are searched. If at least one feature mapping is evaluated to true, i.e., fulﬁlls Condition 2 or 3 of Section 3, the context

10

M. Wimmer et al.

mapping is approved. For the opposite case, the context mapping is marked as false, although the cardinality constraints are fulﬁlled. After giving an overview on the design rational of this strategy, we discuss the three major phases of this strategy based on the state machine illustrated in Fig. 3. Finding context mappings. As already mentioned before, the local strategies search from right to left. Hence, after the Init state where not yet correctly mapped LHS and RHS schema elements are computed, a RHS class is selected in the state Select RHS Class to establish a context mapping. The ﬁrst choice is to select a LHS class to apply the C2C mapping operator (cf. state Search 4 LHS Class). If a C2C mapping has been found by the local strategy (cf. isApplicable method), it is applied and the state Evaluation is entered. Otherwise, the RHS class is stored in the C2C blacklist which has as impact that this RHS class will never be applied to a LHS class in future iterations (cf. condition [not in C2C BL]). Evaluating context mappings. Within the state Evaluate, the LHS instance model is transformed into a RHS actual instance model. After the transformation, the actual and the target instance models are compared and the output of the Fitness Function is analyzed. If the applied mapping is evaluated to true, e.g., for C2C mappings it is required that the cardinality constraint is fulﬁlled, consists of searching for feature mappings for the selected RHS class. In case the evaluation returned false, the RHS class and the LHS element are stored in the blacklist (i.e., this combination will never be applied again) and it is switched into the state Init. Approving context mappings. When the context mapping is evaluated to true, the current state is changed to Select RHS Att in which a not yet correctly mapped attribute of the RHS class is selected. Subsequently, the strategy looks for free LHS attributes. The structure and behavior of the following states for the application of attribute mappings is analogous to the aforementioned application of context mappings. After the evaluation of the found A2A mapping (cf. state Evaluation), three cases have to be distinguished. First, the A2A mapping is evaluated to true and the RHS class has no references, which results in an approved context mapping, i.e., it is not changeable in future iterations, and in a transition to the Init state. Second, no A2A mapping has been evaluated to true and the RHS class has no references which results also in a transition to the Init state but the context mapping is marked as evaluated to false, i.e., it is changeable in future iterations. Third, the RHS class has additional references, then the state is changed to Select RHS Ref independent of the evaluation of the A2A mapping. The mapping of the RHS references and approving of the context mapping is analogous to that of attributes. Running Example. For exemplifying the execution behavior of the presented global strategy, we make use of the running example. In Section 3, we have elaborated on the output of the Fitness Function for the initial mapping model which is summarized in Iteration 0 of Fig. 4. Now, it is explained how the global

On Realizing a Framework for Self-tuning Mappings Iteration 0

Iteration 1

C2C

Rebate

Discount A2A

height

Rebate

gets

R2R

height

A2A

heeight

gets

gets

C2C

D Discount

height

gets R2R

Person

Person Customer name famName

label

C2C

C Customer

belongsTo

naame famName

Family

A2A

label

C2C

belongsTo

Family l

A2C

label

label

c2:Customer name=marge famName=simpson p

p1:Person

p1:Person

label=simpson

label=homer

p2:Person 2P

T

c3:Customer

c1:Discount

label simpson label=simpson

p3:Person 3P

p3:Person

label=flenders

label=ned

f2:Family label=flenders

r1:Rebate

r1:Rebate

height=10%

height=10%

Feedback

p1:Person

nam me =homer h fam mName =simpson

c c2:Customer nam me=marge fam mName=simpson

label=marge

p2:Person

T

Rebate

gets

label=homer

f1:Family

p2:Person

label = simpson label=simpson

f2:Family

p3:Person

f3:Family

p3:Person 3P

f2 F il f2:Family

label =ned

label=ned

label=flenders

gets

R2R

C2C

Rebate height gets R2R

Person

Person Customer name famName

label

name n f famName

Family

label

C2C

Customer

belongsTo A2C agg

belongsTo

Family

A2C

label

label

Target

Actual

c1:Customer

p1:Person p

name =homer h famName =simpson

c2:Customer name=marge famName=simpson p

p2:Person p

p1:Person p

label =homer

label=homer

f1:Family

p2:Person

label=simpson

label =marge

p3:Person p

c3:Customer

Diff

f3:Family y

p3:Person

f2:Family

label =ned

label=ned

label=flenders

r1:Rebate

name =ned = ned famName =flenders

r1:Rebate

height=10%

height=10%

d1:Discount

Feedback

height=10%

{Person.label, Family, Family.label, Person.belongsTo}

Iteration 4

Rebate

name famName

c3:Customer

p3:Person

name =ned famName =flenders

label =simpson

Diff

{Person.label}

f1:Family

p2:Person

label=simpson

label=marge

f2:Family

p3:Person

f2:Family

label=ned

label=flenders

height=10%

Feedback

l b l h label=homer

label =flenders

r1:Rebate

d1:Discount height=10% e g t 0%

T Target t

p1:Person f1 F il f1:Family

p2:Person

p3:Person p label=ned

f2:Family y label=flenders

r1:Rebate

height=10%

height=10%

{Person.label, Family, Family.label, Person.belongsTo}

C2C

Rebate height

A2A

name n f famName

A t l Actual

T

f3:Family label =flenders

gets R2R

Person

p1:Person

c2:Customer

label=marge

Feedback

r1:Rebate height=10%

belongsTo A2A

Familyy

A2C agg

c c2:Customer

label

T Target t

A t l Actual

c c1:Customer

p1:Person

p1:Person

name =homer fam mName =simpson

name=marge mName=simpson fam

label

C2C

Customer

Family

c1:Customer

label=simpson

Diff

r1:Rebate

label

label

name=marge famName=simpson

p3:Person

d1:Discount

gets

belongsTo

name =homer famName =simpson simpson

f1:Family

p2:Person

f2:Family

c c3:Customer

h i ht height

R2R

A2C agg gg

p1:Person label=homer

Discount

height g gets

C2C

f1:Family label =simpson

label =simpson

name =ned fam mName =flenders = flenders

Person C t Customer

p2:Person

T

Iterration 5

C2C

A2A gets

c c2:Customer

height=10% he i h 10%

Discount height

p1:Person

name =homer fam mName =simpson

name =marge fam mName =simpson

label=marge

Target

Actual

c c1:Customer

f1:Family y f2:Family y

T

height=10%

Feedback {Person.label, Family, Family.label, Person.belongsTo}

A2A

h height

r1:Rebate

height=10%

d1 1:Discount

gets

C2C

label=marge

r1:Rebate

Discount

h i ht height

Diff

c c3:Customer

Iterration 3

C2C

A2A

p1:Person

label =homer

nam me =ned = ned fam mName =flenders

heigght=10%

Discount height

f1:Family

label =marge

{Person.label, Family, Family.label, Person.belongsTo}

Iteration 2

Target

Actual

c1:Customer f1:Family

p2:Person

Diff

label=simpson

name =ned = ned famName =flenders

height=10%

Target

Actual

c1:Customer name =homer h famName =simpson

11

T

label = homer label=homer

f1 F il f1:Family

p2:Person

label =simpson

Diff

l b l label=marge

c c3:Customer

label = homer label=homer

f1 F il f1:Family

p2:Person

label=simpson

l b l label=marge

p3:Person

f2:Family

p3:Person

l b l label=ned d

l b l =flenders label fl d

label=ned

r1:Rebate

name =ned f mName fam N =flenders fl d

height=10%

d1:Discount

Feedback

height=10%

{}

f2:Family label=flenders

r1:Rebate height=10%

Fig. 4. The Smart Matcher in action

strategy adapts the mapping model based on this input. In addition, the necessary iterations are illustrated in Fig 4. Iteration 1 and 2. When we reconsider the computed output of the Fitness Function, we ﬁnd out that only one RHS class is not correctly mapped. In particular, the class Family has currently no mapping to a LHS element. Therefore, the global strategy switches from the Init state to the state Select RHS class

12

M. Wimmer et al.

in which the Family class is selected. On the LHS schema no free classes are available now. Thus, it is not possible to apply a C2C mapping and the global strategy searches for a free attribute on the LHS to apply a A2C mapping without aggregation. The Customer.name attribute is selected and an A2C mapping is applied. Subsequently, the Evaluation state is entered, the transformation is started, and the produced actual model is compared with the given target model. The evaluation results amongst others into diﬀerences for Family instances which is an indicator that the applied mapping is not correct. Thus, the found mapping is put onto the blacklist and the Init state is entered. Because there is another conﬁguration possibility for the A2C operator, in iteration 2, an A2C mapping with aggregation is applied which is also not correct and therefore we end up again in the Init state. Iteration 3. The Init state computes the same unmapped RHS and LHS elements as in the previous iterations. Thus, we enter again the Select RHS class state where the class Family is selected. Again, no class on the LHS is free and a LHS attribute is selected. In this iteration, we already know that Customer.name is not the corresponding element. Therefore, another free attribute is searched which results in selecting Customer.famName. After building the A2C mapping marked without aggregation between Family and Customer.famName, the Evaluation state is entered. This time, the evaluation shows that the cardinalities of the class Family are not equal, but some attribute values overlap. However, for evaluating this mapping to true, the similarity value is too low. Therefore, the next iteration is started by switching to the Init state. Iteration 4. After entering the Init state, we have again the same set of incorrectly mapped RHS elements. Thus, the same path is followed as before. But this time, the applied A2C mapping is marked with aggregation. This means, only for unique values new objects are created. The evaluation of this mapping shows that no diﬀerences may be found for the focused RHS elements and the mapping is marked as evaluated to true. Because the Family class has no further unmapped features, the state Init is entered. Iteration 5. This time, the Init state computes that only one RHS attribute is not appropriately mapped. Therefore, we directly navigate to the Select RHS Att state in which Person.label is selected. Within the next state, Customer. name is selected, because this is the only free LHS attribute. The A2A mapping is applied and the evaluation computes no diﬀerence between the actual and target models. Thus, the whole mapping process terminates.

5

Evaluation

In the project ModelCVS1 which was aimed at ﬁnding solutions for the integration of modeling tools, the need for automatic model transformations arose. 1

www.modelcvs.org

On Realizing a Framework for Self-tuning Mappings

13

Fig. 5. The development of quality indicators over time

Due to the lack of appropriate tools for metamodel matching, and the strong relatedness of metamodels and ontologies [11], models were lifted to ontologies, and ontology matching tools where applied. Unfortunately, the quality of the resulting alignments did not meet our expectations [6] and we could not deduce reliable mappings for model transformation. Consequently, the Smart Matching approach was established. Although developed with a certain application domain in mind, the result was a generic matching tool which is not restricted to metamodels only. In the following we present results of a case study where elements of the UML class diagram 1.4 metamodel are matched to elements of the UML class diagram 2.0 metamodel. Both metamodels consist of about 60 elements. The user-deﬁned instance models describe a gas station with about 40 elements for the source model and about 30 elements for the target model. For a detailed description of the setting please refer to our project page2 . Starting from an empty alignment set, the Smart Matcher achieves a precision of 0.963 and a recall of 0.897 resulting in an f-measure of 0.929. When we translate the UML class diagram 1.4 terms into German and match it against the English UML class diagram 2.0, we obtain the same results as the Smart Matcher follows an uninterpreted mapping approach. In contrast, COMA++, the most successful system in our matching systems evaluation [6], yields alignments with a precision of 0.833, a recall of 0.583, and an f-measure of 0.683. The results of the German UML class diagram 1.4 matched with the UML class diagram 2.0 clearly indicate the relevance of similarity information for COMA++: the precision decreases to 0.368 and the recall is 0.117 with an f-measure of 0.178. Fig. 5 shows the evolution of precision, recall, and f-measure with respect to the number of Smart Matching iterations. Whereas the precision value tends abruptly towards one, recall and f-measure steadily increase. At iteration 91, the precision slightly decreases. The found mapping is interpreted as correct with respect to the provided instance models, although it is not. This is due to our heuristic for references, which is not suﬃcient when two references which have the same amount of links are between the same classes. For additional information on the case study, we kindly refer again to our project page. 2

http://big.tuwien.ac.at/projects/smartmatcher

14

6

M. Wimmer et al.

Related Work

The enormous demand for matching systems in order to automatically solve information integration problems led to a vast number of diﬀerent approaches (for surveys cf. [5,10]). Following the classiﬁcation of [10] the Smart Matcher realizes a schema-based approach where user-deﬁned instances are employed to verify and to improve the found alignments iteratively. In this sense, the Smart Matcher is based on supervised machine learning techniques. In addition to the source and the target schema, training data consisting of input/output pairs is provided and a relationship model is generated which maps the input instances to the output instances. Several systems have been proposed which incorporate machine learning techniques in order to exploit previously gathered information on the schema as well as on the data level. For example, LSD, Autoplex, and Automatch use Naive Bayes over data instances, SemInt is based on neural networks, and iMap analyses the description of objects found in both sources by the establishment of a similarity matrix (see [5] for an overview). The probably closest approach to ours is SPICY. Like the Smart Matcher, SPICY [2] enhances the mapping generation process with a repeated veriﬁcation phase by comparing instances of the target schema with transformed instances. The similarity in conceptual architecture of these two approaches contrasts to the signiﬁcant diﬀerences in their realization as well as the moment a human user has to intervent. Whereas SPICY only deals with 1:1 and 1:n alignments, the Smart Matcher is able to resolve more complex heterogenities due to the integration of the CAR-language. This results in very distinct search, comparison, and evaluation strategies. Furthermore SPICY assumes that instances of the target schema are a priori available (e.g. from a Web data source). In contrast, for the application of the Smart Matching approach a human user must explicitly engineer the same instances in the source and in the target schema as the Smart Matcher implements the idea of unit testing. When SPICY terminates, it returns a ranked list of transformations which has then to be evaluated and veriﬁed by the user. When the Smart Matcher terminates, the resulting mappings are correct and complete with respect to the source and target input instances. Finally, the eTuner [9] approach automatically tunes the conﬁgurations of arbitrary matching systems by creating synthetic schemas for which mappings are known. Hence, in contrast to the Smart Matcher which tunes the search process itself, the eTuner adjusts other systems externally in order to increase the accuracy of the found alignments.

7

Conclusion and Future Work

In this paper we have presented the realization of the Smart Matcher, which implements a generic method for the automatic detection of executable mappings between a source schema and a target schema. During the whole search process, human intervention is only necessary once, namely at the beginning. The

On Realizing a Framework for Self-tuning Mappings

15

user has to deﬁne instances which represent the same real world issue in the language of the source schema as well as in the language of the target schema. Those instances allow a permanent evaluation and an incremental improvement of the found mappings by the comparison of the user given target instances with the generated actual instances. The Smart Matcher’s feedback-driven approach guarantees that the resulting mappings are sound and complete with respect to those instances. Therefore an accurate engineering of the instances is inevitable for the successful application of the Smart Matcher, but the eﬀort spent in the preparation phase is compensated with less eﬀort necessary in later phases. For many schemas (e.g., UML diagrams, ontologies, database schemas) the creation of the instances is supported by comfortable editors and once the instances are established for one schema, those instances may be reused in multiple integration scenarios. Furthermore, the required size of the instances does not need to be large for obtaining a satisfying quality of the mappings, as we have experienced in our case study. In future work, we will focus on the development of appropriate methods for the deﬁnition of the example instances in order to provide recommendations and in order to guide the user during the preparation phase. Furthermore, we plan more extensive comparisons with other systems, especially with those which incorporate learning techniques. Concerning the Smart Matcher itself, we plan to improve the search strategies by enhancing them with sophisticated heuristics.

References 1. Bernstein, P.A., Melnik, S.: Model Management 2.0: Manipulating Richer Mappings. In: Proc. of the 2007 ACM SIGMOD Int. Conf. on Management of Data, pp. 1–12. ACM Press, New York (2007) 2. Bonifati, A., Mecca, G., Pappalardo, A., Raunich, S., Summa, G.: Schema Mapping Veriﬁcation: The Spicy Way. In: Proc. of the 11th Int. Conf. on Extending Database Technology (EDBT 2008), pp. 85–96. ACM Press, New York (2008) 3. Erdogmus, H., Morisio, T.: On the Eﬀectiveness of Test-ﬁrst Approach to Programming. IEEE Transactions on Software Engineering 31(1), 226–237 (2005) 4. Euzenat, J.: An API for Ontology Alignment. In: McIlraith, S.A., Plexousakis, D., van Harmelen, F. (eds.) ISWC 2004. LNCS, vol. 3298, pp. 698–712. Springer, Heidelberg (2004) 5. Euzenat, J., Shvaiko, P.: Ontology Matching. Springer, Heidelberg (2007) 6. Kappel, G., Kargl, H., Kramler, G., Schauerhuber, A., Seidl, M., Strommer, M., Wimmer, M.: Matching Metamodels with Semantic Systems—An Experience Report. In: Workshop Proc. of the 12th GI-Fachtagung Datenbanksysteme f¨ ur Business, Technologie und Web (BTW 2007), pp. 38–52. Verlag Mainz (2007) 7. Kappel, G., Kargl, H., Reiter, T., Retschitzegger, W., Schwinger, W., Strommer, M., Wimmer, M.: A Framework for Building Mapping Operators Resolving Structural Heterogeneities. In: Proc. of the 2nd Int. United Information Systems Conf. (UNISCON 2008). LNBIP, vol. 5, pp. 158–174. Springer, Heidelberg (2008) 8. Kargl, H., Wimmer, M.: SmartMatcher—How Examples and a Dedicated Mapping Language can Improve the Quality of Automatic Matching Approaches. In: Proc. of the 2nd Int. Conf. on Complex, Intelligent and Software Intensive Systems (CISIS 2008), pp. 879–885. IEEE Computer Scociety, Los Alamitos (2008)

16

M. Wimmer et al.

9. Lee, Y., Sayyadian, M., Doan, A., Rosenthal, A.: eTuner: Tuning Schema Matching Software Using Synthetic Scenarios. VLDB Journal 16(1), 97–122 (2007) 10. Rahm, E., Bernstein, P.: A Survey of Approaches to Automatic Schema Matching. VLDB Journal 10(4), 334–350 (2001) 11. Wimmer, M., Reiter, T., Kargl, H., Kramler, G., Kapsammer, E., Retschitzegger, W., Schwinger, W., Kappel, G.: Lifting Metamodels to Ontologies: A Step to the Semantic Integration of Modeling Languages. In: Nierstrasz, O., Whittle, J., Harel, D., Reggio, G. (eds.) MoDELS 2006. LNCS, vol. 4199, pp. 528–542. Springer, Heidelberg (2006)

Programming Models for Concurrency and Real-Time Jan Vitek Computer Science Department Purdue University [email protected]

Abstract. Modern real-time applications are increasingly large, complex and concurrent systems which must meet stringent performance and predictability requirements. Programming those systems require fundamental advances in programming languages and runtime systems. This talk presents our work on Flexotasks, a programming model for concurrent, real-time systems inspired by stream-processing and concurrent active objects. Some of the key innovations in Flexotasks are that it support both real-time garbage collection and region-based memory with an ownership type system for static safety. Communication between tasks is performed by channels with a linear type discipline to avoid copying messages, and by a non-blocking transactional memory facility. We have evaluated our model empirically within two distinct implementations, one based on Purdue’s Ovm research virtual machine framework and the other on Websphere, IBM’s production real-time virtual machine. We have written a number of small programs, as well as a 30 KLOC avionics collision detector application. We show that Flexotasks are capable of executing periodic threads at 10 KHz with a standard deviation of 1.2us and have performance competitive with hand coded C programs.

M. Oriol and B. Meyer (Eds.): TOOLS EUROPE 2009, LNBIP 33, p. 17, 2009. c Springer-Verlag Berlin Heidelberg 2009

CIF: A Framework for Managing Integrity in Aspect-Oriented Composition Andrew Camilleri, Geoﬀrey Coulson, and Lynne Blair Computing Department Lancaster University LA1 4WA, UK {a.camilleri,g.coulson,l.blair}@lancaster.ac.uk http://www.infolab21.lancs.ac.uk

Abstract. Aspect Oriented Programming (AOP) is becoming increasingly accepted as an approach to deal with crosscutting concerns in software development. However, AOP is known to raise software integrity issues. For example, join point shadows may easily omit crucial join points or include inappropriate ones. In this paper, we propose an extensible framework called CIF that constrains aspect-oriented software design and composition with the intent to maintain the integrity of the ﬁnal composed system. CIF controls the composition of aspects and the base application in three dimensions: where the composition occurs, how the composition is carried out and what exactly is being composed. The framework is intended to be used in a team-based software development environment. We demonstrate the applicability of the framework through an application case study.

1

Introduction

Aspect Oriented Programming [1] is becoming increasingly accepted as an approach to deal with crosscutting concerns in software development. However, the ﬂexible approach to modularity and composition enabled by AOP has a potential downside: it can easily result in systems that behave in unintended and unanticipated ways – i.e. the integrity [2] of the system can easily become compromised (examples are given in the following paragraphs). According to our analysis, integrity violations can be conveniently grouped under three orthogonal headings: i) where an aspect-oriented composition occurs, ii) how the composition occurs and iii) what exactly is being composed. ‘Where’ refers to the quantiﬁcation [3] of aspect composition as speciﬁed by a pointcut expression. Quantiﬁcation is about choosing the ‘shadows’ [4] within which composition will occur. The diﬃculty here lies in getting the strength of the quantiﬁcation right (and therefore the extent of the shadow). Over-strong (or over-speciﬁc) quantiﬁcation can lead to compositions that miss crucial join points, whereas weak quantiﬁcation can include unintended join points [5]. In M. Oriol and B. Meyer (Eds.): TOOLS EUROPE 2009, LNBIP 33, pp. 18–36, 2009. c Springer-Verlag Berlin Heidelberg 2009

CIF: A Framework for Managing Integrity in Aspect-Oriented Composition

19

either case the integrity of the resulting system can clearly be compromised. It is also worth noting that threats to system integrity posed by inappropriate quantiﬁcation are commonly tangled with issues of maintainability and reusability – e.g. there is often a strong desire for weak quantiﬁcation to help foster aspect reuse (i.e. to avoid ‘fragile pointcuts’ [6]). ‘How’ refers to the compositional semantics employed when weaving aspects; this is especially an issue where multiple aspects are woven at a common join point. Existing AOP systems provide considerable ﬂexibility in this area, oﬀering for example, options such as advice ordering; execution options like ‘before’, ‘after’ or ‘around’; compositional options like ‘call’ or ‘execution’; or aspect instantiation semantics such as ‘per-instance’, ‘per-class’, ‘per-call’ or ‘per-thread’. Inadvertent misuse of these dimensions of ﬂexibility can easily result in integrity violations. For example, if a ‘distribution’ aspect is inadvertently woven before an ‘authentication’ aspect at the same join point, then the resultant system could erroneously make unauthenticated calls. As another example, it is desirable to limit the use of ‘around’ advice: reasoning about aspects that use ‘around’ advice is hard because the advice developer has the luxury to forfeit the original join point (i.e. not call proceed()). Finally, ‘what’ refers to the functionality encapsulated in an aspect. Inadvertent inclusion of inappropriate behaviour in an aspect can again lead to integrity violations; for example, deadlock might result if an aspect unexpectedly accesses a resource shared by the base code. It is also important to have conﬁdence that a given aspect behaves as its name would suggest. For example, it could easily lead to confusion, especially in a team-based software development environment, if an aspect called ‘logging’ failed to call proceed() or used apparently inappropriate modules such as an authentication library. In general, therefore, the libraries and other code used by an aspect should be closely related to the associated concern. Diverging from this goal at the very least makes modular reasoning considerably harder. The contribution of this paper is to propose an extensible framework, called CIF (for “Composition Integrity Framework”), that constrains aspect-oriented software design and composition with the intent to avoid threats such as the above and therefore to maximise the integrity of the composed system. CIF is abstract and independent of any tool speciﬁc concepts (and as such is generally applicable to a range of AOP environments) and is designed to be employed in team-based software development environments. The constraint mechanisms oﬀered by the framework closely follow the above analysis in terms of ‘where’, ‘how’ and ‘what’. The rest of the paper is organised as follows. Section 2 introduces CIF’s basic abstractions. Section 3 illustrates the use of these abstractions in an application case study, and Section 4 discusses our current implementation of CIF. Section 5 surveys related work, arguing that while elements of the integrity issue have been addressed in the literature, no work has yet considered the issue suﬃciently comprehensively. Finally, Section 6 oﬀers our conclusions.

20

2 2.1

A. Camilleri, G. Coulson, and L. Blair

CIF’s Basic Abstractions: Domains, Realms and Configurations Overview

The main contribution of this paper is to propose an extensible ‘integrity framework’ called CIF for aspect-oriented software development. Our objective is to specify and enforce limits for aspects in a very general way. Our approach is to specify what is permissible rather than to explicitly restrict behaviour (this is in line with the well-accepted principle of fail-safe defaults [7]). Although our work to date has largely considered compile-time AOP, our framework is inherently agnostic as to compile-time versus dynamic AOP. In more detail, our approach deals with AOP integrity in terms of programmerdeﬁned, concern-speciﬁc, ‘boundaries’ or ‘ringfences’ that we refer to as domains. A domain scopes a speciﬁc concern and imposes its boundaries in terms of the ‘where’, ‘how’, ‘what’ analysis discussed in Section 1. Each aspect is then allowed to operate only within the limits of one or more associated domains.

Fig. 1. Composition Integrity Framework

On top of domains, CIF provides the higher-level abstraction of realms. The simplest function of a realm is to list and organise the set of domains deﬁned for a given system. But beyond that, realms provide a way for domains to be combined and inter-related while preserving domain autonomy. This allows us, for example, to specify which domains depend on each other (like authorization and authentication), or to specify weaving priorities in cases where a join point falls simultaneously within multiple domains. In essence, a realm provides an abstraction over a set of domains and a discipline to preserve the integrity of their composition. Finally, on top of realms, CIF oﬀers the coarser-grained abstraction of a conﬁguration which allows us to use the foregoing abstractions to deﬁne multiple

CIF: A Framework for Managing Integrity in Aspect-Oriented Composition

21

alternative instantiations or ‘builds’ of a whole application, each of which may have its own dedicated integrity constraints. Architecturally, the CIF approach is to avoid interfering with the either the base code of a system or its associated aspects. Instead, CIF domain, realm, and conﬁguration deﬁnitions sit alongside the system (see Figure 1) which remains oblivious to the existence of CIF. Essentially, CIF separates out the integrity concern (or ‘meta-concern’). CIF also operates throughout the development life cycle. In particular, the domain and realm abstractions constrain the design of aspects; and at build-time, a tool called the CIF veriﬁer (see Section 4) checks the conformance of aspect implementations to domain and realm speciﬁcations, while the conﬁguration abstraction (again, with tool support) helps ensure that application builds or deployments maintain pre-speciﬁed integrity constraints. CIF can even deal with run-time compositional constructs such as cflow. 2.2

Domains

In this and following sections we use a BNF-like notation1 to more precisely deﬁne the three main CIF abstractions and their supporting concepts. In terms of this notation, a domain is deﬁned in Figure 2.

Fig. 2. Domain

As can be seen, a domain consists of a shadow-region (cf. ‘where’), a concernregion (cf. ‘what’), and a set of permissable-compositional-styles (cf. ‘how’). Before discussing these in detail we ﬁrst expand on the general notion of a ‘region’. A region is a set of programmatic-element-instances in the target programming language environment such as speciﬁc classes, methods, ﬁelds, etc. The intention is that a region provides a means to ‘slice’ an application in some appropriate manner using a predicate, which picks up a speciﬁc set of 1

In this notation < > refers to a tuple; and { } to a set.

22

A. Camilleri, G. Coulson, and L. Blair

programming-element-instances. For example, in the case study that will be described in Section 3, we deﬁne predicates over the Java package structure of the system: healthwatcher.view.command.* in Figure 7 matches all the programmatic-element-instances (in this case classes) scoped within healthwatcher.view.command. Given this understanding of a region, a domain’s shadow-region refers to the set of programmatic-element-instances that fall within the scope of this domain. More speciﬁcally, when an aspect is assigned to this domain (see Section 2.4), and a pointcut expression is written to specify the composition of this aspect, the shadow deﬁned by that pointcut expression is constrained by CIF to fall within the domain’s shadow region. This is useful in a number of ways; for example, if not so constrained, a pointcut that targets the execution join points of a class may inadvertently extend over all of the class’s subclasses which may be implemented by diﬀerent team members who may be unaware of such a pointcut. The concern-region deﬁnes the ‘what’ of the domain; this aims to constrain the behaviour that embodies a particular concern. For example, an aspect implementing a distribution concern can be excluded from making use of the behaviour found in a database library. Finally, the permissable-compositional-styles constituent deﬁnes the ‘how’ of the domain. This is expressed as a combination of three other constructs: advicerelated-style deals with advice types, pointcut-related-style is a boolean expression that is written in terms of primitive-pointcut s, and instantiation-related-style deals with aspect instantiation options. Advice types are useful to constrain (using advice-related-style) in a number of scenarios. For example, a synchronization aspect using a semaphore should be restricted to use before and after for wait and release join points. (It is possible to use an around but we do not need to forfeit the original join point. In general, we should always strive for the simpler semantics since it reduces the possibility of introducing errors.) The pointcut-related-style is useful because it allows us to deﬁne any implicit assumptions which an advice might have. For example in AspectJ, the semantics of the implicit this2 object varies depending on captured join point [4]. We can easily violate integrity if we change a pointcut (in a manner that changes the context accessed through thisJoinPoint) without a corresponding change in how the corresponding this object is used. The arguments of an advice can also be a source of confusion and this can be alleviated by an explicit pointcut-relatedstyle. We may constrain the exposure of the arguments at a speciﬁc join point using the args pointcut instead of opting for an advice to access them using reﬂection. Similar reasoning applies for the instantiation-related-style. The beneﬁts of permissable-compositional-styles are especially visible in scenarios where the pointcut is deﬁned separately from the advice implementation; for example, AspectJ provides the use of abstract aspects. In such scenarios it is possible that implicit assumptions regarding the runtime context are violated (as described above), because each developer may have a diﬀerent intuition of 2

The same reasoning applies for the implicit target object and the arguments at a join point. In AspectJ these are all accessed through the implicit thisJoinPoint.

CIF: A Framework for Managing Integrity in Aspect-Oriented Composition

23

how composition should occur. This is especially true if we take into account the subtle diﬀerences between certain primitive pointcuts; the classic example here is call and execution (in AspectJ the reference to the implicit target object is diﬀerent for these pointcuts), but another possible example is within and cflow. These subtle problems are perhaps more common in environments that provide an asymmetric paradigm for the encapsulation of concerns [8,9]. As shown in Figure 2, the permissable-compositional-styles constituent is extensible and can comprise a range of mechanisms, the choice of which is dependent on the target AOP environment in which CIF is being applied3 . To summarise, the primary purpose of a domain is to constrain its associated aspect or aspects. We say that an aspect is suitably constrained by (conforms to) a domain iﬀ: 1. all shadows speciﬁed by pointcuts applied to the aspect are contained within the domain’s shadow-region, 2. all programmatic-element-instances used by the aspect’s advice are contained within the domain’s concern-region, 3. the aspect’s composition is constrained by its associated domain’s permissable-compositional-styles deﬁnition. Examples of the use of the domain abstraction are given in Section 3. 2.3

Realms

The pseudo-BNF for the realm abstraction is shown in Figure 3. As can be seen, a realm is a set of one or more domains plus a regime that constrains relations between the domains. This regime is a boolean expression that is written in terms of an extensible set of so-called integrity functions. The initial set of integrity functions that we have employed in our work to date is shown in Figure 4. These integrity functions are all of type boolean and can express a variety of constraints between domains. In general, we diﬀerentiate between two types of integrity functions: static and dynamic. The key diﬀerence is that static integrity functions perform checks that deal more closely to the internal integrity of CIF and thus are applied before the start of the weaving process; dynamic functions are applied during the weaving process, performing checks or possibly inﬂuencing it in some manner (more about this in Section 4). In Figure 4 all of the functions are static, except for precedence(). The function dependent() declares dependencies between domains: i.e. d1 is dependent on d2. This might be used, for example, to provide a regime for domains that respectively address authorisation and authentication – i.e. the ﬁnal system can only include authorisation (as a concern) if it also includes authentication. Similarly, the function exclusive() declares which domains are mutually exclusive i.e. d1 and d2 are mutually exclusive. For example, we may 3

Another possible ‘style’ that might be applied in an AspectJ [10] environment, for example, is the declare construct.

24

A. Camilleri, G. Coulson, and L. Blair

design a system that implements distribution as an aspect and want to leave open the possibility of using alternative distribution technologies (e.g. CORBA or Java RMI). In such a case we can construct separate domains for each technology and deﬁne them to be mutually exclusive. The closure() function determines the ‘level’ to which the domain’s concernregion is to be veriﬁed against an associated aspect. For example, with level = 0 only code directly included in the aspect’s advice is veriﬁed; with level = 1, veriﬁcation extends to the top level of all the modules and other programmaticelement-instances that are referenced from within the advice; with level = ∞ the closure of all code referenced directly or indirectly from the advice is veriﬁed.

Fig. 3. Realm

Fig. 4. Example Integrity Functions

Finally, the function precedence() imposes a weaving order for advice between domains that share common shadows (thus addressing an element of the ‘how’ dimension that could not be addressed in individual domain deﬁnitions). In more detail, precedence() deﬁnes the advice precedence of d1 over d2 ; the shadow set argument is used to scope the set of shadows at which the precedence relation should apply4 . To round oﬀ the discussion of realms, we can summarise by saying that the purpose of a realm is twofold. First, it gives us an abstraction over all the domains that are relevant for a speciﬁc application. Second, using regimes we can express constraints over domains with the intent to preserve the integrity of a ﬁnal application. In general, regimes (and their consituent integrity functions) are useful because they allow us to go beyond the programming language based constraints expressed using domains. Moreover, the fact that new integrity functions can be added provides CIF’s main mechanism for extensibility which enables us to cover arbitrary application-speciﬁc integrity issues. An example of the use of the realm abstraction is given in Section 3. 4

Incidentally, the precedence relation expressed as a sequence of aspects provided by most AOP environments, can be viewed as a special case of our precedence() function.

CIF: A Framework for Managing Integrity in Aspect-Oriented Composition

2.4

25

Configurations

The purpose of our ﬁnal abstraction, that of a conﬁguration, is to enable us to create diﬀerent application builds from a set of domains. For example, we can use conﬁgurations to deﬁne application builds that are supported by diﬀerent OSs, or debugging versions, or versions that include or omit security, etc. Intuitively, a conﬁguration provides us with an instantiation of a realm.

Fig. 5. Conﬁguration

In more detail, a conﬁguration, as deﬁned in Figure 5, speciﬁes: i) a set of all the domains that are relevant to a particular application build; ii) a congruent set of aspects that are to be associated with and constrained by those domains; and iii) a realm deﬁnition that speciﬁes the regime constraints associated with those mappings. Essentially, a conﬁguration embodies the following assertions: 1. that each aspect conforms to its associated domain (or domains – as we have seen, an aspect can be mapped to multiple domains), 2. that the given realm’s regime evaluates to true. An example of the use of the conﬁguration abstraction is given in Section 3. 2.5

CIF Syntax

Figures 2, 3 and 5 deﬁned CIF’s main concepts in an abstract manner. In practice, CIF speciﬁcations employ a concrete syntax. Due to lack of space, we do not describe this syntax formally; rather we introduce it by example. More specifically, examples of concrete domain, realm and conﬁguration speciﬁcations are given in Figures 7, 8 and 9. These CIF speciﬁcations are related to an application case study that will be described in Section 3. As can be seen in Figures 7, 8 and 9, the various constituents of the framework (regions, styles, etc.) each have their own corresponding labeled sections in a speciﬁcation, and each of these sections comprises a set of deﬁnitions in which CIF-deﬁned attributes are assigned values within curly brackets. Thus, Figure 7 shows a domains section which deﬁnes a number of domains (in this case two) together with their corresponding attributes – i.e. shadow and concern regions from the regions section, and permissible-compositional-styles from the compositional-styles section. The regions sub-section is in turn made up of region deﬁnitions that are associated with predicate-valued attributes that specify sets of programmaticelement-instances that should fall within the region being speciﬁed (the Method qualiﬁer speciﬁes that methods are the only programmatic element type of interest here). Similarly, the compositional-styles section is deﬁned in terms

26

A. Camilleri, G. Coulson, and L. Blair

of advice-related-styles (with options like before, around, etc.), pointcut-related-styles (with options such as call, execution, etc.) and instantiation-related-styles (with options like singleton, per-thread, etc.). Likewise, Figures 8 and 9 respectively deﬁne realms and conﬁgurations, which employ the same syntactic approach. The realms section depends on subsections dealing with regimes and integrity functions. The configurations section simply lists a set of conﬁgurations, each of which is deﬁned as a list of <domain, aspect> pairs.

3 3.1

Application Case Study Case Study Overview

The purpose of the following application case study is to further motivate CIF’s abstractions, but also to clarify the way we envisage CIF being used. The application on which the case study is based is discussed in detail in [11,12]. In outline, the application is called Health Watcher and is a health service complaint logging system which embodies a distributed client server architecture. It was originally implemented in Java, but later refactored to encapsulate crosscutting concerns using AspectJ. The refactored version employs aspects that address distribution, persistence and transaction concerns. The main reason for choosing this application (besides it being a real system) as our case study is the variety of conﬁgurations in which it is available, as shown in Figure 6 (which shows 4 distinct conﬁgurations). Thus, for example, the application could be accessed either through the web with a normal browser or using an application installed on the client side (thus explaining the need for diﬀerent distribution technologies). Moreover, given a speciﬁc platform, the application developers had to cater for diﬀerent options in terms of transaction and persistence mechanisms (in this case relational and object-oriented databases).

Fig. 6. Health Watcher Conﬁgurations

CIF: A Framework for Managing Integrity in Aspect-Oriented Composition

3.2

27

Using CIF’s Abstractions

The Health Watcher application provides an ideal ecosystem in which to encounter integrity violations because it is diﬃcult to manage even such a modest set of concerns, given their multiple implementations. It should be obvious that the integrity of the ﬁnal system can be easily violated if the corresponding aspects are not carefully combined together. CIF is useful in this situation because it gives an abstraction over the concerns and on how they can be combined. Turning to the speciﬁcs of the CIF speciﬁcation, Figure 7 deﬁnes appropriate domains for the application’s distribution concern (which we will employ as our primary example concern). Two domains are speciﬁed, one for the client side and one for the server side. The concern-region of rmiserver-Domain speciﬁes that only methods from the java.rmi.server package can be employed within aspects associated with this domain. This serves to eliminate, for example, an EJB based distribution implementation. The shadow region speciﬁed establishes an explicit set of programmatic-element-instances that constrain the compositional shadow for this domain. This prevents the aspects associated with this domain from accidentally composing with elements that are not listed in this region.

Fig. 7. Example Domains

The compositional-styles section constrains aspect oriented compositions for the client and the server, regardless of the concrete distribution technology. In this case, the pointcut-related-styles for the client and server side are constrained to use call and execution, respectively. This prevents us from confusing the advice arguments (exposed by a pointcut that is deﬁned separately

28

A. Camilleri, G. Coulson, and L. Blair

from the advice implementation) for arguments of intercepted methods. If the style for composition is constrained through compositional-styles then we can have a uniform compositional view for all distribution concerns and our assumptions will always be sound. For example, we can choose to explicitly expose the intercepted method arguments by constraining the use of the args pointcut, instead of relying on each advice to access them through thisJoinPoint. Finally, both domains constrain their advice-related-styles to be around because the distribution aspects need to marshal data (i.e. method arguments) on both requests and replies. Using a before or an after here would be an error, since we want to replace the original join point. The respective instantiation-related-styles are restricted to be singleton since we only want one running instance of the client and of the server. Again, having this kind of constraint capability is useful to maintain a consistency amongst the alternative concern implementations.

Fig. 8. Example Realms

Figure 8 illustrates realm deﬁnitions employed in the case study. As one can see, the regime is a conjunction of ﬁve separate predicates. The dependent() integrity function is used to state that if a transaction concern is to be employed, a persistence concern is also required (this is to ensure safe storage for uncommitted data). Similarly, the exclusive() function is used to disallow the simultaneous use of RMI and EJB distribution concerns. The closure() function speciﬁes that the full closure of the advice code of the server-side distribution aspect(s) should be veriﬁed for conformance to the server-side domain’s concern-region.

CIF: A Framework for Managing Integrity in Aspect-Oriented Composition

29

Fig. 9. Example Conﬁgurations

Finally, Figure 9 shows two sample conﬁgurations: one employs RMI-based distribution with object-oriented database persistence and transaction management; the other employs EJB distribution, with relational database persistence and transaction management. These two conﬁgurations correspond to Figures 6(a) and 6(d). The two conﬁgurations speciﬁed here correspond to RMI-based conﬁguration and EJB-based conﬁguration in Figure 10 (this ﬁgure gives an overview of the whole speciﬁcation process). It is important to note that the conﬁguration speciﬁcations are not eﬀectively specifying any additional integrity constraints. They simply allow us to create our ﬁnal application. Thus, given a speciﬁc realm, the mapping in the conﬁguration can be speciﬁed (perhaps using an appropriate GUI) by a deployer who may be ignorant of the details of implementation, but simply requires to build a system which includes a speciﬁc set of concerns. The conﬁguration allows us to do exactly this, while making sure that there are no integrity violations along the way. 3.3

Using CIF in a Team-Based Software Development Environment

The case study also enables us to illustrate how CIF can be applied in teambased software development environments. For example, consider a development environment that employs three major roles: i) a system architect, ii) a developer, and iii) a quality assurance oﬃcer. In such an environment, the system architect role would deﬁne the fundamental architecture of the application, plus CIF abstractions to constrain the subsequent deﬁnition and composition of the aspects envisaged in the design. In our particular case, this means deﬁning the application’s basic client-server architecture, and its associated domains (e.g. rmiclient-Domain and rmiserver-Domain, plus domains for the persistence and transaction aspects as shown in Figure 10). The system architect would also constrain the way that these domains inter-relate by deﬁning realms and associated regimes (e.g. in our case the system architect speciﬁes that RMI and

30

A. Camilleri, G. Coulson, and L. Blair

Fig. 10. Health Watcher with CIF

EJB communications are mutually exclusive and that transactional functionality is dependent on the presence of persistence functionality). The developer role would then implement the base modules and aspects identiﬁed by the system architect. When complete, the various modules and aspects would all be delivered to the quality assurance oﬃcer who would test the delivered aspects against their associated domains (using the CIF veriﬁer; see Section 4) and deﬁne prototype conﬁgurations (such as our RMI- and EJB-based conﬁgurations as shown in Figure 10) that exercise the system in various application-speciﬁc ways. Finally, the quality assurance role would feed back the results of testing these conﬁgurations (such as integrity violations) for reﬁnement to the systems architects and developers as appropriate.

4

Implementation

We have built a prototype implementation of CIF for the AspectJ language using AspectBench [13]. The implementation is logically structured in two parts: a static part and a dynamic part. The static part comprises a veriﬁer that ensures that aspects conform to domains, and that realm and conﬁguration speciﬁcations are self-consistent e.g. ensuring that any dependencies and mutual exclusion relations (speciﬁed using the dependent() and exclusive() integrity functions) are honoured. Our current implementation supports only “closure(0)” semantics – i.e. only toplevel code is checked for conformance with regions (regions themselves are computed with assistance from the AspectBench weaver). Other application-speciﬁc integrity functions can also be added. For example, in a team-based software development scenario where multiple architects collaborate to deﬁne domains, an ignore(Region r) function could be deﬁned to demarcate an application-level region that should never participate in the deﬁnition of any domain.

CIF: A Framework for Managing Integrity in Aspect-Oriented Composition

31

The second part of the implementation is a dynamic veriﬁer integrated with the AspectBench weaver, and uses callbacks to realise extensible weave-time checking behavior. Integrity functions that are integrated with this part are allowed to have side eﬀects. For example, a weaver callback associated with the precedence() integrity function directs weaving to occur according to precedences speciﬁed by this function. Figure 11 gives a layered view of our implementation. The CORE module implements the CIF’s central abstractions (domains, realms, and conﬁgurations) and performs the above-mentioned static consistency checks. Above this, the CIFParser module reads in and parses CIF speciﬁcations. The CORE and CIFParser modules are generic and entirely independent of any particular target AOP environment, the latter being represented primarily by the Weaver module. The generic parts are sheltered from the Weaver by semi-generic modules, namely the AspectLang, LangParser, and the DynamicVeriﬁer modules: AspectLang provides an AOP-environment-speciﬁc adaptation layer between the CORE and the target AOP environment; LangParser takes care of parsing the programming language speciﬁc aspects that will participate in the composition; and DynamicVeriﬁer provides the above-mentioned callback mechanism for integration with the weaver.

Fig. 11. CIF Implementation Architecture

We handle dynamic pointcuts (e.g. AspectJ’s cflow) in the following manner. Such pointcuts are commonly realised by inserting runtime checks (during compilation) at points (both for entry and exit control ﬂows) where composition might occur [14,15]. In that light, our strategy5 is simply to ensure that these checks are constrained to occur only within appropriately deﬁned shadow regions. This gives us an ‘upper limit’ for constraining dynamic composition without the need to be too restrictive. In future implementation work we plan to extend our exploration of runtime integrity support in dynamic environments. One key strategy here is to explore integrity functions that persist at runtime. For example, we could conceive of an integrity function that ensures that ‘around’ advice in a speciﬁed domain always calls proceed() [16,17]. 5

A possible alternative strategy could be to constrain only where composition should occur i.e. control ﬂow exit points.

32

A. Camilleri, G. Coulson, and L. Blair

We also plan to provide an automated tool that would inspect all the aspects and pointcuts in an existing system and come up with corresponding domain speciﬁcations. This would relieve us of the burden of having to specify domains from scratch; it would provide an initial deﬁnition of regions for each aspect, which can be viewed and reﬁned (at least by specifying integrity functions) as required. Complementary to this automated tool, we also plan to provide a GUI-based workbench to better evaluate the use of CIF in realistic software development environments.

5

Related Work

Although we argue that the literature has not yet comprehensively addressed the issue of integrity in AOP systems, we nevertheless acknowledge that a considerable body of work addresses a range of concerns within this general space. In terms of the where dimension, a number of approaches attempt to restrict the shadows speciﬁed by pointcut expressions. For example, [18,19,20,21] all oﬀer ways to abstract the set of join points accessible to pointcut expressions so as to raise the semantic level and thereby increase the precision of the quantiﬁcation. XPI [22] attempts to achieve similar ends through the use of an information hiding interface called ‘cross-cutting interfaces’. A complementary approach is adopted by [23] which oﬀers a ‘negative’ pointcutting mechanism that allows sets of join points to be explicitly excluded from quantiﬁcations. In general, although this work enables greater control over the ‘where’ dimension, it does not at all address the ‘how’ or ‘what’ dimensions. Furthermore, all the proposed approaches lack an equivalent concept for domains and therefore restrict quantiﬁcation in a coarse-grained manner: i.e. they do not allow per-aspect control over the quantiﬁcation shadow. This is an important omission because shadow management is naturally a per-concern issue. In terms of the how dimension, most work to date has focused on how multiple aspects woven at a common join point should relate to each other. For example, [24] and [25] provide a comprehensive catalogue of aspect interactions and suggest how unwanted interactions can be detected; [26] provides a systematic means of composing multiple aspects, and detecting and resolving potential conﬂicts; and [16] provides a domain-speciﬁc language to deﬁne execution constraints for aspect interactions. This work is largely complementary to ours in oﬀering greater semantic depth in speciﬁc focus areas. However, it does not address the comprehensive range of ‘how’ issues (e.g. accommodating ordering issues, aspect instantiation semantics, or execution options) that we cover (see Section 2). Our approach can also be viewed as an integrating framework for more focused work such as the above. In terms of the what dimension, [27] discusses the notion of a permission system for aspects – i.e. to control which parts of a system a given aspect can access; however, no concrete mechanisms are proposed. Other work (e.g. [28,29,30,31,32]) attempts to classify and categorise advice (e.g. in terms of whether it is read-only or, if not, the degree of invasiveness permitted). Our

CIF: A Framework for Managing Integrity in Aspect-Oriented Composition

33

approach, on the other hand (see Section 2), focuses on a ﬁner-grained restriction of the behaviour that is applicable for a given advice type (i.e. in terms of the modules or libraries that can be used by advice). Furthermore, although the classiﬁcations provided by [28,29] seem very general and widely applicable, they do not deal, as we do, with concern interactions (see Section 2). Also, although [30,32] provide an intuitive classiﬁcation, the reports generated by these tools require a lot of manual analysis, and each change requires the generation (and subsequent analysis) of a new report. Moreover, contrasting with [31], our approach does not require annotation of the underlying system. Finally, some research attempts to verify the behaviour of aspects in terms of static code analysis [33]. However this work (so far) only supports a limited source language, making it unsuitable for most real-world scenarios. “Design by contract” approaches can also be viewed as relating to the ‘what’ dimension. For example, the approach proposed by [34] divides aspects into ‘observers’ and ‘assistants’ to facilitate modular reasoning; and [35] provides a similar classiﬁcation of aspects. Also, [36] describes the notion of ‘aspect integration contracts’ which attempts to preserve integrity by equipping aspects with ‘required’ and ‘provided’ denotations. Although these approaches directly target the integrity issue (albeit only in the ‘what’ dimension) their downside is that they tend to be intrusive, verbose and diﬃcult to manage. In constrast, CIF avoids cluttering source code by factoring integrity speciﬁcation out of both base modules and aspects. Finally, MAO [17] is an interesting system that addresses both ‘what’ and ‘where’ issues, but using an approach quite diﬀerent from ours (see Section 2). In brief, they adopt a language-based annotation approach that enables programmers to explicitly characterise the eﬀects of aspects. In terms of ‘what’, the programmer can, for example, write annotations that ensure that proceed() is called, that return values are returned unchanged, or that an advice may not change arguments. In terms of ‘where’, the programmer can constrain the quantiﬁcation shadow of a given aspect. However, although MAO enables wideranging control over the integrity of an AOP system, it has signiﬁcant disadvantages that mainly accrue from its language-based approach (again, compare our ‘factored out’ approach). In particular, MAO annotations are potentially complex and may need to be applied to base code as well as aspect code; moreover it is always diﬃcult to ﬁnd general acceptance of new language-level constructs.

6

Conclusion

As AOP gains more ground, it becomes increasingly important to provide a means to maintain a tight grip on the compositional ﬂexibility it provides. We feel that the CIF approach oﬀers a very promising way to achieve this, in a very practical manner. We are encouraged with the results we have achieved so far, but we also realize that there is more work to be done. Perhaps the most signiﬁcant limitation of our design that we have discovered to date is the predicate-based region deﬁnition

34

A. Camilleri, G. Coulson, and L. Blair

approach. Experience has shown that this can be tricky to get right (especially using Java-inspired package predication) and may suﬀer from similar weaknesses as pointcuts – i.e. over strong or over weak predication. Currently, we are experimenting with alternative predication approaches and also with the integrity function facility to help minimise this weakness: for example, a ‘monitor’ function can keep track of complete enumeration (automatically, without any manual intervention) of programmatic elements picked out by speciﬁc regions and warn us when there are changes in this set. More generally, we believe that our approach to integrity management is unique in not restricting or even changing the aspect language or the base application: CIF simply sits alongside these two components, to provide the necessarily integrity-preserving machinery. It thus cleanly separates out the integrity concern (or ‘meta-concern’). Furthermore, we believe that our proposal is complementary to several of the more partial approaches discussed in Section 5, and that these could in future work be integrated into our approach. For example, XPIs [22] can be applied alongside CIF at design time to clarify and facilitate the selection of appropriate regions and compositional styles. Similarly, it seems feasible to accommodate the execution constraints deﬁned in [16] by encapsulating them in integrity functions. We can also include advice classiﬁcations similar to augmentation and replacement [30,31,32] in advice-related-style, thus extending them with compositional styles. Finally, it seems conceivable to relate contracts, as described by [36], to domains and to therefore provide stronger guarantees that aspects comply with concern-speciﬁc semantics.

References 1. Kiczales, G., Lamping, J., Mendhekar, A., Maeda, C., Lopes, C., Loingtier, J.M., Irwin, J.: Aspect-oriented programming, pp. 220–242 (1997) 2. Clark, D.D., Wilson, D.R.: A comparison of commercial and military computer security policies. SP, 184 (1987) 3. Filman, R.E., Friedman, D.P.: Aspect-oriented programming is quantiﬁcation and obliviousness. Technical report (2000) 4. Hilsdale, E., Hugunin, J.: Advice weaving in aspectj. In: AOSD 2004: Proceedings of the 3rd international conference on Aspect-oriented software development, pp. 26–35. ACM, New York (2004) 5. McEachen, N., Alexander, R.T.: Distributing classes with woven concerns: an exploration of potential fault scenarios. In: AOSD 2005: Proceedings of the 4th international conference on Aspect-oriented software development, pp. 192–200. ACM Press, New York (2005) 6. Stoerzer, M., Graf, J.: Using pointcut delta analysis to support evolution of aspectoriented software. ICSM, 653–656 (2005) 7. Saltzer, J.H., Schroeder, M.D.: The protection of information in computer systems. Proceedings of the IEEE 63, 1278–1308 (1975) 8. JBossAOP, http://www.jboss.org/jbossaop 9. Rajan, H., Sullivan, K.J.: Classpects: unifying aspect- and object-oriented language design. In: ICSE 2005: Proceedings of the 27th international conference on Software engineering, pp. 59–68. ACM, New York (2005)

CIF: A Framework for Managing Integrity in Aspect-Oriented Composition

35

10. AspectJ, http://www.eclipse.org/aspectj 11. Soares, S., Borba, P., Laureano, E.: Distribution and persistence as aspects. Softw. Pract. Exper. 36(7), 711–759 (2006) 12. Greenwood, P., Garcia, A., Rashid, A., Figueiredo, E., Sant’Anna, C., Cacho, N., Sampaio, A., Soares, S., Borba, P., Dosea, M., Ramos, R., Kulesza, U., Bartolomei, T., Pinto, M., Fuentes, L., Gamez, N., Moreira, A., Araujo, J., Batista, T., Medeiros, A., Dantas, F., Fernandes, L., Wloka, J., Chavez, C., France, R., Brito, I.: On the Contributions of an End-to-End AOSD Testbed. In: EARLYASPECTS 2007: Proceedings of the Early Aspects at ICSE, Washington, DC, USA, p. 8. IEEE Computer Society Press, Los Alamitos (2007) 13. AspectBench, http://abc.comlab.ox.ac.uk 14. Dinkelaker, T., Haupt, M., Pawlak, R., Benavides, L.D., Gasiunas, V.: Inventory of aspect-oriented execution models. Technical report, AOSD-Europe (2006) 15. Bockisch, C., Kanthak, S., Haupt, M., Arnold, M., Mezini, M.: Eﬃcient control ﬂow quantiﬁcation. In: OOPSLA 2006: Proceedings of the 21st annual ACM SIGPLAN conference on Object-oriented programming systems, languages, and applications, pp. 125–138. ACM, New York (2006) 16. Pawlak, R., Duchien, L., Seinturier, L.: CompAr: Ensuring Safe Around Advice Composition. In: Steﬀen, M., Zavattaro, G. (eds.) FMOODS 2005. LNCS, vol. 3535, pp. 163–178. Springer, Heidelberg (2005) 17. Clifton, C., Leavens, G.T., Noble, J.: MAO: Ownership and eﬀects for more eﬀective reasoning about aspects. In: Ernst, E. (ed.) ECOOP 2007. LNCS, vol. 4609, pp. 451–475. Springer, Heidelberg (2007) 18. Kiczales, G., Mezini, M.: Aspect-oriented programming and modular reasoning. In: ICSE 2005: Proceedings of the 27th international conference on Software engineering, pp. 49–58. ACM, New York (2005) 19. Aldrich, J.: Open modules: Modular reasoning about advice, pp. 144–168. Springer, Heidelberg (2005) 20. Gudmundson, S., Kiczales, G.: Addressing Practical Software Development Issues in AspectJ with a Pointcut Interface. In: Advanced Separation of Concerns (2001) 21. Lieberherr, K., Lorenz, D.H.: Aspectual Collaborations: Combining Modules and Aspects. The Computer Journal 46(5), 542–565 (2003) 22. Sullivan, K., Griswold, W.G., Song, Y., Cai, Y., Shonle, M., Tewari, N., Rajan, H.: Information hiding interfaces for aspect-oriented design. SIGSOFT Softw. Eng. Notes 30(5), 166–175 (2005) 23. Larochelle, D., Scheidt, K., Sullivan, K., Wei, Y., Winstead, J., Wood, A.: Join point encapsulation. In: AOSD 2003 Workshop on Software-engineering Properties of Languages for Aspect Technologies (2003) 24. Sanen, F., Truyen, E., De Win, B., Wouter, J., Loughran, N., Coulson, G., Rashid, A., Nedos, A., Jackson, A., Clarke, S.: Study on interaction issues. Technical report, AOSD-Europe (2006) 25. Katz, S., Katz, E., Havinga, W., Staijen, T., Taiani, F., Weston, N., Rashid, A., Sudholt, M., Ha Nguyen, D.: Detecting interference among aspects. Technical report, AOSD-Europe (2006) 26. Composition, reuse and interaction analysis of stateful aspects. In: AOSD 2004: Proceedings of the 3rd international conference on Aspect-oriented software development. ACM, New York (2004) 27. De Win, B., Piessens, F., Wouter, J.: How secure is aop and what can we do about it? In: SESS 2006: Proceedings of the 2006 international workshop on Software engineering for secure systems, pp. 27–34. ACM, New York (2006)

36

A. Camilleri, G. Coulson, and L. Blair

28. Sihman, M., Katz, S.: Superimpositions and aspect-oriented programming. The Computer Journal 46(5), 529–541 (2003) 29. Dantas, D.S., Walker, D.: Harmless advice. In: POPL 2006: Conference record of the 33rd ACM SIGPLAN-SIGACT symposium on Principles of programming languages, pp. 383–396. ACM, New York (2006) 30. Rinard, M., Salcianu, A., Bugrara, S.: A classiﬁcation system and analysis for aspect-oriented programs. SIGSOFT Softw. Eng. Notes 29(6), 147–158 (2004) 31. Munoz, F., Baudry, B., Barais, O.: Improving maintenance in aop through an interaction speciﬁcation framework. In: IEEE International Conference on Software Maintenance, ICSM 2008, September 28-October 4, pp. 77–86 (2008) 32. Zhang, D., Hendren, L.: Static Aspect Impact Analysis. Technical report, AspectBench (2007) 33. Krishnamurthi, S., Fisler, K., Greenberg, M.: Verifying aspect advice modularly. In: SIGSOFT 2004/FSE-12: Proceedings of the 12th ACM SIGSOFT twelfth international symposium on Foundations of software engineering, pp. 137–146. ACM, New York (2004) 34. Clifton, C., Leavens, G.T.: Observers and assistants: A proposal for modular aspect-oriented reasoning. In: Foundations of Aspect Languages, pp. 33–44 (2002) 35. Lorenz, D.H., Skotiniotis, T.: Extending design by contract for aspect-oriented programming. CoRR, abs/cs/0501070 (2005) 36. Lagaisse, B., Joosen, W., De Win, B.: Managing semantic interference with aspect integration contracts. In: International Workshop on Software-Engineering Properties of Languages for Aspect Technologies (2004)

A Diagrammatic Formalisation of MOF-Based Modelling Languages Adrian Rutle1 , Alessandro Rossini2 , Yngve Lamo1 , and Uwe Wolter2 1

Bergen University College, P.O. Box 7030, 5020 Bergen, Norway aru,[email protected] 2 University of Bergen, P.O. Box 7803, 5020 Bergen, Norway {rossini,wolter}@ii.uib.no

Summary. In Model-Driven Engineering (MDE) models are the primary artefacts of the software development process. The usage of these models have resulted in the introduction of a variety of modelling languages and frameworks. Many of these languages and frameworks are based on the Object Management Group’s (OMG) Meta-Object Facility (MOF). In addition to their diagrammatic syntax, these languages use the Object Constraint Language to specify constraints that are difficult to specify diagrammatically. In this paper, we argue for a completely diagrammatic specification framework for MDE, where by diagrammatic we mean specification techniques which are targeting graph-based structures. We introduce the Diagram Predicate Framework, which provides a formal diagrammatic approach to modelling based on category theory – the mathematics of graph-based structures. The development of a generic and flexible formalisation of metamodelling is the main contribution of the paper. We illustrate our approach through the formalisation of the kernel of the Eclipse Modeling Framework. Keywords: Model-Driven Engineering, Meta-Object Facility, Unified Modeling Language, Object Constraint Language, Eclipse Modeling Framework, Diagram Predicate Framework, diagrammatic specification.

1 Introduction Since the beginning of computer science, developing high-quality software at low cost has been a continuous vision. This has boosted several shifts of programming paradigms, e.g. machine code to compilers and imperative- to object oriented programming. In every shift of paradigm, raising the abstraction level of programming languages and technologies has proved to be necessary to increase productivity. One of the latest steps in this direction has lead to the usage of modelling languages in software development processes. Software models are indeed abstract representations of software systems which are used to tackle the complexity of present-day software by enabling developers to reason at a higher level of abstraction. In traditional software development processes, models are used either for sketching the architectural design at a high level of abstraction or for mere documentation purposes. On the contrary, in Model-Driven Engineering (MDE) models are first-class entities of the software development process. These models are used to generate M. Oriol and B. Meyer (Eds.): TOOLS EUROPE 2009, LNBIP 33, pp. 37–56, 2009. c Springer-Verlag Berlin Heidelberg 2009

38

A. Rutle et al.

(parts of) the software systems by means of model-to-model and model-to-text transformations. Hence, models in MDE are required to have two important properties. Firstly, they should be intuitive for humans, such as software developers, domain experts, managers and users. This has motivated for the usage of visual modelling and specification. Secondly, the models should have a precise syntax and semantics to enable analysis, validation, verification and transformation. This has motivated for the usage of formal modelling languages. The terms “diagrammatic-”, “visual-” and “graphical-” modelling are often used interchangeably. Here, we distinguish clearly between visualisation and diagrammatic syntax and focus on precise syntax (and semantics) of models independent of their visualisation. It is first and foremost due to the graph nature of models that we focus on diagrammatic modelling. By diagrammatic modelling we mean techniques which are targeting graph-based structures. Several diagrammatic specification frameworks have emerged in the last years as an attempt to facilitate modelling. The state-of-the-art of diagrammatic modelling languages includes the Unified Modeling Language (UML) [1] and the Eclipse Modeling Framework (EMF) [2]. These languages and frameworks share the same metamodel, the Meta-Object Facility (MOF) [3], which is a metamodelling and metadata repository standard developed by the Object Management Group (OMG). However, MOF is defined only semi-formally which may not guarantee the degree of precision required by MDE [4,5,6]. In this paper, we argue for a completely diagrammatic specification framework for MDE. We introduce the Diagram Predicate Framework (DPF) [7,8,9,10], which provides a formal diagrammatic approach to modelling based on category theory. The remainder of the paper is structured as follows. Section 2 motivates the necessity of formal diagrammatic modelling languages in MDE and presents a running example which we model by a UML class diagram. In the example, we introduce some modelling challenges which are difficult to express by UML alone. Section 3 introduces the basic concepts of our approach and shows how it can be used to model the motivating example precisely. Section 4 presents a generic and flexible scheme for the formalisation of metamodelling and discusses OMG’s 4-layered architecture in view of this scheme. Then in Section 5 the approach is illustrated for the metamodel of EMF in a case-study. Section 6 discusses some related works. Finally, Section 7 concludes the paper and outlines future work.

2 Motivation An appropriate approach to object-oriented modelling is to describe models – in a first approximation – as graphs. Graphs are a well-known, well-understood and frequently used means to represent system structures or states [11]. However, the expressive power offered by present-day MOF-based modelling languages – which are meant to deal with graph-based models – may not be sufficient to express certain constraints a software system must obey. As a consequence, OMG proposes the Object Constraint Language (OCL) 2.0 [12] for the definition of constraints in models defined by these languages [13]. This is just a special case of a general pattern where diagrammatic

A Diagrammatic Formalisation of MOF-Based Modelling Languages

39

modelling languages use textual languages to define constraints that are difficult to express by their own syntax and semantics. While this solution is to some extent accepted among software developers, there are two reasons why a completely diagrammatic approach is worth investigating. Firstly, OCL and MOF-based languages such as UML class diagrams live in different conceptual and technical spaces, which makes automatic reasoning about these models challenging. For example, any modification in a model structure must be reflected in the OCL constraints which are related to the modified structure, but, defining automatic synchronisation of OCL constraints for arbitrary model modifications is not possible [14]. Moreover, the identification of classes of modifications for which an automatic synchronisation of OCL constraints is possible requires a complex machinery to be implemented by tool vendors. Secondly, due to the graph nature of models, we are interested in investigating a completely diagrammatic approach for the specification and reasoning about structural models. This is also to obey the “everything is a model” vision of MDE by having both structure and constraints in the same model-centric format. Among the efforts which are done to clarify the semantics of diagrammatic modelling languages, are the introduction of MOF 2.0 [3] as a common metamodel for languages which are defined by OMG, and the so-called 4-layered modelling architecture. In this architecture, a model in each layer conforms to (or is an instance of ) a model in a higher layer, i.e. each layer represents a metamodel for the layer below it. In particular, a metamodel specifies the abstract syntax of a modelling language by defining the set of rules (or constraints) that can be used to determine whether a given model is well-formed. However, the relationship between a model and its metamodel remains imprecise. This is because how to determine that a model is well-formed according to its metamodel is only defined semi-formally [4,5,6]. Thus, although the usage of graphs for the representation of model structures is a success story, an enhancement of the formal basis is needed to: – Express well-formed diagrammatic constraints, especially constraints which span over multiple model elements, and which go beyond the structural constraints which are formalised as graph constraints [11,15]. – Formalise the relationship – conformance or instance of – between models in different layers. This relationship is expressed by the concepts of type- and typed graphs in graph theory [11], which does not capture the type of constraints mentioned above. A natural choice for such a formal enhancement of graph theory is category theory, and in particular the sketch formalism, which can be used to define semantics of diagrams, and thus of diagrammatic models, without the needs for a supplementary textual language. In the categorical sketch formalism, models are represented as graphs, and model properties are expressed by universal properties such as; limit, colimit, and commutativity constraints [16,17]. This approach has the benefit of being generic and at a high level of abstraction, but it turns models into a complex categorical structure with several auxiliary objects [7]. The proposed formalisation approach in this paper is DPF which is a generalisation and adaptation of the categorical sketch formalism, where user-defined diagrammatic

40

A. Rutle et al.

predicate signatures are allowed to represent the constructs of modelling languages in a more direct and adequate way. In particular, DPF is an extension of the Generalised Sketches [18] formalism originally developed by Diskin et al. in [4,19,20]. DPF aims to combine the mathematical rigour – which is necessary to enable automatic reasoning – with diagrammatic modelling. 2.1 Constraints in UML The motivating example illustrates the usage of some constraints which are not expressible by UML itself. These constraints are specified in OCL. We are convinced that software developers will benefit from a diagrammatic syntax capable of expressing these constraints.

Fig. 1. A UML class diagram for the management of employees and projects

In Fig. 1, a UML class diagram of an information system for the management of employees and projects is presented. We require that the following set of rules are satisfied at any state of the system: 1. 2. 3. 4. 5. 6. 7.

An employee must work for at least one department. An employee may be enrolled in none or many projects. A department may have none or many employees. A department may control none or many projects. A project must be controlled by at least one department. An employee enrolled in a project must work in the controlling department. A set of employees working for a controlling department must not be enrolled in the same controlled project more than once.

The rules above can be categorised into two groups based on their complexity. Rules 1-5 can be forced by constraints which involve one or two model elements. These rules can be expressed by UML syntax. For the rules 6 and 7, constraints which involve more than two elements are required. This can only be achieved by using the text-based OCL constraints as, for example:

A Diagrammatic Formalisation of MOF-Based Modelling Languages

41

1 context 2

3

4 5 6

Enrolment inv rule6: self.department.employees->includesAll(self. employee) inv rule7: Let enrolments:Set(Enrolment)=Enrolment. allInstances in (not enrolment->exists(enr|enr.project=self.project and enr.department=self.department and enr.employees=self.employees))

The solution above has the following drawbacks: 1. Checking the state of the system against the model will involve two steps which are done in two different technical spaces: firstly, checking the structure and some of the constraints in UML; then, checking the rest of the constraints by an OCL engine. 2. Since some the semantics of the model is hidden in the OCL code, the model development process may become complex and error-prone in the long run. In particular, domain experts may have difficulties in understanding the OCL code – something which may force the developers to use the list of rules in a natural language instead of the OCL rules. This may lead to misunderstandings [21]. A completely diagrammatic representation of the model in Fig. 1 is described in Section 3, Fig. 2.

3 Diagram Predicate Framework DPF is a graph-based specification format that takes its main ideas from both categorical and first-order logic (FOL), and adapts them to SE needs. While in FOL the arity of a predicate is given by a collection of nodes only, the arity of a predicate in DPF is given by a graph, i.e. a collection of nodes together with a collection of arrows between nodes. Besides, the main difference between FOL and DPF is that FOL is an “element-wise” logic, i.e. variables vary over elements of sets. In contrast, DPF uses a “sort-wise” logic where variables vary over sets and mappings. Diagrammatic reasoning can provide several benefits to software engineers. As an example we consider model evolution where the difference between artefacts is typically represented and reasoned about in a diagrammatic fashion. This is to help the developer to understand the rationale behind the modifications [22]. DPF has shown to be beneficial for the formalisation of version control systems in MDE. In particular, we exploit the formal foundation for the identification of commonalities and calculation/representation of differences between models [10]. In DPF, software models are represented by diagrammatic specifications. In the remainder of the paper, we use the terms “model” and “diagrammatic specification” interchangeably. In the next sections, we will discuss the syntax and semantics of these diagrammatic specifications.

42

A. Rutle et al.

3.1 Syntax of Diagrammatic Specifications Diagrammatic specifications are structures which consist of a graph and a set of constraints. The graph represents the structure of the model. Predicates from a predefined diagrammatic signature are used to add constraints to the graph. The concepts of signatures, constraints, and diagrammatic specifications are defined as follows. Definition 1 (Signature). A (diagrammatic predicate) signature Σ := (Π, α) consists of a collection of predicate symbols Π with a mapping that assigns an arity (graph) α(p) to each predicate symbol p ∈ Π. Definition 2 (Constraint). A constraint (p, δ) in a graph G is given by a predicate symbol p and a graph homomorphism δ : α(p) → G, where α(p) is the arity of p. Definition 3 (Diagrammatic Specification). A Σ-specification S := (G(S), S(Π)), is given by a graph G(S) and a set S(Π) of constraints (p, δ) in G(S) with p ∈ Π. Table 1 shows a sample signature Σ = (Π, α). The first column of the table shows the names of the predicates. The second and the third columns show the arities of predicates and a possible visualisation of the corresponding constraints, respectively. In the fourth column, the intended semantics of each predicate is specified. The predicates in Table 1 allow to specify properties and constraints that, according to our analysis, should be part of any modelling language which is used to define structural models. These predicates are just generalisations, or general patterns, for properties that are used in different areas of modelling. For example, [mult,(m,n)] for multiplicity constraint on associations in UML class diagrams, [total] for total constraint on relations in ER diagrams and [jointly-key] for uniqueness constraint (i.e. UNIQUE) in SQL. For a given signature the semantics of nodes, arrows and predicates has to be chosen in a way which is appropriate for the corresponding modelling environment. For structural models in object-oriented development, it is appropriate to interpret nodes as sets f

→ B as multi-valued functions f : A → ℘(B). The powerset ℘(B) of and arrows A − B is the set of all subsets of B, i.e. ℘(B) = {X | X ⊆ B}. 3.2 Constraints in DPF Fig. 2 shows an example of a Σ-specification S = (G(S), S(Π)). S specifies the structural model for the information system presented in Section 2.1. G(S) in Fig. 3 is the graph of S without any constraints on it. Here we will revisit some of the rules in Section 2.1. Recall that in rule 1 an employee must work for at least one department. In S, this is forced by the constraint ([total], δ1 ) on the arrow eDeps (see Table 2). Moreover, a property of S is that the functions eDeps and dEmps are inverse to each other, i.e. ∀e ∈ Employees, ∀d ∈ Departments : e ∈ dEmps(d) iff d ∈ eDeps(e). This is forced by the constraint ([inverse], δ3 ) on eDeps and dEmps. According to rule 6, an employee enrolled in a project must work in the controlling department. While this rule has to be forced by means of OCL constraints in the motivating example, we can specify it diagrammatically in S. The rule is forced by the constraint ([subset], δ8 ) between the arrows eEmps and eDep;dEmps. Note that

A Diagrammatic Formalisation of MOF-Based Modelling Languages

43

Table 1. A sample signature Σ p

α(p)

Proposed visualisat. Intended semantics

/2

A

/2

A

[total]

1

x

[key]

1

x

[singlevalued]

1

x

/2

A

[mult,(n,m)] 1

x

/2

A

1

x

/2

A

x

$

[cover]

[inverse]

•

∀a ∈ A : |f (a)| ≥ 1

f

/ B

∀a, a ∈ A : a = a implies f (a) = f (a )

[1]

f

/ B

∀a ∈ A : |f (a)| ≤ 1

/ B

∀a ∈ A : n ≤ |f (a)| ≤ m

2, B

∀b ∈ B : ∃a ∈ A | b ∈ f (a)

f [n,m] f

f

1d

2

A g

'

B

[INV]

/2 O

f

A

y

/ B O

[DC]

g

3 x

[subset]

1

f

$

:2 x

1

/2

y

/ B

∀a, a ∈ A : a = a implies f (a) = f (a ) or g(a) = g(a )

f / B >> 3 >[JK] >> h g > >

∀a, a ∈ A : a = a implies f (a) = f (a ) or g(a) = g(a ) or h(a) = h(a )

A

g f

[JK]2

C

1> >

x

/2

>>y >> >

z

4 [composition]

'

∀a ∈ A : f (a) ⊆ g(a)

[]

3 [jointlykey]

{f (a) | a ∈ A}∩ {g(c) | c ∈ C} = ∅ and {f (a) | a ∈ A} ∪ {g(c) | c ∈ C} = B

7 B

A

g

C

y

[jointlykey]

∀a ∈ A , ∀b ∈ B : b ∈ f (a) iff a ∈ g(b)

g

x

1

/ B

[KEY]

y

[disjointcover]

f

1> >

x

3

/2

>>z >> >

3

A

D

/ B >> >>f ;g g > [COMP] >> A

y

C f

∀a ∈ A : f ; g(a) ≡ f (a)}

{g(b) | b ∈

C

the arrow eDep;dEmps is the composition of the arrows eDep and dEmps. We believe that the usage of the [subset] predicate can help domain experts and developers in grasping the rationale of the constraint.

44

A. Rutle et al.

Fig. 2. A diagrammatic specification S = (G(S), S(Π))

Fig. 3. The graph G(S) of the diagrammatic specification S

Finally, rule 7 forces that a set of employees working for a controlling department must not be enrolled in the same controlled project more than once. Again, it is possible to specify such a rule diagrammatically by using the constraints ([jointly-key], δ8 ), ([single-valued], δ5 ) and ([single-valued], δ7 ). The semantics of the constraint [jointly-key] and [subset] are further explained in Example 2 (Section 3.3), where instances of S are taken into consideration. 3.3 Semantics of Diagrammatic Specifications In this section we discuss the semantics of diagrammatic predicates and show how this semantics is used to check whether a graph is an instance of a diagrammatic specification. As mentioned, for object-oriented modelling it is appropriate to interpret nodes in the underlying graph G(S) of a Σ-specification S by sets, and arrows by multi-valued functions. The main idea in these modelling languages is that the semantics of a model is given by a set of instances. To reflect this idea in DPF, we consider arbitrary graphs I together with a graph homomorphism ι : I → G(S) as instances of G(S). That is, a

A Diagrammatic Formalisation of MOF-Based Modelling Languages

45

Table 2. Some diagrams (p, δ) ∈ S(Π) (p, δ) ([total], δ1 )

α(p) 1

([cover], δ2 )

1

([inverse], δ3 )

1d

δ(α(p))

x

/2

x

/2

x

$

/ Employee

dEmps

Department

eDeps

2

/ Department

eDeps

Employee

Employee m

-

Department

dEmps

y x

/ 2 Enrolment

eDep

/ Department

([single-valued], δ5 ) 1

x

/ 2 Enrolment

eDep

/ Department

1

x

/2

([single-valued], δ7 ) 1

x

/2

x

$

([total], δ4 )

([total], δ6 )

([subset], δ8 )

1

1 1> > z

4

x

Enrolment

eP ro

/ P roject

eP ro

/ P roject

eEmps

:2

-

1 Employee

Enrolment eDep;dEmps

y

([jointly-key], δ9 )

Enrolment

/2

Enrolment P

3

Employee

>>y >> >

eDep / Department PPP PPeP PPro eEmps PPP P'

P roject

f

node A in G(S) is interpreted by the set ι−1 (A) of nodes in I; and an arrow A − →B in G(S) is interpreted by the multi-valued function from ι−1 (A) into ℘(ι−1 (B)) in I. The latter is represented by the set ι−1 (f ) of arrows in I (see Example 1). To define the concept of instance of a Σ-specification we have to fix, however, the semantics of the constraints offered by the signature Σ. We can mention at least three ways to define semantics of these constraints. In Table 1, we have used the “elementwise” language of set theory. In category theory, where the signature is limited to limit, colimit and commutativity constraints, the semantics of these constraints is mathematically “pre-defined” for any category according to the universal nature of these special constraints. In implementations of modelling languages, we rely on a less descriptive but more algorithmic way to define the semantics of constraints, e.g. we implement a validator for each constraint. However, in order to analyse and formalise MOF-based modelling languages and models which are defined by these languages, we do not need to decide for one of the mentioned possibilities. We just need to know that any of these possibilities defines “valid instance of predicates”. Definition 4 (Instances of Predicates). A semantic interpretation of a signature Σ = (Π, α) is given by a mapping that assigns to each p ∈ Π a set [[p]] of graph homomorphisms τ : O → α(p) called valid instances of p, written τ p, where O may vary over all graphs.

46

A. Rutle et al.

x / Example 1. We consider the (arity) graph 1 2 and the following instances of this graph: ⎛ ⎞ e1 / b1 a1 ⎜ ~? ⎟ x / ⎜ ⎟ e2 ~~ 2) τ1 : ⎜ ⎟ −→ (1 ~~ ~ ⎝ ⎠ ~~ a2 b2 ⎞ ⎛ e1 / b1 a1 ⎜ ~? ⎟ ⎟ x / ⎜ e2 ~~ 2) τ2 : ⎜ ⎟ −→ (1 ~~ ~ ⎠ ⎝ ~~ e3 / b2 a2 x / Note that the instances of 1 2 are what is often called “bipartite graphs”. These instances represent multi-valued functions f1 , f2 : {a1 , a2 } → ℘({b1 , b2 }), where ℘({b1 , b2 }) = {∅, {b1}, {b2 }, {b1 , b2 }}, with f1 (a1 ) = f1 (a2 ) = f2 (a1 ) = {b1 } and f2 (a2 ) = {b1 , b2 }. We have τ1 , τ2 [total] and τ2 [cover], but τ1 [cover].

To check the validity of a constraint in a given instance of G(S), it is obviously enough to inspect only that part of G(S) which is affected by the constraint. This kind of “restriction to a subpart” is described as the pullback construction, i.e. a generalisation of δ ι the inverse image construction. Given a span p − →G← − I the pullback PO

δ ι∗

O∗

/G O ι

P.B. δ∗

/I

is given by a graph O∗ and graph homomorphisms δ ∗ : O∗ → I, ι∗ : O∗ → P . Nodes in in O∗ are given by those nodes i in I such that there exists a node n in P with δ(n) = ι(i). Analogously, the arrows fa in O∗ are given by those arrows f in I such that there exists an arrow a in P with δ(a) = ι(f ). Moreover, we have δ ∗ (in ) = i, δ ∗ (fa ) = f and ι∗ (in ) = n, ι∗ (fa ) = a, respectively. In case f is injective there is no need to distinguish between different copies of nodes or arrows from I; thus we can drop the subscripts n or a, respectively, as we have done in Example 2. Definition 5 (Instance of Specification). An instance of a diagrammatic specification S = (G(S), S(Π)) is a graph I together with a graph homomorphism ι : I → G(S), written (I, ι), such that for each constraint (p, δ) ∈ S(Π) we have ι∗ ∈ [[p]], where ι∗ : O∗ → α(p) is given by the following pullback diagram α(p) O ι∗

O∗

δ

/ G(S) O ι

P.B. δ∗

/I

A Diagrammatic Formalisation of MOF-Based Modelling Languages

47

The following example revisits the motivating example in Section 2.1 and explains the usage of Definition 5 to check whether a given graph is an instance of the specification. Example 2. Fig. 4 shows an instance ι : I → G(S) of the diagrammatic specification S from Fig. 2. In Fig. 4, we have used some “user-friendly” notations: e.g. the roundedrectangle with the name Enrolment stands for the set ι−1 (Enrolment), the arrows eEmps1 and eEmps2 represent the set ι−1 (eEmps), and, the ellipsis with the elements e2 and e3 stands for one of the subsets in ℘(ι−1 (Employee)).

Fig. 4. An instance, ι : I → G(S), of the Σ-specification S in Fig. 2

To verify that (I, ι) is indeed an instance of S, we need to construct the pullback for each constraint (p, δ) in S(Π) (see Table 2). In this example, we validate three of these constraints. First, we look at the constraint ([jointly-key], δ9 ). The pullδ

ι

ι∗

9 G(S) ← − I will be α([jointly-key]) ←− back of α([jointly-key]) −→

δ∗

9 O∗ −→ I, where ι∗ : O∗ → α([jointly-key]) is as in Fig. 5. The graph homomorphism ι∗ is a valid instance of the predicate [jointly-key] since the semantics of the constraint which is set by the predicate is not violated. The constraint here is that every enri ∈ ι−1 (Enrolment) must be uniquely identified by a triple ℘(ι−1 (Employee)), ℘(ι−1 (Department)), ℘(ι−1 (P roject)) . Moreover, consider the constraint ([single-valued], δ5 ) ∈ S(Π). Since the arrow eDep is single-valued, the target of eDep1 and eDep2 must be singleton sets, i.e. |ι−1 (eDep)| ≤ 1, This constraint is not violated since the subsets {d1 } and {d2 } are both containing only one value from ι−1 (Department). In contrast, since the arrow eEmps is not marked with [single-valued], the target of eEmps2 , for example, is the subset {e1 , e2 } which contains more than one element from ι−1 (Employee). Finally, we will look at the subset constraint ([subset], δ8 ). We can check that this constraint is not violated by verifying that eEmps1 (enr1 ) ⊆ dEmps1 ( eDep1 (enr1 )) and eEmps2 (enr2 ) ⊆ dEmps2 (eDep2 (enr2 )).

48

A. Rutle et al.

Fig. 5. ι∗ : O∗ → α([jointly-key])

4 MOF-Based Modelling Languages In the previous sections, we argued that formal diagrammatic modelling languages are necessary in order to specify models which are of the quality that MDE requires. Most of present-day’s diagrammatic modelling languages share MOF as their metamodel. However, these languages have still some issues when it comes to their formal semantics, especially considering the relationship between models in different modelling layers. Therefore in this section, we focus on the formalisation of MOF-based modelling languages. First we explain the 4-layered modelling architecture of OMG standards. Then, we show how modelling languages are represented by modelling formalisms in DPF by presenting a generic and flexible scheme for the formalisation of metamodelling. Finally, the notions of metamodels and reflexive (meta)models will be formalised in terms of DPF. 4.1 The 4-Layered Modelling Architecture OMG defines four layers of modelling (Table 3). Models in the M0 -layer are called instances which represent the state of a running system. These instances must conform to models in the M1 -layer, which are structures specifying what instances should look like. These models, in their turn, must conform to a metamodel in the M2 -layer, against which models can be checked for validity. Metamodels correspond to modelling languages, for example UML and CWM (Common Warehouse Metamodel ). The highest modelling layer (as defined by OMG) is the M3 -layer where MOF is located. A model at this layer is often called meta-metamodel; it conforms to itself and it is used to describe metamodels (Fig. 6a). The concepts presented in Section 3 allow to model directly the layers M1 and M0 as Σ-specifications and instances (I, ι) of Σ-specifications, respectively. In Fig. 6b,

A Diagrammatic Formalisation of MOF-Based Modelling Languages

49

Table 3. OMG’s modelling layers Modelling layers M3 : Meta-metamodel M2 : Metamodel M1 : Model M0 : Instance

OMG Standards and examples MOF UML language A UML model: Class Person with attributes name and address An instance of Person: “Ola Nordmann” living in “Sotraveien 1, Bergen”

ιS3

conf ormsT o

MO 3 conf ormsT o

MO 2 conf ormsT o

MO 1 conf ormsT o

M0 (a) OMG’s 4-layered modelling hierarchy.

S3 (Π3 )

Π3 _ _ _/ G(S3 )

O

ιS2 S2 (Π2 ) Π2 _ _ _/ G(S2 )

O

ιS1 S1 (Π1 ) Π1 _ _ _/ G(S1 )

O

ιS0

S0 (b) OMG’s 4-layered modelling hierarchy in DPF.

Fig. 6. OMG’s 4-layered modelling hierarchy, and its formalisation in DPF

these two layers are shown as a Σ1 -specification S1 and an instance (S0 , ιS0 ). We have chosen the names Σ1 = (Π1 , α1 ), (S0 , ιS0 ) and S1 = (G(S1 ), S1 (Π1 )) to underline the correspondence to the modelling layers of OMG. In this section, the layers M2 and M3 will be formalised. 4.2 Modelling Formalisms Each modelling language L, which is located in the M2 layer, is reflected in DPF by a diagrammatic signature ΣL and a metamodel M ML . L-models are represented by ΣL -specifications where the underlying graphs are required to be instances of the metamodel M ML . Moreover, the metamodel M ML itself is assumed to be a ΣMML specification, for an appropriate meta-signature ΣMML . We will call such a triple (ΣL , M ML , ΣMML ) for a modelling formalism FL . In Section 3 we formalised the concept of a model and of instances of a model, i.e. we defined a syntax and a semantics for diagrammatic specifications. In this sense, metamodelling means that syntax of a model is treated as a semantic entity to be specified and constrained by a another model, i.e. by a syntax, at a higher level of abstraction. Especially, we are interested to constrain, in an appropriate way, the underlying graphs

50

A. Rutle et al.

of these models. This “recursion step” within modelling (from syntax to semantics) can be formalised by the following definitions. Definition 6 (Conformance). Given the diagrammatic signatures Σ1 = (Π1 , α1 ), Σ2 = (Π2 , α2 ) and a Σ2 -specification S2 = (G(S2 ), S2 (Π2 )), a Σ1 -specification S1 = (G(S1 ), S1 (Π1 )) conforms to S2 if there exists a graph homomorphism ιS1 : G(S1 ) → G(S2 ) such that (G(S1 ), ιS1 ) is an instance of S2 . S2 (Π2 ) Π2 _ _ _/ G(S2 ) O ιS1 S1 (Π1 )

Π1 _ _ _/ G(S1 ) Definition 7 (Modelling Formalism). A modelling formalism F = (Σ1 , S2 , Σ2 ) is given by diagrammatic signatures Σ1 = (Π1 , α1 ) and Σ2 = (Π2 , α2 ), and a Σ2 specification S2 = (G(S2 ), S2 (Π2 )) called the metamodel of F . An F -specification is a Σ1 -specification S1 = (G(S1 ), S1 (Π1 )) which conforms to S2 . 4.3 Meta-formalism and Reflexive (Meta)Models As discussed above, a modelling language L can be formalised as a modelling formalism F = (Σ1 , S2 , Σ2 ), i.e. (ΣL , M ML , ΣMML ). The idea is to describe the syntactic constraints on S2 by a meta-language M L, i.e. by another modelling formalism FML = (Σ2 , S3 , Σ3 ) such that S2 conforms to S3 (see Fig. 6). Note that the same instance-model pattern is used for the definition of the relationship between instances and model; between models and metamodel; as well as between metamodels and metametamodel. In principle, this chain of metamodelling, i.e. for a modelling formalism Fi = (Σi−1 , Si , Σi ) finding a meta-formalism Fi+1 = (Σi , Si+1 , Σi+1 ) such that Si conforms to Si+1 , can continue ad infinitum. However, if at every modelling layer i + 1, we choose Σi+1 as minimal as possible, i.e. only keep the predicates that are needed to constrain the underlying graph of models at layer i, then this chain ends in an endpoint. Moreover, since Σi+1 only specifies syntactic restrictions on the underlying graph of Si , it is likely Graph

O

ιSi

ιSi Si (Πi ) Πi _ _ _ _ _ _ _/ G(Si )

O

O

ιSi−1

ιSi−1 (Π

)

i−1 i−1 Πi−1 _ _ _ _ _ _/ G(Si−1 )

S

(a) Natural endpoint for metamodelling.

Si (Πi ) Πi _ _ _ _ _ _ _/ G(Si )

(Π

)

i−1 i−1 Πi−1 _ _ _ _ _ _/ G(Si−1 )

S

(b) A reflexive metamodel.

Fig. 7. Modelling layers and reflexive metamodels

A Diagrammatic Formalisation of MOF-Based Modelling Languages

51

that Πi+1 is less expressive than Πi . Thus, the natural endpoint of such a chain of decreasing expressiveness will be a modelling formalism Fn = (Σn−1 , Graph, ∅),

where by Graph we mean • (Fig. 7a). This indicates that graph is the basic structure on which the whole framework is based. Another way to avoid progress ad infinitum is to collapse a potentially infinite chain into a single loop, i.e. to stop when Σi+1 is expressive enough to describe the restrictions which are put on the underlying graph of Si+1 (see Fig. 7b). This motivates the concept of reflexive (meta)model. Definition 8 (Reflexive Metamodel). A Σ-specification S = (G(S), S(Π)) is reflexive if S conforms to itself, i.e. if S is a (Σ, S, Σ)-specification. Fig. 7b shows a reflexive metamodel Si of a modelling formalism Fi = (Σi−1 , Si , Σi ). Note that ιSi will not be, in general, the identity morphism (see Section 5). Examples of reflexive metamodels are MOF (Fig. 6) and Ecore (Fig. 8). A formalisation of the latter is outlined in the next section. ιM

instanceOf

Ecore O

Ecore

ΠEcore _ _ _ _ _ _/ G(MEcore )

instanceOf

M odel O

O

ιS S(ΠCore ) ΠCore _ _ _ _ _ _ _/ G(S)

O

instanceOf

ιI

Instance (a) EMF’s modelling hierarchy.

MEcore (ΠEcore )

I (b) EMF’s modelling hierarchy in DPF.

Fig. 8. EMF’s modelling hierarchy and its formalisation in DPF

5 Case-Study: Formalisation of EMF In this section, we will use DPF to build a diagrammatic modelling formalism for EMF. EMF is a Java open source framework for modelling, data integration and codegeneration [2]. EMF is an implementation of Essential MOF (EMOF), which is a subset of MOF 2.0, but its metamodel is called Ecore to avoid confusion with MOF. EMF is a widely-used modelling framework because: – It is considered a low cost entry to the employment of MDE. – It has a clear semantics due to the simplicity of its metamodel. – There are many tools which are built around EMF and made available as Eclipse plug-ins. Software models in EMF are represented by core models. Core models are located in the M1 -layer of the OMG architecture. These models specify the structure of instances – the M0 -layer. That is, the types of objects that make up instance models,

52

A. Rutle et al.

Fig. 9. MEcore , the metamodel of FEM F , as a ΣEcore-specification

the data they contain and the relationships between them [23]. The metamodel of core models – Ecore – specifies the structure of the core models and is located in the M2 layer of the OMG architecture. Moreover, Ecore is reflexive, meaning that it specifies its own structure (Fig. 8a). Note that EMF has only three layers since Ecore is not intended to be the metamodel of a collection of other languages as it is the case for MOF. Fig. 8b shows EMF’s modelling hierarchy in view of DPF. EMF corresponds to a modelling formalism FEMF = (ΣCore , MEcore, ΣEcore ), where ΣCore = (ΠCore , αCore ), ΣEcore = (ΠEcore , αEcore ), and the metamodel is a ΣEcorespecification MEcore = (G(MEcore ), MEcore(ΠEcore )). Core models correspond to ΣCore -specifications S = (G(S), S(ΠCore )) which conform to MEcore. In addition, MEcore is a ΣCore -specification which conforms to MEcore. This is because MEcore is reflexive and ΣEcore is part of ΣCore . In some application contexts, we extend ΣCore with additional constraints and validators for these constraints. This is to enable software modellers to put additional constraints on core models. These additional constraints may be needed since Ecore supports only a minimal set of constraints, such as; multiplicity constraints and containment- and bidirectional references. In practice, constraints in EMF models (which are specified in OCL or Java) will consist of writing an Eclipse plug-in based on the EMF Validation Framework, or will involve the usage of the EValidator API of EMF. In both cases, models and constraints live in two different technical spaces – something which may lead to challenges as mentioned in Section 2. In the following, we describe the kernel of the metamodel MEcore in terms of DPF; in addition, we show that the the metamodel is reflexive. The kernel of MEcore, which is adopted from [23], is shown in Fig. 9. Each model element in MEcore is typed over MEcore by the graph homomorphism ιMEcore : G(MEcore ) → G(MEcore ). This typing is as follows: ιMEcore (EClassifier) = ιMEcore (EDataType) = ιMEcore (EClass) = EClassifier ιMEcore (eSuperType1 ) = ιMEcore (eSuperType2 ) = ιMEcore (eReferences) = ιMEcore (eAttributes) = ιMEcore (eStructuralFeatures) = eStructuralFeatures

A Diagrammatic Formalisation of MOF-Based Modelling Languages

53

We have some concerns about the mixture of Ecore with Java, for example in MEcore the super-type (or superclass) of EClass and EDataType is an EClassifier. In other words, each instance of the metaclass EClassifier is the super-type of either an EClass or an EDataType. In the specification of Ecore as an EMF-model, the semantics of the super-type arrows is given by the semantics of the “extends” keyword in Java. That is, something outside the modelling language is used to specify Ecore. In contrast, we incorporate this constraint into the framework by means of a single predicate [disjoint-cover] (see Table 1).

6 Related Work Diagrammatic modelling and formalisation of metamodelling have been extensively discussed in the literature. Here we discuss some of the approaches to the specification of constraints and formalisation of MOF-based modelling languages. The work in [5] exploits the higher-order nature of constructive type theory to uniformly treat the syntax of models, metamodels as well as the MOF model itself. Models are formalised as terms (token models) and can also be represented as types (type models) by means of a reflection mechanism. This formalisation ensures that correct typing corresponds to provably correct models and metamodels. The works in [6,24] use algebraic specification to give formal semantics to MOF. In [6] MOF-based metamodels are considered to be graphs with attributed objects as nodes and references as arrows. These graphs are represented by specifications in Membership Equational Logic (MEL), i.e. the logical power of MEL is essentially used to define the concept of a finite (multi)set at different places and levels. This formal semantics is made executable by using the Maude language, which directly supports MEL specifications. In this formalisation, metamodels are seen to play several roles: as data, as type or as theory; these roles are formally expressed by metamodel definition, model type, and metamodel realisation, respectively. In [24] a metamodelling framework, which is also based on Maude, is presented. In this framework, graphs are represented as terms by taking advantage of the underlying term matching algorithm modulo associativity and commutativity. The ability of Maude to execute the specifications allows the implementation of some key operations on models, such as model subtyping, type inference, and metric evaluation. Epsilon Validation Language (EVL) is one of the languages in the Epsilon model management platform [25]. EVL extends OCL conceptually (as opposed to technically) to provide a number of features such as support for constraint dependency management and access to multiple models of different metamodels and technologies. Epsilon provides a mature tool support for management of Ecore-based models. It will be interesting to investigate the formal basis of this language and compare it to DPF in a future work. Visual OCL (VOCL) [26] is an effort to define a graphical visualisation for OCL. It extends the syntax of UML and is used to define constraints on UML models. VOCL allows developers to put models together with constraints without leaving the graphical level of abstraction. However, VOCL does not extend the formal semantics of OCL. It only visualises constraints which are specifiable in OCL. An interesting and open

54

A. Rutle et al.

questions is whether the OCL constraints that are visualised can indeed be seen as sortwise constraints, and thus formalised in DPF. Alloy [27] is a structural modelling language which is capable of expressing complex structural constraints and behaviour. Model analysis in Alloy is based on the usage of FOL to translate specifications into boolean expressions which are automatically evaluated by a boolean satisfiability problem (SAT) solver. Then for a given logical formula F , Alloy attempts to find a model which satisfies F . Alloy models are checked by using the Alloy analyser which attempts to find counterexamples within a limited scope which violates the constraints of the system. Even though Alloy cannot prove the system’s consistency in an infinite scope, the user receives immediate feedback about the system’s consistency.

7 Conclusion and Future Work Independent of the importance of an appropriate and unambiguous visualisation we focus in DPF on the precise syntax (and semantics) of diagrammatic constraints. We allow diagrammatic constraints which modellers can use and reason about. An advantage of using these diagrammatic constraints is that they belong to the same conceptual space as the models which they constrain. This saves software developers from the challenges caused by mixing different technical and conceptual spaces. The paper argues that DPF is a promising candidate for an appropriate diagrammatic specification framework for MDE. Besides the diagrammatic appearance, this is mainly due to the rigorous mathematical foundation of the DPF. We have shown DPF’s usage as a formal diagrammatic framework for the definition of models and modelling languages. In particular, we have presented a generic and flexible scheme for the formalisation of metamodelling and discussed the 4-layered architecture of OMG in view of this scheme. This scheme has justified the extension of type- and typed graph concepts [11] with support for diagrammatic constraints. In view of graph transformations, a characterisation of our approach can be given by considering diagrammatic specifications as type graphs with constraints, and instances as typed graphs which satisfy those constraints. Moreover, an application of the approach is illustrated by representing EMF in DPF. The version of DPF presented in the paper distinguishes clearly between signatures and (meta)models. And it seems that only this “separation of concerns” allows for a clean and well-structured formalisation of metamodelling. In most modelling languages, however, predicates and constraints are internalised in the sense that predicates are represented by classes in the metamodel and constraints by corresponding objects. It seems that such kind of internalisation can be formalised by adapting and generalising an abstract categorical construction presented by Makkai in [18]. We intend to investigate this conjecture in one of our more foundational future work. The proposed visualisation of the predicates can be modified to meet the taste of the developers of the modelling language which uses the signature. A formalisation of parts of UML within DPF is presented in [4,28]. In a future work, we intend to define a transformation for the combination of UML with OCL to DPF. This will provide users of the framework with means to specify their models formally and diagrammatically

A Diagrammatic Formalisation of MOF-Based Modelling Languages

55

without the steep learning curve of a new visualisation or notation. To define such a transformation we need to investigate to what extend OCL constraints can be considered as sort-wise constraints. Due to space limitations we have not considered dependencies between predicates [7] even if they are highly relevant for practical applications of DPF. These dependencies provide a kind of rudimentary logic for DPF. The development of a fully fledged logic for DPF, allowing especially to reason about and to verify models, is one of our main future objectives. The development of a prototype implementation of the DPF framework based on the Eclipse platform is now in its early stage; we plan to boost this development in a near future. The final prototype will also support version control, whose underlying techniques are analysed in [10].

References 1. Object Management Group: Unified Modeling Language Specification (November 2007), http://www.omg.org/cgi-bin/doc?formal/2007-11-04 2. Eclipse Modeling Framework, http://www.eclipse.org/emf/ 3. Object Management Group: Meta-Object Facility Specification (January 2006), http://www.omg.org/cgi-bin/doc?formal/2006-01-01 4. Diskin, Z.: Mathematics of UML: Making the Odysseys of UML less dramatic. In: Practical foundations of business system specifications, pp. 145–178. Kluwer Academic Publishers, Dordrecht (2003) 5. Poernomo, I.: A Type Theoretic Framework for Formal Metamodelling. In: Reussner, R., Stafford, J.A., Szyperski, C. (eds.) Architecting Systems with Trustworthy Components. LNCS, vol. 3938, pp. 262–298. Springer, Heidelberg (2006) 6. Boronat, A., Meseguer, J.: An Algebraic Semantics for MOF. In: Fiadeiro, J.L., Inverardi, P. (eds.) FASE 2008. LNCS, vol. 4961, pp. 377–391. Springer, Heidelberg (2008) 7. Diskin, Z., Wolter, U.: A Diagrammatic Logic for Object-Oriented Visual Modeling. In: ACCAT 2007: 2nd Workshop on Applied and Computational Category Theory. ENTCS, vol. 203, pp. 19–41. Elsevier Science Publishers B.V., Amsterdam (2008) 8. Rutle, A., Wolter, U., Lamo, Y.: A Diagrammatic Approach to Model Transformations. In: EATIS 2008: Euro American Conference on Telematics and Information Systems (to appear) 9. Rutle, A., Wolter, U., Lamo, Y.: A Formal Approach to Modeling and Model Transformations in Software Engineering. Technical Report 48, Turku Centre for Computer Science, Finland (2008) 10. Rutle, A., Rossini, A., Lamo, Y., Wolter, U.: A Category-Theoretical Approach to the Formalisation of Version Control in MDE. In: Chechik, M., Wirsing, M. (eds.) FASE 2009. LNCS, vol. 5503, pp. 64–78. Springer, Heidelberg (2009) 11. Ehrig, H., Ehrig, K., Prange, U., Taentzer, G.: Fundamentals of Algebraic Graph Transformation. Springer, Heidelberg (March 2006) 12. Object Management Group: Object Constraint Language Specification (May 2006), http://www.omg.org/cgi-bin/doc?formal/2006-05-01 13. Warmer, J., Kleppe, A.: The Object Constraint Language: Getting your models ready for MDA, 2nd edn. Addison-Wesley, Reading (2003) 14. Markovi´c, S., Baar, T.: Refactoring OCL annotated UML class diagrams. Software and System Modeling 7(1), 25–47 (2008) 15. Taentzer, G., Rensink, A.: Ensuring Structural Constraints in Graph-Based Models with Type Inheritance. In: Cerioli, M. (ed.) FASE 2005. LNCS, vol. 3442, pp. 64–79. Springer, Heidelberg (2005)

56

A. Rutle et al.

16. Barr, M., Wells, C.: Category Theory for Computing Science, 2nd edn. Prentice Hall International Ltd., Hertfordshire (1995) 17. Fiadeiro, J.L.: Categories for Software Engineering. Springer, Heidelberg (May 2004) 18. Makkai, M.: Generalized Sketches as a Framework for Completeness Theorems. Journal of Pure and Applied Algebra 115, 49–79, 179–212, 214–274 (1997) 19. Diskin, Z., Kadish, B.: Generic Model Management. In: Encyclopedia of Database Technologies and Applications, pp. 258–265. Idea Group (2005) 20. Diskin, Z., Dingel, J.: Mappings, Maps and Tables: Towards Formal Semantics for Associations in UML2. In: Nierstrasz, O., Whittle, J., Harel, D., Reggio, G. (eds.) MoDELS 2006. LNCS, vol. 4199, pp. 230–244. Springer, Heidelberg (2006) 21. Bottoni, P., Koch, M., Parisi-Presicce, F., Taentzer, G.: A Visualization of OCL Using Collaborations. In: Gogolla, M., Kobryn, C. (eds.) UML 2001. LNCS, vol. 2185, pp. 257–271. Springer, Heidelberg (2001) 22. Cicchetti, A., Di Ruscio, D., Pierantonio, A.: A Metamodel Independent Approach to Difference Representation. Journal of Object Technology (Special Issue on TOOLS Europe 2007) 6(9), 165–185 (2007) 23. Budinsky, F., Merks, E., Steinberg, D.: EMF: Eclipse Modeling Framework 2.0, 2nd edn. Addison-Wesley Professional, Reading (2006) 24. Romero, J.R., Rivera, J.E., Durán, F., Vallecillo, A.: Formal and Tool Support for Model Driven Engineering with Maude. Journal of Object Technology 6(9), 187–207 (2007) 25. Epsilon: Book, http://epsilonlabs.wiki.sourceforge.net/Book 26. Visual OCL: Project Web Site, http://tfs.cs.tu-berlin.de/vocl/ 27. Alloy: Project Web Site, http://alloy.mit.edu/community/ 28. Diskin, Z., Easterbrook, S.M., Dingel, J.: Engineering Associations: From Models to Code and Back through Semantics. In: Paige, R.F., Meyer, B. (eds.) TOOLS Europe 2008. LNBIP, vol. 11, pp. 336–355. Springer, Heidelberg (2008)

Designing Design Constraints in the UML Using Join Point Designation Diagrams Vanessa Stricker1, Stefan Hanenberg2, and Dominik Stein2 1 Software Systems Engineering. Data Management Systems and Knowledge Representation Institute for Computer Science and Business Information Systems, University of Duisburg-Essen 45117 Essen, Germany [email protected] [email protected] [email protected] 2

Abstract. Design errors cause high costs during the development of software, since such errors are often detected quite late in the software development process. Hence, it is desirable to prevent design errors as early as possible, i.e. it is necessary to specify constraints on the software design. While there are a number of approaches available that permit to specify such constraints, a common problem of those is that the developers need to specify constraints in a different language than their design language. However, this reduces the acceptance of such approaches, as developers need a different language just for the purpose of specifying constraints. This paper proposes an approach that permits to specify constraints within the UML by using Join Point Designation Diagrams (JPDDs).

1 Introduction Checking constraints from the very beginning of the software development process is essential for the success of the resulting software. It is commonly accepted that errors in the early stages of software development cause very high costs later on. This also applies to recent approaches such as model-driven software development [16]. In this case, it is even more crucial to check whether the design artifacts (i.e. the models) fulfill their constraints since models are the main fragments that are used to generate the resulting software from. Disregards of any constraint at design time are most often detected not before code generation time or (even worse) when the generated code is finally being tested. Since the generation of code from non-trivial models can easily take several hours, disregards of design constraints are a critical factor for the success of a software project. Typically, a number of design constraints result from architectural decisions, or from (external) techniques that are (or have to be) applied in a project. Furthermore, software companies usually have a number of development guidelines that must be followed by the developers. While there are design constraints that cannot be expressed in a formal manner (e.g. guidelines such as “names of classes and methods M. Oriol and B. Meyer (Eds.): TOOLS EUROPE 2009, LNBIP 33, pp. 57–76, 2009. © Springer-Verlag Berlin Heidelberg 2009

58

V. Stricker, S. Hanenberg, and D. Stein

should reflect their semantics”) a large amount of constraints and guidelines can be formally expressed. Instead of defining such constraints in an external document, it is desirable to specify them such that they can be immediately deployed in order to identify corrupt model elements. There are already a number of approaches that permit to specify constraints [27]. However, in practice such approaches did not turn out to be easily applicable in actual projects. The main reason for this is that constraints need to be specified in languages that developers are not familiar with. As a consequence, developers need to learn new languages (in addition to their proper design language) in order to specify constraints. This also implies that developers need to learn a set of new tools that come with the constraint language. Such an approach is costly as developers need to be trained on the language and the tools. Furthermore, applying a new language is a potential source of errors because most often constraint languages introduce concepts that do not (or not directly) appear in the utilized design language. Furthermore, developers that check their models using some constraints (in case a constraint rejects a model) need to understand what exactly the intention of the constraint is and how the model can be adapted in a way that fulfills the constraint’s underlying motivation: in case they do not understand precisely the constraint language, it is hard for them to fix their model in order to fulfill given constraints. Software developers that are using graphical design languages such as the UML [21] are familiar with the language’s concepts as well as with its graphical notation (i.e. with the language's concrete syntax). In order to reduce the previously described problems, it is desirable to provide developers with means to specify constraints in terms of the design language (i.e. UML in this case) itself. That means that the constructs provided by the constraint language as well as their graphical representation should correspond to the constructs provided by the design language. This permits developers to communicate their design constraints in terms of constructs and notational means they already know. Such an approach would increase the ability to understand constraints as well as the ability to specify constraints. Furthermore, it would reduce the training costs as well as the potential sources of errors (caused by applying a new language). This paper presents a constraint language based on the (concrete syntax of) UML in order to increase the developers’ abilities to specify and to communicate design constraints in terms of a design language they are familiar with. This constraint language is based on Join Point Designation Diagrams (JPDDs, see [6, 24, 25, 26]) that were originally developed to model aspect-oriented software in the UML. In order to exemplify the urge for such a constraint language, section 2 presents a real-world problem scenario from a medium-sized company. Section 3 gives detailed description of the underlying problem. Section 4 introduces Join Point Designation Diagrams. Then, section 4 explains how JPDDs are being deployed for the specification of design constraints. Section 5 revisits the problem and expresses it in term of JPDDs. Section 6 presents related work. Finally, section 7 discusses and concludes the paper.

2 Motivation The following example describes a real-world scenario of a medium-sized software company that provides products for health-care insurance companies. Examples for

Designing Design Constraints in the UML Using Join Point Designation Diagrams

59

such products are information systems for insurants, configurations of insurances as well as billing. The company deploys a model-driven software development approach based on UML that requires the software developers to specify most parts of the software with UML models. The company provides its own code generator that generates about 95% of the code base. The remaining code is hand-coded at well-defined hooks provided by the generator. The company generates server-sided code for both the business logic and the middleware connections as well as client-sided code for the user interface. The product under consideration consists of about 15.000 UML classes whose transformation to code takes about eight hours. (Automated) deployment and testing takes about 3 days. The resulting application runs in a J2EE environment. Developers that specify UML models have to follow a large number of constraints which come from different origins. There are constraints that represent the best practices of the company, others are forced by the underlying architecture. Furthermore, there are technical constraints from the underlying code generator. For motivation purposes, we want to cite here a simple (and simplified) example of a constraint in use which needs to be considered by developers that are responsible for designing the user interface. The user interface is based on the Model-View-Controller pattern (MVC, [22, 2]). This pattern suggests the encapsulation of the model, the view and the controller in separate elements. The view is responsible for visualizing data, the controller handles events of the user, and the model represents the data being visualized by the view. For reasons of simplicity we will neglect the model in the pattern, i.e. we only consider controller and view in the following discussion. «view» MyView

1..* view

has

1 «controller» controller MyController

Fig. 1. Correct model of views and controllers

The company’s code generator (as well as the company’s best practices) requires that a view and a controller are represented by separate UML classes. Furthermore, the generator requires that each view is annotated with a stereotype «view» and that each controller is annotated with the stereotype «controller». Each view must be connected to a controller (and vice versa) by a bi-directional association. The connection must have the name “has” and the role names “view” and “controller” respectively. The cardinality of the association is restricted, too: while the cardinality of the controller role must be 1, the cardinality of the view role may be 1..*. Figure 1 illustrates an extraction of a class diagram that fulfills these constraints. Figure 2 illustrates a model that does not fulfill the constraints: the first view does not have the required stereotype. Furthermore, the association between the second view and the controller does not comply with the constraint (since the role name “controller” is missing). Each disregard of the above design constraints had an immensely negative impact on the subsequent development process as erroneous connections between views and controllers would make either the code generation or (even) the final program execution crash. So the company decided to install model checking facilities in order to prevent this.

60

V. Stricker, S. Hanenberg, and D. Stein

1..* view

has

FirstView

1 controller

«controller» ControllerA

«view» SecondView

1..* view

has

1

«controller» ControllerB

Fig. 2. Incorrect model of views and controllers

3 Problem Statement The company’s approach to install model checking facilities was to write Java programs that read the UML models (i.e. their XMI [20] files) and check their validity by consecutively invoking hard-coded constraint-checker functions on them. Figure 3 shows a code fragment that illustrates the specification of the constraint presented above. Furthermore, the company provided its developers with a natural language description of the constraints they had to obey. public void checkViewControllers() { Collection cls = ... get the UML classes... for(Iterator it=cls.iterator(); it.hasNext() ; ) { UMLClass c = (UMLClass) it.next(); if(c.hasSterotypeNamed(“view”) { Collection as = c.getAssociationsNamed(“has”); if (!as.isEmpty()) { ...

...check rolenames.....check cardinality...

for(Iterator it2 = as.iterator(); it2.hasNext() ;) { UMLClass controller = ...

...check for sterotype “controller”

} } else ...throw an Exception...

} ... }

Fig. 3. Constraint specified in Java

Doing so led to two new problems, though: First, the constraints in the Java programs and their natural explanations had to be synchronized. Secondly, it had to be guaranteed that there are no ambiguities in the natural language description. Otherwise developers would not be able to determine why their models were considered erroneous. Due to the complexity of a large number of constraints, a natural language description of the constraints turned out to be satisfying at all: developers were not satisfied with the natural descriptions because of their unclear semantics as well as their - from their point of view - inappropriate representation. One option to overcome these synchronization and ambiguity problems is to make all developers read and understand the constraint semantics already specified in Java. This option comes along with an immense learning effort: Developers need to learn the (Java) data model of the UML diagrams, its API and its usage. This option has not been considered as a serious solution by the company: the programming skills of the developers that mainly use the UML to express their artifacts were not considered to be strong enough. Furthermore, it turned out that developers still require too much time to

Designing Design Constraints in the UML Using Join Point Designation Diagrams

61

someUmlModel.contents ->select(c: Class | c.stereotype->exists(st | st.name='view') and not ( c.associations->exists(a | a.name='has' and a.allConnections->size=2 and a.allConnections->select(ae | ae.participant = self)->exists(ae | ae.name = 'view' and ae.isNavigable = true and ae.multiplicity.range.lower >= 1 and ae.multiplicity.range.upper <= unlimited ) and a.allConnections->select(ae | ae.participant <> self)->exists(ae | ae.name = 'controller' and ae.isNavigable = true and ae.multiplicity.range.lower = 1 and ae.multiplicity.range.upper = 1 and (let c1: Class = ae.participant.oclAsType(Class) in c1.stereotype->exists(st | st.name='controller') ) ) ) ) ) someUmlModel.contents ->select(c: Class | /* analogously for each controller */ )

Fig. 4. Constraint specified in OCL

understand the semantics of the constraint. I.e. a code representation did not turn out to be effective to understand and communicate design constraints among developers. Unfortunately, the option of using a common (model) constraint specification means such as the OCL [27] does not do better in that regard (Figure 4 illustrates a corresponding constraint specified by using the OCL): In order to understand OCL constraints, developers need to spend a large amount of their time to get familiar with this new (i.e. new for them) language – just as in the previous case. Apart from learning keywords, operators, syntax and semantics of OCL, they would furthermore be confronted with the full complexity of UML's meta-model – something that they usually do not encounter during daily work. Again, they would face a significant mapping problem since there are plenty of UML (meta)model elements which have no explicit visual representation in UML's concrete syntax. As a result, the representation of the constraints in terms of OCL is significantly different from the representational means they are familiar with. Although Visual OCL [1, 12] tries to overcome some of these disadvantages, it does not solve these problems. The existing UML notation elements which are used as much as possible within Visual OCL are not rich enough to express all elements defined in the OCL meta-model, which is why developers need to get used to the new notation elements and still would have to learn the semantics of the OCL constructs. Hence, while the real-world scenario described above showed the urgent need to check constraints at design time, the investigation of possible options to do so – in a synchronized, unambiguous way – points to the problems of using a second/a different language for specifying the constraints. Since the representation of the constraints differs fundamentally from the representation of the models themselves, it is hard for developers to understand the relationship between the constraint specification and their models. Consequently, the semantics of the constraints remains unclear to the developer whose model is in conflict with an existing constraint, which hinders him/her from comprehending and correcting his/her design errors. In other words: Current techniques are sufficient and mature enough to detect design errors. However, what is lacking is a means which can be used by constraint developers to easily communicate their design constraints to the application developers (so that

62

V. Stricker, S. Hanenberg, and D. Stein

they know what they have to look for), and which at the same time can be used to automatically check the compliance of the models to the constraints.

4 Specifying Design Constraints This section introduces a graphical constraint language based on the UML. Therefore, so-called Join Point Designation Diagrams (JPDDs) are introduced, which permit to specify queries on UML models. Based on that, the use of JPDDs for the purpose of specifying constraints is explained1. 4.1 Join Point Designation Diagrams (JPDDs) «Join Point Designation Diagrams» (JPDDs [24, 26]) are extensions of the UML [21] in order to specify queries on software artifacts specified in the UML. Hence, JPDDs permit developers to express selections on UML models in terms of the UML itself (i.e. its own concrete syntax). Although JPDDs provide general-purpose constructs for expressing such selections (cf. [24, 26]), the original motivation for proposing JPDDs came from the domain of aspect-oriented software development, where JPDDs have been applied to select those artifacts that need to be handled by aspects (see [6]). Due to their background, JPDDs provide means to specify static selection criteria (e.g. selection criteria on classdiagrams etc.) as well dynamic selection criteria (e.g. selection criteria on interactions, etc.). For the purpose of specifying design constraints that need to be statically checked, the dynamic selection criteria are irrelevant. Hence, the following subsections introduce constructs for specifying static selection criteria, whereas constructs for specifying dynamic selection criteria are neglected. For reasons of simplicity, we furthermore explain the semantics of JPDDs in an informal way. A specification of its semantics in OCL can be found in [26]. JPDDs are defined with help of the graphical notation provided by the UML. That notation makes use of symbols from existing and well-established graphical notations, such as interaction sequence diagrams, state diagrams, activity diagrams, and class/object diagrams. Nevertheless, in order to provide reasonable selection semantics, a small set of constructs needed to be added. This is described in the next subsection. Afterwards, the semantics of selection patterns is explained by means of an example. Finally, relationships between JPDDs are explained. 4.1.1 Extensions to the UML In order to provide reasonable selection semantics, JPDDs extend the UML by the following elements: Identifiers, parameter boxes, regular expressions, list-wildcards, and indirections. Within a selection pattern, a tuple of identifiers (see Figure 5a) may be assigned to particular elements in order to designate those parts of the queried software artifact that should be actually selected by the JPDD. JPDDs possess an export parameter box at their lower right corner (see Figure 5b) which lists the identifiers of all elements that have to be selected. 1

A tool-support for JPDDs and constraint checking using JPDDs can be found at http://www.dawis.wiwi.uni-due.de/forschung/foci/aosd/jpdds/

Designing Design Constraints in the UML Using Join Point Designation Diagrams

63

(c) Deviation Specification Means

(a) Identifiers prin t* identifiers

Stri

regular expressions

[0..*] [0..*]

(b) Export Parameter Box

.* : .* [0..*]

?jp ?obj

paths (set.*|get.*) (..) or-operator list-wildcard

exposed elements

Fig. 5. JPDD-specific symbols (cf. [26])

JPDDs come with a set of means to specify deviations in search patterns. Examples are regular expressions which permit to specify name patterns of elements to be selected as well as the dot-dot-wildcard (..) which abstracts over an arbitrary number of parameters in a parameter list (see Figure 5c). Furthermore, JPDDs provide means to postulate the existence of paths (of arbitrary length) along class/object associations ( ), the inheritance hierarchy ( ), or the call graph ( ). All indirect paths can be annotated with a cardinality which indicates the length of the path by specifying a minimum and maximum bound. The cardinality [0..*] specified in Figure 5c states that the paths may have an arbitrary length (including length 0 meaning that the two modeled classes are the same). Further explanations on the effects of these deviation specification means and examples of their usage will be given in the subsequent subsections. 4.1.2 Example JPDD JPDDs can be specified based on an arbitrary kind of model in the UML (see [24, 26]). Due to our motivation, we consider only class diagrams till now. GUIPOSelection GUI Element

Persistent Object [0..*]

[0..*] .*

.* .*

.* .*

Attributes

.* : .* Operations

set.*(.. ) : .*

?C ?PC

Fig 6. A structural JPDD

Figure 6 illustrates a sample JPDD named GUIPOSelection. The JPDD exports the variables ?C and ?PC. ?C is bound to all classes that fulfill the following requirements: each class must directly or indirectly extend class GUIElement. Furthermore,

64

V. Stricker, S. Hanenberg, and D. Stein

the class must have at least one directed association to a class which is a (direct or indirect) descendant of PersistentObject. Furthermore, that descendant must have at least one attribute and one set-method. Such a class is bound to the variable ?PC. Figure 7 shows a class diagram that the previous JPDD can be applied to. Since POA as well as POB fulfill the criterion of being a PersistentObject and since ClassA as well as ClassB are are descendants of GUIElement that have relationships to POA and POB, a corresponding result set is created. The result set has three elements. The first one binds ?C to ClassA and ?PO to POA. Furthermore, for each relationship between ClassB and a persistent object (i.e. POA and POB), a tuple is returned. GUI Element

Persistent Object

r1

ClassA

POA Attributes

f: F

r2

ClassB

g: G

Operations

setF (f:F ) : void

r3

POB Attributes

Operations

setG (g:G ): void

Result: { {?C=ClassA, ?PC= POA }, {?C=ClassB, ?PC= POA }, {?C=ClassB, ?PC= POB }}

Fig. 7. UML model and result of JPDD

4.1.3 Relationships between JPDDs In order to create modules of JPDDs that can be combined with others, different types of relationships between JPDDs are provided which allow such a combination (cf. [25]). Three different kinds of relationships are defined: union (symbol ∪), confinement (symbol ∩) and exclusion (symbol \). Figure 8 shows how these combination relationships can be defined between two JPDDs. For each relationship it is necessary to specify which variables are considered for the combination of the JPDDs. The union combines the result sets of the source JPDD_D with the target JPDD_A. A confinement considers in the result of the source JPDD_D only those tuples whose JPDD_A

?jpd = ?jpa

JPDD_B

?jpa

?jpb

∪

∩ ?jpd = ?jpb

JPDD_D ?jpd

\

JPDD_C ?pjc

?jpd = ?jpc

Fig. 8. Relationships between JPDDs (cf. [25])

Designing Design Constraints in the UML Using Join Point Designation Diagrams

65

variables have values that are also bound to the corresponding variables in the result set of the target JPDD_B. An exclusion removes all elements from the result set of the source JPDD_D that bind the variables to values that appear in the corresponding variables in the result set of the target JPDD_C. Figure 9 illustrates an application of an exclusion relationship whereby the JPDD GUIPOSelection corresponds to the JPDD specified in Figure 6 (internal details are omitted here). The JPDD Preselection binds the variable ?C to the class ClassA. As specified in the mapping specification (attached to the combination relationship) the exclusion removes all elements from the result set of JPDD GUIPOSelection that bind the variable ?C to values that appear in the corresponding variable ?C in the result set of the target JPDD (i.e. ClassA). Consequently, the result set of GUIPOSelection, applied to the model from Figure 7 is “{?C=ClassB, ?PO=POA}, {?C=ClassB, ?PO=POB}”. Preselection

ClassA ?C

\ ?C=?C GUIPOSelection

?C ?PC

Fig 9. Example of exclusion relationship

4.2 Designing Constraints Using JPDDs This section explains how JPDDs can be used to specify design constraints. The following section discusses aspects that have to be considered when specifying constraints are specified by using JPDDs. A special focus is on the use of identifiers. Then, a necessary extension of JPDDs in order to specify constraints is explained. 4.2.1 Specifying Design Violations Before discussing how to design constraints using JPDDs, it is necessary to determine what a violation of a constraint is. Mostly, constraints are specified in a form of an all quantifier: for all elements that fulfill a certain predicate certain other predicates have to be fulfilled. For example, in order to revisit the motivating example, all classes that fulfill the view predicate (i.e. which have the corresponding stereotype) must have a corresponding relationship to a controller. Of course, an alternative would be to replace the all quantifiers by a negation of an existence qualifier. As a result, a constraint violation would be detected by finding a single element that does not comply with the constraint. A consequence of applying these considerations to JPDDs is that it is sufficient to encode a violation of a constraint in terms of one JPDD. Then, it needs to be checked whether the JPDD specifies a situation that can be found in a set of models to be checked.

66

V. Stricker, S. Hanenberg, and D. Stein

IteratorMethod .*

Iterator

f:Attributes F Operations

.*Iterator(..) : {not} .*

[0..*] .*

Fig. 10. Iterator-method constraint

Although theoretically correct, the previous consideration has a practical disadvantage. Developers who check whether their models fulfill all constraints are not only interested in a result like “the model does not fulfill the constraints”. Such developers are interested in error messages that guide them to those places in their models that violate a constraint so that they can correct the design errors quickly (if possible). Figure 10 illustrates a simple constraint from the same company introduced in section 2. The constraint expresses that each method whose name has the postfix Iterator must have Iterator (or a subtype of Iterator) as return type. Note, that the identifier ?It appears twice in this example: In the return type of the method and as a class. Figure 11 illustrates an excerpt from a class model that violates the constraint: Since there is a method mapIterator within the class IteratorFactory that does not return a subtype of Iterator, the JPDD can detect a violation. Furthermore, there is a method linkedListIterator that returns MyObjectB, which is also not a subtype of Iterator. Factories Iterator

IteratorFactory f:Attributes F Operations

collectionIterator(c:Collection) : Iterator listIterator(l:List) : ListIterator treeIterator(t:Tree) : TreeIterator mapIterator(m:map) : MyObjectA linkedListIterator(l:LinkedList): MyObjectB …

ListIterator TreeIterator MyObjectA

MyObjectB

Fig. 11. Model violating the constraint IteratorMethod

It seems obvious that the constraint checker should notify the developer about both methods that are considered to be wrong. However, a general problem is that it cannot be concluded from the JPDD itself what the failing elements in the model are. Possible wrong elements are the package that contains IteratorFactory, the IteratorFactory, the methods mapIterator and linkedListIterator, or the classes MyClassA and MyClassB (or maybe even combinations of them). Consequently, it cannot be concluded from the JPDD how many design errors exist, nor where the developer should start his/her search for finding and correcting the design errors.

Designing Design Constraints in the UML Using Join Point Designation Diagrams

67

IteratorMethod Attributes

Iterator

.*

Operations

.*Iterator(..) : {not} .*

[0..*] .*

?m

Fig. 12. Iterator-method constraint (revised)

Due to this ambiguity, developers of constraints need to make explicit which elements should be told to the developer who checks his/her model for validity. This is done by specifying corresponding export parameters. Constraint checking based on JPDDs evaluates constraints and creates an error notification for each bound value in the result set of the selection. According to this, figure 12 revises the previous constraint by adding an additional identifier ?m. The identifier binds the method with the postfix Iterator and the wrong return type, and exports it via the parameter box. Consequently, the developer now considers only the method to be wrong. The classes MyObjectA and MyObjectB are not considered to violate the constraint. 4.2.2 Extension of JPDDs for constraints In order to specify constraints using JPDDs it is necessary to annotate those JPDDs that represent constraints. Examples of JPDDs that do not represent constraints are those ones that are used by another JPDD by means of JPDD combination relationships. Such JPDDs should not be evaluated alone in order to determine design errors. ClassesWithCapitalLetters {A-Z}.* ?C

\ ?C=? C WrongClassName .* ?C

Fig. 13. Constraint consisting of more than one JPDD

Figure 13 illustrates an example of such a JPDD. The JPDD ClassesWithCapitalLetters selects all classes whose name starts with capital letters (expressed by the corresponding regular expression {A-Z}.*). The JPDD WrongClassName selects all classes that start with just any letters. The intention of the constraint is to make explicit the correct designation of class names (in ClassesWithCapitalLetters) and to select all classes that do not comply with it. From the example, it seems obvious that the JPDD ClassesWithCapitalLetters should not be executed on a model alone, but only in combination with WrongClassName.

68

V. Stricker, S. Hanenberg, and D. Stein

However, from the relationship between both JPDDs this cannot be concluded: it is also possible that WrongClassName is only an extension/refinement of a previous selection. Hence, JPDDs need to be extended in order to handle constraints. Therefore, we extend JPDDs by means of a special stereotype «Constraint». Only JPDDs annotated with this stereotype are considered for constraint checking, all others are not. 4.2.3 Representing Design Errors While the detection of erroneous UML elements according to the above description is not hard, a problem occurs in the representation of errors. The company mentioned in section 2 has diagrams that consist of more than thousand classes and it turned out that sometimes a few hundred methods, fields or classes violate the set of constraints. Because of that, a pure textual representation of error messages is not acceptable, because it is hard to determine for developers in which parts of the models the constraint violations occur. Furthermore, for developers it is essential that error messages can be used in a way that permits developers to see directly where a given violation of constraints comes from. Due to this, constraint checking based on JPDDs considers the representation of design errors in terms of the UML: the result of a constraint check is a UML diagram in which all elements that violate a given constraint are annotated. This annotation is achieved by special stereotypes that are constructed only for the purpose of representing design errors. Such stereotypes are prefixed with the string “false_” in addition to the name of the responsible constraint. In order to make it easier for developers to determine within a large set of diagrams which elements are erroneous, all elements that contain erroneous elements are annotated with a special stereotype prefixed with “containsFalse_” followed by the corresponding constrained name. <> Factories

<> IteratorFactory f:Attributes F Operations

collectionIterator(c:Collection) : Iterator listIterator(l:List) : ListIterator treeIterator(t:Tree) : TreeIterator <> mapIterator(m:map) : MyObjectA <> linkedListIterator(l:LinkedList): MyObjectB …

Fig. 14. Model violating the constraint IteratorMethod

Figure 14 illustrates the representation of design errors of the Iterator method constraint (Figure 12) applied to the previous UML model (Figure 11). The constraint exports the failing methods as parameter. Consequently, the corresponding methods are annotated with the stereotype <>. All owning elements (the class IteratorFactory as well as the package Factories) receive the stereotype <>.

Designing Design Constraints in the UML Using Join Point Designation Diagrams

69

By annotating owning members that contain erroneous elements, it is easy for the developer to navigate with his/her UML tool to the corresponding element that cause the violation of the constraint.

5 Revisiting the Problem In this section we revisit the specification of constraints in the context of the ModelView-Controller architectural pattern. We discuss how we specify a corresponding constraint definition and what the implications of such constraint definitions are. First, the correct relationships between views and controllers are specified in a JPDD ViewsControllers (see Figure 15). The class descriptions just contain the required stereotypes and the association between the classes. The association has the necessary name “has” with the corresponding role names “view” and “controller”. Since it is permitted to have cardinalities 1 as well as 1..* on the view-side, we specify a corresponding criterion [1..*] which matches both. Since on the controller side only a cardinality of 1 is permitted, our specification includes this cardinality. The specified JPDD matches all classes which have the corresponding stereotypes as well as the corresponding relationship. The JPDD itself does not describe a selection criterion of elements that contradict with the constraint we are about to define. Hence, the JPDD is not annotated with the stereotype «constraint». The benefit of a diagram such as Figure 15 is that it directly illustrates the underlying architectural guideline. Instead of specifying what is not valid, the constraint includes a JPDD that can be directly used by the developers as a template to create their valid models. Since this diagram directly illustrates the correct use of view and models, we consider it as easily readable and understandable. ViewsControllers <> .* view

[1..*] has

[1] controller

<< controller>> .* ?v ?c

Fig. 15. Correct use of views and models

We need to decide which elements should be bound to and exported by variables. In principle, we have the possibility to bind all elements (classes, associations, association names, cardinalities) to corresponding identifiers. The positive benefit of such an approach would be that errors can be communicated on a very fine-grained level: For example, it would be possible to indicate for an association that has e.g. wrong cardinalities which of those cardinalities contradict the constraint. However, the negative impact would be that the diagram becomes polluted with identifiers, since in such a case the export box would have seven instead of two parameters. Furthermore, each element in the diagram would have a corresponding identifier, which from our point of view has a negative impact on the readability and understandability of the JPDD.

70

V. Stricker, S. Hanenberg, and D. Stein

∪

ViewsControllers ?v

?v =?v ?c=?c

?c

AbstractViews <> .*

∪

?v ?c

?v =?v ?c=?c

AbstractControllers <> .* ?v ?c

Fig. 16. Consideration of abstract views and controllers

Since our main intention is to make constraint definitions more understandable for the developers, we decide here to export only the classes. Note that for understanding the constraint definition in Figure 15, it is only necessary to know the binding mechanism of variables in JPDDs and the export of such variables. Based on the ViewsControllers we need to add the possibility that views as well as controllers which are abstract do not violate the constraint. Consequently, we specify a selection criterion that selects all views and controllers which are abstract (and thus extend our "template" of valid elements). Figure 16 illustrates the corresponding two JPDDs in which the classes to be selected are specified to be abstract due to the class identifiers written in italics. Again, the essential part of these JPDDs is the selection of abstract classes by using familiar UML notations.

AbstractControllers

/

?v ?c

?v =?v

<>Views

/

<> .* ?v

?c =?c

<>Controllers <> .* ?c

Fig. 17. Constraints for views and controllers

Based on these JPDDs we are able to specify the constraint (Figure 17): we need to select all views and controllers that do not match the previous selection criterions. Since we are interested in notifying developers about errors in view as well as in controller definitions, we specify a constraint for views as well as for controllers. Both constraints have an exclusion relationship to the AbstractControllers selection. The specification of the view-controller constraint requires five JPDDs. Two of them are the constraints – a single constraint handles the corresponding wrong views and controllers. Only two variables are exposed by the non-constraint JPDDs. As

Designing Design Constraints in the UML Using Join Point Designation Diagrams

71

discussed above, a consequence of this is a slightly reduced way of error notification of the results but a better understandable constraint specification.

6 Tool Support The Modeller for JPDDs (M4JPDD) is an Eclipse-based tool which has specially been implemented to support users with an interface to create, to save and to load JPDDs. It is implemented as an Eclipse plugin which is based on several other Eclipse plugins that allow creating graphical editors within the workbench of Eclipse. These plugins are the Eclipse Modeling Framework (EMF), the Graphical Editing Framework (EMF) and UML2. Thus, the M4JPDD editor is also based on the UML meta-model since it uses the UML2 Eclipse plugin that defines all elements of the UML specification. Besides these UML elements, the models of the M4JPDD contain some JPDDspecific selection mechanisms like identifiers etc., which are defined in the editor by using the UML build-in extension mechanisms. It uses a UML2 profile to adapt the UML2 meta-model to its own needs. The JPDDs are stored as XMI-files, which support the interchange of the diagrams with other EMF/UML-based tools. The availability of this format is essential for the constraint checker as no vendor-relevant information has to be considered. It is easy to traverse tree structures when searching for elements of the models and enables the annotation of the found constraint violations that can be interpreted by all UML2 based modeling tools. The constraint checker is an extension of the M4JPDD. It includes specific modeling elements as part of the UML2 profile and uses the XMI files of the constraint models to extract the modeled query. The query is performed on the XMI files of the models that have to be checked.

7 Related Work Currently, OCL is the de facto standard in UML to define design constraints. Thus, a lot of approaches and tools have been developed that resort to OCL as a constraint specification means. A survey about such approaches and tools can be found in [3]. One of the first systems supporting OCL was the UML Specification Environment (USE) [23]. This approach makes use of snapshots that represent system states at a particular point in time and that can be used to check the constraints. The approach is limited to the check of dynamic behavior of the system, and thus it cannot be used to check design constraints, which is the focus of this paper. Other examples of OCL tools are the Dresden OCL Toolkit2, OCL Environment (OCLE) [4], or Octopus3. These tools are a great help in specifying OCL constraints and verifying their compliance in a given model. However, they do not provide a graphical means for modeling the constraints. Furthermore, they mainly focus on the evaluation of well-formedness-rules and/or invariants. The approach presented in this paper, on the contrary, takes a different (slight and subtle) perspective on the 2 3

http://dresden-ocl.sourceforge.net/ http://octopus.sourceforge.net/

72

V. Stricker, S. Hanenberg, and D. Stein

specification of constraints (i.e. specification of design guidelines instead of the specification of invariants, etc.), and it provides a visual notation to represent them. Checking Design Constraints An approach that specifically focuses on checking the correct usage of design patterns in UML models is presented in [14]. The approach uses patterns described in a Role Based Metamodelling Language (RBML) [13] that defines patterns in terms of pattern roles. The patterns and the models that have to be checked are composed into blocks. If all blocks are correct the class diagram is assumed to be correct regarding the pattern. Tool support is provided by a prototype which is an add-in for Rational Rose. However, a graphical visualization of errors is not supported. Another (rule-based) approach that aims at finding inconsistencies which go beyond the well-formedness-rules of UML is presented in [15]. Different classes of inconsistencies, which also include the class ‘Conformance to Constraints and Standards’, are defined. This class explicitly examines design aspects which are not within the scope of the requirements but are caused by design patterns and standards. The presented tool RIDE can be integrated in existing UML tools but does not support any graphical notation of constraints or support for error correction. There are also approaches for using (aspect-oriented) programming language constructs for checking design failures. First, the programming language AspectJ [5] provides language constructs (so-called error or warning declaration) that permit to specify constraints that are checked at compile time. However, due to the rather restrictive pointcut language, the constraints that can be expressed using such language features are rather trivial. A similar approach is proposed by Morgan et al. [18]: a language is proposed, which is called Program Description Logic (PDL), similar to AspectJ, which actually allows specifying violations of design rules and detecting them. “PDL is based on a fully static and expressive pointcut language.” Instead of imposing design rules on pointcuts, this approach rather uses them to specify design rules on object-oriented systems. Constraint Visualization Other approaches to visualize constraints include Constraint Diagrams [11] or Visual OCL [1, 12]. Constraint Diagrams, as well as their successors Spider Diagrams [8], represent a graphical notation to specify invariants on objects and their associations (i.e., links) depending on the state they are in. They strictly focus on runtime constraints. As a consequence, the notation does not permit the specification of design constraints (which are static). For example, the notation does not provide means to refer to particular attributes and operations, and thus no design constraints can be specified on their signatures. Visual OCL is a graphical notation to express OCL constraints. It provides graphical symbols for all OCL keywords. Although Visual OCL tries to use existing UML notation elements, these are not rich enough to express all elements defined in the OCL meta-model so that developers need to get used to the new notation elements. These symbols are part of every diagram, which sometimes adds significant 'noise' to the visualization. When specifying design constraints, such as those presented in this paper, Visual OCL constraints would be specified in terms of UML meta-model elements – rather than in terms of UML's concrete syntax, as the approach presented in

Designing Design Constraints in the UML Using Join Point Designation Diagrams

73

this paper does. In consequence, developers are burdened with the full complexity of UML's metamodel – something which they usually do not encounter get in touch with during daily work. Use of Concrete Syntax The idea of specifying something in terms of user entities rather than in terms of metadescriptions compares to the approach of Query-By-Example (QBE) [29], which is a common query technique in the database domain nowadays. The idea is to specify sample model entities, having sample properties, and determine how selected model elements must relate to (may deviate from) such samples. JPDDs have followed this idea since a long time (cf. [24]). The idea has been adopted recently by other modeldriven approaches, too, such as model transformation languages (e.g. [9, 28]). Model Queries As proposed in this paper, design constraints on models can be specified in terms of (sequential) model queries. Accordingly, the approach presented in this paper relates to the magnitude of query and transformation languages in the domain of modeldriven development. Examples are UMLAUT [7], KerMeta [17], MOLA [10], and QVT [19]. A common problem of these transformation languages is that they specify model queries in terms of meta-model entities. This is of great value (if not inevitable) when a transformation needs to be applied to multiple model elements of different metatypes, or to model elements that have no explicit graphical representation in user model diagrams. However, it hinders the comprehension of the query by the uneducated developer.

8 Discussion and Conclusion In this paper we described the specification of design constraints on UML models using Join Point Designation Diagrams (JPDDs). We motivated the need for such an approach by describing the difficulties developers have who are familiar with the UML but not with the constraint language. The motivation was given by a concrete industry setting in the area of model-driven development. We introduced JPDDs which are lightweight extensions of the UML and which provide means to specify constraints using JPDDs. We discussed the impact of constraint design using JPDDs by illustrating the use of the provided relationships and identifiers. Furthermore, we described the need for a visual representation of design error notifications in order to guide the developers through finding and fixing design errors. Finally, we showed the applicability of JPDD-based constraints by means of an example. We consider the main advantage of JPDD-based constraints that they permit to specify constraints in a way that developers are immediately aware how a model that does not violate a given constraint should look like. For example, as exemplified in the MVC-example, developers directly see from the constraint specification (in the JPDD ViewsControllers) how views and controllers are related to each other. Consequently, the understanding of constraints and detecting errors in models becomes relatively easy. Furthermore, since JPDDs are closely related to the UML, developers

74

V. Stricker, S. Hanenberg, and D. Stein

can be relatively easy motivated to express further design constraints – although they are not educated in a further, additional constraint language. Although we are able to express most of the constraints desired by the mentioned company, there are situations where the expressiveness of JPDDs is not sufficient. The main reason for this problem is that the value bindings of export parameters are not ordered. Furthermore, we currently see situations where it is desirable to provide identifiers which are not only bound to single variables, but to collections of variables. We currently cannot estimate how essential such language features are and how they can be integrated into JPDDs. Hence, such a study is future work to be done. Our proposal of error visualization turned out to be very useful for the practical application of JPDDs. Nevertheless, we also see difficulties. For example, an element can be considered to be erroneous due to some missing owned elements. In such a situation is seems desirable not only to mark this element as erroneous, but to make a proposal as to which owned elements are missing. Our current approach is not able to make such a proposal, but we are investigating means to integrate such proposals. Our current tool support for JPDDs is quite restricted. We heavily rely on the XMI format provided by the current UML2 API. This reduces the applicability of JPDDbased constraints, since in order to specify such constraints in other tools, it is currently necessary to convert XMI formats. Although these problems exist, we are convinced that constraints on models should be expressed in terms of the modeling language itself, since this reduces the effort for developers to learn and understand the constraint language itself as well as constraints written in it. From our point of view, JPDDs can significantly improve the development of software developed in a model-driven way.

References 1. Bottoni, P., Koch, M., Parisi-Presicce, F., Taentzer, G.: A Visualization of OCL Using Collaborations. In: Gogolla, M., Kobryn, C. (eds.) UML 2001. LNCS, vol. 2185, pp. 257–271. Springer, Heidelberg (2001) 2. Buschmann, F., Meunier, R., Rohnert, H., Sommerlad, P., Stal, M.: Pattern-Oriented Software Architecture: A System of Patterns. John Wiley & Sons, Chichester (1996) 3. Cabot, J., Teniente, E.: Constraint Support in MDA Tools: A Survey. In: Rensink, A., Warmer, J. (eds.) ECMDA-FA 2006. LNCS, vol. 4066, pp. 256–267. Springer, Heidelberg (2006) 4. Chiorean, D., Paşca, M., Cârcu, A., Botiza, C., Moldovan, S.: Ensuring UML models consistency using the OCL Environment. In: Workshop Proc. on OCL 2.0 – Industry Standard or Scientific Playground?, November 2004. ENTCS, vol. 102, pp. 99–110. Elsevier, Amsterdam (2004) 5. Colyer, A., Clement, A., Harley, G., Webster, M.: Eclipse AspectJ: Aspect-Oriented Programming with AspectJ and the Eclipse AspectJ Development Tools. Addison-Wesley, Reading (2005) 6. Hanenberg, S., Stein, D., Unland, R.: From aspect-oriented design to aspect-oriented programs: tool-supported translation of JPDDs into code. In: Proc. of AOSD 2007, Vancouver, Canada, March 2007, pp. 49–62. ACM, New York (2007)

Designing Design Constraints in the UML Using Join Point Designation Diagrams

75

7. Ho, W.M., Jézéquel, J.M., Le Guennec, A., Pennaneac’h, F.: UMLAUT: An Extendible UML Transformation Framework. In: Proc. of ASE 1999, Cocoa Beach, Florida, October 1999, pp. 275–278. IEEE, Los Alamitos (1999) 8. Howse, J., Molina, F., Taylor, J., Kent, S., Gil, J.Y.: Spider Diagrams: A Diagrammatic Reasoning System. Journal of Visual Languages & Computing 12(3), 299–324 (2001) 9. Jayaraman, P., Whittle, J., Elkhodary, A., Gomaa, H.: Model composition in product lines and feature interaction detection using critical pair analysis. In: Engels, G., Opdyke, B., Schmidt, D.C., Weil, F. (eds.) MODELS 2007. LNCS, vol. 4735, pp. 151–165. Springer, Heidelberg (2007) 10. Kalnins, A., Barzdins, J., Celms, E.: Model Transformation Language MOLA. In: Aßmann, U., Aksit, M., Rensink, A. (eds.) MDAFA 2003. LNCS, vol. 3599, pp. 62–76. Springer, Heidelberg (2005) 11. Kent, S.: Constraint Diagrams: Visualizing Assertions in Object-Oriented Models. In: Proc. of OOPSLA 1997, Atlanta, Georgia, October 1997, pp. 327–341. ACM, New York (1997) 12. Kiesner, C., Taentzer, G., Winkelmann, J.: Visual OCL: A Visual Notation of the Object Constraint Language, TR 2002/23, Technical University Berlin (2002) 13. Kim, D., France, R., Ghosh, S., Song, E.: A Role-Based Metamodeling Approach to Specifying Design Patterns. In: Proc. of COMPSAC 2003, Dallas, Texas, November 2003, pp. 452–459. IEEE, Los Alamitos (2003) 14. Kim, D.-K., Shen, S.: An approach to evaluating structural pattern conformance of UML Models. In: Proc. of SAC 2007, Seoul, Korea, March 2007, pp. 1204–1208. ACM Press, New York (2007) 15. Liu, W.Q., Easterbrook, S., Mylopoulos, J.: Rule-Based Detection of Inconsistency in UML Models. In: Workshop Proc. on Consistency Problems in UML-Based Software Development, UML 2002, Dresden, Germany, October 2002, pp. 106–123 (2002) 16. Mellor, S., Scott, K., Uhl, A.: MDA Distilled: principles of model-driven architecture. Addison-Wesley Professional, Reading (2004) 17. Muller, P.A., Fleurey, F., Jézéquel, J.M.: Weaving executability into object-oriented metalanguages. In: Briand, L.C., Williams, C. (eds.) MoDELS 2005. LNCS, vol. 3713, pp. 264–278. Springer, Heidelberg (2005) 18. Morgan, C., Volder, K.D., Wohlstadter, E.: A static aspect language for checking design rules. In: Barry, B.M., de Moor, O. (eds.) Proceedings of the 6th International Conference on Aspect-Oriented Software Development (AOSD 2007), Vancouver, British Columbia, Canada, March 12-16, 2007. ACM International Conference Proceeding Series, vol. 208, pp. 63–72. ACM, New York (2007) 19. OMG, MOF QVT final adopted specification, Version 1.0 beta 2 (OMG Document: ptc/05-11-01) (November 2005) 20. OMG, MOF XMI Mapping Specification, Version 2.1 (OMG Document: formal/05-09-01) (September 2005) 21. OMG, Unified Modeling Language Specification, Version 1.5 (OMG Document: formal/03-03-01) (March 2003) 22. Reenskaug, T.: Models - Views - Controllers, Xerox PARC technical note (December 1979) 23. Richters, M., Gogolla, M.: Validating UML Models and OCL Constraints. In: Evans, A., Kent, S., Selic, B. (eds.) UML 2000. LNCS, vol. 1939, pp. 265–277. Springer, Heidelberg (2000)

76

V. Stricker, S. Hanenberg, and D. Stein

24. Stein, D., Hanenberg, S., Unland, R.: A Graphical Notation to Specify Model Queries for MDA Transformations on UML Models. In: Aßmann, U., Aksit, M., Rensink, A. (eds.) MDAFA 2003. LNCS, vol. 3599, pp. 77–92. Springer, Heidelberg (2005) 25. Stein, D., Hanenberg, S., Unland, R.: On relationships between query models. In: Hartman, A., Kreische, D. (eds.) ECMDA-FA 2005. LNCS, vol. 3748, pp. 254–268. Springer, Heidelberg (2005) 26. Stein, D., Hanenberg, S., Unland, R.: Query Models. In: Baar, T., Strohmeier, A., Moreira, A., Mellor, S.J. (eds.) UML 2004. LNCS, vol. 3273, pp. 98–112. Springer, Heidelberg (2004) 27. Warmer, J., Kleppe, A.: The Object Constraint Language: Precise Modelling with UML. Addison-Wesley, Reading (1998) 28. Whittle, J., Moreira, A., Araújo, J., Rabbi, R., Jayaraman, P., Elkhodary, A.: An expressive aspect composition language for UML state diagrams. In: Engels, G., Opdyke, B., Schmidt, D.C., Weil, F. (eds.) MODELS 2007. LNCS, vol. 4735, pp. 514–528. Springer, Heidelberg (2007) 29. Zloof, M.: Query-by-Example: A Data Base Language. IBM Systems Journal 16(4), 324–343 (1977)

Stream-Based Dynamic Compilation for Object-Oriented Languages Michael Bebenita, Mason Chang, Andreas Gal, and Michael Franz University of California, Irvine

Abstract. Traditional just-in-time compilers operate at the granularity of methods. Compiling a method in such a compiler is an atomic operation that can require substantial amounts of processing time, resulting in execution pauses in interactive computing environments. We describe a new software architecture for dynamic compilers in which the granularity of compilation steps is much finer, forming a “pipeline” with completely linear runtime behavior, and in which there are only two write barriers. This means that on future many-core platforms, the compiler itself can be parallelized, providing high-throughput dynamic compilation without execution pauses. As our prototype for Java demonstrates, stream-based compilation lends itself very naturally to an object-oriented implementation.

1 Introduction Most just-in-time (JIT) compilers operate at the granularity of methods. Apart from the fact that it happens at run-time, the actual process of compilation is often not much different from the way this is done in traditional compilers: First, an intermediate representation incorporating a control-flow graph is built for the entire method. Then, a series of optimization steps is applied, until finally a program in the target instruction set is obtained. An important downside of this approach is that the compilation of each method is an atomic operation that cannot be decomposed, due to complex internal data structures and interdependencies. We have been working on an alternative compilation strategy in which no controlflow graph is ever constructed, but in which relevant (i.e., frequently executed) control flows are instead discovered lazily during execution. We have built a family of compilers [12,8,11] that dynamically detect “hot” code paths, so-called “traces,” and that then generate code on-the-fly for exactly these code paths—all other parts of the program are executed by interpretation only. In this paper, we will focus on our dynamic JIT compiler for the Java Virtual Machine (JVM) bytecode language (JVML). Our compiler first dynamically detects loop headers and then incrementally discovers alternative paths through the resulting loops. For example, an if statement inside of a while statement corresponds to two different paths through the while loop. Each time that a new path through the loop that hasn’t been recorded before becomes sufficiently “hot,” we recompile all known paths through the loop, re-balancing the processor resources expended on the alternatives. In the following, we report how re-compiling the different code paths through a loop can be pipelined in a compiler and implemented quite elegantly in an object-oriented M. Oriol and B. Meyer (Eds.): TOOLS EUROPE 2009, LNBIP 33, pp. 77–95, 2009. c Springer-Verlag Berlin Heidelberg 2009

78

M. Bebenita et al.

fashion as a series of filters through which a program passes on its way from intermediate representation to target code. Each compiler phase is implemented as a separate filter that manipulates the instruction “stream” by altering, inserting, re-ordering, or removing instructions as the “stream” passes through it. Even more interestingly, the pipeline requires only two synchronization points, meaning that the compilation activity can be highly parallelized on a a future many-core platform in which there are idle processor resources available. The remainder of this paper is organized as follows. In Section 2, we describe how alternative paths through a program are discovered lazily and recorded in a representation that we call a “Trace Tree”. In Section 3, we show how such Trace Trees can be serialized in such a way that correct compilation semantics result from a set of linear passes over the serialized tree. This is the basis for the idea of pipelining these passes, which is discussed in Section 4. In Section 5 we report on a series of benchmarks that demonstrate the performance of our compiler pipeline. In particular, run-time is directly proportional to the length of the serialized tree. Related work is discussed in Section 6. We conclude in Section 7.

2 Trace Trees Our stream-based JIT compiler for the Java Virtual Machine (JVM) bytecode language (JVML) is embedded in a mixed-mode execution environment, i.e., bytecode is initially interpreted and only frequently executed (“hot”) code is ever compiled. To detect such “hot” code regions that are suitable candidates for dynamic compilation, we use a simple heuristic first introduced by Bala et al. [2]. In this scheme, the interpreter keeps track of backward branches; the targets of such branches are often loop headers. For each target of a backward branch, we keep a counter of how often a backward branch has terminated at this location. When the counter exceeds a certain threshold, a loop header has been discovered. The interpreter then records the instruction path subsequently taken, starting with the loop header. If that path leads back to the loop header within a certain observation window and fulfills certain other criteria, then we have recorded an initial trace through the loop. There may be other paths through the loop that may be disovered later. The trace is then compiled and the resulting target code is executed. Forward branches inside of a trace indicate alternative paths that may not yet have been discovered by our compiler. For example, in the case of our if statement inside of a while statement, when we initially discover the loop, we only follow one path through the loop. If indeed the if condition is so biased that only one of the two alternatives is ever taken sufficiently often, then our compiler will never generate code for the rare alternative. If the second path is common, then it will eventually be compiled as well. Branches to paths that have not yet been seen by our compiler are compiled into guard instructions. When such a guard fails, that means that we have arrived at a path that has not yet been compiled. This is called a “side exit” from the loop. In this case, the compiled code hands control back to the interpreter. Since this can be expensive, our

Stream-Based Dynamic Compilation for Object-Oriented Languages

79

JIT compiler attempts to compile the alternative path that led to the side exit as well. For this, at every side exit we resume interpretation, but at the same time we restart the trace recorder to record instructions starting at the side exit point. These secondary traces are completed when the interpreter revisits the original loop header. This results in a tree of traces, spanning all observed paths through the “hot” code region. The rest of this paper explains how we compile such Trace Trees. We also inline method invocations into the trace. In case of a static method invocation, the target method is fixed and no additional runtime checks are required. For dynamically dispatched methods, the trace recorder inserts a guard instruction to ensure that the same actual method implementation that was found during the trace recording is executed. If the guard fails, a regular dynamic dispatch is repeated in the interpreter. If the guard succeeds, we have effectively performed method specialization on a predicted receiver type. Since our compiler can handle multiple alternative paths, eventually our compiler specializes method invocations for all commonly occurring receiver classes. Creating separate traces for all alternative paths through a loop can potentially lead to very large Trace Trees, especially if separate specialized versions of the same code are being generated for many different receiver classes. Hence, current trace compilers use heuristics to limit tree growth. While this is necessary in contexts in which resources are limited, such as just-in-time compilers for embedded and portable computers, it may not be the best strategy for server-class dynamic compilers. The work presented here shows that if memory consumption is not a concern and a just-in-time compiler keeps growing Trace Trees to very large sizes to capture all executed program paths, compilation effort will still be linear in the size of the tree. Moreover, compilation itself can possibly be parallelized. This means that it may very well be beneficial to capture all the paths through a server program. The linear runtime complexity behavior that we report in this paper is a significant result that will hopefully aid future designers of server-class dynamic compilers, while the simplicity and elegance of the pipeline-based approach will likely appeal to compiler designers across the whole spectrum of target platforms.

3 Compiling Traces Trees with Tree Serialization During trace recording, we perform stack deconstruction and generate an equivalent three address code intermediate representation of JVM bytecode. Each instruction points to the instructions that generated the values of its operands and also points to the instruction immediately preceding it. If a side exit occurs, we link the first instruction in the side trace to the guard instruction that caused the side exit. Thus, a guard instruction can connect at most two traces and each node in a Trace Tree can have at most two successors linking to it. Effectively, a Trace Tree is structured like a binary tree with reversed edges. For example, the code fragment in Figure 1 is eventually transformed into a Trace Tree such as the one in Figure 2. The exact order in which the different paths through the code are discovered and ultimately appear in the tree is correlated to their execution frequency.

80

M. Bebenita et al.

while (...) do { . . . D := ...; // initial definition of D . . . if (... != ...) then { . . . ... := ...D...; // first use of D in an expression . . . ... := ...D...; // second use of D in an expression } else { . . . ... := ...D...; // third use of D in an expression . . . if (... <= ...) then { . . . } else { . . . ... := ...D...; // 4th use of D in an expression . . . } } }

Fig. 1. A code fragment for the purpose of illustrating Trace Tree dominator ordering. Inside a loop, value D is computed and subsequently used in 4 other expressions. Note that D may be a subexpression identified by the compiler and need not necessarily be explicitly assigned to a variable.

The branches of a Trace Tree observe an inherent ordering in which dominating instructions always precede dominated ones, which follows naturally from the way traces are recorded. A preorder depth first traversal of the Trace Tree guarantees that all instruction definitions are seen before any of the uses while the reverse of this guarantees that all uses are seen before any of the definitions. This reversed preorder traversal allows us to serialize arbitrary Trace Trees without losing any information. Since a preorder depth first traversal is not possible using our reversed binary tree data structure, we maintain a back pointing linked list of tail instructions as shown in Figure 2 which allows us to traverse the tree in reversed preorder. This is the order in which Trace Trees are serialized for the compilation pipeline. 3.1 Compilation Pipeline The compilation pipeline is organized as a series of filters that are chained together. At each point in the pipeline, a filter can remove instructions, modify already existing instructions, re-order instructions, or simply introduce new instructions. Trace Trees are serialized and piped through the first filter in the compilation pipeline. Each filter receives one instruction at a time and forwards it to the next filter until the instruction reaches the final compilation filter, where the instruction is removed from the pipeline. Filters that remove instructions from the pipeline simply do not forward them along, while filters that introduce instructions forward extra instructions. This pipelined architecture makes it easy to construct complex instruction assembly lines. Further in the paper we discuss how the pipeline can be extended to provide various levels of parallelism.

Stream-Based Dynamic Compilation for Object-Oriented Languages

81

HEADER Definition

D

!= Use

U1 U3 U2

> U4

TAIL

TAIL

TAIL

0

1

2

Reverse Dominator Order

Fig. 2. Example of a Trace Tree for the code fragment in Figure 1. Code paths are added to the Trace Tree in the order in which they are discovered, with the respective branch condition reversed as necessary. As alternative paths through the loop are discovered, they are connected to the guards for the respective branch conditions. The dotted line shows the reversed dominator order used to serialize Trace Trees through the pipeline. This order guarantees that all uses of an instruction are seen before its definition. The reversed dominator order for instructions referencing value D in this tree is U4, U3, U2, U1, D.

3.2 Object-Oriented Modeling: Instruction Class Hierarchy We make extensive use of the “Visitor” Pattern [22,13] in our compiler architecture. Each compilation filter is in fact a visitor over a sequence of instruction objects. Instructions in our intermediate representation are modeled as class objects and are organized in a class hierarchy based on the instruction’s form. The base class of the instruction hierarchy is the abstract Instruction class. From this base class, we derive classes based on the instruction’s operands such as the UnaryInstruction and BinaryInstruction classes. The ADD instruction, for instance, is derived from the BinaryInstruction class as it operates on two operands. This architecture allows us to write optimizations that operate at various levels in the class hierarchy rather than just on leaf classes. It also enables us to neatly separate the implementation of our optimization algorithms from the details of our intermediate representation by making use of the Visitor Pattern.

82

M. Bebenita et al.

Message

Control Messages

Instruction

Unary Instruction

Binary Instruction

Neg

Add

Fig. 3. Partial view of the instruction class hierarchy. Instructions are organized based on instruction form. The root of the hierarchy is the Instruction class, however from an implementation standpoint it is convenient to create another level of abstraction, the message class.

Instruction i = new Instruction.ADD(leftOperand, rightOperand); i.accept(new InstructionVisitor() { @Override void visit(Instruction.BinaryInstruction i) { // visited for all binary instructions } });

Fig. 4. Example code using the visitor method hierarchy to implement a visitor action for an entire class of instructions at once (here binary instructions).

The pipeline architecture necessitates a demarcation of individual trees and traces. When Trace Trees are serialized, a TreeBegin notification is sent down the pipeline, followed by a TraceBegin notification, followed by a sequence of instructions, and then terminating with a TraceEnd and TreeEnd notification. If a tree has multiple traces, then multiple trace begin/end notifications are sent. These control messages extend the Instruction hierarchy to include the root Message class from which both Instruction and each of the individual control message classes inherit (Figure 3). In our architecture, the Instruction class provides an abstract accept method with an InstructionVisitor object as an argument. Each concrete implementation of the Instruction class overrides the accept method and invokes the visit method on the visitor object using the instruction’s own declared type. The default implementation of the InstructionVisitor class defines visit methods for all instruction class types. Figure 4 shows a Java program that uses the visitor method hierarchy to implement a visitor action for an entire class of instructions at once. The Instruction.ADD class provides a concrete implementation of the accept method declared in

Stream-Based Dynamic Compilation for Object-Oriented Languages

83

Instruction. Here, the visit(Instruction.ADD) method is invoked on the visitor object that is passed into the accept method. The base class implementation of the method visit(Instruction.ADD) forwards the invocation to the base class virtual method visit(Instruction.BinaryInstruction), which in turn invokes the catch-all method visit(Instruction). This chain of invocations that bubble up to the root of the instruction hierarchy allows us to hook into the instruction hierarchy at any level by extending the InstructionVisitor class and overriding a visit method. In this example program, by overriding the implementation of the method visit(Instruction.BinaryInstruction) in the anonymous class, we provide an implementation for all binary instructions including the Instruction.ADD instruction which derives from BinaryInstruction. 3.3 Filter Pattern In our pipelined compiler architecture, compilation filters are implemented as classes derived from Filter (Figure 5), which derives from InstructionVisitor. Filters also implement the Sink interface which exposes the receive method. This is the entry point through which instructions are fed into a filter. Filters accept the incoming instruction and visit it using themselves as an instruction visitor. By default, filters bubble up instructions through a chain of accept method invocations all the way to the root of the instruction hierarchy, at which point instructions are forwarded to the next filter. The code for a compilation filter that performs Common Subexpression Elimination using Value Numbering [23] is shown in Figure 6. In this example, the CSE class overrides the visit methods for unary, binary, and constant instructions. All other instructions are allowed to pass through the filter untouched. However, for the above mentioned instructions, the CSE filter performs value numbering and keeps a list of previously seen values. If an instruction’s value has been seen before, the instruction whose value has been seen takes the place of the current instruction. This is done by keeping a pointer in every instruction to an instruction that replaces it. Further in the pipeline, instructions that are not referenced are eliminated. The CSE filter cannot eliminate redundant instructions because it has no knowledge of future instructions that may reference a redundant instruction. Therefore a forwarding pointer is used instead to indicate that an instruction is substituted with another. Throughout the pipeline, we follow this forwarding chain of instructions every time an operand is accessed. 3.4 Baseline Compiler Our baseline non-optimizing compiler is assembled from only two filters (Figure 7), a Clone filter followed by the Assembler filter. The Clone filter makes a copy of every instruction it receives and forwards along the copy instead of the original. This is done to ensure that the compilation pipeline does not adversely change the instructions in the original Trace Tree data structure which is constantly extended and recompiled as new traces are added to the tree. The Assembler filter performs platform specific register allocation and instruction selection/assembly. The final machine code output is then executed by the interpreter once it reaches the Trace Tree’s anchor instruction.

84

M. Bebenita et al.

MessageVisitor void visit(Message m) {} void visit(BeginTree m) { visit((Message)m); } void visit(EndTree m) { visit((Message)m); } ...

InstructionVisitor void visit(Instruction i) { visit((Message)m); } void visit(BinaryInstruction i) { visit((Instruction)i); } void visit(Add i) { visit((Instruction)i); } ...

Filter Sink receiver; Filter(Sink receiver) { this.receiver = receiver; } void receive(Message m) { m.accept(this); } @Override void visit(Message m) { receiver.receive(m); }

<> Sink void receive(Message m);

Fig. 5. UML diagram of the filter architecture. The Filter class extends the Instruction Visitor class and implements the Sink interface. Messages received by the filter through the receive(Message) method are accepted and visited using the filter itself, i.e. m.accept(this). Messages that bubble up the instruction hierarchy are eventually caught in the visit(Message m) method and forwarded to the next filter.

In order to notify the interpreter that some compiled code is available, we utilize a custom tag object. The tag object is associated with a program counter (PC). The tag object contains information specific to the PC including: whether or not this PC is being recorded, if this PC has a compiled trace attached to it, the associated compiled code, and the number of times the PC has been jumped to. The tag object is always

Stream-Based Dynamic Compilation for Object-Oriented Languages

85

public class CSE extends Filter { private IdentityHashMap avail = new IdentityHashMap(); public CSE(Sink receiver) { super(receiver); } private void process(Instruction i) { Value v = i.getValue(); if (avail.containsKey(v)) i.redirectTo(avail.get(v)); else avail.put(v, i); receiver.receive(i); } public void visit(Instruction.UnaryInstruction i) { process(i); } public void visit(Instruction.BinaryInstruction i) { process(i); } public void visit(Instruction.Constant i) { process(i); } }

Fig. 6. Example for a compilation filter that performs Common Subexpression Elimination using Value Numbering. Only constants and unary and binary instructions are processed, all other instructions and messages are passed through to the next filter (receiver). Redundant instructions are replaced by equivalent instructions that were previously observed.

1 2

3

6

Trace Tree Serialization

4 5

7

8

8

7

6

5

4

3

2

1

Clone

Assembler

Fig. 7. The Baseline Compiler pipeline is assembled from only two filters. The Trace Tree is serialized in reverse order through a clone filter, which makes a copy of the Trace Tree data structure, and then through the Assembler filter which assembles machine code.

initialized to a default state of unrecorded. The state of the tag object is changed to compiling when a trace head is attached to it. Finally, the state is changed to compiled once the assembler has finished generating the appropriate code. By exploiting the tag object, we can execute any compiled code as soon as possible. We do this by a simple check in the interpreter. Every time the interpreter does a backwards branch, the tag associated with the branch target is checked to see if the tag has a compiled trace associated with it. If the tag does not have a compiled trace, the fre-

86

M. Bebenita et al.

quency counter on the tag is increased by the interpreter, to note that another backwards branch to the same PC has been taken. Once the frequency counter on the tag passes a threshold, the compiler is invoked, and the compiled code is attached to the tag. Now, the tag will note that it has a compiled trace associated with it. The interpreter then fetches the trace code from the tag, stops interpreting, and executes the compiled code. 3.5 Optimizing Compiler The optimizing compiler pipeline extends the baseline pipeline by inserting a sequence of optimization filters between the Clone and Assembler filters. The optimization pipeline is partitioned into three stages. These stages are separated by Barrier filters. Barrier filters are pipeline buffers that collect instructions and flush them back once all instructions have been collected. We detect that all instructions have been collected when a TreeEnd message is found. In a pipeline without barriers, each instruction traverses the entire pipeline before the next instruction can be sent down the pipeline. This behavior is undesirable for optimization filters that need a full code analysis before proceeding. The full code analysis, which is implemented as a filter, must complete before the optimization filter can start. For this reason we introduce instruction buffers between filters with this type of dependency. The instruction buffer, or barrier, ensures that the code analysis filter has seen the entire Trace Tree before the optimization filter sees any instructions. The architecture of the optimized compiler pipeline (Figure 8) is presented below. Most pipeline filters expect Trace Trees to be serialized in forward or reverse order, with the exception of the Clone, Reverse, Fork, Barrier and Proxy filters which operate on trees serialized in any order. The Reverse filter reverses the order in which Trace Trees are serialized. Trace Trees are always serialized in reverse order. By having trees serialized in reverse order, the use of an instruction is always seen before its definition. This property is useful for the assembler which performs register allocation, while most optimization and analysis filters need to see the definition of an instruction before its use. By reversing the default Trace Tree serialization order we can ensure this property. Therefore we apply the Reverse filter right after the Clone filter at the beginning of the optimization pipeline. At the very end of the pipeline we reverse the order again since our assembler performs bottom up code generation. The Fork filter connects the previous filter to two receiving filters. Thus any instruction that is received by a Fork filter will be forwarded to the two receiving filters. This is useful for debugging purposes, as we can send all instructions to a filter which prints all instructions in the pipeline, while sending the instructions to compilation filters. The Proxy filter is described later in the paper. The first stage of the optimized compiler pipeline is designed to perform basic optimizations and prepare the intermediate code for more complex optimizations that follow later in the second stage of the pipeline. The first optimization filter, Invariant Code Analysis, annotates loop invariant instructions with information that is used in subsequent filters. Following this filter, constant propagation and arithmetic optimizations are performed. These optimizations reduce arithmetic expression complexity. The last optimizations in the first stage are induction variable analysis, which determines the monotonicity of loop variables; escape analysis which identifies objects that escape

Stream-Based Dynamic Compilation for Object-Oriented Languages

87

1 2

3

6

Trace Tree Serialization

4 5

7

8

8

7

6

5

4

3

2

1

Clone

Load Propagation

Reverse

Common Subexpression Elimination

Invariant Code Analysis

Guard Elimination

Constant Propagation

Invariant Code Motion

Arithmetic Optimization

Barrier

Induction Variable Analysis

Constant Propagation

Escape Analysis Invariant Allocation Analysis Barrier

Arithmetic Optimization Common Subexpression Elimination Reverse

Assembler

Fig. 8. The optimizing compiler pipeline is assembled from 19 filters, two of which are pipeline barriers. The Trace Tree is serialized in reverse order through a clone filter, and then through a series of basic optimization and analysis filters which make up the first stage of the pipeline. In the second stage, we perform four complex optimizations. Finally in the last stage, we re-apply basic optimizations and assemble machine code.

into memory along a trace; and finally the invariant allocation analysis. Invariant allocation analysis determines if new object allocations are contained along a trace, and can instead be allocated only once and shared in subsequent iterations. The second stage of the pipeline uses the results of the analysis filters in the first stage to actually perform optimizations. Load propagation is the first optimization in this

88

M. Bebenita et al.

stage. Load propagation uses the results of the escape analysis and invariant allocation analysis filter to eliminate memory load instructions. In certain cases where load instructions follow store instructions with the same address, load instructions can be eliminated by replacing them with the value that was originally stored by the store instructions. In cases where the address written to belongs to an object that was determined to be non-escaping by escape analysis, the store can also be eliminated. Following the load propagation filter we perform common subexpression elimination and then guard elimination. Guard elimination is a very important optimization where redundant guards are eliminated. During trace recording, guards are inserted to ensure that the execution of compiled code does not diverge from the path that was originally recorded by the interpreter. Guard instructions are inserted whenever a branch was taken by the interpreter or whenever an implicit type check, array bounds check, or null check occurred during interpretation. The number of guards has a negative impact on compilation performance as well as on the final produced machine code. For every non-optimized guard the compiler must emit code to test the guard at runtime and potentially restore the interpreter state in case the guard fails. Restoring state is done by executing a code stub that is emitted by the compiler for each individual guard instruction. Guard instructions pose a significant overhead and it is therefore especially important to eliminate as many guard instructions as possible. The last filter in the second stage is the Invariant Code Motion filter which hoists invariant instructions out of loop traces into a prolog region that is executed only once. The third and last stage of the pipeline performs constant propagation, arithmetic optimization and common subexpression elimination again. This is because the second pipeline stage may have exposed new optimizations. Following these filters, a reverse filter changes the order of instructions and feeds them into the assembler.

4 Parallel Compilation and Parallel Pipelining Two techniques can be used to take advantage of machine-level parallelism to speed up compilation: by either having a parallel compilation, a parallel pipeline, or both simultaneously. These are completely independent of each other. Parallel compilation refers to the use of a separate thread to compile Trace Trees while the main thread continues to interpret the program. A parallel pipeline splits up sections of the pipeline and each section runs its own thread in parallel. In each of these implementations, the threads are controlled via a thread pool. Every compiler phase that requires a thread implements the Runnable interface, and asks for a thread from the thread pool to execute. In case of parallel compilation, the Java interpreter continues executing the main program while the Trace Tree is optimized and compiled. For this, the compiler requests a new thread from the thread pool and begins to compile the recorded trace in that new thread. At the same time, the interpreter continues to interpret the program. Once the final compilation result is available, the compilation thread (or the last compilation thread in case of pipelined compilation) notifies the interpreter that a compiled version of the frequently executed bytecode region is available by annotating the loop header bytecode instruction with a pointer to the compiled code.

Stream-Based Dynamic Compilation for Object-Oriented Languages

89

Blocking Queue

Filter Thread Proxy Filter 1

Filter

Filter

Thread Proxy Filter 2

Fig. 9. Visual representation of a parallel pipeline. A proxy object sits between two connecting segments of the pipeline. The proxy itself utilizes and controls the thread it is given. When the proxy receives a new message, it forwards the instruction through all the filters in its segment. Once the instruction has passed through its segment, the thread continues to wait for new instructions. The proxy releases its thread when it receives a TreeEnd message.

In case of a parallel pipeline, we split sections of the pipeline into separate independent threads. Each of these segments are connected via a proxy filter. This proxy filter implements the Runnable interface, and uses a message queue to forward incoming messages (instructions) to a new thread that then invokes the receive method of the receiver filter (which is the next filter in the pipeline). It is possible to have a proxy filter between compiler phases. Our current prototype pipeline runs in parallel in 19 different threads. This is, however, not very efficient due to the communication overhead of the message queue. Instead, we split the pipeline into segments, each segment containing multiple filters (Figure 9). The proxy object contains a LinkedBlockingQueue of messages. Filter A, once it has finished processing its current message, adds the message to the LinkedBlockingQueue of the proxy class. The proxy’s run method constantly polls the LinkedBlockingQueue looking for new messages in the queue. When a new message is discovered, the proxy passes the message and directly calls the filter that the proxy designates as the receiving item, which in the case above is filter B. The proxy thread then takes the message, and runs through all filters in the pipeline segment. When the thread reaches the end of the pipeline segment, the thread will go back to waiting for a new message to arrive in the linked queue in the proxy class. The proxy will return and release its hold on the thread once a TreeEnd message type is found in the queue. Parallel compilation and the parallel pipeline, can be enabled/disabled independently. We can utilize this to scale the trace compiler to use as many cores as the system provides, by tweaking how many threads are used in the pipeline. Our trace compiler can also use as few as two threads by parallelizing the compilation process itself or by parallelizing the pipeline.

5 Benchmarks We have implemented a prototype dynamic compiler that compiles Trace Trees using a compiler pipeline. The prototype compiler compiles Trace Trees recorded for Java programs. The compiler and the compiler pipeline are also implemented in Java [1,21].

90

M. Bebenita et al.

Compilation Time vs. Tree Size 6,000

Time (ms)

5,000 4,000 3,000 2,000 1,000 0

2,500

5,000

7,500

10,000 12,500 15,000 17,500 20,000 22,500 25,000 27,500

Size (Instructions) Sample

Fitted Regression Line

Compilation Time vs. Tree Size 1,500

Time (ms)

1,250 1,000 750 500 250 0

2,500

5,000

7,500

10,000 12,500 15,000 17,500 20,000 22,500 25,000 27,500

Size (Instructions) Sample

Fitted Regression Line

Fig. 10. Linear Regression - Compilation Time vs. Tree Size. The first graph shows the compilation times when executing our compiler through bytecode interpretation. The second graph shows compilation times for running the compiler pipeline as compiled native code.

Our system runs on top of Apple’s Java VM 6 (Developer Preview). All benchmarks were performed on a Apple MacBook 2.16 Ghz Intel Core 2 Duo with 2 GB RAM. To compensate for measurement errors, each test was run ten times. Our graphs aggregate the results for all ten samples. Figure 10 shows the results for compiling Trace Trees with increasing size using the prototype compiler pipeline. The compilation time increases linearly with the size of the Trace Tree. The upper part of the figure shows a graph that was recorded by running our compiler on top of an interpreting Java VM. The slower execution reduces the impact of measurement errors and influences such as garbage collection. The lower graph shows the performance of the compiler when running as compiled code, in which case our compiler as almost 6 times faster and compiles and optimizes graphs with more than 27,000 instructions in less than one second.

Stream-Based Dynamic Compilation for Object-Oriented Languages

91

Compilation Time vs. Tree Size 5,500 5,000 4,500

Time (ms)

4,000 3,500 3,000 2,500 2,000 1,500 1,000 500 0

2,500

5,000

7,500

10,000 12,500 15,000 17,500 20,000 22,500 25,000 27,500

Size (Instructions) Sample

Fitted Regression Line

Fig. 11. Linear Regression - Compilation Time vs. Tree Size (Mixed Mode)

Figure 11 shows the same benchmark for an underlying “mixed mode” environment that has both a compiler and an interpreter. The performance is still linear in trend, but there are some outliers representing the cases in which the underlying justin-time compiler doesn’t kick in fast enough to accelerate our compilation pipeline. We are confident that in practical use these cases could be handled by appropriate hinting mechanisms to jump-start these particular compilation steps. We performed several experiments with different parallelization granularities, executing different parts of the compiler pipeline in parallel using “Proxy” filters. Our compiler is extremely fast and scales very well. However, the overhead of using a message queue between two threads running different parts of the pipeline for now outweighs the performance gain from overlapping those parts of the pipeline. Splitting the pipeline in 4 threads and compiling a large tree with 27,000 instructions, for example, requires sending 108,000 message across the various message queues involved. Amongst others, this requires allocating 108,000 temporary heap objects to transport the messages, and the threads involved have to perform a synchronization protocol to avoid race conditions. Hence, on current hardware our method is not yet efficient, but the balance will tip in favor of our architecture as many-core processors with higher levels of parallelism become available.

6 Related Work Trace-based dynamic compilation is related to trace scheduling, which was proposed by Fisher [10] to efficiently generate code for VLIW (very long instruction word) architectures. In Fisher’s system, all code was compiled using trace scheduling. Dynamic detection and optimization of “hot” program traces was first introduced in the Dynamo system by Bala et al. [2]. Dynamo is a transparent binary optimizer that records frequently executed traces and optimizes instructions in those traces. In contrast to Dynamo, which optimized only individual traces, our system can optimize

92

M. Bebenita et al.

hierarchies of (potentially nested) traces. Our work also significantly improves on Dynamo in that we are able to optimize away the creation of activation records for inlined procedures and methods. This requires a mechanism that can dynamically create all the “missing” activation records on-the-fly when a “deep bailout” occurs from a guard inside of such an inlined piece of code. The trace-tree representation is closely related to superblocks. In particular, Chang et al.’s work on tail duplication [9] produces an intermediate representation that closely resembles trace-trees. However, in contrast to most superblock approaches that use static analysis to form superblocks [15], our representation is built dynamically at runtime. Our method of selecting only certain “compilation-worthy” parts of methods is similar to region-based compilation by Suganuma et al. [24]. Region-based compilation uses runtime profiling to select code regions for compilation and uses partial method inlining to inline profitable parts of method bodies only. The authors observed not only a reduction in compilation time, but also achieved better code quality due to rarely executed code being excluded from analysis and optimization. Whaley [26] also found that excluding rarely executed code when compiling a program significantly increases compilation performance and code quality. Our approach further reduces the cost for analysis and optimization by eliminating complex control-flows altogether, merely operating on linear sequences of instructions. Path profiling was first proposed by Ball et al. [3]. To collect profiling information about program paths, the authors use a path encoding that produces unique index numbers for program paths. Profiling code is injected into the code such that a unique index number is generated when a certain sequence of basic blocks (path) is executed, and a profiling counter is associated with each such index number that counts the execution frequency. Our trace-based compilation approach is similar to path profiling in that we perform path specific optimization along each trace. Also, the iterative extension of Trace Trees can be seen as a form of dynamic hot path graph construction. The most significant difference is that we do not incur the actual path profiling runtime overhead, because path specificity is an inherent property of our intermediate representation. Thus, we also do not have to deal with compensation code generation, because we already implicitly perform tail duplication as we record traces. Dynamic compilation with traces uses dynamic profile information to identify and record frequently executed code traces (paths). By dynamically adding them to a Trace Tree, and thus iteratively extending that Trace Tree as more traces are discovered, we perform a form of feedback directed optimization, which was first proposed by Hansen [16]. Feedback-directed optimization was used heavily in the SELF system [7] to produce efficient machine code from a prototype-object based language. Feedback directed optimization was subsequently also used in conjunction with other highly dynamic languages such as Scheme [6], Smalltalk [14], and Java. H¨olzle later extended the SELF system to not only use profile-guide compilation, but also adaptive code re-compilation [17]. Kistler and Franz [19,20] further extended this idea and proposed a continuous program optimizer for the Oberon system that continuously re-schedules program instructions and dynamically rearranges the layout of objects during execution to maximize performance.

Stream-Based Dynamic Compilation for Object-Oriented Languages

93

Similar to traditional feedback-oriented and continuous compilers, our trace compiler compiles an initial version of a Trace Tree consisting of a single trace, and then iteratively extends the Trace Tree by recompiling the entire tree. Berndl et al. [4] also investigated the use of traces in a Java Virtual Machine. However, while providing a mechanism for trace selection, the authors do not actually implement any code compilation or optimization techniques based on those traces. Bradel et al. [5] propose using traces for inlining methods calls in a Java Virtual Machine. Similar to our work they use trace recording to extract only those parts of a method that are relevant for the specific caller site. Off-loading dynamic compilation to another execution unit while continuing interpretation is frequently used in commercial virtual machines such as Sun’s Hotspot Virtual Machine [25] and IBM’s J9 virtual machine [18]. Our compilation differs significantly, because it is not merely parallelizable on a macro level at the atomic unit of a method. Instead, its parallelizable in itself and individual compiler passes can execute overlappingly on different execution units. In our own work, our original Trace Tree based compiler [12] for the Java Virtual Machine’s bytecode language has evolved into a family of related compilers based on the same underlying principles, including a just-in-time compiler for Adobe’s Flash language [8] and the TraceMonkey just-in-time compiler for JavaScript that is shipping as part of Mozilla’s Firefox Web browser from version 3.1 onwards [11].

7 Conclusions and Outlook We have presented a new architecture for dynamic compilers that uses a pipeline of “filters” implementing individual compiler phases. Mapping this architecture onto an object-oriented implementation is completely natural and leads to a clear and elegant solution. In our architecture, a Trace Tree is dynamically recorded and extended at runtime and is compiled by serializing the traces in the tree into a single linear sequence of instructions. It is possible to serialize trees in this fashion because the dominance information between traces is preserved if the serialization occurs in reverse recording order of traces. In the resulting serialized stream, every definition occurs before all of its uses. To compile such a serialized tree, it is sent through the pipeline. Each filter in the pipeline can manipulate the instruction stream by modifying, inserting, removing, and re-ordering instructions. The optimization and compilation filters can be put together in different configurations. By directly connecting the final instruction assembly filter to the tree serialization output, for example, we obtain a non-optimizing compiler. Without modifying either of these filters, we can insert optimization filters in between to obtain an optimizing compiler. Beyond being clear and elegant, the object-oriented stream-based architecture of our compiler pipeline lends itself very well for parallelization. “Proxy” filters can be inserted to execute parts of the compiler pipeline in a separate thread. The compiler pipeline in our prototype compiler only requires two synchronization points that cannot be overlapped with the rest of the pipeline. In the benchmarking section we provided a preliminary evaluation of the runtime performance of our compiler. In particular, the compilation cost of the pipeline

94

M. Bebenita et al.

architecture increases linearly with the size of the compiled Trace Trees. This result is remarkable, considering that we provide several aggressive compiler optimizations that have highly non-linear performance characteristics in traditional compilers. Key to the linearity of our compiler’s performance is the linearity of the underlying data structure. On current few-core processors, the communication overhead of using a pipeline in this manner outweighs the advantage of parallelizing compilation. The main contributions of our paper are the decoupling of key components via the object-oriented stream model, and the fact that our compiler pipeline architecture is parallelizable at all. We are not aware of any other dynamic compiler architecture that has the potential to reduce the compilation time for individual compilation units using parallelism. As hardware manufacturers create many-core platforms in which processing resources are abundantly available, the attractiveness of our software architecture will rise quickly.

Acknowledgements The authors would like to thank the anonymous referees for their constructive comments. Parts of this effort have been sponsored by the National Science Foundation under grants CNS-0615443 and CNS-0627747, as well as by the California MICRO Program and industrial sponsor Sun Microsystems under Project No. 07-127. Further support in the form of generous unrestricted gifts has come from Google and from Mozilla, for which the authors are immensely grateful. The U.S. Government is authorized to reproduce and distribute reprints for Governmental purposes notwithstanding any copyright annotation thereon. Any opinions, findings, and conclusions or recommendations expressed here are those of the author and should not be interpreted as necessarily representing the official views, policies or endorsements, either expressed or implied, of the National Science foundation (NSF), any other agency of the U.S. Government, or any of the companies mentioned above.

References 1. Arnold, K., Gosling, J.: The Java Programming Language, 2nd edn. ACM Press/AddisonWesley Publishing Co., New York (1998) 2. Bala, V., Duesterwald, E., Banerjia, S.: Transparent Dynamic Optimization: The Design and Implementation of Dynamo. Hewlett Packard Laboratories Technical Report HPL-1999-78 (June 1999) 3. Ball, T., Larus, J.: Efficient Path Profiling. In: Proceedings of the 29th Annual International Symposium on Microarchitecture, pp. 46–57 (1996) 4. Berndl, M., Hendren, L.: Dynamic Profiling and Trace Cache Generation for a Java Virtual Machine. Technical Report, McGill University (2002) 5. Bradel, B.: The Use of Traces in Optimization. PhD Thesis, University of Toronto (2004) 6. Burger, R.: Efficient Compilation and Profile-Driven Recompilation in Scheme. PhD Thesis, Department of Computer Science, Indiana University (1997) 7. Chambers, C., Ungar, D., Lee, E.: An Efficient Implementation of SELF, a DynamicallyTyped Object-Oriented Language Based on Prototypes. Higher-Order and Symbolic Computation 4(3), 243–281 (1991)

Stream-Based Dynamic Compilation for Object-Oriented Languages

95

8. Chang, M., Smith, E., Reitmaier, R., Gal, A., Bebenita, M., Wimmer, C., Eich, B., Franz, M.: Tracing for Web 3.0: Trace Compilation for the Next Generation Web Applications. In: 5th International Conference on Virtual Execution Environments (VEE) (2009) 9. Chang, P., Mahlke, S., Chen, W., Warter, N., Hwu, W.: IMPACT: An Architectural Framework for Multiple-Instruction-Issue Processors. In: Proceedings of the 18th Annual International Symposium on Computer Architecture, pp. 266–275 (1991) 10. Fisher, J.: Trace Scheduling: A Technique for Global Microcode Compaction. IEEE Transactions on Computers 30(7), 478–490 (1981) 11. Gal, A., Eich, B., Shaver, M., Anderson, D., Kaplan, B., Hoare, G., Mandelin, D., Zbarsky, B., Orendorff, J., Ruderman, J., Smith, E., Reitmaier, R., Haghighat, M.R., Bebenita, M., Chang, M., Franz, M.: Trace-based Just-in-Time Type Specialization for Dynamic Languages. In: ACM SIGPLAN 2009 Conference on Programming Language Design and Implementation (PLDI) (2009) 12. Gal, A., Probst, C., Franz, M.: HotpathVM: An Effective JIT Compiler for ResourceConstrained Devices. In: Proceedings of the 2nd International Conference on Virtual Execution Environments (VEE), pp. 144–153 (2006) 13. Gamma, E., Helm, R., Johnson, R., Vlissides, J.: Design patterns: Elements of Reusable Object-Oriented Software. Addison-Wesley Longman Publishing Co., Inc., Boston (1995) 14. Goldberg, A., Robson, D.: Smalltalk-80: the Language and its Implementation. Addison-Wesley Longman Publishing Co., Inc., Boston (1983) 15. Hank, R., Mahlke, S., Bringmann, R., Gyllenhaal, J., Hwu, W.: Superblock Formation Using Static Program Analysis. In: MICRO 26: Proceedings of the 26th Annual International Symposium on Microarchitecture, pp. 247–255. IEEE Computer Society Press, Los Alamitos (1993) 16. Hansen, G.: Adaptive Systems for the Dynamic Run-Time Optimization of Programs. PhD Thesis, Department of Computer Science, Carnegie-Mellon University (March 1974) 17. H¨olzle, U.: Adaptive Optimization for Self: Reconciling High Performance with Exploratory Programming. PhD Thesis, Stanford University, Department of Computer Science (1994) 18. IBM. WebSphere Everyplace Custom Environment J9 Virtual Machine (October 2006), http://www-306.ibm.com/software/wireless/wece/ 19. Kistler, T., Franz, M.: Continuous Program Optimization: Design and Evaluation. IEEE Transactions on Computers 50(6), 549–566 (2001) 20. Kistler, T., Franz, M.: Continuous Program Optimization: A Case Study. ACM Transactions on Programming Languages and Systems (TOPLAS) 25(4), 500–548 (2003) 21. Lindholm, T., Yellin, F.: The Java Virtual Machine Specification, 2nd edn. The Java Series. Addison Wesley Longman, Inc., Amsterdam (1999) 22. Palsberg, J., Jay, C.: The Essence of the Visitor Pattern. In: The Twenty-Second Annual International Computer Software and Applications Conference, COMPSAC 1998. Proceedings, pp. 9–15 (1998) 23. Rosen, B., Wegman, M., Zadeck, F.: Global Value Numbers and Redundant Computations. In: POPL 1988: Proceedings of the 15th ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages, pp. 12–27. ACM Press, New York (1988) 24. Suganuma, T., Yasue, T., Nakatani, T.: A Region-Based Compilation Technique for Dynamic Compilers. ACM Transactions on Programming Languages and Systems (TOPLAS) 28(1), 134–174 (2006) 25. Sun Microsystems. The Java Hotspot Virtual Machine v1.4.1 (September 2002) 26. Whaley, J.: Partial Method Compilation Using Dynamic Profile Information. In: ACM Conference on Object-Oriented Programming Systems, Languages and Applications, pp. 166–179. ACM Press, New York (2001)

Algebraic Semantics of OCL-Constrained Metamodel Specifications Artur Boronat1 and Jos´e Meseguer2 1

2

Department of Computer Science, University of Leicester [email protected] Department of Computer Science, University of Illinois at Urbana-Champaign [email protected]

Abstract. In the deﬁnition of domain-speciﬁc modeling languages a MOF metamodel is used to deﬁne the main types of its abstract syntax, and OCL invariants are used to add static semantic constraints. The semantics of a metamodel deﬁnition can be given as a model type whose values are well-formed models. A model is said to conform to its metamodel when it is a value of the corresponding model type. However, when OCL invariants are involved, the concept of model conformance has not yet been formally deﬁned in the MOF standard. In this work, the concept of OCL-constrained metamodel conformance is formally deﬁned and used for deﬁning style-preserving software architecture conﬁgurations. This concept is supported in MOMENT2, an algebraic framework for MOF metamodeling, where OCL constraints can be used for both static and dynamic analysis. Keywords: Membership equational logic, OCL invariants, MOF metamodel, static and dynamic analysis of models.

1

Introduction

Model-driven development (MDD) constitutes a paradigm for representing software artifacts with models, for manipulating them and for generating code from them in an automated way. A model can be deﬁned with a so-called domainspeciﬁc modeling language, which provides high-level modeling primitives to capture the semantics of a speciﬁc application domain, such as business processes, conﬁguration ﬁles or web-based languages (see [1] for an annotated bibliography), or it can be deﬁned using a general-purpose modeling language such as the UML. In both cases, the corresponding modeling language is constituted by an abstract syntax and a concrete syntax, which can be either graphical or textual. In this paper, we focus on the formal semantics of a modeling language, by considering its abstract syntax, when it is enriched with additional semantic requirements speciﬁed by OCL constraints. M. Oriol and B. Meyer (Eds.): TOOLS EUROPE 2009, LNBIP 33, pp. 96–115, 2009. c Springer-Verlag Berlin Heidelberg 2009

Algebraic Semantics of OCL-Constrained Metamodel Speciﬁcations

97

The Meta-Object Facility (MOF) [2] standard provides a UML-based modeling language for deﬁning the abstract syntax of a modeling language as a metamodel M , where types are metarepresented in a UML-like class diagram. A MOF metamodel M provides the abstract syntax of a modeling language, but not the semantics of the model conformance relation. We identiﬁed this problem and provided a formal framework where the notions of metamodel realization, of model type M and of model conformance are formally speciﬁed [3,4]. These notions are implemented in MOMENT2 [5], where a metamodel realization is algebraically characterized by a theory in membership equational logic (mel) [6] that is automatically generated from a MOF metamodel M . Within this theory, the carrier of a speciﬁc sort in the initial algebra of the metamodel realization constitutes a model type M , which deﬁnes the set of well-formed models M that conform to the metamodel M . We call such a relation the structural conformance relation, denoted M : M . The static semantics of a metamodel M can be enriched by adding constraints such as invariants, where the concepts that are deﬁned in M can be used. These invariants can be deﬁned with the standard Object Contraint Language (OCL) [7], enhancing the expressiveness of MOF. However, in the MOF and OCL standards, it is not clear how OCL constraints aﬀect the semantics of a model type M deﬁned in a metamodel M , and only implementation-oriented solutions are provided. In this paper, we extend the executable algebraic semantics of MOF metamodels [4] by considering OCL constraints. We build on previous experience on encoding OCL expressions as equationally deﬁned functions [8]. The main new contributions of this work are: (i) the notion of metamodel specification as a pair (M , C ), where M is a MOF metamodel and C is a set of OCL constraints that are meaningful for M ; (ii) an algebraic semantics for metamodel speciﬁcations, so that the structural conformance relation is enriched with the satisfaction of OCL constraints and characterized by equational axioms; (iii) the use of OCL expressions for dynamic analysis using Maude-based veriﬁcation techniques [9]; and (iv) the implementation of these new concepts in the MOMENT2 framework by enabling the validation of OCL constraints over models in the Eclipse Modeling Framework (EMF) [10]. In the following subsection, we describe the application of OCL constraints to deﬁnitions of style-preserving software architecture conﬁgurations. We use this as a running example throughout the paper. 1.1

An Example: Architectural Style Preservation

To illustrate our approach we use a basic speciﬁcation of a software architecture component type, shown in Fig. 1.(a), where a component can be deﬁned as client or server (type attribute) and can be connected to other components. This component type is given as a MOF metamodel M . As explained above, such a metamodel metarepresents a model type M , whose terms constitute well-formed software architecture conﬁgurations. By using such a model type M , we can deﬁne a speciﬁc software architecture conﬁguration where client component instances can connect to other clients or servers, as shown

98

A. Boronat and J. Meseguer

Fig. 1. (a) Metamodel M . (b) Valid conﬁguration. (c) Non style-preserving conﬁguration.

in Fig. 1.(b). A software architecture conﬁguration is a model M that conforms to the model type M , denoted M : M . While a given conﬁguration of component instances can be changed to improve communication among components, there may still be some structural constraints that must be preserved. Constraints of this kind are know as architectural styles, which are speciﬁed by sets of rules indicating which components can be part of the architecture and how they can be legally interconnected [11]. In this paper, we use as an example the client/server architectural style, in which client component instances can only connect to server component instances. The OCL invariant in Fig. 1.(a) represents such an architectural style in the software architecture deﬁnition of the example. We can therefore view the client/server architecture as a metamodel specification (M , C ), where M is the above-deﬁned metamodel for connecting components, and where C consists of the single OCL constraint just mentioned. A style-conformant conﬁguration for the client/server architecture is then a wellformed architectural model M that satisﬁes the previous OCL invariant. The conﬁguration provided in Fig. 1.(c) is not client/server-conformant due to the marked link between the objects c1 and c2. In this paper we provide an algebraic, executable semantics for the constrained conformance relation to both formalize the use of OCL invariants for the deﬁnition of the static semantics of a domain-speciﬁc modeling language by means of a metamodel speciﬁcation (M , C ), and to provide automatic veriﬁcation of the constrained conformance relation. From an operational point of view, this formalization does not only provide a mechanism to check whether or not a model conforms to a metamodel speciﬁcation (M , C ) but also enhances the use of OCL constraints for verifying behavioral speciﬁcations of DSMLs with model checking techniques. In Section 2, we summarize preliminary concepts about mel and Maude. Section 3 gives a summary of the main concepts of the algebraic semantics for MOF: model type M and structural conformance between a model M and its metamodel M , i.e., M : M . These concepts are used in Section 4 to deﬁne an algebraic, executable semantics for OCL expressions. Section 5 gives the algebraic semantics for metamodel speciﬁcations (M , C ), illustrating how OCL can be used for static analysis. Section 6 shows how OCL can be used for

Algebraic Semantics of OCL-Constrained Metamodel Speciﬁcations

99

dynamic, model checking analysis in MOMENT2. Section 7 compares our approach with other approaches that formalize OCL for diﬀerent purposes, and Section 8 summarizes the main contributions of the paper.

2

Preliminaries

A membership equational logic (mel) [6] signature is a triple (K, Σ, S) (just Σ in the following), with K a set of kinds, Σ = {Σw,k }(w,k)∈K ∗ ×K a many-kinded signature and S = {Sk }k∈K a K-kinded family of disjoint sets of sorts. The kind of a sort s is denoted by [s]. A mel Σ-algebra A contains a set Ak for each kind k ∈ K, a function Af : Ak1 × · · ·× Akn → Ak for each operator f ∈ Σk1 ···kn ,k and a subset As ⊆ Ak for each sort s ∈ Sk , with the meaning that the elements in sorts are well-deﬁned, while elements without a sort are errors. TΣ,k and TΣ (X)k denote, respectively, the set of ground Σ-terms with kind k and of Σ-terms with kind k over variables in X, where X = {x1 : k1 , . . . , xn : kn } is a set of kinded variables. Given a mel signature Σ, atomic formulae have either the form t = t (Σequation) or t : s (Σ-membership) with t, t ∈ TΣ (X)k and Σ s ∈ Sk ; and sentences are conditional formulae of the form (∀X) ϕ if i pi = qi ∧ j wj : sj , where ϕ is either a Σ-equation or a Σ-membership, and all the variables in ϕ, pi , qi , and wj are in X. A mel theory is a pair (Σ, E), with Σ a mel signature and E a set of Σ-sentences. The paper [6] gives a detailed presentation of (Σ, E)-algebras, sound and complete deduction rules, and initial and free algebras. In particular, given a mel theory (Σ, E), its initial algebra is denoted T(Σ/E) ; its elements are E-equivalence classes of ground terms in TΣ . Under appropriate executability requirements explained in [9], such as conﬂuence, termination, and sort-decreasingness modulo A, a mel theory (Σ, E), where E = E0 ∪ A, becomes executable by rewriting with the equations and memberships E0 modulo some structural axioms A. Furthermore, the initial algebra T(Σ/E) then becomes isomorphic to the canonical term algebra Can Σ/E0 ,A whose elements are Aequivalence classes of ground Σ-terms that cannot be further simpliﬁed by the equations and memberships in E0 . A rewrite theory [12] is a triple R = (Σ, E, R), where (Σ, E) is a mel theory, and R is a collection of (possibly conditional) rewrite rules of the form t −→ t if C, where t, t are Σ-terms of the same kind, and C is the rule’s condition. Intuitively, what the rewrite theory R speciﬁes is a concurrent system, whose states are elements of the algebraic data type TΣ/E deﬁned by the mel theory (Σ, E), and whose concurrent transitions are speciﬁed by the rules R. That is, a rule t −→ t if C speciﬁes transitions in which a fragment of the current state matches the pattern t, and then in the resulting state that state fragment is transformed by the corresponding instance of t , provided that the condition C is satisﬁed.

100

A. Boronat and J. Meseguer

2.1

Maude

Maude [9] is a declarative language where programs are rewrite theories, and where Maude computations are logical deductions using the axioms speciﬁed in the theory/program. We use an example of a simpliﬁed vending machine for illustrating how Maude can be used to specify systems. A mel theory is represented as a functional module in Maude, enclosed within fmod. . .endfm, where types are syntactically represented as sorts and terms (data) are deﬁned by means of constructors for these sorts. In the module below, we have the sorts Coin, Item and Marking where the sort Marking represents the state of the system. Constants are constructors with no arguments, for instance, $ is a constant of sort Coin, representing a coin, and c, a are constants of sort Item, representing a cake and an apple, respectively. The constructor (blank space) has two arguments of sort Marking. The subsorts Coin Item < Marking are subtype declarations indicating that coins and items is deﬁned as associative and are also terms of sort Marking. The operator commutative. Therefore, a term of sort Marking is a multiset of coins and items. For instance, we could represent two coins, an apple and a cake in our system as a term $ $ a c. fmod VENDING-MACHINE-SIGNATURE is sorts Coin Item Marking . op $ : -> Coin . op c : -> Item . op a : -> Item . subsorts Coin Item < Marking . op __ : Marking Marking -> Marking [assoc comm] .

As described in the Section 2 a mel theory may also contain equationally deﬁned functions. Below we deﬁne a boolean predicate that indicates when a vending machine state (a term of sort Marking) only contains apples. The ﬁrst and second equations indicate that the predicate fails if we ﬁnd either a coin $ or a cake c in the multiset of sort Marking. The last equation indicates that the predicate holds when no other equations can be applied, i.e., when there are only apples a. Note that c M:Marking is a term pattern that is applied modulo associativity and commutativity. op eq eq eq

onlyApples? : Marking -> onlyApples?( $ M:Marking onlyApples?( c M:Marking onlyApples?( M:Marking )

Bool . ) = false . ) = false . = true [owise] .

In the following sections, we encode models as multisets of objects where pattern matching modulo associativity and commutativity is used to match the links between objects. To illustrate the use of memberships, we deﬁne a subsort AppleMarking of Marking for categorizing states that only contain apples. The

Algebraic Semantics of OCL-Constrained Metamodel Speciﬁcations

101

carrier (set of values) of the sort AppleMarking is deﬁned by means of a membership indicating that only those states M:Marking in which there are no cakes can be found in AppleMarking, that is onlyApples?(M:Marking) = true. sort AppleMarking . subsort AppleMarking < Marking . cmb M:Marking : AppleMarking if onlyApples?(M:Marking) = true . endfm

To check whether a machine state has no cakes, we only have to check that the corresponding term of sort Marking belongs to the sort AppleMarking as follows reduce ($ $ c a) :: AppleMarking . rewrites: 5 in 0ms cpu (0ms real) (238095 rewrites/second) result Bool: false reduce ($ $ a a) :: AppleMarking . rewrites: 7 in 0ms cpu (0ms real) (3500000 rewrites/second) result Bool: true

In the following sections, this notion of membership is used to deﬁne the set of models that satisfy OCL constraints, which are encoded as boolean functions for a metamodel speciﬁcation (M , C ). Some dynamics are added to the system by means of two simple rules to buy an apple with a coin (buy-a) and to buy a cake with two coins (buy-c). Rewrite theories are deﬁned in so-called system modules, enclosed in between mod. . .endm. mod VENDING-MACHINE is including VENDING-MACHINE-SIGNATURE . var M : Marking . rl [buy-a] : $ => a . rl [buy-c] : $ $ => c . endm

Maude provides several veriﬁcation techniques, including model checking, for systems speciﬁed as rewrite theories. For instance, the search command allows one to explore (following a breadth-ﬁrst strategy) the reachable state space in a system. In the example below, system states are terms of sort Marking and we search all canonical terms that can be reached from the initial state $ $ where there are two coins. search $ $ =>! M:Marking .

The solutions that the search command provides are two states in which we obtained either a cake or two apples. Solution 1 (state 1) states: 3 rewrites: 2 in 0ms cpu (0ms real) (125000 rewrites/second) M --> c

102

A. Boronat and J. Meseguer

Solution 2 (state 3) states: 4 rewrites: 3 in 0ms cpu (0ms real) (32967 rewrites/second) M --> a a No more solutions.

We show how the search command is applied to model analysis with OCL invariants in Section 6.

3

Algebraic Semantics of MOF Metamodels

In this section, basic notions of the MOF standard and their formalization are presented. These notions are speciﬁed in the MOMENT2 framework and constitute the basis for the rest of the paper. MOF is a semiformal approach to deﬁne modeling languages by deﬁning their abstract syntax in a so-called metamodel M , which metarepresents a model type M . What this metamodel describes is, of course, a set of models. We call this the extensional semantics of M , and denote this semantics by M , which can be informally deﬁned as follows: M = {M | M : M }. In [4], we presented a formalization of the MOF framework in which the informal MOF semantics just described is made mathematically precise in terms of the initial algebra semantics of mel. Let MOF denote the set of all MOF metamodels M , and let SpecMEL denote the set of all mel speciﬁcations. Our algebraic semantics is then deﬁned as a function : MOF −→ SpecMEL that associates to each MOF metamodel M a corresponding mel speciﬁcation (M ), which constitutes the metamodel realization. Recall that any mel signature Σ has an associated set S of sorts. Therefore, in the initial algebra T(Σ,E) each sort s ∈ S has an associated set of elements T(Σ,E),s . The key point is that in any mel speciﬁcation of the form (M ), there is always a sort called Model , whose data elements in the initial algebra are precisely the data representations of those models that conform to M . That is, the sort Model syntactically represents the model type M associated to a metamodel M . The structural conformance relation between a model and its metamodel is then deﬁned mathematically by the equivalence M :M

⇔

M ∈T

(M ),Model .

Therefore, we can give a precise mathematical semantics to our informal MOF extensional semantics by means of the deﬁning equation M = T

(M ),Model .

Algebraic Semantics of OCL-Constrained Metamodel Speciﬁcations

103

Note that this algebraic semantics gives a precise mathematical meaning to the entities lacking such a precise meaning in the informal semantics, namely, the notions of: (i) model type M , (ii) metamodel realization (M ), and (iii) conformance relation M : M . For the metamodel M in Fig. 1.(a), the model depicted in Fig. 1.(c) can be deﬁned as a term of sort Model in the (M ) theory as follows: << < ’c1 : Component | type = "client", connectsTo = ’c2 ’c3 ’c4 > < ’c2 : Component | type = "client", connectsTo = ’c4 > < ’c3 : Component | type = "server", connectsTo > < ’c4 : Component | type = "server", connectsTo > >>,

where each tuple < Oid : ClassName | Properties > represents an object that is typed with a speciﬁc object type of the corresponding metamodel. Objects are deﬁned with properties of two kinds: attributes, typed with simple data types, and references, typed with object identiﬁer types. Each property is deﬁned by a pair (name = value). All the constructors that are used in the previous term are deﬁned in the signature of the (M ) theory. Note that a term of this kind represents an attributed typed graph with labelled unidirectional edges where the type graph is the metamodel. This representation of models (graphs) as algebraic terms is automatically generated by MOMENT2 from EMF models.

4

Algebraic Semantics of OCL Expressions

In this section, we introduce the algebraic, executable speciﬁcation of OCL available in MOMENT2, which is based on [8,3]. OCL permits deﬁning expressions that are useful to perform queries upon models M that conform to a given metamodel M . OCL expressions are parameterized with user-deﬁned types, such as classes, that are provided in a metamodel M . In MOMENT2, OCL expressions acquire an algebraic semantics that can be used for many purposes. One of them is the executable formalization of the constrained conformance relation between a model M and a metamodel speciﬁcation (M , C ) in the following section. Let OclExpression denote the set of well-formed OCL expressions. Not all OCL expressions in OclExpression make sense for a given metamodel. We say that a speciﬁc OCL expression e, such that e ∈ OclExpression, is meaningful for a metamodel M , denoted by (M , e), iﬀ all user-deﬁned types, if any, that are used in the expression e are metarepresented in M . An OCL expression is evaluated over a root object in a model M from which other objects in the model can be traversed. In the example in Fig. 1.c), we can traverse components through the connectsTo reference. In particular, those objects that are reachable through references can be traversed. In the OCL expression, the object type of the root object is indicated in the so-called context of the OCL expression. In this setting, the object type is called contextual type and the root object, to which an OCL expression is applied, is called contextual instance. In an OCL expression, the contextual instance can be explicitly referred to by means of the self keyword.

104

A. Boronat and J. Meseguer

In MOMENT2, we represent the concrete syntax of OCL as a membership equational theory OCLGrammar having a sort OclExpressionCS whose terms represent syntactically well-formed OCL expressions1 . Therefore, the following OCL expression is a valid term of sort OclExpressionCS: ’self . ’connectsTo -> forAll( ’c | ’c . ’type == # "server")

and it is a meaningful OCL expression for the metamodel M of the example if we take into account the object type Component as contextual type of the expression. 4.1

Algebraic Executable Semantics of Meaningful OCL Expressions

OCL expressions can be deﬁned by using generic OCL types, such as collections, and user-deﬁned types. The generic OCL types can be split in two groups: basic datatypes and collection types. OCL provides a set of operators that can be applied on values of these types. In particular, among collection operators we can distinguish between regular operators and loop operators. On the one hand, regular operators provide common functionality such as the size of a collection or the inclusion of elements within a collection. On the other hand, loop operators constitute second-order operators that receive a user-deﬁned OCL expression as argument, normally called loop body expression, and apply it to the elements of a collection depending on the nature of the operator. For example, the forAll operator receives a boolean loop body expression and checks that all the elements in the collection satisfy the predicate. The algebraic semantics of a meaningful OCL expression (M , e) is given by the partial function

: MOF × OclExpression SpecMEL, which is deﬁned for all meaningful OCL expressions (M , e) and maps (M , e) to a mel theory (M , e) such that (M ) ⊆ (M , e). In addition, the (M , e) theory provides an algebraic speciﬁcation for OCL basic datatypes and OCL collection types, where their operators are provided as equationally-deﬁned functions2 . The function can be viewed as an internal compiler from OCL to mel. The function traverses the term-based abstract syntax tree of the input OCL expression e by generating equationally-deﬁned functions for user-deﬁned OCL expressions, such as loop body expressions, in a top-down manner. The outermost OCL expression e is represented by an equationally-deﬁned function of the form exp : Object × Model −→ OclType , 1

2

There are a few minor syntactic diﬀerences with the concrete syntax of OCL: identiﬁers are preceded by a quote, literal values are wrapped by the # operator, and the equals operator symbol = is ==. In this paper we have abstracted from the diﬀerent types of collections that can be used in OCL. See [3] for a detailed explanation of the complete algebraic speciﬁcation of OCL.

Algebraic Semantics of OCL-Constrained Metamodel Speciﬁcations

105

where exp is a symbol that is generated for the corresponding OCL expression e, Object represents the set of objects that can be deﬁned with user-deﬁned types from a metamodel M , Model is the model type that corresponds to M , and OclType represents a valid OCL type depending on the type of e. In our running example, we obtain exp : Object Model -> Bool.

OCL expressions can then be executed over a contextual instance o in a model deﬁnition M by means of a term of the form exp(o, M ), whose return type depends on the user-deﬁned expression. For the expression ’self . ’connectsTo -> forAll( ’c | ’c . ’type == # "server"),

the exp function is deﬁned by the equation: exp(c , M) = c . ’connectsTo(M) -> forAll ( body ; empty-env ; M ),

where body is a constant that identiﬁes the function that represents the loop body expression, c . ’connectsTo(M) projects all the objects in the model M that can be traversed from the object c through the reference connectsTo, empty-env is a list of environment variables (used when there are variables that are bound in an outer OCL expression), and M is a variable that represents the model M . The following equation indicates how to apply the boolean body expression of the forAll operator to each member Obj of a collection of objects recursively (where Col is a collection of objects, B is a body, E is an environment of variables, and M is a model): Obj Col -> forAll(B ; E ; M) = Obj . B( E ; M ) and Col -> forAll(B ; E ; M) .

and the expression Obj . B( E ; M ) is again deﬁned by an equation that evaluates the expression ’type == # "server" for an object Obj Obj . body( E ; M ) = Obj . get("type") == "server"

If we denote the model in Fig. 1.(c) by model2 and the objects with identiﬁers c1 and c2 by o1 and o2, repectively, we obtain the following results with the above OCL expression: exp(o1, model2) = false and exp(o2, model2) = true. Therefore, the function provides an algebraic executable semantics of meaningful OCL expressions (M , e) by means of an executable mel theory where all OCL datatypes are algebraically deﬁned, and where user-deﬁned OCL expressions are available as equationally deﬁned functions. The presented OCL formalization can be used for evaluating OCL queries within a MOF-like modeling environment, such as the Eclipse Modeling Framework through MOMENT2. In addition, and even more importantly, this algebraic semantics for OCL expressions can be used at a theoretical level to provide a formal semantics to concepts that are not yet suﬃciently precise in the MOF standard, such as the constrained conformance relation, and, at a more practical level, to enhance static and dynamic model analysis with OCL invariants.

106

5

A. Boronat and J. Meseguer

Algebraic Executable Semantics of Metamodel Specifications

OCL permits imposing constraints upon speciﬁc object types in a metamodel M , thus constraining the set of models M that conform to M . We call the resulting conformance relation the constrained conformance relation. In this section, we deﬁne the concept of metamodel specification (M , C ), which is used to attach a set C of meaningful OCL constraints to a metamodel M . Relying on the aforementioned algebraic semantics of OCL expressions, the algebraic semantics of a metamodel speciﬁcation (M , C ) is also given, deﬁning how a model type can be semantically enriched with OCL constraints. 5.1

Metamodel Specifications

An OCL invariant c is a constraint that is deﬁned using a boolean body expression that evaluates to true if the invariant is satisﬁed. The body expression of an invariant c, denoted body(c), is a well-formed boolean OCL expression, i.e., body(c) ∈ OclExpression. An invariant is also deﬁned with a contextual type by means of the clause context as follows: context ’Component inv : <meaningful OCL expression>

where ’Component is the name of an object type, in this case deﬁned in the metamodel M of the example. An OCL invariant c must hold true for any instance of the contextual type at any moment in time. Only when an instance is executing an operation, is c allowed not to evaluate to true. An invariant c is meaningful for a metamodel M iﬀ (M , body(c)) is a meaningful boolean OCL expression. A set of OCL invariants that are meaningful for a metamodel M may be evaluated over a speciﬁc model M : M . More precisely, each OCL invariant c ∈ C is evaluated for each contextual instance o ∈ M . We say that a model M satisfies a set C of OCL invariants that are meaningful for a metamodel M iﬀ all such invariants evaluate to true for every contextual instance of the model M . We write M C to denote this OCL constraint satisfaction relation. A metamodel speciﬁcation (M , C ) consists of a metamodel M , such that M : MOF, and a set C of OCL invariants that are meaningful for M . A metamodel speciﬁcation (M , C ) deﬁnes a model type whose values are models M that both conform to the metamodel M and satisfy the set C of meaningful OCL invariants. We deﬁne the extensional semantics of a metamodel speciﬁcation (M , C ) by the equality: (M , C ) = {M | M : M ∧ M C }. 5.2

Algebraic Executable Semantics of Metamodel Specifications

Let SpecMOF denote the set of well-deﬁned metamodel speciﬁcations (M , C ). To realize a metamodel speciﬁcation (M , C ), we deﬁne a function : SpecMOF −→ SpecMEL

Algebraic Semantics of OCL-Constrained Metamodel Speciﬁcations

107

that maps a metamodel speciﬁcation (M , C ) to an executable mel theory (M , C ). The resulting (M , C ) theory, called metamodel specification realization, also formalizes the body expressions of all constraints C involved in the metamodel speciﬁcation, i.e., we have the mel theory inclusion (M , body (c)) ⊆ (M , C ). c∈C

Given a metamodel speciﬁcation (M , C ), the model type M is deﬁned in the (M ) theory. The model type M is preserved in the (M , C ) theory by means of the subtheory inclusion (M ) ⊆ (M , C ), where the constrained model type (M , C ) is deﬁned as a subset of the model type M , i.e., (M , C ) ⊆ M . (M , C ) constitutes the constrained model type that is syntactically represented by the sort CModel in the theory (M , C ). This model type inclusion is syntactically deﬁned by the subsort relation CModel < Model . The function deﬁnes the constrained model type (M , C ), in the (M , C ) theory, by means of a membership axiom of the form consistent (M ) : CModel if M : Model ∧ condition 1 (M ) = true ∧ · · · ∧ condition n (M ) = true,

(†)

where the partial operator consistent : Model [CModel ] is deﬁned by the membership and each constraint deﬁnition ci , in C , corresponds to a boolean function condition i that is generated by means of the function and that is evaluated over a model M as explained in Section 4.1. When C = ∅, M = (M , ∅) and therefore (M ) = (M , ∅). The above membership axiom forces the evaluation of OCL constraints C over a model M , so that M : (M , C ) iﬀ M C . This means that a model deﬁnition M , such that M : M , satisﬁes all the constraints that are deﬁned in C , iﬀ M is a value of the constrained model type (M , C ). Each condition i function evaluates the boolean body expression of an invariant for all the corresponding contextual instances. In the invariant of our running example we obtain the function: op condition : Model -> Bool . eq condition(M) = M . allInstances( Component ) -> forAll(c-body ; empty-env ; M) . eq Obj . c-body( E ; M ) = exp(Obj, M) .

where the expression M . allInstances( Component ) obtains all instances of the Component class in the model M and the operation -> forAll(c-body ; empty-env ; M) evaluates the expression exp(Obj, M) deﬁned in the section above, corresponding to the expression ’self . ’connectsTo -> forAll( ’c | ’c . ’ type == # "server"). The resulting membership that is automatically compiled for the OCL constraint of the example is expressed, in Maude notation, as follows cmb consistent(M) : CModel if M : Model /\ condition(M).

108

A. Boronat and J. Meseguer

Using the mel theory (M , C ), the semantics of the constrained model type (M , C ) is deﬁned in terms of the initial algebra semantics of (M , C ) as follows (M , C ) = T (M ,C ),CModel the OCL constraint satisfaction is deﬁned by the equivalence M C ⇔ M ∈ (M , C ) and the constrained conformance relation M : (M , C ) is deﬁned by the equivalence M : (M , C ) ⇔ M ∈ (M , C ). The (M , C ) theory constitutes a formal realization as a theory in mel of the metamodel speciﬁcation (M , C ). In addition, (M , C ) is executable, providing a decision procedure for the OCL constraint satisfaction relation. Furthermore, by being a mel theory with initial algebra semantics, it gives an algebraic semantics for the types that are deﬁned as data in (M , C ). 5.3

MOMENT2-OCL

The presented executable algebraic semantics for OCL is available in MOMENT2 [5], a formal speciﬁcation and veriﬁcation framework for MOF-based software artifacts, speciﬁed in Maude and plugged into the EMF. MOMENT2 provides a collection of tools, among which MOMENT2-OCL provides a front-end for deﬁning metamodel speciﬁcations (M , C ), where M is an EMF metamodel and C is a set of textual OCL invariants. MOMENT2-OCL deﬁnes the semantics of a metamodel speciﬁcation (M , C ) by means of the function , and oﬀers a mechanism to automatically check the constrained conformance relation between a model M and a constrained model type (M , C ), i.e., M : (M , C ) by evaluating the membership axiom †.

6

Dynamic Analysis with OCL Invariants

In this section, we show how the aforementioned algebraic semantics for OCL expressions and metamodel speciﬁcations (M , C ) can be used for formal dynamic analysis. In our running example, software architectures are conﬁgurations of components that may be connected to each other through the connectsTo reference. Given a speciﬁc initial conﬁguration M of components, we can use model checking in order to verify whether or not all possible reconﬁgurations of an initial conﬁguration are style-preserving w.r.t. a metamodel speciﬁcation (M , C ), where the set of OCL constraints C deﬁnes the corresponding architectural style to be preserved.

Algebraic Semantics of OCL-Constrained Metamodel Speciﬁcations

109

We add a dynamic connection load balancing strategy, so that a server component should not have more than two connections at a time, i.e., when a component has more than two connections, the spare connections are forwarded to other components with less than two incoming connections. We depict the reconﬁguration as a graph transformation rule in Fig. 2, where a rule is deﬁned with a left-hand side (LHS) pattern and a right-hand side (RHS) pattern. Each pattern is comprised of nodes that represent Component objects in a model (graph) M and edges representing connectsTo references between them. A reconﬁguration can be applied whenever the LHS pattern of the rule can be matched against a speciﬁc conﬁguration M of components, and then the edges are manipulated as follows: an edge in the LHS and not in the RHS is removed from the conﬁguration, an edge not in the LHS but in the RHS is added, all other edges remain unmodiﬁed. Marked edges indicate that the references must not exist in order to apply the rule; this is known as a negative application condition in the graph transformation community.

Fig. 2. Reconﬁguration rule

The graph-theoretic nature of models is axiomatized in our algebraic semantics as a set of objects modulo the associativity, commutativity, and identity axioms of set union. The semantics of a reconﬁguration can then be naturally expressed as a rewrite theory [12] extending the algebraic semantics (M , C ) of our metamodel speciﬁcation with rewrite rules that are applied modulo the equational axioms. In this way, the above graph-transformation rule can be summarized at a high level as follows: op free-reconfiguration : Model -> Model . crl free-reconfiguration(M) => free-reconfiguration(<< < O1 : Component | PS1 > < O2 : Component | connectsTo = O1 S2, PS2 > < O3 : Component | connectsTo = O1 S3, PS3 > < O4 : Component | connectsTo = O6 S4, PS4 > < O6 : Component | PS6 > ObjCol >>) if << < O1 : Component | PS1 > < O2 : Component | connectsTo = O1 S2, PS2 > < O3 : Component | connectsTo = O1 S3, PS3 > < O4 : Component | connectsTo = O1 S4, PS4 > < O6 : Component | PS6 > ObjCol >> := M /\ nac(O6, M) .

where the terms with variables that constitute the LHS and RHS of the equation represent the graph patterns deﬁned in Fig. 2, the matching equation P := M

110

A. Boronat and J. Meseguer

allows us to use the variable M as a model pattern P, and the condition corresponds to the negative application condition that enables the application of the rule: op nac : Oid Model -> Bool . eq [counterexampleNAC] : nac(O1, << < O2 : Component | connectsTo = O1 S2, PS2 > < O3 : Component | connectsTo = O1 S3, PS3 > ObjCol >>) = false . eq [satisfiedNAC] : nac(O1, M) = true [owise] .

The metamodel speciﬁcation realization (M , C ), corresponding to Fig. 1.(a) and to the singleton set C of OCL constraints, and the rewriting rule just presented above deﬁne a state transition system, where states represent conﬁgurations M and M of components and transitions M −→ M represent an application of the reconﬁguration rule. However, a reconﬁguration of this kind could conceivably produce conﬁgurations M of components that are not client/server style-preserving, i.e., such that M : M but M C . Therefore, it is important to formally verify whether or not a given reconﬁguration, like the one above, is style-preserving.

Fig. 3. Initial software architecture conﬁguration (a) and conﬁguration that does not preserve the client/server architectural style (b).

In Maude, the search command allows one to exhaustively explore (following a breadth-ﬁrst strategy) the reachable state space deﬁned by a state transition system as the one above, checking whether an invariant is violated. We can use the search command to ﬁnd out if the reconﬁguration free-reconfiguration produces such an illegal conﬁguration as follows: search [1] free-reconfiguration(model) =>+ free-reconfiguration(M:Model) such that not(consistent(M:Model) :: CModel) .

where model is a constant that represents the model M in Fig. 3.a. This command ﬁnds the counterexample, shown in Fig. 3.b, where a client component is connected to another client component. An alternative reconﬁguration rule can be deﬁned to avoid this problem as shown below, by indicating that the node 6 in the graph patterns of the rule in Fig. 2 is of type server. No counterexamples are found when running again the search command with the new reconﬁguration rule.

Algebraic Semantics of OCL-Constrained Metamodel Speciﬁcations

111

op safe-reconfiguration : Model -> Model . crl safe-reconfiguration(M) => safe-reconfiguration(<< < O1 : Component | PS1 > < O2 : Component | connectsTo = O1 S2, PS2 > < O3 : Component | connectsTo = O1 S3, PS3 > < O4 : Component | connectsTo = O6 S4, PS4 > < O6 : Component | type = "server", PS6 > ObjCol >>) if << < O1 : Component | PS1 > < O2 : Component | connectsTo = O1 S2, PS2 > < O3 : Component | connectsTo = O1 S3, PS3 > < O4 : Component | connectsTo = O1 S4, PS4 > < O6 : Component | type = "server", PS6 > ObjCol >> := M /\ nac(O6, M) .

7

Related Work

The formal semantics of OCL was introduced in [13] and was included in the standard speciﬁcation [7]. The MOF standard speciﬁcation [2] provides the semantics of the MOF meta-metamodel and indicates how OCL constraints can be attached to a MOF metamodel from a syntactical point of view. Clark, Evans and Kent formalized the use of UML and OCL with the MML language in [14]. In their approach the concrete syntax of MML is mapped to the MML Calculus, which provides an operational semantics for both UML modeling constructs and OCL operations. In MOMENT2, we also follow a translational approach to provide semantics to MOF (as presented in [15]) and to OCL. MOF (through EMF) and OCL constitute our concrete syntax and the mapping provides the translation of OCL expressions, deﬁned in a metamodel speciﬁcation, into a mel theory. Due to the fact that we focus on MOF, we have not considered class methods at this stage, so that OCL pre- and post-conditions are not currently supported. Our goal in MOMENT2 consists of leveraging the use of rewriting logic and Maude-based formal veriﬁcation techniques in model-driven development, in particular in the use of OCL in this paper. Therefore, we discuss below other approaches that provide support for formal analysis based on OCL. On the one hand, several tools provide support for static analysis with OCL constraints: USE [16] and MOVA [17] with validation of OCL constraints, and HOL-OCL [18], UML2Alloy [19] and UMLToCSP [20] for veriﬁcation of UML/OCL models, among others. In particular, HOL/OCL provides an elegant approach to encode OCL iterator operators as higher order constructs. However, we have chosen mel for our OCL formalization because it is a sublogic of rewriting logic, hence equationally-deﬁned OCL expressions can be used to deﬁne properties that can be veriﬁed in rewrite systems by means of Maude’s facilities for reachability analysis and LTL model checking. MOVA also uses mel as underlying formalism and focuses on the speciﬁcation of UML/OCL models, which are represented as mel theories. In MOMENT2, we focus on metamodel speciﬁcations (M , C ), and a model M is deﬁned as a term modulo associativity commutativity and identity, i.e., a graph [4], and not as a mel theory as in MOVA.

112

A. Boronat and J. Meseguer

On the other hand, dynamic analysis with OCL is usually supported by mapping UML/OCL models into a given formalism for model checking, such as the Object-Based Temporal Logic (BOTL), a logic based on branching temporal logic CTL and OCL, in [21]. The tool SOCLe [22] provides an extension of OCL (EOCL) with CTL temporal operators and ﬁrst-order features, inspired in BOTL, that allows model checking EOCL predicates on UML models expressed as abstract state machines. In our case, we automatically map metamodel speciﬁcations (M , C ) to mel theories enabling model checking of OCL invariants in rewriting logic as shown above. Although without using OCL, in [23] the authors present how to directly use Maude to provide the structural and dynamic semantics of DSMLs, where metamodels are encoded as rewrite theories and models as collections of objects. Therefore, Maude-based formal veriﬁcation techniques can be applied as described in [9]. Despite the similar use of veriﬁcation techniques, there are several diﬀerences between both approaches. MOMENT2 uses OMG standards, such as MOF and OCL, as interface between industrial environments, such as EMF, and the formalism based on Maude so that the use of the formalism remains hidden to the user. In addition, we have identiﬁed and formalized the notions of model type and conformance relation where OCL constraints can be taken into account. A complete description of these concepts and their formalization is provided in [3]. In the graph transformation ﬁeld, several analysis techniques have been developed [24], but they usually work with simple type graphs, less expressive than metamodel speciﬁcations. [25] shows how graph transformations can be mapped to OCL pre- and post-conditions, so that the aforementioned tools for OCLbased formal veriﬁcation can be applied. In addition, the authors considered a number of analysis techniques with OCL properties, such as correctness preservation when a transformation rule is applied, among others. Our approach and strategy are just the opposite: we have mapped metamodel speciﬁcations (M , C ) into mel theories so that the function is used to include OCL expressions in model transformations in the tool MOMENT2-MT [5,26]. As for the running example, Architectural Design Rewriting (ADR) [27] is an approach for hierarchical style-based reconﬁgurations of software architectures that is based on rewriting logic. ADR allows deﬁning style-preserving reconﬁgurations while in our approach style preservation should be explicitly veriﬁed. However, our approach is not speciﬁc to the service-oriented computing domain and relies on OMG standards for formalizing DSMLs.

8

Conclusions and Future Work

We have presented several contributions towards the main goal of increasing the formal analysis power of model-based software engineering. Our ﬁrst contribution has focused on the fact that metamodel speciﬁcations frequently include both the metamodel syntax itself and additional semantic constraints, thus making the issue of checking conformance of a model to a metamodel speciﬁcation a semantic one. To automate the cheking of what we have called constrained

Algebraic Semantics of OCL-Constrained Metamodel Speciﬁcations

113

model conformance, we have given an algebraic, executable semantics of metamodel speciﬁcations, embodied in the function . This provides a static analysis feature for models in a modeling language with semantic constraints, and we have illustrated its use with a client-server architectural style example. Our second contribution has been to show how the same function can also be used for dynamic analysis when models are transformed, so that one can check that the models obtained by transforming a given model using reconﬁguration rules satisfy some correctness criteria. We have illustrated this by showing through model checking how a given reconﬁguration rule is incorrect, producing models that fail to satisfy the client-server metamodel speciﬁcation and giving a better rule not making such violations. More generally, the semantics and infrastructure developed here can be used in conjuntion with Maude’s LTL model checker to verify any dynamic properties expressed as LTL formulas whose atomic predicates are deﬁned by OCL invariants. Our third and last contribution has been to incorporate support for automatic veriﬁcation of OCL invariants and of the constrained conformance relation in the latest version MOMENT2 tool [5]. The presented semantics for OCL expressions is integrated in a QVT-like model transformation language, MOMENT2-MT (also available in the MOMENT2 framework), so that OCL expressions can be used to perform queries and to manipulate data. In MOMENT2-MT, a model transformation is deﬁned as a collection of graph production rules, which are compiled to equations and rewrites [26]. Therefore, Maude-based model checking facilities can be used for model checking model transformations with OCL predicates. In future work, we plan to apply MOMENT2 and Maude-based formal veriﬁcation techniques to perform formal analysis of real-time embedded systems in the avionics speciﬁc domain, by using model-based languages like the Architecture Analysis and Design Language (AADL) [28]. Acknowledgments. This work has been partially supported by the ONR Grant N00014-02-1-0715, by the NSF Grant IIS-07-20482, by the NAOMI project funded by Lockheed Martin, by the EU project SENSORIA IST-2005-016004 and by the Spanish project META TIN2006-15175-C05-01.

References 1. van Deursen, A.v., Klint, P., Visser, J.: Domain-speciﬁc languages: an annotated bibliography. SIGPLAN Not. 35(6), 26–36 (2000) 2. OMG: Meta Object Facility (MOF) 2.0 Core Speciﬁcation (ptc/06-01-01) (2006) 3. Boronat, A.: MOMENT: a formal framework for MOdel manageMENT. PhD in Computer Science, Universitat Polit`enica de Val`encia (UPV), Spain (2007), http://www.cs.le.ac.uk/~ aboronat/papers/2007_thesis_ArturBoronat.pdf 4. Boronat, A., Meseguer, J.: An Algebraic Semantics for MOF. In: Fiadeiro, J.L., Inverardi, P. (eds.) FASE 2008. LNCS, vol. 4961, pp. 377–391. Springer, Heidelberg (2008) 5. MOMENT2 (2008), http://www.cs.le.ac.uk/~ aboronat/tools/moment2

114

A. Boronat and J. Meseguer

6. Meseguer, J.: Membership algebra as a logical framework for equational speciﬁcation. In: Parisi-Presicce, F. (ed.) WADT 1997. LNCS, vol. 1376, pp. 18–61. Springer, Heidelberg (1998) 7. Object Management Group: OCL 2.0 Speciﬁcation (2006), http://www.omg.org/cgi-bin/doc?formal/2006-05-01 ´ An Algebraic Speciﬁca8. Boronat, A., Oriente, J., G´ omez, A., Ramos, I., Cars´ı, J.A.: tion of Generic OCL Queries Within the Eclipse Modeling Framework. In: Rensink, A., Warmer, J. (eds.) ECMDA-FA 2006. LNCS, vol. 4066, pp. 316–330. Springer, Heidelberg (2006) 9. Clavel, M., Dur´ an, F., Eker, S., Meseguer, J., Lincoln, P., Mart´ı-Oliet, N., Talcott, C.: All About Maude. In: Clavel, M., Dur´ an, F., Eker, S., Lincoln, P., Mart´ı-Oliet, N., Meseguer, J., Talcott, C. (eds.) All About Maude - A High-Performance Logical Framework. LNCS, vol. 4350, Springer, Heidelberg (2007) 10. Eclipse Organization: The Eclipse Modeling Framework (2007), http://www.eclipse.org/emf/ 11. Shaw, M., Garlan, D.: Software Architecture: Perspectives on an Emerging Discipline. Prentice-Hall, Englewood Cliﬀs (1996) 12. Meseguer, J.: Conditional rewriting logic as a uniﬁed model of concurrency. Theoretical Computer Science 96(1), 73–155 (1992) 13. Richters, M.: A Precise Approach to Validating UML Models and OCL Constraints. PhD thesis, Universit¨ at Bremen, Logos Verlag, Berlin, BISS Monographs, No. 14 (2002) 14. Clark, T., Evans, A., Kent, S.: The Meta-modeling Language Calculus: Foundation Semantics for UML. In: Hussmann, H. (ed.) FASE 2001. LNCS, vol. 2029, pp. 17–31. Springer, Heidelberg (2001) ´ Ramos, I.: Algebraic Speciﬁcation of a Model Trans15. Boronat, A., Cars´ı, J.A., formation Engine. In: Baresi, L., Heckel, R. (eds.) FASE 2006. LNCS, vol. 3922, pp. 262–277. Springer, Heidelberg (2006) 16. Gogolla, M., B¨ uttner, F., Richters, M.: USE: A UML-based speciﬁcation environment for validating UML and OCL. Sci. Comput. Program 69(1-3), 27–34 (2007) 17. Egea, M.: An Executable Formal Semantics for OCL with Applications to Model Analysis and Validation. PhD in Computer Science (to appear), Universidad Complutense de Madrid, Spain (2008) 18. Brucker, A.D., Wolﬀ, B.: The HOL-OCL book. Technical Report 525, ETH Z¨ urich (2006) 19. Anastasakis, K., Bordbar, B., Georg, G., Ray, I.: UML2Alloy: A Challenging Model Transformation. In: Engels, G., Opdyke, B., Schmidt, D.C., Weil, F. (eds.) MODELS 2007. LNCS, vol. 4735, pp. 436–450. Springer, Heidelberg (2007) 20. Cabot, J., Claris´ o, R., Riera, D.: UMLtoCSP: a tool for the formal veriﬁcation of UML/OCL models using constraint programming. In: ASE 2007, pp. 547–548. ACM, New York (2007) 21. Distefano, D., Katoen, J.P., Rensink, A.: On a temporal logic for object-based systems. In: Smith, S.F., Talcott, C.L. (eds.) Formal Methods for Open Objectbased Distributed Systems, pp. 305–326. Kluwer Academic Publishers, Dordrecht (2000); Report version: TR–CTIT–00–06, Faculty of Informatics, University of Twente 22. Mullins, J., Oarga, R.: Model Checking of Extended OCL Constraints on UML Models in SOCLe. In: Bonsangue, M.M., Johnsen, E.B. (eds.) FMOODS 2007. LNCS, vol. 4468, pp. 59–75. Springer, Heidelberg (2007)

Algebraic Semantics of OCL-Constrained Metamodel Speciﬁcations

115

23. Rivera, J.E., Vallecillo, A.: Adding behavioral semantics to models. In: Proceedings. 11th IEEE International Enterprise Distibuted Object Computing Conference. EDOC 2007, Annapolis, Maryland, USA, October 15-19, 2007, pp. 169–180. IEEE Computer Society, Los Alamitos (2007) 24. Ehrig, H., Ehrig, K., Prange, U., Taentzer, G.: Fundamentals of Algebraic Graph Transformation. Springer, Heidelberg (2006) 25. Cabot, J., Claris´ o, R., Guerra, E., de Lara, J.: Analysing graph transformation rules through OCL. In: Vallecillo, A., Gray, J., Pierantonio, A. (eds.) ICMT 2008. LNCS, vol. 5063, pp. 229–244. Springer, Heidelberg (2008) 26. Boronat, A., Heckel, R., Meseguer, J.: Rewriting Logic Semantics and Veriﬁcation of Model Transformations. In: Chechik, M., Wirsing, M. (eds.) FASE 2009. LNCS, vol. 5503, pp. 18–33. Springer, Heidelberg (2009) 27. Bruni, R., Lluch-Lafuente, A., Montanari, U., Tuosto, E.: Style-Based Architectural Reconﬁgurations. Bulletin of the EATCS (94) (February 2008) 28. SAE: AADL (2007), http://www.aadl.info/

Specifying and Composing Concerns Expressed in Domain-Specific Modeling Languages Aram Hovsepyan, Stefan Van Baelen, Yolande Berbers, and Wouter Joosen Katholieke Universiteit Leuven, Departement Computerwetenschappen, Celestijnenlaan 200A, B-3001 Leuven, Belgium {Aram.Hovsepyan,Stefan.VanBaelen,Yolande.Berbers, Wouter.Joosen}@cs.kuleuven.be

Abstract. Separation of concerns and levels of abstraction are key software engineering principles that can help master the increasing complexity of software applications. Aspect-oriented modeling (AOM) and domain-speciﬁc modeling languages (DSML) are two important and promising approaches in this context. However, little research is done to investigate the synergy between AOM and DSMLs. In this paper we present an asymmetric approach to compose modularized concerns expressed in diﬀerent DSMLs with an application base model expressed in a general-purpose modeling language (GPML). This allows to specify each concern in the most appropriate modeling language. We introduce the concept of a concern interface, expressed in a GPML, that serves as a common language between a speciﬁc concern and the application base. In addition, we use an explicit composition model to specify the syntactic and the semantic links between entities from the diﬀerent concerns. We explore these concepts using an application where we modularize the user interface modeled in WebML and the access control speciﬁed in XACML. The modularized concerns are then composed with an application base that has been speciﬁed in UML.

1

Introduction

The increasing complexity of software applications requires improved development techniques. The introduction of aspect-oriented modeling (AOM) and the evolution of domain-speciﬁc modeling languages (DSML) are very promising in this context. AOM [1] is a recent development paradigm that aims at providing support for separation of concerns at higher levels of abstraction by using ModelDriven Engineering (MDE) techniques [2]. The main goal of AOM approaches is to enable both vertical and horizontal separation of concerns [3].

The described work is part of the EUREKA-ITEA EVOLVE project, and is partially funded by the Flemish government institution IWT (Institute for the Promotion of Innovation by Science and Technology in Flanders), by the Interuniversity Attraction Poles Programme Belgian State, Belgian Science Policy, and by the Research Fund K.U.Leuven.

M. Oriol and B. Meyer (Eds.): TOOLS EUROPE 2009, LNBIP 33, pp. 116–135, 2009. c Springer-Verlag Berlin Heidelberg 2009

Specifying and Composing Concerns Expressed in DSML

117

In order to specify each concern, we have the possibility of using either a general purpose modeling language (GPML), such as the UML, or a DSML. Most of the current AOM approaches [4,5,6,7,8,9] use UML along with its extension mechanisms for expressing all concerns. However, DSMLs can potentially improve the current AOM methods for specifying concern models since the developer can use an optimal DSML for each of the concerns involved. In this paper we analyze the problem of concern compositions when these are expressed in diﬀerent DSMLs. We present a pragmatic AOM approach based on existing MDE building blocks and on the concept of a concern interface that we introduce. This approach allows the composition of a base concern expressed in a GPML with concerns speciﬁed in DSMLs, which do not necessarily conform to the same metametamodel. We explore these ideas on a case study that modularizes two concern types: an essential part of security, i.e., access control expressed in XACML [10], and a web-based user interface expressed in WebML [11]. We further provide support for code generation towards an AO platform. The paper is structured as follows. In section 2 we sketch the problem statement in detail. In section 3 we describe our asymmetric framework for modularizing and composing reusable concerns expressed in DSMLs. In section 4 we demonstrate a sample use of this framework for two diﬀerent concern types and illustrate the concepts on a case study. In section 5 we evaluate our approach and discuss the possible alternative solutions. We present the related work in section 6. Finally, we conclude and give insights on our future work.

2

Problem Statement

Concerns are an important motivation for organizing and decomposing software into manageable and comprehensible parts [12]. We use the term concern to uniformly refer to what AOSD practitioners often call an aspect concern and a base concern [13]. The base concern typically represents the functional backbone of a given application, whereas diﬀerent aspect concerns represent functional and non-functional modules that augment the core. In addition, we deﬁne the term concern type to refer to a certain concern domain, such as access control, user interface, transactions, etc. 2.1

Introduction

Modularity and abstraction are fundamental principles of any Aspect Oriented Software Development approach [14]. Both concepts are central in the current state-of-the-art AOM approaches [4,5,6,7,8,9]. These approaches provide techniques to decompose a system by separating concerns into modules. In addition, they provide adequate means to model each concern on a high abstraction level and then reﬁne it all the way to the implementation. However, most of the current AOM approaches are based on the use of a GPML, i.e., UML with its extension mechanisms [15]. Even though UML is a well-known modeling language and can be used to specify almost any software system, it is not optimal and may lead to cumbersome and hardly usable models

118

A. Hovsepyan et al.

[16]. DSMLs are said to greatly improve the comprehensibility for each concern domain by providing notations and constructs at the level of abstraction of the problem domain [16,17]. DSMLs oﬀer substantial gains in expressiveness and ease of use compared to GPMLs for the domain in question. In addition, DSMLs narrow the gap between the problem and implementation domain [2]. Using a GPML, one needs to describe the solution in domain terms, then map it to the GPML in question and ﬁnally transform it to code. DSMLs support the description of the solution directly in domain terms. 2.2

Concern Composition

Modularity must be complemented by composability. The various concerns need to relate to each other in a systematic and coherent fashion. One may further explore these relationships by performing the composition at various levels in the development cycle. Figure 1 presents the actual problem statement where four diﬀerent concerns are modeled using four diﬀerent DSMLs from three diﬀerent technical spaces. The notion of a technical (or technological) space is deﬁned as a model management framework along with a set of tools that operate on the models that can be deﬁned within the framework [18]. Ideally, all DSMLs should belong to the same technical space, however as diﬀerent technical spaces oﬀer complementary features in reality this is not the case. Concern1 and Concern2 conform to DSML1 and DSML2 deﬁned in the Ecore technical space, Concern3 conforms to DSML3 deﬁned by the XSD (XML) technology and Concern4 conforms to a DSML4 deﬁned by yet another Metametamodel. Bezivin et al. [19] have outlined the importance of bridging the technical spaces, rather then recreating artefacts throughout the diﬀerent spaces.

Ecore

XSD

Metametamodel

DSML1

DSML2

DSML3

DSML4

Concern1

Concern2

Concern3

Concern4

conforms to

Fig. 1. Problem overview

Given the various concerns of the system expressed in diﬀerent modeling languages across diﬀerent technical spaces, we would like to specify and perform their composition in order to obtain a combined system. Note that throughout this paper we assume that suitable DSMLs for specifying modularized concerns are readily available. In the next section, we present our approach for combining modularized concerns expressed in diﬀerent modeling languages.

Specifying and Composing Concerns Expressed in DSML

3

119

Concern Composition Framework

Once the modularized concerns are modeled, as illustrated in ﬁgure 1, one should be able to specify how the elements between diﬀerent concerns relate to each other and to establish links between them. It is practically unmanageable to create one monolithic composition model to express all interrelationships between all concerns. This is why we will use pair-wise compositions between concern models. Only compositions between concerns related by a dependency relationship [20] should be speciﬁed. Dependency is a situation where one concern explicitly needs another concern and thus depends on it. 3.1

Asymmetric Approach Using a GPML

We categorize the pair-wise concern compositions into two categories depending on the technical spaces each concern belongs to. Compositions between concerns that lie in the Ecore technical space, i.e. intraspace compositions, are speciﬁed using AMW [21], which is an existing stateof-the-art framework for specifying model compositions. Each composition model conforms to a customized composition metamodel that extends the base abstract weaving metamodel provided by AMW. For instance, consider two concerns expressed in diﬀerent DSMLs that conform to the Ecore metametamodel (ﬁgure 2). Ecore

Base WMM

DSML1

Concern1

CompositionMM

DSML2 conforms to Concern2 refers to

Composition Model

extends

Fig. 2. Intraspace compositions with AMW

In order to specify their composition, one should extend the base abstract weaving metamodel and create a composition metamodel (CompositionMM ) that is speciﬁc for DSML1 and DSML2. One can then specify the composition between any two concerns expressed in DSML1 and DSML2 respectively. For concerns that belong to diﬀerent technical spaces, we will use the socalled interspace compositions. In order to specify an interspace composition we introduce a new concept called concern interface. A concern interface serves as a “lingua franca” (common language) between concerns expressed in diﬀerent modeling languages. The use of a concern interface is illustrated in ﬁgure 3. In order to specify the composition between Concern1 and Concern2, we create a ConcernInterface1 that represents the information required by Concern1 expressed in DSML2. Concern interfaces are bridges between modeling languages

120

A. Hovsepyan et al.

MMM1

MMM2

DSML1

Concern1

CompositionMM

DSML2

Concern2

ConcernInterface1 Composition Model

conforms to refers to

Fig. 3. Concern interface

from diﬀerent technical spaces. In theory, concerns can be treated equally, i.e., we could also create a ConcernInterface2 expressed in DSML1 instead of ConcernInterface1. As ConcernInterface1 and Concern2 are expressed in the same modeling language, we can use the intraspace composition speciﬁcation methods. Unfortunately, this approach may not always work in practice. Even though DSMLs in general oﬀer powerful abstraction mechanisms, they are limited to a speciﬁc domain. Hence, it could be impossible to specify ConcernInterface1 in DSML2, because the latter does not oﬀer the necessary concepts and constructs. The same holds for the inverse, where ConcernInterface2 for Concern2 should be speciﬁed in DSML1. In order to solve this issue we introduce two constraints to the previous approach. First of all, we remove the symmetry from it. Even though, in theory, all concerns are equally important and should play symmetrical role [8,22], an asymmetric approach is more practical. In an asymmetric approach one of the concerns, called a base concern, represents the functional backbone of the system. All other concerns are composed with this base concern. The second constraint is the usage a GPML for specifying the base concern. These restrictions ensure that it is mostly possible to create a concern interface. Finally, as the Ecore is one of the most advanced technical spaces in terms of tools we will use it as the base technical space. Figure 4 shows the overall approach and the diﬀerent composition types. Concern1 is speciﬁed in DSML1 that is native to Ecore, so it is composed with Base using an intraspace composition. Concern2 and Base lie in diﬀerent technical spaces and therefore we specify their composition using the interspace composition approach. We create a concern interface (ConcernInterface2 ) speciﬁed in UML and specify its composition with Base using the intraspace composition technique. 3.2

Composition Application

Once the composition pairs are speciﬁed, it is possible to perform each composition. There are two alternative solutions depending on the desired composition output. One possibility is to perform a model composition transformation and obtain a composed model. The composed model can then be further used to

Specifying and Composing Concerns Expressed in DSML

Ecore

121

MMM

DSML1

UMLEMF

Concern1

Base

DSML2

ConcernInterface2

Concern2 conforms to

Composition Model

Composition Model

refers to

Fig. 4. The uniﬁed asymmetric solution

generate code for traditional OO platforms. The second possibility is translating each concern immediately into source code. In this case the target platform should take care of the composition itself and will typically be an AO platform. Note that in both solutions the composition application output will lose the domain speciﬁcity and be conform to a general purpose modeling or an AO programming language. In this paper we use the latter alternative and transform each concern into AO source code. By targeting an AO platform, we can actually postpone the composition step towards the AOP level. The base concern model is transformed into a base code-level artefact. Each aspect concern model is translated into a code-level advice. Finally, the composition models are transformed into codelevel pointcuts that specify how exactly the aspect concerns should augment the base code. The actual composition application is delegated to the AO code weaver that can completely automatically perform the composition of concerns and create a byte-level source code of the system. In the next section we show an instantiation of our asymmetric approach for two diﬀerent concern types. We illustrate both the intraspace and the interspace composition speciﬁcations and we use a model-to-code transformation tool to generate code for the CaesarJ [23] platform.

4

Case Study

In this section we explore the presented approach on a case study taken from the Electronic Health Information and Privacy (EHIP) domain. 4.1

Overview of the Method

First of all we present an overview of the method that we followed. Figure 5 presents an activity diagram showing the sequence of steps to be performed in order to develop an application by modularizing its access control model and its user interface model. In the ﬁrst activity the requirements are gathered and deﬁned using existing approaches for requirements engineering. In activity 2 and 2a, the access control

122

A. Hovsepyan et al.

2. Define access control model initial 1. Define requirements

2a. Define access control interface

5. Specify composition

3. Define base model

4. Define hypertext model

final 6. Generate code

7. Manually complete the code

5. Specify composition

Fig. 5. Overview of the method

policies are written and an access control interface is created. In activity 3, the software engineer deﬁnes the base concern model. Activity 4 requires the speciﬁcation of the user interface using a hypertext model. Activities 2, 3 and 4 can be done in parallel. In activity 5, the base-access control composition and base-user interface composition are speciﬁed. The last two activities involve semi-automatic code generation and its manual completion. Because of space restrictions we will not handle the ﬁrst activity explicitly. 4.2

Conceptual Instantiation of the Concern Composition Framework

Our case study uses a screening lab application taken from the Electronic Health Information and Privacy (EHIP) domain. The application is an information management system in a screening lab and provides authorized access to relevant patient data. Figure 6 illustrates a high-level overview of two concerns applied on a screening application. We use UML as the GPML for the base concern model of the screening application. We specify the user interface of this application in WebML. We only use the WebML hypertext model in our case study. The WebML data model is actually completely deﬁned by the base concern. A WebML implementation as an Ecore DSML is available, thus we can specify the composition with the base using the intraspace composition techniques. The access control is speciﬁed in XACML, which is a de facto standard language for specifying access control policies. As XACML lies in a diﬀerent technical space, we also need to create the access control interface in UML in order to specify the composition with the base concern. The application of the composition speciﬁcation targets the CaesarJ platform. Using the process described in the previous section we now illustrate the concepts from this ﬁgure on our case study. 4.3

Define Base Model

The screening lab application controls the information system of a screening lab. Figure 7 shows the structural base concern model expressed in UML. Patients (ScreeningSubject) make an appointment with a Secretary and have their radiographic pictures (Screening) taken by a Radiographer. Two diﬀerent Radiologists perform a Reading of the radiographic screening. In case the reading results are

Specifying and Composing Concerns Expressed in DSML

Ecore

XSD

WebML

UMLEMF

Web-based UI

Screening Application

Composition Model

conforms to

123

XACML

Access Control Interface

Access Control Concern

Composition Model

refers to base code Java

transformation

pointcuts CaesarJ

advice CaesarJ

Fig. 6. Screening Application Employee -salary : double -name : String Radiographer 1 performs conduct

+getDate() : date +screen() +setDate( d : date)

-license : String

1

* Reading -outcome : int * -results : String

* Screening -date : date -image

Doctor

Radiologist

1

* Conclusion

GeneralPractitioner

-report : String

1 1

*

1

1

1

ScreeningSubject

1

-sisID : String -name : String

Fig. 7. ScreeningLab Application Model

the same an automatic Conclusion is generated. Otherwise a third reading takes place, where after a third radiologist creates a ﬁnal conclusion. 4.4

Define Access Control Model

Many real world applications need to obey an access control policy in order to be deployed in practice. XACML is currently a de facto standard language in practice that enables a relatively straightforward speciﬁcation of security policies and its enforcement in applications. The access control policy for our application is largely deﬁned by the general requirements in the health care domain. A central concept in the health care domain is contact, which is essentially a workﬂow. A contact is a medical logical unit that can be an outpatient visit, a hospitalization, a surgery, a chemotherapy treatment, etc. A contact may result in a report after going through the following phases: – reception: the patient and the department is associated with the contact; – assignment: the contact is assigned to a physician and a supervisor;

124

A. Hovsepyan et al.

<Subjects><Subject> <SubjectMatch MatchId="urn:oasis:names:tc:xacml:1.0:function:string-equal"> <SubjectAttributeDesignator AttributeId="urn:e-hip:access-subject:attributes:role"/> general practitioner ehip:med:record view <Apply FunctionId="urn:oasis:names:tc:xacml:1.0:function:string-one-and-only"> <SubjectAttributeDesignator AttributeId="urn:oasis:names:tc:xacml:1.0:subject:subject-id"/> <Apply FunctionId="urn:oasis:names:tc:xacml:1.0:function:string-one-and-only">

Fig. 8. XACML policy sample

– report generation: the assigned physician generates the report, which is then placed on the work list of the supervisor for validation; – report validation: the assigned supervisor validates the report. This also closes the contact. The assigned physician may append and modify relevant patient data. The assigned physician may also close the contact by formulating an explicit statement that no report is necessary. External general practitioners (GP) are able to view their patients’ hospital records, so that they are kept informed of the health condition of their patients. The data that needs to be protected encompasses the contact and all the reports that are associated with the contact. We will refer to all these documents as patient data. The access control policy that has to be enforced, consists of the following rules: (1) Physicians who are assigned to a contact are granted access to patient data related to that contact, until 30 days after the contact was closed, (2) a general practitioner retains his access rights as long as he remains registered as the patient’s general practitioner, and (3) a physician can overrule an access denial, provided that a detailed reason is speciﬁed. We specify these access control policies in XACML. Figure 8 shows a snippet of an XACML policy stating that a general practitioner may view his patient’s medical record (some XACML speciﬁcs have been omitted for readability purposes). Each XACML policy or Rule has a feature called a Target, which is a set of simpliﬁed conditions for the Subject, Resource and Action that must be met for a Rule to apply to a given request. In our case general practitioner is the Subject, patient’s medical record (ehip:med:record) is the Resource and view is the Action. Each Subject, Resource and Action is identiﬁed using Attributes, which are central concepts in XACML. Attribute values are resolved from a request using AttributeDesignators that specify an attribute with a given

Specifying and Composing Concerns Expressed in DSML

125

name and type. For instance, to identify the Subject of our policy we use the AttributeDesignator with the name urn:e-hip:access-subject:attributes:role and type http://www.w3.org/2001/XMLSchema#string. Once the Target is matched the Condition is evaluated. Each Condition is a boolean function. In our policy we use a standard XACML function string-equal with two Attributes the subject-id of the patient and the gp-id of the general practitioner. If the Condition evaluates to true then the Rule’s Eﬀect is returned (Permit). If the Condition evaluates to false, the Condition does not apply and NotApplicable is returned. Due to space restrictions we will not present the full XACML policies for our application1 . 4.5

Define Access Control Interface

As the access control concern is expressed in a DSML from a technical space different from Ecore, we deﬁne an access control interface that serves as a bridge between the access control and the base concern. The access control interface consists of object interfaces that declare the available resources and a number of subject interfaces that declare the available subject roles. Both interfaces declare information that the policy may need, in the form of a number of attribute declarations. In addition, the object interface declares semantic actions. These semantic actions represent the security sensitive operations to which policies apply. All interfaces, attribute declarations and semantic actions are an abstract representation of the Attributes needed for the XACML policies from the previous subsection. In order to discern between security objects and subjects, we have created the object and subject stereotypes. The ﬁrst step towards creating an access control interface for the EHIP domain is ﬁltering out security objects and security subjects from the organizational requirements and access control policies. We have deﬁned four security subjects: Doctor and its specializations GP, ResponsiblePhysician and Supervisor ; and one security object: MedicalData. The next step is ﬁnding the relevant security actions, which will be executed on the security objects. We have selected 4 security actions on MedicalData relevant to the case study: view, append, close and validate. Finally, we have to specify the necessary security attributes for both security subjects and objects. For the Doctor security subject we have selected one relevant attribute - licenseID - which uniquely identiﬁes each doctor in general. GP and Supervisor introduce no additional attributes. ResponsiblePhysician has a reason attribute for the overrule of an access denial (policy rule 3). In addition it has a supervisor attribute to denote the supervisor. MedicalData has a patientID attribute that determines the patient. It has a closed and closingTime attributes that denote whether the contact has been closed, and if so the exact closing date and time. Finally, gp and responsiblePhysician attributes are used to specify the license ID for the general practitioner and responsible physician 1

The complete design models, sources, transformation and code generation templates and other relevant data used in this paper can be found at http:// www.cs.kuleuven.be/~aram/index.php?page=implementation1.html

126

A. Hovsepyan et al.

Fig. 9. EHIP Access Control Interface

for the contact. Fig. 9 shows the access control interface for our application that we have created from the access control concern model. 4.6

Define Hypertext Model

We have speciﬁed the user interface concern using WebML. Figure 10 illustrates a partial hypertext model of the application. The initial page (Screening Home) contains a menu that can be used to navigate to the linked pages. The Patients page shows an indexed list of patients and is further navigable to view each patient’s detailed information including the conclusion of the screening procedures. The Screenings page contains an index of screenings that are to be read by the radiologists. The users can then navigate to view each screening detail and the readings associated with it.

Fig. 10. User interface model

4.7

Specify Base-Access Control Composition Model

We need to map the concepts from the screening application, which are relevant for access control, to the access control interface. Based on the code-level solution by Verhanneman et al. [24,25] we have created a composition metamodel that can be used to specify any composition between an application and its

Specifying and Composing Concerns Expressed in DSML

127

Fig. 11. Composition Metamodel

access control interface. The composition metamodel, shown in ﬁgure 11 is implemented as an extension of the generic weaving metamodel speciﬁed by AMW [21]. MapObject and MapSubject link types extend the WLink type from the AMW metamodel and are used to map classes from the base concern to security objects and subjects from the access control interface. SecurityObject, SecuritySubject and BaseClass are link end types that extend the WLinkEnd type. MapAction link is used to link security sensitive operations from the base concern with the corresponding security actions from the access control interface. Finally, MapAttribute links are used in order to specify the necessary attribute calculation used to perform the access control. As AMW weaving tool has limited visualization capabilities, ﬁgure 12 shows only a partial snapshot of the access control composition model for the ScreeningLab application. The highlighted mapping links Screening class from the base with the MedicalData security object from the access control interface. 4.8

Specify Base-User Interface Composition Model

We have created another composition metamodel based on AMW base abstract weaving metamodel in order to specify the composition of the WebML model with the base concern. We have deﬁned two types of composition links. The ﬁrst type maps the WebML model entities (Entity) to the base concern. The second type deﬁnes the key condition that guides the how the relevant entities are selected. The entity mappings from the WebML model to the base are rather straightforward. The entities on PatientsList and PatientDetail pages map to ScreeningSubject class from the base, entities on Conclusion to Conclusion, entities on ScreeningsList and ScreeningDetail to Screening, and the ones on Readings to Reading. 4.9

Generate and Manually Complete Code

We have created a model-to-code transformation engine using MOFScript [26] that uses the base concern model, the XACML policies, its concern interface

128

A. Hovsepyan et al.

Fig. 12. Composition Model ScreeningLab

and the composition model as inputs and produces source code for the CaesarJ platform as an output. Currently, only the base-access control composition is realized in the generator. For a more detailed description of the generation engines, we refer to [27].

5

Discussion and Evaluation

In this paper we have identiﬁed two diﬀerent concern composition categories based on the notion of a technical space: intraspace and interspace. The concept of modular reasoning is important throughout all technical spaces as technical spaces already provide the fundamentals and the toolset of concern modularization and composition using diﬀerent modeling languages within the technical space. We have used these building blocks and we have demonstrated the feasibility of both intraspace and interspace compositions using a GPML for the base concern and DSMLs for the aspect concerns. However, we have introduced a number of restrictions in our pragmatic approach. In this section we provide an analysis of possible alternatives given the problems presented in section 2 and evaluate our approach. 5.1

Intraspace Compositions

There are two possible solutions in order to specify the composition between concerns that belong to the same technical space [16]. One possible solution is to integrate the metamodels of the concern models by the means of shared or linked modeling constructs. This approach is very similar to the way the UML metamodel combines the diﬀerent types of diagrams and information. For instance, the entity concept can be used on one hand to specify objects in a sequence diagram model and on the other hand to specify the classes

Specifying and Composing Concerns Expressed in DSML

129

in a structural data model. The advantage of this approach is that the concern composition is already speciﬁed at the meta-level. However, such an integration is very impractical especially if the involved metamodels are complex and evolving. Moreover, we would need to integrate every pair of DSMLs for which a composition speciﬁcation is necessary. A second option is to use a special composition model that speciﬁes the syntactic and semantic mapping between the elements of the involved concerns. Currently, many technical spaces already provide techniques and tools for composition speciﬁcation. The XML technical space supports schema modularization and composition out of the box, ATLAS Model Weaver (AMW) [21] is a generic model composition framework for the Eclipse Modeling Framework (EMF) technical space, AspectJ [28] could be used as one of the modularization and composition mechanisms for the Java technical space, etc. In our approach we have demonstrated the feasibility of the second option. Our approach is constrained to an asymmetric setting where the base concern is speciﬁed using a GPML. However, these constraints are not essential for the intraspace composition speciﬁcation, but are introduced only for the interspace composition speciﬁcations. Intraspace composition application is also supported by the technical spaces, which provide powerful transformation tools. Tools like ATLAS Transformation Language (ATL) [29] for Ecore, XSLT and XPath for XSD, and aspect weaver for AspectJ are capable of handling the composition speciﬁcations and transforming them into composed models. However, given two concerns that belong to diﬀerent DSMLs it remains unclear what their composed model should look like. Obviously, one would need to provide an intelligent combination of the input metamodels in order to enable the intracomposition application. One of the alternative solutions is using a GPML as an output metamodel rather than trying to calculate the output DSML. In order to do so, besides the composition engine, one would also need a mapping scheme for each DSML concept onto a GPML. Another approach is generating code for an AO platform where the aspect weaver will ultimately compose the models at the code level. This approach, however, will not produce an integrated design view that could be used, e.g., to better understand the interactions across the composed design views. Currently, we have not yet implemented the intraspace composition application of the screening lab from the case study. 5.2

Interspace Compositions

In theory, it is possible to create similar solutions for the interspace composition speciﬁcations as for the intraspace ones. The ﬁrst approach would require one to integrate two metamodels from diﬀerent technical spaces. Obviously, this option will be even more complex than that for intraspace composition speciﬁcation. The second alternative would mean deﬁning a more generic base abstract weaving metamodel that allows the speciﬁcation of links between metamodels from two technical spaces. However, this solution would be quite complex as well. First of all, one would need abstract weaving metamodels for each pair of technical

130

A. Hovsepyan et al.

spaces. Although the number of technical spaces used will probably be rather limited, creating such a metamodel is a daunting task on its own as it has to conform to a sophisticated combination of the metametamodels of the two technical spaces. Moreover, one would also need to create the tools that are capable of analyzing and transforming such interspace composition speciﬁcations. An diﬀerent approach presented by Bezivin et al. [19] proposes the concept of a projector, which is a special sort of transformation that can transform a concern from one technical space to another. A number of such projectors are already available to translate concerns between technical spaces (e.g. XMI, JMI). As the DSMLs we are using are typically small, they would require relatively little eﬀort to develop a projector. One of the advantages of this approach is that all concerns can be transformed to the same technical space and eventually be treated equally. Hence, a symmetrical AOM approach could be feasible. However, this approach may be less practical in certain cases. For instance, if applied to our case study, we would need to translate each XACML policy to the base technical space and specify an equal number of compositions. In this paper we have proposed a pragmatic approach that relies on the concept of a concern interface, which serves as a bridge between concerns expressed in diﬀerent modeling languages. In this case the GPML use is essential as we need a language that is expressive enough to specify the concern interface. We have explored the interspace compositions by illustrating how one can compose access control concern expressed in XACML and base concern expressed in GPML. Initially, UML seems a good choice as it is expressive enough and we expect that one could specify any concern interface in UML. As EMF implements only the structural part of the UML standard, it might be impossible to create a concern interface for, e.g., a behavioral concern. However, we see this as a limitation of EMF rather than UML. Even though the Ecore technical space is central in our case study, it is not essential for the ideas presented. There are several possible solutions in order to apply the interspace composition speciﬁcation. The ﬁrst alternative is to provide an intelligent combination of the input metamodels as a target output metamodel. However, as mentioned previously, this is quite complex to realize in practice. The second possibility is to use the projector approach for the composition speciﬁcation and rely on the intraspace composition application techniques. Finally, it is also possible to use the code generating approach like in the case of intraspace compositions. The approach presented in this paper uses the last alternative, for which the input concerns are transformed into AO code where the real composition is done by the aspect weaver. Given the power of the AO programming languages, we assume that any model-based composition can be translated into an appropriate pointcut code-level construct. In [27] we have illustrated some initial insights on how targeting an AO platform may oﬀer beneﬁts over the traditional OO platforms in a given AOM approach. A number of pure code-level approaches have also shown that AO source code may improve certain software qualities (e.g. [30,31]). Obviously, this approach will not work if OO source code is required.

Specifying and Composing Concerns Expressed in DSML

131

Moreover, there are no means to analyze concern interactions in order to identify for instance conﬂicts and undesirable emergent behaviors between concerns. A potential drawback of our approach is that instantiations for each concern type are very speciﬁc and require a substantial eﬀort. As we have illustrated in the previous section, the instantiation for the access control concern is completely diﬀerent from the instantiation for the user interface concern. The cost of building a concern interface and a composition model may also be a disadvantage compared to using an integrated non-AOM based approach. For instance, one could use the SecureUML [32] approach in order to specify the access control policies immediately on the base concern models.

6

Related Work

To our best knowledge, there is no existing AOM approach that can specify and perform the composition of concerns expressed in diﬀerent modeling languages from diﬀerent technical spaces. However, there are a number of related approaches that we describe according to several categories. The ﬁrst category outlines the relationships to the code-level AO approaches that implement the combination of a general purpose programming language (GPL) and a domainspeciﬁc programming language (DSL). The second category investigates the relationship with the most prominent AOM approaches. Finally, the last category presents the most prominent works in the area of DSMLs, access control modeling and data-intensive web application development in the context of MDE. 6.1

Domain Specific AOP Languages

Recent research on the combination of AO DSLs in order to tackle the increasing complexity and size of software applications has produced a number of interesting approaches for diﬀerent domains [33,34,35]. The AO DSLs typically specify a crosscutting concern using an expressive DSL, while the rest of the system is written in Java, which is a GPL. These approaches come with a low-level composition engine that can perform the code-weaving and produce plain Java byte-level code. The composition speciﬁcation, i.e. the pointcuts, are written in the DSL as well. The intraspace compositions in our approach are conceptually very similar to these code-level solutions. However, in the code-level solution the base is typically unaware or oblivious [36] to the aspect. Because of the fragile pointcut problem [37] where changes to the base may break the aspect, uncontrolled use of the powerful pointcut constructs may require the base to deal with aspect details. This will lead to actually reduced modularity. In our approach, this problem is substantially reduced by using explicit links in the composition model rather than using a pattern matching mechanism oﬀered by the pointcut language.

132

A. Hovsepyan et al.

6.2

AOM Approaches

There are many GPML-based AOM approaches, each pursuing a distinguished goal, providing diﬀerent concepts as well as notations [1]. UML is often used to specify the base concern. For aspect concerns, the existing approaches typically use a light-weight UML extension with the concepts speciﬁc for each approach. Typically most of the current approaches [4,5,6,7,8,9], whether symmetric or not, allow the modularization and composition of concerns in order to obtain a combined view. All these approaches are intraspace compositions in our terminology where all concerns are expressed in a GPML, i.e., UML. These approaches are complementary to our approach when it comes to concerns that are better expressed using a GPML rather than a DSML, e.g., concerns that belong to the solution domain rather than the problem domain (e.g. design patterns [38]), or concerns for which no suitable DSML exists and creating one would be too costly. All these approaches are in theory generic and can be used to model any concern type using UML. Song et al. [39] have demonstrated the use of a role-based access control (RBAC) within the AOM approach [4]. However, the approach where XACML can be used as it is to specify the access control policies is more suitable. Aspect-Oriented Domain Modeling (AODM) of Gray et al. [40] is the most prominent AOM approach that explicitly focuses on the usage of DSMLs. AODM is an extension of Model-Integrated Computing [41] with a model weaver framework called the Constraint-Speciﬁcation Aspect Weaver. Using AODM we can model an application using a DSML created by the Generic Modeling Environment tool and weave it with domain constraints written in the Embedded Constraint Language. However, AODM is limited to only one type of concerns, i.e. constraints. Our approach is complementary to AODM. 6.3

Other Approaches

A number of pure MDE approaches have successfully used DSMLs for specifying the software system on a higher abstraction level [16]. Even though these approaches are very practical, concern modularization is only a desired option and is not yet achieved. A number of approaches deﬁne UML extensions to provide support for access control. SecureUML [32] is the most prominent approach that uses extended RBAC with authorization constraints speciﬁed in OCL. However, SecureUML does not oﬀer any mechanisms for specifying conﬂict resolution strategies, whereas deﬁning them in XACML is straightforward. In addition, there is no support for policy grouping in SecureUML. Grouping of policies improves the performance of the system, as we can easily exclude certain policy groups from the authorization checks. Both conﬂict resolution and policy grouping are crucial in structuring and managing policies in complex applications. Finally, XACML oﬀers support for obligations in policies. Moreover, even though we have used XACML, our approach is easily extensible towards other access control models

Specifying and Composing Concerns Expressed in DSML

133

and modeling languages based on RBAC. The only thing that would need to be reworked is the model transformation engine. Ceri et al. [11] deﬁne an MDE framework that focuses on the design and semi-automatic implementation of data intensive web applications. The webbased user interface concern that we have illustrated is similar to their approach. However, our framework targets the modularization of multiple heterogeneous concerns and targets any kind of software systems.

7

Conclusions and Future Work

In this paper we have introduced the notion of a concern interface to bridge diﬀerent technical spaces. We have explored a combined intra- and interspace composition framework that is built on existing MDE approaches and tools. We have successfully illustrated the usage of this framework on an application with modularized access control concern and web-based user interface concern. In the future, we plan to further investigate this composition framework for other DSMLs and perform a detailed cost/beneﬁt analysis of using a combination of diﬀerent modeling languages.

References 1. Schauerhuber, A., Schwinger, W., Kapsammer, E., Retschitzegger, W., Wimmer, M., Kappel, G.: A survey on aspect-oriented modeling approaches. Technical report (2006) 2. France, R., Rumpe, B.: Model-driven development of complex software: A research roadmap. In: FOSE 2007: 2007 Future of Software Engineering, Washington, DC, USA, pp. 37–54. IEEE Computer Society, Los Alamitos (2007) 3. Simmonds, D., Reddy, R., France, R., Ghosh, S., Solberg, A.: An aspect oriented model driven framework. In: EDOC 2005, Washington, DC, USA, pp. 119–130. IEEE Computer Society, Los Alamitos (2005) 4. Reddy, Y.R., Ghosh, S., France, R.B., Straw, G., Bieman, J.M., McEachen, N., Song, E., Georg, G.: Directives for composing aspect-oriented design class models, pp. 75–105 (2006) 5. Baniassad, E., Clarke, S.: Aspect-Oriented Analysis and Design: The Theme Approach. Addison-Wesley, Reading (2005) 6. Hanenberg, S., Stein, D., Unland, R.: From aspect-oriented design to aspectoriented programs: tool-supported translation of jpdds into code. In: AOSD 2007: Proceedings of the 6th international conference on Aspect-oriented software development, pp. 49–62. ACM, New York (2007) 7. Sanchez, P., Fuentes, L.: Designing and weaving aspect-oriented executable uml models. Journal of Object Technology (6) (2007) 8. Jézéquel, J.M.: Model driven design and aspect weaving. Software and Systems Modeling (2008) 9. Hovsepyan, A., Van Baelen, S., Berbers, Y., Joosen, W.: Generic reusable concern compositions. In: Fourth European Conference on Model Driven Architecture Foundations and Applications, pp. 231–245 (2008)

134

A. Hovsepyan et al.

10. OASIS: Core speciﬁcation: Extensible access control markup language (XACML) v2.0, www.oasis-open.org/commitees/xacml 11. Ceri, S., Fraternali, P., Bongio, A.: Web modeling language (webml): a modeling language for designing web sites. In: Proceedings of the 9th international World Wide Web conference on Computer networks: the international journal of computer and telecommunications networking, pp. 137–157. North-Holland Publishing Co., Amsterdam (2000) 12. Ossher, H., Tarr, P.: Multi-Dimensional Separation of Concerns and The Hyperspace Approach. In: Proceedings of the Symposium on Software Architectures and Component Technology: The State of the Art in Software Development (2000) 13. Kiczales, G.: Aspect-oriented programming. ACM Comput. Surv. 154 14. Rashid, A., Moreira, A.: Domain models are NOT aspect free. In: Nierstrasz, O., Whittle, J., Harel, D., Reggio, G. (eds.) MoDELS 2006. LNCS, vol. 4199, pp. 155–169. Springer, Heidelberg (2006) 15. OMG: UML superstructure, v2.0. OMG Document number formal/05-07-04 (2005) 16. Kelly, S., Tolvanen, J.P.: Domain-Speciﬁc Modeling: Enabling Full Code Generation. Wiley-IEEE Computer Society Press (2008) 17. Mernik, M., Heering, J., Sloane, A.M.: When and how to develop domain-speciﬁc languages. ACM Comput. Surv. 37, 316–344 (2005) 18. Kurtev, I., Bezivin, J., Aksit, M.: Technological spaces: An initial appraisal. In: CoopIS, DOA 2002 Federated Conferences, Industrial track (2002) 19. Bezivin, J., Kurtev, I.: Model-based technology integration with the technical space concept. In: Metainformatics Symposium 2005, Esbjerg, Denmark (2005) 20. Sanen, F., Truyen, E., Joosen, W.: Managing concern interactions in middleware. In: Indulska, J., Raymond, K. (eds.) DAIS 2007. LNCS, vol. 4531, pp. 267–283. Springer, Heidelberg (2007) 21. Del Fabro, M.D., Bezivin, J., Valduriez, P.: Weaving models with the eclipse amw plugin. In: Eclipse Modeling Symposium, Eclipse Summit Europe 2006 (2006) 22. Moreira, A., Rashid, A., Araujo, J.: Multi-dimensional separation of concerns in requirements engineering. In: RE 2005: Proceedings of the 13th IEEE International Conference on Requirements Engineering, Washington, DC, USA, pp. 285–296. IEEE Computer Society, Los Alamitos (2005) 23. Aracic, I., Gasiunas, V., Mezini, M., Ostermann, K.: An overview of CaesarJ. In: Transactions on Aspect-Oriented Software Development, pp. 135–173 (2006) 24. Verhanneman, T., Piessens, F., De Win, B., Truyen, E., Joosen, W.: A modular access control service for supporting application-speciﬁc policies, vol. 7, p. 1. IEEE Educational Activities Department, Piscataway (2006) 25. Verhanneman, T., Piessens, F., De Win, B., Joosen, W.: Uniform applicationlevel access control enforcement of organizationwide policies. In: ACSAC 2005, pp. 431–440. IEEE Computer Society, Los Alamitos (2005) 26. SINTEF: MOFScript, http://modelbased.net/mofscript/ 27. Hovsepyan, A., Van Baelen, S., Yskout, K., Berbers, Y., Joosen, W.: Composing application models and security models: on the value of aspect-oriented technologies. In: Proc. of the 11th Int. Workshop on AOM@MODELS 2007 (2007) 28. Kiczales, G., Hilsdale, E., Hugunin, J., Kersten, M., Palm, J., Griswold, W.G.: An overview of aspectJ. In: Knudsen, J.L. (ed.) ECOOP 2001. LNCS, vol. 2072, pp. 327–355. Springer, Heidelberg (2001) 29. Jouault, F., Kurtev, I.: Transforming models with ATL. In: Bruel, J.-M. (ed.) MoDELS 2005. LNCS, vol. 3844, pp. 128–138. Springer, Heidelberg (2006)

Specifying and Composing Concerns Expressed in DSML

135

30. Garcia, A., Sant’Anna, C., Figueiredo, E., Kulesza, U., Lucena, C., von Staa, A.: Modularizing design patterns with aspects: A quantitative study. In: 4th international conference on AOSD (2005) 31. Greenwood, P., Bartolomei, T., Figueiredo, E., Dosea, M., Garcia, A., Cacho, N., Sant’Anna, C., Soares, S., Borba, P., Kulesza, U., Rashid, A.: On the impact of aspectual decompositions on design stability: An empirical study. In: Ernst, E. (ed.) ECOOP 2007. LNCS, vol. 4609, pp. 176–200. Springer, Heidelberg (2007) 32. Basin, D., Doser, J., Lodderstedt, T.: Model driven security: from uml models to access control infrastructures. ACM Transactions on Software Engineering and Methodology 15, 39–91 (2006) 33. Fabry, J., Tanter, É., DHondt, T.: Kala: Kernel aspect language for advanced transactions. Sci. Comput. Program. 71, 165–180 (2008) 34. Amor, M., Garcia, A., Fuentes, L.: Agol: An aspect-oriented domain-speciﬁc language for mas. In: EARLYASPECTS 2007: Proceedings of the Early Aspects at ICSE, Washington, DC, USA, p. 4. IEEE Computer Society, Los Alamitos (2007) 35. Dinkelaker, T., Mezini, M.: Dynamically linked domain-speciﬁc extensions for advice languages. In: DSAL 2008: Proceedings of the 2008 AOSD workshop on Domain-speciﬁc aspect languages, pp. 1–7. ACM, New York (2008) 36. Filman, R., Friedman, D.: Aspect-oriented programming is quantiﬁcation and obliviousness. In: Workshop on Advanced Separation of Concerns, OOPSLA 2000, Minneapolis (2000) 37. Sullivan, K., Griswold, W.G., Song, Y., Cai, Y., Shonle, M., Tewari, N., Rajan, H.: Information hiding interfaces for aspect-oriented design. In: ESEC/FSE-13: Proceedings of the 10th European software engineering conference held jointly with 13th ACM SIGSOFT international symposium on Foundations of software engineering, pp. 166–175. ACM, New York (2005) 38. Gamma, E., Helm, R., Johnson, R., Vlissides, J.: Design Patterns: Elements of Reusable Object-Oriented Software. Addison-Wesley Professional, Reading (1995) 39. Song, E., Reddy, R., France, R., Ray, I., Georg, G., Alexander, R.: Veriﬁable composition of access control and application features. In: SACMAT 2005: Proceedings of the tenth ACM symposium on Access control models and technologies, pp. 120–129. ACM, New York (2005) 40. Gray, J., Bapty, T., Neema, S., Schmidt, D.C., Gokhale, A., Natarajan, B.: An approach for supporting aspect-oriented domain modeling. In: Pfenning, F., Smaragdakis, Y. (eds.) GPCE 2003. LNCS, vol. 2830, pp. 151–168. Springer, Heidelberg (2003) 41. Sztipanovits, J., Karsai, G.: Model-integrated computing. IEEE Computer (1997)

Early Crosscutting Metrics as Predictors of Software Instability José M. Conejero1, Eduardo Figueiredo2, Alessandro Garcia3, Juan Hernández1, and Elena Jurado1 1

Quercus Software Engineering Group, University of Extremadura, Spain 2 Computing Department, Lancaster University, United Kingdom 3 Informatics Department,Pontifical Catholic University of Rio de Janeiro, Brazil {chemacm,juanher,elenajur}@unex.es, [email protected], [email protected]

Abstract. Many researchers claim that crosscutting concerns, which emerge in early software development stages, are harmful to software stability. On the other hand, there is a lack of effective metrics that allow software developers to understand and predict the characteristics of “early” crosscutting concerns that lead to software instabilities. In general, existing crosscutting metrics are defined for specific programming languages and have been evaluated only against source-code analysis, when major design decisions have already been made. This paper presents a generic suite of metrics to objectively quantify key crosscutting properties, such as scattering and tangling. The definition of the metrics is agnostic to particular language intricacies and can be applied to all early software development artifacts, such as usecases and scenarios. We have performed a first stability study of crosscutting on requirements documents. The results pointed out that early scattering and crosscutting have, in general, a strong correlation with major software instabilities and, therefore, can help developers to anticipate important decisions regarding stability at early stages of development. Keywords: Concern Metrics, Modularity, Stability, Requirements Engineering.

1 Introduction There is growing empirical evidence that software stability is often inversely proportional to the presence of crosscutting concerns [8, 9, 12, 13]. A software system is stable if, when observed over two or more versions of the software, the differences between its quality measures are insignificant [14]. It is claimed that crosscutting concerns often lead to harmful software instabilities, such as increased modularity anomalies [9, 13] and higher number of introduced faults [8]. The problem of crosscutting concerns is usually described in terms of scattering and tangling [3]. Scattering occurs when the realization of a concern is spread over the software modules whilst tangling occurs when the concern realization is mixed with other concerns in a module. Most of the crosscutting concerns manifest in early development artifacts, such as requirements descriptions [3] and architectural models [18, 11], due to their M. Oriol and B. Meyer (Eds.): TOOLS EUROPE 2009, LNBIP 33, pp. 136–156, 2009. © Springer-Verlag Berlin Heidelberg 2009

Early Crosscutting Metrics as Predictors of Software Instability

137

widely-scoped influence in software decompositions. They can be observed in every kind of requirements and design representations, such as usecases and component models [3, 18, 11, 2]. Over the last years, aspect-oriented software development (AOSD) [15] has emerged with the goal of supporting improved modularity and stability of crosscutting concerns throughout the software lifecycle. However, the use of aspect-oriented decompositions cannot be straightforwardly applied without proper assessment mechanisms for early software development stages. This became more evident according to recent empirical studies of AOSD based on source-code analysis (e.g. [9, 12, 13]). First, not all types of crosscutting concerns were found to be harmful to design stability. Second, there are certain measurable characteristics of crosscutting concerns that seem to recurrently lead to design instabilities [9, 13]. However, there is little or no knowledge about how characteristics of crosscutting concerns, observable in early artefacts, are correlated with design instabilities. Most of the systematic studies of crosscutting concerns (e.g. [8, 9, 12, 13]) concentrate on the analysis of source code, when architectural decisions have already been made. Even worse, a survey of existing crosscutting metrics has pointed out that they are defined in terms of specific OO and aspect-oriented (AO) programming languages [10]. However, inferring design stability after investing in OO or AO implementations can be expensive and impractical. In addition, crosscutting metrics defined for early design representation are very specific to certain models, such as component-andconnector models [18]. These metrics are overly limited as many crosscutting concerns are visible in certain system representations, but not in others [10]. In this context, the major contributions of this paper are threefold. First, it presents a language-agnostic metrics suite for early quantification of crosscutting (Section 3). This is particular useful with the transition to model-driven software engineering gaining momentum, where analysis of crosscutting concerns should also be undertaken in early system representations. The definition of the metrics is based on a conceptual framework (Section 2) that is independent of specific requirements and architectural models. Second, canonical instantiations of the crosscutting metrics are given for usecases. Third, we also present a first exploratory study investigating the correlation of early crosscutting measures and design instabilities (Section 4). The results obtained help developers to anticipate important decisions regarding stability at early stages of development. Finally, Sections 5 and 6 discuss related work and conclude this paper.

2 Characterizing and Identifying Crosscutting Concerns The operational definitions of concern-oriented metrics need to be conceived in an unambiguous manner. However, concern properties, such as crosscutting and scattering, are often not formally defined. Our proposed concern-oriented metrics (Section 3) are based on a previously-defined conceptual framework [3] that supports the characterization and identification of crosscutting. Section 2.1 describes the key definitions of this conceptual framework. Section 2.2 illustrates its instantiation to requirements-level artefacts of a software system.

138

J.M. Conejero et al.

2.1 A Conceptual Framework for Crosscutting Our previous work [3] presented a conceptual framework where a formal definition of crosscutting was provided. This framework is based on the study of matrices that represent particular features of a traceability relationship between two different domains. These domains, generically called Source and Target, could be, for example, concerns and usecases respectively or, in a different situation, design modules and programming artefacts. We used the term Crosscutting Pattern to denote this situation (see Fig. 1).

Fig. 1. Abstract meta-model of the crosscutting pattern

The relationship between Source and Target can be formalized by two functions f and g, where g can be considered as a special inverse function of f. Let f: Source tions defined by:

⎯⎯→

P

(Target) and g: Target

⎯⎯→

P

(Source) be these func-

∀ s ∈ Source, f(s) = {t ∈ Target :there exists a trace relation between s and t } ∀ t ∈ Target, g(t) = {s ∈ Source : there exists a trace relation between s and t}. The concepts of scattering, tangling, and crosscutting are defined as specific cases of these functions. Definition 1. [Scattering] We say that an element s ∈ Source is scattered if card(f(s)) > 1, where card refers to cardinality of f(s). In other words, scattering occurs when, in a mapping between source and target, a source element is related to multiple target elements. Definition 2. [Tangling] We say that an element t ∈ Target is tangled if card(g(t))>1. Tangling occurs when, in a mapping between source and target, a target element is related to multiple source elements. There is a specific combination of scattering and tangling which we call crosscutting. Definition 3. [Crosscutting] Let s1, s2 ∈ Source, s1 ≠ s2, we say that s1 crosscuts s2 if card(f(s1)) > 1 and ∃ t ∈ f(s1): s2 ∈ g(t). Crosscutting occurs when, in a mapping between source and target, a source element is scattered over target elements and where in at least one of these target elements, some other source element is tangled. In [6] we formally compared our definition with others existing in the literature, such as [17].

Early Crosscutting Metrics as Predictors of Software Instability

139

2.2 Identification of Crosscutting In [3], we defined the dependency matrix to represent function f. An example of dependency matrix with five source and six target elements is shown in Table 1. A 1 in a cell means that the target element of the corresponding column contributes to or addresses the source element of the corresponding row. Based on this matrix, two different matrices called scattering matrix and tangling matrix are derived.

Source

Table 1. Example dependency matrix

s[1] s[2] s[3] s[4] s[5]

dependency matrix Target t[1] t[2] t[3] t[4] 1 0 0 1 1 0 1 0 1 0 0 0 0 1 1 0 0 0 0 1

t[5] 0 1 0 0 1

t[6] 0 1 0 0 0

The crosscutting product matrix is obtained through the multiplication of scattering matrix and tangling matrix. The crosscutting product matrix shows the quantity of crosscutting relations (Table 2) and is used to derive the final crosscutting matrix (Table 3). A cell in the final crosscutting matrix denotes the occurrence of crosscutting, but abstracts the quantity of crosscutting. More details about the conceptual framework and the matrix operations can be found in [3].

s[1] s[2] s[3] s[4] s[5]

s[1] 2 1 0 0 1

s[2] 1 3 0 1 1

Source s[3] s[4] 1 0 1 1 0 0 0 1 0 0

s[5] 1 1 0 0 2

Table 3. Crosscutting matrix

Source

Source

Table 2. Crosscutting product matrix

s[1] s[2] s[3] s[4] s[5]

s[1] 0 1 0 0 1

s[2] 1 0 0 1 1

Source s[3] 1 1 0 0 0

s[4] 0 1 0 0 0

s[5] 1 1 0 0 0

3 Concern-Oriented Metrics for Early Development Assessment In this section, we propose a concern-oriented metric suite based on the framework presented in Section 2. These metrics allow developers to quantify the degree of scattering, tangling, and crosscutting at earlier development stages of a software system, such as requirements and architecture modeling. The metrics defined are based on the relation between source and target domains represented by the crosscutting pattern. In order to illustrate the metrics, we rely on requirements descriptions of a running example (MobileMedia). 3.1 The MobileMedia System The MobileMedia [9] is a product line system built to allow the user of a mobile device to perform different options, such as visualizing photos, playing music or videos,

140

J.M. Conejero et al.

and sending photos by SMS (among other concerns). It has about 3 KLOC. The system has been built as a product line in 8 different releases. In this section we show a simple usecase diagram (Fig. 2) which corresponds to a part of the usecase diagram used for release 0 in the MobileMedia system. In this part of the diagram, the actions for adding albums and photos to the system are implemented. These actions include the option for providing a label. Some actions for recording the data into a persistent storage are also included. Then, we consider that four main concerns are involved in this part of the system: album, photo, label, and persistence.

Fig. 2. Simplification of the usecase diagram for release 0 in MobileMedia

In Table 4, we show a simplified description of the usecases shown in Fig. 2. We have shadowed in light and dark grey colors the flows and relations in these usecases corresponding to Label and Persistence concerns, respectively. Although there are other two concerns involved in the example (album and photo), we have not shadowed any flow related to them to keep the example clear. Table 4. Usecase descriptions for Add Album, Add Photo and Provide Label usecases Usecase: Add Album

Usecase: Add Photo

Actor: Mobile Phone (system) and User Description: The user can store (add) an album to the mobile phone Pre/Posconditions: (…) Basic flows: 1. (add)The user selects the option to add an album. 2. (label) User provides label to the new created album 3. (saved) A new album is available 4. (listing) The list of photos is displayed Includes: 1.Provide Label usecase 2. Store Data usecase

Actor: Mobile Phone (system) and User Description: User can store (add) a photo in an album available Pre/Posconditions: (…) Basic flows: 1. (select) The user selects an album to store the photo. 2. (add)The user selects the option to add photo. 3. (path) User provides the path for uploading the photo. 4. (label) User assigns a label to the photo. 5. (saved) The new photo is stored in the album. Includes: 1. Provide Label usecase 2. Store Data usecase

Usecase: Provide Label Actor: Mobile Phone (system) and User Description: The user provides label for the photo and album Pre/Posconditions: (…) Basic flows: 1. (label) The users provides a name for a photo or an album 2. (save) The edited label is saved Includes: 1. Store Data usecase

Usecase: Store Data Actor: Mobile Phone (system) Description: The data of a photo or an album must be stored into the device storage Pre/Posconditions: (…) Basic flow: 1. (space) The device select a space in the storage 2. (save) The data are saved in the storage

Early Crosscutting Metrics as Predictors of Software Instability

141

In order to clarify the metrics presented in next sections, each metric is illustrated using the previously described example. In particular, we show the values of each metric for the partial usecase diagram (Fig. 2) and usecase descriptions (Table 4). 3.2 Metrics for Scattering According to Definition 1 in Section 2.1, Nscattering of a source element sk as the number of 1’s in the corresponding row (k) of the dependency matrix: | |

(1)

where |T| is the number of target elements and dmkj dm is the value of the cell [k,j] of the dependency matrix. We may also express this metric according to the functions defined in Section 2.1 as NScattering (sk) = card {t є Target : f’(sk)=t}, i.e. card(f(sk)). This metric measures how scattered a concern is. In the example shown in Section 3.1, the NScattering for Label and Persistence concerns is 3 and 4, respectively. As we can see in Table 4, all the usecases descriptions have some flows or relations shadowed with dark grey (related to Persistence), however there are only 3 usecases with light grey (related to Label). Then we may assure that the Persistence concern is more scattered than Label. This NScattering metric can be normalized in order to obtain a value between 0 and 1. Then, we define Degree of scattering of the source element sk as: ∑|

| |

|

1

| |

(2)

| |

1

0

The closer to zero this metric for a source element (i.e., a concern), the better encapsulated the source element. Conversely, when the metric has a value closer to 1, the source element is highly spread over the target elements and it is worse encapsulated. This metric could have been also defined in terms of the scattering matrix (instead of dependency matrix). In order to have a global metric for how much scattering the system’s concerns are, we define the concept of Global scattering (Gscattering) which is obtained just by calculating the average of the Degree of scattering values for each source elements: ∑|

|

| |

(3)

where |S| is the number of analyzed source elements. In our particular case, it represents the number of concerns of interest in the system.

142

J.M. Conejero et al.

3.3 Metrics for Tangling Similarly to Nscattering for scattering, we also defined the Ntangling metric for the target element tk, where |S| is the number of source elements and dmjk is the value of the cell [i,k] of the dependency matrix: | |

(4)

Again, according to the functions introduced in Section 2.1, Ntangling (tk) = card {s є Source : f’(s)=tk}, i.e., card(f(tk)). Then, this metric measures the number of source elements addressed by a particular target element. In the MobileMedia example (Section 3.1), the NTangling for the Add Album and Store Data usecases are 2 and 1, respectively. As we can see in Table 4, Store Data usecase is only shadowed in dark grey color so that it just addresses the Persistence concern (whilst Add Album addresses Persistence and Label). We follow the same steps performed for the scattering metrics and to define two tangling metrics: Degree of tangling and Gtangling. These metrics represent the normalized tangling for the target element tk and the global tangling, respectively: ∑|

|

| |

| | | |

0 ∑|

1 (5)

1

|

| |

(6)

Like Degree of scattering, the Degree of tangling metric may take values between 0 and 1, where the value 0 represents a target element addressing only one source element. The number of source elements addressed by the target element increases as the metric is closer to 1. 3.4 Metrics for Crosscutting Finally, this section defines three metrics for crosscutting: Crosscutpoints, NCrosscut and Degree of crosscutting. These metrics are extracted from the crosscutting product matrix and the crosscutting matrix of the framework presented in Section 2.1. The Crosscutpoints metric is defined for a source element sk as the number of target elements where sk is crosscutting to other source elements. This metric is calculated from the crosscutting product matrix (remember that this matrix is calculated by the product of scattering and tangling matrices). The Crosscutpoints metric for sk corresponds to the value of the cell in the diagonal of the row k (cell [k,k] or ccpmkk). (7)

According to our running example of Section 3.1, we can see that Crosscutpoints metric for Persistence has a value of 3. Note that there are three usecases descriptions

Early Crosscutting Metrics as Predictors of Software Instability

143

(Table 4) which are shadowed with both light and dark color (Add Album, Add Photo and Provide Label). Then, the Persistence and Label concerns cut across each other in these usecases. The NCrosscut metric is defined for the source element sk as the number of source elements crosscut by sk. The NCrosscut metric for sk is calculated by the addition of all the cells of the row k in the crosscutting matrix: | |

(8)

In our example, NCrosscut for Persistence is 1 since it is crosscutting just to the Label concern. Finally, the two crosscutting metrics above allow us to define the Degree of crosscutting metric of a source element sk. Note that, Degree of crosscutting is normalized between 0 and 1, so that those source elements with lower values for this metric are the best modularized. | |

(9)

| |

We summarize all our metrics in Table 5. In this table we show the definition of each metric and the relation with the matrices used by the crosscutting pattern. Table 5. Summary of the concern-oriented metrics based on the Crosscutting Pattern Metric

Definition

NScattering (sk)

Number of target elements addressing source element sk

Degree of scattering (sk)

Normalization of NScattering (sk) between 0 and 1

Relation with matrices Addition of the values of | cells in row k in depen- = ∑ dency matrix (dm)

Calculation |

∑| |

1

| |

∑

Average of Degree of scattering of the source elments

NTangling (tk)

Addition of the values of Number of source elements | cells in column k in =∑ addressed by target element tk dependency matrix (dm)

Degree of tangling (tk)

Normalization of NTangling (tk) between 0 and 1 Average of Degree of tangling of the target elments

1

| |

∑

0

Gscattering (sk)

Gtangling (tk)

| |

∑

| |

=

| | |

∑| |

| |

1

| |

1

∑

| |

=

∑

0 | |

∑

Number of target elements Diagonal cell of row k in Crosscutpoints where the source element sk the crosscutting product = (sk) crosscuts to other source matrix (ccpm) elements Number of source elements Addition of the values of | NCrosscut (sk) crosscut by the source elecells in row k in the =∑ ment sk crosscutting matrix (ccm) Addition of the two last Degree of metrics normalized between 0 = crosscutting (sk) and 1

| |

|

∑| | | | | |

144

J.M. Conejero et al.

4 Evaluation and Discussion In this section we present a first empirical study using the metrics presented in Section 3. The main goal of the analysis presented is to observe how early crosscutting metrics may help in predicting instabilities. Then, these metrics provide indications that the crosscutting concerns identified should be modularized, e.g., using aspects [2]. This hypothesis is tested by using a double validation: (1) an internal validation of the crosscutting metrics with respect to their ability of accurately quantifying certain crosscutting properties, and (2) an external validation of the crosscutting metrics in terms of their predictability of software stability [14]. 4.1 Survey of Related Metrics In this section we briefly discuss several concern-oriented metrics [7, 8, 19, 22] which have been used in our internal and external validations. These metrics are summarized in Table 6. Unlike the metrics presented in this paper, the metrics summarized in Table 6 are mainly defined in terms of specific design or implementation artefacts. Accordingly, we have adapted these metrics to the requirements level in order to compare the results obtained by our metrics with those obtained by the rest of metrics (in Section 4.2). The adaptation has mainly consisted of a change in the target element Table 6. Survey of metrics defined by other authors

[19] [18]

Metric Concern Diffusion over Components (CDC) Concern Diffusion over Operations (CDO) Concern Diffusion over Lines of Code (CDLOC) Lack of Concern Cohesion (LOCC) Component-level Interlacing Between Concerns (CIBC) Size Spread Focus

Wong et al. [22]

Touch Concentration (CONC)

Eaddy et al. [8]

Ducasse et al. [7]

Sant’Anna et al.

Authors

Degree of scattering (DOS)

Dedication (DEDI) Disparity (DISP)

Degree of tangling (DOT)

Definition It counts the number of components addressing a concern. It counts the number of methods and advices addressing a concern. It counts the number of lines of code related to a particular concern. It counts the number of concerns addressed by the assessed component. It counts the number of other concerns with which the assessed concerns share at least a component. It counts the number of internal members of classes (methods or attributes) associated to a concern. It counts the number of modules (classes or components) related to a particular concern It measures the closeness between a module and a property or concern It assesses the relative size of a concern or a property (Size of property divided into total size of system) It measures how much a concern is concentrated in a component It quantifies how much a component is dedicated to a concern. It measures how many blocks related to a particular property (or concern) are localised in a particular component It is defined as the variance of the Concentration of a concern over all program elements with respect to the worst case It is defined as the variance of the Dedication of a component for all the concern with respect to the worst case

Early Crosscutting Metrics as Predictors of Software Instability

145

used for the different measures. For example, where a metric was defined for measuring concepts using components or classes, we have adapted the metric to the requirements domain by using usecases as the target entity. We have also taken into account the different granularity levels used by the metrics. For instance, there are some metrics which use operations or lines of code (instead of components or classes) as the target entity. In order to adapt these metrics, we changed operations by usecase flows or steps, since flows represent a finer granularity level (similar to operations or lines of code) than usecases. Then, the granularity level used at requirements keeps consistent with the used by the original metrics. 4.2 Internal Validation In this section we show the results obtained by calculating our metrics to the MobileMedia system [9]. The application has been used for performing different analyses in software product lines mainly at architectural and programming level [9]. Our work complements those previous analyses since we focus on modularity at the requirements level. The reason for calculating the metrics at this level is to identify the concerns with a higher Degree of crosscutting (a poor modularity) as soon as possible. Then, the developer may anticipate important decisions regarding quality attributes at early stages of development. Table 7. Different releases of MobileMedia Release Description r0 MobilePhoto core r1 Error handling added Sort Photos by frequency and Edit Label r2 concerns added r3 Set Favourites photos added r4 Added a concern to copy photo to an album r5 Added a concern for sending photos by SMS r6 Added the concern for playing music Added the concern for playing videos and r7 capture media

Table 8. Concerns and releases where are included Concern Releases Concern Releases Album r0 - r7, Copy r4 - r7 Photo r0 - r7, SMS r5 - r7 Label r0 - r7, Music r6, r7 Persistence r0 - r7, Media r6, r7 Error Handling r1 - r7 Video r7 Sorting r2 - r7 Capture r7 Favourites r3 - r7

As discussed in Section 3.1, MobileMedia has evolved to 8 successive releases by adding different concerns to the product line. For instance, release 0 implements the original system with just the functionality of viewing photos and organizing them by albums. In Table 7 we show the different releases with the concerns added in each release (see [9] for more details). The reasons for choosing this application for our first analysis are several. (1) The MobileMedia application is a product line, where software instability is of upmost importance; instabilities affect negatively not only the Software Product Line (SPL) architecture, but also all the instantiated products. (2) The software architecture and the requirements had all-encompassing documentation; e.g., the description of all the usecases were made available as well as a complete specification of all the component interfaces. (3) The architectural components were independently defined and provided by the real developers, rather than ourselves. The architectural part could be used by a second study to analyze traceability of crosscutting concerns (observed using our metrics). (4) The developers had implemented an

146

J.M. Conejero et al.

aspect-oriented version of the system, which may be also used in a different analysis for comparing the metrics applied in different paradigms. We have calculated the metrics presented in Section 3 for the requirements of each release. We have considered the different concerns of each release and the usecases implementing the system as the source and target domains, respectively. Table 8 shows the concerns used for the analysis and the releases in which these concerns were included. We do not show the usecases diagrams for each release due to space constraints, the whole analysis of the experiment may be found in [1]. 4.2.1 Calculating the Metrics Based on the two domains (concerns and usecases as source and target, respectively) we build the dependency matrix for each release showing the usecases contributing to the different concerns. Our metrics and those summarized in Table 6 are automatically calculated using as input the dependency matrix. Based on this dependency matrix, we derive the rest of matrices presented in Section 2.2 (Scattering, Tangling, Crosscutting Product, and Crosscutting Matrices). Due to space reasons, we just show the dependency matrix for the MobileMedia system in release 7, which includes all the concerns of the system (Table 9). Table 9. Dependency matrix for the MobileMedia system in release 7

Concerns

Album Photo Label Persistence Error Handling Sorting Favourites Copy SMS Music Media Video Capture

2 2 1

Capture Media

Add Album Delete Album Add Media Delete Media View Photo View Album Provide Label Store Data Remove Data Retrieve Data Edit Label Count Media View Sorted Media Set Favourite View Favourites Copy Media Send Media Receive Media Music Control Access Media Play Video

Usecases

3 1

2 2 2 2 2 2 2 2 2 3 2 2 1

1 1 1 2 1 2 2 2 2 2 2 2 3 1 1 2 2 2 2 1 1 1 1 1 2 2 2 1 1 2 2 1 1 1 1 1 1 1

2 2 1

1

1 1 3

1 2 1

3

2

3

1 1 1

Although our original dependency matrix is a binary matrix, in this case we have used a not-binary matrix in order to allow the calculation of metrics which utilize a granularity level different from usecase. That means that a cell represents the number of control flows or steps of the usecase addressing a particular concern. For instance, in Table 9 we can see how the View Album usecase has 3 and 1 control flows addressing the Album and Label concerns, respectively. In order to relate concerns and

Early Crosscutting Metrics as Predictors of Software Instability

147

usecases (i.e. fill in the dependency matrix), we have used a shadowing technique (see an example in Section 3.1) which was used at source code level in [9]. Using the dependency matrix for each release, we automatically calculate all metrics for these releases. In Table 10 we show the average of the metrics for all releases. In this table we have shown only the metrics calculated for concerns (source elements in the Crosscutting Pattern). In [1] we show the calculation for all the metrics presented in Table 6. We have performed a pairwise systematic comparison of the metrics and an in-depth discussion is presented at the website [1]. In next section we focus on the key results. Table 10. Average of the metrics for all the releases Releases Authors

Concern Difussion over Flows

Usecase level Interlacing between Concerns

Size

Spread

Focus

Degree of Scattering

5,25 4,13 6 6,38 7 7,33 6 6,25 6,67 0 8,5 0 0

Eaddy

Concern Difussuion over Usecases

3,63 3,88 5,38 12,4 15,9 4,33 3 2,5 3,67 0 6,5 0 0

Ducasse

Degree of crosscutting

Degree of scattering 0,26 0,3 0,34 0,85 0,98 0,25 0,17 0,13 0,18 0 0,31 0 0 0,27

NCrosscut

3,63 4,13 5,38 12,8 15,9 4,33 3 2,5 3,67 1 6,5 1 1

Crosscutpoints

Album Photo Label Persistence Error Handling Sorting Favourites Copy SMS Music Media Video Capture Globals/Avg

Nscattering

Metrics

Concerns

Average of all releases Sant’Anna

Ours

0,39 0,38 0,46 0,77 0,89 0,43 0,32 0,29 0,32 0 0,44 0 0 0,34

3,63 4,13 5,38 12,8 15,9 4,33 3 2,5 3,67 1 6,5 1 1

8 7,38 7,88 25,1 27,6 5,33 4 2,5 3,67 3 12,5 1 1

5,25 5,38 6,13 6,38 7 7,33 6 6,25 6,67 0 8,5 0 2

8 7,38 7,88 25,1 27,6 5,33 4 2,5 3,67 3 12,5 1 1

3,63 4,13 5,38 12,8 15,9 4,33 3 2,5 3,67 1 6,5 1 1

0,25 0,24 0,25 0,39 0,36 0,34 0,26 0,09 0,16 1 0,25 1 0,33

0,77 0,62 0,82 0,98 0,99 0,78 0,66 0,61 0,76 0 0,85 0 0

4.2.2 Discussion on Internal Validation The main goal of the measures shown in previous section is to analyze the accuracy of the crosscutting metrics. However, by means of this validation we have also extracted important conclusions about the used metrics. First of all, we have observed through an analytical comparison (which was confirmed by an analysis of the MobileMedia data) that some of our proposed metrics are generic enough to embrace existing codelevel metrics currently used in studies based on source-code analysis [8, 9, 12, 13]. Examples of these metrics are Sant`Anna’s Concern Diffusion over Components or Eaddy’s Degree of Scattering. A full pairwise comparison and discussion about the metrics is presented in [1]. In Table 10 we have shown the metrics which are more interesting for extracting conclusions on source elements (concerns). Fig. 3 shows three charts where Degree of scattering and Degree of crosscutting metrics are represented. Using these charts we observed that the metrics tend towards the same values for the same concerns.

148

J.M. Conejero et al.

One important conclusion that we have extracted from the analysis is the need for using the Degree of crosscutting metric (exclusive of our metrics suite). This metric is calculated using Crosscutpoints and NCrosscut metrics (see Section 3.4), and it is a special combination of scattering and tangling metrics.

a)

Video

Capture

Media

SMS

Music

Copy

Sorting

Favourites

Exception Handling

Label

Persistence

Photo

Album

Video

Capture

Media

SMS

Music

Degree of crosscutting

Copy

Sorting

Favourites

Exception Handling

Label

Persistence

Album

Photo

Degree of scattering

1 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 0

b)

Capture

Video

Media

Music

SMS

Copy

Favourites

Sorting

Exception…

Persistence

Label

Photo

Album

Eaddy's Degree of scattering

c) Fig. 3. Charts showing Degree of scattering (ours and Eaddy’s) and Degree of crosscutting

Fig. 4 shows the Degree of scattering and Degree of tangling metrics for releases 0 and 1. Note that in these releases, the Album concern presents the same value for Degree of scattering. However, the Degree of crosscutting metric for this concern is higher in release 1 than in release 0 (see Fig. 5). This is due to the tangling of the usecases where the Album concern is addressed (see in Fig. 4b). Accordingly, we observed that the Album concern is worse modularized in release 1 than in release 0 (there are other examples, such as Persistence or Photo). Note that this situation could not be discovered using only the Degree of scattering metric. Although the combination of the Degree of scattering and Degree of tangling metrics could help to disclose the problem, it would be a tedious task since the metrics do not provide information about which target elements are addressing each source element. Thus, the utilization

Early Crosscutting Metrics as Predictors of Software Instability

149

of Degree of crosscutting allows the detection of this problem just observing the values for this metric. The same analysis could be done for Eaddy’s metrics since they do not have a specific metric for crosscutting (see the Album concern in releases 0 and 1, in Fig. 3c). release0 release1

Degree of scattering

release 0 release 1

Add Album Delete Album Add Photo Delete Photo View Photo View Album Provide Label Store Data Remove Data Retrieve Data Edit Label Count Photo View Sorted Photos Set Favourite View Favourite Copy Photo Send Photo Receive Photo Music Control Access Media Play video Capture Media

Album Photo Label Persistence Exception Handling Sorting Favourites Copy SMS Music Media Video Capture

1 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 0

Degree of Tangling 1 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 0

a)

b)

Fig. 4. Degree of scattering and Degree of tangling for releases 0 and 1

Video

Media

SMS

Music

Copy

Sorting

Favourites

Exception…

Label

Persistence

Photo

Album

Capture

release0 release1

Degree of crosscutting

1 0.8 0.6 0.4 0.2 0

Fig. 5. Degree of crosscutting for releases 0 and 1

4.3 External Validation To date, there is no empirical study that investigates whether scattering and crosscutting negatively affect to software stability. In this section, our analysis shows that the concerns with a higher degree of scattering and crosscutting are addressed by more unstable usecases than concerns with lower degree of scattering and crosscutting. Stability is highly related to change management so that the more unstable a system is, the more complicated the change management becomes (decreasing quality of the system) [4]. Then, we can infer that crosscutting has also a negative effect on software quality. In this analysis we mainly focus on changes in the functionality of the system. We do not focus on changes performed to correct bugs or in maintainability tasks (we do not rule out this kind of changes in future analyses).

150

J.M. Conejero et al.

4.3.1 Relating Crosscutting Metrics with Stability In order to perform our empirical study, we have shown in Table 11 the usecases (rows) which change in the different releases (columns). A change in a usecase is due mainly to either the concerns which it addresses have evolved or it has been affected by the addition of a new concern to the system. In this table a 1 in a cell represents that in that release, the corresponding usecase has changed. As an example, in release 1 (r1) all the cells in the column present the value 1. This is due to the fact that error handling is added in this release, and this concern affects to all the usecases. An “a” in a cell represents that the usecase is added in that release. There are also some usecases which change their names in a release. These usecases are marked in the “Renaming” column, where the release which introduces the change in the name is shown (e.g. Add Photo usecase changes its name to Add Media in release 6). Finally, usecases with a number of changes higher than a threshold value (e.g. 2 in our analysis) are marked as unstable. Table 11. Changes in usecases in the different releases

Renaming

r6 r6

r6 r6

r6 r6 r6

Requirements Element Add Album Delete Album Add Photo [Media] Delete Photo [Media] View Photo View Album Provide Label Store Data Remove Data Retrieve Data Edit Label Count Photo [Media] View Sorted Photo Set Favourites View Favourites Copy Photo [Media] Send Photo [Media] Receive Photo [Media] Play Music Access Media Play Video Capture Media

r0 a a a a a a a a a a

r1 1 1 1 1 1 1 1 1 1 1

r2 0 0 0 0 1 1 0 0 0 0 a a a

Releases r3 r4 0 0 0 0 0 0 0 0 0 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 a 0 a 0 a

r5 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 a a

r6 0 0 1 1 1 1 0 0 0 0 0 1 1 0 0 1 1 1 a a

r7 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 a a

#Changes Unstable? 1 no 1 no 2 yes 2 yes 5 yes 4 yes 1 no 1 no 1 no 1 no 0 no 1 no 1 no 0 no 0 no 1 no 1 no 1 no 0 no 0 no 0 no 0 no

Once the changes affecting each usecase are known, the number of unstable usecases which realise each concern is calculated. Table 12 shows the unstable usecases (those with two changes or more) in the columns and the concerns in the rows. A cell with 1 represents that the usecase addresses the corresponding concern. The last column of the table shows the total number of unstable usecases contributing to each concern. We relate the number of unstable usecases for each concern with the degree of scattering and crosscutting for such concerns. In particular, Fig. 6 shows the linear

Early Crosscutting Metrics as Predictors of Software Instability

151

regression between the number of unstable usecases and the Degree of scattering and Degree of crosscutting metrics, respectively. We have used the least squares criteria to estimate the linear regression between the variables assessed so that the higher the degree of scattering or crosscutting for a concern, the more unstable usecases addressing such a concern. We can anticipate that usecases addressing scattered or crosscutting concerns are more prone to be unstable.

Concerns

Table 12. Number of unstable usecases addressing each concern

Album Photo Label Persistence Error Handling Sorting Favourites Copy SMS Music Media Video Capture

Usecases Add Media Delete Media View Photo View Album 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1

1

1

Unstable usecases 2 4 2 4 4 2 1 1 1 0 3 0 0

We have also related Eaddy’s Degree of Scattering metric with stability (Fig. 7). This figure complements the internal validation previously presented by showing consistency in the correlations of Fig. 6a) and Fig. 7 (they follow the same tendency). 4.3.2 Discussion on External Validation In this section we present some conclusions extracted from the analysis performed in previous sections. As we can see in Fig. 6 and Fig. 7 correlations follow a linear tendency so that the higher the degree of scattering or crosscutting for a concern, the more unstable usecases addressing this concern. This analysis allows the developer to decide which parts of the system are more unstable just observing the degree of scattering or crosscutting. Also, since the analysis is performed in requirements, the developer may anticipate important decisions about stability at this early stage of development, improving the later architecture or detailed design of the system. Fig. 6 and Fig. 7 also show the value for Pearson’s r (a common measure of the linear dependence between two variables) [21]. The values of r shown in Fig. 6a) and Fig. 6b) are 0.844 and 0.879 respectively. These values indicate that Degree of scattering and Degree of crosscutting are highly correlated with the number of unstable components. Using the critical values table for r [21], we calculated the probability after N measurements (in our case 13) that the two variables are not correlated. For Fig. 6a) and Fig. 6b), the value obtained for this probability is 0.1%. Accordingly, the probability that these variables are correlated is 99.9%. For Fig. 7, we obtained that r is 0.788. Analogously, Eaddy’s Degree of Scattering is also linearly correlated with

152

J.M. Conejero et al.

6

5

4

Unstable usecases

Unstable usecases

5 Photo r = 0,844 3 2

Photo

4 3 2 r = 0,879 1

1 0

0 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 Degree of Scattering

0

0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 Degree of Crosscutting

a)

1

b)

Fig. 6. Correlation between Degree of scattering and Degree of crosscutting and stability 5 Unstable usecases

Photo 4 3 2 r = 0,788 1 0 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 Eaddy's Degree of Scattering

Fig. 7. Correlation between Eaddy’s Degree of Scattering and stability

the number of unstable components. In particular, the probability that the variables assessed in Fig. 7 are not correlated is only 0.8%. We observed that, in general, we obtained a better correlation for the Degree of crosscutting with stability than for Degree of scattering with stability. After analyzing the data, we observed that the correlations between Degree of scattering metrics (both ours and Eaddy’s) and stability were much influenced by those concerns either without scattering or completely scattered. As an example, we can see in Fig. 6a) that there is a point with a Degree of scattering of almost 1 while most of the points present a Degree of scattering lower than 0.4. This situation is even more evident in Fig. 7 where the correlation coefficient obtained is lower than for the other correlations. The reason is the aforementioned commented: the difference between the values obtained for this metric in cases without scattering and the rest of cases. This metric obtained high values (greater than 0.5) for almost all the concerns assessed. However, when a concern does not present scattering the result of the metric is 0, highly influencing the correlation. Finally, we concluded that Degree of crosscutting presents a better correlation with stability since this metric somehow takes into account not only scattering but also tangling. This conclusion supports the need for having a specific metric for assessing crosscutting.

Early Crosscutting Metrics as Predictors of Software Instability

153

We have also annotated in all the correlations a point called Photo. These points are the most digressed from the linear regression in all the figures. We observed that although this concern (Photo) presents values for the scattering and crosscutting metrics not very high, the number of unstable usecases was high. After analyzing this situation, we observed that this concern presents a high degree of scattering and crosscutting in the six first releases. After release 5, a new concern is added (Media) which is responsible for addressing the actions common to photo, music and video, and carrying out many actions previously assigned to the Photo concern. This is why Degree of scattering and crosscutting for Photo drastically decrease in releases 6 and 7. It highly influences to the average of the metrics and this is the reason why although having non-relatively high values for the metrics; the number of unstable usecases remains high.

5 Related Works In [19], Sant’Anna et al. introduce different metrics (summarized in Table 6), namely Concern Diffusion over Components (CDC), Concern Diffusion over Operations and Concern Diffusion over Lines of Code. These metrics allow the developer to assess the scattering of a concern using different levels of granularity. The authors also define the Lack of Concern-based Cohesion to assess the tangling in the system. However, these metrics are mainly defined to assess modularity using specific deployment artefacts so that they are focused on specific abstraction levels (design or programming). In [18], the same authors adapted the metrics to the architectural level and introduced new metrics. However, the metrics still keep tied to specific deployment artefacts and they are not generic enough to be used at any abstraction level. In [9], the metrics are used for analyzing stability in product lines. However, the work is very tied to the programming level, relegating the benefits of the metrics to the latest phases of development. In [7], Ducasse et al. introduce four concern measures: Size, Touch, Spread and Focus (see Table 6). Again, these metrics are tied to the implementation. Wong et al. introduce in [22] three concern metrics called Disparity, Concentration and Dedication (Table 6). Eaddy et al., use an adaptation of Concentration and Dedication metrics for defining two new concern metrics [8]: Degree of Scattering (DOS) and Degree of Tangling (DOT). Whilst DOS is defined as the variance of the Concentration of a concern over all program elements with respect to the worst case, DOT is defined as the variance of the Dedication of a component for all the concern with respect to the worst case. Both works are focused on assessing modularity at programming level as well. In [16] Lopez-Herrejon and Apel define two concern metrics: Number of Features (NOF) and Feature Crosscutting Degree (FCD), measuring number of features and number of classes or components crosscut by a particular concern respectively. Ceccato and Tonella also introduce a metric called Crosscutting Degree of an Aspect (CDA) [5] which counts the number of modules affected by an aspect. These metrics are defined to assess attributes of an aspect-oriented implementation. They could not be used to anticipate decisions in not aspect-oriented systems.

154

J.M. Conejero et al.

In [20], the authors use a tool to analyze change impact in Java applications. This tool allows the classification of changes so that they detect the more failure inducing changes. However, like most of the aforementioned approaches, this work is focused on programming level when the system is already designed.

6 Conclusions and Future Work In this paper, we proposed a concern-oriented metrics suite to complement traditional software metrics. The metrics proposed are based on the crosscutting pattern presented in our previous work which establishes a dependency between two generic domains, source and target, based on traceability relations. This metric suite allows the developer to perform a modularity analysis, identifying the crosscutting concerns in a system but also quantifying the degree of crosscutting of each concern. The metrics are generic and they are not tied to a specific deployment artifact. Then, they may be used at different abstraction levels, allowing developers to assess modularity and infer quality properties at early stages of development. Even, with the transition to model-driven software engineering gaining momentum, the assessment of abstract models (usecases or components models) becomes more important. Through the internal validation, we observed not only that our metrics are consistent with other metrics but also that they complement other metrics since they are not defined in terms of any deployment artefact. Moreover, we showed the need for introducing a specific metric for crosscutting. The external validation was focused on demonstrating the utility of the metrics for other software quality attributes. In particular, we show how the Degree of scattering and Degree of crosscutting metrics present a linear correlation with stability and that the Degree of crosscutting metric is better correlated with stability than Degree of scattering. As future work, we plan to perform several empirical studies. In a first study we expect to compare the results obtained by our metrics at requirements level with those obtained at different abstraction levels (e.g., architectural level, design or implementation). Also, the application of the metrics at source-code level would allow us to compare our results with the obtained by other studies were authors analyze instability at this level (e.g., [9, 13]). By this analysis we could test different hypotheses, such as, whether similar properties of crosscutting concerns are found to be indicators of instabilities or what probabilities of early crosscutting measurements lead to false warnings (i.e. false positives or negatives) at source-code level. In these analyses we may also utilize an aspect-oriented version of the system assessed to check the improvements obtained by the utilization of different paradigms. We also plan to apply the metrics in several case studies (projects) demonstrating that the results obtained are not just coincidental.

Acknowledgements This work has been supported in part by the European Commission grant IST-2004349: European Network of Excellence on AOSD (AOSD-Europe) and by MEC under contract: TIN2008-02985. Eduardo is supported by CAPES, Brazil.

Early Crosscutting Metrics as Predictors of Software Instability

155

References 1. Analysis of modularity in the MobileMedia system (2008), http://quercusseg.unex.es/chemacm/research/ analysisofcrosscutting 2. Baniassad, E., Clements, P., Araújo, J., Moreira, A., Rashid, A., Tekinerdogan, B.: Discovering Early Aspects. IEEE Software 23(1), 61–70 (2006) 3. van den Berg, K., Conejero, J., Hernández, J.: Analysis of Crosscutting in Early Software Development Phases based on Traceability. In: Rashid, A., Aksit, M. (eds.) Transactions on AOSD III. LNCS, vol. 4620, pp. 73–104. Springer, Heidelberg (2007) 4. van den Berg, K.: Change Impact Analysis of Crosscutting in Software Architectural Design. In: Workshop on Architecture-Centric Evolution at 20th ECOOP, Nantes (2006) 5. Ceccato, M., Tonella, P.: Measuring the Effects of Software Aspectization. In: Proceedings of the 1st Workshop on Aspect Reverse Engineering, Delft University of Technology, the Netherlands (2004) 6. Conejero, J., Hernandez, J., Jurado, E., van den Berg, K.: Crosscutting, what is and what is not? A Formal definition based on a Crosscutting Pattern. Technical Report TR28_07, University of Extremadura (2007) 7. Ducasse, S., Girba, T., Kuhn, A.: Distribution Map. In: Proc. of the Int’l Conference on Software Maintenance (ICSM), Philadelphia, USA (2006) 8. Eaddy, M., Zimmermann, T., Sherwood, K., Garg, V., Murphy, G., Nagappan, N., Aho, A.: Do Crosscutting Concerns Cause Defects? IEEE Transactions on Software Engineering 34(4), 497–515 (2008) 9. Figueiredo, E., Cacho, N., Sant’Anna, C., Monteiro, M., Kulesza, U., Garcia, A., Soares, S., Ferrari, F., Khan, S., Filho, F., Dantas, F.: Evolving Software Product Lines with Aspects: An Empirical Study on Design Stability. In: Proceedings of the 30th International Conference on Software Engineering (ICSE), Leipzig, Germany (2008) 10. Figueiredo, E., Sant’Anna, C., Garcia, A., Bartolomei, T., Cazzola, W., Marchetto, A.: On the Maintainability of Aspect-Oriented Software: A Concern-Oriented Measurement Framework. In: Proceedings of CSMR 2008, pp. 183–192 (2008) 11. Garcia, A., Lucena, C.: Taming Heterogeneous Agent Architectures. Commun. ACM 51(5), 75–81 (2008) 12. Garcia, A., Sant’Anna, C., Figueiredo, E., Kulesza, U., Lucena, C., Staa, A.: Modularizing design patterns with aspects: A quantitative study. In: Rashid, A., Aksit, M. (eds.) Transactions on Aspect-Oriented Software Development I. LNCS, vol. 3880, pp. 36–74. Springer, Heidelberg (2006) 13. Greenwood, P., Bartolomei, T., Figueiredo, E., Dosea, M., Garcia, A., Cacho, N., Sant’Anna, C., Soares, S., Borba, P., Kulesza, U., Rashid, A.: On the Impact of Aspectual Decompositions on Design Stability: An Empirical Study. In: Ernst, E. (ed.) ECOOP 2007. LNCS, vol. 4609, pp. 176–200. Springer, Heidelberg (2007) 14. Kelly, D.: A Study of Design Characteristics in Evolving Software Using Stability as a Criterion. IEEE Trans. Software Eng. 32(5), 315–329 (2006) 15. Kiczales, G., Lamping, J., Mendhekar, A., Maeda, C., Lopes, C.V., Loingtier, J.-M., Irwin, J.: Aspect-Oriented Programming. In: Aksit, M., Matsuoka, S. (eds.) ECOOP 1997. LNCS, vol. 1241, pp. 220–242. Springer, Heidelberg (1997) 16. Lopez-Herrejon, R., Apel, S.: Measuring and Characterizing Crosscutting in Aspect-Based Programs: Basic Metrics and Case Studies. In: Proc. of the Int’l Conference on Fundamental Approaches to Software Engineering (2007)

156

J.M. Conejero et al.

17. Masuhara, H., Kiczales, G.: Modeling Crosscutting in Aspect-Oriented Mechanisms. In: Cardelli, L. (ed.) ECOOP 2003. LNCS, vol. 2743, Springer, Heidelberg (2003) 18. Sant’Anna, C., Figueiredo, E., Garcia, A., Lucena, C.J.P.: On the modularity of software architectures: A concern-driven measurement framework. In: Oquendo, F. (ed.) ECSA 2007. LNCS, vol. 4758, pp. 207–224. Springer, Heidelberg (2007) 19. Sant’Anna, C., Garcia, A., Chavez, C., Lucena, C., von Staa, A.: On the Reuse and Maintenance of Aspect-Oriented Software: an Assessment Framework. In: Proc. of the Brazilian Symposium on Software Engineering (SBES), Manaus, Brazil (2003) 20. Stoerzer, M., Ryder, B.G., Ren, X., Tip, F.: Finding Failure-Inducing Changes in Java Programs using Change Classification. In: Proc. of 14th International Symposium on Foundations of Software Engineering, Portland, USA, pp. 57–68. ACM, New York (2006) 21. Taylor, J.: An Introduction to Error Analysis. The Study of Uncertainties in Physical Measurements, 2nd edn. University Science Books (1997); ISBN: 0-935702-75-X 22. Wong, W., Gokhale, S., Horgan, J.: Quantifying the Closeness between Program Components and Features. Journal of Systems and Software (2000)

Extensibility in Model-Based Business Process Engines Mario S´ anchez1,2, , Camilo Jim´enez1 , Jorge Villalobos1 , and Dirk Deridder2, 1

2

Universidad de los Andes, Bogot´ a, Colombia {mar-san1,camil-ji,jvillalo}@uniandes.edu.co System and Software Engineering Lab, Vrije Universiteit Brussels, Belgium [email protected]

Abstract. An organization’s ability to embrace change, greatly depends on systems that support their operation. Speciﬁcally, process engines might facilitate or hinder changes, depending on their ﬂexibility, their extensibility and the changes required: current workﬂow engine characteristics create diﬃculties in organizations that need to incorporate some types of modiﬁcations. In this paper we present Cumbia, an extensible MDE platform to support the development of ﬂexible and extensible process engines. In a Cumbia process, models represent participating concerns (control, resources, etc.), which are described with concern-speciﬁc languages. Cumbia models are executed in a coordinated way, using extensible engines specialized for each concern. Keywords: Business Process Management, Workﬂows, Workﬂow Extensibility, Model Driven Engineering.

1

Introduction

Nowadays, many organizations operate in agile environments, where they have to deal with frequent changes caused by a great number of internal and external forces. Thus, in order to be competitive, organizations need to be capable of rapidly adapting to new conditions. This can be seen in rapid evolutions of business processes within an organization. Since they are part of the core assets that make the organization competitive, they need to be updated frequently to remain eﬃcient and eﬀective. Most successful organizations have strong dependencies towards information systems that support their operation (databases, portals, etc.) and change has therefore an important impact on those systems. Furthermore, the speed of change of an entire organization can be limited by the capacity of its systems to accommodate new requirements. Consequently, organizations have adopted a

Supported by the VLIR funded CARAMELOS project http://ssel.vub.ac.be/ caramelos/ and by Colciencias. Funded by the Interuniversity Attraction Poles Programme - Belgian State Belgian Science Policy.

M. Oriol and B. Meyer (Eds.): TOOLS EUROPE 2009, LNBIP 33, pp. 157–174, 2009. c Springer-Verlag Berlin Heidelberg 2009

158

M. S´ anchez et al.

number of technologies that oﬀer strategies to address change. The most common among these technologies are Service Oriented Architectures and Enterprise Service Buses, Rules’ Engines, and Business Process Managers. However, each technology also presents limitations and therefore it is impossible to have an application stack that can support every kind of future change. In this paper we focus on the limitations, posed by current business process engines and languages, and we propose a solution to overcome some of these limitations. There are multiple factors that limit the capacity of engines and languages to accommodate changes. One factor is that low-level knowledge about tools and languages is required, and therefore every change has to include technical and business parties. Another factor is that the most used languages are big, general, and monolithic. This combination makes it diﬃcult to introduce modiﬁcations or extensions to the languages. Furthermore, these languages are tightly coupled to the engines that run them. As a result, even small extensions to the languages require extensions to the engines, which can be impossible to introduce when proprietary engines are used. The ﬁnal factor is that most workﬂow languages use a single artifact per process, where all the characteristics of a business process have to be deﬁned using the same language. This also implies that for complex processes this artifact can become very big and diﬃcult to manage. In this paper we present Cumbia: an extensible platform which addresses the problems of ﬂexibility and extensibility in business process engines and languages. The main goal of Cumbia, is to oﬀer a foundation for the development of new workﬂow engines, compatible with custom languages or some existing workﬂow languages such as BPEL or BPMN. Our proposal is based on three main ideas [1]: the ﬁrst one is that elements in a process can be separated in concerns, such as control, resources and data, and that extensible concern-speciﬁc languages can be used to describe those elements; the second idea is that an extensible platform can be used to develop extensible concern-speciﬁc engines; the third idea is that the platform can coordinate the execution of the diﬀerent concerns, using a common composition language. In order to illustrate our proposal, this paper introduces three diﬀerent types of extensions, applicable to a process taken from the banking context (see section 4); afterwards, the paper presents a complete case study of extensibility in a workﬂow based application. The rest of the paper presents some strategies, used for extensibility and ﬂexibility in workﬂow engines and languages (section 2), and the fundamental elements of Cumbia (section 3).

2

Strategies for Workflow Flexibility and Extensibility

Literally, there are hundreds of diﬀerent workﬂow engines and languages, and many of them include strategies to handle ﬂexibility and extensibility requirements. In this section we analyze some and identify their disadvantages. Please note that extensibility refers to the capacity of the software to accommodate new functionalities, while ﬂexibility refers to the capacity of the software to support changes to existing functionalities. Moreover, we do not intend to address the problem of ﬂexibility at schema or instance level.

Extensibility in Model-Based Business Process Engines

159

Perhaps the most common extensibility strategy is to oﬀer extension points: explicit elements modiﬁable according to some rules. Some examples of such a strategy can be found in BPEL, which oﬀers the tag <extensionActivity> and supports extra attributes and elements [2]. Similar examples are artifacts in BPMN [3], or Vendor or User specific Extensions of XPDL [4]. Although this strategy can be successfully used in many situations, it also has some important drawbacks. The ﬁrst one is the support of a somewhat limited set of extensions. Furthermore, if extensions were complex, then artifacts could become cluttered and diﬃcult to manage. Finally, extensions points also have an impact on the engines, which in general are not very easy to extend. These drawbacks can be seen in BPEL4PEOPLE (B4P) [5], one of the best known BPEL extensions, which oﬀers a very expressive way to introduce human interaction to BPEL processes. B4P makes use of the extension points of BPEL to introduce complex people-related information and new types of activities. This makes BPEL ﬁles longer and more complex. Furthermore, the extension of existing BPEL engines to support B4P, is not trivial [6]. A diﬀerent strategy to facilitate ﬂexibility and extensibility in workﬂow engines and languages, involves the separation of concerns and the usage of aspects. This strategy oﬀers several advantages over simple extension points. One of them is an increase in the number of available extension points. Another one is related to the division of process descriptions among several artifacts, reducing complexity and making it easier to manage and maintain the processes. This division also opens the possibility to include new concerns, which can introduce new functionalities. This strategy has been applied in various tools, albeit with some variations. Padus [7] is an aspect-oriented workﬂow language for BPEL, which uses a static weaver producing processes that can be run on standard BPEL engines. AO4BPEL [8] is also BPEL-based, but it oﬀers dynamic weaving. The main downside of this is that processes have to be run in AO4BPEL-capable engines. A further disadvantage of these two approaches is that advices have to be speciﬁed using BPEL, and thereby the same language and engine are used for every concern. Finally, AMFIBIA [9] is a model-based tool, which deﬁnes several workﬂow concerns specializing a set of common concepts, oﬀering a way to coordinate their execution. A radically diﬀerent strategy to introduce ﬂexibility and extensibility in workﬂows, is the usage of domain speciﬁc workﬂow languages (DSWL). Since they are applicable in a reduced set of contexts, they can be designed to include the most adequate extension points for each context. Examples of DSWLs can be found in the context of scientiﬁc applications [10] and ubiquitous computing [11]. The main disadvantage of this approach, is that DSWLs are usually expensive to design, support and maintain. However, the technologies included in the Windows Workﬂow Foundation and the Visual Studio DSL Tools from Microsoft, are now a real possibility to reduce the costs associated with the implementation of DSWL. Using diﬀerent techniques, general purpose engines are used in several contexts to execute workﬂows deﬁned with diﬀerent languages (usually high-level

160

M. S´ anchez et al.

languages). For instance, processes deﬁned using BPMN are frequently executed using BPEL engines. This strategy facilitates ﬂexibility and extensibility, because it separates a language from the engine implementation. However, if the original language is extended or modiﬁed, it becomes necessary to modify the tools in charge of performing the transformation to the executable language. This strategy also has to deal with limitations imposed by the executable platform; if this platform is not very expressive, high-level languages are limited in their expressiveness as well. Currently, JBoss has an active project using this strategy to develop a Process Virtual Machine [12], which aims to oﬀer a basic execution platform for processes deﬁned using any language. Nevertheless, from our perspective this approach has limitations, because it only focuses on the control concern. The Cumbia platform combines ideas from the aforementioned strategies. In the ﬁrst place, separation of concerns is a fundamental technique in Cumbia, combined with concern-speciﬁc languages. Furthermore, each language is designed to be extensible by using the capabilities oﬀered by the platform. Finally, the execution is managed by concern-speciﬁc engines, built using functionalities oﬀered by Cumbia. The ensuing section presents details on the platform and explains its relationships with concern-speciﬁc engines.

3

The Cumbia Platform

The Cumbia platform is our proposal to support the deﬁnition, execution and evolution of business processes in a ﬂexible manner. The techniques used to achieve these goals (separation of concerns, the development of concern-speciﬁc languages, model weaving, and coordinated execution) introduce some diﬃculties the platform needs to resolve. These diﬃculties are mainly caused by diﬀerences between the concerns, which create diﬀerent extension and coordination requirements. Cumbia copes with these problems by providing a base platform that oﬀers powerful mechanisms for interaction, coordination, and extension. The current implementation of this platform is Java based: it can be used in standalone applications or in more complex systems such as JEE containers. Metamodels are a central concept in Cumbia: for each concern that may appear in a process, a metamodel has to be deﬁned. These metamodels are deﬁned with a special editor, and they deﬁne which elements make part of the concern and how they relate to each other. Figure 1a shows a screenshot of the editor while it is used to edit a metamodel. The most important feature of Cumbia Metamodels, is being based in something that we have called open objects. Open objects are formed by an entity, a state machine associated with the entity, and a set of actions. An entity is an object with attributes and methods, which provides an attribute-based state to the open object, while its methods implement part of its behavior. The state machine materializes an abstraction of the entity’s life-cycle, and it allows other elements to know the state of the open

Extensibility in Model-Based Business Process Engines

161

Fig. 1. Screenshots of the Cumbia editor while editing a) a metamodel, and b) an open object

object and react to its changes. Finally, actions are pieces of behavior, associated with transitions of the state machine. When a transition takes place, its actions are executed in a synchronized way. Figure 1b also shows the editor, but in this case it is used to edit the structure of an open object. The usage of open objects in Cumbia metamodels has two important consequences. In the ﬁrst place, it is possible to deﬁne a weaving mechanism capable of coordinating any concern, because they share the same base element. Secondly, the extension mechanisms oﬀered by open objects, are applicable in every concern. In order to use Cumbia to model and execute business processes, after deﬁning the metamodels, it is necessary to follow some extra steps. The ﬁrst one is the deﬁnition of concrete languages to describe the concerns. By default, Cumbia supports a trivial xml-based syntax, derived from the structure of the metamodel. However, if more complex languages are desired (e.g. graphical), it is necessary to implement ad-hoc interpreters for them. The second step is the development of an engine for the metamodel, using the common functionalities oﬀered by the platform. The development of these engines only requires the implementation of concern-speciﬁc functionalities, such as the interaction with existing and external applications, and the management and persistence of special data structures. 3.1

The Control Concern and XPM

The control concern is critical in the majority of workﬂow applications. This concern speciﬁes the tasks that have to be executed as part of the process, and the conditions that may alter their order of execution. There are many languages available to describe the elements in this concern, such as BPEL, and BPMN. However these languages are composed by a large number of elements, including several from other concerns.

162

M. S´ anchez et al.

In order to facilitate the explanations in the rest of the paper, this section introduces XPM (eXtensible Process Metamodel); a simple metamodel to describe the control concern in business processes. This metamodel is introduced using the sample process used in section 4 to motivate some extensibility requirements.

Fig. 2. Control concern in the sample process

The process shown in ﬁgure 2 is taken from the context of ﬁnancial services, and it deﬁnes the steps to study and approve a credit request. The process starts when a customer applies for a credit. Then, three tasks are performed in parallel: the request is evaluated by an account manager; the customer’s credit rating is consulted in an external system using a web-service; and the credit history of the customer is studied by a team of junior analysts. Finally, the senior account manager approves or rejects the request based on the previous results. Figure 2 shows the sample process using a graphical version of XPM; however, it is speciﬁed with an xml-based textual language. Figure 3a shows the structure of the XPM metamodel. The elements in it are visible in the sample process. The top level element is called process and it contains all the other elements. This particular process is composed by four activities and a multiactivity connected through ports and dataflows. Each activity has a distinct workspace executing a speciﬁc atomic task. Activities enclose workspaces and handle their synchronization and data management issues; Multiactivities, like the one used for Study Credit History, execute in parallel the same workspace a number of times determinable during runtime. An important characteristic of XPM is the control ﬂow being determined by the data ﬂow. This occurs because activities can only be executed when they receive data to be processed. Each ProcessTask (an activity, multiactivity or process, see ﬁgure 3a) has at least one entry port, expecting a certain set of data. Ports are connected by dataﬂows, moving data between ports. When an entry port of a ProcessTask receives all the data expected, it can be executed. If the task is an activity or multiactivity, the data is given to the workspaces,

Extensibility in Model-Based Business Process Engines

163

Fig. 3. a) Metamodel of XPM and b) simpliﬁed workspaces state machine

to be executed and to produce some output data. The output is then placed in one of the exit ports of the task, and the execution of the process continues. In a process, entry ports provide the initial data required, and exit ports publish the ﬁnal results of the process execution. XPM does not have gateways such as the ones found in BPMN; instead, gateways’ functions can be fulﬁlled by activities and workspaces. Every element in XPM is an open object. Thus, each element in XPM has an entity, a state machine, and some actions. Consequently, they can interact by using the two mechanisms provided by the open objects: event passing and method calling. In a state machine speciﬁcation, each transition has an associated source event. When the open object receives that event, that particular transition takes place. This mechanism does not only serve to synchronize open objects, but also to keep the state machine consistent with the internal-state of the entity. This is because each time the state is modiﬁed, an event is generated changing the state in the automaton. Additionally, other events are generated when transitions take place, and it is therefore possible to synchronize open objects using state changes. The other interaction mechanism, method calling, is synchronous, and is used to invoke methods oﬀered by the entities. These entities can receive method calls from two sources: There could be calls coming from external sources, such as user interactions and other related applications. Alternatively, actions associated with transitions can invoke methods of other entities. For this reason, they play a central role in model coordination. To understand the interaction between open objects, consider the simpliﬁed state machine of a workspace shown in ﬁgure 3b1 : in the state machine, events are originated from two sources: [ACT] refers to the activity that encloses the workspace, and [ME] refers to the workspace itself. Thus, the transition from Waiting to Executing is taken when the activity generates the event enterActive, and the transition from Executing to Delivering Results takes place when the workspace ﬁnishes its execution and generates the event executionEnded. Additionally, the transition starting in Waiting has an associated action called ExecuteWorkspace, which invokes a method of the workspace entity to start its execution. 1

In the ﬁgure we removed some intermediate states and actions, and it does not include source events for every transition.

164

4

M. S´ anchez et al.

Extensibility in Cumbia

In some occasions, the changes to the requirements of an existing business process are not supported by the language or engine in use. In those cases it is necessary to somehow extend the languages and engines, for instance using the mechanisms discussed in section 2. It is impossible to foresee every future extension required. Nevertheless, we have identiﬁed three types of changes that are likely to happen, and we have incorporated in Cumbia, mechanisms to support them. The three types of changes are the following: extensions to the behavior of an element, inclusion of additional elements, and the inclusion of additional concerns. It is important to note that these three types of changes are only applicable to future business process instances: they do not aﬀect processes already in execution. 4.1

Extensibility in a Concern: Extended Element Behavior

The ﬁrst extension to the sample process, implements the new requirement of keeping a log of the process results, by storing the ﬁnal answer about the credit request. In order to support this, it is necessary to extend the behavior of the process’ exit-port, achievable by adding an extra action in one of the port’s transitions. This is the simpler extension mechanism oﬀered by open objects, and it is applicable at design time or at run time. To apply this kind of extension, it is only necessary to know in which transition the new action has to be inserted. The following is the snippet of code that deﬁnes a new extension to a Port, called ProcessExitPort. <extended-type name="ProcessExitPort" extends="Port"> <extension transitionName="Receive">

This extension is local to the process, and consequently does not aﬀect any other processes built with XPM. The code speciﬁes a ProcessExitPort to have an action called Store Final Result, associated with the transition Receive. The new action is implemented in the class cumbia.actions.StoreData. When the extension is applied during run time, the open objects API can be used for adding the desired action instead of the code. 4.2

Extensibility in a Concern: New Element

The second extension to the sample process presented in section 3, indicates that the query of the credit rating of the customer has to be done in a secure way, using the WS-Security standard [13]. This standard establishes mechanisms to cipher and sign messages before sending them, and mechanisms to decipher

Extensibility in Model-Based Business Process Engines

165

and verify the signature of responses. Thus, the behavior of the workspace that consumes the web service has to be extended with the ws-security behavior. In order to support this extension, we are going to use the mechanism of Cumbia that allows the addition of elements. This can be done, either by creating new elements from scratch, or by extending existing elements. In this case, we will create a new element called WSSecureWorkspace, which extends the behavior of workspaces by extending its entity and state machine. The introduction of new elements in a metamodel can be troublesome because of incompatibilities with existing elements. However, the event-based interaction mechanism used by open objects, reduces coupling, and therefore it reduces the likelihood of having problems. Nevertheless, we are currently not working on discovering these incompatibilities [14]. This can be a very diﬃcult problem, and is closely related with the ﬁeld of behavioral analysis of component interfaces [15, 16].

Fig. 4. Extended workspace and state machine

Figure 4 shows the new hierarchy of workspaces and the state machine of the new, secure workspace. In this hierarchy, the element WSWorkspace is capable of consuming web-services, and WSSecureWorkspace implements the extension required in the example. Since WSSecureWorkspace extends WSWorkspace, its entity already includes all the necessary code to consume web-services, and so the entity is only modiﬁed to include security related code, which is encapsulated in the methods sign( ), cipher( ), verifySignature( ), and decipher( ). This entity also adds an attribute to hold the key used for the encryption. The state machine of the workspace is also modiﬁed to support extra requirements. Strictly speaking, this extension is unnecessary, because all the security code can be added to the method execute( ) of the workspace. However, adding these extra states makes it possible to observe the execution of the new workspace in more detail. 4.3

Extensibility with Additional Concerns

In some cases the best strategy is not to modify an existing concern, but to add an entirely new concern. One reason for this is that most extensions can be integrated in existing languages and metamodels, but over time, such an additional complexity can have a negative impact on their usability. If a single model

166

M. S´ anchez et al.

includes many unrelated elements, its complexity could become unmanageable. On the other hand, the identiﬁcation of concerns and their representation with metamodels opens the possibility to have, in a single process, several models being much focused and deﬁnable using concern-speciﬁc languages. Cumbia facilitates the creation of new concerns, by providing a base to describe the metamodels and execute its models in a coordinated way. Firstly, open objects provide services to handle communication, coordination and concurrency, required in most concerns. Furthermore, Cumbia oﬀers a language to describe relationships between elements of diﬀerent concerns. This language is concern-agnostic, meaning that it only describes relationships between open objects without caring for the concern they belong to. To illustrate the addition of a concern to an existing application, we add the time concern to the sample process. This new concern describes conditions and restrictions over another concern, depending on dates and durations. Two sample time restrictions that could be added to the process are the following: R1. The activity called Evaluate Request cannot last longer than 1 week; if after one week it is not ﬁnished, a standard positive review of the request is automatically generated. R2. During the execution of the process, the requester should receive a daily email with the status of the process. In order to model these restrictions, and the rest of the elements in the time concern, we deﬁned a metamodel called XTM (see ﬁgure 5a). In this metamodel TimeRestriction is the central element: it has the responsibility of abstracting and coordinating the time restrictions. Similarly to workspaces in XPM, specialized to perform speciﬁc atomic tasks, TimeRestrictions are specialized as necessary for each situation. In the proposed example, two extensions are required. In order to be executed, a TimeRestriction requires a reference calendar and timers, keeping track of time and generating events when necessary. TimeRestrictions also have associated task sets, executed when certain conditions are met. Finally, there are also EntryPoints, which are XTM acting as proxies for elements of XPM or any other metamodel. Figure 5b shows the state machine of the TimeRestriction used to model R1. This state machine includes actions to control a timer, associated with the restriction, and to start the execution of a task set, if the condition is not met. The events expected by the state machine come from two sources: they can be

Fig. 5. a) Simpliﬁed XTM’s metamodel and b) state machine of a restriction

Extensibility in Model-Based Business Process Engines

167

events produced by the timer (timeOut), or events produced by XPM’s elements, and received through the EntryPoint (start and end). Finally, it should be noted that the structure of this state machines is what gives the semantics to the TimeRestriction. If the events end and timeOut were swapped, the logic of the TimeRestriction would be inversed.

Fig. 6. Weaving of the Activity and the EntryPoint

The composition and coordination between a time restriction and an XPM process, is described with a language that speciﬁes relations between open objects based models. This language states how events should be propagated between the models. Further details of this language and its implementation can be found in [1]. In the case of the restriction R1, the composition is implemented with two actions and installed in the state machine of the activity Evaluate Request, and in the occurring transitions when the activity starts and ends its execution. Those actions are responsible for notifying the EntryPoint of the start and end of the activity (see ﬁgure 6). Since this composition mechanism is external and non-invasive, it is unnecessary to modify the description of models in order to compose them. This generates several beneﬁts, like the capacity to easily introduce new concerns and increase reusability. Additionally, relationships can be created and removed at run time, creating an extra degree of ﬂexibility.

5

Extending a Workflow Engine

To illustrate the extensibility properties of our approach in a case study, we now explain how a specialized workﬂow engine was extended. PaperXpress is an engine built using the Cumbia platform to support the deﬁnition and execution of processes for writing and reviewing research articles. PaperXpress allows an author to deﬁne and perform an ordered set of activities, like writing and reviewing, and relating them with sections in an article. The author starts using PaperXpress, by ﬁrst deﬁning the structure of the paper he wants to write. This is achieved by using the application shown in ﬁgure 7. In this image, the author already deﬁned a structure composed by seven sections and two subsections.

168

M. S´ anchez et al.

Fig. 7. Example of an article structure deﬁnition using PaperXpress

Fig. 8. Example of a process deﬁnition for writing and reviewing an article using PaperXpress: only a part of the process is visible

After the article structure is deﬁned, the author can plan writing and reviewing activities. For this purpose, he can use the application shown in ﬁgure 8, which oﬀers an ad-hoc language with very simple control structures. In the shown sample process, the author deﬁnes a process with steps to write the abstract, write the main sections, review the sections, and ﬁnally write the conclusions. There are two concerns in PaperXpress. Firstly, the control concern has the activities that are executed as part of the process. This concern is very simple and it only supports basic control-ﬂow patterns [17], since the context of PaperXpress does not require complex control structures. Such simplicity can be seen in the

Extensibility in Model-Based Business Process Engines

169

Fig. 9. Metamodels of both a) the control concern and b) the article concern in PaperXpress

size of its metamodel that includes only 3 major constructs: Process, Activity and Flow (see ﬁgure 9a). A Process element always has an initial and a ﬁnal activity. Only when the ﬁnal activity is ﬁnished, the Process can end its execution. On the other hand, the article concern represents the elements that structure a complete research article. Its metamodel (see ﬁgure 9b) includes ﬁne grained elements like paragraphs and images. Every element in these metamodels is an open object. There are several relations between the control and the article concern in PaperXpress. One example of those relations, is a rule specifying that article sections can only be changed when the related activity is active. In this way, an activity unlocks its section as soon as it starts its execution, and locks it again when it ﬁnishes. The composition between these elements is implemented with two actions installed in the state machine of the activity. In particular, these actions are associated with the transitions occurring when the activity starts and ends its execution. These actions notify the corresponding sections to be unlocked or locked. Another example can be found when the process ends. At that moment, PaperXpress automatically generates a pdf ﬁle with the contents of the article using a predeﬁned format. This composition is implemented with actions installed in the process state machine. In this case, an action notifying the article element to generate the pdf ﬁle is automatically installed in the transition taken when the process ends its execution. 5.1

Extending PaperXpress: Storing the Article Contents in a Remote Repository

The ﬁrst extension to PaperXpress includes a new persistence requirement: the contents of article sections must be kept in a remote repository and they must be checked out only when the activity planned to write or review them is active. Similarly, modiﬁcations to these contents must be saved in the repository, when the activity ﬁnishes its execution. To support this requirement, it was necessary to extend the behavior of the section. More speciﬁcally, two actions were implemented and installed in its state machine. A ﬁrst action, checkOut, was installed in the transition taken place when the section is unlocked (see sections state machine in ﬁgure 9). This action checks out the contents from a speciﬁc repository and gives them to the

170

M. S´ anchez et al.

Fig. 10. New actions in the a) state machine of the Activity and in the b) state machine of a Section

entity. A second action, save, was installed in the transition occurring when the section is locked. This action retrieves the contents from the entity and saves them in the remote repository. After installing the new actions, the coordination between the activity and its section changes as follows (see ﬁgure 10). As soon as an activity is activated, its state machine takes the transition from the state Init to the state Active. This executes the unlock action notifying the section that must be unlocked. At that moment, the state machine of the Section takes the transition to the Unlocked state, executing the checkout action to retrieve the contents from the remote repository. The contents remain available and are changeable until the activity is ﬁnished. At that time, the transition to the Finalizing state is taken. The action lock is then executed, notifying the section that it must be locked again. Finally, the state machine of the section takes the transition to the Locked state, executing the save action. Implementing this requirement implied two simple steps. First, the actions were implemented in separate Java classes. The classes include the logic necessary to connect and disconnect from a speciﬁc remote repository, as9 well as to save and retrieve contents from it. Secondly, the extension was deﬁned using the following snippet of code: <extended-type name=PersistentSection extends=Section> <extension transitionName=Unlock> <extension transitionName=Lock>

5.2

Extending PaperXpress: Revision Control for Article Sections

The second extension to PaperXpress includes a simple revision control mechanism for article sections. This mechanism must be activated automatically every time an activity modiﬁes a section and it applies the following two criteria: First,

Extensibility in Model-Based Business Process Engines

171

a new revision is created, if and only if, the contents of a section are diﬀerent from the contents of its current revision. Second, if the diﬀerences between those contents are greater than 30%, the new revision should then increase its revision number by 1; this is called a major revision. Otherwise, the revision number is increased by 0.1; this is called a minor revision. In order to support this new requirement in PaperXpress, we used Cumbia extension mechanisms to allow the inclusion of additional elements to the metamodels. In particular, two new elements within the article metamodel were deﬁned (see ﬁgure 11): Firstly, a new element representing the current revision of a section (right side of the ﬁgure); and secondly, a new extended section composed and coordinated with its revision (left side of the ﬁgure).

Fig. 11. New elements within the article metamodel

The element that represents the current revision of a section (Revision), keeps the last version of the section contents, and is identiﬁed by a revision number that changes according to the second criterion. Within its state machine, the complete revision process required by the two criteria, is represented and coordinated by particular actions that activate the transitions in the state machine. This revision process is triggered by the extended section (VersionedSection) when it is locked after an activity has ﬁnished. In order to do this, its state machine is being enriched with a new state between the unlocked and locked states. In this way, when the section is locked it must be revised. As soon as the revision steps ﬁnish, the section is locked again. Implementing this requirement implied three simple steps. Firstly, the article metamodel was extended to include the new elements and their state machines. To do this, the metamodel was updated using the Cumbia editor. Secondly,

172

M. S´ anchez et al.

the new entities (Revision and VersionedSection) were implemented. They were implemented in separate java classes and their logic includes simple attributes handling and call-back methods. Finally, the new actions were implemented. Note that no changes to other concerns were needed. 5.3

Extending PaperXpress: Supporting Several Authors

Usually, research articles are written by more than one author. For this reason, another extension to PaperXpress is to support the participation and collaboration of several authors within the process. The authors and their activities must be deﬁned at the same time as the article structure and the process deﬁnition. Once they are deﬁned and the process started, the authors must be able to login to PaperXpress and check their assigned activities. In particular, authors must be assigned one activity at a time. In case an activity is assigned to a busy author (he has unﬁnished activities assigned), that activity must be queued in his pending activities; pending activities can later be manually assigned. In order to support the new requirement, we extended PaperXpress with a whole new concern: the authors concern. This concern includes two principal elements in its metamodel: Authors and Assignments (see ﬁgure 12). In particular, an author might be waiting for an assignment or he might already have begun working on it. These two possible states are represented in the author’s state machine, and they deﬁne whether or not an activity is marked as pending during process. While an author is waiting, an assignment can be given to him. This assignment is triggered by the predeﬁned process when the corresponding activity starts its execution. At this particular moment, the availability of the author must be checked: If he is busy, the activity is marked as pending and must be activated later manually; otherwise, the assignment is given to the author, and his state changes to Working. This assignment policy is represented in the state machine of the assignment element shown in ﬁgure 12. Implementing this requirement in PaperXpress implied three steps. Firstly, the elements of the new concern had to be developed. The Cumbia editor was used for the metamodel and state machine deﬁnitions, and separate java classes were implemented for the entities and the actions. Secondly, the state machine

Fig. 12. Elements in the authors concern: a) Assignment and b) Author

Extensibility in Model-Based Business Process Engines

173

of the activity was extended with two actions, to be coordinated with the assignments. In particular, for each activity there is one assignment to a corresponding author. In this way, as soon as the state machine of the activity changes its state to Active, an action notifying the corresponding assignment is executed. The assignment policy is then triggered and the author can start working, or the activity can be marked as pending. The other action installed in the state machine of the activity notiﬁes the assignment to be ﬁnished when the activity ends its execution. Finally, the client applications to use the PaperXpress workﬂow engine had to be extended too. In particular, the process deﬁnition editor required the most changes.

6

Conclusions

Organizations wanting to be agile and to embrace change, need information systems that can be easily adapted to new requirements. In particular, workﬂow engines should oﬀer features to introduce functionalities whenever new business requirements are not supported with the current languages or implementations. In this paper we presented our platform to develop extensible workﬂow engines, which is based in the idea of separating process concerns such as control, data and resources. Other workﬂow tools use strategies based on the development of huge languages, which support a great number of requirements and, in consequence, are very complex. But on the other hand, our strategy is based on the usage of concern-speciﬁc languages, and the development of small, customized engines. This strategy improves ﬂexibility and extensibility within a concern, and allows the inclusion of new concerns as required. To support this, Cumbia uses model-based techniques, and proposes the usage of open objects to build metamodels. This has several beneﬁts. One of them is the common presence of mechanisms of communication, interaction and extension are common to every concern. Another is that a single weaving language, provided by the platform, is necessary to specify relationships between any pair of concerns. The beneﬁts of the Cumbia extensible platform were illustrated in a context speciﬁc workﬂow engine called PaperXpress. Three diﬀerent extensions were easily included, showing the capacity of the platform to introduce new concerns and functionalities, and reusing elements. A common point between the extensions presented is that they are only applied to future instances of the business processes. However, in certain cases it may be necessary to introduce changes that aﬀect processes already in execution. In those situations, mechanisms similar to those presented in the paper can be used, but many additional considerations are required. For instance, it is necessary to maintain execution and data consistency. We have not yet directly addressed these problems, but it is part of our ongoing research to deﬁne a taxonomy of changes to running processes. The correction of the evolving workﬂows is an important issue, and we have presented several scenarios where inconsistencies can be easily introduced. We have not studied these problems yet, but they are part of out future research, and they will be applied to the PaperXpress workﬂow engine as well.

174

M. S´ anchez et al.

References 1. S´ anchez, M., Villalobos, J.: A ﬂexible architecture to build workﬂows using aspectoriented concepts. In: Proceedings of the 2008 AOSD Workshop on AspectOriented Modeling, AOM 2008, pp. 25–30. ACM, New York (2008) 2. Web Services Business ProcessExecution Language, Version 2.0, http://docs.oasis-open.org/wsbpel/2.0/wsbpel-v2.0.pdf 3. Business Process Modeling Notation, V1.1, http://www.omg.org/spec/BPMN/1.1/PDF 4. Process Deﬁnition Interface – XML Process Deﬁnition Language, Version 2.1, http://www.wfmc.org 5. WS-BPEL Extension for People (BPEL4People), Version 1.0, http://www.ibm.com/developerworks/webservices/library/specification/ ws-bpel4people/ 6. Holmes, T., Vasko, M., Dustdar, S.: VieBOP: Extending BPEL Engines with BPEL4People. In: 16th Euromicro International Conference on Parallel, Distributed and network-based Processing 2008, PDP 2008, pp. 547–555. IEEE Computer Society, Los Alamitos (2008) 7. Braem, M., Verlaenen, K., Joncheere, N., Vanderperren, W., Van Der Straeten, R., Truyen, E., Joosen, W., Jonckers, V.: Isolating process-level concerns using Padus. In: Dustdar, S., Fiadeiro, J.L., Sheth, A.P. (eds.) BPM 2006. LNCS, vol. 4102, pp. 113–128. Springer, Heidelberg (2006) 8. Charﬁ, A., Mezini, M.: Aspect-oriented workﬂow languages. In: Meersman, R., Tari, Z. (eds.) CoopIS, DOA, GADA, and ODBASE 2006. LNCS, vol. 4275, pp. 183–200. Springer, Heidelberg (2006) 9. Axenath, B., Kindler, E., Rubin, V.: AMFIBIA: a meta-model for integrating business process modelling aspects. International Journal of Business Process Integration and Management 2007 2(2), 120–131 (2007) 10. Taylor, I.J., Deelman, E., Gannon, D.B., Shields, M. (eds.): Workﬂows for e-Science: Scientiﬁc Workﬂows for Grids. Springer, Heidelberg (2007) 11. Han, J., Cho, Y., Choi, J.: Context-Aware Workﬂow Language Based on Web Services for Ubiquitous Computing. In: Gervasi, O., Gavrilova, M.L., Kumar, V., Lagan´ a, A., Lee, H.P., Mun, Y., Taniar, D., Tan, C.J.K. (eds.) ICCSA 2005. LNCS, vol. 3481, pp. 1008–1017. Springer, Heidelberg (2005) 12. Baeyens, T., Valdes, M.: The Process Virtual Machine, http://docs.jboss.com/jbpm/pvm/article/ 13. WS-Security Core Speciﬁcation 1.1, http://www.oasis-open.org 14. S´ anchez, M., Villalobos, J., Deridder, D.: Co-Evolution and Consistency in Workﬂow-based Applications. In: 1st International Workshop on Model CoEvolution and Consistency Management, Toulouse, France (2008) 15. Inverardi, P.: Compositionality, Coordination and Software Architecture. In: De Nicola, R., Ferrari, G.-L., Meredith, G. (eds.) COORDINATION 2004. LNCS, vol. 2949, pp. 3–4. Springer, Heidelberg (2004) 16. Brogi, A., Canal, C., Pimentel, E.: Behavioural types for service integration: Achievements and challenges. Electronic Notes in Theoretical Computer Science 180(2), 41–54 (2007) 17. Russell, N., ter Hofstede, A.H.M., van der Aalst, W.M.P., Mulyar, N.: Workﬂow Control-Flow Patterns: A Revised View. Technical Report, BPM Center Report BPM-06-22 (2006)

Guaranteeing Syntactic Correctness for All Product Line Variants: A Language-Independent Approach Christian Kästner1 , Sven Apel2 , Salvador Trujillo3 , Martin Kuhlemann1, and Don Batory4 1

4

School of Computer Science, University of Magdeburg ckaestne/[email protected] 2 Dept. of Informatics and Math., University of Passau [email protected] 3 IKERLAN Research Centre, Mondragon, Spain [email protected] Dept. of Computer Science, University of Texas at Austin [email protected]

Abstract. A software product line (SPL) is a family of related program variants in a well-defined domain, generated from a set of features. A fundamental difference from classical application development is that engineers develop not a single program but a whole family with hundreds to millions of variants. This makes it infeasible to separately check every distinct variant for errors. Still engineers want guarantees on the entire SPL. A further challenge is that an SPL may contain artifacts in different languages (code, documentation, models, etc.) that should be checked. In this paper, we present CIDE, an SPL development tool that guarantees syntactic correctness for all variants of an SPL. We show how CIDE’s underlying mechanism abstracts from textual representation and we generalize it to arbitrary languages. Furthermore, we automate the generation of plug-ins for additional languages from annotated grammars. To demonstrate the language-independent capabilities, we applied CIDE to a series of case studies with artifacts written in Java, C++, C, Haskell, ANTLR, HTML, and XML.

1 Introduction A Software Product Line (SPL) is a set of software-intensive systems that shares a common, managed set of features, satisfying the specific needs of a domain [7]. A feature is an end-user visible requirement that is used to describe commonalities or differences in this domain [20,27]. Different programs of the SPL, called variants, can be generated from a common code base by combining features. Already with a few features, a high number of distinct variants can be generated [27]. The ability to generate many variants is beneficial because variants can be tailored to specific scenarios according to customer requirements. At the same time, this raises new challenges: Traditional application development focuses on the design, implementation, and testing of a single program; detecting errors in an SPL is difficult as an error may appear only in some variants with specific features or feature combinations [28,9]. Because of the sheer number of variants (already with 33 independent optional features, M. Oriol and B. Meyer (Eds.): TOOLS EUROPE 2009, LNBIP 33, pp. 175–194, 2009. c Springer-Verlag Berlin Heidelberg 2009

176

C. Kästner et al.

there is a distinct variant for every person on the planet), generating, separately compiling every variant in isolation is usually infeasible. Thus, novel techniques for checking the entire SPL instead of checking each variant separately are needed [9,35]. Approaches for checking SPLs are especially challenging considering the fact that an SPL typically contains artifacts written in different languages. Beyond source code, an SPL can also contain non-code artifacts as build scripts, models, and documentation. SPL tools should handle different artifact types in a uniform way (a.k.a. principle of uniformity) [3]. While some SPL implementation mechanisms (preprocessors [27], XVCL [18], Gears [26], pure::variants [4], etc.) are so general that they work on plain text files, even simple errors can go undetected. Even syntax errors in the target language – like missing a closing bracket – can be difficult to detect if the error occurs only in few variants. On the other hand, more sophisticated tools and languages that can detect certain errors in SPLs [3,9,24,8,16,21] are usually available only for a single language. In this paper, we present a tool called Colored Integrated Development Environment (CIDE) for SPL development that guarantees syntactical correctness for all variants and multiple languages. CIDE is similar to #ifdef -preprocessors, but by abstracting from plain text files and considering internal structure, simple rules can prevent syntax errors in a language-independent way. However, while a previous version of CIDE presented in [22] (with focus on expressiveness of language constructs) supported only Java, in this work, we generalize CIDE to a wide variety of languages. We extract underlying principles necessary for correctness in Java and propose a language-independent model that can be used for other languages as well. Finally, to minimize human effort for using this model with a new language, we automate the generation of language plug-ins for CIDE based on a grammar file of the target language. Detecting syntax errors is only a first step in our endeavor to ensure a languageindependent safe implementation of SPLs, which detects errors as early as possible in the development process. With this paper, we contribute a foundation for this endeavor: simple mechanisms can prevent common and difficult to find syntax errors, independent of the used language. On this foundation, further mechanisms to detect type errors [21,25] or for verification [36] and model-checking [30] (which all require syntactically correct code and so far exist for single specific languages only) can be added for more detailed checks and multiple languages in future steps. In order to demonstrate the practicality of our approach, we generated a number of plug-ins for (among others) the following code and non-code languages: Java, C, C++, C#, Haskell, JavaScript, ANTLR, HTML, XML. Subsequently, we used CIDE in seven small to medium-sized SPL projects written with these languages. In all projects, CIDE guarantees syntactical correctness of all generated variants in all languages. CIDE together with all languages and case studies presented in this paper can be downloaded from the project’s web site: http://fosd.de/cide/

2 Taxonomy of Errors in Software Product Lines and Related Work There are many possible errors that can occur in an SPL’s implementation. We provide a taxonomy to explain the problems we are addressing and to distinguish our approach

Guaranteeing Syntactic Correctness for All Product Line Variants Kind of Error Error Detection Languages Implementation

177

Syntax Typing Semantic Check Variants Check SPL Single Multiple Inter-Language Annotative Compositional Generators ...

Fig. 1. Taxonomy of errors and corresponding checks in SPLs (morphological box)

from related work. In Figure 1, we give an overview and set the focus of this paper (highlighted boxes). First, we classify three kinds of errors: syntactic errors, type errors and semantic errors (first line in Fig. 1). Syntax errors occur when a variant is ill-formed regarding the language’s syntax, for example when an opened bracket is not closed. Type errors occur when the variant is ill-formed regarding the language’s type system, e.g., a statement invoking a method that is not defined in that variant. Finally, semantic errors occur when the variant behaves incorrectly according to some (formal or informal) specification, and are the most difficult to detect. In this paper, we begin with syntax errors but give an outlook on detecting typing and semantic errors. Second, we distinguish two error detection approaches: check variants or check the SPL itself (second line in Fig. 1). In the first case, not the SPL itself but (some or all) generated variants are checked [28]. A brute force strategy of generating and checking all variants is usually infeasible, because already with few features the number of variants that can be generated from an SPL explodes (for n independent optional features, there are 2n distinct variants). This typically means that only some sampled variants are checked. In contrast, some approaches check the entire SPL itself [9,35,21] and guarantee correctness for all variants when this check passes. In our work, we want to check the SPL, not each variant separately. Third, we classify checks by their coverage of different programming languages: single language, multiple languages, and inter-language errors. Some checks are specific to a single language. For example, different languages require different type checks. Next, there are errors that can occur in different languages and can be addressed by the same tool. Finally, there are errors that occur only at the interaction of multiple languages, e.g., the interface specification of a web service in a WSDL file and its implementation might not match. In our work, we focus on a mechanism that is languageindependent. Fourth, we distinguish checks by the general implementation mechanism of the SPL, which is usually put on top of a programming language. Although there are many different mechanisms, for brevity, our taxonomy considers only three groups. In annotative approaches – common in industry – code is annotated and removed for variant generation; typical examples include ‘#ifdef’ preprocessor directives, Frames/XVCL [18], and commercial SPL tools like Gears [26] and pure::variants [4]. In contrast, compositional approaches – favored in academia – implement features in physically separated modules and compose them to generate variants; examples include frameworks [19] and different forms of components, aspects or feature modules [27,3,24,2,1]. Furthermore, several other implementation mechanisms like generators [8,16] or version control systems [33] exist. All approaches have different advantages and disadvantages, e.g., regarding SPL

178

C. Kästner et al.

adoption or expressiveness as discussed in [6] and [22], which justifies research on error detection for all of them. In this work, we focus on annotative approaches. Related Work by Kind of Error. There is a large body of research on checking SPLs for errors. A first group of approaches focus on SPL testing, i.e. detecting semantic errors by running test cases [34,28,27]. Testing can be applied to different implementation mechanisms and different languages and can detect even inter-language defects, but only variants are tested, not the entire SPL. Another approach to detect semantic errors is to apply formal methods. Earlier work suggested specific languages to verify SPLs for compositional approaches [36,29] or to check an SPL with annotations using model checking [30], but both have yet to show how they scale for real world SPLs. Also for type errors, there has been effort to switch from type-checking individual variants to type-checking the entire SPL. Approaches exist for individual languages with annotative [21,9,25], compositional [35,10] and generative [16,17] implementations. Although suggested as a future extension in [9] and [35], to the best of our knowledge, approaches that cover multiple languages or inter-language typing have not been applied to SPL checking. Finally, regarding syntax errors, compositional approaches separate 1 s t a t i c i n t __rep_queue_filedone( feature modules physically so that the dbenv, rep, rfp) 2 DB_ENV *dbenv; SPL can be checked by checking all 3 REP *rep; features in isolation. Recent compo4 __rep_fileinfo_args *rfp; { 5 # i f n d e f HAVE_QUEUE sition tools that support multiple lan6 COMPQUIET(rep, NULL); guages in a uniform way [3,1] can 7 COMPQUIET(rfp, NULL); 8 r e t u r n (__db_no_queue_am(dbenv)); even check the SPL’s syntax for multi9 #else ple languages. For (language-specific) 10 db_pgno_t first, last; 11 u_int32_t flags; generators there have been approaches 12 i n t empty, ret, t_ret; to check the generator, to ensure 13 # i f d e f DIAGNOSTIC 14 DB_MSGBUF mb; syntactical correct output for any in15 # e n d i f put [16]. In contrast, annotative ap16 // over 100 lines of additional code proaches (all features located in the 17 } # e n d i f same annotated code base) are typically so general that they work only on Fig. 2. Code excerpt of Berkeley DB, with syntax plain text, and detecting syntax errors error in variants without HAVE_QUEUE is largely unexplored. Syntactic Correctness in Annotative Approaches. Consider the code fragment in Figure 2 that shows a fragment of C code from Oracle’s Berkeley DB1 that uses C’s preprocessor with multiple (partly nested) ‘#ifdef’ annotations to generate different variants. Already, this short code fragment illustrates the complexity that may occur when implementing variability in SPLs, but more importantly, it illustrates a subtle error, which we deliberately introduced. Note that the opening curly bracket in Line 4 is only closed in Line 17 when the feature HAVE_QUEUE is selected, while other variants contain a syntax error. Although this example may appear trivial, it is a matter of scale. In our projects (see Sec. 5), we experienced such problems frequently and it was often not 1

http://www.oracle.com/database/berkeley-db

Guaranteeing Syntactic Correctness for All Product Line Variants

179

obvious to find their cause. In large SPLs with many features, syntax errors can easily occur in some variants and can be difficult to detect, especially when located in nested annotations and thus only in very few variants [32]. Interestingly, there are some annotative tools that can guarantee syntactic correctness for some languages by transforming and annotating software artifacts on a higher level of abstraction. One example is Czarnecki’s tool fmp2rsm [9] to generate variants of annotated UML models. Using this tool, syntax errors (e.g., a class without a name) cannot occur because annotations and variant generation is not performed on the textual representation of the model, but on an abstract level with the Rational Software Modeler engine, which does not allow transformations that would invalidate UML syntax. In other fields of software engineering, this abstraction principle is also frequently applied. For example, refactorings in IDEs such as Eclipse are usually not performed directly on the textual source code, but on an abstract representation like an abstract syntax tree [12]. In CIDE, we used this principle of abstraction for removing annotated code safely. To summarize, although there are approaches to address syntax, typing, and semantic errors in SPLs for single languages, there are no language-independent solutions. In the remainder of this paper, we explore language-independent checks for SPLs implemented with annotated source code. We begin by describing CIDE’s guarantee for syntactic correctness in Java and subsequently generalize it for other languages.

3 Checking Syntactic Correctness of Java SPLs In this section, we revisit CIDE, a tool for developing Java SPLs using annotations on source code, which we presented in prior work [22]. With simple design principles, CIDE guarantees that all variants are syntactically correct Java programs. To provide context, we first give an overview of the origins of CIDE and the motivation behind it. CIDE was originally designed to analyze and discuss how code fragments that implement a feature are scattered and interact inside legacy applications. In our discussions, we originally highlighted those code fragments on printouts with text markers using a different color for each feature. This turned out to be useful, so that the motivation for CIDE was to convey this color metaphor to a Java IDE based on Eclipse. In CIDE, developers assign code fragments to features. These annotations are then represented with a background color in the editor (one color per feature), just as with the text marker on paper.2 When creating a variant, code annotated with unwanted features is removed like when using ‘#ifdef’ directives in C. The main visual difference lies in the fact that CIDE uses background colors instead of ‘#ifdef’ directives for annotations. Besides the visual representation, another key difference between CIDE and traditional preprocessors (a difference we later use to guarantee syntactic correctness) is that, in CIDE, features are assigned to elements of the underlying structure of the Java code, instead of assigning them to a sequence of characters (possibly determined by 2

In case multiple features are assigned to the code fragment, the corresponding background colors are blended. Though this does not allow to recognize feature code solely from the background colors, it indicates where feature code starts and ends, so that the user can lookup the actual features in a tool tip or even infer them from the context.

180

C. Kästner et al. ClassDeclaration Name=C MethodDeclaration Name=m

1 class C { 2 v o i d m( i n t p){ 3 s1(); 4 s2(p,true); 5 } 6 }

ReturnType Type=void

Parameter Name=p

Block

MethodCall Name=s1

MethodCall Name=s2

Parameter Value=p

Parameter Value=true

model

Fig. 3. Source code fragment and according AST

AST

binary

source

1

parse

Legacy Application

assign features 2

Annotated AST 3

save/load

Java code with external annotations

remove features 4

Variant AST 5

serialize

Variant code 6

compile

Program

Fig. 4. CIDE’s process for implementing an SPL and creating a variant

offset and length). As underlying structure, we use the abstract syntax tree (AST) that represents a Java artifact at a fine granularity. The AST provides flexibility to annotate even small code fragments in the middle of a method, which we often needed in our projects. In CIDE, we assign features to AST nodes that represent the selected code fragment, other annotations are not possible. Nevertheless, for users, the underlying structure is transparent, they simply annotate code fragments, which are then mapped to AST nodes internally. In Figure 3, we illustrate this concept of using the underlying structure with a simple example. It shows a code fragment and its AST. When assigning a feature to a code fragment (in this example, Line 4, underlined), the code fragment is internally mapped to the corresponding AST nodes (grayed). Code fragments that cannot be mapped to AST nodes cannot be annotated. In the user front-end, the code fragments belonging to annotated AST nodes are shown with a background color according to their assigned features (of course other visual representations would be possible as well, e.g., using additional keywords like #ifdef instead of colors). The overall process to develop an SPL with CIDE is depicted in Figure 4. Developers begin with a syntactically correct base implementation (possibly a legacy application), which is parsed into an AST (Step 1). Then, they assign features to AST nodes inside the development environment (Step 2). The feature-annotated AST can be saved and loaded again (Step 3), so that it is still possible to edit the source code. To generate a variant, developers select a set of features and CIDE removes all AST nodes that are annotated with features that are not selected (Step 4). Note, this step is a transformation

Guaranteeing Syntactic Correctness for All Product Line Variants

181

ClassDeclaration Name=C MethodDeclaration Name=m

1 class C { 2 v o i d m(){ 3 try { 4 s1(); 5 } catch(Exception e) { 6 handleException(e); 7 } 8 } 9 }

… Block

TryStatement

Block

Catch Exception=e

MethodCall Name=s1

Block

MethodCall Name=... …

Fig. 5. Wrapper for try-catch Statement as Exception to the Subtree Rule

from one AST to another. The generated AST is then serialized (‘unparsed’) as source code (Step 5), which finally can be compiled into a program (Step 6). Enforcing Syntactic Correctness. Representing source code as ASTs instead of plain text and the AST transformation in Step 4 are the key to CIDE’s guarantee for syntactic correctness. With only two rules, we can ensure that every transformation in Step 4 transforms the annotated AST into another AST that also adheres to Java’s syntax specification. Thus, we can prevent by construction the generation of code with incorrect syntax. The rules are: – Optional-Only Rule: Only AST nodes that are optional according to the Java syntax specification (as described in [14]) can be removed. For example, we cannot remove a class’s name without invalidating the AST, but we can remove a method. Incidentally, the AST provided by the Eclipse Java framework already enforces this rule; it only removes optional elements and throws exceptions otherwise. – Subtree Rule: When an AST node is removed, all its child nodes must be removed as well. For example, when method m is removed in Figure 3 also its parameter and statements must be removed; when class C is removed all content therein must be removed as well. For CIDE that means, when a user annotates an AST node, CIDE automatically propagates this annotation to all subnodes. In CIDE, these two rules provide a foundational mechanism for syntactic correctness. Even without consulting a feature model, this mechanism guarantees that no transformation can invalidate the AST. At the same time, it provides a fine granularity so that even individual statements or parameters can be annotated, as discussed in [22]. Nevertheless, we found some situations in Java in which these rules are too restrictive and more flexibility is needed. For those situations, we manually implemented an exception. Due to the Subtree Rule, we were not able to independently annotate code fragments that wrap other code. A typical example is a try-catch statement as in Figure 5, which could belong to an exception handling feature. The try-catch statement wraps a code block “s1();”. When deselecting the exception handling feature in a variant, the Subtree

182

C. Kästner et al.

Rule would automatically remove all child elements including the wrapped code block. The same effect occurs with several other Java statements like try-finally, synchronize, if, for, while, and do, which wrap other statements and which we therefore call wrapping elements. To offer more flexibility, we allow an exception to the Subtree Rule in Java: When a user annotates a wrapping element, specific child elements can be excluded despite the Subtree Rule. When removing this wrapper in a variant, it is replaced with the wrapped element.

4 Generalizing CIDE beyond Java CIDE was created for Java and builds directly on Eclipse’s Java framework, which made implementing the two rules simple because Eclipse already encodes the Java language specification. Nonetheless, SPLs usually consist of code and non-code artifacts written in different languages, e.g., source code, scripts, make files, documentation, models, or grammar files. All these artifacts should be handled uniformly by product line tools, as stated by the principle of uniformity [3,1]. Therefore, our goal is to provide CIDE – including its guarantee for syntactic correctness – for multiple languages of different kinds of code and non-code artifacts. In this section, we generalize CIDE beyond Java and proceed in two steps. First, we analyze the underlying principles behind the rules and exceptions we found for Java in order to derive a general model. Second, we describe our approach to automate the process of extending CIDE for new languages to minimize human effort. 4.1 Generalizing Correctness Rules: The gCIDE Model CIDE’s key mechanism for Java was abstracting from plain text, using the underlying structure, and allowing only operations on that structure which do not invalidate Java’s syntax. To generalize these concepts, we need an underlying structure for other languages as well. This structure can be an AST as common in programming languages, a document object model as in XML, or another structure that represents the artifact. In order not to implement distinct mechanisms for every language, we develop a generalized model for CIDE (gCIDE model) that language-independently represents a common underlying structure on which rules for syntactic correctness are defined. Thus, instead of statements, classes, AST nodes, XML elements, or others, we just generally speak of structural elements. For concrete languages, the language structure is mapped to this language-independent model. The full gCIDE model (which we explain next in several steps) is depicted in Figure 6. Basics. The Subtree Rule can be applied directly to arbitrary tree structures: whenever a structural element is annotated, its children are annotated as well. In the gCIDE model, this tree structure is represented such that structural elements have exactly one parent each (except for the root that represents the entire file) and may have child elements which are again structural elements. For technical reasons, to be able to make the mapping between code fragments and structural elements transparent in CIDE, each structural element must store a mapping to the actual location in the artifact (modeled as mappingToArtifact).

Guaranteeing Syntactic Correctness for All Product Line Variants

183

StructuralElement +type +wrappee +mappingToArtifact 1

0..*

+isOptional() +remove()

+children 0..1 +parent

0..1

ParentChildRelation +isOptional +childType

WrappingElement

+wrapper

Fig. 6. gCIDE Model

To transfer the Optional-Only Rule from Java to other artifacts, there must be a description which elements in the tree structure are optional. In Java this can be derived from the Java Language Specification [14], in XML the allowed structure is specified by a W3C recommendation [5], for other languages such specification either exists or must be formulated to determine which (removal) transformations on the structure are safe. In the gCIDE model, independent of any specific language, the isOptional attribute of an element’s relationship to its parent specifies whether the element can be removed safely. The values of the isOptional attribute must be assigned individually to every element for each language. Wrappers and Types. For the wrapper exception to the Subtree Rule (removing a trycatch statement in Java without removing the inner statements), we introduce the notion of a wrapping element in our model (cf. Fig. 6). In the gCIDE model, a wrapping element is a special case of a structural element and specifies exactly one child element it wraps (implications of allowing to wrap multiple child elements are discussed below). When removed, it is replaced by this child element. Wrapping elements cannot be placed at arbitrary places or wrap arbitrary elements. For example, a Java class cannot wrap a method such that the class is replaced by this method if removed, as this would invalidate the AST. In the original implementation of CIDE, we manually defined specific exceptions for selected Java elements and implemented them individually. For a general solution in gCIDE, we use a type-based mechanism instead.3 Each structural element belongs to a type and there is a subtype relation on those types. Parent-child relations between structural types state the type of elements they expect. For example, Java classes expect members as child elements, blocks expect statements. The subtype relation describes which structural types represent a member or a statement and can be used as child element. For example, ‘method’ and ‘field’ are subtypes of ‘member’ and can be child elements of classes, while ‘method invocation’ and ‘try-catch’ are subtypes of ‘statement’ and can be used as a child element for blocks. Types, subtype relations, and accepted types are modeled in the gCIDE model and must be provided for each language. 3

For clarification: the types used in this mechanism represent syntactical categories of the host language as statements, parameters, or blocks. They are not to be confused with types of terms in the host language.

184

C. Kästner et al.

With types, a straightforward algorithm can determine where wrappers are allowed. For this, it needs to consider three elements: the wrapper, its parent, and the wrapped child. The wrapped child is allowed if it is accepted as a child of the parent. For example, in Figure 5 the try-catch statement can wrap a block but not the CatchBlock element, because only the block is acceptable as child to the parent.4 4.2 Automating Language Plug-in Creation Using the gCIDE model, we can now extend CIDE to support multiple languages. We evolved CIDE and removed all Java specific code and replaced it by an abstract framework following the gCIDE model. Concrete target languages can now be added as extensions – so called language plug-ins – which implement this framework. Thus, a language plug-in creates structural elements for a specific target language and fills values like types and isOptional. This way, we can separate the infrastructure that is common for all languages (user frontend, feature management, tree transformation, cf. Step 2 and 4 in Figure 4) from the implementation of specific target languages. However, the effort to create language plug-ins for CIDE is still high. First, we need a parser for each language that transforms the artifact into the tree structure (Step 1 in Figure 4). Second, we need to serialize (‘unparse’) transformed structures in each language to write them back to an artifact (Step 5 in Figure 4). Finally, and most importantly, we need to define the rules and exceptions for each specific language: we need to define which structural elements are optional or wrappers, so that transformations always transform valid trees into other valid trees. A straightforward way to develop language plug-ins is to bridge the internal structures of an existing open compiler or graphical editor to the gCIDE model. For example, we could bridge the AST from Eclipse’s Java framework or the internal structure of an UML editor and thus reuse this infrastructure. However, for many languages, industrialstrength compilers that can be accessed and reused are not available. Furthermore, implementing the bridge might still require considerable effort. Instead, we pursue an approach, in which we can uniformly generate language plug-ins.5 Fortunately, creating language plug-ins (parser generation, serializer implementation, rule definition) can be automated to a high degree from the grammar of the target language, as we will show in the remainder of this section. First, existing parser generators can generate a parser from a grammar. Second, some parser generators can also create a ‘pretty printer’ that can be used for serialization (‘unparsing’). Finally, even information for the Subtree Rule and the Optional-Only Rule can be derived from the 4

5

Note, it is conceptually possible to model structural elements that wrap multiple elements under some conditions, but it would require a more complex model and reasoning. Required conditions are: (1) all wrapped elements must be of the correct type and (2) it must be allowed to replace one wrapper with multiple elements. Alternatively, more complex custom transformations could be specified. To keep the model simple, we disallow wrappers around a multiple elements. Note, our generation approach targets artifacts which are usually edited with a textual editor like source code or textual modeling and specification languages such as Alloy. For artifacts that are usually edited in a graphical editor such as UML, often a tailored approach of mapping the gCIDE model to the representation of a specific editor is more suitable.

Guaranteeing Syntactic Correctness for All Product Line Variants

185

target language’s grammar, because a grammar specifies (1) the child-parent relationship between structural elements, (2) which elements are optional, and (3) subtyping information. That is, we can use the grammar of a target language as the single source to generate a language plug-in. To generate language plug-ins for CIDE from grammar specifications, we built our own tool chain, because common parser generators (e.g., JavaCC, yacc, or ANTLR) do not propagate sufficient information from the grammar to the created tree structure. For example, from the parse tree that JavaCC generates, we cannot determine which elements are optional. Therefore, we defined our own grammar specification language called FeatureBNF and built a tool called astgen which generates LL(k) parsers, serializers, and trees with all information required by the gCIDE model.6 FeatureBNF Basics. FeatureBNF uses an extended Backus-Naur notation and supports some additional annotations for astgen. From a given grammar, elements are recognized as optional when followed by a question mark, or when they are part of a list (expressed with an asterisk symbol). In Figure 7, we illustrate an excerpt from a sample programming language. A compilation unit consists of any number of type declarations. The type declaration consists of one mandatory identifier, a second optional one, an optional ‘implements’ list, and a class body. The class body contains any number of fields or methods. 1 CompilationUnit : 2 TypeDeclaration : ClassBody; 3 ClassBody : 4 Member : 5 ImplementsList :

(TypeDeclaration)* <EOF> ; "class" ( "extends" )? ( ImplementsList )? "{" (Member)* "}" ; Method | Field ; "implements" ("," )* ;

Fig. 7. FeatureBNF grammar example

From a given grammar for a target language, astgen generates the language plug-in, consisting of a parser that builds a tree structure for a given artifact in that language, and a pretty printer that writes it back into a file. The tree structure generated by the parser not only represents the structure of the source code, but can also reflect its structural properties derived from the grammar, e.g., each tree node knows whether it is optional as described in the gCIDE model. So, in the given example in Figure 7, the type declarations are optional (there can be any number of type declarations in a compilation unit, Line 1) and thus can be annotated in CIDE. Inside the type declaration the first identifier and the class body are mandatory and cannot be annotated, but the second identifier and the ‘implements’ list are optional (Line 2). Also members are optional and can be annotated (Line 3). In a full grammar, typically also statements, parameters, or parts of expressions are optional and can be annotated in CIDE. 6

Technically, the FeatureBNF grammar specification language is a meta-grammar. It is a grammar that specifies how developers can specify and annotate grammars for a specific target language. For example, we can write a Java grammar in the FeatureBNF format. From this Java grammar, astgen generates all required parts for a Java language plug-in. For parser generation, we internally reuse JavaCC. Note, FeatureBNF is reused with some extensions in another line of research on language-independent software composition [1].

186

C. Kästner et al.

Concrete Syntax vs. Abstract Syntax. A parser generated from a grammar for a target language with standard parser generator tools creates a Concrete Syntax Tree (CST). This tree contains those elements that are required for parsing. However, a CST does not necessarily reflect the abstract syntax of the language, and mapping a CST to the gCIDE model (instead of an AST) can result in reduced flexibility. A typical example how the concrete syntax may reduce flexibility is the use of lists, as exemplified in Figure 7, Line 5. The ‘implements’ list is optional inside the type declaration, but inside the list the first entry is mandatory due to special parsing requirements for the separating comma. Using the CST, the first entry cannot be annotated individually, although in the abstract syntax all entries are optional elements of a list. To ensure the full flexibility of the abstract syntax, it is necessary to transform the CST into an AST. In many tools this is a separate step after parsing. For serialization, the inverse transformation must be performed on the modified AST. To guarantee syntactic correctness for all variants, both transformations must be performed safely without loss of information. To bridge this gap, we follow the lead of Wile, who used an extended grammar specification language to derive the abstract syntax directly from a grammar file [37]. Wile proposed a series of additional constructs in the grammar specification language, so that the abstract syntax and its relationship to the concrete syntax are directly specified in the extended grammar file. This way, we can generate a parser that directly produces an AST instead of an CST. Wile further proposed a semi-automated process to transform an existing grammar describing a concrete syntax into the extended format. For example, to solve problems like the ‘implements’ list described above, he proposes a special ‘list’ construct. In Wile’s notation, the ImplementsList production is expressed as ImplementsList: ID ^ ",";, in which the ^ symbol is a special construct for lists followed by the token that separates list entries. Using this construct, the parser can interpret identifiers directly as lists and build the AST accordingly. We adopted Wile’s concept and added those extensions that are relevant for our case studies in FeatureBNF. This way, we can generate a parser that creates structural elements based on the target language’s abstract syntax from a grammar file. Only a single tree is created, no manual mapping between CST and AST is required, and all further transformations to remove annotated fragments can be directly performed on the AST of the artifact. Wrappers and Other Exceptions. Although technically possible, wrappers are not automatically recognized from the grammar for a target language, but a language expert has to decide where wrappers make sense. Wrappers should only be used where the additional flexibility is needed. Otherwise, it would still enforce syntactic correctness, but be harder to use.7 Feature BNF supports additional constructs to specify exceptions like wrappers. Other exceptions, e.g., making a mandatory production optional by providing a default value, or marking an optional production mandatory, can also be defined safely. Due to space restrictions we defer the interested reader to the language description at CIDE’s web site. 7

For example, classes could automatically be interpreted as wrappers around inner classes in Java. While this is syntactically correct and also fulfills the typing rules, annotating a whole class except an inner class, usually does not make sense. Offering such flexibility is only confusing to the developer; workarounds by rewriting the code are much easier to use.

Guaranteeing Syntactic Correctness for All Product Line Variants

187

5 Experience After extending CIDE and building the tool infrastructure, we generated 15 language plug-ins from grammars of various code and non-code languages. Next, we conducted several small case studies applying CIDE to different projects to test its practicality regarding those languages. Due to space restrictions, we only give an overview of language plug-ins and SPLs projects. Further information is provided in an accompanying technical report [23] and on CIDE’s website which contains the source code of all language plug-ins and SPL projects (except for the water boiler SPL which we cannot disclose to protect intellectual properties of our partners). Language plug-ins. In Table 1, we list all supported languages and some information describing optional structures, wrappers, and the number of production rules each. The number of production rules can be used as a rough indicator of the complexity of the language. For most languages the FeatureBNF grammar was derived from an existing grammar in the JavaCC or ANTLR format (which required mostly just syntactic changes) and usually took less than one hour. The language extensions for C and C++ were the most problematic, due to C’s preprocessor. Although we could straightforwardly adapt an existing JavaCC grammar for C, this would only work on C code that was already preprocessed. That is, a C parser requires source code in which ‘#include’ statements were resolved, ‘#ifdef’ statements were removed, and macros were evaluated. Without preprocessing (or even just with ignoring preprocessor directives) C code cannot be parsed. Unfortunately, working on preprocessed code is only an option for downstream tools, but in CIDE developers work on unprocessed code that still contains preprocessor directives. This problem was already faced much earlier, e.g., by intentional programming [31] or refactoring tools [13] and required complex workarounds. To overcome this problem in CIDE, we wrote a Table 1. CIDE Language plug-ins generated from FeatureBNF grammar Language

Featherweight Java Java 1.5 C (plain) C (pseudo) C++ (pseudo) C# Haskell (pseudo) Haskell 98 JavaScript/ECMAScript JavaCC , Bali , ANTLR Property files HTML XML

Optional structures

Wrappers

#Prod.

methods, fields, parameters members, statements, parameters, ... functions, statements, parameters, ... functions, statements, preprocessor, ... classes, methods, statements, prepr., ... classes, members, statements, parameters, ... types, imports, data, classes, ... types, imports, data, classes, parameters, ... functions, statements, expressions, ...

if, for, try, ... if, for, ... if, #ifdef, ... if, #ifdef, ... if, for, try, ... if if, for, ...

16 133 80 46 65 215 54 71 111

productions, terminals, ... lines headings, paragraphs, list items, ... nodes, parameters

[], ()* , ... -

14–166 1 11 13

handwritten based on external specification, adapted from JavaCC grammar, adapted from ANTLR grammar.

188

C. Kästner et al.

pseudo-parser, which does not actually parse the code based on the full language specification, but recognizes only important constructs like functions, variable declarations, or statements. For example, statements are recognized by the terminating semicolon, functions by the typical pattern for return type and parameter declarations. Preprocessor directives are recognized as part of the language, as long as they are used within certain limitations (e.g. ‘#include’ may not occur inside functions). With this pseudo-parser, we are able to use CIDE on C projects, but we weaken the guarantee for syntactic correctness to some degree (see discussion in Sec. 6). The same pseudo-parser approach was used for C++, and also for an initial version of Haskell, because of Haskell’s complex syntax. The XML grammar supports only plain XML files, i.e., all elements or parameters are optional and can be annotated. This guarantees that every variant is well-formed in the XML terminology. Guaranteeing that all variants of an XML artifact are valid based on the given DTD or XML Schema description requires additional information. Note, DTD is a meta-grammar itself that can be used for XML documents instead of FeatureBNF. We implemented a prototype transformation tool dtdgen, which converts a DTD into a FeatureBNF grammar, as a proof of concept. This way, we generated a parser specifically for XHTML (version ‘1.0 strict’). However, in ongoing work, we pursue a direct transformation from a DTD or XML Schema to the gCIDE model by extending an off-the-shelf XML parser [11]. Finally, for the Java language plug-in it is worth emphasizing that it is also generated from a grammar and not bridged from the Eclipse Java framework. Still, for enforcing syntactic correctness, the generated version is as flexible as the original one. All in all our experience shows that creating language plug-ins for new languages is simple for a variety of different languages, even for complex or less popular languages like Haskell or JavaScript. If a target language has a well-specified grammar, creating a language extension is a matter of few hours. All generated languages share CIDE’s guarantee for syntactical correctness (with some limitations for pseudo-parsers). SPL projects. In order to demonstrate that CIDE can actually be used for developing non-trivial SPLs written in different languages, we used CIDE in a series of small to medium-sized projects, as listed in Table 2. For brevity, we again only give an overview and refer the interested reader to our accompanying technical report [23]. The given durations for each project are rough estimates, and it has to be taken into account that in most projects the features were previously known, so only little code exploration was necessary. However, the point of these projects is not to discuss the feasibility or effort of creating SPLs by annotating legacy applications, but we focus on CIDE’s applicability to different languages. The first two projects are interesting, because with the generalized version, we could not only annotate the Java code of these SPLs, but also the documentation (e.g., 120 000 lines of HTML in Berkeley DB). This way, the documentation is also tailored specific for the generated variant, e.g., for a Berkeley DB variant without transactions, the ‘Transaction Processing Guide’ and all references to it are removed. In the third project, we additionally annotated the build scripts which are ANT files in XML format. Despite their small size, the prototypes of SQL Parser SPL for embedded systems and Arithmetic SPL demonstrate CIDE’s capabilities to languages beyond mainstream

Guaranteeing Syntactic Correctness for All Product Line Variants

189

Table 2. SPL projects implemented with CIDE Project Berkeley DB Graph Product Line AHEAD Tool Suite SQL Parser SPL Arithmetic SPL FAME-DBMS Water Boiler SPL

Features

LOC

38 14 24 4 3 14 14

204 000 2 300 45 000 60 120 6 000 10 000

Annotated Parts Code (Java), Documentation (HTML) Code (Java), Documentation (XHTML) Doc. (HTML), Build scripts (XML) Grammar (ANTLR) Code (Haskell) Code (C++) Code (C)

Time 4 days 3 hours 2 hours 20 min 1 hour 2 days 2 days

prototype, closed-source, industrial.

object-oriented languages. Finally, the last two projects are again real projects for embedded systems written in C and C++ from academia and industry that are actively developed and maintained. Even though, we provide only a pseudo-parser for C and C++, we could successfully annotate these projects. In these projects, we experienced tedious syntax errors quite frequently in earlier implementations before using CIDE. In all projects, we were able to create different variants and found (using tools, tests, or manual evaluation) that all generated variants are syntactically correct. For example, for FAME-DBMS and the Water Boiler SPL we successfully generated, compiled, and executed different variants.

6 Discussion: Flexibility vs. Safety During development and testing, we found a trade off between two properties: flexibility and safety. By imposing a tree structure on a source code artifact, we provide safety (guaranteeing syntactic correctness for all generated variants). However, at the same time, we impose restrictions on what developers are allowed to annotate and thus reduce their flexibility, i.e., they have fewer possibilities to express variability. For example, compared to an implementation using the C preprocessor that works on token level8 , CIDE only annotates optional elements of the underlying AST. Hence, using such preprocessor, a method can have two alternative return types, but in CIDE a return type cannot be annotated independently when it is mandatory in the language’s grammar. Programming languages already define a certain structure for code artifacts by their language syntax. For example, the Java syntax defines a top-down structure for Java code (e.g., Java files contain classes, which contain methods, which contain statements). By using ASTs to annotate programs, we expose this structure in CIDE and employ it with the Subtree Rule and the Optional-Only Rule to prevent syntax errors. Different languages provide a different amount of structure in their syntax. For example, JavaScript artifacts only contain a list of statements or function declarations, grammar artifacts only contain a list of production rules with a simple inner structure, and XML nodes are nested completely arbitrarily. 8

To be precise the C preprocessor works on lines of source code. However, due to the code layout flexibility in many languages, it can usually be used for token-based annotations.

190

C. Kästner et al.

Safety

This raises two questions. (1) How much structure does an artifact language need to be usable in CIDE? (2) Is the structure defined by a language’s syntax a limitation when adding further language plug-ins for other artifact types in the future? To answer the second question first, consider a ‘README.txt’ file. It is a valid artifact in an SPL, but will probably not provide any structure, at least none that is described by a LL(k) grammar. Fortunately, such artifacts, for which no specific language grammar is specified, can still be parsed by a dummy grammar that matches any file as a list of arbitrary optional tokens or even as a list of optional characters. With a dummy grammar, every character is optional with respect to the document, i.e., every single character in this document can be annotated independently just as when using preprocessors. This shows that a required structure is not a limitation of our approach. Even if no structure is available, as in the ‘README.txt’ file, we can still use the same tool to annotate this file uniformly next to other artifacts in an SPL. Nevertheless, structure is beneficial. When using the dummy grammar, the guarantee of syntactic correctness is lost, because any artifact adheres to this grammar. This shows that any structure – although it reduces flexibility – is beneficial for safety, because the grammar defines CIDE’s syntax checks. It restricts the possible parts of the artifact that can be annotated and enforces ‘reasonable’ annotations. In this context, the pseudo-parsing approach presented for C, C++, and Haskell artifacts in Section 5 is interestPseudo- CST AST parser A ing. Pseudo-parsing does not use the full structure as provided by the language Pseudoparser B grammar (in these particular cases because of technical limitations caused by Token-based the preprocessor in C/C++ and because Character-based of Haskell’s complexity). Instead, it uses Flexibility a simpler approach that recognizes only certain elements like functions and stateFig. 8. Safety vs. Flexibility ments, but ignores inner fragments like parameters, or expressions. There are two possibilities to handle those inner fragments which are ignored by the parser. First, we can regard these fragments as a single mandatory node each, i.e., there is no substructure inside such fragment (used in Haskell, referred to as alternative A below). Alternatively, we can parse these fragments with a dummy grammar as a list of optional tokens or characters (used in C and C++, referred to as alternative B). Pseudo-parsing can be used to balance between flexibility and safety, as alternative A guarantees syntactic correctness at a significantly reduced flexibility, while alternative B is more flexible but guarantees syntactic correctness only for the recognized parts and not for their inner structure. In Figure 8, we visualize the relative differences in safety and flexibility of all approaches. For annotations based on a concrete syntax tree or an abstract syntax tree, we can guarantee syntactic correctness, while the AST provides more flexibility (see Sec. 4.2). Pseudo-parsing approaches in which the internal structure of recognized elements is opaque (alternative A) can also guarantee safety, but with a significantly reduced flexibility. In contrast, a character based or token based annotation – as with the

Guaranteeing Syntactic Correctness for All Product Line Variants

191

dummy grammar or ‘#ifdef’ preprocessors – provides the highest flexibility (every single character or token can be annotated) but no safety at all. A pseudo-parser that uses a dummy grammar for inner elements (alternative B) lies in the middle, it provides high flexibility, but reduced safety. This discussion shows that a structure given by a grammar is not necessary for an artifact to be handled by CIDE. However, as we experienced in our case studies, when a reasonable grammar is provided, CIDE can ensure syntactic correctness and take advantage of the artifact’s structure to support the developer toward reasonable annotations. The ability to use the artifact’s structure (if available) in every language to ensure syntactic correctness distinguishes CIDE from naive ‘#ifdef’-like preprocessor approaches.

7 Perspective: Language-Independent Checks beyond Syntax We showed how CIDE guarantees that every variant of an SPL is syntactically correct, independent of the artifact’s language. This is helpful to prevent errors when developing SPLs, and in fact it this is more than provided by any current language-independent SPL technologies and tools like Frames/XVCL [18], C preprocessor, pure::variants [4], or Gears [26]. Still, many other kinds of errors are possible in CIDE (see Taxonomy in Sec. 2). In the remainder, we provide an outlook on future work to detect also typing and semantic errors, while still supporting multiple languages. We addressed syntactic correctness in this paper first, because it is a necessary precondition for all these additional checks. Detecting Type Errors. CIDE, as described so far, covers only the syntax of a language, not its type system. In a variant that is syntactically correct, there can still be type errors like dangling method invocations (when only the method declaration but not the invocation has been removed in this variant). Type errors can be detected in many (statically typed) languages by static program analysis. Ideally, an SPL tool should be able to check both syntax and typing in multiple languages in order to detect all compilation errors without actually compiling all variants. Based on CIDE, we implemented type checks for SPLs written in Java that can guarantee that all variants are well-typed once certain checks pass [21,25,23]. If there is a problem like a dangling method reference – even though it might only occur in few variants – an error is reported in CIDE. For Featherweight Java (a subset of Java), we have formalized these type-checks and proved them complete [21]. The basic mechanism of these type-checks is simple and can be generalized to other languages. Type checks are broken down into pairs of code elements that reference each other. For example, a method invocation references a method declaration; a field access references a field declaration; in UML an association references two elements; and so on. For these pairs, CIDE makes sure that in every possible variant in that the referencing element (e.g., method invocation) is present, also the referenced element (e.g., method declaration) is reachable. This condition can be expressed based on the elements’ annotations and the SPL’s feature model and evaluated using a SAT solver as described in [21]. The agenda for providing type checks for multiple languages is similar to the path we took for syntax checks. As first step, we generalized the common parts of type checking

192

C. Kästner et al.

SPLs in a framework in CIDE, similar to the gCIDE model. We successively use this framework to implement type-checks for language plug-ins, currently Java and Bali [3] are implemented. Whether these steps can be automated is an open research question. Detecting Inter-Language Type Errors. Another interesting question, when dealing with multiple languages inside an SPL is inter-language typing. It is often not sufficient to check artifacts from a single language, but annotated artifacts from different languages may reference each other and must be consistent in all variants – e.g., a web service description (WSDL file) in XML format references the implementation in a programming language like C#. As type checks in a single language, checks can be broken down to pairs of elements that reference each other. However, language plug-ins must be able to detect these interlanguage pairs. The main research questions are finding the right abstractions and a suitable polylingual type system (e.g., [15]) for these inter-language checks of SPLs. Advances in inter-language refactorings in Eclipse can be used as starting point [12]. Detecting Semantic Errors. Semantic errors are difficult to detect even without SPL technologies. While there are several approaches of how to formally specify behavior of programs and how to automatically check these specifications, they hardly scale for industrial-size programs and are used only in few scenarios. First steps for verifying SPL behavior against some formal specifications [36] and for using annotations in an SPL for model checking [30] were suggested. To support such verification and model-checking mechanisms in CIDE, we yet again have to find the right abstractions and extend language plug-in mechanism to map the according language-specific conditions to a generalized model for CIDE. Still, there are open research questions how to provide or automate such checks in CIDE and how to scale them for large SPLs. Nevertheless, this is a promising avenue for future work, especially for safety critical SPLs in embedded systems.

8 Conclusion Software product lines (SPLs) usually contain artifacts written in different languages. To handle different artifacts uniformly, current SPL technologies either (a) use an approach that is so general that it works for arbitrary artifacts, but can easily introduce subtle errors for some variants (preprocessors, XVCL, Gears, pure::variants, etc.); or (b) they provide specialized tools for a low number of languages (frameworks, AHEAD, aspects, generators, etc.). Errors that only occur in certain variants of the SPL are a serious problem, as the exploding number of variants makes testing by generating, compiling, and running each variant infeasible. We have shown how CIDE, an SPL development tool that can be considered as disciplined preprocessor, guarantees syntactic correctness for all variants that the SPL can generate by abstracting from the concrete textual representation in a file and using its internal structure. We have shown the underlying principles and generalized them from Java into a language-independent model. In a further step, we have even automated the process of creating language plug-ins from annotated grammar files, so that extending CIDE (including its guarantee for syntactical correctness) for new languages requires minimal human effort.

Guaranteeing Syntactic Correctness for All Product Line Variants

193

We have demonstrated CIDE’s applicability by generating plug-ins for a series of code and non-code languages including Java, C, C#, Haskell, JavaScript, ANTLR, and XML. We have further shown CIDE’s usability for concrete problems in several different case studies that consist of artifacts written in different languages. Acknowledgments. We thank Peter Kim for fruitful comments on earlier drafts of this paper, Marko Rosenmüller and Norbert Siegmund for their help with C and C++, Armin Größlinger for help with Haskell and for providing programs for the Haskell case study, and Sagar Sunkle for releasing the annotated SQL grammars. Apel’s work was funded partly by the German Research Foundation, project #AP 206/2-1. Trujillo’s work was co-supported by the Spanish Ministry of Science & Education under contract TIN2008-06507-C02-02. Batory’s work was supported by NSF’s Science of Design Project #CCF-0724979.

References 1. Apel, S., Kästner, C., Lengauer, C.: FeatureHouse: Language-Independent, Automatic Software Composition. In: Proc. Int’l Conf. on Software Engineering (ICSE) (2009) 2. Apel, S., Leich, T., Saake, G.: Aspectual Feature Modules. IEEE Trans. Softw. Eng. 34(2), 162–180 (2008) 3. Batory, D., Sarvela, J.N., Rauschmayer, A.: Scaling Step-Wise Refinement. IEEE Trans. Softw. Eng. 30(6), 355–371 (2004) 4. Beuche, D., Papajewski, H., Schröder-Preikschat, W.: Variability Management with Feature Models. Sci. Comput. Program. 53(3), 333–352 (2004) 5. Bray, T., et al.: Extensible Markup Language (XML) 1.1. 2nd edn. W3C Recommendation, W3C (2006) 6. Clements, P., Krueger, C.: Point/Counterpoint: Being Proactive Pays Off/Eliminating the Adoption Barrier. IEEE Software 19(4), 28–31 (2002) 7. Clements, P., Northrop, L.: Software Product Lines: Practices and Patterns. Addison-Wesley, Reading (2001) 8. Czarnecki, K., Eisenecker, U.: Generative Programming: Methods, Tools, and Applications. ACM Press, New York (2000) 9. Czarnecki, K., Pietroszek, K.: Verifying Feature-Based Model Templates Against WellFormedness OCL Constraints. In: Proc. Int’l Conf. Generative Programming and Component Eng. (GPCE), pp. 211–220 (2006) 10. Delaware, B., Cook, W., Batory, D.: A Machine-Checked Model of Safe Composition. In: Proc. AOSD Workshop on Foundations of Aspect-Oriented Languages (FOAL), pp. 31–35 (2009) 11. Dörre, J.: Feature-Oriented Composition of XML Artifacts. Master’s thesis, University of Passau, Germany (2009) 12. Fuhrer, R., Keller, M., Kie˙zun, A.: Advanced Refactoring in the Eclipse JDT: Past, Present, and Future. In: Proc. ECOOP Workshop on Refactoring Tools (WRT), pp. 31–32 (2007) 13. Garrido, A.: Program Refactoring in the Presence of Preprocessor Directives. PhD thesis, University of Illinois at Urbana-Champaign (2005) 14. Gosling, J., Joy, B., Steele, G., Bracha, G.: JavaTM Language Specification, 3rd edn. The JavaTM Series. Addison-Wesley Professional, Reading (2005) 15. Grechanik, M., Batory, D., Perry, D.: Design of Large-Scale Polylingual Systems. In: Proc. Int’l Conf. on Software Engineering (ICSE), pp. 357–366 (2004)

194

C. Kästner et al.

16. Huang, S., Zook, D., Smaragdakis, Y.: Statically Safe Program Generation with SafeGen. In: Glück, R., Lowry, M. (eds.) GPCE 2005. LNCS, vol. 3676, pp. 309–326. Springer, Heidelberg (2005) 17. Huang, S.S., Smaragdakis, Y.: Expressive and Safe Static Reflection with MorphJ. In: Proc. Conf. Programming Language Design and Implementation (PLDI), pp. 79–89 (2008) 18. Jarzabek, S., et al.: XVCL: XML-based Variant Configuration Language. In: Proc. Int’l Conf. on Software Engineering (ICSE), pp. 810–811 (2003) 19. Johnson, R.E., Foote, B.: Designing Reusable Classes. Journal of Object-Oriented Programming 1(2), 22–35 (1988) 20. Kang, K., et al.: Feature-Oriented Domain Analysis (FODA) Feasibility Study. Technical Report CMU/SEI-90-TR-21, Software Engineering Institute (1990) 21. Kästner, C., Apel, S.: Type-checking Software Product Lines - A Formal Approach. In: Proc. Int’l Conf. Automated Software Engineering (ASE), pp. 258–267 (2008) 22. Kästner, C., Apel, S., Kuhlemann, M.: Granularity in Software Product Lines. In: Proc. Int’l Conf. on Software Engineering (ICSE), pp. 311–320 (2008) 23. Kästner, C., Apel, S., Trujillo, S., Kuhlemann, M., Batory, D.: Language-Independent Safe Decomposition of Legacy Applications into Features. Technical Report 2/08, School of Computer Science, University of Magdeburg, Germany (2008) 24. Kiczales, G., et al.: Aspect-Oriented Programming. In: Aksit, M., Matsuoka, S. (eds.) ECOOP 1997. LNCS, vol. 1241, pp. 220–242. Springer, Heidelberg (1997) 25. Kim, C.H.P., Kästner, C., Batory, D.: On the Modularity of Feature Interactions. In: Proc. Int’l Conf. Generative Programming and Component Eng. (GPCE), pp. 23–34 (2008) 26. Krueger, C.: Easing the Transition to Software Mass Customization. In: Proc. Int’l Workshop on Software Product-Family Eng., pp. 282–293 (2002) 27. Pohl, K., Böckle, G., van der Linden, F.J.: Software Product Line Engineering: Foundations, Principles and Techniques. Springer, Heidelberg (2005) 28. Pohl, K., Metzger, A.: Software Product Line Testing. Commun. ACM 49(12), 78–81 (2006) 29. Poppleton, M., Fischer, B., Franklin, C., Gondal, A., Snook, C., Sorge, J.: Towards Reuse with Feature-Oriented Event-B. In: Proc. GPCE Workshop on Modularization, Composition and Generative Techniques for Product Line Engineering (2008) 30. Post, H., Sinz, C.: Configuration Lifting: Verification meets Software Configuration. In: Proc. Int’l Conf. Automated Software Engineering (ASE), pp. 347–350 (2008) 31. Simonyi, C.: The Death of Computer Languages, the Birth of Intentional Programming. In: NATO Science Committee Conference (1995) 32. Spencer, H., Collyer, G.: #ifdef Considered Harmful or Portability Experience With C News. In: Proc. USENIX Conf., pp. 185–198 (1992) 33. Staples, M., Hill, D.: Experiences Adopting Software Product Line Development without a Product Line Architecture. In: Proc. Asia-Pacific Software Engineering Conf. (APSEC), pp. 176–183 (2004) 34. Tevanlinna, A., Taina, J., Kauppinen, R.: Product Family Testing: a Survey. SIGSOFT Softw. Eng. Notes 29(2), 12 (2004) 35. Thaker, S., Batory, D., Kitchin, D., Cook, W.: Safe Composition of Product Lines. In: Proc. Int’l Conf. Generative Programming and Component Eng. (GPCE), pp. 95–104 (2007) 36. Uzuncaova, E., Garcia, D., Khurshid, S., Batory, D.: A Specification-Based Approach to Testing Software Product Lines. In: Proc. Europ. Software Engineering Conf./Foundations of Software Engineering (ESEC/FSE), pp. 525–528 (2007) 37. Wile, D.: Abstract Syntax from Concrete Syntax. In: Proc. Int’l Conf. on Software Engineering (ICSE), pp. 472–480 (1997)

A Sound and Complete Program Logic for Eiﬀel Martin Nordio1 , Cristiano Calcagno2 , Peter M¨ uller1 , and Bertrand Meyer1 1 ETH Zurich, Switzerland {Martin.Nordio,Peter.Mueller,Bertrand.Meyer}@inf.ethz.ch 2 Imperial College, London, UK [email protected]

Abstract. Object-oriented languages provide advantages such as reuse and modularity, but they also raise new challenges for program veriﬁcation. Program logics have been developed for languages such as C# and Java. However, these logics do not cover the speciﬁcs of the Eiﬀel language. This paper presents a program logic for Eiﬀel that handles exceptions, once routines, and multiple inheritance. The logic is proven sound and complete w.r.t. an operational semantics. Lessons on language design learned from the experience are discussed. Keywords: Software veriﬁcation, program proofs, operational semantics, Eiﬀel.

1

Introduction

Program veriﬁcation relies on a formal semantics of the programming language, typically a program logic such as Hoare logic. Program logics have been developed for mainstream object-oriented languages such as Java and C#. For instance, Poetzsch-Heﬀter and M¨ uller presented a Hoare-style logic for a subset of Java [18]. This logic includes the most important features of object-oriented languages such as abstract types, dynamic binding, subtyping, and inheritance. However, exception handling is not treated in their work. Huisman and Jacobs [4] developed a Hoare-stype logic which handles abrupt termination. It includes not only exception handling but also break, continue, and return statements. Their logic was developed to verify Java-like programs. Eiﬀel has several distinctive features not present in mainstream languages, for instance, a diﬀerent exception handling mechanism, once routines, and multiple inheritance. Eiﬀel’s once routines (methods) are used to implement global behavior, similarly to static ﬁelds and methods in Java. Only the ﬁrst invocation of a once routine triggers an execution of the routine body; subsequent invocations return the result of the ﬁrst execution. The development of formal techniques for these concepts does not only allow formally verifying Eiﬀel programs, but also allows comparing the diﬀerent concepts, and analyzing which concepts are more suitable to be applied for formal veriﬁcation. The main contributions of this paper are an operational and an axiomatic semantics for Eiﬀel, and soundness and completeness results. The semantics includes: (1) basic instructions such as loops, compounds and assignments; (2) routine invocations; (3) exceptions; (4) once routines, and (5) multiple inheritance. M. Oriol and B. Meyer (Eds.): TOOLS EUROPE 2009, LNBIP 33, pp. 195–214, 2009. c Springer-Verlag Berlin Heidelberg 2009

196

M. Nordio et al.

During this work, we have found that Eiﬀel’s exception mechanism was not ideal for formal veriﬁcation. The use of retry instructions in a rescue clause complicates its veriﬁcation. For this reason, a change in the Eiﬀel exception handling mechanism has been proposed, and will be adopted by a future revision of the language standard. Outline. Section 2 describes the subset of Eiﬀel and its operational semantics. Section 3 presents the Eiﬀel program logic. An example that illustrates the use of the logic is described in Section 4. The soundness and completeness theorems are presented in Section 5. Section 6 discusses related work. Finally, in Section 7 we compare the Eiﬀel features such as exception handling and once routines with other object-oriented languages features such as try-catch instructions and static methods. Proof of soundness and completeness for our logic are presented in a technical report [11].

2

A Semantics for Eiﬀel

2.1

The Source Language

The source language is a subset of Eiﬀel, which includes the most important features of Eiﬀel, although agents are omitted being beyond the scope of this work. The most interesting concepts supported by this subset are: (1) multiple inheritance, (2) exception handling, and (3) once routines. An Eiﬀel program is a sequence of class declarations. A class declaration consist of an optional inheritance clause, and a class body. The inheritance clause supports multiple inheritance, and allows undeﬁning, redeﬁning, and renaming routines. If the routine is redeﬁned, preconditions of subclasses can be weaker, and postconditions can be stronger. A class body is a sequence of attribute declarations or routine declarations. For simplicity, routines are functions that take always one argument and return a result. However, we do not assume that functions are side-eﬀect free, that is, our logic fully handles heap modiﬁcations. The source language also supports once routines. The syntax of the subset of Eiﬀel is presented in Figure 1. Class names, routine names, variables and attributes are denoted by ClassId, RoutineId, VarId, and AttributeId, respectively. The set of variables is denoted by Var ; VarId is an element of Var . The arithmetic functions ∗ and + are deﬁned as usual, and list of denotes a comma-separated list. For space restrictions, we omit attribute reading, attribute writing, and object creation here, but these features are presented in our technical report [11]. Boolean expressions and expressions (BoolExp and Exp) are side-eﬀect-free, and do not trigger exceptions1 . In addition, we write ExpE for the expressions which might raise exceptions. For simplicity, expressions ExpE are only allowed in assignments. This assumption simpliﬁes the presentation of the logic, especially the rules for routine invocation, if then else, and loop instructions. However, the logic could be easily extended. 1

the necessary checks are delegated to the compiler.

A Sound and Complete Program Logic for Eiﬀel

197

One of the design goals of our logic is that programs behave in the same way when contracts are checked at runtime and when they are not. For this reason, we demand that expressions occurring in contracts to be side-eﬀect-free, and to not trigger exceptions. Program ClassDecl Type Parent Undefine Redefine Rename MemberDecl Routine

::= ::= ::= ::= ::= ::= ::= ::= ::=

Instr

::= | | | ::= ::= ::= ::= ::=

Exp, ExpE BoolExp Op Bop CompOp

ClassDecl ∗ class ClassId [inherit Parent+] MemberDecl ∗ end boolT | intT | ClassId | voidT Type [Undefine] [Redefine] [Rename] undefine list of RoutineId redefine list of RoutineId rename list of (RoutineId as RoutineId ) AttributeId Type | Routine RoutineId (VarId : Type) : Type require BoolExp [ local list of (VarId : Type) ] (do | once) Instr [ rescue Instr ] ensure BoolExp end VarId := ExpE | Instr ; Instr | check BoolExp end from invariant BoolExp until BoolExp loop Instr end if BoolExp then Instr else Instr end VarId := VarId .Type : RoutineId (Exp ) Literal | VarId | Exp Op Exp | BoolExp Literal | VarId | BoolExp Bop BoolExp | Exp CompOp Exp + | − | ∗ | // and | or | xor | and then | or else | implies < | > | <= | >= | = | / = Fig. 1. Syntax of the subset of Eiﬀel

2.2

The Memory Model

The state of an Eiﬀel program describes the current values of local variables, arguments, the current object, and the current object store $. A value is either a boolean, or an integer, or the void value, or an object reference. An object is characterized by its class and an identiﬁer of inﬁnite sort ObjId . The data type Value models values; its deﬁnition is the following: data type Value = boolV Bool | intV Int | objV ClassId ObjId | voidV The function τ : Value → Type returns the dynamic type of a value, where τ (boolV b) = boolT ; τ (intV n) = intT ; τ (objV cId oId ) = cId ; and τ (voidV) = voidT .

198

M. Nordio et al.

The state of an object is deﬁned by the values of its attributes. The sort Attribute deﬁnes the attribute declaration T @a where a is an attribute declaration in the class T . We use a sort Location, and a function instvar where instvar (V , T @a) yields the instance of the attribute T @a if V is an object reference, and the object has an attribute T @a; otherwise it yields undef . The datatype deﬁnitions and the signature of instvar are the following: data type Attribute = Type AttributeId data type Location = ObjId AttributeId instvar : Value × Attribute → Location ∪ {undef } The object store models the heap describing the states of all objects in a program at a certain point of execution. An object store is modeled by an abstract data type ObjectStore. We use the object store presented by Poetzsch-Heﬀter and M¨ uller [17,18]. The following operations apply to the object store: os(l ) denotes reading the location l in the object store os; alive(o, os) yields true if and only if object o is allocated in the object store os; new (os, C ) yields a reference of type C to a new object in the store os; os < l := v > updates the object store os at the location l with the value v ; os < C > denotes the store after allocating a new object of type C . An axiomatization of these functions is presented in [17]. ( ) : ObjectStore × Location → Value alive : Value × ObjectStore → Bool new : ObjectStore × ClassId → Value < := > : ObjectStore × Location × Value → ObjectStore < > : ObjectStore × ClassId → ObjectStore 2.3

Operational Semantics

Program states are a mapping from local variables and arguments to values, and from the current object store $ to ObjectStore. The program state is deﬁned as follows: State ≡ Local × Heap Local ≡ VarId ∪ {Current , p, Result , Retry} → Value ∪ {undef } Heap ≡ {$} → ObjectStore Local maps local variables, the receiver object Current (this in Java), arguments, Result , and Retry to values. Arguments are denoted by p. The variables Result and Retry are special variables used to store the result value, and the retry value but they are not part of VarId. For this reason, these variables are included explicitly. For σ ∈ State, σ(e) denotes the evaluation of the expression e in the state σ. Its signature is the following: σ : ExpE → Value ∪ {exc}

A Sound and Complete Program Logic for Eiﬀel

199

The evaluation of an expression e can yield exc meaning that an exception was triggered in e. For example, σ(x /0) yields exc. Furthermore, the evaluation σ(y = 0 ∧ x /y = z ) is diﬀerent from exc because σ ﬁrst evaluates y = 0 and then evaluates x /y = z only if y = 0 evaluates to true. The state σ[x := V ] denotes the state obtained after the replacement of x by V in the state σ. The transitions of the operational semantics have the form: σ, S → σ , χ where σ and σ are states, S is an instruction, and χ is the current status of the program. The value of χ can be either the constant normal or exc. The variable χ is required to treat abrupt termination. The transition σ, S → σ , normal expresses that executing the instruction S in the state σ terminates normally in the state σ . The transition σ, S → σ , exc expresses that executing the instruction S in the state σ terminales with an exception in the state σ . In the following, we present the operational semantics. First, we present the operational semantics for basic instructions. Second, we deﬁne the semantics for routine invocation. Finally, we show the semantics for exception handling and once routines. The operational semantics for exception handling and once routines is one of the contributions of this paper. Basic Instructions. Figure 2 presents the operational semantics for basic instructions such as assignment, compound, conditional, and loop instructions. The operational semantics for assignment, conditional, and loop instructions is standard. Compound is deﬁned by two rules: in rule (2.3) the instruction s1 is executed and an exception is triggered. The instruction s2 is not executed, and the state of the compound is the state produced by s1 . In rule (2.4), s1 is executed and terminates normally. The state of the compound is the state produced by s2 . The check instruction helps to express a property that one believes will be satisﬁed. If the property is satisﬁed then the system does not change. If the property is not satisﬁed then an exception is triggered. The semantics for check consist of two rules: if the condition of the check instruction evaluates to true, then the instruction terminates normally, rule (2.5); otherwise the check instruction triggers an exception, rule (2.6). Routine Invocations. Poetzsch-Heﬀter and M¨ uller [18] have developed an operational and axiomatic semantics for a Java-like languages which handle inheritance, dynamic binding, subtyping and abstract types. However, the source language used in their work has single inheritance. In this section, we extend their logic to support multiple inheritance. Poetzsch-Heﬀter and M¨ uller distinguish between virtual routines and routine implementation. A class T has a virtual routine T:m for every routine m that it declares or inherits. A class T has a routine implementation T @m for every routine m that it deﬁnes (or redeﬁnes). We assume in the following that every invocation is decorated with the virtual method being invoked. The semantics of routine invocations uses two functions: body and impl . The function impl (T , m) yields the implementation of routine m in class T . This implementation could

200

M. Nordio et al.

Assignment Instruction σ(e) = exc (2.1) σ, x := e → σ, exc

σ(e) = exc σ, x := e → σ[x := σ(e)], normal

Compound σ, s1 → σ , exc

σ, s1 → σ , normal

σ, s1 ; s2 → σ , exc

(2.3)

(2.5)

Conditional Instruction σ(e) = true σ, s1 → σ , χ σ, if e then s1 else s2 end → σ , χ σ(e) = false

σ , s2 → σ , χ

σ, s1 ; s2 → σ , χ

Check Instruction σ(e) = true σ, check e end → σ, normal

σ, s2 → σ , χ

σ, if e then s1 else s2 end → σ , χ

(2.2)

(2.4)

σ(e) = false (2.6) σ, check e end → σ, exc

(2.7)

(2.8)

Loop Instruction

σ(e) = true σ, from invariant I until e loop s1 end → σ, normal σ(e) = false

σ, s1 → σ , exc

σ, from invariant I until e loop s1 end → σ , exc σ(e) = false σ, s1 → σ , normal σ , from invariant I until e loop s1 end → σ , χ σ, from invariant I until e loop s1 end → σ , χ

(2.9)

(2.10)

(2.11)

Fig. 2. Operational Semantics for Basic Instructions

be deﬁned by T or inherited from a superclass. The function body yields the instruction constituting the body of a routine implementation. The signatures of these functions are as follows: impl : Type × RoutineId → RoutineImpl ∪ {undef } body : RoutineImpl → Instr The complications of multiple inheritance can be elegantly captured by a revised deﬁnition of impl . While impl (T , m) traverses T ’s parent classes, it can take redeﬁnition, undeﬁnition, and renaming into account. In particular, impl is undeﬁned for deferred routines (abstract methods) or when an inherited routine has been undeﬁned. Figure 3 shows an example of inheritance using the features rename and redefine. Table 1 presents an example of the application of the function impl using the class declarations of Figure 3. For brevity, refer to our technical report [11] for a deﬁnition of impl .

A Sound and Complete Program Logic for Eiﬀel class A feature m do ... end end

201

class B feature m do ... end end

class C inherit A B rename m as n end end class D inherit C redefine m end feature m do ... end end

class E inherit C rename m as m2 end end

Fig. 3. Example of Inheritance using Rename and Redeﬁne Table 1. Example of the Application of the Function impl

impl(A,m) impl(B,m) impl(C,m) impl(C,n)

= = = =

A@m B@m A@m B@m

impl(D,m) impl(D,n) impl(E,m) impl(E,m2) impl(E,n)

= = = = =

D@m B@m undeﬁned A@m B@m

The operational semantics of routine invocation for non-once routines is deﬁned with the following rules: T:m is not a once routine σ(y) = voidV σ, x := y.T:m(e) → σ, exc

σ(y) = voidV

(1)

T:m is not a once routine σ[Current := σ(y), p := σ(e)], body(impl (τ (σ(y)), m)) → σ , χ σ, x := y.T:m(e) → σ [x := σ (Result)], χ (2)

In rule (1), since the target y is Void , the state σ is not changed and an exception is triggered. In rule (2), the target is not Void, thus, the Current object is updated with y, and the argument p by the expression e, and then the body of the routine is executed. To handle dynamic dispatch, ﬁrst, the dynamic type of y is obtained using the function τ . Then, the routine implementation is determined by the function impl . Finally, the body of the routine is obtained by the function body. Note that multiple inheritance is handled using the function impl , which yields the appropriate routine implementation for m. Exception Handling. Exceptions [7] raise some of the most interesting problems addressed in this paper. A routine execution either succeeds - meaning

202

M. Nordio et al.

that it terminates normally - or fails, triggering an exception. An exception is an abnormal event, which occurred during the execution. To treat exceptions, each routine may contain a rescue clause. If the routine body is executed and terminates normally, the rescue clause is ignored. However, if the routine body triggers an exception, control is transferred to the rescue clause. Each routine deﬁnes a boolean local variable Retry (in a similar way as for Result ). If at the end of the clause the variable Retry has value true, the routine body (do clause) is executed again. Otherwise, the routine fails, and triggers an exception. If the rescue clause triggers another exception, the second one takes precedence and it can be handled through the rescue clause of the caller. The Retry variable can be assigned to in either a do clause or a rescue clause. The operational semantics for the exception handling mechanism is deﬁned by rules 3-6 below. If the execution of s1 terminates normally, then the rescue block is not executed, and the returned state is the one produced by s1 (rule 3). If s1 terminates with an exception, and s2 triggers another exception, the rescue terminates in an exception returning the state produced by s2 (rule 4). If s1 triggers an exception and s2 terminates normally but the Retry variable is false, then the rescue terminates with an exception returning the state produced by s2 (rule 5). In the analogous situation with Retry being true, the rescue is executed again and the result is the one produced by the new execution of the rescue (rule 6). Note that the rescue is a loop that iterates over s2 ; s1 until s1 terminates normally or Retry is false. σ, s1 → σ , normal σ, s1 rescue s2 → σ , normal

(3)

σ, s1 → σ , exc σ , s2 → σ , exc σ, s1 rescue s2 → σ , exc

(4)

σ, s1 → σ , exc σ , s2 → σ , normal σ, s1 rescue s2 → σ , exc

¬σ (Retry)

σ , s2 → σ , normal σ, s1 → σ , exc σ , s1 rescue s2 → σ , χ σ, s1 rescue s2 → σ , χ

σ (Retry)

(5)

(6)

Once Routines. The mechanism used in Eiﬀel to access a shared object is once routines. This section focuses on once functions; once procedures are similar. The semantics of once functions is as follows. When a once function is invoked for the ﬁrst time in a program execution, its body is executed and the outcome is cached. This outcome may be a result in case the body terminates normally or an exception in case the body triggers an exception. For subsequent invocations, the body is not executed; the invocations produce the same outcome (result or exception) like the ﬁrst invocation. Note that whether an invocation is the ﬁrst or a subsequent one is determined solely by the routine implementation name.

A Sound and Complete Program Logic for Eiﬀel

203

To be able to develop a semantics for once functions, ﬁnally, we also need to consider recursive invocations. As described in the Eiﬀel ECMA standard [8], a recursive call may start before the ﬁrst invocation ﬁnished. In that case, the recursive call will return the result that has been obtained so far. The mechanism is not so simple. For example the behavior of following recursive factorial function might be surprising: factorial ( i : INTEGER): INTEGER require i>=0 once 4 if i<=1 then Result := 1 else 6 Result := i Result := Result ∗ factorial (i−1) 8 end end 2

This example is a typical factorial function but it is also a once function, and the assignment Result := i ∗factorial (i −1) is split into two separate assignments. If one invokes factorial (3) we observe that the returned result is 9. The reason is that the ﬁrst invocation, factorial (3), assigns 3 to Result . This result is stored for a later invocation since the function is a once function. Then, the recursive call is invoked with argument 2. But this invocation is not the ﬁrst invocation, so the second invocation returns the stored value (in this case 3). Thus, the result of invoking factorial (3) is 3 ∗ 3. If we do not split the assignment, the result would be 0 because factorial (2) would return the result obtained so far which is the default value of Result , 0. This corresponds to a semantics where recursive calls are replaced by Result . To be able to develop a sound semantics for once functions, we need to consider all the possible cases described above. To fulﬁl this goal, we present a pseudocode of once functions. Given a once function m with body b, the pseudo-code is the following: 1 3 5

not m done then m done := true; execute the body b if body triggers an exception e then m exception := e end end if m exception /= Void then throw m exception else Result := m result end if

We assume the variables m done, m exception and m result are global variables, which exist one per routine implementation and can be shared by all invocations of that function. Furthermore, we assume the body of the function sets the result variable m result . Now, we can see more clearly why the invocation of factorial (3) returns 9. In the ﬁrst invocation, ﬁrst the global variable m done is set to false, and then the function’s body is executed. The second invocation returns the stored value 3 because m done is false. To deﬁne the semantics for once functions, we introduce global variables to store the information whether the function was invoked before or not, to store

204

M. Nordio et al.

whether it triggers an exception or not, and to store its result. These variables are T @m done, T @m result , and T @m exc. There is only one variable T @m done per every routine implementation T @m. Descendants of the class T that do not redeﬁne the routine m inherit the routine implementation T @m, therefore they share the same global variable T @m done. Given a once function m implemented in the class T , T @m done returns true if the once function was executed before, otherwise it returns false; T @m result returns the result of the ﬁrst invocation of m; and T @m exc returns true if the ﬁrst invocation of m produced an exception, otherwise it returns false. Since the type of the exception is not used in the exception mechanism, we use a boolean variable T @m exc, instead of a variable of type EXCEPTION. We omit the deﬁnition of a global initialization phase T @m done = false, T @m result = default value, and T @m exc = false. This initialization is performed in the make routine of the ROOT class. The invocation of a once function is deﬁned in four rules (rules 7-10, Figure 4). Rule (7) describes the normal execution of the ﬁrst invocation of a once function. Before its execution, the global variable T @m done is set to true. Then, the function body is executed. We assume here that the body updates the variable T @m result whenever it assigns to Result . Rule (8) models the ﬁrst invocation of an once function that terminates with an exception. The function is executed and terminates in the state σ . The result of the once function m is the state σ where the variable T @m exc is set to true to express that an exception was triggered. In rule 9, the ﬁrst invocation of the once function terminates normally, the remaining invocations restore the stored value using the variable T @m result . In rule 10, the ﬁrst invocation of m terminates with an exception, so the subsequent invocations of m trigger an exception, too.

3

A Program Logic for Eiﬀel

The logic for Eiﬀel is based on the programming logic presented by PoetzschHeﬀter and M¨ uller [18,19]. We take over many of the rules, especially all the language-independent rules such as the rules of consequence. We do not repeat those rules here. We have developed new rules to model exceptions and once routines. Poetzsch-Heﬀter et al. [19] uses a special variable χ to capture the status of the program such as normal or exceptional status. We instead use Hoare triples with two postconditions to encode the status of the program execution. The logic is a Hoare-style logic. Properties androutine bodies are of routines Qn , Qe , where P , Qn , Qe expressed by Hoare triples of the form P S are formulas in ﬁrst order logic, and S is a routine or an instruction. The third component of the triple consists of a normal postcondition (Qn ), and an exceptional postcondition (Qe). The triple P S Qn , Qe deﬁnes the following reﬁned partial correctness property: if S ’s execution starts in a state satisfying P , then (1) S terminates normally in a state where Qn holds, or (2) S throws an exception and Qe holds,

A Sound and Complete Program Logic for Eiﬀel

T @m = impl (τ (σ(y)), m) T @m is a once routine σ(T @m done) = false σ[T @m done := true, Current := y, p := σ(e)], body(T @m) → σ , normal σ, x := y.S:m(e) → σ [x := σ (Result)], normal T @m = impl (τ (σ(y)), m) T @m is a once routine σ(T @m done) = false σ[T @m done := true, Current := y, p := σ(e)], body(T @m) → σ , exc σ, x := y.S:m(e) → σ [T @m exc := true], exc T @m = impl (τ (σ(y)), m) T @m is a once routine σ(T @m done) = true σ(T @m exc) = false σ, x := y.S:m(e) → σ[x := σ(T @m result)], normal T @m = impl (τ (σ(y)), m) T @m is a once routine σ(T @m done) = true σ(T @m exc) = true σ, x := y.S:m(e) → σ, exc

205

(7)

(8)

(9)

(10)

Fig. 4. Operational Semantics for Once Routines

or (3) S aborts due to errors or actions that are beyond the semantics of the programming language, e.g., memory allocation problems, or (4) S runs forever. Boolean Expressions. Preconditions and postcondition are formulas in ﬁrst order logic. Since expressions in assignments can trigger exceptions, we cannot always use these expressions in pre- and postconditions of Hoare triples. For example, if we want to apply the substitution P [e/x ] where e is an ExpE expression, ﬁrst, we need to check that e does not trigger any exception, and then we can apply the substitution. To do this, we introduce a function safe that takes an expression, and yields a safe expression. A safe expression is an expression whose evaluation does not trigger an exception. The deﬁnition of safe expression is the following: Definition 1 (Safe Expression). An expression e is a safe expression if and only if ∀ σ : σ(e) = exc. Definition 2 (Function Safe). The function safe : ExpE → Exp returns an expression that expresses if the input expression is safe or not. The definition of this function is the following: safe : ExpE → Exp safe (e1 oper e2 ) = safe(e1 ) ∧ safe(e2 ) ∧ safe op (oper , e1 , e2 ) safe op : op × ExpE × ExpE → Exp safe op (oper , e1 , e2 ) = if (oper = //) then (e2 = 0) else true

206

M. Nordio et al.

Lemma 1. For each expression e, safe(e) satisfies: – safe(e) is a safe expression – σ(safe(e)) = true ⇔ σ(e) = exc Lemma 2 (Substitution). If the expression e is a safe expression, then: ∀ σ : (σ |= P [e/x ] ⇔ σ[x := σ(e)] |= P ) We deﬁne σ |= P as the usual deﬁnition of |= in ﬁrst order logic but with the restriction that the expressions in P are safe expressions. Signatures of Contracts. Contracts refer to attributes, variables and types. We introduce a signature Σ that represents the constant symbols of these entities. Given an Eiﬀel program, Σ denotes the signature of sorts, functions and constant symbols as described in Section 2.1. Arguments, program variables, and the current object store $ are treated syntactically as constants of Value and ObjectStore. Preconditions and postconditions of Hoare triples are formulas over Σ ∪ {Current , p, Result , Retry} ∪ Var (r ) where r is a routine, and Var (r ) denotes the set of local variables of r . Note that we assume Var (r ) does not include the Result variable and the Retry variable, it only includes the local variables declared by the programmer. Routine preconditions are formulas over Σ ∪ {Current , p, $}, and routine postconditions are formulas over Σ ∪ {Current , p, Result , Retry, $}. We treat recursive routines in the same way as Poetzsch-Heﬀter and M¨ uller [18]. We use sequents of the form A | A where A is a set of routine annotations and A is a triple. Triples in A are called assumptions of the sequent, and A is called the consequent of the sequent. Thus, a sequent expresses that we can prove a triple based on the assumptions about routines. In the following, we present the logic for Eiﬀel. First, we present the basic instructions. Second, we deﬁne the logic for invocation. Finally, we present the semantics for exception handling and once routines. The logic for exception handling and once routines is another contribution of this paper. 3.1

Basic Rules

Figure 5 presents the axiomatic semantics for basic instructions such as assignment, compound, loop, and conditional instructions. In the assignment rule, if the expression e is safe (it does not throw any exception) then the precondition is obtained replacing x by e in P . Otherwise the precondition is the exceptional postcondition. In the compound instruction, ﬁrst the instruction s1 is executed. If s1 triggers an exception, s2 is not executed, and Re holds. If s1 terminates normally, s2 is executed, and the postcondition of the compound is the postcondition of s2 . Note that for conditionals, check instructions, and loops, the expression e is syntactically restricted not to trigger an exception, which simpliﬁes the rules signiﬁcantly.

A Sound and Complete Program Logic for Eiﬀel

207

Assignment Instruction (safe(e) ∧ P [e/x ]) ∨ x := e P , Qe | (¬safe(e) ∧ Qe ) Compound Q n , Re A | Qn s2 Rn , Re A | P s1 A | P s1 ; s2 Rn , Re Conditional Instruction Q , Qe A | P ∧ e s 1 n A | P ∧ ¬e s2 Qn , Qe A | P if e then s1 else s2 end Qn , Qe Check Instruction A | P check e end (P ∧ e ) , (P ∧ ¬e ) Loop Instruction I , Re A | ¬e ∧ I s1 A | I from invariant I until e loop s1 end (I ∧ e) , Re Fig. 5. Axiomatic Semantics for Basic Instructions

3.2

Routine and Routine Invocation Rules

Routine Invocation. Routine invocations of non-once and once routines are veriﬁed based on properties of the the virtual method being called: A | P T:m Qn , Qe (y = Void ∧ P [y/Current, e/p])∨ A | x := y.T:m(e) Qn [x /Result] , Qe (y = Void ∧ Qe )

In this rule, if the target y must not beVoid , the current object is replaced by y and the formal parameter p by the expression e in the precondition P . Then, in the postcondition Qn , Result is replaced by x to assign the result of the invocation. If y is Void the invocation triggers and exception, and Qe holds. To prove a triple for a virtual method T : m, one has to derive the property for all possible implementations, that is, impl (m, T ) and S : m for all sublasses S of T . The corresponding rule is identical to the logic we extend [18]. Routine Implementation. The following rule is used to derive properties of routine implementations from their bodies. Qn , Qe body(T @m) A, {P } T @m {Qn , Qe } | P Qn , Qe A | P T @m Eiﬀel pre and postconditions are often too weak for veriﬁcation, for instance because they cannot contain quantiﬁers. Therefore, our logic allows one to use stronger conditions. To handle recursion, we add the assumption {P } T @m(p) {Qn , Qe } to the set of routine annotations A.

208

3.3

M. Nordio et al.

Exception Handling

The operational semantics presented in Section 2.3 shows that a rescue clause is a loop. The loop body s2 ; s1 iterates until no exception is thrown in s1 , or Retry is false. To be able to reason about this loop, we introduce an invariant Ir . We call this invariant rescue invariant. The rule is deﬁned as follows: P ⇒ Ir s1 Qn , Qe A | Ir A | Qe s2 Retry ⇒ Ir ∧ ¬Retry ⇒ Re , Re A | P s1 rescue s2 Q n , Re

This rule is applied to any routine with a rescue clause. If the do block, s1 , terminates normally then the rescue block is not executed and the postcondition is Qn . If s1 triggers an exception, the rescue block executes. If the rescue block, s2 , terminates normally, and the Retry variable is true then control ﬂow transfers back to the beginning of the routine and Ir holds. If s2 terminates normally and Retry is false, the routine triggers an exception and Re holds. If both s1 and s2 trigger an exception, the last one takes precedence, and Re holds. 3.4

Once Routines

To deﬁne the logic for once routines, we use the global variables T @m done, T @m result , and T @m exc, which store if the once routine was executed before or not, the result, and the exception. Let P be the following precondition, where T M RES is a logical variable: (¬T @m done ∧ P )∨ T @m done ∧ P ∧ T @m result = T M RES ∧ ¬T @m exc ∨ (T @m done ∧ P ∧ T @m exc)

P≡

and let Qn and Qe be the following postconditions: Qn ≡ Qe ≡

∧ ¬T @m exc ∧ T @m done Qn ∨ ( P ∧ Result = T M RES ∧ T @m result = T M RES ) T @m done ∧ T @m exc ∧ (Qe ∨ P )

The rule for once functions is deﬁned as follows: A, {P } T @m {Qn , Qe } |

body(T @m) Qn , Qe P [false/T @m done] ∧ T @m done A | P T @m Qn , Qe

In the precondition of the body of T @m, T @m done is true to model recursive call as illustrated in the example presented in Section 2.3. In the postcondition of the rule, under normal termination, either the function T @m is executed and Qn holds, or the function is not executed since it was already executed and P holds. In both cases, T @m done is true and T @m exc false. In the case an exception is triggered, Qe ∨ P holds.

A Sound and Complete Program Logic for Eiﬀel

4

209

Example

Figure 6 presents an example of the application of the logic. The function safe division implements an integer division which always terminates normally. If the second operand, y, is zero, this function returns the ﬁrst operand, x ; otherwise it returns the integer division x //y. This function is implemented in Eiﬀel using a rescue clause. If the division triggers an exception, this exception is handled by the rescue block setting z to 1 and retrying. To verify this example, we ﬁrst apply the routine implementation rule (Section 3.2). Then, we apply the rescue rule (Section 3.3) to verify the rescue block. The retry invariant is (y = 0 ∧ z = 0) ∨ (y = 0 ∧ (z = 1 ∨ z = 0)). Finally, we verify the body of the do block and the body of the rescue block using the assignment, and the compound rule. 1 safe division (x,y: INTEGER): INTEGER local 3 z : INTEGER do 5 { (y = 0 ∧ z = 0) ∨ (y = 0 ∧ (z = 1 ∨ z = 0)) } Result := x // (y+z)

(y = 0 ⇒ Result = x ) ∧ 7 , (y = 0 ∧ z = 0) (y = 0 ⇒ Result = x /y) ensure 9 zero : y = 0 implies Result = x not zero : y /= 0 implies Result = x // y 11 rescue { y =0∧z =0 } 13 z := 1 { (y = 0 ∧ z = 1), false } 15 Retry := true Retry ∧ (y = 0 ∧ z = 1) , false 17 end Fig. 6. Example of an Eiﬀel source proof

5

Soundness and Completeness

We have proved soundness and completeness of the logic. The soundness proof runs by induction on the structure of the derivation tree for P s Qn , Qe . The completeness proof runs by induction on the structure of the instruction s using a sequent which contains a complete speciﬁcation for every routine implementation T @m. In this section, we state the theorems. The proofs are presented in a technical report [11].

210

M. Nordio et al.

Qn , Qe if and only if: Definition 3. The triple |= P s for all σ |= P : σ, s → σ , χ then – χ = normal ⇒ σ |= Qn , and – χ = exc ⇒ σ |= Qe Theorem 1 (Soundness Theorem) Qn , Qe Qn , Qe ⇒ |= P s | P s Theorem 2 (Completeness Theorem) Qn , Qe ⇒ | P Qn , Qe |= P s s

6

Related Work

Huisman and Jacobs [4] have developed a Hoare-style logic with abrupt termination. It includes not only exception handling but also while loops which may contain exceptions, breaks, continues, returns and side-eﬀects. The logic is formulated in a general type theoretical language and not in a speciﬁc language such as PVS or Isabelle. Oheimb [23] has developed a Hoare-style calculus for a subset of JavaCard. The language includes side-eﬀecting expressions, mutual recursion, dynamic method binding, full exception handling and static class initialization. These logics formalize a Java-like exception handling which is diﬀerent to the exception handling presented in this paper. Logics such as separation logic [20,13], dynamic frames [5,22], and regions [1] have been proposed to solve a key issue for reasoning about imperative programs: framing. Separation logic has been adapted to verify object-oriented programs [14,15,3]. Parkinson and Bierman [14,15] introduce abstract predicates: a powerful means to abstract from implementation details and to support information hiding and inheritance. Distefano and Parkinson [3] develop a tool to verify Java programs based on the ideas of abstract predicates. Logics have been also developed for bytecode languages. Bannwart and M¨ uller [2] have developed a Hoare-style logic a bytecode similar to Java Bytecode and CIL. This logic is based on Poetzsch-Heﬀter and M¨ uller’s logic [17,18], and it supports object-oriented features such as inheritance and dynamic binding. The Mobius project [9] has also developed a program logic for bytecodes. This logic has been proved sound with respect the operational semantics, and it has been formalized in Coq. With the goal of verifying bytecode programs, Pavlova [16] has developed an operational semantics, and a veriﬁcation condition generator (VC) for Java Bytecode. Furthermore, she has shown the equivalence between the veriﬁcation condition generated from the source program and the one generated from the bytecode. Furthermore, M¨ uller and Nordio [10] present a logic for Java and its proof-transformation for programs with abrupt termination. The language considered includes instructions such as while, try-catch, try-finally, throw, and break.

A Sound and Complete Program Logic for Eiﬀel

211

An operational semantics and a veriﬁcation methodology for Eiﬀel has been presented by Sch¨ oller [21]. The methodology uses dynamic frame contracts to be able to address the frame problem, and applies to a substantial subset of Eiﬀel. However, Sch¨oller’s work only presents an operational semantics, and it does not include exceptions. Our logic is based on Poetzsch-Heﬀter and M¨ uller’s work [17,18], which we extended by new rules for Eiﬀel instructions. The new rules support Eiﬀel’s exception handling, once routines, and multiple inheritance. This work is based on our earlier eﬀort [12] on proof-transforming compilation from Eiﬀel to CIL. In this earlier work, we have developed an axiomatic semantics for the exception handling mechanism, and its proof transformation to CIL. This earlier work does not present the operational semantics, and the logic was neither proved sound nor complete. Furthermore, once routines and multiple inheritance were not covered.

7

Lessons Learned

We have presented a sound and complete logic for a subset of Eiﬀel. Here we report on some lessons on programming language design learned in the process. Exception Handling. During the development of this work, we have formalized the current Eiﬀel exception handling mechanism. In the current version of Eiﬀel, retry is an instruction that can only be used in a rescue block. When retry is executed, the control ﬂow is transferred to the beginning of the routine. If the execution of the rescue block ﬁnishes without invoking a retry, an exception is triggered. Developing a logic for the current Eiﬀel would require the addition of a third postcondition, to model the execution of retry (since retry is another way control ﬂow). Thus, we would use Hoare oftransferring P Qn , Qr , Qe where s is an instruction, Qn triples of the form s is the postcondition under normal termination, Qr the postcondition after the execution of a retry, and Qe the exceptional postcondition. Such a formalization would make veriﬁcation harder than with the formalization we use in this paper, because the extra postcondition required by the retry instruction would have to be carried throughout the whole reasoning. In this paper, we have observed that a rescue block behaves as a loop that iterates until no exception is triggered, and that retry can be modeled simply as a variable which guards the loop. Since the retry instruction transfers control ﬂow to the beginning of the routine, a retry instruction has a similar behavior to a continue in Java or C#. Our proposed change of the retry instruction to a variable will be introduced in the next revision of the language standard [8]. Since Eiﬀel does not have return instructions, nor continue, nor break instructions, Eiﬀel programs can be veriﬁed using Hoare triples with only two postconditions. To model object-oriented programs with abrupt termination in languages such as Java or C#, one needs to introduce extra postconditions for

212

M. Nordio et al.

return, break or continue (or we could introduce a variable to model abrupt termination). If we wanted to model the current version of Java, for example, we would also need to add postconditions for labelled breaks and labelled continues. Thus, one would need to add as many postcondition as there are labels in the program. These features for abrupt termination make the logic more complex and harder to use. Another diﬀerence between Eiﬀel and Java and C# is that Eiﬀel supports exceptions using rescue clauses, and Java and C# using try-catch and try-finally instructions. The use of try-finally makes the logic harder as pointed out by M¨ uller and Nordio [10]. The combination of try-finally and break instructions makes the rules more complex and harder to apply because one has to consider all possible cases in which the instructions can terminate (normal, break, return, exception, etc). However, we cannot conclude that the Eiﬀel’s exception handling mechanism is always simpler for veriﬁcation; although it eliminates the problems produced by try-finally, break, and return instructions. Since the rescue block is a loop, one needs a retry invariant. When the program is simple, and it does not trigger many diﬀerent exceptions, deﬁning this retry invariant is simple. But, if the program triggers diﬀerent kinds of exception at diﬀerent locations, ﬁnding this invariant can be more complicated. Note that ﬁnding this retry invariant is more complicated than ﬁnding a loop invariant since in a loop invariant one has to consider only normal termination (and in Java and C#, also continue instructions), but in retry invariants one needs to consider all possible executions and all possible exceptions. Multiple Inheritance. Introducing multiple inheritance to a programing language is not an easy task. The type system has to be extended, and this extension is complex. However, since the resolution of a routine name can be done syntactically, extending Poetzsch-Heﬀter and M¨ uller’s logic [18] to handle multiple inheritance was not a complicated task. The logic was easily extended by giving a new deﬁnition of the function impl . This function returns the body of a routine by searching the deﬁnition in the parent classes, and considering the clauses redeﬁne, undeﬁne, and rename. The experience with this paper indicates that the complexity of a logic for multiple inheritance is similar to a logic for single inheritance. Once Routines. To verify once routines, we introduce global variables to express whether the once routine has been executed before or not, and whether the routine triggered an exception or not. With the current mechanism, the use of recursion in once functions does not increase the expressivity of the language. In fact, every recursive call can be equivalently replaced by Result . However, the rule for once functions is more complicated than it could be if recursion were omitted. Recursive once function would be more interesting if we changed the semantics of once routines. Instead of setting the global variable done before the execution of the body of the once function, we could set it after the invocation. Then the

A Sound and Complete Program Logic for Eiﬀel

213

recursive once function would be invoked until the last recursive call ﬁnishes. Thus, for example, the result of the ﬁrst invocation of factorial (n) would be n! (the function factorial is presented in Section 2.3). Later invocations of factorial would return the stored result. However, this change would not simplify the logic, and we would need to use global variables to mark whether the once function was invoked before or not. Analyzing the EiﬀelBase libraries, and the source code of the EiﬀelStudio compiler, we found that the predominant use of once functions is without arguments, which makes sense because arguments of subsequent calls are meaningless. Even though our rules for once functions are not overly complicated, veriﬁcation of once functions is cumbersome because one has to carry around the outcome of the ﬁrst invocation in proofs. It is unclear whether this is any simpler than reasoning about static methods and ﬁelds [6].

References 1. Banerjee, A., Naumann, J.D.A., Rosenberg, S.: Regional Logic for Local Reasoning about Global Invariants. In: Vitek, J. (ed.) ECOOP 2008. LNCS, vol. 5142, pp. 387–411. Springer, Heidelberg (2008) 2. Bannwart, F.Y., M¨ uller, P.: A Logic for Bytecode. In: Spoto, F. (ed.) Bytecode Semantics, Veriﬁcation, Analysis and Transformation (BYTECODE). ENTCS, vol. 141(1), pp. 255–273. Elsevier, Amsterdam (2005) 3. Distefano, D., Parkinson, M.J.: jStar: Towards Practical Veriﬁcation for Java. In: OOPSLA 2008: Proceedings of the 23rd ACM SIGPLAN conference on Object oriented programming systems languages and applications, pp. 213–226 (2008) 4. Huisman, M., Jacobs, B.: Java program veriﬁcation via a hoare logic with abrupt termination. In: Maibaum, T. (ed.) FASE 2000. LNCS, vol. 1783, pp. 284–303. Springer, Heidelberg (2000) 5. Kassios, I.T.: Dynamic Frames: Support for Framing, Dependencies and Sharing Without Restrictions. In: Misra, J., Nipkow, T., Sekerinski, E. (eds.) FM 2006. LNCS, vol. 4085, pp. 268–283. Springer, Heidelberg (2006) 6. Leino, K.R.M., M¨ uller, P.: Modular Veriﬁcation of Static Class Invariants. In: Fitzgerald, J.S., Hayes, I.J., Tarlecki, A. (eds.) FM 2005. LNCS, vol. 3582, pp. 26–42. Springer, Heidelberg (2005) 7. Meyer, B.: Object-Oriented Software Construction, 2nd edn. Prentice-Hall, Englewood Cliﬀs (1997) 8. Meyer, B. (ed.): ISO/ECMA Eiﬀel standard (Standard ECMA-367: Eiﬀel: Analysis, Design and Programming Language) (June 2006), http://www.ecma-international.org/publications/standards/Ecma-367.htm 9. MOBIUS Consortium. Deliverable 3.1: Byte code level speciﬁcation language and program logic (2006), http://mobius.inria.fr 10. M¨ uller, P., Nordio, M.: Proof-transforming compilation of programs with abrupt termination. In: SAVCBS 2007: Proceedings of the 2007 conference on Speciﬁcation and veriﬁcation of component-based systems, pp. 39–46 (2007) 11. Nordio, M., Calcagno, C., M¨ uller, P., Meyer, B.: Soundness and Completeness of a Program Logic for Eiﬀel. Technical Report 617, ETH Zurich (2009) 12. Nordio, M., M¨ uller, P., Meyer, B.: Proof-Transforming Compilation of Eiﬀel Programs. In: Paige, R., Meyer, B. (eds.) TOOLS-EUROPE 2008. LNBIP, vol. 11. Springer, Heidelberg (2008)

214

M. Nordio et al.

13. O’Hearn, P.W., Yang, H., Reynolds, J.C.: Separation and information hiding. In: POPL 2004, pp. 268–280 (2004) 14. Parkinson, M.J., Bierman, G.: Separation logic and abstraction. In: POPL 2005, vol. 40, pp. 247–258. ACM Press, New York (2005) 15. Parkinson, M.J., Bierman, G.M.: Separation logic, abstraction and inheritance. In: POPL 2008, pp. 75–86. ACM Press, New York (2008) 16. Pavlova, M.: Java Bytecode veriﬁcation and its applications. PhD thesis, University of Nice Sophia-Antipolis (2007) 17. Poetzsch-Heﬀter, A., M¨ uller, P.: Logical Foundations for Typed Object-Oriented Languages. In: Gries, D., De Roever, W. (eds.) Programming Concepts and Methods (PROCOMET), pp. 404–423 (1998) 18. Poetzsch-Heﬀter, A., M¨ uller, P.O.: A Programming Logic for Sequential Java. In: Swierstra, S.D. (ed.) ESOP 1999. LNCS, vol. 1576, pp. 162–176. Springer, Heidelberg (1999) 19. Poetzsch-Heﬀter, A., Rauch, N.: Soundness and Relative Completeness of a Programming Logic for a Sequential Java Subset. Technical report, Technische Universit¨ at Kaiserlautern (2004) 20. Reynolds, J.C.: Separation logic: A logic for shared mutable data structures. In: LICS (2002) 21. Schoeller, B.: Making classes provable through contracts, models and frames. PhD thesis, ETH Zurich (2007) 22. Smans, J., Jacobs, B., Piessens, F.: Implicit dynamic frames. In: Formal Techniques for Java-like Programs (2008) 23. von Oheimb, D.: Analyzing Java in Isabelle/HOL - Formalization, Type Safety and Hoare Logic. PhD thesis, Universit¨ at M¨ unchen (2001)

A Coding Framework for Functional Adaptation of Coarse-Grained Components in Extensible EJB Servers Olivier Caron1 , Bernard Carr´e1, Alexis Muller1 , and Gilles Vanwormhoudt1,2 1

LIFL, UMR CNRS 8022 Lille University France - 59655 Villeneuve d’Ascq cedex [email protected] 2 Institut TELECOM

Abstract. Separation of functional concerns is a major step for managing the complexity of large scale component systems. It allows the identiﬁcation of well deﬁned functional or non-functional dimensions in order to facilitate their assembly, adaptation and ﬁnally their reuse. Current component-based middleware and application servers oﬀer container services that much more facilitate the reuse of the functional core throughout technical concerns, such as persistence, components distribution, transactions, security management, and so on. But they rarely oﬀer facilities for the separation of functional concerns within this core itself. This contribution is dedicated to this question, speciﬁcally within the ﬁeld of ”coarse-grained view components”. In the Information System domain, these components are common and capture ”real-world semantics”, that is a structured set of entities related to a speciﬁc functional concern. Inspired by previous works on Subject-Oriented or View-Oriented Modeling, we retain model templates as good candidates for the deﬁnition of coarse-grained reusable view components. We propose a framework to provide such components onto the EJB technology. It takes advantages of extensible containers and exploits their Aspect-Oriented Programming facilities in order to inject speciﬁc view services compatible with standard J2EE/EJB ones. Keywords: Coarse-grained Components, Separation of Concerns, Views, MDE, Model Templates, Framework, AOP, Component Middleware.

1

Introduction

Separation of concerns is a major step for managing the complexity of large scale component systems. We are interested here in Enterprise Information System (IS) components, and particularly in their entity-centric dimension (as opposed to process-centric components [27]), well exempliﬁed by the EJB technology [16]. Many studies recognized the very distinction between coarse-grained and fine-grained components [13,17,18]. Within the IS domain, coarse-grained components are common and capture “real-world semantics”, that is a structured set of entities related to a speciﬁc functional concern. M. Oriol and B. Meyer (Eds.): TOOLS EUROPE 2009, LNBIP 33, pp. 215–230, 2009. c Springer-Verlag Berlin Heidelberg 2009

216

O. Caron et al.

At the design or modeling level, there exists methodologies that decompose systems according to their functional dimensions such as Catalysis [13], SubjectOriented Design [11], Role-Oriented Design [24], View-Oriented Design [8] or Aspect-Oriented Modeling [25]. All these methodologies help in identifying “model components” that capture well-deﬁned parts of models, dedicated to some functional dimension of the system. Following the Model Driven Engineering (MDE) trend [20], these methodologies concretized their modeling products through “model artefacts”. The next step was to make these model components functionally adaptable, so reusable, thanks to template techniques [9,1,22]. The point here is model parameterization by a structured model of applicable systems, not only by individual parameters as oﬀered by the general UML template notion [6]. Inspired by these previous works on templates at a model level, we explore now the deﬁnition of coarse-grained reusable view components at a technological level. These components call for speciﬁc services in order to facilitate their assembly, adaptation and ﬁnally their reuse. Indeed, current component-based middleware and application servers oﬀer container services that much more ensure non-functional properties such as persistence, component distribution, transactions, security management, and so on. They facilitate the reuse of the functional core throughout these technical concerns. But they rarely oﬀer facilities for the separation of functional concerns within this core itself. We implement such facilities onto the EJB technology. Components have to be “context independent”, so parameterized by some required context consisting in requirements on the applicable Information System. A coding framework is speciﬁed and allows to the deﬁnition of coarse-grained view components through a speciﬁc set of Java annotations. Then it is necessary to oﬀer a connection mechanism to actual IS. We take advantages of advances on extensible containers [14] which exploit Aspect-Oriented Programming (AOP) [23] for the injection of new services. Such facilities are oﬀered by the JBOSS Application Server [19] that we use to oﬀer view connection facilities fully compatible with other standard J2EE ones. This paper is organized as follows. In Section 2, we recall our groundings for separation of functional concerns through the deﬁnition of coarse-grained view components, at the modeling and architectural (patterns) levels. Section 3 details their implementation in the EJB technology. Then Section 4 shows the usage of extensible server AOP facilities to support their assembly and execution. Section 5 presents related works before concluding.

2 2.1

Designing Is with Reusable Views The Model Level

As introduced above parameterized models are used by methodologies in order to model reusable functional concerns or views and allow the construction of a complete system by their application to some primary model [22]. For example,

A Coding Framework for Functional Adaptation

217

let us consider a family of parameterized models dedicated to resource management, inspired from [10], where each model provides a generic functional concern such as stock management, searching or resource allocation. Figure 1 shows the building of a car rental system by the application of these functionalities (parameterized models at the borders) to a primary model capturing primitives of car agency management. Parameterized models are represented by model templates. The template signature depicts the required model. Other model elements are speciﬁc to the template functionality and will be added to the targeted system. They can be properties, operations, associations or classes.

Location name address

at 0..*

Search

Location name address findAll() : Resource[]

Resource key date

Resource key date

at

0..* location(): Location findByKey(key): Resource findByDate(date): Resource[]

<> Agency, Resource -> Client, name -> name, address -> address, key -> name, date-> birthday, at -> cli>

<> Agency,Resource -> Car, name -> name,address -> address, key -> number,date-> date,at -> ac> CarRental <<primary model>>

Stock identifier

Stock identifier capacity add(Resource r) delete(Resource r)

in 0..*

Resource ref

cli

name birthday 0..* phone address

in 0..* Resource ac

ref

0..*

Car <> <Stock -> Agency, Resource -> Car, identifier -> name, ref -> number, in -> ac>

transfer(Stock l)

number date constructor model

<> < Product -> Car, Client -> Client, code -> number, id -> name>

Allocation

Allocation assign_date due_date return_date

Product code nbAssignment(): int free(date begin, date end): bool

Client

Agency name address

StockManager

0..*

Client id

Client to 0..*

cost() : float

Product code

id nbAssignment(): int

Fig. 1. Applying Stock Manager, Searching and Resource Allocation to a Car Rental System

218

O. Caron et al.

Template application (ﬁgured by the ”apply” dependency link) sets the substitution of formal parameters by eﬀective model elements of the target model. The consistency of the model binding is checked according to a set of constraints [6], which allows early veriﬁcation. Details on this application process can be found in [22] where it is justiﬁed concretely and formally, speciﬁcally in regard to composition and ordering properties which master consistent alternative interpretations. Let us only insist on reusability. First, a parameterized model can be applied several times to diﬀerent parts of a same primary one. It is shown on the preceding example where the Search template is applied twice, respectively for cars and clients searching. Second, parameterized models can be reused in distinct projects, applying them to their respective primary model. For example, Figure 2 shows the application of the preceding StockManager model template to a book library management project. BookLibrary Stock identifier StockManager

add(Resource r) delete(Resource r)

ref

0..* <>

0..*

Resource ref

Team name

in 0..* Resource

Stock identifier capacity

in

<<primary model>>

Location identifier address

<Stock -> Location, Resource -> Document, identifier -> identifier, ref -> title, in -> stored>

Book ISBN

transfer(Stock l)

stored

Document title publication_date

Revue number

Researcher name 1..* arrival_date

publishes 0..*

Client name subscription_date

Fig. 2. Applying Stock Manager to a Library System

These capacities are near to component properties and constitute a good background for the furniture of coarsed-grained reusable view components at the technological level. To achieve this, implementation is guided by relevant design patterns at the following architectural level. 2.2

The Architectural Level

Pattern for Supporting Views with Fragmented Representation. In [5], we proposed a pattern for supporting views with fragmented representation. This pattern allows to design a domain entity using a common base fragment and a set of view fragments. This pattern also sketches several mechanisms in order to maintain the consistency of this representation during the entity life cycle. It introduces a conceptual identity for each domain entity, this identity being registered as a key in all its fragments. It is a valuable feature of view representation problems which distinguishes them from general composition semantics.

A Coding Framework for Functional Adaptation

219

Indeed, all the view fragments denote the same conceptual entity and are delegates of it [2], that is, must act ”as if” they were the entity itself within the view context. This phenomenon occurs speciﬁcally in the management of shared associations between views. In fragmented representation, an entity (via its fragments) has several ways to be referenced. It is its base fragment that must be used to reference the entity within the base context while, within a view one, the corresponding view fragment must be used instead. The consequence is that references to entities need to be coerced as references to view fragments when using a shared association. A mapping between base fragments and corresponding view fragments is necessary. It can be done in each view class thanks to the conceptual identity. Figure 3 presents the structure of the pattern. All fragments inherit from EntityPart that deﬁnes characteristics common to fragments: a unique conceptual identity to each entity (entityKey), an access operation to this identity and a method to test if two fragments belong to the same entity. The Base fragment knows View fragments of the entity (link ”views”) and computes the conceptual identities (generateEntityKey). The rest of the method protocol is dedicated to the management of the view fragments life cycle. This ﬁgure also shows the instantiation of the pattern for searching cars, thanks to the application of the Search template (see Figure 1). Agency and Car, which are base fragments, inherit from the Base class. Location and Resource, which are view fragments, inherit from the View class. This ﬁgure also illustrates the pattern principles applied to the ”agency” association, viewed as ”location” between Resource and Location view fragments. The Resource fragment communicates with its related base fragment in order to retrieve the base fragment

for each v in views b = b & v.acceptEntityRemoval() if (b) for each v in views v.removeView()

EntityPart

this.base=base this.entityKey= base.getEntityKey() base.attachView(this)

entityKey getEntityKey() boolean isSameEntity(EntityPart)

Base

View base

attachView(View) detachView(View) removeEntity() static generateEntityKey()

views

1

*

acceptEntityRemoval() removeView() initView(base) pattern example

Car

Agency ...

agency 1

... getAgency() setAgency(Agency)

ag = base.getAgency() agKey = ag.getEntityKey() loc= Location.findByEntityKey(agKey) return loc

Location ... location static Location(Agency) static findByEntityKey(key)

Resource static findByEntityKey(key) getLocation():Location setLocation(Location)

ag=location.getBase() base.setAgency(ag)

Fig. 3. Pattern: Split Representation of Views

220

O. Caron et al.

(one Agency) corresponding to the opposite end of the association. Then, the base fragment is replaced by its corresponding fragment through the conceptual identiﬁcation (Location.findByEntityKey). This pattern allows the design of a system as a set of coarse-grained view components connected to a base, each component being composed of the corresponding view fragments. In order to obtain reusable view components, it is necessary to make them independent from any speciﬁc base. For that, we propose to resort to the Adapter pattern [15,7]. Adapter Pattern for View Reusability . Recall that this pattern consists in deﬁning an abstract adapter that speciﬁes the methods provided for the client and is reﬁned into an concrete adapter that forwards method calls to an adaptee.

Adapter <>

Base attachView(id, adapt) getView(id):Adapter

base 1

1

views *

View

view id

1

Adapter(View)

initView(id,adapt) pattern

Car getNumber()...

Client getName()...

getKey() { return base.getNumber() }

AdResource <>

example Resource getKey()...

getKey()...

AdSearchCar getKey()...

AdSearchClient

getKey() { return base.getName() }

getKey()...

Fig. 4. Adapter Pattern Applied to Views

Figure 4 shows the application of this pattern for view fragments. We can see that each view fragment is linked to an abstract adapter that deﬁnes a set of methods corresponding to its required elements. Each abstract adapter associated with a view fragment is intended to be specialized for achieving its connexion to a base fragment. In order to allow multiple usages of a same reusable view component, we introduce a view identifier for each of its applications. This identiﬁer is shared by all fragments which participate to the resulting view. Methods of the previous pattern now integrate this view identiﬁer. This parameter completes the conceptual identity for fragment identiﬁcation needs. In Figure 4, AdResource plays the role of the abstract adapter for the Resource class of the Search component. This adapter must translate the calls to view fragments depending on the connexion conﬁguration, that is, either cars or clients. Here the concrete adapters AdSearchCar and AdSearchClient do the work. For example, look at the translation of the getKey getter method of Figure 1, depending on the application. This adaptation can be done dynamically, speciﬁcally if reﬂection capacities are available. It is the case in the present contribution as we will see in Section 4.

A Coding Framework for Functional Adaptation

221

These patterns oﬀer architectural choices for rendering separation of functional concerns through reusable coarsed-grained view components at some technological level. In the next section, we propose a coding framework for such components in the J2EE/EJB technology. Then, we will present server facilities for the management of such an architecture.

3

An EJB Coding Framework for Adaptable Coarse-Grained View Components

Figure 5 shows an overview of this framework. On the left, we can see the correspondence between deﬁnitions introduced in Section 2.1 and the framework constructs. On the right, the development cycle is described and contains the four following steps which will be detailed in the next sections : 1. For each concern, the developer writes a view component, that is a Java package with speciﬁc annotations dedicated to the speciﬁcation of required elements. The assembly of these components with a base package is described in an XML ﬁle apply.xml (Section 3.1). 2. The second step consists in checking the consistency of the annotations and generating EJB packages (Section 3.2). 3. After compiling, a standard EJB archive is generated and then deployed in a J2EE/EJB server. This archive includes the apply.xml ﬁle (Section 3.2). 4. At running time, the EJB container uses the standard services (persistence, security, . . . ) augmented by our new dedicated service (Section 4).

... component model template model

... component model EJB Package + annotations

... component model Java Package + annotations 2

check and generate EJBs

apply.xml

<> 1

archive and deploy

Develop, select and compose

ejb

base model

Model Level

4 run

Java Package + annotations Code Level

3

ejb view

standard services

EJB Container

Fig. 5. View EJB Component Development Cycle

222

3.1

O. Caron et al.

Description of Reusable View Components

The description of a reusable view component is derived from the deﬁnition of parameterized models (see Section 2.1). Such a component captures a functional concern and is implemented in a Java package which contains classes linked together by inheritance link and binary associations. A pure Java package can not directly embed neither the associations nor the required model. We are using the standard EJB Java annotation framework in order to code associations and we introduce our own set of annotations in order to specify the required model. According to this standard, attributes and associations are developed via getter/setter methods. Then, EJB annotations are added to the getter methods to fully specify associations (cardinality, inverse role. . . ). Constructs that participate to the required model of a view component have to be annotated with the speciﬁc annotation set presented in Table 1. These annotations are respectively applicable to packages (view component), classes, methods and ﬁnally getters/setters that correspond to attributes and association ends. There is an additional annotation base.Class which will be applied to all classes of the base package. Table 1. Java Annotation Set for View Components Model Element Annotation package view.Component view.Class class view.Attribute attribute view.Method operation view.Role association

Applied Java Element Meaning package view component class required class getter/setter methods required attribute method required method getter/setter method required association

For instance, Figure 6 shows the Java sources of the StockManager view component from Figure 1. In particular, Stock is a required class (annotated with view.Class), contains a required attribute identifier and a required association end resources which corresponds to the association end of the required association in; the attributes className and roleName of the annotation Role refer to the inverse association end1 . In addition to this code, the developer has to write the application assembly via an XML ﬁle, exampliﬁed Figure 7. It can be considered as an XML translation of the apply operator of Section 2.1. Each view section delimited by ... denotes some application of a reusable view component to the base package and enumerates the needed parameter substitutions. The name and linkedTo XML attributes respectively refer to the source element component and the target base one. Note that, following Section 2.2, each view 1

Note that the bodies of the getter/setter methods are not signiﬁcant but required for Java compilation needs. Indeed, as we will see later, they are virtual and will be delegated to the corresponding base elements.

A Coding Framework for Functional Adaptation

223

// Stock.java file package StockManager ; import java.util.Collection ; @view.Class public class Stock { private int capacity ;

// package-info.java file @view.Component package StockManager ;

public Stock() { } public int getCapacity() { return this.capacity ;} public void setCapacity(int value) { this.capacity=value ; }

// Resource.java file package StockManager ; @view.Class public class Resource

public void add(Resource r) { Collection resources=this.getResources(); if (resources.add(r)) { capacity++ ; this.setResources(resources) ; } }

{

public Resource() {} public void transfer(Stock s) { this.getStock().delete(this) ; this.setStock(s) ; s.add(this) ; }

public void delete(Resource r) { Collection resources=this.getResources(); if (resources.remove(r)) { capacity-- ; this.setResources(resources) ; } } // Model Required Section @view.Attribute public String getIdentifier() { return null ; } public void setIdentifier(String value) {}

// Model Required Section @view.Attribute public String getRef() { return null ; } public void setRef(String value) { } @view.Role(className="StockManager.Stock", roleName="resources") public Stock getStock() { return null ; } public void setStock(Stock s) { }

@view.Role(className="StockManager.Resource", roleName="stock") public Collection getResources() { return null ; } public void setResources(Collection value){} ;

} }

Fig. 6. The StockManager View Component

application is identiﬁed by an id XML attribute, which allows to disambiguate multiple application of a same reusable view component, for example SearchClient/SearchCar. 3.2

Checking and Archiving Steps

The set of added annotations embedded into the Java code must be consistent according to the following rules: – All classes annotated with view.Class must reside in a package annotated with view.Component. – Only classes annotated with view.Class can have methods annotated by view.Method, view.Role or view.Attribute. – An association must be speciﬁed by two ends annotated with view.Role whose mandatory attributes (className and roleName) must reference to each other (see getResources() / getStock() in Figure 6). These rules are veriﬁed via a checking tool which inspects the annotated code of each view component. One also need to validate the assembly description ﬁle. The XML document structure is validated according to the rules described in the DTD ﬁle. Additional

224

O. Caron et al.

Fig. 7. An XML Assembly File

constraints must be checked like the veriﬁcation that java elements denoted by name and linkedTo XML attributes respectively refer existing view elements and base elements and are type compatible. Once this checking step achieved, sources are automatically transformed in order to make them fully EJB compatible. Classes annotated with view.Class (respectively base.Class) inherit from functionalAspect.View class (respectively functionalAspect.Base). These classes are an EJB implementation of the patterns of Section 2.2. Let’s outline that the implementation policy has to ensure that view properties (attributes and roles) are virtual. Indeed, in conformance with the view pattern, access to these properties must be delegated to the connected base. So, the corresponding getter methods will be annotated by javax.persistence.Transient. Finally, packages can be compiled, archived with the assembly descriptor ﬁle and then deployed into an EJB server (step 3 of the development cycle).

4

Server Facilities

To support the assembly of coarse-grained view components with primary components at the execution level, we propose to extend the capacity of EJB servers with a new service. The role of this service consists in ensuring the proper execution of conceptual objects throughout their views according to the architectural choices made in Section 2.2. So, it has to address the following requirements : – support for adapting the behavior of view components to map their required elements with elements of primary system components as speciﬁed by the assembly XML descriptor. – support for managing conceptual objects made of base fragments and view fragments. As a consequence, the EJB life cycle of these fragments are closely linked (both at creation and deletion time). – support for multiple application of views.

A Coding Framework for Functional Adaptation

225

We present here an implementation of this view service using dynamic interception capacities of EJB servers for adaptation needs. These capacities are oﬀered by extensible EJB servers such as the JBOSS/J2EE application server. JBoss is an open-ended middleware, in the sense that developers can extend middleware services by aspect introduction. The core technology is the JBoss/AOP framework [23] that intensively uses Java annotations. Adding a new service consists in adding a new aspect. Each aspect is made of its implementation and an XML description which declares necessary pointcuts and advices. Figure 8 shows the XML declaration of our new aspect.

Fig. 8. Aspect Declaration of the New Service

The pointcuts allow to intercept method calls to a view component, identiﬁed by our set of annotations. The advices consist in installing a JBOSS-AOP interceptor class called aspect.ViewInterceptor. This interceptor allows to execute some code around a method identiﬁed by one of the previous pointcuts. This code performs the mapping between required elements and provided ones using the assembly descriptor and Java introspection. This achieves a dynamic implementation of the Adapter pattern (see Section 2.2) thanks to reﬂexive invocation. As a consequence, the developer has no additional work other than using the previous annotation framework and writing the assembly descriptor ﬁle. Figure 9 illustrates the execution of the interception mechanism applied to the access to the view association ”location” like in Section 2.2. The getLocation method of the Search.Resource class is invoked. As this method is annotated with view.Role, the bindRole method of the ViewInterceptor interceptor is called. The interceptor retrieves the view identiﬁer (getId()) and the name of the corresponding association role of the base class in the assembly XML descriptor ﬁle, here ”agency”. The getAgency() method can now be retrieved (using Java reﬂection) and dynamically invoked on this object, returning its agency. Finally, it is necessary to retrieve the corresponding view fragment (”location”) through the getView(id) call which terminates the substitution process. This example shows that this delegating behaviour is completely transparent for the developer when a view object is connected to a base object.

226

O. Caron et al.

:location

client:

:Resource

base:Car

:ViewInterceptor

ag:Agency

:XMLBinding

<> bindRole

getLocation()

getId() id getAttributeName(id, "Resource","location") "agency"

getBase() base

<> getAgency() ag getView(id)

location

location

...

Fig. 9. a Sequence Diagram of the View Service

1. Agency ag1 = (Agency) em.find(Agency.class,"key-ag1") ; 2. Search.Location location = (Search.Location) ag1.getView("SearchCar") ; 3. Collection<Search.Resource> resources = location.findAll() ; 4. // resources are view fragments of cars 5. 6. location =(Search.Location) ag1.getView("SearchClient") ; 7. resources = location.findAll() ; 8. // resources are view fragments of clients 9. ( (Search.Resource) resources.toArray()[0] ).getLocation() ; 10. 11. Car newCar=new Car() ; // => creation of view fragments ; 12. entityManager.persist(aCar) ; // the entity manager now 13. // manages automatically 14. // view fragments 15. StockManager.Stock stock = ag1.getView("StockManagerCarRental") ; 16. StockManager.Resource r = newCar.getView("StockManagerCarRental") ; 17. stock.add(r) ; // now ag1 manages newCar

Fig. 10. Using Views, an excerpt

The code excerpt of Figure 10 now illustrates how to use views. At the beginning of this code sequence, we suppose there already exists an entity Agency. By using the standard EJB entity manager em, we retrieve the existing agency ag1 (line 1). Lines 2-3 and lines 6-7 are identical except the view considered so that, the findAll method applied to the agency returns a collection of resources which are either cars or clients. Figure 9 details the execution of line 9 (treatment of a view association)

A Coding Framework for Functional Adaptation

227

The rest of the code creates a new base object and we use the StockManager component to add this car to the previous agency if its capacity allows it (this capacity control is a functionality provided by the view component). Line 11 instantiates a car, which calls the functionalAspect.Base super-class constructor that creates view fragments using the assembly descriptor. When the EJB entity manager persists an instance of the base class (line 12), all the connected view fragments are automatically managed by the entity manager. This mechanism is done because the EJB association between base and views is speciﬁed with the ”cascade persist” standard mode. In this example, we show that the persistent EJB life cycle between view fragments and base fragment are closely linked. Then, the view StockManagerCarRental is used (lines 15-17).

5

Related Works

We ﬁnd studies on separation of functional concerns through reusable coarsegrained components which also starts from initial works on generic models. [13] has dealt in some detail with various implementations of reusable pluggable code components based on their framework notion. Diﬀerent ways are compared (template classes, class hierarchies,. . . ). [12] proposes a mapping between ThemeUML and AspectJ. The solution is a pure AOP one where each model becomes an aspect that one can weave into a primary base. However, these studies have not been directly explored within the enterprise component server sphere. [3] proposes a MDA process which transforms PIM models to EJB components. The MDA process generates speciﬁc glue in order to reuse EJB components. This work oﬀers a set of primitives for general model composition but does not deal with speciﬁc properties of functional concerns modeling. As far as coding is concerned, most of these approaches are model driven and generative. We ourselves already used a generative strategy to obtain EJB [5] implementation of our model templates. However, the generated EJB components were hard-wired, that is, speciﬁc to a particular system and then not reusable enough. Though the present framework is usuable in a standalone manner, it can be a valuable target technology for MDE environments. We are integrating such a targeting strategy into our proper UML CASE tool ”CocoaModeler” [22] using ”EMFScript” [26] for transformation needs. From a middleware point of view, there are a lot of works on extensible platforms [14,4]. They concentrate on non-functional concerns and facilitate the reuse of the functional core of the components throughout the corresponding services. The present work is dedicated to the structuring of this core itself according to functional (real world domain) concerns. All these works are complementary contributions to the quest for rich IS components, fully reusable whatever the concerns. More, despite the fact that we concentrated here on IS component servers, solutions are somehow generalizable to other domains, components architectures and middleware, like Corba and Fractal components that we have already experimented [22].

228

6

O. Caron et al.

Conclusion

This work provides separation of functional concerns in EJB servers by the deﬁnition of coarse-grained view components. It takes advantages of AOP facilities of an open-ended server to extend the EJB framework with new dedicated services. This extension does adopt the same writing idioms as standard EJB components: package structuring, an annotation language and XML descriptors for assembly. So it conforms to the EJB developer practice. Reusing view components is facilitated because one has only to describe the architecture assembly consisting in the speciﬁcation of their connections to a base EJB standard package. The resulting architecture is compatible with the EJB standard and takes advantages of all existing J2EE services: persistence, transaction and security. As far as component architectures and middleware are concerned, it is a contribution to the reusability of coarsed-grained IS components complementary to researches on separation of non-functional concerns. Though it is a result that this framework is usable in a standalone manner by EJB developers and architects, it is also a good target technology for MDE environments. We are integrating such a targeting strategy into our MDE environment in order to automatize the transformation from modeling separation of functional concerns to platform speciﬁc code. Lastly, we have experimented the framework through a fully dynamic implementation that makes extensive use of reﬂection. However, reﬂection ﬂexibility comes at the expense of performances. In future works, we will manage this issue by studying optimization techniques for reﬂection and experimenting more generative implementation strategies as we already studied for CORBA middleware [21].

References 1. Baniassad, E., Clarke, S.: Theme: An approach for aspect-oriented analysis and design. In: ICSE 2004: Proceedings of the 26th International Conference on Software Engineering, pp. 158–167. IEEE Computer Society Press, Los Alamitos (2004) 2. Bardou, D., Dony, C.: Split Objects: a Disciplined Use of Delegation within Objects. In: Proceedings of the 11th Conference on Object-Oriented Programming Systems, Languages, and Applications (OOPSLA 1996), San Jose, California, USA, October 1996, pp. 122–137. ACM Press, New York (1996) 3. Bouzitouna, S., Gervais, M.P., Blanc, X.: Model Reuse in MDA. In: International Conference on Software Engineering Research and Practice. CSREA Press (2005) 4. Bruneton, E., Riveill, M.: An architecture for extensible middleware platforms. Software - Practice and Experience 31(13), 1237–1264 (2001) 5. Caron, O., Carr´e, B., Muller, A., Vanwormhoudt, G.: A Framework for Supporting Views in Component Oriented Information Systems. In: Konstantas, D., L´eonard, M., Pigneur, Y., Patel, S. (eds.) OOIS 2003. LNCS, vol. 2817, pp. 164–178. Springer, Heidelberg (2003) 6. Caron, O., Carr´e, B., Muller, A., Vanwormhoudt, G.: An OCL Formulation of UML 2 Template Binding. In: Baar, T., Strohmeier, A., Moreira, A., Mellor, S.J. (eds.) UML 2004. LNCS, vol. 3273, pp. 27–40. Springer, Heidelberg (2004)

A Coding Framework for Functional Adaptation

229

7. Caron, O., Carr´e, B., Muller, A., Vanwormhoudt, G.: Mise en oeuvre d’aspects fonctionnels r´eutilisables par adaptation. Revue L’objet, Programmation par Aspects, Hermes Ed. 11(3), 105–118 (2005) 8. Caron, O., Carr´e, B., Debrauwer, L.: Contextualization of OODB schemas in CROME. In: Database and Expert Systems Applications, pp. 135–149 (2000) 9. Clark, T., Evans, A., Kent, S.: A Metamodel for Package Extension with Renaming. In: J´ez´equel, J.-M., Hussmann, H., Cook, S. (eds.) UML 2002. LNCS, vol. 2460, pp. 305–320. Springer, Heidelberg (2002) 10. Clarke, S.: Extending standard UML with Model Composition Semantics. In: Science of Computer Programming, vol. 44, pp. 71–100. Elsevier Science, Amsterdam (2002) 11. Clarke, S., Harrison, W., Ossher, H., Tarr, P.: Subject-Oriented Design: Towards Improved Alignment of Requirements, Design and Code. In: Procs. of the Object-Oriented Programming Systems, Languages and Applications Conference (OOPSLA), Denver (1999) 12. Clarke, S., Walker, R.J.: Towards a Standard Design Language for AOSD. In: AOSD 2002: Proceedings of the 1st International Conference on Aspect-Oriented Software Development, pp. 113–119. ACM Press, New York (2002) 13. D’Souza, D., Wills, A.: Objects, Components and Frameworks With UML: The Catalysis Approach. Addison-Wesley, Reading (1999) 14. Fleury, M., Reverbel, F.: The JBoss extensible server. In: Endler, M., Schmidt, D.C. (eds.) Middleware 2003. LNCS, vol. 2672, pp. 344–373. Springer, Heidelberg (2003) 15. Gamma, E., Helm, R., Johnson, R., Vlissides, J., Booch, G.: Design Patterns: Elements of Reusable Object-Oriented Software. Addison-Wesley Professional Computing, USA (1995) 16. Grundy, J., Patel, R.: Developing Software Components with the UML, Enterprise Java Beans and Aspects. In: ASWEC 2001: Proceedings of the 13th Australian Conference on Software Engineering, p. 127. IEEE CS, Los Alamitos (2001) 17. Helton, D.: Coarse-Grained Components as an Alternative to Component Frameworks. In: ECOOP Workshop on Component Oriented Programming, Lisbon, Portugal (1999) 18. Helton, D.: Closing the Gap between Business Functions and Software Components in Distributed Enterprise Systems. In: Weck, W., Reussner, R., Szyperski, C. (eds.) Proceedings of the 13th International ECOOP Workshop on Component-Oriented Programming (WCOP), Karlsruhe, Germany (October 2008) 19. JBoss Aspect-Oriented Programming, http://labs.jboss.com/portal/jbossaop 20. Kent, S.: Model Driven Engineering. In: Butler, M., Petre, L., Sere, K. (eds.) IFM 2002. LNCS, vol. 2335, pp. 286–298. Springer, Heidelberg (2002) 21. Muller, A.: Construction de syst`emes par application de mod`eles param´etr´es. PhD thesis, LIFL, Universit´e de Lille (2006) 22. Muller, A., Caron, O., Carr´e, B., Vanwormhoudt, G.: On Some Properties of Parameterized Model Application. In: Hartman, A., Kreische, D. (eds.) ECMDA-FA 2005. LNCS, vol. 3748, pp. 130–144. Springer, Heidelberg (2005) 23. Pawlak, R., Retaill´e, J.-P., Seinturier, L.: Foundations of AOP for J2EE Development. APress (2005) 24. Reenskaug, T., Wold, P., Lehne, O.A.: Working with Objects: The OORam Software Engineering Method. Prentice-Hall, Inc., Englewood Cliﬀs (1995)

230

O. Caron et al.

25. France, R., Georg, G., Ray, I.: Supporting Multi-Dimensional Separation of Design Concerns. In: AOSD Workshop on AOM: Aspect-Oriented Modeling with UML (March 2003) 26. Tombelle, C., Vanwormhoudt, G.: Dynamic and Generic Manipulation of Models: From Introspection to Scripting. In: Nierstrasz, O., Whittle, J., Harel, D., Reggio, G. (eds.) MoDELS 2006. LNCS, vol. 4199, pp. 395–409. Springer, Heidelberg (2006) 27. Wand, Z., Xu, X., Zhan, D.: A Survey of Business Component Identiﬁcation Methods and Related Techniques. International Journal of Information Technology 2(4) (2005)

A Leasing Model to Deal with Partial Failures in Mobile Ad Hoc Networks Elisa Gonzalez Boix , Tom Van Cutsem , Jorge Vallejos , Wolfgang De Meuter, and Theo D’Hondt Programming Technology Lab - Vrije Universiteit Brussel - Belgium {egonzale,tvcutsem,jvallejo,wdmeuter,tjdhondt}@vub.ac.be

Abstract. In mobile ad hoc networks (MANETs) many partial failures are the result of temporary network partitions due to the intermittent connectivity of mobile devices. Some of these failures will be permanent and require application-level failure handling. However, it is impossible to distinguish a permanent from a transient failure. Leasing provides a solution to this problem based on the temporal restriction of resources. But to date no leasing model has been designed speciﬁcally for MANETs. In this paper, we identify three characteristics required for a leasing model to be usable in a MANET, discuss the issues with existing leasing models and then propose the leased object references model, which integrates leasing with remote object references. In addition, we describe an implementation of the model in the programming language AmbientTalk. Leased object references provide an extensible framework that allows programmers to express their own leasing patterns and enables both lease holders (clients) and lease grantors (services) to deal with permanent failures. Keywords: mobile ad hoc networks, partial failures, leasing, remote object references, language design.

1

Introduction

The recent progress in the ﬁeld of wireless technology has proliferated a growing body of research that deals with mobile ad hoc networks (MANETs): networks composed of mobile devices connected by wireless communication links with a limited communication range. Such networks have two discriminating properties, which clearly set them apart from traditional, ﬁxed computer networks [13]: intermittent connectivity of the devices in the network (called the volatile connections phenomenon) and lack of any centralized coordination facility (called the

Author funded by Prospective Research for Brussels program of the Institute for the encouragement of Scientiﬁc Research and Innovation of Brussels (IWOIB-IRSIB). Postdoctoral Fellow of the Research Foundation - Flanders (FWO). Author supported by the VariBru project of the ICT Impulse Programme of IWOIB-ISRIB and the MoVES project of the Interuniversity Attraction Poles Programme of the Belgian State, Belgian Science Policy.

M. Oriol and B. Meyer (Eds.): TOOLS EUROPE 2009, LNBIP 33, pp. 231–251, 2009. c Springer-Verlag Berlin Heidelberg 2009

232

E. Gonzalez Boix et al.

zero infrastructure phenomenon). The volatile connections phenomenon states that a disconnection should not be treated as a “failure” by default: due to the limited communication range of wireless technology, devices may move out of earshot unannounced. The resulting disconnections are usually transient : the devices may meet again requiring their connection to be re-established and allowing their collaboration to be continued where they left oﬀ. The zero infrastructure phenomenon states that it is more diﬃcult to rely on server infrastructure (e.g. a name server for service discovery) since devices spontaneously join and disjoin the network due to their physical mobility. Our research focuses on distributed programming language support for mobile ad hoc networks. In this paper, we explore support to deal with the eﬀects engendered by partial failures. Due to the above mentioned phenomena, it can be expected that many partial failures in MANETs are the result of temporary network partitions. However, not all network partitions are transient, e.g. a remote device has crashed or has moved out of the wireless communication range and does not return. Such permanent failures should also be dealt with by means of compensating actions, e.g. application-level failure handling code. Because it is impossible to distinguish a transient from a permanent failure [15], some arbitrary criteria should be agreed upon so that a device can determine when the logical communication with another remote device has terminated. Leasing [5] provides a solution to this problem based on the temporal restriction of resources [15]. A lease denotes the right to access a resource for a speciﬁc duration that is negotiated by the owner of a resource and a resource claimant (called the lease grantor and lease holder, respectively) when the access is ﬁrst requested. The advantage of leasing is that allow both lease grantor and holder to distinguish a transient from a permanent failure by approximating permanent failures as disconnections that exceed the agreed time interval. However, to date no leasing model has been designed to operate in a mobile ad hoc network setting where leasing needs to (1) be combined with computational models that deal with transient failures, (2) provide diﬀerent leasing patterns for the diﬀerent kinds of collaboration that can be set up in MANETs, and (3) allow both lease holders and grantors to deal with permanent failures. This paper proposes a leasing model specially designed for MANETs. Our contribution lies in integrating leasing as a special kind of remote object reference, called leased object references, which tolerate both transient and permanent failures. The leased object references model incorporates diﬀerent leasing patterns at its heart and allows programmers to schedule clean-up actions at both client and server side of a leased reference upon expiration. In addition, we describe a concrete instantiation of such a model in a distributed programming language, called AmbientTalk [13], as an extensible framework in which many leasing patterns can be expressed.

2

Leasing in Mobile Ad Hoc Networks

In this section, we discern a number of criteria that need to be exhibited by a leasing model for dealing with partial failures in MANETs and describe how

A Leasing Model to Deal with Partial Failures in Mobile Ad Hoc Networks

233

existing leasing models for distributed computing fail to deal with them. We derive these criteria from the analysis of an illustrative ad hoc networking application. Throughout this paper, we assume an object-oriented system where devices can be abstracted as a container for a set of objects implementing some functionality. Objects can be exported in the network either explicitly or implicitly by passing them as parameter or return value in a message sent to a remote object. We denote such remotely accessible objects as service objects. Service objects can be referenced from other machines by means of remote object references. 2.1

Running Example: The Mobile Music Player

Consider a music player running on mobile devices. The music player contains a library of songs. When two people using the music player enter one another’s personal area network (deﬁned by for example the bluetooth communication range of their cellular phones), the music players set up a so-called ad hoc network and exchange their music library’s list (not necessarily the songs themselves). After the exchange, the music player can calculate the percentage of songs both users have in common. If this percentage exceeds a certain threshold, the music player can e.g. warn the user that someone with a similar musical taste is nearby. MusicPlayerServiceObject@ device A

MusicPlayerServiceObject@ device B openSession() session

session@ device B

uploadSong(song) 'ok endExchange()

Fig. 1. The music library exchange protocol

Figure 1 gives a graphical overview of the music library exchange protocol modeled via a distributed object-oriented system where communication between devices is asynchronous. The ﬁgure depicts the stages of the protocol from the point of view of the music player on the device A. In fact, this protocol is executed simultaneously on both devices. Once both devices have discovered each other, the music player running on A asks the remote peer B to start a session to exchange its library index by sending an openSession message. In response to it, the remote peer returns a new session object which implements methods that allow the remote music player to send song information (uploadSong) and to signal the end of the library exchange (endExchange).

234

2.2

E. Gonzalez Boix et al.

Analysis

This section describes a set of criteria that need to be exhibited by a leasing model to be used for dealing with partial failures in MANETs. While some of the above described criteria can be observed in diﬀerent leasing models for distributed computing platforms and frameworks, to the best of our knowledge, no single leasing model exhibits all of the criteria presented in this section. Leasing an Intermittent Connection. A lease denotes a time restriction on the logical connection between lease holder and grantor. At the software level, a logical connection is represented by a communication link. Because of the volatile connections phenomenon in MANETs, communication links are often intermittent: devices often disconnect for an unknown period of time and then reconnect. However, that does not imply that the logical connection should be terminated. In our running example, once the service objects that represent the music player application have discovered one another, they need to set up a session to exchange their music libraries. Such a session is leased, such that both music players can gracefully terminate the exchange process in the face of a persistent disconnection. However, if a user moves temporarily out of range, the resulting transient disconnection should not cause the exchange to fail immediately as the user may reconnect. A leasing model for MANETs must take this into account: the disconnection of a device does not indicate that resources associated with the logical connection can already be cleared, since the communication link may be restored later. Leasing Patterns. Mobile ad hoc networking applications are conceived as a collection of several devices setting up a collaboration. As diﬀerent kinds of collaboration can be set up, diﬀerent kinds of leasing patterns are also possible. In our running example, devices collaborate in the exchange of their music library indexes. As long as the exchange is active, i.e. uploadSong messages are received, the session should remain active. The session could be thus exported using a lease which is automatically renewed each time it receives a message. The lease should be then revoked either explicitly when a client sends the endExchange message to indicate the end of the library exchange, or implicitly if the lease time has elapsed. Other collaborations may involve objects adhering to a single call pattern such as callback objects. For example, in asynchronous message passing schemes, callback objects are often passed along with a message in order for service objects to be able to return values. These callback objects are typically remotely accessed only once by service objects with the computed return value. In this case, a lease which is automatically revoked upon a method call is more suitable. A leasing model for MANETs should be ﬂexible enough to allow developers to specify diﬀerent leasing patterns to manage the lifetime of leases. Symmetric Expiration Handling. Leasing allows lease grantors to remain in control of the resource by maintaining the right to free the resource once the lease expires. In MANETs, both communicating parties should be aware of a lease expiration since it allows them to detect a permanent disconnection. Once a lease expires, both lease holder and grantor should be able to

A Leasing Model to Deal with Partial Failures in Mobile Ad Hoc Networks

235

properly react and do some clean-up actions in order to gracefully terminate their collaboration. In our running example, the session object is clearly only relevant within the context of a single music library exchange. If the exchange cannot be completed (e.g. due to a persistent network partition), the session object and the resources allocated during the session, e.g the partially uploaded library index, should be eventually reclaimed. A leasing mechanism for MANETs should allow both lease holder and lease grantor to deal with the termination of their logical connection. 2.3

Related Work

Leases were originally introduced as a fault-tolerant technique in the context of distributed ﬁle cache consistency [5]. Jain and Kircher introduced the concept of leasing as a software design pattern to simplify resource management in [6]. In distributed object-oriented systems, leasing has been used to describe the lifetime of remote objects in frameworks like Java RMI and .NET Remoting [10], and as a general abstraction for resource management in platforms like CORBA [2] and Jini [14]. In this section, we further evaluate these leasing mechanisms in the light of the criteria for a leasing model in MANETs. Java RMI. In Java RMI leases are tightly coupled to the distributed garbage collector (DGC) and are used as a way to manage the lifetime of remote objects in a fault-tolerant fashion. Although Java RMI integrates leasing with remote references, leases are only used to reclaim unused connected remote references. In fact, its leasing model is combined with a synchronous communication model (RPC) which does not decouple objects in time or synchronization [3], which makes it unsuitable for MANETs. Leases are transparent to the programmer as a part of the DGC and the lease duration is controlled by means of a system property. If developers need to deviate from the default leasing behaviour, the system provides an interface to the DGC based on so-called dirty and clean calls. Leasing patterns need to be built on top of these low-level operations. For example, automatic renewal of leases can be only accomplished by making lowlevel dirty calls on the remote references. Expiration handling is not provided upon lease expiration in Java RMI. If a client attempts to access a service object whose lease expired, an exception is raised. This allows clients to schedule some clean-up actions in the exception handler, but forces them to install exception handlers on every remote call. .NET Remoting framework. The .NET Remoting framework incorporates leasing in combination with the concept of sponsorship for managing the lifetime of remote objects [9]. Sponsors are third-party objects which are contacted by the framework when a lease expires to check if that party is willing to renew the lease. Clients can register a sponsor on a lease and thus decide on the lifetime of server objects. Similar to JavaRMI, the .NET Remoting framework leases are used to reclaim unused connected remote references. In contrast to the simplicity of the language constructs oﬀered in JavaRMI, the .NET Remoting

236

E. Gonzalez Boix et al.

framework incorporates a leasing pattern at the heart of its design. Leases are automatically extended on every call on the remote object by the time speciﬁed in the RenewOnCallTime property. If that property is not set, lease renewal can be achieved by registering a sponsor. Variations on the integrated pattern need to be built on top of sponsor and lease interface abstractions. The lease interface provides methods for overriding some leasing properties (e.g. RenewOnCallTime), renewing the lease, and the registration of sponsors. However, no means are provided for explicit revocation of a lease. Expiration handling is not provided either in the .NET Remoting framework. Although the system does indeed contact sponsors upon lease expiration, there are no guarantees that the system will contact the sponsor of a speciﬁc client as it may ask several sponsors until it ﬁnds one willing to renew the lease. CORBA. Leasing has been introduced to CORBA as a technique for resource management [2]. A broad deﬁnition of resource is adopted: a resource can be practically any CORBA entity. In order to provide reusable leasing functionality for diﬀerent CORBA-based applications, leasing is modeled as a dedicated CORBA service [2]. A resource is expected to implement two methods used by leases to start and stop the use of the resource. A resource claimant has to obtain a lease from a lessor in order to use such a resource. A lease oﬀers methods to be renewed and revoked. Two types of leases are granted depending on the type of claimants. Observed claimants receive a lease which observes the claimant so that if it terminates, the lease gets cancelled. The object is periodically queried to detect if it is still alive. Due to the volatile connections phenomenon, such leases do not seem appropriate for MANETs: the claimants may only be disconnected temporarily, causing the lease to be cancelled erroneously. Notified resource claimants receive a lease which notiﬁes the claimant as soon as it expires. The lease is then automatically renewed once at server side to give the claimant suﬃcient time to renew the lease if necessary. Leasing patterns need to be built on top of the above mentioned architecture, e.g. automatic renewal of leases can be accomplished by making explicit renew calls on the lease interface. Expiration handling can be achieved at client side using notiﬁed claimants. Jini. Jini is a framework built on top of Java which allows clients and services to discover and set up an ad hoc network. Jini provides support to deal with the fact that clients and services may join and leave the network at any time, unannounced. Leasing was introduced to allow clients and services to leave the network gracefully without aﬀecting the rest of the system. However, Jini relies on the communication model of Java RMI [15]. This means that transient disconnections are not supported, i.e. a disconnection blocks the connection between objects. Although Jini’s architecture is ﬂexible enough to accommodate a leasing model which takes into account intermittent connections, to the best of our knowledge, Jini does not implement this functionality. Jini advocates the use of leasing to mediate the access to any kind of resource: objects, ﬁles, certiﬁcates that grant the lease holder certain capabilities or even the right to request for some actions to execute while the lease is valid. By default, leases have been

A Leasing Model to Deal with Partial Failures in Mobile Ad Hoc Networks

237

only integrated in the lookup service. Services get a lease when they advertise themselves with the lookup service which must be explicitly renewed; if they cannot, the lookup service will remove the service advertisement such that it does not provide stale information. Jini provides a data structure for the systematic renewal of a set of leases. Leasing patterns can be built using a lease renewal service to implement the protocol to communicate with the remote server. Expiration handling can be achieved at client side by registering an event listener on a lease renewal set. When the lease is about to expire, an expiration warning event is generated notifying all registered listeners for that set. Table 1. Summary of related work

Java RMI .NET Remoting Framework CORBA Jini

Leasing an Intermittent Connection N (RPC)

Leasing Patterns N (need to be built)

N (RPC)

Y (integration of a leasing pattern) N (RPC) Y (using notiﬁed and observer claimants) N (reliance on JavaRMI) N (need to be built)

Symmetric Expiration Handling N (no notiﬁcation upon lease expiration) N (notiﬁcation not guaranteed) Y (only at client side) Y (only at client side)

Table 1 summarizes our related work. We observe that no single approach adheres to all of the criteria for a leasing model in MANETs. This shortcoming forms the main motivation for our work. In particular, no approach takes into account the intermittent connections criterion since they rely on synchronous communication by RPC, which is not designed for MANETs. However, these approaches provide a foundation on which we base our leasing model.

3

Leased Object References

To address all the criteria discussed in the previous section, we introduce our leasing model for MANETs where remote object references play the role of the lease and the service objects they refer to play the role of the resource, leading to the concept of leased object references. A leased object reference is a remote object reference that transparently grants access to a service object for a limited period of time. When a client ﬁrst references a service object, a leased reference is created and associated to the service VM A

client object

VM B

server object

Fig. 2. A leased object reference

238

E. Gonzalez Boix et al.

object. From that moment on, the client accesses the service object transparently via the leased reference until it expires. Figure 2 illustrates an allocated leased reference. Each end point of the leased reference has a timer initialized with a time period which keeps track of the lease time left. When the time period has elapsed, the access to the service object is terminated and the leased reference is said to expire. The lifetime of leased references can be explicitly controlled by renewing or revoking them before they expire. Once a leased reference expires, both the client object and service object know that access to the service object is terminated. Leased object references provide dedicated parameter-passing semantics to ensure that all remote interactions to a service object are subject to leasing. Leasing an Intermittent Connection. In order to abstract over the transient disconnections inherent to MANETs, a leased reference decouples the client object and the service object it refers to in time. This means that a client object can send a message to the service object even if the leased reference is disconnected at that time. Client objects can only send messages to service objects asynchronously: when a client object sends a message to the service object, the message is transparently buﬀered in the leased reference and the client does not wait for the message to be delivered1 . Figure 3 shows a diagram of the diﬀerent states of a leased reference. When the leased reference is connected and active, i.e. there is network connection and the lease has not yet expired, it forwards the buﬀered messages to the remote object. While disconnected, messages are accumulated in order to be transmitted when the reference becomes reconnected at a later point in time. When the lease expires, the client loses the means of accessing the service object via the leased reference. Any attempt in using it will not result in a message transmission since an expired leased reference behaves as a permanently disconnected remote reference.

disconnect

Connected

Disconnected

(messages are forwarded)

(messages are buffered) reconnect

expire

expire

Expired (messages are dropped)

Fig. 3. States of a leased object reference

1

We are not the ﬁrst ones on providing buﬀering of messages, this behaviour can be also found in the Rover toolkit [7].

A Leasing Model to Deal with Partial Failures in Mobile Ad Hoc Networks

239

Leasing Patterns. Leased object references incorporate two leasing variants on leased object references which transparently adapt their lease period under certain circumstances. The ﬁrst variant is a renew-on-call leased reference which automatically prolongs the lease upon each method call received by the service object. This pattern has been inspired by the renewOnCall property of the .NET Remoting framework [9]. As long as the client uses the service object, the leased reference is transparently renewed by the interpreter. The second variant is a single call leased reference which automatically revokes the lease upon performing a successful method call on the service object. Such leases are useful for objects adhering to a single call pattern, such as callback objects. As previously explained, callback objects are often used in asynchronous message passing schemes in order for service object to be able to return values. These callback objects are typically remotely accessed only once by service objects with the computed return value. Other variants are deﬁnitely possible and can be built on top of the leased object reference abstraction as we illustrate later in the implementation of a concrete instantiation of the leased object references model. Symmetric Expiration Handling. We adopt a Jini-like solution for expiration handling based on event listeners which can be registered with a leased reference at both client and server side. When the reference expires, the registered listeners are notiﬁed asynchronously. This allows client and service objects to treat a failure as permanent (i.e. to detect when the reference is permanently broken) and to perform appropriate compensating actions. At server side, this has important beneﬁts for memory management. Once all leased references to a service object have expired, the object becomes subject to garbage collection once it is no longer locally referenced. Because both sides of a leased reference have a timer, no communication with the server is required in order for a client to detect the expiration of a leased reference. However, having a client-side and server-side timer introduces issues of clock synchronisation. Keeping clocks synchronised is a well known problem in distributed systems [12]. This issue is somewhat more manageable with leases since they use time intervals rather than absolute time and the degree of precision is expected to be of the magnitude of seconds, minutes or hours. Once the leased reference is established, the server side of the reference periodically sends the current remaining time by piggybacking it onto application-level messages. At worst, the asynchrony causes a leased reference to be temporarily in two inconsistent states: either the client-side of the reference expires while the server-side is still active, or the client-side of the reference is active while the server-side expired. In the ﬁrst case, a client will not attempt a lease renewal and thus, the server-side timer will eventually expire as well. In the second case, when a client requests a lease renewal, the server will ignore it and the clientside timer will expire soon thereafter. When the server-side timer is expired, the client perceives the remote object as disconnected due to a network failure.

240

4

E. Gonzalez Boix et al.

Leased Object References in AmbientTalk

We have implemented leased object references in AmbientTalk, an objectoriented programming language designed for distributed programming in mobile ad hoc networks [13]. Before describing the concrete instantiation of the leased object reference model, we ﬁrst brieﬂy introduce AmbientTalk. We will use the mobile music player example introduced in Section 2.1 to illustrate the language and the language support for leased object references. 4.1

AmbientTalk in a Nutshell

AmbientTalk is a prototype-based object-oriented distributed language. The following code excerpt shows the deﬁnition of a simple Song object in AmbientTalk: def Song := object: { def artist := nil; def title := nil; def init(artist, title) { self.artist := artist; self.title := title; }; def play() { /* play the song */ }; }; def s := Song.new("Mika", "Relax");

A prototypical song object is assigned to the variable Song. A song object has two ﬁelds; a constructor called init in AmbientTalk, and a method play. Sending new to an object creates a copy of that object, initialised using its init method. Distributed programming. AmbientTalk is a concurrent actor-based language [1]. AmbientTalk’s actors are based on the communicating event loops model of the E language [11] in which each actor owns a set of regular objects. Objects owned by the same actor communicate using sequential message sending (expressed as o.m()), as in Java. Objects owned by diﬀerent actors can only send asynchronous messages to one another (expressed as o<-m()). This ensures by design that all distributed communication is asynchronous. Additionally, an actor can explicitly export objects which can then be discovered by remote objects. Objects can acquire a remote reference to an object when the remote object is implicitly parameter-passed as argument or return value in a asynchronous message sent. Additionally, an actor can explicitly export objects representing certain services which can then be discovered by remote objects. In AmbientTalk, a service object is always exported together with a service type, a descriptor used to categorise which objects export what kinds of services. One can deﬁne event handlers that are triggered whenever a remote object of a certain service type has become available in the network. AmbientTalk’s remote references by default mask partial failures: messages may be sent to a disconnected reference, where are buﬀered until the remote reference becomes reconnected.

A Leasing Model to Deal with Partial Failures in Mobile Ad Hoc Networks

4.2

241

Leasing in AmbientTalk

We now describe a concrete instantiation of the leased object reference model introduced in Section 3 in the AmbientTalk language. A description of how leased references have been implemented in AmbientTalk is postponed until Section 5. Our language support features three diﬀerent language constructs for creating leased object references which correspond to basic leased object references and the two variants described in Section 3. The most basic form of a leased reference is created by the lease:for: construct which requires two parameters: an object corresponding to the service object to which the leased reference grants access, and an initial time period (in milliseconds). In our running example, the session object that represents the exchange process between two music players should be subject to leasing in order for both music players to gracefully terminate the exchange process in the presence of network failures. Such a session object can be leased as follows: def leasedRef := lease: minutes(10) for: session

The leased reference created with the lease:for: construct is valid for the given time period unless a renewal or revocation is explicitly issued. After this time period, access to the session is terminated and the leased reference expires. Explicit manipulation of the lifetime of a leased reference is provided by means of the renew: and revoke: constructs. The renew: construct requests a prolongation of the speciﬁed leased reference with a new interval of time which can be diﬀerent than the initial time while the revoke: construct cancels the given leased reference. Cancelling a lease is in a sense analogous to a natural expiration of the lease, but it requires communication between the client and server side of the leased reference. Note that the lease:for: construct and the other two constructs (described in the next section) are executed at the server side. The virtual machine hosting the service object hands out the proper leased object reference to a client object. In our running example, a music player application asks a remote peer to start a session to exchange its library index by sending it an openSession message which returns a new session object. The music player can then send song information to the remote peer via the obtained leased reference as follows: session<-uploadSong("Mika", "Relax", ...);

Because we chose to model leasing by means of a special kind of remote object reference, the client can use the leased reference as if it were the service object itself. The use of leasing is thus made transparent to the client. 4.3

Language Constructs for Leasing Patterns

In order to create renew-on-call and single-call leases explained in Section 3, we provide the renewOnCallLease:for: and singleCallLease:for: constructs,

242

E. Gonzalez Boix et al.

respectively. The renewOnCallLease:for: construct creates a leased reference which is automatically prolonged on every remote method invocation on the service object. When no renewal is performed due to a network partition or in the absence of utilization, the leased reference expires once its lease time elapses. In the running example, once a music player establishes a session with another music player to exchange their music library index, the session should remain active as long as the exchange is active, i.e. uploadSong messages are received. A renew-on-call lease can be used for the session object to model that kind of collaboration as follows: def openSession(sessionCallback) { def senderLib := Set.new(); // store sender’s music library in a set def session := renewOnCallLease: minutes(10) for: ( object: { def uploadSong(artist, title, ackCallback) { senderLib.add(Song.new(artist, title)); ackCallback<-ok(); // tell sender that song was successfully received }; def endExchange() { revoke: session; def matchRatio := calculateMatchRatio(senderLib); if: (matchRatio >= THRESHOLD) then: { // notify user of match }; }; } ); sessionCallback<-receive(session); // return the session object };

As previously mentioned, the openSession message is sent by a music player to a remote peer which returns a session object that can be used to start a library exchange. A session implements the uploadSong method to send song information and the endExchange method to signal the end of the library exchange. The session object is exported using a lease for 10 minutes which is automatically renewed each time it receives a message. The renewal time applied on every call is the initial interval of time speciﬁed at creation. The leased reference is revoked either explicitly when a client sends the endExchange message to indicate the end of the library exchange, or implicitly if the lease time has elapsed. Since the session object was only referred to by the leased reference, it can be reclaimed once the lease has expired. Any resources it transitively occupied such as the partially uploaded library of songs (i.e. senderLib) can be reclaimed as well. The singleCallLease:for: construct allows developers to create leased references that remain valid for only a single call. In other words, the leased reference expires after the service object receives a single message. However, if no message has been received within the speciﬁed time interval, the leased reference also expires. As shown in the code above, the sessionCallback is parameter-passed in the openSession message to asynchronously receive a session object. A singlecall lease can be used for unexporting this callback object upon receipt of the receive message as follows: remotePlayer<-openSession( singleCallLease: minutes(10) for: ( object: { def receive(session) { /* start to exchange of its library via the session (explained later) */ } }));

A Leasing Model to Deal with Partial Failures in Mobile Ad Hoc Networks

243

A lease time of 10 minutes is speciﬁed to wait for the reply of the openSession message. If a disconnection would occur after the message was sent but before the receive reply was received, the session object could have already been allocated. Since a session’s lease only lasts 10 minutes by default, it does not make sense to wait any longer for the reply. If the session callback’s lease expires, the music library exchange terminates before it was actually started, requiring no additional cleanup code. 4.4

Integrating Leasing with Future-Type Message Passing

In the previous section, we used an explicit callback object to return the result of the openSession asynchronous message. This is motivated by the fact that in AmbientTalk, an asynchronous message send has no return value by default (i.e. it returns nil). To avoid forcing programmers to rely on explicit, separate callback methods to obtain the result of an asynchronous computation, futuretype message passing [16] was introduced in AmbientTalk. Futures are a classic technique to reconcile asynchronous message sends with return values, by making an asynchronous send immediately return a future object. A future is a placeholder for the return value of an asynchronous message send which allows the sender of an asynchronous message to access the return value of that message at a later point in time [16]. In our running example, we have used callback objects to circumvent the lack of return values in asynchronous message sends. With the introduction of futures, explicit callbacks are no longer necessary: the future serves as an implicit callback. The asynchronous invocation of openSession can be rewritten using futures as follows: def sessionFuture := remotePlayer<-openSession(); when: sessionFuture becomes: { |session| // open session with remotePlayer }

We have integrated leasing into futures by parameter-passing a future attached to an asynchronous message via a singe-call lease which either expires due to a timeout or upon the reception of the computed return value. The timeout for the implicit single-call lease on a future can be set by annotating the asynchronous message with a @Due annotation as follows: def sessionFuture := remotePlayer<-openSession()@Due(minutes(10)); when: sessionFuture becomes: { |session| // open session with remotePlayer }catch: TimeoutException using: { |e| system.println("unable to open a session."); }

In AmbientTalk, it is possible to register a block of code with a future which is executed asynchronously when the future becomes resolved with a return value by means of the when:becomes:catch: construct. If the future is resolved to a proper value, the block of code in the becomes: argument is applied. If the

244

E. Gonzalez Boix et al.

asynchronously invoked method raises an exception, the catch: argument is applied to the exception. A TimeoutException is raised when the future’s lease expires. If the future is resolved, the session variable stores a leased object reference to the remote session object. A music player then sends all of its own songs one by one to this session object as follows: def sessionFuture := remotePlayer<-openSession()@Due(minutes(10)); when: sessionFuture becomes: { |session| def iterator := myLib.iterator(); // iterate over own music library def sendNextSong() { // auxiliary function to send each song if: (iterator.hasNext()) then: { def song := iterator.next(); def ackFut := session<-uploadSong(song.artist, song.title)@Due(leaseTimeLeft: session); when: ackFut becomes: { |ack| sendNextSong(); // recursive call to send next song info }catch: TimeoutException using: { |e| notification("stopping exchange: " + e) }; } else: { session<-endExchange(); }; }; sendNextSong(); }; };

As already observed, the uploadSong method can be directly sent to the session variable storing the leased reference as if it were the service object itself since leasing is transparent to the client. The auxiliary function sendNextSong sends the music player’s songs one by one to the remote session object. This serial behaviour is guaranteed because each subsequent uploadSong message is only sent after the previous one returned an acknowledgement. Since the return value of the uploadSong message is only useful in the context of the current library exchange session, it only makes sense to wait for the future resolution for the remaining duration of the session (which can be acquired from a leased reference by means of the leaseTimeLeft: construct). If the future’s lease expires, the library exchange is stopped without requiring additional cleanup code. The integration of futures and leasing by means of the @Due annotation, illustrates that low-level memory management concerns (e.g. the callback objects) can be cleanly incorporated into more high-level abstractions, decreasing the mental overhead for the developer. 4.5

Supporting Expiration Handling

In order to allow both client and service objects to properly react to the expiration of a leased reference and schedule clean-up actions, the when:expired: construct is provided. A music player can detect when a session with a remote music player expires as follows: when: session expired: { system.println("session timed out."); // clean the partially received music library }

A Leasing Model to Deal with Partial Failures in Mobile Ad Hoc Networks

245

The construct takes as parameters a leased reference and a block of code that is asynchronously triggered upon the lease expiration. In the example, when:expired: is installed at the server side so that if the exchange cannot be completed the resources a session transitively keeps alive (i.e. the senderLib set storing incoming songs) can be cleared. Note that in the integration of leasing with futures, specifying a catch: block for the TimeoutException is equivalent to install a when:expired: observer on the future’s (server-side) lease.

5

Implementation

Leased object references have been implemented as part of the AmbientTalk language2. The language has been implemented as an interpreter written on top of the Java Virtual Machine which runs on the J2ME platform. The mobile music player has been implemented and tested on HTCP3650 Touch Cruise smartphones connected by a WiFi network. In this section, we describe the necessary features of the implementation of leased object references to show how custom leasing patterns can be expressed. Leased references have been built reflectively in the AmbientTalk language itself. They have been implemented as remote object references whose default semantics has been altered using the language’s meta-object protocol (MOP) [8]. The implementation of leased references basically applies two changes to this semantics. First, the lifetime of a remote reference is limited by means of a timer which is initialized when the remote reference is created and associated to a service object. Second, any asynchronous message received in a leased reference is managed as shown in ﬁgure 3. 5.1

Leased Object References

A leased object reference is a unidirectional communication link from a client to a service object as depicted in ﬁgure 4 with a dotted line. At the implementation level, as also shown in ﬁgure 4, a leased reference actually consists of an ensemble of object references and two set of objects at client and server side: a lease object which implements methods for managing the life cycle of a leased reference, and an interceptor object which intercepts the messages sent to the server object and exposes the diﬀerent variation points of a leased reference. The Lease Object. A lease object contains a timer and implements methods to handle the life cycle of a leased object reference. Figure 5 (on the left-hand side) shows the API of the lease object. The expire method terminates the remote access to the service object by taking oﬄine the leased reference. The revoke method is analogous to the expire method but it does not notify the expiration observers. The renew method prolongs the lease timer with a speciﬁed renewal time (by default set to the initial time). The given time is not directly added to 2

The language is available at http://prog.vub.ac.be/amop/at/download

246

E. Gonzalez Boix et al. VM A

VM B lease obj

client object

client lease interceptor

lease obj

server object

server lease interceptor

Conceptual leased reference

Implementation references

Fig. 4. Implementation of a leased object reference

the initial time interval but, a new interval is calculated by taking the maximum of the current time left and speciﬁed the renewal time. Finally, the whenExpired method registers a block of code to be applied upon lease expiration. def lease := object: { def expire(); def revoke(); def renew(renewalTime); def whenExpired(block); };

def interceptor := object: { def receive(msg, lease); def pass(lease); def resolve(lease); };

Fig. 5. The lease object API (left) and the interceptor API (right)

The Interceptor API. Figure 5 (on the right-hand side) shows the API of the interceptor object. receive is called every time an object receives a message. By default, the server lease interceptor overrides receive to forward the messages to the service object to which a leased reference grants access as shown below. def receive(msg, lease) { if: !(lease.isExpired) then: { forward(msg, lease.serviceObject) } }

The pass and resolve methods are called when an object is marshalled and unmarshalled to another device, respectively. Interceptors override pass and resolve methods to modify the parameter-passing semantics of the service object referenced to ensure a pass-by-lease semantics. More concretely, these methods are overridden by interceptors to create the client and server side of a leased object reference when a service is parameter passed. The client side of a leased reference behaves slightly diﬀerent than its server counterpart. The key diﬀerence is that it does not grant access to a service object but to the server side of the leased reference pointing to the actual service object (as shown in ﬁgure 4). Messages intercepted by the client lease interceptor are forwarded to the server lease interceptor. The client side of a leased reference also has a lease object (which maintains its own timer kept in synchronization with its server counterpart) and its own when:expired: observers (which allow clients to be notiﬁed upon the lease expiration reference without requiring

A Leasing Model to Deal with Partial Failures in Mobile Ad Hoc Networks

247

communication with the server). Another diﬀerence is that the client side of a leased reference does not adhere to the described pass-by-lease semantics since they do not provide access to a service object but to the server side of a leased reference. Integrated Leasing Patterns. A leased object reference created by means of the lease: construct uses the default interceptors explained above. We now describe how such default interceptors has been extended to implement the singlecall and renew-on-call variants explained in Section 3. The single-call-lease and renew-on-call interceptors override pass and resolve with their own speciﬁc strategies to ensure pass-by-single-call-lease and pass-by-renew-on-call-lease semantics, respectively. receive is overridden in single-call and renew-on-call interceptors to provide automatic renewal and revocation of the leased reference upon message reception, respectively as follows: // Renew-on-call server interceptor def receive(msg, lease) { if: !(lease.isExpired) then: { lease.renew(lease.renewalTime); }; super.receive(msg); };

// Single-call server interceptor def receive(msg, lease) { def result := super.receive(msg); if: !(lease.isExpired) then: { lease.revoke(); }; result; };

Both server lease interceptors delegate the forwarding of the message to the default interceptor by means of a super-send. In the case of a renew-on-call lease, it renews its timer before delegating the forwarding of the message. In the case of a single-call lease, it cancels its lease upon receiving a ﬁrst message by calling the revoke method, which also sets the isExpired property to true so that future received messages are dropped. Other Variations on Leased Object References. Using the provided APIs developers can extend the default leasing semantics to encode custom leasing patterns. For example, in our running example, the session object is leased with a renew-on-call lease which needs to be explicitly revoked upon the endExchange message. We have implemented a custom interceptor and lease object to encode this behaviour. The receive method for the interceptor is shown below: // server interceptor def receive(msg, lease) { if: !(lease.isExpired) then: { if: (lease.isRevoke(msg)) then: { lease.revoke(); } else: { lease.renew(lease.renewalTime) }; } }

The lease object needs has been extended with the isRevoke method which checks if the received message should automatically revoke the lease (in the case of the running example: if it is an endExchange message).

248

6

E. Gonzalez Boix et al.

Discussion

Now that the leased object references model and its instantiation in AmbientTalk have been properly described, we evaluate them in the light of the criteria for leasing in MANETs identiﬁed in Section 2. Leasing an Intermittent Connection. Leased object references combine leasing with asynchronous communication into one coherent language abstraction that deals with both transient and permanent disconnections. A leased reference deﬁnes a connection which supports intermittent disconnections by default: client objects can send messages via a leased reference as long as it is not expired, independently of the state of the connection, because a leased reference buﬀers messages while disconnected. Leasing Patterns. Useful leasing patterns have been made available in the form of dedicated leased object references, e.g. renew-on-call and single-call leases. These lease variants illustrate that managing the lifetime of leased references can be done implicitly by means of message passing reducing the programming eﬀort for the developer. In addition, the lease object and interceptor API explained in the implementation section form an extensible framework in which developers can plug in their own custom leasing patterns. Symmetric Expiration Handling. By means of the registration of dedicated listeners triggered upon the expiration of a leased object reference, both sides of the reference can gracefully deal with the termination of their logical connection and schedule the appropriate compensating actions. We consider the mobile music player application to be an illustrative example which exhibits a set of key issues that are typical in collaborative ad hoc networking applications. Its implementation, shown in Section 4, demonstrates how developers can concisely deﬁne and manipulate leased references and how the language support eases the development of mobile applications that deal with both transient and permanent disconnections and properly reclaim their service objects. Thanks to the language constructs presented, its implementation counts merely 90 lines of code. We have implemented a music player application which exhibits similar semantics in Java RMI which counts no less than 462 lines 3 . Table 2. Summary of the lines of code (and %) for the music player application Memory man- Concurrency Failure hand- Application Total lines agement code control code ling code code of code Java RMI 145 (31,38%) 148 (32,03%) 78 (16,88%) 91 (19,69%) 462 AmbientTalk 7 (7,77%) 7 (7,77%) 6 (6,66%) 70 (77,77%) 90

Table 2 summarizes the lines of code for both implementations according to four diﬀerent concerns4 : 1) memory management: includes the code to setup a 3 4

Both implementations are available at http://code.google.com/p/ambienttalk/downloads The code for service discovery has not been taken into account in this comparison.

A Leasing Model to Deal with Partial Failures in Mobile Ad Hoc Networks

249

renew-on-call lease for the music session and reclaim the used data structures upon lease expiration, 2) concurrency control: includes the code to ensure the responsiveness of the application in the face of transient disconnections, 3) failure handling: includes the code to have time-based delivery policy guarantees on remote messages and 4) application-level code. While the application code has a similar magnitude in both implementations, the Java RMI implementation has required to manually deal with the following issues: – First, programmers have to manually deal with the impact of the volatile connections phenomenon on remote references. Upon a disconnection, the thread executing a remote method call blocks (as Java RMI uses synchronous communications). This makes the application unresponsive to GUI events and to discovery events notifying new music player peers in the network. To solve this issue, two diﬀerent threads need to be spawned: a transmission thread that sends the remote messages and a callback thread that handles the return values. These threads need to communicate with each other by means of message objects which wraps a remote call. Five message classes were encoded corresponding to the diﬀerent remote messages shown in ﬁgure 1. In addition, the transmission thread must ensure that messages are buﬀered while the reference is disconnected. This is not required in AmbientTalk as leased references themselves abstract the connection state. – Second, the lifetime of Java RMI leases is controlled by the leaseValue property which is applied to all remote objects in the entire VM. It is not possible to specify lease periods on a per-application, per-class or per-object basis to provide application-speciﬁc policies. This needs to be manually encoded with timers. This property is also associated with the socked connection timeout which controls when a remote message send fails. Thus, every remote message send has the same timeout. In the music player application, uploadSong and openSession messages have diﬀerent delivery guarantees (speciﬁed by the @Due annotation in our approach). In order to have similar semantics in Java RMI, the application needs to implement its own timers and modify the transmission thread to take into account the message expiration. – Third, in Java RMI a remote reference is considered in use as long as the client holds it. When the client stops using the reference, the runtime sends an “unreferenced” control message to the server side which may be then able to garbage collect the object (once it is no longer remotely nor locally referenced). To stop using a remote reference, clients have to explicitly make sure to clear local references to the remote reference. For example, in the music player application, code was added to stop the transmission and callback threads and to remove the session from the hashmap storing active opened sessions. If clients do not add this code, the remote object cannot be collected unless there is a disconnection exceeding the leaseValue timeout. This code is avoided with renew-on-call or single-call leases in AmbientTalk. Leases created by means of lease: construct need to be manually revoked. To revoke a lease, only one revoke message must be sent rather than having to wait for the system to detect when the remote reference is no longer used.

250

E. Gonzalez Boix et al.

– Fourth, Java RMI does not provide a renew-on-call lease pattern as the one used in the music player application. Rather, the client-side runtime system renews the lease to a remote object implicitly once the leaseValue reaches half of its value. This leads to unnecessary network traﬃc since control messages are sent to keep a remote object alive which may not be used anymore. To implement a renew-on-call pattern in Java RMI, the system provides the dgc interface which is not extensible. The interface is not meant to be used by application programmers, thus the pattern needs to be implemented explicitly requiring repetitive renewal code at client and server-side. – Finally, no notiﬁcation is performed in Java RMI upon lease expiration. Client objects are notiﬁed of the expiration of a lease only if they issue a remote method call upon an expired lease (which throws an exception). At server side, only when all clients disconnect, the unreferenced method is called on a remote object. In the music player application, the registration of listeners with leases had to be explicitly encoded. In AmbientTalk, dedicated language constructs are provided for the registration of expiration listeners at both sides of the leased reference.

7

Conclusion and Future Work

This paper focuses on the use of leasing for dealing with partial failures in mobile ad hoc networks. We identify a number of criteria for a leasing model specially designed for MANETs. We require a leasing model that (1) takes into account the volatile connections phenomenon, (2) provides diﬀerent leasing patterns to manage the lifetime of leases, and (3) allows both lease holder and grantor to react to and schedule clean-up actions upon lease expiration. We subsequently propose the leased object reference model which exhibits such criteria. The contributions of leased object references are threefold: we provide leasing as a special kind of remote reference which tolerates both transient and permanent disconnections, we design leased references as an extensible framework which integrates useful patterns, e.g. renew-on-call and single-call leases, and we provide means for allowing client and service objects to detect and react to a permanent failure. The applicability of the language constructs have been assessed by means of the development of a typical collaborative ad hoc networking application. By making use of leased references, programmers can distinguish transient from permanent disconnections and react accordingly. Disconnections that exceed the lease period are considered permanent requiring the collaboration between two communicating parties to be for example restarted in the music player application. However, leasing can only provide an approximation of when a disconnection is permanent. The quality of the approximation depends on the accuracy of the selected lease period. Selecting a suitable lease period is not straightforward and it requires to consider the behaviour of mobile devices in the physical world and factors such as the frequency of disconnections and reconnections. Any leasing mechanism has to deal with this issue but it is exacerbated in MANETs due to the unpredictability of the mobility patterns of the end-users.

A Leasing Model to Deal with Partial Failures in Mobile Ad Hoc Networks

251

In future work, we would like to allow the system to choose an appropriate lease timeout for the developer based on previously observed mobility patterns. Questions also arise in a leasing system regarding when to renew a lease. This is in a high degree application-speciﬁc. Although renew-on-call and single-call leases alleviate this problem by providing automatic renewal or revocation of leases, we would like to explore the use of leasing patterns which dynamically adapt the lease period under certain circumstances [4], e.g. changes in the observed mobility pattern.

References 1. Agha, G.: Actors: a Model of Concurrent Computation in Distributed Systems. MIT Press, Cambridge (1986) 2. Aleksy, M., Korthaus, A., Schader, M.: Realizing the leasing concept in corba-based applications. In: Proc. of Symp. on Applied Comp., pp. 706–712 (2005) 3. Eugster, P.T., Felber, P.A., Guerraoui, R., Kermarrec, A.-M.: The many faces of publish/subscribe. ACM Comput. Surv. 35(2), 114–131 (2003) 4. Gonzalez Boix, E., Vallejos, J., Van Cutsem, T., Dedecker, J., De Meuter, W.: Context-aware leasing for mobile ad hoc networks. In: ECOOP Workshop on Object-Oriented Technology for AmI and Pervasive Comp. (2007) 5. Gray, C., Cheriton, D.: Leases: an eﬃcient fault-tolerant mechanism for distributed ﬁle cache consistency. In: SOSP 1989: Proceedings of the twelfth ACM symposium on Operating systems principles, pp. 202–210 (1989) 6. Jain, P., Kircher, M.: Leasing. In: Proceedings of the 7th Patterns Languages of Programs Conference, PLoP (2000) 7. Joseph, A.D., de Lespinasse, A.F., Tauber, J.A., Giﬀord, D.K., Kaashoek, M.F.: Rover: a toolkit for mobile information access. In: Proc. of the 15th ACM Symposium on Operating Systems Principles, pp. 156–171 (1995) 8. Kiczales, G., Rivieres, J.D., Bobrow, D.G.: The Art of the Metaobject Protocol. MIT Press, Cambridge (1991) 9. Lowy, J.: Managing the lifetime of remote.net objects with leasing and sponsorship. MSDN Library (December 2003) 10. McLean, S., Williams, K., Naftel, J.: Microsoft.Net Remoting. Microsoft Press, Redmond (2002) 11. Miller, M., Tribble, E.D., Shapiro, J.: Concurrency among strangers: Programming in E as plan coordination. In: De Nicola, R., Sangiorgi, D. (eds.) TGC 2005. LNCS, vol. 3705, pp. 195–229. Springer, Heidelberg (2005) 12. Tanenbaum, A.S., Steen, M.V.: Distributed Systems: Principles and Paradigms. Prentice Hall PTR, Upper Saddle River (2001) 13. Van Cutsem, T., Mostinckx, S., Gonzalez Boix, E., Dedecker, J., De Meuter, W.: Ambienttalk: object-oriented event-driven programming in mobile ad hoc networks. In: Int. Conf. of the Chilean Comp. Science Society (2007) 14. Waldo, J.: The Jini Architecture for Network-centric Computing. Commun. ACM 42(7), 76–82 (1999) 15. Waldo, J.: Constructing ad hoc networks. In: IEEE Inter. Symposium on Network Computing and Applications (NCA), p. 9 (2001) 16. Yonezawa, A., Briot, J.-P., Shibayama, E.: Object-oriented concurrent programming in ABCL/1. In: Proceedings on OOPSLA, pp. 258–268 (1986)

Reusing and Composing Tests with Traits Stéphane Ducasse1 , Damien Pollet1 , Alexandre Bergel1 , and Damien Cassou2 1 RMoD team, INRIA Lille – Nord Europe & University of Lille 1 Parc Scientiﬁque de la Haute Borne – 59650 Villeneuve d’Ascq, France 2 Phoenix team, University of Bordeaux 1 Building A29bis – 351, cours de la libération – 33405 Talence, France

Abstract. Single inheritance often forces developers to duplicate code and logic. This widely recognized situation aﬀects both business code and tests. In a large and complex application whose classes implement many groups of methods (protocols), duplication may also follow the application’s idiosyncrasies, making it diﬃcult to specify, maintain, and reuse tests. The research questions we faced are (i) how can we reuse test speciﬁcations across and within complex inheritance hierarchies, especially in presence of orthogonal protocols; (ii) how can we test interface behavior in a modular way; (iii) how far can we reuse and parametrize composable tests. In this paper, we compose tests out of separately speciﬁed behavioral units of reuse —traits. We propose test traits, where: (i) speciﬁc test cases are composed from independent speciﬁcations; (ii) executable behavior speciﬁcations may be reused orthogonally to the class hierarchy under test; (iii) test ﬁxtures are external to the test speciﬁcations, thus are easier to specialize. Traits have been successfully applied to test two large and critical class libraries in Pharo, a new Smalltalk dialect based on Squeak, but are applicable to other languages with traits. Keywords: Reuse, Traits, Multiple Inheritance, Tests, Unit-Testing.

1

The Case

One fundamental software engineering principle is to favor code reuse over code duplication. Reusing unit tests is important, because they are valuable executable speciﬁcations that can be applied to classes in diﬀerent inheritance hierarchies. How can we factor tests and reuse them to validate the conformance of diﬀerent classes to their common protocols? By protocol, we mean an informal group of methods, that is implemented orthogonally to inheritance hierarchies1. This is a typical situation where single inheritance hampers code reuse by forcing developers to copy and paste [4]. Still, there are some techniques to reuse tests: one way is to write unit tests that are parametrized by the class they should test. This way, it is possible to deﬁne tests for protocols (group of methods) and interfaces and to reuse 1

We avoid the term interface because protocols are typically of ﬁner granularity and implemented in a more ﬂexible way, e.g., incompletely or with naming variations.

M. Oriol and B. Meyer (Eds.): TOOLS EUROPE 2009, LNBIP 33, pp. 252–271, 2009. c Springer-Verlag Berlin Heidelberg 2009

Reusing and Composing Tests with Traits

253

them for the diﬀerent classes that implement such interfaces. Even if this does not seem widely used, JUnit 4.0 oﬀers a parametrized test runner to run tests on a collection of ﬁxtures returned by a factory method. Another approach is to deﬁne hook methods in the generic protocol test case, which can be overridden in a test case subclass for each implementor of that protocol. Both approaches have the limit that they organize tests per protocol rather than per implementor. This encourages to duplicate ﬁxtures —which are indeed speciﬁc to the implementor— across protocols, but also makes implementor-speciﬁc adjustments diﬃcult: for instance, to cancel inherited tests, subclasses may have to deﬁne phony ones2 . Traits, pure composable units of behavior[8], recently got some attention and have been introduced in several languages including Squeak [12], AmbientTalk [7], Fortress [10], the DrScheme object layer [9], Slate and Javascript3. Traits are groups of methods that can be reused orthogonally to inheritance. A class inherits from a superclass and may be composed of multiple traits. To evaluate the expressivity of traits on real case studies, several works redesigned existing libraries using traits [3,5]. So far, traits have been applied to and used when deﬁning and designing applications. Since traits are composable units of behavior, the question whether traits may be successfully applied for testing naturally raises. The research questions that drive this paper are the following ones: – Are test traits reusable in practice? If each feature was tested by a separate trait, how much test reuse could we obtain? What is a good granularity for test traits that maximizes their reusability and composition? – How far should a test ﬁxture be adapted to speciﬁc requirements? – How far should a test be parametrized to be reusable? What is the impact on the maintainability of test code? To answer these research questions, over the last two years, we performed the following experiment: while designing a complete new implementation of the stream library of the Squeak open-source Smalltalk [5], we deﬁned test traits (i.e., traits that deﬁne test methods, and which can be reused by composition into test classes). In addition, we initiated the development of a new body of tests for the collection library [6], in preparation for a future complete redesign. We deﬁned a large body of tests and reused them using traits. In this paper, we report our results which are now part of the Pharo Smalltalk4 . The contributions of this article show that traits are really adapted to specifying and reusing tests, especially in the context of multiple inheritance. A test trait deﬁnes an abstraction over its test data (a ﬁxture), and a set of test methods. Test traits may then be composed and adjusted to build concrete test classes that can parallel the domain code organization. 2 3 4

And canceling tests is not easily possible with the JUnit 4.0 approach. respectively http://slate.tunes.org and http://code.google.com/p/jstraits http://www.pharo-project.org

254

S. Ducasse et al.

The article is structured as follows: Section 2 shortly presents xUnit, and expose the problems we are currently facing; Section 3 describes the libraries we selected for our study; we then explain our experimental process (Section 4) and present some selected examples and situations (Section 5); in Section 6, we present the traits we deﬁned, and how they are reused to test the classes of the libraries; ﬁnally, we discuss the beneﬁts and limits of our approach, before exploring related work and concluding (Sections 7, 8, 9).

2

Illustrating the Problem

In this section we brieﬂy present the implementation principles of the xUnit frameworks, some examples of problems found in the Squeak libraries. 2.1

xUnit in a Nutshell

The xUnit family of testing frameworks originated with its Smalltalk incarnation: SUnit. In essence, a unit test is a short piece of code which stimulates some objects then checks assertions on their reaction [2,14]. The stimulated objects are collectively called a test ﬁxture, and unit tests that have to be executed in the context of a given ﬁxture are grouped with it into a test case. The framework automates executing the unit tests, ensuring each is run in the context of a freshly initialized ﬁxture. In SUnit, a test case is a subclass of TestCase which deﬁnes unit tests as methods, and their shared ﬁxture as state. In the example below5 , we deﬁne such a subclass, named SetTest, with instance variables full and empty. We then deﬁne the method setUp, which initializes both instance variables to hold diﬀerent sets as a ﬁxture, and the unit test method testAdd, which checks that adding an element works on the empty set. TestCase subclass: #SetTest instanceVariableNames: ’full empty’ SetTest >> setUp empty := Set new. full := Set with: 5 with: 6.

5

SetTest >> testAdd empty add: 5. self assert: (empty includes: 5).

Readers unfamiliar with Smalltalk might want to read the code aloud, as approximate english. Message sends interleave arguments with keywords of the message name: receiver message: arg1 with: arg2 (the message name is message:with:). Messages are sent from left to right with priorities: unary, then operator, then keyword messages, so that self assert: 120 = 5 factorial does not need parentheses. A dot separates full statements, and semi-columns cascade several messages to a single receiver: receiver msg1; msg2. Single quotes denote ’strings’, double quotes denote “comments”. #foo is a symbol, and #(a b 42) is a literal array containing #a, #b, and 42. Curly braces are syntactic sugar for building arrays at runtime. Square brackets denote code blocks, or anonymous functions: [:param | statements]. The caret ^ result returns from the method.

Reusing and Composing Tests with Traits

2.2

255

Analyzing the Squeak Collection Library Tests

As explained above, in absence of parametrized unit tests, it is not simple to reuse tests across hierarchies (i.e., to run tests against classes in separate hierarchies). Indeed, the well-known limits of single inheritance apply to tests as well, and this is particularly true when we want to reuse tests for protocol compliance. Since protocol implementations often crosscut inheritance, a developer cannot simply rely on inheritance and is forced to either copy and paste or use delegation [8]. In addition to such common problems we wanted to understand the other problems that may arise when programmers cannot reuse tests. We studied the Squeak collection library tests and we complement the problems mentioned above with the following points: Test duplication. We found redundancies in testing some features, not due to straight code duplication, but to duplicated test logic. The programmers probably were not aware of the other tests or had no means to reuse them. For example, the feature “adding an element in a collection after a particular element in the collection” is implemented with the add:after: method in LinkedList and OrderedCollection. This feature is tested in LinkedListTest and OrderedCollectionTest. For example, compare the following tests: LinkedListTest >> test09addAfter | collection last | collection := LinkedList new. last := self class new n: 2. collection add: (self class new n: 1); add: last. self assert: (collection collect:[:e | e n]) asArray = #(1 2). collection add: (self class new n: 3) after: last. self assert: (collection collect:[:e | e n]) asArray = #(1 2 3). OrderedCollectionTest >> testAddAfter | collection | collection := #(1 2 3 4) asOrderedCollection. collection add: 88 after: 1. self assert: (collection = #(1 88 2 3 4) asOrderedCollection). collection add: 99 after: 2. self assert: (collection = #(1 88 2 99 3 4) asOrderedCollection).

Logic duplication is a problem since it limits the impact of tests and costs more in maintenance. No systematic feature testing. In most of the cases, the collection tests were written to assess the compliance of Squeak with the ANSI standard. However, instead of an overall eﬀort, the tests are the result of punctual and independent initiatives. The only way to assess which feature a test method covers, is by its name and by what messages the test method sends. As a consequence: – some features are tested only for one particular type (e.g., after: and after:ifAbsent: are only tested for instances of OrderedCollection in SequenceableCollectionTest);

256

S. Ducasse et al.

– some features are tested twice for the same type, in diﬀerent classes (e.g., in both SequenceableCollectionTest and OrderedCollectionTest, testCopyWith tests the same copying features for OrderedCollection). Testing ad hoc behavior. Since the tests were not reused, some tests ended up covering uninteresting implementation decisions, e.g., the default capacity of newly created collections. While it would be an interesting test if it was generic, applied class by class it often leads to ad-hoc values being documented in test methods. This practice can even be counter productive since tests that assume ﬁxed values will break when the value changes, and require unnecessary ﬁxes [14].

3

Experimental Context: Two Large Libraries

We experimented on two Smalltalk librairies structured around large hierarchies: streams and collections . We selected them for the following reasons: (i) they are complex and essential parts of a Smalltalk system, (ii) they mix subtyping with subclassing, (iii) they are industrial quality class hierarchies that have evolved over 15 years, and (iv) they have been studied by other researchers [13,6,11,4]. We identiﬁed the problems cited above in the Squeak open-source Smalltalk [12]. Squeak, like all Smalltalk environments, has its own implementation of such libraries which are solely based on single inheritance without traits. In Squeak, the abstract classes Stream and Collection have around 40 and 80 subclasses, respectively, but many of these (like Bitmap or CompiledMethod) are specialpurpose classes crafted for use in the system or in applications. Here, we only take into account the core classes of each hierarchy (Figures 1 and 2). 3.1

Streams

Streams are used to iterate over sequences of elements such as sequenced collections, ﬁles, and network streams. Streams may be either readable (with methods like next and skip), or writable (with methods like write and flush), or both. There are diﬀerent kinds of streams to read/write diﬀerent kinds of data (characters, bytes, pixels. . . ) and to iterate over diﬀerent kind of sequences (collection, single ﬁle, compressed ﬁle, picture. . . ). This multiplicity of properties implemented in a single inheritance object-oriented language involves either a combinatorial number of classes or trade-oﬀs. Since the ﬁrst solution is hardly realizable, all Smalltalk dialects chose the second approach with trade-oﬀs (copy and paste, too many responsibilities, methods implemented too high in the hierarchy, unused superclass state) [5].

Stream

PositionableStream

WriteStream

ReadStream

ReadWriteStream

Fig. 1. The Squeak core stream hierarchy

FileStream

Reusing and Composing Tests with Traits

257

Object Collection SequenceableCollection

Bag

Set

LinkedList

PluggableSet

ArrayedCollection

Interval

Dictionary

OrderedCollection Array

String

Text

IdentityDictionary SortedCollection

ByteString

Symbol

PluggableDictionary

Fig. 2. Some of the key collection classes in Squeak

3.2

The Varieties of Collections

In Smalltalk, when one speaks of a collection without being more speciﬁc about the kind of collection, he or she means an object that supports well-deﬁned protocols for testing membership and enumerating elements. Here is a summary. All collections understand testing messages such as includes:, isEmpty and occurrencesOf:. All collections understand enumeration messages do:, select:, collect:, detect:ifNone:, inject:into: and many more. Table 1 summarizes the standard protocols supported by most of the classes in the collection hierarchy. These methods are deﬁned, redeﬁned, optimized or occasionally even cancelled in Collection subclasses6 . Large set of diﬀerent behaviors. Beyond this basic uniformity, there are many different collections. Each collection understands a particular set of protocols where each protocol is a set of semantically cohesive methods. Table 2 presents some of the diﬀerent facets of collections and their implementation. For overall understanding, the key point to grasp is that protocols presented in Table 1 crosscut the large set of diﬀerent behaviors presented in Table 2. Here is an enumeration of the key behaviors we can ﬁnd in the Smalltalk collection framework: Sequenceable: Instances of all subclasses of SequenceableCollection start from a ﬁrst element and proceed in a well-deﬁned order to a last element. Set, Bag and Dictionary, on the other hand, are not sequenceable. Sortable: A SortedCollection maintains its elements in sorted order. Indexable: Most sequenceable collections are also indexable. This means that elements of such a collection may be retrieved from a numerical index, using the message at:. Array is the most familiar indexable data structure with a ﬁxed size. LinkedLists and SkipLists are sequenceable but not indexable, that is, they understand first and last, but not at:. 6

Note that canceling methods in subclasses is a pattern that exists outside the Smalltalk world. The Java runtime cancels many methods in its collection framework to express immutability by throwing UnsupportedOperationException.

258

S. Ducasse et al. Table 1. Standard collection protocols Protocol

Methods

accessing

size, capacity, at: anIndex, at: anIndex put: anElement

testing

isEmpty, includes: anElement, contains: aBlock, occurrencesOf: anElement

adding

add: anElement, addAll: aCollection

removing

remove: anElement, removeAll: aCollection, remove: anElement ifAbsent: aBlock

enumerating

do: aBlock, inject: aValue into: aBinaryBlock, collect: aBlock, select: aBlock, reject: aBlock, detect: aBlock, detect: aBlock ifNone: aNoneBlock

converting

asBag, asSet, asOrderedCollection, asSortedCollection, asArray, asSortedCollection: aBlock

creation

with: anElement, with:with:, with:with:with:, with:with:with:with:, withAll: aCollection

Keyed: Instances of Dictionary and its subclasses may be accessed by nonnumerical indices. Any object may be used as a key to refer to an association. Mutable: Most collections are mutable, but Intervals and Symbols are not. An Interval is an immutable collection representing a range of Integers. It is indexable with Interval >> at:, but cannot be changed with at:put:. Growable: Instances of Interval and Array are always of a ﬁxed size. Other kinds of collections (sorted collections, ordered collections, and linked lists) may grow after creation. Accepts duplicates: A Set ﬁlters out duplicates, but a Bag will not. This means that the same elements may occur more than once in a Bag but not in a Set. Dictionary, Set and Bag use the = method provided by the elements; the Identity variants of these classes use the == method, which tests whether the arguments are the same object, and the Pluggable variants use an arbitrary equivalence relation supplied by the creator of the collection. Contains speciﬁc elements: Most collections hold any kind of element. A String, CharacterArray or Symbol, however, only holds Characters. An Array will hold any mix of objects, but a ByteArray only holds Bytes. A LinkedList is constrained to hold elements that conform to the Link accessing protocol, and Intervals only contain consecutive integers, and not any numeric value between the bounds.

4

Experimental Process

Experience with traits [4,5] shows that they eﬀectively support reuse. We identiﬁed the problems of Section 2 in Squeak, a dialect of Smalltalk. Our ultimate

Reusing and Composing Tests with Traits

259

Table 2. Some of the key behaviors of the main Squeak class collection Implementation kind Arrayed Array, String, Symbol Ordered OrderedCollection, SortedCollection, Text, Heap Hashed Set, IdentitySet, PluggableSet, Bag, IdentityBag, Dictionary, IdentityDictionary, PluggableDictionary Linked LinkedList, SkipList Interval Interval Sequenceable access by index Array, String, Symbol, Interval, OrderedCollection, SortedCollection not indexed LinkedList, SkipList Non-sequenceable access by key Dictionary, IdentityDictionary, PluggableDictionary not keyed Set, IdentitySet, PluggableSet, Bag, IdentityBag

goal is to redesign core libraries of Squeak, favoring backward compatibility, but fully based on traits. Our results are available in Pharo, a fork of Squeak that provides many improvements, including a ﬁrst redesign of the stream library [5]. The current paper takes place in a larger eﬀort of specifying the behavior of the main collection classes, as a preliminary of designing a new library. Our approach to reuse tests is based on trait deﬁnition and composition. In both cases, our experimental process iterated over the following steps: Protocol identiﬁcation and selection. We selected the main protocols as deﬁned by the main classes. For example, we took the messages deﬁned by the abstract class Collection and grouped them together into coherent sets, inﬂuenced by the existing method categories, the ANSI standard, and the work of Cook [1,6]. Trait deﬁnitions. For each protocol we deﬁned some traits. As we will show below, each trait deﬁnes a set of test methods and a set of accessor methods which make the link to the ﬁxture. Note that one protocol may lead to multiple traits since the behavior associated to a set of messages may be diﬀerent: for example, adding an element to an OrderedCollection or a Set is typically diﬀerent and should be tested diﬀerently (we deﬁned two traits TAddTest, TAddForUniquenessTest for simple element addition). Now these traits can be reused independently depending on the properties of the class under test. Composing test cases from traits. Using the traits deﬁned in the previous steps, we deﬁned test case classes by composing the traits and specifying their ﬁxture. We did that for the main collection classes, i.e., often the leaves of the Collection inheritance tree (Figure 2). We ﬁrst checked how test traits would ﬁt together in one main test class, then applied the traits to other classes. For example, we deﬁned the traits TAddTest and TRemoveTest for adding and removing elements. We composed them in the test cases for OrderedCollection.

260

S. Ducasse et al.

Then we created the test cases for the other collections like Bag, etc. However, the tests would not apply to Set and subclasses, which revealed that variants of the tests were necessary in this case (TAddForUniquenessTest and TRemoveForMultiplenessTest).

5

Selected Examples

We now show how we deﬁne and compose traits to obtain test case classes. 5.1

Test Traits by Example

We illustrate our approach with the protocol for inserting and retrieving values. One important method of this protocol is at:put:. When used on an array, this Smalltalk method is the equivalent of Java’s a[i] = val: it stores a value at a given index (numerical or not). We selected this example for its simplicity and ubiquity. We reused it to test the insertion protocol on classes from two diﬀerent sub-hierarchies of the collection library: in the SequenceableCollection subclasses such as Array and OrderedCollection, but also in Dictionary, which is a subclass of Set and Collection. First, we deﬁne a trait named TPutTest, with the following test methods: TPutTest >> testAtPut self nonEmpty at: self anIndex put: self aValue. self assert: (self nonEmpty at: self anIndex) == self aValue. TPutTest >> testAtPutOutOfBounds "Asserts that the block does raise an exception." self should: [self empty at: self anIndex put: self aValue] raise: Error TPutTest >> testAtPutTwoValues self nonEmpty at: self anIndex put: self aValue. self nonEmpty at: self anIndex put: self anotherValue. self assert: (self nonEmpty at: self anIndex) == self anotherValue.

Finally we declare that TPutTest requires the methods empty, nonEmpty, anIndex, aValue and anotherValue. Methods required by a trait may be assimilated as the parameters of the traits, i.e., the behavior of a group of methods is parametrized by its associated required methods [8]. When applied to tests, required methods and methods deﬁning default values act as customization hooks for tests: to deﬁne a test class, the developer must provide the required methods, and he can also locally redeﬁne other trait methods. 5.2

Composing Test Cases

Once the traits are deﬁned, we deﬁne test case classes by composing and reusing traits. In particular, we have to deﬁne the ﬁxture, an example of the domain

Reusing and Composing Tests with Traits

261

objects being tested. We do this in the composing class, by implementing the methods that the traits require to access the ﬁxture. Since overlap is possible between the accessors used by diﬀerent traits, most of the time, only few accessors need to be locally deﬁned after composing an additional trait. The following deﬁnition shows how we deﬁne the test case ArrayTest to test the class Array: CollectionRootTest subclass: #ArrayTest uses: TEmptySequenceableTest + TIterateSequenceableTest + TIndexAccessingTest + TCloneTest + TIncludesTest + TCopyTest + TSetAritmetic + TCreationWithTest + TPutTest instanceVariableNames: ’example1 literalArray example2 empty collection result’

ArrayTest tests Array. It uses 9 traits, deﬁnes 10 instance variables, contains 85 methods, but only 30 of them are deﬁned locally (55 are obtained from traits). Among these 30 methods, 12 methods deﬁne ﬁxtures. The superclass of ArrayTest is CollectionRootTest. As explained later the class CollectionRootTest is the root of the test cases for collections sharing a common behavior such as iteration. ArrayTest deﬁnes a couple of instance variables that hold the test ﬁxture, and the variables are initialized in the setUp method: ArrayTest >> setUp example1 := #(1 2 3 4 5) copy. example2 := {1. 2. 3/4. 4. 5}. collection := #(1 -2 3 1). result := {SmallInteger . SmallInteger . SmallInteger . SmallInteger}. empty := #().

We then make the ﬁxture accessible to the tests by implementing trivial but necessary methods, e.g., empty and nonEmpty, required by TEmptyTest: ArrayTest >> empty ^ empty

ArrayTest >> nonEmpty ^ example1

TPutTest requires the methods aValue and anIndex, which we implement by returning a speciﬁc value as shown in the test method testAtPut given below. Note that here the returned value of aValue is absent from the array stored in the instance variable example1 and returned by nonEmpty. This ensures that the behavior is really tested. ArrayTest >> anIndex ^2

ArrayTest >> aValue ^ 33

TPutTest >> testAtPut self nonEmpty at: self anIndex put: self aValue. self assert: (self nonEmpty at: self anIndex) = self aValue.

These examples illustrate how a ﬁxture may be reused by all the composed traits. In the eventuality where a trait behavior would require a diﬀerent ﬁxture, new state and new accessors could be added to the test class.

262

S. Ducasse et al.

The class DictionaryTest is another example of a test case class. It also uses a slightly modiﬁed version of TPutTest. This trait is adapted by removing its method testAtPutOutOfBounds, since bounds are for indexed collections and do not make sense for dictionaries. The deﬁnition of DictionaryTest is the following: CollectionRootTest subclass: #DictionaryTest uses: TIncludesTest + TDictionaryAddingTest + TDictionaryAccessingTest + TDictionaryComparingTest + TDictionaryCopyingTest + TDictionaryEnumeratingTest + TDictionaryImplementationTest + TDictionaryPrintingTest + TDictionaryRemovingTest + TPutTest - {#testAtPutOutOfBounds} instanceVariableNames: ’emptyDict nonEmptyDict’

DictionaryTest uses 10 traits and deﬁnes 2 instance variables. 81 methods are deﬁned in DictionaryTest for which 25 are locally deﬁned and 56 are brought by the traits. For this class, a similar process happens. We deﬁne the setUp method for this class. Note that here we use a hook method classToBeTested, so that we can also reuse this test case class by subclassing it. DictionaryTest >> setUp emptyDict := self classToBeTested new. nonEmptyDict := self classToBeTested new. nonEmptyDict at: #a put: 20; at: #b put: 30; at: #c put: 40; at: #d put: 30. DictionaryTest >> classToBeTested ^ Dictionary

And similarly we redeﬁne the required methods to propose a key that is adapted to dictionaries: DictionaryTest >> anIndex ^ #zz

5.3

DictionaryTest >> aValue ^ 33

Combining Inheritance and Trait Reuse

It is worth noting that we do not oppose inheritance-based and trait-based reuse. For example, the class CollectionRootTest uses the following traits TIterateTest, TEmptyTest, and TSizeTest (see Figure 3). CollectionRootTest is the root of all tests, therefore methods obtained from these three traits are inherited by all test classes. We could have deﬁned these methods directly in CollectionRootTest, but we kept them in traits for the following reasons: – Traits represent potential reuse. It is a common idiom in Smalltalk to deﬁne classes that are not collections but still implement the iterating protocol. Having separate traits at hand will increase reuse.

Reusing and Composing Tests with Traits

empty nonEmpty

TSizeTest testSizeWhenEmpty

empty nonEmpty

263

TEmptyTest testifEmptyifNotEmpty testIfEmptyDo TIterateTest

collection expectedSizeAfterReject speciesClass

CollectionRootTest

OrderedCollectionTest

testCollect testDetect testAllSatisfy

T...

SetTest ArrayTest ...

TSetArithmetic testUnion testIntersection

Fig. 3. Test reuse by both trait composition and inheritance (the representation of traits shows required methods on the left and provided methods on the right)

– A trait is a ﬁrst class entity. It makes the required methods explicit and documents a coherent interface. – Finally, since our plans are to design a new collection library, we will probably reuse these protocol tests in a diﬀerent way.

6

Results

We now present the results we obtained by reusing tests with traits. Section 6.3 discusses the questions we faced during this work. 6.1

In the Nile Stream Library

Excluding tests, Nile is composed of 31 classes, structured around 11 traits: TStream, TDecoder, TByteReading, TByteWriting, TCharacterReading, TCharacterWriting, TPositionableStream, TGettableStream, TPuttableStream, TGettablePositionableStream, TPuttablePositionableStream. More details about the design of Nile may be found in the literature [5]. Nile has 18 test classes among which the ones listed on Table 3 use traits. The columns of Table 4 and Table 6 describe: Users: how many test case classes use each trait, either directly by composition or indirectly through both inheritance and composition; Methods: the trait balance in terms of required methods vs. provided tests5 ; Ad-hoc: deﬁned requirements or redeﬁned methods vs. overridden tests; Eﬀective: the number of unit tests executed during a test run that originate from the trait in column 1. For instance, TPuttableStreamTest is composed into 7 test case classes in total, through 4 explicit composition clauses; it requires 3 methods and provides 5 unit 5

The small numbers indicate any additional non-test methods that the traits provide.

264

S. Ducasse et al.

Table 3. Trait compositions in the Nile tests (only these 6 test classes use traits) Test case class

Traits composed

CollectionStreamTest

TGettablePositionableStreamTest + TPuttablePositionableStreamTest FileStreamTest TGettablePositionableStreamTest + TPuttablePositionableStreamTest HistoryTest TGettableStreamTest + TPuttableStreamTest LinkedListStreamTest TGettableStreamTest + TPuttableStreamTest RandomTest TGettableStreamTest SharedQueueTest TGettableStreamTest + TPuttableStreamTest

Table 4. Use, structure, adaptation, and beneﬁt of test traits in Nile Test trait

Users dir. / inh.

TGettablePositionableStreamTest TGettableStreamTest TPositionableStreamTest TPuttablePositionableStreamTest TPuttableStreamTest

2/4 5/8 2/8 2/4 4/7

Methods5 req. prov.

12 1 10 19 01 3 5 +1

Ad-hoc mth. tests

40 80 80 00 22 0

Eﬀective unit tests 8 80 72 4 35

tests and an auxiliary method. The test classes provide 22 deﬁnitions for its required methods: 3 requirements implemented in 7 test cases, plus 1 redeﬁnition of the auxiliary method. Overrides are to be expected since the ﬁxtures often can not be deﬁned in a completely abstract way; none of the tests had to be adapted, though, which is good. In total, the 5 tests are run for each of the 7 test cases, so TPuttableStream generates 35 unit tests. 6.2

In the Collection Library

The collection hierarchy is far richer than the stream hierarchy. It contains several behaviors, often orthogonal, that are intended to be recomposed. As previously, Tables 5 and 6 describe how we composed and deﬁned the test traits. The coverage of the collection hierarchy is in no way complete, and we expect to deﬁne other traits to cover behavior, like identity vs. equality of elements, homogeneous collections, weak-referencing behavior. . . When writing the test traits, we decided to make do with the collection classes as they exist, so the traits are much less balanced than with Nile. For instance, TGrowableTest only applies to collections that reallocate their storage, so we tested it only for OrderedCollection. However this situation is due to a lack of time since several other collections exhibit this behavior. In contrast, TIterateTest and

Reusing and Composing Tests with Traits

265

Table 5. Trait composition in the collection hierarchy tests Test case class

Traits composed

ArrayTest

TEmptySequenceableTest + TIterateSequenceableTest + TIndexAccessingTest + TCloneTest + TIncludesTest + TCopyTest + TSetAritmetic + TCreationWithTest + TPutTest BagTest TAddTest + TIncludesTest + TCloneTest + TCopyTest + TSetAritmetic + TRemoveForMultiplenessTest CollectionRootTest TIterateTest + TEmptyTest + TSizeTest DictionaryTest TIncludesTest + TDictionaryAddingTest + TDictionaryAccessingTest + TDictionaryComparingTest + TDictionaryCopyingTest + TDictionaryEnumeratingTest + TDictionaryImplementationTest + TDictionaryPrintingTest + TDictionaryRemovingTest + TPutTest − {#testAtPutOutOfBounds} IntervalTest TEmptyTest + TCloneTest + TIncludesTest + TIterateSequenceableTest + TIndexAccessingTest + TIterateTest − {#testDo2. #testDoWithout} LinkedListTest TAddTest − {#testTAddWithOccurences. #testTAddTwice} + TEmptyTest OrderedCollectionTest TEmptySequenceableTest + TAddTest + TIndexAccessingTest + TIncludesTest + TCloneTest + TSetAritmetic + TRemoveForMultiplenessTest + TCreationWithTest + TCopyTest + TPutTest SetTest TAddForUniquenessTest + TIncludesTest + TCloneTest + TCopyTest + TSetAritmetic + TRemoveTest + TCreationWithTest − {#testOfSize} + TGrowableTest + TStructuralEqualityTest + TSizeTest StackTest TEmptyTest + TCloneTest − {#testCopyNonEmpty} StringTest TIncludesTest + TCloneTest + TCopyTest + TSetAritmetic SymbolTest TIncludesTest + TCloneTest − {#testCopyCreatesNewObject} + TCopyPreservingIdentityTest + TCopyTest + TSetAritmetic − {#testDiﬀerenceWithNonNullIntersection}

TEmptyTest are composed by nearly all test cases, so they have a much higher reuse. We also had to adapt composed tests more: while the ﬁxtures are deﬁned abstractly, they have to be specialized for each tested class and, sometimes, we excluded or overrode test methods in a composition because they would not work in this particular test case. Table 6 shows that traits can achieve important code reuse. The presented results should also be interpreted with the perspective that the collection hierarchy is large and that we did not cover all the classes. For example, several kind of dictionary (open vs. closed implementation, keeping order) exist and were not covered. Therefore the results are deﬁnitively encouraging.

266

S. Ducasse et al.

Table 6. Use, structure, adaptation, and beneﬁt of test traits in the collection hierarchy Test trait TAddForUniquenessTest TAddTest TCloneTest TCopyPreservingIdentityTest TCopyTest TCreationWithTest TDictionaryAccessingTest TDictionaryAddingTest TDictionaryComparingTest TDictionaryCopyingTest TDictionaryEnumeratingTest TDictionaryImplementationTest TDictionaryPrintingTest TDictionaryRemovingTest TEmptySequenceableTest TEmptyTest TGrowableTest TIdentityAddTest TIncludesTest TIndexAccessingTest TIterateSequenceableTest TIterateTest TPutTest TRemoveForMultiplenessTest TRemoveTest TSizeTest TStructuralEqualityTest

6.3

Users dir. / inh. 1/1 3/4 9 / 11 1/1 6/7 3/3 1/2 1/2 1/2 1/2 1/2 1/2 1/2 1/2 2/2 6 / 13 1/1 1/1 8 / 10 3/3 2/2 2/9 3/4 2/3 2/4 2/9 2/1

Methods5 req. prov. 34 37 23 11 25 17 3 13 34 01 32 39 08 3 2 +1 3 4 +1 3 6 +3 28 53 2 1 +1 56 1 13 33 7 20 +2 43 11 24 22 24

Ad-hoc mth. tests

30 10 1 18 0 10 12 0 30 30 30 00 30 30 01 30 30 70 22 0 51 30 41 0 32 60 49 6 12 1 21 60 14 0 20

Eﬀective unit tests 4 28 33 1 35 21 26 8 2 4 18 16 4 8 12 104 3 1 60 39 6 180 12 3 16 18 4

What Did We Gain?

In the introduction of this paper we stated some research questions that drove this experiment. It is now time to revisit them. – Are test traits reusable in practice? If each feature was tested by a separate trait, how much test reuse could we obtain? What is a good granularity for test traits that maximizes their reusability and composition? We created 13 test classes that cover the 13 classes from the collection framework (Table 5). These 13 classes use 27 traits. The number of users for each trait ranges from 1 to 13. Globally, the traits require 29 unique selectors, and provide 150 test and 8 auxiliary methods. In the end, the test runner runs 765 unit tests, which means that on average, reuse is 4.7 unit tests run for each test written. If we do not count just tests but all (re)deﬁned methods, the ratio to unit tests run is still 1.8.

Reusing and Composing Tests with Traits

267

Moreover, since the classes we selected often exhibit characteristic behavior, we expect that once we will cover much more cases, the reuse will increase. – How far should a test ﬁxture be adapted to speciﬁc requirements? We had to deﬁne speciﬁc test ﬁxtures. For example, to test a bag or a set we need diﬀerent ﬁxtures. However, we could ﬁrst share some common ﬁxtures which were abstracted using trait required methods, second we could share them between several traits testing a given protocol. It was not our intention to reuse test ﬁxtures optimally, though. To optimize the ﬁxture reuse, we could have followed a breadth-ﬁrst approach by collecting all the constraints that hold on a possible ﬁxture (having twice the same element, being a character...) before writing any tests. However, this approach makes the hypothesis that the world is closed and that a ﬁxture can be really shared between diﬀerent test classes. We took a pragmatic approach and wanted to evaluate if our approach works in practice. In such a context, we had to deﬁne speciﬁc ﬁxtures but we could share and abstract some of the behavior using required methods. – How far should a test be parametrized to be reusable? What is the impact on the maintainability of test code? In the test traits, one out of 6 methods is required. Those unique 29 requirements lead to 237 implementations in the test classes to deﬁne the concrete ﬁxtures, but often they are trivial accessors. The diﬃculty was in striking a balance between reusing ﬁxtures constrained to test several aspects, or deﬁning additional independent ones.

7

Discussion

Threats to validity. The collection hierarchy is complex and dense in terms of the represented behavior. As we showed in Section 3.2, collections possess many similar elementary facets: order and objects (OrderedCollection), order and characters (String), no order and duplication (Bag), no order and uniqueness (Set)... therefore we imagine that the potential of reuse is higher than in normal domain classes. Still we believe that lots of systems designed with interfaces in mind would beneﬁt from our approach. For example, user interface or database mapping frameworks also often exhibit such multiple inheritance behavior. Factoring test code or grouping test data? As said in the introduction, without traits, it is still possible to test protocols and interfaces orthogonally to classes, provided the test case are parametrizable, like in JUnit 4.0 (See Section 8.2). With this approach, test cases only test a speciﬁc protocol but are applied to each domain class that should respect that protocol. Alternatively, with the traditional approaches like JUnit, generic test code can be factored through inheritance or auxiliary methods. The natural question is then what is the advantage of using traits vs. parametrized test cases. The JUnit scheme implies that generic test code must be grouped with the code that passes the test data, either all in one class, or in one hierarchy per tested protocol. Reusing generic test code is indeed a typical case of

268

S. Ducasse et al.

implementation inheritance: to group the tests for several protocols and one domain class together, one needs either multiple inheritance or delegation and lots of glue code. In contrast, with test traits deﬁning the generic test code, it is possible to compose the tests for several protocols into a class that deﬁnes the test data for a single domain, thus nicely separating both generic and domain-speciﬁc parts of the tests code. Moreover, since the domain-speciﬁc code controls trait composition, it may ignore or redeﬁne the generic test code on a case-by-case basis. Traits with a single use. In the current system, some test traits are not reused; for instance, the ones dealing with speciﬁc OrderedCollection behavior (e.g., removing the last element, or inserting an element at a precise position) are only used by OrderedCollectionTest, so we could have deﬁned the methods directly in this class. Whether it makes sense or not to have such speciﬁc protocol tests grouped as a trait is a design question —which is not speciﬁc to test traits but applies to using traits in general and to interface design. We believe that it still makes sense to deﬁne such cohesive test methods as a trait. The current collection library has a large scope but misses some useful abstractions; even if a trait is not reused currently, it is still a potentially reusable cohesive group of methods. For example, there is no collection which simultaneously maintains the order of elements and their uniqueness; if a new UniqueOrderedCollection is implemented, test traits specifying the ordering and uniqueness protocols will make it easy to ensure that UniqueOrderedCollection behaves like the familiar Set and OrderedCollection. The case of implementation or speciﬁc methods. When writing test traits we focused on public methods. In Smalltalk, since all methods are public, it is diﬃcult to know if one should be covered or not. Methods may be categorized into three groups: (i) implementation and internal representation methods, (ii) methods in a public interface speciﬁc to a given collection, and (iii) unrelated method extensions. We think that tests for the ﬁrst category are not worth reusing and should be deﬁned locally in the particular test case. As mentioned above, we believe it is worth to create a trait for the second group because it proactively promotes reuse. The last group of methods are convenience methods, so they belong to a diﬀerent library and should be tested there. For example, OrderedCollection >> inspectorClass should be covered by the tests for the GUI library —where it belongs. Beneﬁts for protocol/interface design. Our experience trying to identify common protocols highlighted a few inconsistencies or asymmetries; for instance OrderedCollection >> ofSize: and Array >> new: both create a nil-initialized collection of the given size, but OrderedCollection >> new: creates an empty collection that can grow up to the given capacity. We imagine that these inconsistencies could appear because the tests were not shared between several classes. Since protocol tests are easily reused speciﬁcations, we believe they support the deﬁnition of more consistent interfaces or uniform class behavior.

Reusing and Composing Tests with Traits

269

Designing traits for tests vs. for domain classes. When designing traits for domain classes (as opposed to test traits), we often found ourselves deﬁning required methods that could be provided by other traits. The idea there is that the method names act as join points and glue between traits, propagating behavior from one trait to another. However, in the context of this work, it is better to design traits with required methods named to avoid accidental conﬂicts with methods of other traits. This way, it is easier to provide speciﬁc values and behavior in the context of the speciﬁc collection class under test. If a value or behavior must be shared, a simple redirection method does the trick. Pluggable ﬁxtures and test traits. To support reuse between test traits, we tried to share ﬁxtures by making them pluggable. However, it was sometimes simpler to deﬁne a separate ﬁxture speciﬁc to a given protocol. Indeed, each protocol will impose diﬀerent constraints on the ﬁxture, and devising ﬁxtures that satisfy many constraints at the same time quickly becomes not practical. It is also important to understand how far we want to go with pluggability; for example, we did not use the possibility to execute a method given its name (which can be trivial in Smalltalk using the perform: message). We limited ourselves to use required methods to change collection elements, because we wanted to favor readability and understandability of the tests, even at the cost of potential reuse.

8 8.1

Related Work Inheritance-Based Test Class Reuse

The usual approach to run the same tests with several ﬁxtures is to write the tests in terms of an abstract ﬁxture and to redeﬁne setUp to provide various concrete ﬁxtures. This approach can be extended with Template/Hook Methods to organize and share tests within a class hierarchy; for example, the Pier and Magritte frameworks [15] use this approach. Their test suites total more than 3200 SUnit tests when run, for only about 600 actual test methods deﬁned; it is then interesting to understand the pros and cons of the approach. The principle is the following: the test hierarchy mirrors the class hierarchy (e.g., MAObjectTest tests MAObject). Some tests are abstract or just oﬀer default values and deﬁne hook methods to access the domain classes and instances under test (e.g., a method actualClass returns the domain class to test, and instance and others return instances under test). This approach allows one to test diﬀerent levels of abstraction. In the leaves of the hierarchy, the tests deﬁned in the superclass can be reﬁned and made more precise. While this approach works well, it shows the following limits: – Sometimes, test methods have to be cancelled in subclasses. – When the domain exhibits multiple inheritance behavior, this approach does not support reuse: it exhibits the same limits as single inheritance in face of need for multiple inheritance reuse. Indeed, besides copy/paste or delegation, there is no speciﬁc mechanism to reuse tests.

270

S. Ducasse et al.

This approach may be improved to reuse protocol tests. The idea is to deﬁne one test case class for each protocol and deﬁne an abstract ﬁxture. Then for each of the classes implementing the protocol, a subclass of the test case class is created and a ﬁxture is speciﬁcally deﬁned. The drawback of this approach is that the ﬁxture has to be duplicated in each of the speciﬁc subclasses for each protocol. 8.2

Parametrized Test Classes in JUnit 4.0

JUnit 4.0 provides a way to parametrize a test class intended to run a set of test cases over a diﬀerent data. For example, the code excerpt given below shows a test class that veriﬁes the interface IStack for two implementations. The method annotated with @Parameters must return a collection of test data objects, in this case instances of the stack implementations JavaStack and ArrayStack. The Parameterized test runner instantiates the test class and run its tests once for each parameter. @RunWith(Parameterized.class) public class IStackTest { private IStack stack; public IStackTest(IStack stack) { this.stack = stack; } @Parameters public static Collection stacks() { return Arrays.asList(new Object[ ][ ] { { new ArrayStack() }, { new JavaStack() } }); } @Test public void newStackIsEmpty() throws Exception { assertTrue(stack.isEmpty()); } }

When the ﬁxtures are easy to build, the parametrized test runner is thus a very convenient alternative to the usual xUnit approach of redeﬁning setUp in subclasses. However, this is not so clear if we consider complex ﬁxtures composed from several interrelated domain values, like we needed in our tests, and the @Parameters annotation introduces a dependency from the test code to the implementations: to test a new IStack implementor, one must modify IStackTest — or subclass it, which is the non-parametrized approach. In contrast, test traits group the protocol-level test code in an independent entity, while test classes encapsulate details speciﬁc to the tested implementation like building the ﬁxture. Especially, test classes control the composition of tests from several traits, and can deﬁne additional ad-hoc tests. This enables a workﬂow where the protocol can be documented by a generic test trait, and implementation tests can be organized in test classes that parallel the domain class hierarchy.

Reusing and Composing Tests with Traits

9

271

Conclusion

Single inheritance hampers code reuse of business and testing code. Currently, programming languages lack constructs that help reuse across diﬀerent unit tests. We propose test traits, an approach based on traits which reduces code duplication and favors composition of feature testing code. Our approach is particularly adapted to test protocols in class hierarchies with many polymorphic classes, and is applicable to other trait-based languages. We applied test traits to two large and critical Smalltalk libraries; in average we reused each test 4.7 times. This experiment shows deﬁnitive advantages of our approach: test reuse across several classes, test composition, simple ﬁxture parametrization. We will pursue our eﬀort to fully cover the collection hierarchy, and we expect to get higher test code reuse.

References 1. ANSI. American National Standard for Information Systems—Programming Languages—Smalltalk, ANSI/INCITS 319-1998 (1998) 2. Beck, K.: Simple Smalltalk testing: With patterns, http://www.xprogramming.com/testfram.htm 3. Black, A.P., Schärli, N.: Traits: Tools and methodology. In: ICSE (2004) 4. Black, A.P., Schärli, N., Ducasse, S.: Applying traits to the Smalltalk collection hierarchy. In: OOPSLA, vol. 38, pp. 47–64 (2003) 5. Cassou, D., Ducasse, S., Wuyts, R.: Traits at work: the design of a new trait-based stream library. Journal of Computer Languages, Systems and Structures 35(1), 2–20 (2009) 6. Cook, W.R.: Interfaces and speciﬁcations for the Smalltalk-80 collection classes. In: OOPSLA, vol. 27, pp. 1–15. ACM Press, New York (1992) 7. Dedecker, J., Van Cutsem, T., Mostinckx, S., D’Hondt, T., De Meuter, W.: Ambient-oriented programming in ambientTalk. In: Thomas, D. (ed.) ECOOP 2006. LNCS, vol. 4067, pp. 230–254. Springer, Heidelberg (2006) 8. Ducasse, S., Gîrba, T., Wuyts, R.: Object-oriented legacy system trace-based logic testing. In: European Conference on Software Maintenance and Reengineering (CSMR 2006), pp. 35–44. IEEE Computer Society Press, Los Alamitos (2006) 9. Flatt, M., Finder, R.B., Felleisen, M.: Scheme with classes, mixins and traits. In: AAPLAS (2006) 10. The Fortress language speciﬁcation, http://research.sun.com/projects/plrg/fortress0866.pdf 11. Godin, R., Mili, H., Mineau, G.W., Missaoui, R., Arﬁ, A., Chau, T.-T.: Design of class hierarchies based on concept (Galois) lattices. Theory and Application of Object Systems 4(2), 117–134 (1998) 12. Ingalls, D., Kaehler, T., Maloney, J., Wallace, S., Kay, A.: Back to the future: The story of Squeak, a practical Smalltalk written in itself. In: OOPSLA, pp. 318–326. ACM Press, New York (1997) 13. LaLonde, W., Pugh, J.: Subclassing = Subtyping = Is-a. Journal of ObjectOriented Programming 3(5), 57–62 (1991) 14. Meszaros, G.: XUnit Test Patterns – Refactoring Test Code. Addison-Wesley, Reading (2007) 15. Renggli, L.: Magritte — Meta-described web application development. Master’s thesis, University of Bern (2006)

Flow-Centric, Back-in-Time Debugging Adrian Lienhard, Julien Fierz, and Oscar Nierstrasz Software Composition Group, University of Bern, Switzerland http://scg.unibe.ch

Summary. Conventional debugging tools present developers with means to explore the run-time context in which an error has occurred. In many cases this is enough to help the developer discover the faulty source code and correct it. However, rather often errors occur due to code that has executed in the past, leaving certain objects in an inconsistent state. The actual run-time error only occurs when these inconsistent objects are used later in the program. So-called back-intime debuggers help developers step back through earlier states of the program and explore execution contexts not available to conventional debuggers. Nevertheless, even Back-in-Time Debuggers do not help answer the question, “Where did this object come from?” The Object-Flow Virtual Machine, which we have proposed in previous work, tracks the flow of objects to answer precisely such questions, but this VM does not provide dedicated debugging support to explore faulty programs. In this paper we present a novel debugger, called Compass, to navigate between conventional run-time stack-oriented control flow views and object flows. Compass enables a developer to effectively navigate from an object contributing to an error back-in-time through all the code that has touched the object. We present the design and implementation of Compass, and we demonstrate how flow-centric, back-in-time debugging can be used to effectively locate the source of hard-to-find bugs.

1 Introduction When debugging object-oriented systems, the hardest task is to find the actual root cause of a failure as this can be far from where the bug actually manifests itself [1]. In a conventional debugger, the developer is provided with an interface to explore the run-time context at a particular point in time in the execution of the program. Very often defective code leads very quickly to a run-time failure, so in these cases the run-time context is exactly what we need to identify bugs. Unfortunately there are also frequent situations where the failure occurs an arbitrary period of time after the defective code has executed, in which case the run-time context at the time of the failure contains little or no useful information about the source of the error. In a recent study, Liblit et al. examined bug symptoms for various programs and found that in 50% of the cases the execution stack contains essentially no information about the bug’s cause [2]. Back-in-time debuggers have been developed to help developers step back through earlier states of the program and explore execution contexts not available to conventional debuggers. Back-in-time debugging has gained increasing attention recently [3, 4, 5, 6]. Still, even Back-in-Time Debuggers are of only limited help, since they provide no dedicated mechanism to identify which past contexts led to the defective M. Oriol and B. Meyer (Eds.): TOOLS EUROPE 2009, LNBIP 33, pp. 272–288, 2009. © Springer-Verlag Berlin Heidelberg 2009

Flow-Centric, Back-in-Time Debugging

273

state. As a consequence, developers have to manually inspect the execution history to backtrack the origins of an object. Manual inspection of the execution history can be tedious because of the overwhelming amount of data that is typically recorded from program runs. Essentially, we need an effective way to answer the question: “Where did this object come from?”. We have previously proposed an approach to track the flow of objects in a dedicated Object Flow Virtual Machine to answer precisely such questions [7], but the prototype VM does not include any debugging support to exploit object-flows. This paper proposes the novel concept of flow-centric debugging to support an effective navigation of object-oriented execution history during debugging, and presents Compass, a proof-ofconcept flow-centric debugger based on the Object Flow VM. Compass provides several novel views to aid the developer in navigating between the historical control flow and object flow information spaces. One of these views represents the entire history of a running application as an interactive call graph using a “fisheye” view. In this view, the method execution currently in focus is represented as a large circle, and more distant nodes are correspondingly smaller. As one navigates through the call graph, nodes shift and are resized as though they were viewed through a fisheye lens. The contributions of this paper are: (i) a novel approach, called flow-centric debugging, to bridge control flow views offered by conventional debuggers and historical object state views of Back-in-Time Debuggers, and (ii) a scalable and practical implementation strategy with a proof-of-concept prototype called Compass. Outline. In Section 2 we present a simple example to motivate the need for flow-centric debugging. In Section 3 we review existing approaches to back-in-time debugging, and demonstrate how they fall short in tracing back through object flows. Section 4 describes Compass, a proof-of-concept, object-centric, Back-in-Time Debugger. Section 5 describes a realistic example that illustrates how Compass can be used effectively to discover hard-to-track software bugs. In Section 6 we present our implementation, which is based on a metamodel for a debugger that integrates control and object flow into a common approach. Section 7 presents conclusions and remarks on possible future extensions.

2 Motivating Example Let us consider a simple toy example that illustrates the shortcomings of both conventional debuggers and Back-in-Time Debuggers for tracking down a common class of software defects. Figure 1 shows a UML sequence diagram of a scenario leading to a software failure. A Bank is responsible for logging new deposits and adding interest to individual BankAccount instances. It makes a request to an InputReader instance for (1) a new deposit, which is (2) returned to the Bank. The returned value happens to be nil, but this does not lead to an immediate failure. The value is (3) returned to the Bank from a helper method. The value is (4) saved in a BankAccount and (5) stored in a private field. The Bank then triggers the BankAccount to compute interest, so it (6) reads

274

A. Lienhard, J. Fierz, and O. Nierstrasz Bank BankAccount

a BankAccount

an InputReader

an InterestCalculator

example getDeposit

readDeposit nil

1

2 nil 3 4 deposit: (deposit:nil) 5

addInterest deposit

6 nil

7

8 calculateNewDeposit (deposit : nil, interest: 0.05)

Fig. 1. Sequence diagram of the crashing example

the value with an accessor that (7) returns it. The BankAccount then (8) passes the value to the InterestCalculator instance, which blows up when it tries to compute interest for a nil value. Now consider a developer trying to debug this with a conventional debugger. We can clearly see that the failure is due to the nil value, but the debugger only gives us the run-time context back to the point where the value was (6) read from the field where it was stored. We would have to set a breakpoint where the variable is stored (5) and re-run the program. Now we can see that the value was returned (3) from the method getDeposit, but since this method context is not on the stack anymore, we have to set another breakpoint to find out where the value comes from, etc. A Back-in-Time Debugger would allow us to step back in time, but we do not know how far back in time we must travel. We have no information to tell us where to go. Although the given scenario is tiny, a real-world scenario might include many thousands of events between the failure and the point where the faulty value is computed and stored. As we shall see in the following section, present Back-in-Time Debuggers do not offer adequate mechanisms to identify the contexts that the faulty object has passed through.

Flow-Centric, Back-in-Time Debugging

275

3 Limitations of Current Back-in-Time Debuggers Next we look briefly at three examples of Back-in-Time Debuggers and summarize the mechanisms they provide for navigating through historical contexts. In particular we look at how well they support the task of tracing object flow. TOD. The Trace-Oriented Debugger [4] is a Back-in-Time Debugger for Java, implemented as an Eclipse plugin, that improves the performance by using a high-speed distributed database backend. Omniscient Debugger. The Omniscient Debugger [3] is a Back-in-Time Debugger for Java, which instruments bytecode at load time, and offers a user interface to step through the generated traces and inspect the state of objects at any time. Unstuck. The Unstuck debugger [5] is a Back-in-Time Debugger for Squeak Smalltalk1 . These approaches adopt a common perspective of the program execution history. All three debuggers provide views that are stack-oriented and state-centric. The user can browse and step through the method execution trace (run-time control flow) and inspect the past state of objects. Figure 2 shows a debugging session of our running example introduced in Section 2 with the Trace-Oriented Debugger (TOD). The stack is displayed in view (A) and the current values of variables are shown in view (D). With the buttons (B) the user can step forward and backward in the recorded control flow. This perspective is a natural extension from conventional debuggers, which show the current call stack and a snapshot of the current state of objects. Back-in-time debuggers have extended these views with historical run-time data. The call stack is extended to the complete execution trace, and the object inspector is extended to all previous values of an object’s fields. Considering our example, with this perspective we can identify the faulty value of the deposit field in the bank account instance. Furthermore, we can jump to any past method execution. However, none of the three debuggers provides sophisticated support to link the symptom of the bug (null pointer exception) with its root cause (where the null value is introduced). Hence, we do not know how far back we need to jump. If we knew the origin of null, the root cause could be identified in one step. In practice, however, bugs tend to be more complex as faulty variables may infect other variables [8,9]. Therefore, to trace back the infection chain, often several objects have to be backtracked. The features that the TOD, Omniscient Debugger, and Unstuck provide to backtrack object flow are ad hoc: – TOD: a “why?” hyperlink (see Figure 2 (D)) allows the focus to be moved to the statement that assigned the current value – Omniscient Debugger offers similar functionality like TOD – Unstuck: allows one to color all methods in the call trace in which a selected object was used 1

http://squeak.org

276

A. Lienhard, J. Fierz, and O. Nierstrasz

C

B

A

D

Fig. 2. TOD. Control flow (A) can be stepped (B), and stays in sync with events in the source code (C). Current objects are also shown (D).

The features provided by TOD and Omniscient Debugger support only one backtracking step (from a field value to where it was assigned or from a method parameter to where the method was called). Hence, to get to the origin of the value, manual steps are required. This includes figuring out by which method execution an object was returned, which can be hard if loops are involved. Therefore, backtracking non-trivial object flows can quickly become a tedious task. The coloring feature provided by Unstuck (see Figure 3) gives a good overview of how often an object was used but it provides only limited support for tracking object flows. For objects that are often used, a lot of code gets highlighted even if the code is not in the path of the object that the developer is interested in. This limitation becomes especially obvious if one wants to backtrack the flow values like null, true, and false. Since these values occur in a very large number of methods, the highlighting of Unstuck is of little use. Further related work. Omnicore’s CodeGuide2 and RetroVue3 , two commercial products, provide similar stack-oriented functionality like the three above presented approaches. Probably the most interesting earlier related work is ZStep95, a reversible debugger for Lisp. A notable feature of ZStep95 is to directly access from graphical 2 3

http://www.omnicore.com http://www.visicomp.com

Flow-Centric, Back-in-Time Debugging

277

Fig. 3. Unstuck. The method trace (at left) stays in sync with the source code view (at right). Methods of the currently selected object are highlighted (reprinted from reference [5]).

Control flow space

? Object flow space

Fig. 4. A debugger must offer mechanisms to navigate between control and object flow views

objects the code that drew them [10]. Approaches, such as the Data Display Debugger [11] and Reference Patterns [12], explicitly focus on graphical state-centric views. Most other back-in-time debugging related work is focused on improving performance and addressing the memory explosion problem rather than researching ways to exploit the gathered data [13, 14, 15]. Problem summary. Since existing Back-in-Time Debuggers adopt the traditional stack-oriented and state-centric views and introduce only minimal innovations to take history into account, they fail to capture the orthogonal issue of object flow. As Figure 4 illustrates, the problem we face is to link the control flow and object flow spaces. What we are missing is a means to link these spaces and to provide views to navigate between them.

278

A. Lienhard, J. Fierz, and O. Nierstrasz

4 Compass: A Flow-Centric Back-in-Time Debugger Compass4 is a proof-of-concept debugger for object-oriented programs. We chose the name “Compass” because its goal is to be an instrument that provides directions to effectively navigate the large information space. Compass is implemented using the Object Flow VM, which efficiently tracks object-flow at run-time [7]. The Compass user interface is shown in Figure 5. We briefly introduce each of the views provided by Compass, and then we focus on the views that are especially interesting from the perspective of navigating between the object and control flow spaces.

8

1

2

3

4

5

6

9

7

Fig. 5. The Compass debugger

1. The method trace view visualizes the entire run-time control flow as a tree of nodes in a fisheye view. Object flows are shown as green arrows between nodes. 2. Call stack of the currently selected method execution. 3. Lists each reference transfer of the flow of the object selected by the developer, by, e.g., double-clicking on an item in the sub-method statements list (5). 4. Source code of the selected method and buttons to step forward and backward through the method trace. 5. List of reference transfers and message sends of current method execution. 6. Variable inspector panes. 7. Lists of control flow statements the method depends on. 8. Displays changes in relationships between objects in the subtrace of the selected method execution. 9. Lets the developer go back and forth, or jump to bookmarked method executions. We now focus on how these views present control flow and object flow, and how they enable navigation between these two spaces. 4

http://scg.unibe.ch/Research/ObjectFlow/

Flow-Centric, Back-in-Time Debugging

279

Fig. 6. A method trace view synchronized with a stack view. Mousing over a node pops up a tool tip identifying the method execution. Clicking on it shifts the focus to that node.

4.1 Navigating Method Execution Traces Compass provides two synchronized views of method execution traces. The method trace view ((1) in Figure 5) shows the entire method execution trace as a tree, and view (2) offers a more conventional stack-oriented view of a single slice of the trace, focused on a given method execution. The challenge in presenting the entire trace as a tree is that the obvious approach of using a conventional tree list does not scale well. For this reason Compass offers a novel approach to navigating traces based loosely on the notion of a fisheye view [16]. The key idea is to present method executions not as text but as circles, showing the currently selected method execution as a large circle in the center of the view. More distant method executions are shown as progressively smaller circles, progressively closer to the edge of the view. In Figure 6 we see the currently selected method execution simultaneously highlighted in the method trace view and the call stack view. Note that we sacrifice the textual feedback of a tree list view for a high level graphical overview of the method execution tree, but we lose little since the synchronized stack view gives us detailed information of the method execution in focus. We can rapidly navigate to nearby nodes in the tree by clicking on them. Since nodes in neighbouring branches are not represented in the stack view, the fisheye view offers tooltips with the name of each method execution as the mouse hovers over its node.

280

A. Lienhard, J. Fierz, and O. Nierstrasz

The layout of the method trace tree is not strictly a conventional fisheye view because we independently scale the x and y coordinates of each node. We compute the x coordinate as a function ft of the timestamp t of a given node, as shown in the following equation, where tf ocus is the start timestamp of the currently selected method execution, Ct is a constant configured in the tool, and w is the width of the view: w + ft · w2 (t ≥ tf ocus ) 1 ft = 1 − x = w2 w |tf ocus − t| · Ct + 1 (t < tf ocus ) 2 − ft · 2 The value of ft is always in the interval [0, 1); the more ft tends to 0, the closer the node is positioned horizontally to the focused node. The y coordinate is computed similarly, but using the nesting level (depth of the call stack) of method executions instead of their timestamp. The size s of each circular node is computed as follows: s = (1 − ft ) · sf ocus where the constant sf ocus is the size of the focused method execution. Both constants sf ocus and Ct , where the latter defines the relative distance between nodes, can be adjusted by the user. When the focused node changes, the transition from the old to the new state of the method trace view is animated. The smooth transition is intended to help the user follow how the point of view changes. This proved especially useful when backtracking object flows that jump between a large number of method executions. 4.2 Navigating Object Flows Object flow is visualized as follows: for a given method execution selected in either the method trace view (1) (Figure 5) or the stack view (2), all object flow events are listed in view (5). An object flow is selected by double-clicking on one of these events. The entire object flow is then simultaneously displayed textually as a list in view (3), and graphically as green arrows connecting nodes in the method trace view (1), as shown in Figure 7. An object flow answers the question “Where did this object come from?” by displaying the path of reference transfers from the instantiation event to the selected reference. One may then navigate through the object flow by using either the textual or graphical views. The beginning of an object flow represents the creation event. An arrow from a parent to a child node usually means that an object reference was passed as a method argument, and an arrow from a child to its parent node means that the reference was returned from that method. Arrows between more distant nodes mean that the first node wrote the object to a field and that the second node read it. Arrows between parent and child nodes may also indicate that a field was written and then read, but this is relatively rare.

Flow-Centric, Back-in-Time Debugging

281

Fig. 7. Object flow visualized in method trace. Synced with the stack view at bottom left and the list of reference transfers of the selected object flow at bottom right.

To directly navigate to where the object was instantiated one can select the last item in the textual list. (Like the call stack, which grows from bottom to top, the most recent transfer of the object is at the top of the list and its instantiation is at the bottom of the list.) Often, not only the flow of an object is of interest but also where one of its field values comes from. Compass and the Object Flow VM treat objects and values of primitive type the same. Hence, to track back the origin of a field value (even if it is nil) one simply changes the focus of Compass to this value. For more details about the other views and the design and implementation of Compass, the interested reader is referred to the Masters thesis by Fierz [17].

5 Flow-Centric Debugging in Action In this section we demonstrate how Compass can be used in practice with an example of an actual, hard-to-debug defect in a multi-threaded application. The defect occurred in Pharo Smalltalk5 in the software update feature, which downloads and installs both kernel and library updates. The example has been simplified for presentation purposes, but otherwise reflects the actual defect. Problem. The example application downloads and installs software updates using two concurrent processes. The download process is handled by an UpdateDownloader class. 5

http://pharo-project.org

282

A. Lienhard, J. Fierz, and O. Nierstrasz

It puts updates into a queue shared with an installer process and puts a special marker in the queue to indicate that all updates have been downloaded: self getUpdates withIndexDo: [ :code :i | aSemaphore wait. aQueue nextPut: i. aQueue nextPut: code ]. aQueue nextPut: ''. aQueue nextPut: #finished

The installer process is handled by an UpdateLoader class, which simply gets updates from the shared queue and evaluates them: [ this := docQueue next. nextDoc := docQueue next. nextDoc ∼= #finished ] whileTrue: [ Compiler evaluate: nextDoc. docQueueSema signal ].

Unfortunately the application freezes unexpectedly. If we interrupt the application, a traditional debugger tells us that the update loader is blocked on an empty queue. This suggests that the downloader has not only stopped putting updates into the queue, but it has also failed to put the “finished” marker into the queue. Inspecting the stack trace and reading the source code does not help us to get any further in tracking down the source of the problem. Solution. In Compass, when more than one process is traced, the method trace view simply displays more than one root method execution. Starting Compass, we find ourselves in the context of the block method execution of next in the shared queue. Stepping up the stack one level, we find ourselves in the UpdateLoader method responsible for sending the blocking request (Figure 8). We see here that next has been sent 5 times, after which the method blocks on the shared queue. Therefore, this context is clearly interesting and we bookmark it so that we can easily return here. Now we want to understand where the objects in the shared queue are coming from, so we double-click the statement return '4+4' to backtrack the flow of the string '4+4' (Figure 9).

Fig. 8. Execution of the update loader

Flow-Centric, Back-in-Time Debugging

283

Fig. 9. Object flow of a element in the queue

Fig. 10. Loop on downloaded updates

Fig. 11. Side effects of the executed update code

We follow back the object flow, which brings us from the update loader process to the execution of withIndexDo: in the download process (see Figure 10). We can clearly see that a fifth update has been downloaded, but then the downloader stops abruptly. Something has interrupted this process, but we do not know what. The other active process is the update loader, so we follow our bookmark back to there. We note (Figure 8) that updates read from the queue are evaluated by the compiler, so now we step into the last evaluation and explore what happened during the last update. Quickly we arrived at the cleanup method of the update loader and remark something unusual in the side effects graph (right side of Figure 11), namely that the update loader used to have a reference to the downloader, but now it points to nil. Furthermore, the downloader no longer refers to its process. Stepping further we discover that the cleanup method terminates the downloader and nils it out (Figure 12). Evidently this update has erroneously initiated the cleanup procedure, which is only supposed to be invoked when all updates have been downloaded. Alternative solution. Of course, there are other ways to home in on the defect. For example, the update loader initializes and starts the downloader as follows:

284

A. Lienhard, J. Fierz, and O. Nierstrasz

retrieveUpdatesOntoQueue: aQueue withWaitSema: aSemaphore self downloader ifNotNil: [ self downloader terminate ]. self downloader: (UpdateDownloader new initializeWith: aQueue with: aSemaphore). self downloader start

Fig. 12. Cleanup method that stops the download process

Fig. 13. The forward flow of the downloader object

We can then use another feature of Compass, the forward flow explorer, to see what happens with the downloader afterwards (Figure 13). The forward flow shows us that the downloader is later instructed to terminate itself. When we shift our focus to this point, the run-time stack tells us that this request has originated from one of the downloaded updates.

6 Implementation: A Flow-Centric Debugging Metamodel Existing approaches commonly collect run-time data by introducing sensors into the bytecode to emit events whenever a method is entered or exited, or when a variable is assigned. The result is a trace of events organized into a control flow. The data for the user interface is then obtained by querying this trace. In contrast, we implement our new approach to flow-centric debugging by making the relevant metamodel explicit. In order to effectively exploit the opportunities offered by Back-in-Time Debuggers, we must connect these views at the metamodel level. The metamodel is populated by run-time data gathered by the Object Flow VM, which tracks object-flow by creating an alias whenever an object reference is transferred [7]. Figure 14 presents our proposal for a combined metamodel. The upper portion of the figure shows our metamodel for representing control flow information [18]. The lower portion presents our metamodel for expressing object flow.

Flow-Centric, Back-in-Time Debugging

285

Fig. 14. A metamodel combining control flow and object flow

Control flow. The execution trace is modeled by the entity MethodExecution, which points to the caller and holds a timestamp to capture the order and nesting of method executions in the control flow. Each method execution is associated with the entity Method, its static counterpart. The code of a method is represented as a list of statements (we could also use an abstract syntax tree representation, but a flat list is sufficient for our purpose). This control flow metamodel provides the means to step through the runtime control flow and to map each method execution to the source code statement from which it was called (notice the association between MethodExecution and the Send statement). Hence, one main reason for capturing single statements is to be able to map dynamic events, like method executions, to the source code. Object flow. Object-flow is captured by representing every object reference created at run-time as an instance of the entity Alias. A new alias is created whenever an object is created, passed as a parameter to a method, returned from a method, written to a field, or read from a field. The flow of an object is then given by all its aliases. Through the origin association, each alias points to the alias from which it originates. This allows us to track the flow of an object from a specific point in the program execution back to

286

A. Lienhard, J. Fierz, and O. Nierstrasz

where it was instantiated. Furthermore, aliases capture historical object state, modeled by the predecessor association. The combined control flow and object flow metamodel establishes the links between these two: each alias is linked to the control flow that caused the alias to be created. That is, each alias captures the method execution in which it is created and it captures the static statement that caused its creation. This linking in our metamodel allows us to precisely map object flow to control flow. Hence, the metamodel not only supports stepping in the execution trace, but it also directly supports navigating along the flow of objects. Scalability. Benchmarks of our Object Flow VM have shown significant improvements over existing approaches. By only keeping track of still-relevant past data, memory consumption stays within reasonable bounds and the run-time overhead is reduced. Our solution makes use of the garbage collector to release the objects that are not referenced anymore in memory and that are not relevant anymore in the program’s history. The relevance of a historical data point depends, among other properties, on the flow of objects. For instance, a method context (and its associated call stack) is retained in history as long as objects exist in memory that were passed through this method context. The Object Flow VM tracks object flow and hence the Compass debugger can directly access this information without requiring a static analysis. For a more detailed description of the Object Flow VM and its performance characteristics we refer the interested reader to our previous work [7, 19].

7 Conclusions and Future Work We have proposed flow-centric, back-in-time debugging as a novel way to improve the effectiveness of Back-in-Time Debuggers by tracking the flow of object references back through the method executions responsible for their flow. We have presented Compass, a proof-of-concept prototype that offers several novel views to aid the developer in navigating between historical control flow and object flow views. We have presented a typical scenario that demonstrates how empirically hard-to-find defects can be effectively tracked down with flow-centric debugging. Finally we have formalized our approach by presenting a simple metamodel that integrates conventional control-flow analysis of running applications with object flow analysis. Although Compass was not especially designed for debugging multi-threaded applications, it turns out to be remarkable effective since object flows capture precisely the interesting interactions between processes. The fisheye view displays the method executions of each relevant process as a tree. These trees then interleave. The view could be further improved by better distinguishing the different processes, yet keeping the property of revealing object reference transfers between processes. We have not yet carried out an empirical study to evaluate the effectiveness of flowcentric debugging. We hypothesize that Compass will considerably speed up the process of identifying the root cause of typical bugs in object-oriented programs compared to the identical tool lacking object-flow navigation features.

Flow-Centric, Back-in-Time Debugging

287

Acknowledgments We gratefully acknowledge the financial support of the Swiss National Science Foundation for the project “Bringing Models Closer to Code” (SNF Project No. 200020121594, Oct. 2008 - Sept. 2010). We would also like to thank David R¨othlisberger and Tudor Gˆırba for their help in reviewing drafts of this paper.

References 1. Zeller, A.: Why Programs Fail: A Guide to Systematic Debugging. Morgan Kaufmann, San Francisco (2005) 2. Liblit, B., Naik, M., Zheng, A.X., Aiken, A., Jordan, M.I.: Scalable statistical bug isolation. In: Proceedings of the 2005 ACM SIGPLAN conference on Programming language design and implementation (PLDI 2005), pp. 15–26. ACM, New York (2005) 3. Lewis, B.: Debugging backwards in time. In: Proceedings of the Fifth International Workshop on Automated Debugging (AADEBUG 2003) (October 2003) 4. Pothier, G., Tanter, E., Piquer, J.: Scalable omniscient debugging. In: Proceedings of the 22nd Annual SCM SIGPLAN Conference on Object-Oriented Programming Systems, Languages and Applications (OOPSLA 2007), vol. 42(10), pp. 535–552 (2007) 5. Hofer, C., Denker, M., Ducasse, S.: Design and implementation of a backward-in-time debugger. In: Proceedings of NODE 2006, Gesellschaft f¨ur Informatik (GI), September 2006. Lecture Notes in Informatics, vol. P-88, pp. 17–32 (2006) 6. Maruyama, K., Terada, M.: Debugging with reverse watchpoint. In: Proceedings of the Third International Conference on Quality Software (QSIC 2003), Washington, DC, USA, p. 116. IEEE Computer Society, Los Alamitos (2003) 7. Lienhard, A., Gˆırba, T., Nierstrasz, O.: Practical object-oriented back-in-time debugging. In: Vitek, J. (ed.) ECOOP 2008. LNCS, vol. 5142, pp. 592–615. Springer, Heidelberg (2008) 8. Zeller, A.: Isolating cause-effect chains from computer programs. In: SIGSOFT 2002/FSE10: Proceedings of the 10th ACM SIGSOFT symposium on Foundations of software engineering, pp. 1–10. ACM Press, New York (2002) 9. Cleve, H., Zeller, A.: Locating causes of program failures. In: ICSE 2005: Proceedings of the 27th international conference on Software engineering, pp. 342–351 (2005) 10. Lieberman, H., Fry, C.: ZStep 95: A reversible, animated source code stepper. In: Stasko, J., Domingue, J., Brown, M.H., Price, B.A. (eds.) Software Visualization — Programming as a Multimedia Experience, pp. 277–292. The MIT Press, Cambridge (1998) 11. Zeller, A., L¨utkehaus, D.: DDD — a free graphical front-end for Unix debuggers. SIGPLAN Not. 31(1), 22–27 (1996) 12. De Pauw, W., Sevitsky, G.: Visualizing reference patterns for solving memory leaks in Java. In: Guerraoui, R. (ed.) ECOOP 1999. LNCS, vol. 1628, pp. 116–134. Springer, Heidelberg (1999) 13. Feldman, S.I., Brown, C.B.: Igor: a system for program debugging via reversible execution. In: Proceedings of the 1988 ACM SIGPLAN and SIGOPS workshop on Parallel and distributed debugging (PADD 1988), pp. 112–123. ACM, New York (1988) 14. Boothe, B.: Efficient algorithms for bidirectional debugging. In: Proceedings of the ACM SIGPLAN 2000 conference on Programming language design and implementation (PLDI 2000), pp. 299–310. ACM, New York (2000) 15. Xu, G., Rountev, A., Tang, Y., Qin, F.: Efficient checkpointing of java software using contextsensitive capture and replay. In: Proceedings of the the 6th joint meeting of the European software engineering conference and the ACM SIGSOFT symposium on The foundations of software engineering (ESEC-FSE 2007), pp. 85–94. ACM, New York (2007)

288

A. Lienhard, J. Fierz, and O. Nierstrasz

16. Furnas, G.W.: Generalized Fisheye View. In: Proceedings of CHI 1986 (Conference on Human Factors in Computing Systems), pp. 16–23. ACM Press, New York (1986) 17. Fierz, J.: Compass: Flow-centric back-in-time debugging. Master’s thesis, University of Bern (January 2009) 18. Richner, T.: Recovering Behavioral Design Views: a Query-Based Approach. Ph.D thesis, University of Bern (May 2002) 19. Lienhard, A.: Dynamic Object Flow Analysis. Ph.D thesis, University of Bern (December 2008)

A Classification Framework for Pointcut Languages in Runtime Monitoring Karl Klose and Klaus Ostermann University of Aarhus, Denmark {klose,ko}@cs.au.dk Abstract. In runtime monitoring and aspect-oriented programming, the execution of a program is monitored to check whether a property – formulated in a pointcut language – holds at some point in the execution of the program. Pointcut languages diﬀer signiﬁcantly in their expressiveness and the amount of information they utilize about the state of the program’s execution, and the relation between diﬀerent pointcut languages is often not clear. We propose a formal framework that provides the common abstractions of these languages and identiﬁes the points of variability, such that pointcut languages can be compared and classiﬁed with regard to their expressiveness and cost. Besides its usage as a common frame of reference for pointcut languages, our framework also gives a precise model of the design space of pointcut languages and can hence help to design future pointcut languages in a more principled way.

1

Introduction

There are many diﬀerent languages that are used to describe conditions on the state or structure of a running program. These conditions are used to react on speciﬁc events or states in the software system. Examples are pointcut languages in aspect-oriented programming [9], break-point conditions in debugging systems [15], or speciﬁcation languages in runtime veriﬁcation and security enforcement (e.g., [10]). We will use the terminology from aspect-oriented programming and use the term pointcut languages to subsume all of these languages in the subsequent. Depending on the application domain, the program may be halted or additional functionality can be executed (so called advice) if a pointcut matches the current point in execution. In contrast to general purpose programming languages, pointcut languages diﬀer signiﬁcantly in two important dimensions. First, the expressive power of the languages ranges from simple forms of pattern matching to Turing-complete languages. Second the model of the program execution available in the pointcut language is quite diﬀerent between systems. Examples for the diﬀerent ﬂavors of pointcut languages range from simple line numbers used in debuggers via sophisticated domain-speciﬁc languages such as the AspectJ [8] pointcut language to the usage of various general computation models, such as regular expressions or context-free grammars over callstacks or execution history [17], or Datalog/Prolog/XQuery queries over some representation of the program execution [1,14] or the static program structure [4,5,7]. M. Oriol and B. Meyer (Eds.): TOOLS EUROPE 2009, LNBIP 33, pp. 289–307, 2009. c Springer-Verlag Berlin Heidelberg 2009

290

K. Klose and K. Ostermann

The aim of this work is to bring order into the huge and so far quite heterogenous design space of pointcut languages. This heterogeneity is multi-dimensional, and it includes the kinds of information the pointcut language operates on, and the granularity and expressiveness of the language on this information. This makes comparisons between pointcut languages and principled design of new pointcut languages quite diﬃcult, which is an important task, because the differences between the expressiveness of the language and the richness of the execution model have signiﬁcant impact on the performance of the resulting system. Using a formal model we can identify groups of pointcut languages which have similar runtime and memory performance and can make use of the same implementation and optimization techniques. Finally, a formalization allows to prove whether or not two pointcut languages are equal, and what the diﬀerences are, if they are not. At the moment, the lack of a formalization makes analyses very ad-hoc and diﬃcult. The aim of this work is to improve the current, undesirable situation by the following contributions: – We give precise formal deﬁnitions of pointcut-related terminology as mathematical structures based on the base language’s semantics. – We present a multi-dimensional decomposition of the pointcut language design space and discuss both the signiﬁcance of the dimensions and classiﬁcations of existing languages with regard to these dimensions. In particular we make a clear distinction between the richness of the underlying data model and the expressiveness of the pointcut language itself. This distinction is often not made or left implicit when comparing pointcut languages, leading to confusion about how to compare expressiveness at all. – We give a methodology on how to compare pointcut languages, based on the formal deﬁnitions. – We discuss how our framework can be used a basis for the formal description of static analysis and program transformations for aspect oriented programs. The remainder of this paper is structured as follows: First we introduce our notion of joinpoint abstractions and models (Sec. 2). We give examples for models and present a small case study showing two diﬀerent pointcut models. Next we deﬁne pointcuts and pointcut languages (Sec. 3). Building on these prerequisites, we show how pointcut languages can be compared in Sec. 4. In Sec. 5 we discuss language implementation issues in the terms of our framework. Finally, we discuss related work and conclude (Sec. 6 and 7).

2

Semantics and Joinpoint Models

Our ﬁrst step towards a formal framework for pointcut languages is to deﬁne the data model upon which joinpoints and pointcuts can be deﬁned. In the following we introduce small-step semantics as the base notation for the semantics of the underlying base language and show how properties of execution traces in this semantics can be described.

A Classiﬁcation Framework for Pointcut Languages

2.1

291

The Semantics of the Base Language

There are several alternative formalism to model the semantics of programming languages, such as denotational semantics, big-step operational semantics, and small-step operational semantics. Of these semantics frameworks, small-step operational semantics is the most suitable framework to talk about pointcuts, since it has a precise and direct notion of a computation step, which allows to talk about traces, execution order, etc. Hence in the following we assume that the base language is deﬁned by a small-step operational semantics. We will model the small-step semantics of our base languages by a binary relation −→ on a set Σ of states: −→⊂ Σ × Σ. A state σ ∈ Σ can often be split into several parts, such as environment, expression, heap etc., but for our purposes it is suﬃcient to assume that the state contains all relevant information that is necessary to compute the respective next computation step. In some situations it is desirable to change the usual small-step semantics in such a way that information relevant for pointcuts can be extracted more easily. For example, in a Featherweight Java [6] trace, it is not easy to determine the lexical origin of an expression that is about to be reduced, or to see when a method call ends. It is possible to extract this information from a full execution trace, but it is easier to modify the semantics such that these kinds of information are directly available, e.g., by adding labels to expressions in the ﬁrst example or introducing a “return” expression in the second example. We will see an example for this situation later on. 2.2

Joinpoint Models

Pointcut languages usually do not refer to the state of evaluation directly, because it contains both too much and too little information: While, say, the exact shape of the heap may be irrelevant for a pointcut language, it may need other information which is not included in the current state, such as information about states in the history of the computation. For this reason, pointcut languages are usually based on joinpoint models, which contain the information that is necessary to evaluate pointcuts. This information may be an abstraction of the evaluation state, static program information, and even information unrelated to the evaluation, or a mixture thereof. To this end, we deﬁne joinpoints to be abstractions over the current state1 . States that are of no interest to the joinpoint model (such as entering a for loop in AspectJ) are mapped to a special value . Definition 1 (Joinpoints and Joinpoint Abstraction). A joinpoint abstraction (JPA) J = (J, αJ ) over Σ is a set J of joinpoints together with a mapping αJ : Σ → J ∪ {}, from states to the corresponding abstraction. Using a joinpoint abstraction, a concrete trace can be turned into an abstract trace. The actual information that is available to a pointcut, however, is a kind of summary of the abstract trace, which we call joinpoint model : 1

In terms of Masuhara et al., we use the point-in-time model for joinpoints in this work, because it is more ﬂexible than the region-in-time model [11].

292

K. Klose and K. Ostermann

Definition 2 (Joinpoint Model). A joinpoint model M = (M, TM ) over a JPA (J, αJ ) is a set M of model values together with a joinpoint transfer function: TM : M × J → M . The joinpoint transfer function deﬁnes how the joinpoint model value for a state σ is calculated using the previous model value and the state abstraction α(σ), so eventually the current model value is a kind of fold over the abstract trace. This means that an initial value has to be provided to calculate the model value corresponding to the ﬁrst state of an evaluation. 2.3

Example: A simple OO Language

In this section we show how the joinpoint abstraction and the joinpoint models of two simple pointcut languages can be expressed in terms of our framework. The joinpoint models of these two pointcut languages we consider are similar to the joinpoint models found in the AOP languages AspectJ and Alpha [14]. We focus on a minimal OO language in the style of Featherweight Java (FJ) [6], a simple calculus that models a minimal core of Java and is deﬁned in terms of a small-step semantics. FJ features classes that may contain ﬁelds and methods, and can inherit declarations from another class. We give the syntax of expressions E in this language by deﬁning its set of redexes (R) and evaluation contexts (E[·]): R ::=v.f | v.m(¯ v ) | return(v) E ::=[·] | new C(¯ v , E, e¯) | E.f | E.m(¯ e) | v.m(¯ v , E, e¯) | E; e | return(E) where v is a value, e is an expression, C is the name of a class, f the name of a ﬁeld, m the name of a method, and e¯ is a list of expressions.2 The basic terms are ﬁeld accesses (v.f ), method calls v.m(¯ v ) and method returns return(v). Values are objects of the form new C(¯ v ), where the arguments to the constructor are in the same order as the ﬁelds of the class. This deﬁnition diﬀers from the original FJ deﬁnition in two regards: First, we use an evaluation context [18] to conveniently identify the redex in an expression. Second, we introduce a new return redex, which is used to identify the end of a method call. This is an example for a modiﬁcation of the operational semantics as discussed at the end of Sec. 2.1. We will identify the set E of all expressions in our language with contexts from E with a redex or a value placed in the hole [·] and write E[r] for the expression that has context E and redex r. Following [6], the states of the semantics have the form Σ = P × E, where P is a representation of the program that contains the class declarations. The reduction rules are identical to that of FJ, except that there is one new reduction rule E[return(v)] −→ E[v], which discards a return wrapper once the wrapped expression has been reduced to a value. 2

We write x ¯ for a list x1 · . . . · xn , [] for the empty list and x · x ¯ for list construction.

A Classiﬁcation Framework for Pointcut Languages

293

An AspectJ-style Joinpoint Model. The joinpoint model in this section is meant to capture the core of the model of AspectJ: the current joinpoint and the callstack abstraction, as required for the cflow pointcuts. The ﬁrst step is to deﬁne a joinpoint abstraction as a basis for the joinpoint model. The states which are of interest for a callstack-based joinpoint model are method calls, i.e., redexes of the form v.m(¯ v ), where v is a value and v¯ is a list of values, and returning from method calls, i.e., redexes of the form return(v), where v is a value. In AspectJ, it is also possible to react to ﬁeld reads, hence we could add redexes of the form v.f , too, but we want to illustrate the case how to deal with redexes that are not of interest to pointcuts, hence we assume that in this pointcut language ﬁeld reads can not be addressed by pointcuts. In AspectJ, the joinpoint model does not include receiver and argument values but only their types3 . To create the joinpoint abstraction we have thus to replace the receiver and argument values in the method call redex by their respective types. Since values in FJ have the form new C(¯ v ) we can give a simple value-type mapping by type(new C(¯ v )) = C. Similar to AspectJ we do not treat the return joinpoint as a “joinpoint event”, i. e., as a entity directly addressable by a construct of the pointcut language. Instead, we only need them to maintain the callstack model. Thus we can safely ignore the return value in our joinpoint model. Based on these considerations, joinpoint abstractions in the callstack based ¯ return}, where C ranges over class model are of the form JAJ = {C.m(C), names. The abstraction mapping αAJ simply removes the unnecessary information from the states in the semantics: ⎧ ¯ ⎪ ⎨type(v).m(C)) αAJ (σ) = return ⎪ ⎩

if σ = (p, E[v.m(¯ v )]), Ci = type(vi ) if σ = (p, E[return(v)]) otherwise

The joinpoint model MAJ consists of a list of joinpoint abstractions represent∗ ing the callstack: MAJ = JAJ . The corresponding transfer function maintains the call stack and the current joinpoint: TAJ (¯j, j) =

¯ j · ¯j if j = C.m(C) k¯ if j = return and ¯j = k · k¯

An Alpha-like Joinpoint Model. In Alpha, not only the types but also the dynamic values of the objects involved in a state are available when formulating pointcuts. Hence the joinpoints in Alpha are more ﬁne-grained4 : JAlpha = {v.m(¯ v ), return(v)} 3 4

The actual values may be needed for the execution of an advice in AspectJ, but this work focuses on pointcuts only. Again we omit ﬁeld reads to illustrate how reductions can be ignored.

294

K. Klose and K. Ostermann

Furthermore, not only the callstack but the complete execution history is available for pointcuts. Accordingly, the abstraction mapping αAJ is the projection on the current redex: ⎧ ⎪ v) if σ = (p, E[v.m(¯ v )]) ⎨v.m(¯ αAlpha (σ) = return(v) if σ = (p, E[return(v)]) . ⎪ ⎩ otherwise The model values in this model are meant to keep the complete (abstracted) trace of the execution history. We will model this again as a list of joinpoints: ∗ , together with a transfer function which accumulates all releMAlpha = JAlpha vant joinpoint abstractions in the list: TAlpha (¯j, j ) = j · ¯j. In contrast to the AspectJ-like model, return joinpoint abstractions are stored and not discarded. The reason is that Alpha pointcuts frequently reconstruct former callstacks from the history, hence this information is needed to evaluate such pointcuts. 2.4

Classification of Joinpoint Models

In the previous section, we saw two diﬀerent joinpoint models for a small base language. In order to illustrate the design space of the combination of joinpoint abstraction and model deﬁnition, we will now look at some of typical cases of joinpoint models. The space dimension. The ﬁrst axis of the design space of joinpoint models is concerned with how much information about the past can be stored in a model value. The simplest such joinpoint model is the constant model, where Tconst (m, j) = m. In this model, the initial model value is present at every point in the program. This kind of model can be used to represent conﬁguration values in a system or to pass the AST of the program or parts of it around. A constant model alone is not very useful, since the model value never changes, but it is useful if combined with other models, see later discussion about model constructions. In contrast, global models are not restricted in the way they collect information about the execution. Given a ﬁxed JPA, the model values of a global model are lists representing a view on the past execution and the transfer function is of the form Tglobal (m, j) = f (j) · m for some function f . The idea behind this deﬁnition is that the joinpoint model accumulates information (possibly transformed by f ) from every state in the execution in a list. The most general global model is the full trace model, which has f = idJ . The Alpha joinpoint model falls in this category. A slightly more restricted model is a bounded model, where the size of the model (according to some size metric) is restricted by a function on the current state. The AspectJ joinpoint model falls in this category: The length of the stored list is bounded by the depth of the current callstack. Finally, a local model is a joinpoint model that is restricted in the amount of information that it can keep in one model value. The canonical example is a

A Classiﬁcation Framework for Pointcut Languages

295

model with Tlocal (m, j) = j, where the model only has access to the most recent state. The joinpoint value can of course be transformed by any function instead of the identity in Tlocal . The AspectJ pointcut language without the cflow-style pointcuts could be evaluated in terms of a local joinpoint model. The same holds for the breakpoint conditions typically found in debuggers. Static and dynamic. The other axis of the design space of joinpoint models is the kind of information that can be accessed. Here we distinguish joinpoint models whose only information about the current state is the lexical position of the current redex from joinpoint models that also refer to other dynamic information. Henceforth, a static model is one whose corresponding joinpoint abstraction function α has the form α(σ) = loc(σ), where loc is a function that retrieves the lexical position of the current redex. Such pointcut languages are particularly easy to implement, since every pointcut can be directly mapped to a set of locations in the code. An example is the pointcut language by Eichberg et al. [4]. In contrast, dynamic models are not restricted in the way they information about the state. Product Models. In most pointcut languages the joinpoints models are combinations of the classes of models described above. For example, even if a very generic trace based model may be available, there may be static source code information that is not part of the trace, like the subtype relation. This means that models will usually be cartesian products of diﬀerent kinds of models whose components can be classiﬁed using the categories above. The product model of two joinpoint models M = (M, TM ) and M = (M , TM ) is MP = M × M , where the product of the transfer functions is deﬁned componentwise as TMP ((m, m ), j) = (TM (m, j), TM (m , j)). Of course, the product construction can easily be generalized by replacing the pair constructor with an arbitrary function. Such a construction can be used to model dependencies between the joinpoint models in such a way that one model value inﬂuences the other one. The categorization of joinpoint models (or their components) is not only useful when comparing diﬀerent joinpoint models and pointcut languages but also in design and implementation of aspect oriented systems: the kind of mapping indicates which information must be stored or calculated in the runtime system when augmenting or implementing a system for a pointcut language.

3

Pointcuts

So far we have deﬁned what we mean by joinpoint models and how these models are connected to the underlying operational semantics by abstraction of states into joinpoints. Based on these deﬁnition we will now describe what pointcuts are in our framework and how pointcut languages can be modeled. 3.1

Pointcuts and Matching

We have already stated that pointcuts are a means to identify points in the execution of a program that share a certain property. In our framework, the

296

K. Klose and K. Ostermann

states of the program are not directly available but through the abstraction of joinpoint models. Thus pointcuts are modeled by stating on which joinpoint model values they match. Definition 3 (Pointcut). Let M = (M, TM ) be a joinpoint model. A pointcut P is a set of model values, i. e., P ⊆ M. 5 Pointcut P is said to match a model value m, if m ∈ P . The deﬁnition of pointcut matching can be extended to execution traces, i. e., lists of the form t = σ1 , . . . , σn of states with σi −→ σi+1 . To this end we have to deﬁne how model values are transformed over traces of states. We begin by ∗ : M × (J ∪ {})∗ → M × (J ∪ {}) lifting the transfer function to a function TM that maps the transfer function over abstract traces, that is, over lists of joinpoint ∗ (m, []) = (m, ) and abstractions by TM ⎧ ⎪ ⎨(TM (m, j), j) ∗ ∗ ¯ TM (m, j · j) = TM (m, ¯j) ⎪ ⎩ ∗ TM (TM (m, j), ¯j)

if j = and length(¯j) = 0 if j = if j = and length(¯j) > 0

This is a gloriﬁed fold of the transfer function over the abstract trace, which is a little more complex than a normal fold due to the fact that abstract traces may contain values, which are no valid arguments for the model transfer function TM . However, we cannot simply ignore them; if a trace ﬁnishes on , then the model value assigned to this trace is the result of an earlier state in the execution, and a pointcut should match only at that earlier state and not on each subsequent state, that is mapped to . Thus we keep track of the last joinpoint that has been used to calculate the model value and use to indicate, that no pointcut should match this trace. ∗ operates on lists of joinpoint abstractions, we need to lift traces, which As TM consist of states, to abstract traces, which consist of the corresponding joinpoint σ ) = map(αJ , σ ¯ ). Based on this we abstractions. We deﬁne this operation α∗J (¯ deﬁne what it means that a pointcut matches a trace. Definition 4 (Pointcut Matching on Traces). Let M be a joinpoint model with joinpoint abstraction J. A pointcut P matches a trace t with initial model ∗ value m0 , if TM (m0 , α∗J (t)) = (m, j) with j ∈ J and m ∈ P . 3.2

Pointcut Languages

The means by which a pointcut is described is deﬁned by a pointcut language. A pointcut languages consists of syntax – a set of valid expressions – and a semantics, mapping syntactical constructs to their meaning in terms of pointcuts. 5

Pointcuts can be identiﬁed with predicate functions by looking at the characteristic function χP .

A Classiﬁcation Framework for Pointcut Languages

297

Definition 5 (Pointcut Language). Let M = (M, TM ) be a joinpoint model. A pointcut language L over M is a set L of language expressions called pointcut designators together with a semantics function [[·]]L : L → P(M ), which maps pointcut designators to their extension, i.e., the set of model values on which the corresponding pointcut matches. We define the extension of a pointcut language as [[L]] = {[[π]] | π ∈ L}. In any non-trivial pointcut language, there will be multiple pointcut designator with the same extension – in most cases even inﬁnitely many. For example, in AspectJ one can ﬁnd arbitrary many pointcut designators matching all possible method calls: call(*.*(..)), call(*.*(..)) && call(*.*(..)), and so forth. It is further noteworthy that some of the extension sets will be inﬁnitely large, because we do not ﬁx a program in the deﬁnition of pointcut languages. For example, if an identiﬁer – like a method name – is part of the model values and a pointcut designator does not impose any restrictions on this identiﬁer, the extension of this pointcut designator will comprise model values for each identiﬁer expressible in the language at hand. It is, however, an interesting property of a pointcut language, which extensions are ﬁnite for any or some ﬁxed program. For example, in the case of identiﬁers, extensions will be ﬁnite (at least with respect to the identiﬁer) because the set of identiﬁers is ﬁnite for any given program. On the other hand, in pointcut languages over joinpoint models containing the whole execution history, many pointcut designators will have inﬁnite extensions, because there are inﬁnitely many possible traces satisfying the constraints given in the pointcut designator. 3.3

Example Pointcut Languages

We have already given the deﬁnition of the pointcut model of Alpha and AspectJ like languages deﬁned over a small OO base language. We will discuss how to compare pointcut languages next and for that purpose we want to discuss brieﬂy the pointcut languages of these two approaches. Due to place constraints we will not give a complete deﬁnition of the syntax and semantics in terms of our framework. AspectJ pointcuts are basically built from a set of primitive pointcut expressions using boolean the operators && (and), || (or) and the higher-order pointcut cflow, which is true if its argument denotes a joinpoint on the callstack. For our purpose, only the primitive pointcut call(R C.m(a)) is important. In this pointcut R is the return type, C the class name, m the method name and a argument. For each of these parameters, wildcards can be used. Alpha pointcuts are basically Prolog queries over a representation of the execution trace. This representation contains facts for each joinpoint that occurred in the execution, paired with a time stamp. These time stamps can be used to relate the joinpoints and to deﬁne abstractions like cflow. Since the programmer can also provide his/her own predicates, Alpha pointcuts can compute arbitrary functions over the trace representation.

298

K. Klose and K. Ostermann

Both of these approaches make the quantiﬁcation over the joinpoint model implicit. In order to come up with a semantics in the form of Def. 5, we would have to make this relation explicit by presenting the semantics as predicate functions over the joinpoint model and then use these predicate functions as characteristic functions for the pointcut extension.

4

Comparing Pointcut Languages

In this section we will develop a methodology to systematically compare pointcut languages. Since we introduced three dependent layers – joinpoint abstractions, joinpoint models and pointcut languages – we have to deﬁne comparison on the lower layers ﬁrst, to make constructs on the higher layers comparable. Thus we start by describing, what it means that one of two joinpoint abstractions is more abstract. Then we talk about the relation between joinpoint models over the same joinpoint abstraction. Finally we show how pointcut languages can be compared and how constructive proofs can be constructed for one language being more/less expressive than an other. 4.1

Comparison of Joinpoint Abstractions

We begin by describing how joinpoint abstractions can be compared. We assume the same base language semantics for both abstractions, because we want to talk about the way that the abstractions act on the same states. We deﬁne a joinpoint abstraction to be more abstract than another (more detailed) joinpoint abstraction, if it identiﬁes joinpoints of the more detailed model. Definition 6 (Abstractness of Join Point Abstraction). Let J1 = (J1 , αJ1 ) and J2 = (J2 , αJ2 ) be joinpoint abstractions over Σ. Then J2 is more abstract than J1 if there exists a surjective mapping δ : J1 → J2 , such that δ ◦ αJ1 = αJ2 . Consider the joinpoint abstractions JAJ and JAlpha from Sec. 2.3. According to Def. 6, the joinpoint abstraction JAJ is obviously more abstract than JAlpha , because its elements contain only type names, while the elements of JAlpha contain values. We show this formally by constructing δ : JAlpha → JAJ as follows: ⎧ ¯ ⎪ v ) and Ci = type(vi ) ⎨type(v).m(type(C)) if j = v.m(¯ δ(j) = return if j = return (v) ⎪ ⎩ otherwise and showing that δ satisﬁes the condition given in Def. 6: v )]) = δ(v.m(¯ v )) = type(v).m(type(¯ v )) δ ◦ αAlpha (E[v.m(¯ = αAJ (E[v.m(¯ v )]) δ ◦ αAlpha (E[return(v)]) = δ(return(v)) = return = αAJ (E[return(v)]).

A Classiﬁcation Framework for Pointcut Languages

299

All other expressions are mapped to by both joinpoint abstraction mappings, so these are the only two cases to consider. Thus we have shown that δ ◦ αAlpha = αAJ , and therefore that the call stack based model is more abstract than the trace model. 4.2

Comparison of Joinpoint Models

When comparing joinpoint models, one important problem is that the underlying joinpoint abstractions may be diﬀerent. In order to ﬁnd a common joinpoint abstraction to compare both models, we can use the joinpoint abstraction translation δ to compare both models on the more abstract joinpoints. Given this common joinpoint abstraction, the key idea is that values of the more expressive (richer) joinpoint model contain enough data to reconstruct values of the less expressive model. This means that there exists a mapping between the model values which is “compatible” with the model transfer function. Definition 7 (Sub-Model). A joinpoint model M1 = (M1 , T1 ) is a sub-model of M2 = (M2 , T2 ), if both have the same joinpoint abstraction J and there exists a surjective function β : M2 → M1 , such that ∀j ∈ J, m ∈ M2 . β(T2 (j, m)) = T1 (j, β(m)). The mapping β identiﬁes all elements of M2 that can not be distinguished in M1 by mapping them to the same element. The condition on β requires the mapping to be consistent with the model transfer function. If β is a bijection then both models are equal (up to renaming), otherwise M1 is a proper sub-model of M2 . For example, consider again the call stack and trace based models from Sec. 2.3. We deﬁne β : MAlpha → MAJ as follows: δ(j) · β(¯j) if j = v.m(¯ v) ¯ β([]) = [], β(j · j) = ¯ k if j = return(v) and β(¯j) = k · k¯ It is easy to prove that the result of β is indeed the corresponding call stack. That β is correct with respect to the deﬁnition can be seen from the fact, that δ is correct and by simulating the semantics for one step. 4.3

Criteria for the Comparison of Pointcut Languages

In the following we compare pointcut languages, and similar to the joinpoint model comparisons, we assume that both languages are deﬁned over the same joinpoint model, since we can translate the richer joinpoint model into the less expressive and compare the translated languages over the less expressive model. Since pointcut languages consist of both a syntax and a semantics we can compare pointcut languages with respect to both. First, we ignore syntax and concentrate on the extensions of the languages, that is, the sets of model values that are extensions of pointcuts in the languages. Later we describe how languages

300

K. Klose and K. Ostermann

can be compared with respect to syntax and ﬁnally we give a third formulation based on the the ability of a language to distinguish program executions. Using the deﬁnition of the extension of a pointcut language (Def. 5), we can formalize our ﬁrst comparison criterion. The idea is that a language L1 is a sub-language of L2 , if the extension of L2 contains every pointcut that is in the extension of L1 : Definition 8 (Expressiveness of Pointcut Languages). Let L1 and L2 be two pointcut languages over the same model M . Then, L1 is less expressive, if [[L1 ]] ⊆ [[L2 ]] and L1 is proper less expressive than L2 , if [[L1 ]] ⊂ [[L2 ]]. This deﬁnition means that if there is a pointcut designator π1 ∈ L1 (i. e. [[π1 ]] ∈ [[L1 ]]) then there exists a pointcut designator π2 ∈ L2 such that [[π1 ]] = [[π2 ]]. Consider, for example, the Alpha pointcut calls(R, ’m’, V), typeof(R, ’C’), odd(V) 6 , which matches all traces ending on a call of a method m on instances of class C, where the argument is an odd integer. To compare the extension of this pointcut to AspectJ pointcuts, we need to embed the AspectJ pointcut language into the Alpha pointcut model. To this end we identify a pointcut P ⊆ MAJ with the Alpha pointcut β −1 (P ) = {m ∈ MAlpha | β(m) ∈ P }. Since we cannot reason about values in AspectJ, β −1 (P ) contains traces ending in calls with every possible argument value. Hence, there is no AspectJ pointcut that corresponds to our Alpha pointcut and thus AspectJ is not equally expressive as Alpha. However, since it is possible to reason about types of values in Alpha, every AspectJ pointcut can be expressed in Alpha. This means that AspectJ is a proper less expressive language than Alpha. The expressiveness of a pointcut language can also be expressed on the level of syntax. On this level, a less expressive or sub-language is characterized by a mapping between the pointcut designators of the two languages. This mapping maps each pointcut designator of the less expressive language to one of the more expressive language: Definition 9 (Sub-Language). Let L1 and L2 be two pointcut languages over the same model M . L1 is a sub-language of L2 , if there exists a mapping of pointcut designators ι : L1 → L2 with: ∀π ∈ L1 . [[π]] = [[ι(π)]]. Of course, the extension of the pointcuts must be invariant under the syntactic mapping, which is what the condition imposed on ι ensures. In a sense, this comparison criterion is more useful, because it requires to construct a concrete syntactical mapping. In practice, this mapping can be used for the emulation of the sub-language in an implementation of the super-language. We have already mentioned that there is an Alpha pointcut for every possible AspectJ pointcut, which means that we can construct a mapping ι : LAspectJ → LAlpha by translating the primitive and cflow AspectJ pointcuts into the corresponding Prolog terms and translate the boolean operators into logical connectives in Prolog. 6

We assume that some representation of integers is available and write them as numbers to keep the examples short.

A Classiﬁcation Framework for Pointcut Languages

301

On the other hand it is often desirable to have a constructive proof that a language is not a sub-language of another language. With the above criteria, this can only be established by proving the non-existence of such a mapping. The deﬁnition of pointcut matching over traces (Def. 4) allows for another characterization of pointcut languages. A pointcut can be identiﬁed with the set of traces on which it matches for a given initial model value. Thus we can classify pointcut languages with the traces that are separable by any pointcut designator belonging to the language. First note that this set of traces is restricted by the joinpoint abstraction: each abstract trace can be the abstraction of several traces. The set of traces that correspond to an abstract trace tα is: (α∗J )−1 (tα ) = {t | α∗J (t) = tα }. Thus pointcuts can only be compared by the abstract traces that they are able to distinguish. We will call two traces t1 and t2 separable by a pointcut π, if there is a model value ∗ ∗ m on which TM (α∗J (t1 ), m) matches π iﬀ TM (α∗J (t2 ), m) does not match π. This notion of separability can be used to deﬁne an equivalence relation on traces that identiﬁes all traces which can be separated by a given pointcut language: Definition 10 (Separability Equivalence). Let L be a pointcut languages over a model M = (M, TM ). The separability equivalence relation ≡L for the language L is defined as follows t1 ≡L t2 :⇔ ∗ ∗ ∀π ∈ L, m ∈ M. TM (m, α∗J (t1 )) ∈ ([[π]] × J) ⇔ TM (m, α∗J (t2 )) ∈ ([[π]] × J).

Thus we can give an alternative deﬁnition for pointcut language expressiveness based on the pointcut languages’ ability to separate sets of model values: Definition 11 (Precision of Pointcut Language). plusLet L1 and L2 be two pointcut languages over the same model M . L1 is less precise than L2 , if ≡L2 ⊆≡L1 . L1 is proper less precise than L2 , if ≡L2 is a proper subset of ≡L1 . From the previous examples we see that Alpha is much more precise than AspectJ, because it is possible to refer to runtime values in the pointcut. This is, however, not due to the lack of values in our modeling of AspectJ: Even if we had values instead of types, AspectJ cannot distinguish traces that diﬀer in arbitrary computable properties, like the sum of all arguments, the structure of the heap (reachability) and so forth. 4.4

A Methodology for Language Comparison

Using the criteria above we propose the following methodology for comparing pointcut languages: First, we have to ensure, that the two pointcut languages we want to compare are deﬁned over the same joinpoint model and thus over the same joinpoint abstraction. The latter can be achieved by using δ to translate detailed joinpoints into abstracted ones. Then we construct the β mapping from the more expressive model to the less expressive and use it perform the comparison either on the less expressive model or on the more expressive model using β −1 as in the examples.

302

K. Klose and K. Ostermann

To show that one language is a sub-language of/less-expressive than another language, we construct a mapping from pointcut designators of the sub-language to pointcut designators of the richer language. To prove that a language L1 is not a sub-language of another language L2 , two traces have to be provided that the language L1 can separate, but L2 can not. In our example comparison we have seen that there are two diﬀerent reasons, why AspectJ is less expressive than Alpha: The lack of values in the joinpoint model and the simpler computational expressiveness. Care must be taken not to confuse these diﬀerent sources of expressiveness by comparing the languages with respect to both the more expressive and the less expressive model.

5

Discussion

Finally we will give an outlook on how our formalization of pointcuts and models is connected to typical issues found in the implementation of programming languages and runtime monitoring systems. 5.1

Shadows and Optimization

We have identiﬁed pointcuts with sets of joinpoint model values. By the joinpoint abstraction, each of these model values can be identiﬁed with one or more traces in the program execution. To be precise, we can identify each model value m ∈ M with the set Tm of pairs of traces and initial values that produce m given by: ∗ {(t, m0 ) | (∃j ∈ J) TM (m0 , α∗J (t)) = (m, j)}. For a pointcut π, the set Tm = Sπ = m∈[[π]] {t | (∃m0 ) (t, m0 ) ∈ Tm } is the set of traces that may lead to a model state that the pointcut matches. This set is of particular interest, if there is a function loc mapping states to elements of the program representation. For example, when using labeled expressions, the labels can be used to identify the source code (or, program) element that is currently evaluated. These source code locations are typically called shadows of the pointcut and the set of residues Resπ (l) = {(σ0 , . . . , σn ) ∈ Sπ | loc(σn ) = l}, can be thought of as deﬁning the dynamic test to be performed at the labeled element to decide if the pointcut matches. Shadows and the corresponding dynamic checks are important concepts in the implementation and optimization of pointcut languages. However, without further information about the structure of states (and, therefore, of traces), the elements of Resπ (l) are not a very good descriptions of the dynamic checks, because they simply enumerate all matching traces. But if it is possible to exploit the semantics of the base language and knowledge about the joinpoint model to compute a set Tl of approximated traces for the location l, we can eliminate all traces from Resπ (l) which are no approximations, possibly yielding a much smaller set of traces. In the extreme case the remaining set could be empty. In this case we know that the pointcut can never match an execution trace reaching

A Classiﬁcation Framework for Pointcut Languages

303

this location and we do not have to check for matching at runtime. But even in the case of a non-empty result, an optimization is possible if we are able to ﬁnd a “more eﬃcient” pointcut expression π with Resπ (l) ∩ Tl = Resπ (l) ∩ Tl . These optimizations are too dependent on the exact details of the base language, joinpoint model and pointcut language to be described in this work, but a specialization of our framework for a particular language is likely to yield concrete and useful optimization strategies. 5.2

Observational Equivalence

Two expressions are observationally equal, if they cannot be distinguished by any context in which they are inserted. Observational equivalence is obviously important both in terms of program understanding and optimization. If we consider a runtime monitoring system over a ﬁxed language, the observational equivalence relation will in general become smaller, because a pointcut observing the program execution can suddenly distinguish expressions that would otherwise be observationally equivalent. For example, a call to a method that returns a constant cannot be replaced with this constant, because a trace model could distinguish these terms. Let us assume that the expressions M and N are observationally equivalent in the unobserved program. We can now look at the modiﬁed observational equality relation on diﬀerent levels. If we allow arbitrary models, we have to consider all traces that the evaluation of the terms may produce. If the traces in all contexts are equal for both expressions, then the expressions are also observationally equal with respect to the runtime monitoring system. If we ﬁx the model, we have more observationally equal expressions, but we still have to check all possible traces in all contexts. Finally, with a ﬁxed pointcut language, we have to check if the terms have separable traces in at least one context, otherwise they are observationally equivalent. This means that we can ﬁnd better approximations for observational equivalence, if we have a compact description of separability for the given pointcut language. A typical application of identifying observationally equivalent expressions is in program optimization. Like with shadows, in this generality we cannot give concrete optimizations. However, for a ﬁxed base and pointcut language, the separability relation may have a simple approximation that allows to derive simple static analyses to approximate observational equivalence. 5.3

Advice and Context Binding

Using pointcuts to trigger additional functionality is only one possible application of pointcuts, but it is useful in many contexts, like debugging and tracing, enforcements of trace and security properties, and aspect-oriented modularization. In this section we show how a “aspect-aware” semantics can be deﬁned on top of a base semantics and a pointcut language over this semantics. To make this elaboration a bit less abstract, we use the convention from our former examples and and deﬁne the states of the base semantics as pairs (p, E[e]) of the program representation and a context E with a redex e.

304

K. Klose and K. Ostermann

We give a semantics which augments the original semantics by adding the required context, a syntactical element for proceeding with the evaluation of the original state of execution, and derivation rules for pointcut matching and advice evaluation to the semantics of the underlying language. These extensions describe when and how pointcuts are evaluated and how they trigger the evaluation of advice functionality. The additional functionality is speciﬁed in aspects, which bind a piece of code to the pointcut which describes when to execute that code. There are diﬀerent ways to introduce the additional functionality: before, after or instead (often called around ) the original expression. In our model we use the around strategy, because both other strategies can be expressed in terms of around. It is not suﬃcient, however, to simply use a pair of a pointcut and an expression to model aspects, because aspects may bind information from the model value and they may choose to proceed with a modiﬁed expression instead of the original one. Taking this possibilities into account we model aspects as the set A = L × (M → E) × (M → E → E) .

bind

target

The bind function describes how the context is used to deﬁne the additional ¯ → print(m); proceed functionality. For example, the bind function C.m(C) prints the method name of the intercepted method before proceeding with the call. The target is used to modify how to proceed with the evaluation. It takes the model value at the point proceed is about to be evaluated and the original expression and produces the expression that has to be evaluated instead of proceed. To model this augmented semantics, we ﬁx a pointcut language P with execution model M and joinpoint abstraction J. The syntax of the base language (E) is extended by the expression proceed, which can be used in advice to resume the evaluation of the expression that lead to the advice activation. Since the deﬁnition of the base language’s syntax will in most cases be a recursive constructor, the deﬁnition of the extended syntax depends on the base syntax and can not be deﬁned in a general way. We assume that such a syntax E can be constructed from E and that this set contains all the base expressions and additionally – at least – the expression proceed. We deﬁne the lifted semantics as follows: the new states are of the form Σ = M × [A] × [e] × Σ. ¯ An element (m, a ¯, e¯, σ) ∈ Σ consists of the current model value m, a list a of aspects, a list e¯ of expressions whose evaluation has been “shadowed” by invocation of an aspect, and the original state σ. The evaluation relation −→ ⊂ Σ × Σ is deﬁned by the rules in Figure 1. The (Advice App) rule explains how matching advice is applied. Rule (Cong) allows to use other aspects than the head of the list. This rules introduces a nondetermism for advice application. Note that we did not model multiple aspects to match at the same state. The third rule describes the semantics of the new expression proceed, and the ﬁnal rule embeds the base semantics.

A Classiﬁcation Framework for Pointcut Languages a = (π, b, t)

m = TM (αJ (σ), m)

m ∈ [[π]]

(m, a · a ¯, e¯, (p, E[e])) −→ (m, a · a ¯, e · e¯, (p, E[b(m )])) (m, a ¯, e¯, σ) −→ σ

e = proceed

(m, a · a ¯, e¯, σ) −→ σ ∀(π, b, t) ∈ a ¯. TM (αJ (σ), m) ∈ / [[π]] (m, a ¯, e · e¯, (p, E[proceed])) −→ (m, a ¯, e¯, (p, E[t(m, e)])) σ −→ σ

σ = (p, E[e])

e = proceed

305

(Advice App)

(Cong)

(Proceed)

∀(π, b, t) ∈ a ¯. TM (αJ (σ), m) ∈ / [[π]]

(m, a ¯, c¯, σ) −→ (TM (σ, m), a ¯, c¯, σ ) (Base) Fig. 1. Evaluation Rules for the Lifted Semantics

6

Related Work

Many formal deﬁnitions of the semantics of aspect oriented languages have been proposed to clarify the semantics concepts like advice application, aspect precedence, and proceed. For example, Masuhara et al. [13] present a model that explains compilation and optimization of an AspectJ-like language. The model is based on an interpreter which is partially evaluated to explain issues like shadow ﬁnding and removal of runtime checks which are unnecessary. The resulting model is not as general as our formalization, because it is tailored to AspectJ. This model (and similar ones for other pointcut languages) can be used to describe the semantic of AspectJ-pointcuts in our formal model, because it deﬁnes the characteristic functions for pointcut extension. This and similar approaches use ﬁxed base and pointcut languages; a general exploration and organization of the pointcut language design space is not in the scope of these works. Another branch of related work is the development of meta-models and ontologies for pointcut-advice (PA) languages [2,3]. These models are used to identify the parts in PA-languages in a very general way that can be used as a basis for implementations of aspect-oriented systems. Although these models contain informal representations of pointcuts, pointcut context (similar to our joinpoint models), as well as a distinction between static and dynamic context, these entities do not have a formally speciﬁed meaning and are hence not amenable to constructive comparisons between pointcut languages. St¨ orzer and Hanenberg [16] give a categorizations of pointcut language constructs with respect to a ﬁxed set of three diﬀerent model classes: speciﬁcation based pointcut languages, which are similar to our constant models, state-based constructs, which are similar to our local models, and so-called progress-based pointcut constructs, which are a subset of our global (trace-based) models. In contrast to our work, there is no notion of joinpoint abstraction or pointcut language expressiveness, and no methodology to compare pointcut languages.

306

K. Klose and K. Ostermann

Masuhara and Kiczales [12] present a comparison framework in which they evaluate four diﬀerent AOP languages. The focus of that work is to characterizing and comparing the weaving process that these languages use. This is in contrast to our framework, which focuses only on the language part and ignores advice and weaving, thus making the framework applicable in “non-AOP” systems.

7

Conclusions

We have presented a formal framework for the classiﬁcation and comparison of pointcut languages. This model gives precise meaning to joinpoints, pointcuts and pointcut languages. The most important usage of this framework is a methodology for the comparison of pointcut languages, which is based on giving constructive proofs for sub-language relations. These proofs can be used to construct embeddings of pointcut languages into more expressive ones, or to characterize the reason why one language is less expressive than another. Furthermore, our framework illustrates the design dimensions and corner cases of pointcut language design and implementation, and hence we hope that it can be used as guidance in both the design of new pointcut languages and the development of eﬃcient compilation techniques for pointcut languages. We consider our framework both as a starting point to improve the modeling, design and implementation of pointcut languages and as way to combine the existing approaches to give semantics to particular pointcut languages.

References 1. Allan, C., Avgustinov, P., Christensen, A.S., Hendren, L., Kuzins, S., Lhot´ ak, O., de Moor, O., Sereni, D., Sittampalam, G., Tibble, J.: Adding trace matching with free variables to AspectJ. In: Johnson, R.E., Gabriel, R.P. (eds.) OOPSLA 2005: Proceedings of the Conference on Object Oriented Programming, Systems, Languages, and Applications, pp. 345–364. ACM Press, New York (2005) 2. Bockisch, C., Mezini, M.: A ﬂexible architecture for pointcut-advice language implementations. In: VMIL 2007: Proceedings of the workshop on Virtual machines and intermediate languages for emerging modularization mechanisms. ACM Press, New York (2007) 3. Bockisch, C., Mezini, M., Gybels, K., Fabry, J.: Initial deﬁnition of the aspect language reference model and prototype implementation adhering to the language implementation toolkit architecture. Technical report, AOSD Europe Deliverable, Technische Universit¨ at Darmstadt (2007) 4. Eichberg, M., Mezini, M., Ostermann, K.: Pointcuts as functional queries. In: Chin, W.-N. (ed.) APLAS 2004. LNCS, vol. 3302, pp. 366–381. Springer, Heidelberg (2004) 5. Hajiyev, E., Verbaere, M., de Moor, O.: codeQuest: scalable source code queries with datalog. In: Thomas, D. (ed.) ECOOP 2006. LNCS, vol. 4067, pp. 2–27. Springer, Heidelberg (2006) 6. Igarashi, A., Pierce, B., Wadler, P.: Featherweight Java: A minimal core calculus for Java and GJ. In: Berman, A.M. (ed.) OOPSLA 1999: Proceedings of the Conference on Object-Oriented Programming, Systems, Languages and Applications, NY, pp. 132–146 (1999)

A Classiﬁcation Framework for Pointcut Languages

307

7. Janzen, D., De Volder, K.: Navigating and querying code without getting lost. In: Aksit, M. (ed.) AOSD 2003: Proceedings of the International Conference on AspectOriented Software Development, pp. 178–187. ACM Press, New York (2003) 8. Kiczales, G., Hilsdale, E., Hugunin, J., Kersten, M., Palm, J., Griswold, W.G.: An Overview of AspectJ. In: Knudsen, J.L. (ed.) ECOOP 2001. LNCS, vol. 2072, pp. 327–353. Springer, Heidelberg (2001) 9. Kiczales, G., Lamping, J., Mendhekar, A., Maeda, C., Lopes, C.V., Loingtier, J.M., Irwin, J.: Aspect-Oriented Programming. In: Aksit, M., Matsuoka, S. (eds.) ECOOP 1997. LNCS, vol. 1241, pp. 220–242. Springer, Heidelberg (1997) 10. Martin, M., Livshits, B., Lam, M.S.: Finding application errors and security ﬂaws using PQL: a program query language. In: Ralph, R.P.G., Johnson, E. (eds.) OOPSLA 2005: Proceedings of the 20th annual ACM SIGPLAN conference on Object oriented programming, systems, languages, and applications, vol. 40, pp. 365–383. ACM, New York (2005) 11. Masuhara, H., Endoh, Y., Yonezawa, A.: A Fine-Grained Join Point Model for More Reusable Aspects. In: Kobayashi, N. (ed.) APLAS 2006. LNCS, vol. 4279, pp. 131–147. Springer, Heidelberg (2006) 12. Masuhara, H., Kiczales, G.: Modeling crosscutting in aspect-oriented mechanisms. In: Cardelli, L. (ed.) ECOOP 2003. LNCS, vol. 2743, pp. 219–233. Springer, Heidelberg (2003) 13. Masuhara, H., Kiczales, G., Dutchyn, C.: A compilation and optimization model for aspect-oriented programs. In: Hedin, G. (ed.) CC 2003. LNCS, vol. 2622, pp. 46–60. Springer, Heidelberg (2003) 14. Ostermann, K., Mezini, M., Bockisch, C.: Expressive Pointcuts for Increased Modularity. In: Black, A.P. (ed.) ECOOP 2005. LNCS, vol. 3586, pp. 214–240. Springer, Heidelberg (2005) 15. Pothier, G., Tanter, E., Piquer, J.: Scalable omniscient debugging. In: Bacon, D.F. (ed.) OOPLSA 2007: Proceedings of the International Conference on ObjectOriented Programming, Systems, Languages and Applications, pp. 535–552. ACM Press, New York (2007) 16. Stoerzer, M., Hanenberg, S.: A classiﬁcation of pointcut language constructs. In: SPLAT 2005: Proceedings of the workshop on Software-Engineering Properties of Languages and Aspect Technologies (2005) 17. Walker, R.J., Viggers, K.: Implementing protocols via declarative event patterns. In: Taylor, R.N., Dwyer, M.B. (eds.) FSE 2004: Proceedings of the International Symposium on Foundations of Software Engineering, pp. 159–169. ACM, New York (2004) 18. Wright, A.K., Felleisen, M.: A syntactic approach to type soundness. Information and Computation 115, 38–94 (1994)

Fast Simulation Techniques for Design Space Exploration Daniel Knorreck, Ludovic Apvrille, and Renaud Pacalet System-on-Chip Laboratory (LabSoC), Institut Telecom, Telecom ParisTech, LTCI CNRS, 2229, Route des Crˆetes, B.P. 193, F-06904 Sophia Antipolis, France {daniel.knorreck,ludovic.apvrille,renaud.pacalet}@telecom-paristech.fr

Abstract. This paper addresses an open-source UML based toolkit named TTool - for performing eﬃcient system-level design space exploration of Systems-On-Chip. Main modeling, veriﬁcation and simulation capabilities of TTool are ﬁrst presented, and exempliﬁed by an MPEG2 application. Then, an innovative simulation strategy to signiﬁcantly reduce simulation time is introduced. The basic idea is to take beneﬁt from high level descriptions of applications by processing transactions spanning potentially hundreds of clock cycles as a whole. When a need for inter task synchronization arises, transactions may be split into smaller chunks. The simulation engine is therefore predictive and supports backward execution thanks to transaction truncation. Thus, simulation granularity adapts automatically to application requirements. Emphasis is more particularly put on procedures taking place under the hood after having pushed the TTool simulation button. Finally, the new simulation strategy is assessed and compared to an earlier cycle-based version of the simulation engine. Keywords: System-On-Chip, Design Space Exploration, System Level Modeling, UML, DIPLODOCUS, TTool, Fast Simulation Techniques.

1

Introduction

System-level design space exploration in a System-on-Chip (SoC) design cycle is an issue of great concern in today’s rapidly growing and heavily constrained design processes. The increasing complexity of SoC requires a complete re-examination of design and validation methods prior to ﬁnal implementation. We believe that the solution to this problem lies in developing an abstract model of the system intended for design, on which fast simulations and formal static analysis could be performed in order to test the satisﬁability of both functional and non functional requirements. In this context, we have previously introduced a UML-based environment named DIPLODOCUS. The strength of our approach relies on formal veriﬁcation capabilities and fast simulation techniques ( [7] [18] ). DIPLODOCUS design approach is based on the following fundamental principles: M. Oriol and B. Meyer (Eds.): TOOLS EUROPE 2009, LNBIP 33, pp. 308–327, 2009. c Springer-Verlag Berlin Heidelberg 2009

Fast Simulation Techniques for Design Space Exploration

– – – –

309

Use of a high level language (UML) Clear separation between application and architectural matters Data abstraction. Use of fast simulation and formal static analysis techniques, both at application and mapping levels

The designer is supposed to model in an orthogonal fashion the application and the architecture of the targeted system. Thereafter, a mapping stage associates application and architectural components. The strength of our approach relies in simulation and formal proofs techniques that can be applied to modeled systems at all methodological stages. Also, our environment totally hides knowledge of simulation or formal proofs techniques: knowledge of our UML proﬁle is the only asset for engineers. Today, this methodology is supported by an open-source toolkit named TTool [3]. The ﬁrst sections of the paper are dedicated to the presentation of TTool. Simulation is an integral part of the DIPLODOCUS methodology and enables the designer to get ﬁrst estimates of the performance of the ﬁnal system. The ﬁrst version of our simulation environment relied on the standard SystemC kernel and took advantage of the integrated discrete event simulator. In that simulator, a SystemC process is assigned to each active hardware node, and a global SystemC clock is used as a means of synchronization. Abstract communication and computation transactions deﬁned in the application model are broken down to the corresponding number of wait cycles. Unfortunately, the inherent latency of the SystemC scheduler due to frequent task switches made the simulation suﬀer from low performance. However, simulation speed is an issue of concern as it represents one argument to justify accuracy penalties as a result of abstractions applied to the model. In order to achieve a better performance, a new simulation strategy is introduced in the paper. It leverages these abstractions by processing high-level instructions as a whole whenever possible. Furthermore, its simulation kernel incorporates a slim discrete event scheduler so that SystemC libraries are not necessary any more. The paper is composed of seven sections. Section 2 gives an overview of the DIPLODOCUS methodology and points out the features of the integrated development environment TTool. Section 3 elaborates on the modeling stages and illustrates involved models with an MPEG2 case study. Section 4 introduces the main concepts of the simulation strategy and provides an insight into some important algorithmic aspects. Section 5 benchmarks the new simulation engine and compares its performance to the previous simulator version. Section 6 puts our approach in context with related works. Section 7 ﬁnally concludes the paper and draws perspectives.

2

The DIPLODOCUS Environment in a Nutshell

DIPLODOCUS is a UML proﬁle targeting the design of System-on-Chip at a high level of abstraction. A UML proﬁle customizes UML [12] for a given domain, using UML extension capabilities. Also, a UML proﬁle commonly provides a methodology and is supported by a speciﬁc toolkit.

310

2.1

D. Knorreck, L. Apvrille, and R. Pacalet

Methodology

DIPLODOCUS includes the following 3-step methodology (see Figure 1): 1. Applications are ﬁrst described as a network of abstract communicating tasks using a UML class diagram. The latter represents the static view of the application. Each task behavior is described with a UML activity diagram based on the given functional requirements. 2. Targeted architectures are modeled independently from applications as a set of interconnected abstract hardware nodes. UML components have been deﬁned to model those nodes (e.g. CPUs, buses, memories, hardware accelerators, bridges). 3. A mapping process deﬁnes how application tasks can be bound to execution entities and also how abstract communications between tasks are assigned to communication and storage devices.

Fig. 1. Global view of our Design Space Exploration Approach

One main DIPLODOCUS objective is to help designers to ﬁnd a hardware architecture that satisﬁes functional and non-functional requirements. To achieve this, DIPLODOCUS relies on fast simulation and formal proof techniques. The application models are designed independently of targeted underlying architectures. Due to the high abstraction level of both application and architecture, simulation speed can be increased signiﬁcantly with regards to simulations usually performed at lower abstraction level (e.g. TLM level, RTL level, etc.). Additionally, eﬃcient formal static analysis techniques may be applied before and after mapping. Within a SoC design ﬂow, Design Space Exploration is supposed to be carried out at a very early stage, right after a speciﬁcation document has been issued. In the scope of hardware/software partitioning, several appropriate architectures could be assessed by means of our methodology. Thus, it should be emphasized that DIPLODOCUS designs are settled above more detailed but still abstract models like TLM. Indeed, the main diﬀerence between DIPLODOCUS and TLM models relies in data (and not only signal) abstraction and the modeling of

Fast Simulation Techniques for Design Space Exploration

311

computational complexity. Hence, with DIPLODOCUS, a designer can already get a ﬁrst estimate of the behavior of ﬁnal applications even if some algorithmic details have not yet been stipulated. 2.2

Toolkit

The DIPLODOCUS UML proﬁle - including its methodology - has been implemented in TTool [3]. TTool is an open-source toolkit that supports several UML2 / SysML proﬁles, including TURTLE [6] and DIPLODOCUS [5]. The main idea behind TTool is that all modeling may be formally veriﬁed or simulated. In practice, UML diagrams are ﬁrst automatically translated into an intermediate speciﬁcation expressed in a formal language, which serves as starting point for deriving formal speciﬁcation and simulation code. TTool has the following main modeling capabilities (see Figure 2): – Modeling of applications using either a UML class and activity diagrams or a speciﬁc language called TML. From both formats, it is possible to generate simulation code or formal veriﬁcation speciﬁcations from which formal proofs may be performed. – Modeling of architectures using UML deployment diagrams (or a textual format named TARCHI). – Modeling of mappings using UML deployment diagrams (or a textual format named TMAP). A mapping scheme associates applications and architectures. Based on these three building blocks (mapping, application, architecture), TTool automatically generates simulation code or formal speciﬁcations at the push of a button. From DIPLODOCUS application or mapping models, TTool may perform simulations or formal proofs: – For simulation purpose, TTool generates SystemC or C++ code. Indeed, TTool is currently equipped with two simulators: one based on the SystemC simulation engine, and another and more recent one implemented in pure C++. That generated code is linked with libraries - included in TTool before being executed. Also, The C++-based simulator is the main contribution described later on in the paper. This C++ simulator outperforms its SystemC counterpart in terms of simulation speed, but it is yet not compatible with third-parties SystemC-based components. – For formal proof purpose, TTool generates LOTOS or UPPAAL code. Formal veriﬁcation of application or mapping models is out-of-scope of this paper. Before being translated into either SystemC, UPPAAL or LOTOS, models are ﬁrst translated into intermediate languages. Figure 2 refers to this process.

312

D. Knorreck, L. Apvrille, and R. Pacalet

Fig. 2. Modeling and veriﬁcation capabilities of TTool

3 3.1

Modeling SoCs with DIPLODOCUS Application Modeling

An application model is the description of functions to be performed by the targeted SoC. One has to bear in mind that, at application modeling level, there is no notion of time but only a partial ordering between actions. As mentioned before, functions are modeled as a set of abstract tasks described within UML class diagrams. One UML activity diagram per tasks represents its behavior and the diagram is built upon the following operators: control ﬂow and variable manipulation operators (loops, tests, assignments, etc.), communication operators (reading/writing abstract data samples in channels, sending/receiving events and requests), and of computational cost operators. This section brieﬂy describes a subset of the aforementioned operators as well as their semantics and provides deﬁnitions for Channels, Events and Requests: – Channels are characterized by a point-to-point unidirectional communication between two tasks. The following Channel types exist: • Blocking Read-Blocking Write (BR-BW) • Blocking Read-Non Blocking Write (BR-NBW) • Non Blocking Read-Non Blocking Write (NBR-NBW) – Events are characterized by a point-to-point unidirectional asynchronous communication between two tasks. Events are stored in an intermediate FIFO between the sender and the receiver. This FIFO may be ﬁnite or inﬁnite. In case of an inﬁnite FIFO, incoming events are never lost. When adding an event to a ﬁnite FIFO, the incoming event may be discarded or the oldest event may be dropped if the FIFO is full. Thus, a single element

Fast Simulation Techniques for Design Space Exploration

313

FIFO may be used to model hardware interrupts. In tasks, events can be sent (NOTIFY), received (WAIT) and tested for their presence (NOTIFIED). – Requests are characterized by a multi-point to one point unidirectional asynchronous communication between two tasks. A unique inﬁnite FIFO between senders and the receiver is used to store all incoming requests. Consequently, a request cannot be lost. As some of the constituting operators of activity diagrams are referred to in later sections, a brief survey is provided which is not meant to be exhaustive (see Table 1). Table 1. Commands for modeling of task behavior Icon:

Command: Semantics: READ x

Read x samples in a channel

WRITE x

Write x samples in a channel

NOTIFY e

Notify event e with 3 parameters

WAIT e

Wait for event e carrying 3 parameters

REQUEST Ask the associated task to run EXECI x

Computational complexity of x execution units

IF

Branch command, can be deterministic or random Sequence command

Let us consider an example to illustrate our modeling approach: The class diagram depicted in Figure 3 is taken from a case study that we carried out recently. In the scope of that case study, we conceived a model of an MPEG decoder based on the DIPLODOCUS methodology and mapped it on several diﬀerent architectures. The star structure of the diagram reveals that the task called Parser directs four other tasks by means of requests. The four remaining tasks are dedicated to the processing stages of an MPEG decoder: Variable Run Length Decoding (VLC), Inverse Quantization, Inverse Discrete Cosine Transform (IDCT) and Motion Compensation respectively. A chain of channels accounts for data dependencies between processing stages. As the parsing process and the run length decoding process are closely coupled, a synchronization becomes necessary. Thus, the VLC task is able to send an event to the Parser after having completed the decoding procedure.

314

D. Knorreck, L. Apvrille, and R. Pacalet

An excerpt of an activity diagram describing the behavior of the Parser task is given in Figure 4. The ﬁrst branch of the sequence operator is intended to assign values to task variables depending on the picture type which is treated. Within the remaining three branches requests carrying parameters are sent to processing tasks in order to launch them. The whole structure is nested in a loop which is run through for each macro block, the second branch comprises a loop iterating through the coded blocks. These examples should give the reader a rough idea of how more complex models can be realized based on our approach. In addition to simulation, formal veriﬁcations could be conducted in order to verify the absence of deadlock situations and liveness properties.

Fig. 3. MPEG class diagram

Fig. 4. Extract of the activity diagram of the parser task

Fast Simulation Techniques for Design Space Exploration

3.2

315

Architecture Modeling

A DIPLODOCUS architecture is built upon the following parameterized hardware nodes: – Computation nodes. Typically, an abstract CPU model merges both the functionality of the hardware component and its operating system. The behavior of a CPU model can be customized by the following parameters (amongst others): data size, pipeline size, cache miss ratio and scheduling algorithm. – Communication nodes. A communication node is either a bus or a bridge. The bus model exhibits the following parameters: data size, latency and scheduling policy. Note that links which are established during the mapping stage are meant to interconnect a hardware node - except for buses - with a bus. A link may be annotated by a priority if the respective bus has a priority-based scheduling policy. – Storage nodes. Typically, a memory. Memories are parametrized by two measures: latency and data size. A DIPLODOCUS architecture is modeled in terms of a UML deployment diagram where DIPLODOCUS links and DIPLODOCUS nodes are depicted by their corresponding UML counterparts. 3.3

Mapping of Applications on Architectures

A DIPLODOCUS mapping is meant to describe the association of application elements - i.e. tasks, channels, requests and events - and hardware nodes. Thereby the following rules apply: – Abstract tasks must be mapped on exactly one computation node. – Abstract communication entities must be mapped on communication and storage nodes. For the time being, the simulation engine stipulates that a channel is mapped on n buses, n-1 bridges and exactly one storage element. Furthermore, all connected communication links have to form a continuous path without loops. A future version of the simulator should support several memory elements per channel. Depending on the mapping semantics, additional parameters may become necessary. For example, when mapping a task on a CPU node having a priority-based scheduling policy, task priorities have to be deﬁned. The mapping stage is carried out based on previously created DIPLODOCUS architecture diagrams: symbols representing tasks and channels are simply bound to hardware components in a drag and drop fashion. Post-mapping formal speciﬁcations contain less traces than pre-mapping formal speciﬁcations since a mapping is intended to resolve shared resource allocations; i.e. the application model does not stipulate any temporal order of concurrent actions. Finally, traces obtained after mapping are supposed to be a subset of traces obtained before mapping.

316

D. Knorreck, L. Apvrille, and R. Pacalet

To illustrate the notions of architecture and mapping, let us come back to our MPEG decoder case study. Figure 5 illustrates an interesting subset of an architecture on which the decoder application is mapped. Two CPUs are in charge of evaluating the MPEG stream (CPU Controller) and reading back output images (CPU DisplaySim) for display purposes respectively. Three channels shall be considered exemplarily in the following: The Parser task reads samples from channel inputBuﬀer which is mapped on memory Mem Input and Bus Input. The DisplaySim task reads samples from channel outputBuﬀer which is mapped on memory Mem Output and Bus Output. The hardware accelerators executing task VLC and task Motion Compensation which communicate by means of channel decMotVec are omitted for the sake of simplicity. The latter channel is shown anyway to justify the presence of the bridge. Thanks to simulation capabilities of the DIPLODOCUS environment, the designer is capable of quickly comparing the given architecture to another one which for instance does not comprise a bridge (and thus the two buses converge to one). Without modifying the application, the designer will discover that in case of a bus utilization of 0.2 (Bus Output) and 0.07 (Bus Input), the average latency of bus transactions issued by a speciﬁc component is not signiﬁcantly improved when introducing a bridge. For example the average latency per transaction seen by the hardware accelerator executing the VLC task amounts to 1.8 time units without bridge and to 0 time units when considering a bridge.

Fig. 5. Extract of the MPEG architecture diagram

In the remaining part of this paper, emphasis is put on post-mapping simulation of tasks mapped onto a given hardware architecture. More precisely, we introduce our new simulation scheme. It is indeed especially suited to exploit characteristics of our high level task and architecture descriptions. The two next sections of the paper are more particularly focused on the architecture of the simulation engine, and on its performance.

Fast Simulation Techniques for Design Space Exploration

4

317

Fast Simulation Techniques

Before going into details, it should be clearly emphasized what we understand by fast simulation. Our fast simulation approach is centered around two basic principles: – A modeling methodology which allows for conceiving abstract application models by applying both data and functional abstractions – A simulation strategy which exploits eﬃciently these characteristics of the high level model. The granularity of the simulation thereby matches the granularity of the application model. This implies that the gain in terms of simulation speed can hardly be expressed in cycles/per second as it highly depends on the application model. Indeed, in theory the simulator could attain any ratio of cycles/second if transactions were suﬃciently large as we will see in the following sections. But in this case the respective model would be far from reﬂecting key characteristics of the real system due to the lack of expressiveness. The trade-oﬀ between detailedness and simulation speed is still subject to our research and can be varied smoothly thanks to our simulation environment. Nevertheless we tried to obtain some key ﬁgures based on several special simulation scenarios (refer to section 5) on the one hand and based on our case study of an MPEG decoder on the other hand. 4.1

Basic Principles

The simulator detailed in this paper is transaction-based. A transaction refers to a computation internal to a task, or a communication between tasks. Those transactions may obviously last from one to hundreds of clock cycles. Transaction durations are initially deﬁned according to the application model, that is to say their maximum duration is given by the length of corresponding operators within tasks description. At simulation runtime, a transaction may have to be broken down into several chunks of smaller size, just because for example, a bus is not accessible and so the task is put on I/O wait on its CPU. Transaction cutting is more likely to happen when the amount of inter task communication is high and hence the need for synchronization arises. The aggregation of cycles may slightly impact simulation semantics when incorrect branch predictions arise on CPUs. The exact point in time of their occurrence cannot be resolved and thus the resulting delay is merely determined on a per-transaction base. Unlike a conventional simulation strategy where all tasks are running in lockstep, a local simulation clock is assigned to each active hardware component. Thus, simulation granularity automatically adapts to application requirements as abstract measures for computational, and communicational costs of operations are speciﬁed within the application model. The coarse granularity of the high level description is exploited in order to increase simulation speed.

318

D. Knorreck, L. Apvrille, and R. Pacalet

A problem at stake when considering transaction-based simulation is the scheduling of transactions themselves. This procedure embraces mainly four elements (see Figure 6): tasks, CPUs, buses and the main scheduler. The sequence of the entities of the aforementioned list also reﬂects their hierarchy during the scheduling process: tasks are settled at the top layer and the main scheduler constitutes the lowermost layer. Based on the knowledge of their internal behavior, tasks are able to determine the next operation which has to be executed within their scope. This operation is encapsulated subsequently in a transaction data structure and forwarded consecutively to hardware components being in charge of its execution. The basic idea is that a transaction carries timing information taken into account by hardware components to update their internal clock. Moreover, a hardware component may delay transactions and modify their duration according to the execution time needed by that speciﬁc component. By doing so, the simulation algorithm accounts for the speed of CPUs, the data rate of buses, bus contention and other parameters characterizing the hardware conﬁguration.

Task

Task

CPU

Bus

Task

Task

Task

CPU

Task

CPU

Task

Scheduling Trans . length calculation

Scheduling Add bus delay to trans . length nextTransaction proposal

Main scheduler

Selects the transaction which Terminates first

Transaction to schedule

Fig. 6. Object collaboration during scheduling process

More precisely, on time management, tasks tag each transaction with its earliest start time (which is equal to the ﬁnish time of their previously executed transaction) and an abstract measure of the computational complexity or of the amount of data to transfer respectively. CPUs in turn have to recalculate the start time based on their internal schedule and are able to convert the abstract length to time units. Buses are in charge of handling communication transactions which are treated in a similar way. If several devices are involved in the execution process, transactions are simply passed from one device to another device and modiﬁed accordingly. Correct values for start time and duration are hence determined incrementally by forwarding transactions to dedicated hardware nodes. Finally, the main scheduler acts as a discrete event simulator and assures the causality of the simulation.

Fast Simulation Techniques for Design Space Exploration

4.2

319

Example Illustrating the Main Scheduler

To illustrate the cooperation of entities discussed previously and the main scheduler, we consider the following example (Figure 7): there are two CPUs, referred to as CPU1 and CPU2. A CPU model merges both an abstract hardware component and an abstract real time operating system. Both CPUs (CPU1 and CPU2) have already executed a transaction of a task T11 and T21 respectively. The transaction on CPU1 ﬁnished at tSI , the one on CPU2 ﬁnished at tSII . The schedulers of both CPUs subsequently propose the next transaction to execute. The simulation algorithm selects the transaction which will terminate ﬁrst because its execution could have an impact on other tasks (making them runnable for instance). This decision is in line with traditional mechanisms used for discrete event simulation. As we have to be sure that during the execution of the selected transaction no events will occur, special care has to be taken when considering communication-related commands (read to channel, write to channel, wait on event, notify event, ...). The length of these transactions are therefore calculated based on the number of samples to read, the number of samples to write, the content and the size of the channel as well as the size of atomic burst transfers on the bus. After T12 has been scheduled, tSI is set to the end time of T12 , referred to as tSInew . If the completed transaction T12 causes a task to become runnable on CPU2, T22 is truncated at tSInew and the remaining transaction is scheduled on CPU2. As a channel always links only two tasks, revealing dependencies becomes trivial. If a portion of T22 has been added to the schedule, tSII is changed accordingly (tSII = tSInew ). After this, all schedulers of CPUs which have executed a transaction are invoked. The algorithm has now reached its initial state where all schedulers have selected a potential next transaction. Again, the transaction which ﬁnishes ﬁrst is scheduled...

Fig. 7. Scheduling scenario

4.3

Transactions and the Timestamp Policy

This chapter elaborates on the attributes which characterize transactions. One has to discern two time scales within a transaction: a virtual one and an absolute

320

D. Knorreck, L. Apvrille, and R. Pacalet

one. The virtual time scale bases on the number of execution units (for commands modeling computational complexity) and the amount of data to transfer (READ/WRITE commands for abstract channels) respectively. Thus, the virtual time scale does not depend on architecture speciﬁc parameters (such as the speed/data rate of devices). The absolute time is calculated as a function of the virtual time and device parameters. The main attributes of a transaction are tabulated below: – startTime: Stands for the absolute point in time when the transaction starts. – length: Indicates the number of time units needed to execute the transaction. – virtualLength: Speciﬁes the amount of data to read/write or the amount of processing units to carry out (example: for a READ 3 and a channel having a width of 2, the corresponding transaction spanning the whole operation would have a virtualLength of 6). – runnableTime: Indicates the point in time when the transaction gets runnable, that is when the dedicated task is ready to execute it. As stated in the introductory part of this paper, timestamps contained in transactions are exploited by hardware components as well as by the main scheduler. This paragraph elaborates on the contribution of each entity which modiﬁes transactions - including commands and channels which have not been introduced yet. Transactions are initiated by commands as the latter keep track of their internal progress. Our modeling methodology provides abstract channels which have their representation within the simulation environment. Diﬀerent semantics like Blocking Read-Blocking Write, Non Blocking Read-Non Blocking Write and Blocking Read-Non Blocking Write have to be respected. Each object depicted in Figure 8 is thus able to (re)calculate one or more transaction attribute(s) based on its internal state. Arrows denote the data ﬂow of transaction pointers. 1. TMLCommand initially instantiates a transaction. The amount of remaining processing units (the diﬀerence between the virtualLength of the command and its progress) is assigned to virtualLength. Furthermore, a ﬁrst estimate of the runnableTime is stored in the dedicated variable. The value simply amounts to the time when the last transaction of the command terminated. 2. Due to its inner state (i.e. ﬁlling level, references to transaction attempting to read/write,...), a channel is able to deduce the deﬁnite value of runnableTime. The time determined by the command is delayed if the channel is blocked due to an overﬂow or an underﬂow of data. TMLChannel recalculates virtualLength depending on the value proposed by TMLCommand and the content and the size of the channel (a detailed description of this procedure given later in this paper). In other words: a channel may truncate a transaction if its inner state does not allow for executing the transaction as a whole.

Fast Simulation Techniques for Design Space Exploration

321

3. The CPU class ﬁnally computes the start time of the transaction and its real duration. To determine the startTime, runnableTime has to be deferred to the end of the schedule (EndLastScheduledTransaction). The absolute length of a transaction depends on the product of the processor frequency and virtualLength as well as on possible task penalties (such as incorrect branch predictions, wake-up delays of CPUs, task switching times). If the objective of the transaction is a bus transfer, length is calculated within the bus class. 4. This last step is only carried out for transactions necessitating bus access. Possibly startTime is delayed once again due to bus contention and the real length of a transaction is calculated based on the bus frequency.

Fig. 8. Data ﬂow of a transaction

4.4

Simulation Phases

The structure of the simulation algorithm is such that each command has to be prepared before it is able to execute. Thus, simulation enters alternately prepare and execution phases of commands. When the simulator is launched initially, the ﬁrst command of each task is prepared. During simulation, only commands which were able to execute their current transaction have to prepare a new one. At the end of the prepare phase, transactions which might be processed by CPUs are known, only bus scheduling decisions are pending. That way, the prepare phase is crucial for the causality of the simulation. Following tasks are accomplished during the prepare phase: 1. It must be checked if the current command has been processed entirely. In this case, the progress of the command is equal to its virtual length and the following command is activated. 2. The number of remaining virtual execution units is calculated. 3. A new transaction object is instantiated. 4. If the concerned command tends to read/write samples from/to channels, it must be checked if the current state of the channel permits this operation. Therefore, read and write transactions are registered at the channel.

322

D. Knorreck, L. Apvrille, and R. Pacalet

5. Task variables may be evaluated or modiﬁed. The objective of the execution phase is to update state variables of the simulation after a transaction has been carried out. To achieve this, the following actions have to be taken: 1. Issue read/write operations on channels deﬁnitely so that state variables of channels are updated. 2. Update the progress of a command achieved by a completed transaction. 3. Add transactions to the schedule of CPUs and buses. Now, the basic phases have been introduced and attention should be drawn to the way they are arranged to conduct the simulation procedure: 1. At ﬁrst, the prepare phase of the the ﬁrst command of all tasks is entered. 2. Schedulers of all CPUs are invoked to determine the next transaction. Each CPU may have its own scheduling algorithm. 3. CPU schedulers register their current transaction at the appropriate bus in case the transaction needs bus access. 4. The main scheduler is in charge of identifying the runnable transaction t1 having the least end time. Therefore, the main scheduler queries all CPUs which may check in turn if bus access was granted for their communication transaction. Bus scheduling is triggered in a lazy manner when the scheduling decision is required by CPUs. 5. The execution phase for transaction t1 is entered. 6. If transaction t1 makes another task T runnable which is in turn able to execute a new transaction t3 , transaction t2 currently running on the CPU on which T is mapped is truncated at the end time of transaction t1 . The execution phase t2 is entered. During the next scheduling round, the scheduler of the aforementioned CPU is able to decide whether to switch to t3 or to resume t2 . 7. Commands which have been executing are prepared. 8. CPUs which have carried out a transaction are rescheduled. 9. Return to step 3

5

Experimental Results

Two important cases are considered to evaluate the performance of our simulation engine: event notiﬁcation/reception on the one hand and communication via a data channel on the other hand. Therefore, two task sets have been executed on both simulators - the SystemC one, and the C++ one - and the execution time has been captured. The measurements have been performed under the following conditions: 1. Output capabilities of both simulators are disabled in order to measure the pure simulation time. No trace ﬁles are created. 2. The time consumed by the initialization procedure of both simulation environments is not taken into account for the same reason.

Fast Simulation Techniques for Design Space Exploration

323

3. All measurements are subject to noise caused by the multi tasking operation system and interfering tasks running on the CPU which executes the simulation. Therefore, the average of several measured values has been taken into account for all diﬀerent tests. Thus the noise should distort similarly the results for both simulators. 4. As the analysis has been carried out on a less powerful machine, only the comparison of both simulators is meaningful in Figures 9 and 10, and not the absolute execution times. According to this, Figure 11 merely reﬂects the qualitative behavior of the new simulator; it performs much faster on recent machines. The ﬁrst task set is composed of two tasks communicating by means of a Blocking Read-Blocking Write channel with the width parameter set to 1 and the length parameter set to 100. All commands are nested in a 1 million-loop structure. This way, the ratio of execution time of the simulator and noise due to the multitasking operating system (task switches,...) is increased. Let us now have a closer look at the example tasks. Task 1 writes x samples into channel ch1, performs x integer operations and ﬁnally reads x samples from channel ch2. Task 2 carries out the complementary operations, it reads x samples from channel ch1, performs x integer operations and writes x samples to channel ch2. The second task set is conﬁgured similarly: the only diﬀerence is that channels are replaced by events based on inﬁnite FIFOs. Figure 9 and Figure 10 depict the execution time as a function of x which varies from 1 to 10. It can be deduced that the execution time of the old simulator grows more or less linearly with the command length. For the new simulator however, the execution time does not depend on the command length as one command corresponds to one transaction in this simulation. The execution time of the new simulator is only impacted by the number of transactions (linear growth, see Figure 11). But even in the worst case, when all transactions have a length of 1, the new simulator outperforms the old version by approximately a factor of 10. Depending on the respective model, the gain can be much higher given that a transaction comprises several cycles. Thus, the new simulator automatically adapts to the granularity of the application model: if a task issues commands having a length of one cycle, the simulator falls back on a cycle based consideration. If transactions are long, simulation eﬃciency increases accordingly.

6

Related Work

There are many approaches in practice today which tackle the Design Space Exploration issue. In the following, a few of the most relevant and widely practiced approaches are discussed very brieﬂy. Some of the key features and limitations are highlighted and a comparison with our approach is provided: The Ptolemy environment [11] is based on a simulator that oﬀers diﬀerent levels of abstraction for jointly simulating - or verifying - hardware and software components. Ptolemy is rather focused on embedded systems than speciﬁcally on design space exploration.

324

D. Knorreck, L. Apvrille, and R. Pacalet

70 Old simulator New simulator 60

time [sec]

50

40

30

20

10

0

1

2

3

4

5 6 command length

7

8

9

10

Fig. 9. Data channel performance for old and new simulator

70 Old simulator New simulator 60

time [sec]

50

40

30

20

10

0

1

2

3

4

5 6 command length

7

8

9

10

Fig. 10. Event performance for old and new simulator

In POLIS environment, applications are described as a network of Codesign Finite State Machines (CFSMs) [9]. Each element in the network can be mapped onto either a hardware or a software entity. This approach diﬀers from our work because it deals with application models mainly and there is no separation of architecture and application models. However, it is also intended for formal analysis. Metropolis is a design environment for heterogeneous systems [8]. It oﬀers various models of computation (MoCs). However, both application and architecture are modeled in the same environment. It uses an Instruction Set Simulator (ISS) to generate execution traces which relies on executable application code. Hence, there is no abstraction at application level.

Fast Simulation Techniques for Design Space Exploration

325

22 20 18

t [sec]

16 14 12 10 8 6 4

6

8

10

12

14

16 18 MTransactions

20

22

24

26

Fig. 11. Transactions per second, new simulator (less powerful machine!)

In SPADE/SESAME, applications are modeled as Kahn Process Networks (KPN) which are mapped onto the architecture models and the performance is analyzed through simulation [13] [14] . There is separation of application and architecture. This approach utilizes symbolic instructions (abstractions of RISC instructions) and the models are not generic. In ArchAn (Architecture Simulation Environment), the architecture is modeled at the cycle-accurate level using a mixture of synchronous language Esterel and C. ArchAn focuses on the optimization of hardware architecture by early performance analysis [16] [10]. However the approach in ArchAn does not describe the application and architecture in an orthogonal fashion. It is not intended for formal analysis as well. [17] proposes a methodology to speed up simulations of multi-processor SoCs at TLM level with additional timing information. The idea of time propagation through transaction passing bears some resemblance with our new simulation approach. However, our environment is settled at a higher abstraction level so that the procedure of transaction passing has been extended to optimally support high level models. As our aim is fast simulation, our simulation engine is not based on the SystemC kernel any more. [15] also follows the Y-Chart approach. Mapping of applications to architectures is performed on graph-based descriptions and based on side-information provided by the system designer. The mapping methodology uses abstract information such as cycle counts, and as opposed to our approach the control ﬂow of algorithms cannot be reﬁned. The aforementioned framework is not intended for formal analysis. System level simulators like [4] and [2] are typically platform based, they rely on a bottom-up modeling methodology and require speciﬁc models of architecture components (processors, buses,...) to perform transaction level simulation. In that context, ISS may constitute an adequate means to provide cycle accurate behavior of CPU models. [1] yet goes a step further by introducing generic

326

D. Knorreck, L. Apvrille, and R. Pacalet

HW components being parameterizable by the designer and a partially graphical way to describe application functionality. The latter approach bears resemblance with our architecture model in the way that it is also generic and that architecture exploration is accomplished on application level. But in addition to that, the DIPLODOCUS methodology is based on data and functional abstractions which may further reduce simulation time. Due to that functional abstraction, all involved models are completely graphical and no line of code has to be produced.

7

Conclusions and Future Work

The paper ﬁrst focuses on the description of TTool, a toolkit for performing eﬃcient design space exploration at a high-level of abstraction. Formal veriﬁcation and simulation techniques are integrated into TTool to assist designers in SoC architecture decisions. Modeling techniques are more particularly presented using an MPEG2 case study. A second important contribution introduced in the paper is the description of a lightweight discrete event simulation engine which is especially optimized for DIPLODOCUS high level application are architecture models. Simulation granularity automatically adapts to application models thanks to our transaction-based approach. When experimenting with models such the one of an MPEG2 decoder, we experienced performance gains in terms of execution time up to factor 30 as compared to a cycle-based SystemC simulator. It should be reemphasized that the gain depends on the application model, it could potentially be much higher. Several improvements are currently under development. At ﬁrst, hardware components model could be enhanced. For example, the bus model shall reﬂect modern communication architectures comprising several possible communication paths (e.g. Network-on-Chip). Furthermore, latency due to buﬀers in buses and bridges has been omitted. Thus, model realism versus simulation complexity is still under study for hardware components. Second, we expect to perform proﬁling of our simulation engine so as to identify time-consuming sections of code and recode them (e.g. in assembly language). Also, multiprocessor / core architectures could improve simulation performance signiﬁcantly. At last, more advanced simulation capabilities shall be introduced. Indeed, we expect the simulation engine to be able, at run-time, to check for functional requirements (e.g. if a client requests the bus, access is granted within 10ms). Also, it would also be very interesting to automatically explore several branches of control ﬂow in order to enhance the coverage of simulations.

References 1. 2. 3. 4. 5.

CoFluent Studio, http://www.cofluentdesign.com Coware Virtual Platforms, http://www.coware.com TTool, the Turtle Toolkit, http://labsoc.comelec.enst.fr/turtle Vast System Engineering Tools, http://www.vastsystems.com Apvrille, L.: TTool for DIPLODOCUS: An Environment for Design Space Exploration. In: Proceedings of the 8th Annual International Conference on New Technologies of Distributed Systems (NOTERE 2008), Lyon, France (June 2008)

Fast Simulation Techniques for Design Space Exploration

327

6. Apvrille, L., de Saqui-Sannes, P., Pacalet, R., Apvrille, A.: Un environnement UML pour la conception de syst`emes distribu´es. Annales des T´el´ecommunications 61(11/12), 1347–1368 (2006) 7. Apvrille, L., Muhammad, W., Ameur-Boulifa, R., Coudert, S., Pacalet, R.: A umlbased environment for system design space exploration. In: 13th IEEE International Conference on Electronics, Circuits and Systems, ICECS 2006, December 2006, pp. 1272–1275 (2006) 8. Balarin, F., Watanabe, Y., Hsieh, H., Lavagno, L., Passerone, C., SangiovanniVincentelli, A.: Metropolis: an integrated electronic system design environment. Computer 36(4), 45–52 (2003) 9. Balarin, F., Chiodo, M., Giusto, P., Hsieh, H., Jurecska, A., Lavagno, L., Passerone, C., Sangiovanni-Vincentelli, A., Sentovich, E., Suzuki, K., Tabbara, B.: Hardwaresoftware co-design of embedded systems: the POLIS approach. Kluwer Academic Publishers, Norwell (1997) 10. Chatelain, A., Mathys, Y., Placido, G., La Rosa, A., Lavagno, L.: High-level architectural co-simulation using esterel and c. In: Proceedings of the Ninth International Symposium on Hardware/Software Codesign, CODES 2001, pp. 189–194 (2001) 11. Eker, J., Janneck, J.W., Lee, E.A., Liu, J., Liu, X., Ludvig, J., Neuendorﬀer, S., Sachs, S., Xiong, Y.: Taming heterogeneity - the ptolemy approach. Proceedings of the IEEE 91(1), 127–144 (2003) 12. Object Management Group. UML 2.0 Superstructure Speciﬁcation, Geneva (2003), http://www.omg.org/docs/ptc/03-08-02.pdf 13. Pimentel, A.D., Polstra, S., Terpstra, F.: Towards eﬃcient design space exploration of heterogeneous embedded media systems. In: Deprettere, F., Teich, J., Vassiliadis, S. (eds.) SAMOS 2001. LNCS, vol. 2268, pp. 57–73. Springer, Heidelberg (2002) 14. Pimentel, A.D., Erbas, C., Polstra, S.: A systematic approach to exploring embedded system architectures at multiple abstraction levels. IEEE Transactions on Computers 55(2), 99–112 (2006) 15. Ristau, B., Limberg, T., Fettweis, G.: A mapping framework for guided design space exploration of heterogeneous mp-socs. In: Design, Automation and Test in Europe, DATE 2008, March 2008, pp. 780–783 (2008) 16. Silbermintz, M., Sahar, A., Peled, L., Anschel, M., Watralov, E., Miller, H., Weisberger, E.: Soc modeling methodology for architectural exploration and software development. In: Proceedings of the 2004 11th IEEE International Conference on Electronics, Circuits and Systems, ICECS 2004, December 2004, pp. 383–386 (2004) 17. Viaud, E., Pecheux, F., Greiner, A.: An eﬃcient tlm/t modeling and simulation environment based on conservative parallel discrete event principles. In: Design, Automation and Test in Europe, DATE 2006. Proceedings, March 2006, vol. 1, pp. 1–6 (2006) 18. Waseem, M., Apvrille, L., Ameur-Boulifa, R., Coudert, S., Pacalet, R.: Abstract application modeling for system design space exploration. In: 9th EUROMICRO Conference on Digital System Design: Architectures, Methods and Tools, DSD 2006, pp. 331–337 (2006)

PyGirl: Generating Whole-System VMs from High-Level Prototypes Using PyPy Camillo Bruni and Toon Verwaest Software Composition Group University of Bern – Switzerland

Abstract. Virtual machines (VMs) emulating hardware devices are generally implemented in low-level languages for performance reasons. This results in unmaintainable systems that are diﬃcult to understand. In this paper we report on our experience using the PyPy toolchain to improve the portability and reduce the complexity of whole-system VM implementations. As a case study we implement a VM prototype for a Nintendo Game Boy, called PyGirl, in which the high-level model is separated from low-level VM implementation issues. We shed light on the process of refactoring from a low-level VM implementation in Java to a high-level model in RPython. We show that our whole-system VM written with PyPy is signiﬁcantly less complex than standard implementations, without substantial loss in performance. Keywords: PyPy, Python, RPython, whole-system virtual machine, translation toolchain, high-level language, compile-time meta-programming Game Boy, 8-bit CPU.

1

Introduction

The research ﬁeld revolving around virtual machines (VMs) is mainly split up in two large subﬁelds. On the one hand we have the whole-system VM (WSVM) domain focusing on new ways to build and optimize emulators for hardware devices. These VMs mimic closely the actual hardware which they are emulating. On the other hand we have the language domain focusing on building high-level language VMs (HLLVM). These VMs only exist virtually. There are no hardware counterparts which natively understand the code running on those VMs. Although both domains share conceptual and implementation similarities, only recently has awareness been growing about the overlap of ideas and acknowledgement that the two ﬁelds can enforce each other. As a clear example of this fact we see that modern VM books discuss both ﬁelds [12]. Historically, the two ﬁelds developed independently of each other. Therefore the tools and techniques that are used in each of them are very diﬀerent. Although concepts for performance enhancement, like just-in-time compilation, are used in both ﬁelds, especially the infrastructure and tools used to realize the systems are very diﬀerent. An important progress that recently emerged in the ﬁeld of language virtual machines is the use of higher-level models for describing virtual machines. Final M. Oriol and B. Meyer (Eds.): TOOLS EUROPE 2009, LNBIP 33, pp. 328–347, 2009. © Springer-Verlag Berlin Heidelberg 2009

PyGirl: Generating Whole-System VMs from High-Level Prototypes

329

VMs are then generated from these prototypes and enhanced by speciﬁc, low-level optimization techniques. This approach has been realized in the PyPy project. The PyPy project aims at building a complete high-level Python VM in Python rather than in a low-level language like C. All VM implementation details such as garbage collection and JIT compilation are excluded from this prototype. Performance and VM speciﬁc details are then reintroduced by applying several model transformation steps from the Python sources to a highly optimized target VM. In this paper we report on our experience using the PyPy translation toolchain1 to prototype WSVMs in a high-level language without sacriﬁcing too much performance in the resulting VM, resulting in only about 40% slowdown compared to a similar WSVM in Java. By separating the high-level VM prototype from low-level implementation details we reduce code complexity. The high-level transformations provided by the PyPy toolchain ensure that the performance of the resulting VM is preserved. Our case study is a custom high-level VM prototype similar to the Squeak VM SPy [3]. SPy is written in RPython as a clean high-level implementation and uses the PyPy toolchain to reintroduce all VM implementation details. Rather than implementing a HLLVM we concentrate on emulating a Game Boy. We port an existing Java implementation of the virtual machine, Mario [5], to RPython. We focus on building a high-level and abstract but executable prototype rather than an inﬂexible and early optimized system. We then show that by using the PyPy toolchain we are able to generate performant low-level virtual machines from those prototypes. The main contributions of this paper are: – We show how the execution and implementation details of WSVMs are separated in the same way as those of HLLVMs. – We show how the use of preprocessing-time meta-programming minimizes the code and decreases the complexity. – We provide a sample implementation of a WSVM prototype for PyPy which exhibits a simpliﬁed implementation without substantial loss of performance (about 40% compared to a similar WSVM in Java). The remainder of this paper is structured as follows. In Section 2 we give an introduction to the PyPy project. Section 3 covers the technical details of the Game Boy, followed by the actual implementation details of PyGirl in Section 4. In Section 5 we compare the performance of the diﬀerent WSVMs implementations. In Section 6 the future work is discussed. Finally in Section 7 we provide a brief overview of our achievements.

2

PyPy in a Nutshell

In this section we describe the PyPy project, which produced the toolchain that we use to translate our VM model. The PyPy toolchain transforms high-level 1

http://codespeak.net/pypy/

330

C. Bruni and T. Verwaest

prototypes into highly optimized executable binaries that incorporate all needed general VM features. The initial goal of PyPy was to write a full-featured, customizable and fast interpreter for Python written in Python itself, in order to have the language described in itself, i.e. a meta-circular interpreter. Running an interpreter on top of another interpreter results in execution so slow as to be almost useless. PyPy addresses this by providing a “domain speciﬁc compiler”, a toolchain that translates high-level VM prototypes in Python down to executables for diﬀerent back ends, such as C/Posix [10]. Just like the interpreter, the toolchain itself is written in Python. The eﬀort of generating a VM from a model in the language itself is similar to other self-sustaining systems such as Squeak where the VM is written in Slang. Slang is a subset of Smalltalk [4] which can be directly translated to C. The major diﬀerence between Slang and PyPy is that Slang is a thinly veiled Smalltalk-syntax on top of the semantics of C, whereas PyPy focuses on making a more complete subset of the Python language translatable to C. This diﬀerence is clearly visible in the level of abstraction used by programs written for the respective platforms [6]. For instance, in the Squeak VM exception handling is manually added to check some bit ﬂags each time after returning from a function call. Not only is this a tedious task, but can also easily result in bugs by omitting a manual check. Using PyPy simpliﬁes this task, since it is possible to use highlevel exception handling in the VM prototype which eventually gets translated down automatically to something similar in the executable. There are many examples of high-level language virtual machines (HLLVM) that are realized using higher-level languages. Jikes [7] and earlier Jalapeño [1] realized a complete, modern optimizing Java virtual machine in Java. Inspired by the idea of Squeak, the Squawk2 Java VM [11] realizes a Java virtual machine in Java. The virtual machine is implemented in Java and then translated aheadof-time to an executable. Klein [14] was a research project to explore how to realize and especially bootstrap a virtual machine for the dynamic prototype based language Self [13]. In all these examples the virtual machines realized were language virtual machines rather than virtual machines simulating real hardware. To our best knowledge none of the used frameworks and tools have been evaluated in the context of hardware VMs. 2.1

The Interpreter

The starting point for PyPy is to create a minimal but full interpreter for Python written in Python itself. By minimal we mean that all interpreter implementation details such as garbage collection and optimizations are not implemented but provided by the environment running the interpreter. This results in a very clean and concise implementation of the interpreter, modelling how the language works without obscuring it with implementation details. 2

http://research.sun.com/projects/squawk/

PyGirl: Generating Whole-System VMs from High-Level Prototypes

2.2

331

The Translation Toolchain

In this subsection we describe PyPy’s translation toolchain, a “domain speciﬁc compiler” that translates VM prototypes to executables binaries. The translation of high-level VM prototypes to low-level back ends is necessary since the prototypes do not run fast by themselves. The Python interpreter written in Python running on top of standard CPython runs code about 2000 times slower than CPython.3 The PyPy translation toolchain is designed as a ﬂexible toolchain where front and back ends can be replaced so that it can generate VMs for diﬀerent languages running on diﬀerent platforms. Not only the front and back end can be changed, but also the set of transformations applied during the translation process. This results in fast and portable VMs. The following ﬁgure shows how prototype VM models can be translated to diﬀerent back ends:

Prolog

Python Interpreter

Scheme Other interpreters…

JavaScript

Type and Flow Analysis

Specialize for object oriented environment

CLI backend

pypy.net

JVM backend

js.net

prolog.net

JS backend

JPyPy

js-jvm

Specialize for lowlevel environment

pypy-c

prolog-js

C backend

LLVM backend

scheme-c

prolog-c

…-llvm

pypy-llvm

Fig. 1. PyPy translation toolchain architecture

This overall architecture of PyPy is shown in Figure 1. On top we have the front end or VM prototype which is the input to the translation toolchain. As output we get a self-containing VM which contains all the required implementation details. This VM is compatible with one of the many back ends which PyPy targets. From here on we will discuss all the steps which the toolchain undergoes to go from prototype to back end speciﬁc VM. In short, the PyPy toolchain builds a dynamically modiﬁable ﬂow graph from the target program’s sources. Then this prototype is transformed in several steps 3

http://codespeak.net/pypy/dist/pypy/doc/faq.html

332

C. Bruni and T. Verwaest

until a ﬁnal binary results. Using the same intermediate representation for a large part of the translation and applying transformations in small steps allows us to customize every translation aspect. Translation Steps. In the ﬁrst step the translator loads the source-code it translates. Unlike standard compilers that start by parsing the source code, the PyPy toolchain never accesses the Python source code. The toolchain can use its hosting Python interpreter to load its input ﬁles. This is because the input code for the translator is RPython code (which is a subset of Python code), and the toolchain itself is running on top of a Python interpreter. While loading, the Python interpreter evaluates all top-level statements and adds the loaded method and class deﬁnitions to the global Python memory. Afterwards the toolchain uses the the globally loaded main function as entry point for its input graph. This setup allows the input prototype to apply meta-programming in plain Python code at preprocessing time. Only the object graph resulting from metaprogramming has to be RPython compatible (Listing 2 shows an example of this feature). Because PyPy mostly targets statically typed back ends the graph is annotated with inferred types. Starting with the speciﬁed entry point, the type inference engine works its way through the object ﬂow graph and tries to infer most speciﬁc types. If multiple types are possible for a certain node, the type inference engine tries to select the least common superclass as type. If the least upper bound degenerates to Object, an error is thrown to show that no specialization is possible. The same happens when two types have no common superclass, like booleans and objects. If such type-errors arise, they have to be ﬁxed by the programmer. These problems are often solved by introducing a new common superclass or moving a method higher up the hierarchy. In other cases they really are semantic errors and require restructuring. Section 4.3 covers some aspects of resolving errors discovered by the toolchain. Here we see that the compiling process has an impact on the structure of the input source code. It eﬀectively limits the expressiveness of the input language. For this reason we call the restricted input language understood by the PyPy toolchain RPython instead of Python. RPython is described in more detail in Section 2.3. The annotation step is followed by the conversion from a high-level ﬂow-graph into a low-level one. Up to now there are two converters, one which specializes towards low-level back ends like C and one which specializes towards objectoriented back ends like CLI. To the low-level ﬂow-graph optional back end optimizations are applied. These optimizations are rather similar to optimizations found in standard compilers, like function inlining and escape analysis. After the low-level ﬂow-graph is optimized, it gets specialized for a speciﬁc back end. The preparation for code generation covers the following steps:

PyGirl: Generating Whole-System VMs from High-Level Prototypes

333

– Insertion of explicit exception handling. – Adding memory management details. Diﬀerent garbage-collection strategies are available4. Note that these garbage collectors themselves are also written in Python code. They also get translated and woven into the VM deﬁnition. – Creation of low-level names for generated function and variables. Eventually the language-speciﬁc ﬂow-graph is transformed into source ﬁles. These source ﬁles are then again processed by the back end, which can perform further domain-speciﬁc optimizations. For example generated C source ﬁles are compiled with GCC using the -O3 ﬂag. 2.3

RPython

The PyPy translation toolchain is designed to boost the performance of the Python interpreter written in Python. More than just that, it translates general VM prototypes to fast executable binaries. The translation from a dynamicallytyped language like Python to a statically-typed language like C is not straightforward however. As mentioned before, in order to be able to preserve the semantics we are forced to limit the expressiveness of the input language. For this reason, when we talk about the language accepted by the translation toolchain, we do not refer to Python but rather to RPython or restricted Python. The language is deﬁned implicitly by the translation toolchain5 The main diﬀerences between the full Python language and RPython are summarized in the following list: – – – –

Variables need to be type consistent, Runtime reﬂection is not supported, All globals are assumed to be constants, Types of all variables in the code must be inferable.

Although these restrictions seem to be substantial for a dynamic language such as Python, it is still possible to use high-level features like single inheritance, mixins and exception handling. More importantly, since RPython is a proper subset of Python, it is possible to test and debug the input programs with all Python tools before trying to translate it. Since VM prototypes we build for PyPy are executable by themselves, there is a great development speedup against a classical compile-wait-test cycle.

3

Game Boy Technical Details

As a case study we implement PyGirl, an executable VM prototype of a Game Boy. We will then translate this prototype using the PyPy translation toolchain 4 5

http://codespeak.net/pypy/dist/pypy/doc/garbage_collection.html http://codespeak.net/pypy/dist/pypy/doc/coding-guide.html# restricted-python

334

C. Bruni and T. Verwaest

presented in the previous section. In this section we list the technical details of the gaming hardware. The oﬃcial documentation is available on Nintendo’s website6 Cartridge

CPU

Sound

ROM

Device

Video

RAM

3.1

JoyPad

Hardware Pieces

The Game Boy system is composed of six essential pieces which are accessible through shared memory. External events are supported through an 8 bit maskable interrupt channel. There are two kind of 8 bit opcodes: – First-order opcodes are executed directly – Second-order opcodes fetch the next instruction for execution. The combined opcode doubles the range of possible instructions at the cost of execution speed. The second-order opcodes are mostly used for bit testing and bit setting on the diﬀerent registers. The following list shows some more speciﬁc details of the diﬀerent parts of Nintendo’s gaming device. – The 8 bit CPU is a slightly modiﬁed version of the Zilog 80 with a speed of 4.19 MHz. The CPU supports two power-saving mechanisms both working in a similar way. After a certain interrupt, the CPU is put into a low power consumption mode which is left only after another interrupt has occurred. – The cartridge contains ROM with the embedded game and possibly additional RAM and/or other devices. The size of the RAM depends on the type of cartridge. Some types of cartridges support an additional battery to store game-state. Switchable memory banks are used to extend the 8 bit limited address range. A checksum in the header and a startup procedure are used to guarantee that the device is working correctly and the cartridge is not corrupted. – The supported resolution is 160×144 pixels with 4 shades. It is possible to show maximally 40 sprites of 8×8 or 8×16 pixels at the same time. The video chip has two tile-map memory regions, one for the background and one for the foreground. The actual background and foreground drawn on the screen 6

http://www.nintendo.co.uk/NOE/en_GB/support/game_boy__pocket__color_ 559_562.html

PyGirl: Generating Whole-System VMs from High-Level Prototypes

335

are cropped versions of the data available in the tile maps. This allows the Game Boy to easily refresh the tile maps. – A serial connection can be used to communicate with another device. – The sound chip supports stereo sound and has four internal mono sound channels. Sound can be either read directly from the RAM, thus creating arbitrary samples at the cost of its calculation, or it can be produced via a noise-channel or via two diﬀerent wave-pattern generators.

4

PyGirl Implementation

In this section we highlight the most relevant implementation details of the PyGirl VM. We present a list of refactorings that we applied while going from a source implementation to our own model. We especially stress the details regarding the application of preprocessing-time meta-programming. Instead of starting to implement from a formal speciﬁcation for the Game Boy we adapt an existing stereotypical VM implementation to PyPy. This allows us to show the diﬀerences of both implementation styles more easily. We then compare the complexity of our resulting prototype to various other Game Boy VM implementations: Mario which we refactor to our own implementation, JavaBoy7 and AEPgb8 . Table 1 shows how our ﬁnal prototype diﬀers from the other Game Boy VM implementations in terms of McCabe cyclomatic code complexity (MCC) [8]. It assigns a number to a piece of code corresponding to the number of possible traces through the code. Table 1. McCabe cyclomatic complexity of the Cartridge related classes Cartridge JavaBoy AEPgb Mario PyGirl r54984 r63242 KLoC 1.0 0.5 1.2 0.8 0.8 Number of methods 28 24 84 89 90 MCC Sum over all methods 220 103 268 217 211 Methods over MCC > 10 3 1 5 0 0 Max MCC 70 17 23 9 8 average 7.86 4.29 3.19 2.44 2.34

4.1

Source Implementation

In this section we present the important details of the Game Boy VM written in Java from which we started. The Game Boy VM Mario [5] is developed in a portable manner by abstracting out platform-speciﬁc details from certain components. Hence it provides variants of the emulator for the diﬀerent versions of Java architectures like the Java Standard Edition and Applets for the web. 7 8

http://www.millstone.demon.co.uk/download/javaboy/ http://sourceforge.net/projects/aepgb/

336

C. Bruni and T. Verwaest

The application is structured by providing one class for each physical piece of hardware. Platform-speciﬁc parts are factored out by providing a set of abstract driver interfaces handling input and output. These drivers are then implemented for each architecture separately, adapting to the platform-speciﬁc requirements. Even though this is a fairly abstract and portable design, this already is an indicator that without a toolchain VM implementations are bound to be cluttered with back end speciﬁc details. While at ﬁrst glance the code appears to be written in an object-oriented manner, many parts of Mario strictly follow the low-level execution details of the hardware. On top of this, the implementation is cluttered with local optimizations. Two types of optimization strategies clearly stand out: manual inlining of code and manual unrolling of loops. Both strategies result in an overly expanded code-base, obscuring the overall design and semantics. For example, the CPU class is cluttered with such speed optimizations. The reason is that a CPU is a very low-level general-purpose device which does not provide many possibilities for abstraction. However, even the video chip is implemented in a non-abstract procedural way. This is so even though there are more conceptual components ready for abstract representation, such as sprites, background and foreground. PyGirl uses high-level abstractions for these components resulting in less complex code. Table 2 shows that our Video classes are less complex than the ones from Mario. Table 2. McCabe cyclomatic complexity of the Video related classes Video JavaBoy AEPgb Mario PyGirl r54984 r63242 KLoC 1.0 1.1 1.2 0.6 1.1 Number of methods 53 108 68 76 182 MCC Sum over all methods 212 288 223 201 293 Methods over MCC > 10 5 5 5 3 0 Max MCC 36 24 20 15 8 average 4.00 2.67 3.28 2.64 1.61

4.2

From Java to Python

Now we show how we migrated the source VM from to Java to Python. In a ﬁrst step we ported the code one-to-one, in order to easily track the upcoming refactoring progress. During the whole process we keep the overall structure of the existing system because it directly corresponds to the hardware. The following sections cover diﬀerent refactorings we apply and abstractions introduced to go from a low-level detailed implementation to a high-level prototype of the VM. Memory Usage Considerations. The Java code is cluttered with type-casts between bytes and integers. Bytes are used to represent the 8 bit hardware

PyGirl: Generating Whole-System VMs from High-Level Prototypes

337

architecture, whereas integers are used for all sorts of arithmetic operations. Instead of using integers whenever possible, the Java version focuses on reducing the memory footprint of the running emulator and focuses on the implementation details of the original hardware. Only at very few places in the code is the use of bytes justiﬁed by the resulting two’s-complement interpretation of the numbers. In our prototype type-casts are removed wherever possible to improve readability and maintainability. Firstly we consider the memory footprint of the device we emulate too small to justify optimization of memory usage. Even if we use four to eight times as much memory as the original device would have used, corresponding to an expansion from 8 to 32 or 64 Bit, this would mean that we end up with about 20Mb memory usage. Running PyGirl on a 64 Bit machine results in a total memory usage of 24Mb. This is a negligible amount for modern computers. Secondly and more importantly it is, hypothetically speaking, possible to plug an additional transformation into the toolchain converting all integers to bytes. By doing so the memory footprint of the ﬁnal VM would again be equal to the one of the original device. Metaprogramming. As described in Section 2.2 we can use the full Python language at preprocessing-time for meta-programming. The most prominent candidate for refactoring is the CPU class. It is packed with duplicated and inlined code and has a typical opcode dispatch switch. Table 3 shows the MCC for the CPU class in Mario, the two other Game Boy VM implementations AEPgb and JavaBoy and two snapshots of PyGirl. Table 3. McCabe cyclomatic complexity of the CPU related classes CPU JavaBoy AEPgb Mario PyGirl r54984 r63242 KLoC 2.7 2.9 4.2 1.2 1.0 Number of methods 22 111 372 173 180 MCC Sum over all methods 801 704 996 272 252 Methods over MCC > 10 5 1 2 0 0 Max MCC 415 536 513 10 9 average 36.41 6.34 2.28 1.57 1.4

PyGirl’s CPU class has half of the number of methods that Mario has. The sum of the code complexity over all methods is drastically reduced. One reason for the high complexity sum of Mario’s CPU class is its size. The original class is around 4000 lines long, featuring an unpleasant 1700 line switch delegating the incoming opcodes. On top of that there is a nested switch of 800 LoC handling the second-order opcodes. An excerpt of this dispatch switch is given in the following listing:

338

C. Bruni and T. Verwaest

public void execute(int opcode) { switch (opcode) { case 0x00: this.nop(); break; ... case 0xFF: this.rst(0x38); break; default: throw new RuntimeException(ERR); } }

In both switches we identify patterns which can be used as basis for abstractions. In the following excerpt from the original Java code we see how bytecodes directly encode their semantics in a structured way: public void execute(int opcode) { ... case 0x78: this.ld_A_B(); case 0x79: this.ld_A_C(); ... } public final void ld_A_B() { this.a = this.b; this.cycles -= 1; } Listing 1. Java: Grouped opcode mappings

The Java code covers all these switch cases by manually specifying them and by encoding the logic in one function per opcode. Since all these operations are symmetrical in terms of semantics and use of cycles the code can be compacted by using meta-programming. Even while there are only few lines of code per operation there is quite some redundancy. Instead of separate functions PyGirl uses a single load function for all register combinations. We reuse the function for multiple opcodes by applying the load function to each time two register objects. Every combination of the load function with two registers is related to a single opcode encoding its meaning. def load(self, register1, register2): register1.set(register2.get())

PyGirl: Generating Whole-System VMs from High-Level Prototypes

339

To refactor Listing 1, we create such reusable functions for all the diﬀerent types of operations. Then we replace the switch with a compact lookup in an opcode table generated from the abstract functionality descriptions. def execute(self, op_code): OP_CODES[op_code](self)

Instead of hard-coding the mapping to the respective functions, we use metaprogramming to compute the deﬁnition at translation time. The mapping of opcodes to functions in the example Listing 1 can be replaced with a more compact deﬁnition. We specify the connected opcodes in a set of entries, each consisting of a starting opcode, an oﬀset, a function and a set of registers. At runtime a sequence of opcodes is mapped to such a function. A corresponding register out of the register set is passed as an argument to this function. An example of this compact opcode deﬁnition is given in the following listing: REGS = [CPU.get_bc, CPU.get_de, CPU.get_hl, CPU.get_sp] SET = [ (0x01, (0x03, (0x09, (0x0B,

0x10, 0x10, 0x10, 0x10,

CPU.fetch_double_register, REGS), CPU.inc_double_register, REGS), CPU.add_hl, REGS), CPU.dec_double_register, REGS),

... (start, step, func, registers) ... ] OP_CODE_TABLE += create_op_codes(SET) Listing 2. RPython: Compacted deﬁnitions of opcodes

In the ﬁrst line we see that we do not directly specify the register. Instead we use getter methods of the CPU class returning these registers at runtime. The ﬁrst line of the SET speciﬁes that the opcode 0x01 is mapped to fetch_double_register passing in the register returned by get_bc(). The opcode is 0x11 using the result of get_de() as argument to the function. Since there are 4 registers in the SET the last opcode in this sequence is 0x31. Next we have to create the concrete methods from this deﬁnition. Helper function take the opcode deﬁnition set at preprocessing time and create corresponding closures. The create_op_codes method in the following listing creates such closures: def create_op_codes(table): op_codes = [] for entry in table: op_code = entry[0] step = entry[1]

340

C. Bruni and T. Verwaest function = entry[2] for getter in entry[3]: op_codes.append( (op_code, register_lambda(function, getter))) op_code += step return op_codes

Since Python handles the scope of variables at the function rather than the block level we introduced a further helper method register_lambda for creating the closures. In the following excerpt you can see how this method creates a speciﬁc closure depending on the incoming registerOrGetter argument: def register_lambda(function, registerOrGetter): if callable(registerOrGetter): return lambda s: function(s, registerOrGetter(s)) else: return lambda s: function(s, registerOrGetter)

We create opcode table entries at preprocessing time not only for most of the register operations such as loading and storing, but also for nearly all other register operations. In total we apply meta-programming to generate about 450 out of all 512 opcodes. As a note on performance, when translating this code to C, PyPy is able to take the preprocessed opcode table and translate it back into an optimized switch. So the source code stays compact and maintainable without substantial loss in performance. Even better, future versions of PyPy are expected to automatically optimize running bytecode interpreters dynamically towards the bytecode they evaluate, i.e. JIT compiling the bytecodes. PyPy can also inline small methods. Although there are no inlined methods in Mario, it would be the next logical step for manually optimizing the code. By letting PyPy handle the optimizations, we maintain a clean implementation. 4.3

Translation

Directly trying to translate our VM prototype, which is initially full Python code, to a low-level back end raises conﬂicts. This is due to the fact that the PyPy translation toolchain does not take full Python code as input, but rather a restricted subset of Python called RPython. The restrictions imposed by RPython only become apparent when you try to translate prototypes with PyPy. Most bugs come from the fact that while Python is fully polymorphic, RPython enforces the static types of all variables to be correct. Every variable needs to be inferable to a speciﬁc type. The most generic type in the system, i.e. Object, is not allowed as type for any variable. All messages sent to instances of a class must be declared in that class, or in any of its superclasses.

PyGirl: Generating Whole-System VMs from High-Level Prototypes

341

The easiest translation bugs are straightforward syntactic bugs, like typos. Such bugs often go unnoticed in dynamic programming languages since they are not statically enforced and are only a problem at runtime. After ﬁxing these, you typically encounter type conversion errors. Some of these problems are related to assignments of objects of diﬀerent types to the same variable. One possible solution for static type inconsistencies is to introduce common abstract superclasses. This ensures that objects assigned to a single variable have this superclass as a common type. Call Wrappers. We handle operations on registers and other CPU functions elegantly by passing function closures around. This allows us to reuse methods for diﬀerent actions. A very common example is the following load function: def load(self, getter, setter): setter(getter()) load(self.flag.get, self.a.set) load(self.fetch, self.a.set)

The load function is called with diﬀerent arguments. In the ﬁrst example the function is used to copy the values from the ﬂag-register into the register a. The second example loads the next instruction into register a. When we ﬁrst translated this code fragment the toolchain was unable to transform the code into a typed counterpart. PyPy was unable to create a strict typed function due to the diﬀerent origins of the passed function closures. We resolve this by creating call-wrappers with a common superclass. Introducing call-wrappers for the passed function closures adds some overhead, but it is a simple way to keep the code minimal and close to the original idea of passing around closures. The following code shows the same function calls using wrappers for each passed type of closure: class CallWrapper(object): def get(self, use_cycles=True): raise Exception("called CallWrapper.get") def set(self, value, use_cycles=True): raise Exception("called CallWrapper.set")

class RegisterCallWrapper(CallWrapper): def __init__(self, register): self.register = register def get(self, use_cycles=True): return self.register.get(use_cycles) def set(self, value, use_cycles=True): return self.register.set(value, use_cycles)

342

C. Bruni and T. Verwaest

class CPUFetchCaller(CallWrapper): def __init__(self, CPU): self.CPU = CPU def get(self, use_cycles=True): return self.CPU.fetch(use_cycles)

RegisterCallWrapper takes a register and calls get or set on it. The CPUFetchCaller is an abstraction for the fetch method of the CPU that allows it to be used as an argument for the load function. These are two out of the ﬁve total callwrappers we use to handle closure passing. The common superclass CallWrapper makes it possible for PyPy to infer a common type for the argument of every method. Note that return-types of methods are also required to be type-correct. For our wrappers this implies that all get methods are required to return a value of the same type. Thus all get methods return integers. To support the callwrappers we replaced the closure calls. Instead of directly invoking the argument as in Listing 4.3 we call get or set on the call wrappers. Applied to the previously presented load method this resulted in the following code: def load(self, getCaller, setCaller): setCaller.set(getCaller.get())

Then the method-calls from the original example look now like load(RegisterCallWrapper(self.flag), RegisterCallWrapper(self.a)) load(CPUFetchCaller(self), RegisterCallWrapper(self.a))

While it is possible to do the same in Java, this imposes a huge runtime overhead. If we create the CallWrappers in Java, this form of meta-programming is not handled at preprocessing time. Instead the objects will be around at runtime, which implies a performance as well as a memory overhead.

5

Performance Evaluation

In this section we show that using the PyPy toolchain to reduce complexity of WSVMs does not result in substantial performance loss. To do so we compare the performance of our Game Boy VM PyGirl to the performance of our source implementation, Mario. We run benchmarks on three diﬀerent versions of the Game Boy VM. The original Java emulation Mario, the interpreted variant of PyGirl and ﬁnally the translated binary version of PyGirl. We benchmark the interpreted PyGirl by running it on top of CPython whereas the translated version is built from those sources. Each test shows the average execution time over 100 runs using Java 1.6.0_10, Cacao 0.97 and CPython 2.5.2 (additionally using Psyco 1.6-1) on a 64 Bit Ubuntu 8.04.1 server machine with an Intel Xeon CPU QuadCore 2.00 GHz processor. We use revision number r63242 of the PyPy project.

PyGirl: Generating Whole-System VMs from High-Level Prototypes

5.1

343

Benchmark Details

In this section we discuss the benchmark parameters. The binaries for the RPython benchmark are created using the default arguments for PyPy resulting in C code which is then transformed to an executable binary via GCC. PyPy uses the following GCC optimizations when creating the binary executable: -c -O3 -pthread -fomit-frame-pointer In order to show the diﬀerence and impact of a JIT and dynamic optimization for the interpreted PyGirl we use Psyco [9], a just-in-time specializer for Python. Since the current version of Psyco (1.6-1) only emits machine code for 32 bit intel-based systems, we benchmark our code using Psyco on a similar Ubuntu installation with the appropriate 32 bit processor. Since those results are less important than the comparison with Java, we simply scaled them to ﬁt the 64 Bit machine. Those results should only display the possible performance gain by using a JIT. Still, it highlights that running the emulator on top of Psyco equally results in unacceptable performance. For those who are missing the -server switch in the Java 1.6 benchmark, we can say that this option did not result in any signiﬁcant performance gain for the tests. The speedup is less than a percent and about the order of the standard deviation, thus the results are merely distinguishable from the standard conﬁguration. 5.2

Runtime Optimization Comparison

In this section we compare the performance of Mario and PyGirl. In order to test performance we let both systems run a ROM which exercises the video output9 but produced no sound10 . The ROM simply prints hello world! on the background while a smiley moves over the screen as a moving sprite, a simple but complex enough benchmark. In the Table 4 you can see the actual CPU time used by the emulators in relation to the actual game time. By game time we mean the actual time that a user spends playing a game at normal speed. By CPU time we mean the time spent by the processor of the host actually running the emulator. Notice that as game time increases, the diﬀerence in CPU time between Mario and the translated PyGirl emulator shrinks. Eventually Mario gets faster than PyGirl due to the runtime optimizations of the JVM. Figure 2 shows the performance for both emulators running on diﬀerent platforms. It is clearly visible that the translated PyGirl runs at linear speed, whereas the Mario behaves diﬀerently. The JVM’s JIT is only warmed up after about 30 to 40 seconds of actual game time. The Cacao VM performs somewhat diﬀerently from the standard JVM implementations. Cacao’s JIT already runs Mario optimized after very little game time, but ends up being even a bit 9 10

The pixels are drawn in internal buﬀers but are not actually written to a screen. In both systems we fully disabled the sound drivers so that they cannot have any impact on performance.

344

C. Bruni and T. Verwaest

Table 4. Inﬂuence of the JIT on the benchmark results. Average execution in seconds over 100 runs per test. Game Time RPython CPU time 1 0.03 4 0.12 10 0.29 15 0.43 30 0.86 60 1.73 300 8.60 600 17.23 1800 51.81 3600 103.62

Java 1.6 CPU time Ratio 0.19 0.15 0.25 0.46 0.37 0.78 0.46 0.93 0.74 1.17 1.28 1.36 5.47 1.57 10.73 1.61 31.65 1.64 63.08 1.64

CPU time for emulation [Seconds]

100

10

1

0.1

0.01

RPython r63242 Java 1.6.0.10 Java Cacao 0.97 CPython 2.5.2 CPython 2.5.4 Psyco 10

100

Game time on the Game Boy [Seconds]

Fig. 2. This graph shows the CPU time of the host needed to emulate an example ROM, in relation to the game time on the Game Boy. It compares the performance of PyGirl and Mario emulating all device parts excluding sound. The results are discussed in Section 5.2.

slower than PyGirl. While Mario on the standard JVM has the advantage of the availability of a JIT compiler and dynamic optimizations, it is only about 60% faster than PyGirl.

PyGirl: Generating Whole-System VMs from High-Level Prototypes

345

Simple benchmarks running on simple language interpreters which are compiled with PyPy to its CLI-back end11 have shown that a JIT compiler can also be used here to get signiﬁcant performance gains. We strongly believe that future versions of PyPy will directly improve the performance of the PyGirl VM, thanks to a generated JIT compiler, by the same order of magnitude. Future versions of PyPy featuring dynamic optimizations for the C-back end might even help us outplay the performance of Mario running on the standard JVM.

6

Future Work

In this section we discuss the future work needed for PyGirl and related tasks for PyPy. 6.1

Future Work for the Game Boy VM PyGirl

While most hardware parts already have a fully functioning software counterpart, this is not the case yet for the sound unit. This is mostly the case since it is the least important piece of hardware for the Game Boy emulator to be immediately usable. For it to work it still needs to be fully ported and refactored. 6.2

Future Work for PyPy

PyGirl’s current state of implementation only allows the code to be translated using the C-back end. The current implementation of the I/O drivers are based on libSDL12 , thus it is not yet compatible with other back ends. The fact most VMs only need very basic graphics support, like simple bitblitting, makes enabling graphics support for the other back ends a straightforward task. Translating PyGirl with the JVM as target makes it possible to directly compare the performance of the original Java implementation with our approach. Since the Java Virtual Machine is widely available and compatible with many diﬀerent platforms, this would eventually even allow us to run PyGirl on mobile devices [2].

7

Conclusion

In this paper we have shown that the use of high-level prototypes for the deﬁnition of whole-system virtual machines and the application of meta-programming reduces code complexity signiﬁcantly without substantial loss of performance. By using the translation toolchain PyPy we keep our high-level WSVM model free from low-level implementation details, such as garbage collection and exception handling. When we build a ﬁnal WSVM from out prototype, those details are reintroduced by the translation toolchain. 11 12

http://morepypy.blogspot.com/2008/11/porting-jit-to-cli-part-1.html http://www.libsdl.org/

346

C. Bruni and T. Verwaest

We have supported our claims by comparing the performance and complexity of two VM implementations for the Game Boy, PyGirl and Mario. We found that for speciﬁc classes the average McCabe cyclomatic complexity is reduced to less than the half compared to other Game Boy VMs. In other cases, like the CPU class, we even reduced the maximum MCC from over 500 down to 9. In order to strengthen our case about the reduced complexity, we compared the complexity of our prototype with yet another two Game Boy emulators, exhibiting the same complexity issues as Mario. In Section 5 we have shown that using PyPy does not result in a signiﬁcant performance loss. PyGirl only runs about 40% slower than the Game Boy VM Mario on the standard JVM. Acknowledgments. We would like to thank Marcus Denker, Adrian Kuhn, Jorge Ressia, Oscar Nierstrasz and the anonymous reviewers for kindly reviewing this paper. We gratefully acknowledge the ﬁnancial support of the Swiss National Science Foundation for the project “Bringing Models Closer to Code” (SNF Project No. 200020-121594, Oct. 2008 - Sept. 2010).

References 1. Alpern, B., Attanasio, C.R., Cocchi, A., Lieber, D., Smith, S., Ngo, T., Barton, J.J., Hummel, S.F., Sheperd, J.C., Mergen, M.: Implementing Jalapeño in Java. In: Proceedings of the 14th ACM SIGPLAN conference on Object-oriented programming, systems, languages, and applications (OOPSLA 1999), pp. 314–324. ACM, New York (1999) 2. Aubry, L., Douard, D., Fayolle, A.: Case study on using PyPy for embedded devices. Technical report, PyPy Consortium (2007) 3. Bolz, C.F., Kuhn, A., Lienhard, A., Matsakis, N.D., Nierstrasz, O., Renggli, L., Rigo, A., Verwaest, T.: Back to the future in one week — implementing a Smalltalk VM in PyPy. In: Workshop on Self-sustaining Systems (S3) 2008, page TBA (to appear) (2008) 4. Goldberg, A., Robson, D.: Smalltalk 80: the Language and its Implementation. Addison Wesley, Reading (1983) 5. Hasan, C.: Mario (2007) 6. Ingalls, D., Kaehler, T., Maloney, J., Wallace, S., Kay, A.: Back to the future: The story of Squeak, a practical Smalltalk written in itself. In: Proceedings of the 12th ACM SIGPLAN conference on Object-oriented programming, systems, languages, and applications (OOPSLA 1997), November 1997, pp. 318–326. ACM Press, New York (1997) 7. The Jikes research virtual machine, http://jikesrvm.sourceforge.net/ 8. McCabe, T.J.: A measure of complexity. IEEE Transactions on Software Engineering 2(4), 308–320 (1976) 9. Rigo, A.: Representation-based just-in-time specialization and the psyco prototype for Python. In: PEPM 2004: Proceedings of the 2004 ACM SIGPLAN symposium on Partial evaluation and semantics-based program manipulation, pp. 15–26. ACM, New York (2004) 10. Rigo, A., Pedroni, S.: PyPy’s approach to virtual machine construction. In: Proceedings of the 2006 conference on Dynamic languages symposium, OOPSLA 2006: Companion to the 21st ACM SIGPLAN conference on Object-oriented programming systems, languages, and applications, pp. 944–953. ACM, New York (2006)

PyGirl: Generating Whole-System VMs from High-Level Prototypes

347

11. Simon, D., Cifuentes, C., Cleal, D., Daniels, J., White, D.: Java on the bare metal of wireless sensor devices: the squawk java virtual machine. In: VEE 2006: Proceedings of the 2nd international conference on Virtual execution environments, pp. 78–88. ACM Press, New York (2006) 12. Smith, J.E., Nair, R.: Virtual Machines. Morgan Kaufmann, San Francisco (2005) 13. Ungar, D., Smith, R.B.: Self: The power of simplicity. In: Proceedings OOPSLA 1987, ACM SIGPLAN Notices, December 1987, vol. 22, pp. 227–242 (1987) 14. Ungar, D., Spitz, A., Ausch, A.: Constructing a metacircular virtual machine in an exploratory programming environment. In: OOPSLA 2005: Companion to the 20th annual ACM SIGPLAN conference on Object-oriented programming, systems, languages, and applications, pp. 11–20. ACM, New York (2005)

Using Grammarware Languages to Define Operational Semantics of Modelled Languages Daniel A. Sadilek and Guido Wachsmuth Humboldt-Universit¨ at zu Berlin, Unter den Linden 6, 10099 Berlin, Germany {sadilek,guwac}@informatik.hu-berlin.de

Abstract. Abstract State Machines, Prolog, and Scheme are wellestablished for language engineering in the technological space of grammarware. In this paper, we show how they can be integrated into the technological space of modelware to describe operational semantics of modelled languages. Integration is based on three bridges: a physical, a logical, and a pragmatical bridge between grammarware language and modelling framework. We applied our approach to integrate Abstract State Machines, Prolog, and Scheme as description languages in EProvide 2.0, an extensible, uniﬁed framework for prototyping operational semantics of modelled languages. In this paper, we discuss in detail the bridging of Abstract State Machines as well as an industrial case study based on this.

1

Introduction

Technological spaces. This paper is concerned with describing operational semantics of computer languages. For this task, a language engineer needs a description language. Our goal is to enable language engineers to use grammarware description languages for deﬁning the operational semantics of modelled computer languages, i.e., of languages that are, apart from their operational semantics, deﬁned with modelware technologies. Modelware and grammarware are technological spaces. The term was initially proposed by Kurtev et al. to name a “working context with a set of associated concepts, body of knowledge, tools, required skills, and possibilities” [9]. The grammarware technological space is concerned with grammars, grammar-based description languages, and associated tools. The modelware technological space is concerned with metamodels, model-based description languages, and associated tools. Operational Semantics. In the grammarware technological space, operational semantics as pioneered by Plotkin [13] is a common way for specifying language semantics in terms of sequences of computational steps. A transition system Γ, → forms the mathematical foundation, where Γ is a set of configurations and → ⊆ Γ × Γ is a transition relation. This approach has several beneﬁts: First, we can describe the operational semantics in terms of the language’s structure. Thus, we stay in the language’s domain so that experts of that domain can understand these structures and can explain the expected eﬀects of execution. M. Oriol and B. Meyer (Eds.): TOOLS EUROPE 2009, LNBIP 33, pp. 350–358, 2009. c Springer-Verlag Berlin Heidelberg 2009

Using Grammarware Languages to Deﬁne Operational Semantics

Puzzle + iDimension: int 1..* cells

Cell

1..* ﬁelds

Field

ﬁelds 1..*

cells + iCellValue: int 1..* + iPossibleValues: int[0..*]

351

Run(s:Sudoku) =def forall f in s.ﬁelds do RunElimination (f) forall f in s.ﬁelds do RunSingleCell (f) forall f in s.ﬁelds do RunHiddenCell (f) forall f in s.ﬁelds do RunLockedCandidates (f) RunElimination(f:Field) =def forall c in f.cells with c.iCellValue<>0 do forall cc in f.cells do delete c.iCellValue from cc.iPossibleCellValues RunSingleCell(f:Field) =def forall c in f.cells with c.iCellValue = 0 and c.iPossibleCellValues.size = 1 do choose v in c.iPossibleCellValues do c.iCellValue := v

Row

Column

Box

...

Fig. 1. Simpliﬁed Sudoku metamodel and ASM semantics description from [7]

Second, if expressed in an appropriate language, an operational semantics description can be executed and, thus, be used as a prototypical interpreter. Certain grammarware languages are commonly used for this purpose, e.g., Abstract State Machines (ASMs), Prolog, or Scheme. In the modelware technological space, the standard way to prepare the execution of models is to translate them (by model transformation or code generation) into an executable target language. However, several proposals were made that allow to deﬁne operational semantics directly in the modelware technological space [11,14]. Their unifying idea is to model Γ , the possible runtime states of language instances, with a metamodel. But they use diﬀerent description languages to model the transition relation →. In [19], we employ QVT Relations [12] for this purpose. We combine this approach with existing model-driven editor creation technology in order to support rapid prototyping of animated visual interpreters and debuggers [15]. We called the implementation of that approach EProvide. Motivating example. As described above, there are established languages for prototypically implementing operational semantics in grammarware. People are used to these languages and there are semantics descriptions already expressed in them. We ﬁnd this, for example, when Gjøsæter et al. model a language to express Sudokus, the very famous mathematical puzzles [7]. A Sudoku consists of 9x9 cells, some of them containing numbers from 1 to 9, some of them empty. The Sudoku is solved by ﬁlling the empty cells with numbers from 1 to 9 in such a way that in each row, column, and each of the nine (non-overlapping) 3x3 sub-areas each number is unique. Besides deﬁning a Sudoku metamodel and some other artefacts, Gjøsæter et al. also describe the step-wise Sudoku solving process in terms of an ASM in an operational style. Though the description was only created for humans to read, an ASM was chosen because it is precise and concise. Fig. 1 shows the metamodel and excerpts from the ASM description.

352

D.A. Sadilek and G. Wachsmuth

Notably, the ASM description uses the class and attribute names from the metamodel. However, we have to translate the ASM into a modelware description language in order to derive an interpreter from it that works on instances of the metamodel. For example, we could translate it to QVT Relations, and EProvide would provide us with a visual interpreter. But using the ASM directly would be less expensive and much more elegant. The aim of this paper is, therefore, to directly use grammarware languages for implementing operational semantics of modelled languages. Structure of the paper. In the following section, we present a pattern for integrating a grammarware description language into modelware. We implemented this pattern in bridges between ASMs, Prolog, and Scheme on the one side and the Eclipse modelling framework EMF on the other side. In Sec. 3, we explain the implementation of the ASM bridges and their usage for the Sudoku example and for an ongoing industrial case study. The paper is concluded in Sect. 4.

2

General Approach

In each description language that should be used to express →, one needs to query and modify models. This is naturally supported by languages from the modelware technological space. Supporting model access in a grammarware description language requires to bridge the language with a modelling framework. A typical bridging approach is to translate models into a representation in the grammarware language [4,14,18] (usually by generating text). This is adequate when one only wants to query models. But it is insuﬃcient if one also wants to modify models because then one needs a back-propagation that merges the changes made in the grammarware language back into the model. Because this merging problem cannot be solved generically without a unique identiﬁer for model objects [2], we opted for another approach. Our bridging approach is to use proxy representations of the model objects in the grammarware language. Any modiﬁcation on the proxy representation is directly applied to the model, thus, avoiding the merging problem. Although the concrete implementation of such a bridge depends on the description language, we identiﬁed three typical facets that each must have: 1. Physical bridge: We must connect the runtime environments of the modelling framework and the description language physically to enable communication. 2. Logical bridge: We must integrate both environments logically to represent models with the data structures provided by the description language. 3. Pragmatical bridge: We must integrate models into the description language pragmatically. This means we can work with models in a similar way as with the description language’s native data structures. In the rest of this section, we discuss these facets in detail.

Using Grammarware Languages to Deﬁne Operational Semantics

(d) (b)

Semantics Description

(a)

Semantics Description

353

Semantics Description Proxy Representation

Proxy Representation

Model Description Language Interpreter

Description Language Interpreter

Proxy Repr.

JVM

(c) Semantics

Description

CPU Legend: Interpreted data Runtime environment

Program

Model or its Proxy Repr.

Transformation works on representation

Logical bridge

Fig. 2. Four cases (a – d) of description and model environment nesting

Physical Bridge. Description language and modelling framework can have different runtime environments, which we call description environment and model environment, respectively. In order to execute a semantics description, data from the model environment must be accessible in the description environment. This may require a physical bridge for communication. Whether such a bridge is necessary and how it looks like depends on how the environments are nested. We can distinguish four cases (a – d), which are visualised in Fig. 2. To discuss them, we assume the JVM as model environment. (a) Description and modelling framework share a runtime environment. A physical bridge is unnecessary. Java as a description language, for example, is executed on the JVM. Therefore, semantics descriptions written in Java can work directly with the Java representation of models. (b) Description environment is embedded into the model environment. A physical bridge is necessary, which consists of a foreign function interface (FFI). An FFI of a language allows one to call functions of the environment surrounding the language’s runtime environment. ASMs and Scheme are examples for this case. For these description languages, we use interpreters implemented in Java (SISC [10] and CoreASM [6]). Scheme already provided an appropriate FFI to Java, for ASMs we were able to adapt the interpreter. (c) Model environment is embedded into the description environment. A physical bridge is necessary. Description languages that get compiled to native code, e.g. C++, account for this case. Here, the Java Native Interface must be used as a bridge from the native description environment to the Java model environment.

354

D.A. Sadilek and G. Wachsmuth

(d) Both model and description environments are embedded into a surrounding runtime environment. A physical bridge is necessary. In this case, a natively implemented interpreter executes the semantics descriptions. For example, we use a native interpreter for Prolog (SWI-Prolog [20]). If both description environment and model environment provide FFIs, we can use them to implement the bridge. Logical Bridge. In each description language, the language engineer works on a model representation that uses the data structures available in that language (Fig. 2). A logical bridge is needed to map this representation to the representation provided by the modelling framework. In a dedicated model-transformation language, such as QVT Relations, the native data structures are already models. In contrast, for Java, for example, an object-oriented API for model access is necessary. It is already provided by the modelling framework EMF we use. In other description languages, model representation requires substantial work. This is especially the case with languages whose native data structures are designed for holding trees and cannot directly hold graphs. Models, which are typically graphs, would have to be encoded as trees with additional symbol tables. Fortunately, the three grammarware languages for which we implemented bridges fall not into this category. Pragmatical Bridge. One motivation to support several description languages is to allow a language engineer to select a language he is familiar with. Therefore, it is important to not only provide the language engineer with some interface for working with models in his language of choice but to provide him with one that integrates naturally into the language. There should be a pragmatical bridge that allows the language engineer to query and modify models in a programming style that ideally is indistinguishable from the style used to work with the language’s native data structures.

3

Integrating ASMs into Modelware

In the last section, we identiﬁed the three facets of each grammarwaremodelware-bridge. In this section, we explain their implementation for an ASMEMF-bridge. Physical Bridge. To execute ASMs, we use CoreASM [6], a Java-based interpreter (case b). For CoreASM, there is already a plugin available that provides data types to hold references to Java objects and an FFI to call Java methods. However, from a pragmatical point of view, this interface is not optimal because querying and modifying the model must be encoded in uses of the FFI. Furthermore, CoreASM does not provide metaprogramming facilities that would enable us to generate a better interface at runtime. Therefore, we decided not to use the existing FFI but to extend the CoreASM interpreter. We added additional interpreter classes that adapt existing ASM concepts to represent models as described in the following.

Using Grammarware Languages to Deﬁne Operational Semantics

355

Logical Bridge. In ASMs, data is represented as an algebraic structure, or algebra for short, that conforms to a certain signature. An algebra consists of a set of all its elements, called the superuniverse, and a number of functions. The functions can denote special elements (0-ary functions), the membership of elements to universes/domains (1-ary characteristic functions), or relations between elements (n-ary functions). A signature deﬁnes a vocabulary of function and universe names. To “program” an ASM, one writes guarded update rules that update the functions. For example, one may use a 0-ary function as an analogue to a variable in traditional programming. ASM execution is performed step-wise according to the rules of the ASM. In each step, the algebra is updated. (That is why ASMs were formerly called “evolving algebras”.) To use ASMs for describing operational semantics, we must represent models as algebras. For this, we derive an ASM signature from the metamodel: for each class in the metamodel, a universe name is deﬁned; for class features (attributes and associations), function signatures are deﬁned. Instances of metamodel classes can then be represented as universe elements; feature values are represented with function values. Pragmatical Bridge. The structures used in object-oriented modelling and in ASMs are surprisingly similar. In modelling, we deal with models that contain objects. A model conforms to its metamodel. An object is an element of the instance set of its class, and it can be related to other objects via links and attribute values that are instances of the features declared by its class. Analogue in ASMs, we deal with algebras that contain abstract entities. An algebra conforms to its signature. An abstract entity is an element of a universe with a universe symbol, and it can be related to other entities via functions with function signatures. Because of these structural similarities, we are able to map the modelling concepts directly to ASM concepts and, thus, to provide a very natural model representation in ASMs. Using the ASM-EMF Bridge for the Sudoku Example. With the ASMEMF bridge, we are able to copy the ASM description of the Sudoku solving semantics from [7] and execute it. We only have to make some minor syntactic modiﬁcations. For example, we must replace the dot-notation for accessing attributes with a function notation, e.g., we change “c.iCellValue” to “iCellValue(c)”. Furthermore, our implementation does not support the delete primitive, so we replace “delete c.iCellValue from cc.iPossibleCellValues” with “iPossibleValues(cc, iCellValue(c)) := false”. With an additional editor for Sudoku models, we can now create Sudokus and solve them step-wise. Using the ASM-EMF bridge, the ASM works directly on a model representation. This allows us to animate a Sudoku’s solving process using the Sudoku model editor.1 Industrial Case Study. We use the ASM bridge in an ongoing industrial case study conducted in co-operation with the Fraunhofer FOKUS. The goal of this 1

Full source code of the Sudoku example is available under http://eprovide.svn.sf.net/svnroot/eprovide/trunk/no.uia.sudoku*

356

D.A. Sadilek and G. Wachsmuth

case study is to develop a prototypical interpreter for Continous TTCN-3 [17], a proposed extension of the test speciﬁcation language TTCN-3. Continous TTCN3 extends TTCN-3 with concepts for testing embedded control systems, e.g., control systems in cars. To create a prototypical interpreter, we create a metamodel for Continuous TTCN-3 and describe its operational semantics with an ASM. Presenting the case study in detail is out of scope of this paper. Therefore, we just report about the necessary work for the ﬁrst prototype: The metamodel we created consisted of 17 classes. We could keep the metamodel so small because we modelled expressions as strings. Normally, expressions account for the most classes in a language’s metamodel. The ASMs that describe the semantics were (without empty lines or comments) around 130 lines with 650 tokens. We could also keep the ASMs quite small because we delegated the interpretation of the string expressions to an external (stub) interpreter. Creating the metamodel and the ASMs took us one day. We spent another day of work for creating (with a generative technique [16]) a textual editor for Continuous TTCN-3 instances that could also visualise the runtime states. This means, the presented approach allowed us to create and animatedly execute the ﬁrst Continous TTCN-3 example after just two days.

4

Conclusion

Contribution. We described a general pattern for the integration of grammarware languages into modelware based on a physical, a logical, and a pragmatical bridge. We demonstrated the integration of ASMs with the modelling framework EMF. We implemented our approach for ASMs, Prolog, and Scheme within EProvide 2.02 , an extensible, uniﬁed Eclipse-based framework for describing and executing operational semantics of modelled languages. EProvide 2.0 supports operational semantics description languages from both modelware and grammarware. Thus, it oﬀers language engineers the ﬂexibility to choose the description language that suits their skills and requirements best. Languages like ASMs, Prolog, and Scheme are well-established in grammar-based language engineering with a vast body of knowledge. Language engineers familiar with these languages are now able to use their knowledge in a model-driven language engineering process—exploiting the available tool support. Related Work. There are many description languages and corresponding tools concerning operational semantics in the modelware space. We discuss these approaches in [19,15]. In [15], we additionally discuss related work for prototyping visual interpreters. Related work about bridges between grammarware and modelware can be classiﬁed into approaches that use formalism mapping, transformations, and proxy representations. The idea of formalism mapping is to bridge the formalisms underlying grammarware and modelware, i.e. context-free grammars and metamodels. In [3], Alanen and Porres provide a bidirectional mapping between EBNF grammars and 2

E clipse plugin for PROtotyping V isual I nterpreters and DE buggers; available at http://eprovide.sf.net

Using Grammarware Languages to Deﬁne Operational Semantics

357

MOF metamodels. Additionally, they suggest a corresponding bidirectional mapping between syntax trees and models. A similar approach can be found in [21]. Mapping approaches pose restrictions on the metamodels, e.g., multi-valued features must be ordered and non-compositional associations are forbidden, which means one can handle only trees, not graphs. In contrast, we do not restrict the expressiveness of metamodels because we keep models in the modelware technological space and provide access to them in the grammarware technological space. The most common way of bridging is to use transformations. For example, this is the approach of MoMaT [18], with which Prolog can be used for model querying. It is also done for describing the semantics of modelled languages with ASMs, e.g., semantic anchoring by Chen et al. [4] and an extension of AMMA by Di Ruscio et al. [14]. These approaches transform models (language instances) to a grammarware (Prolog or ASM) representation, but they do not solve the back-propagation merging problem. With our approach of proxy representations, we circumvent this problem. There are also bridges that use proxy representations like we do. Similarly to our approach, in the Moses tool suite [5] one can write an ASM that works directly on an attributed graph (which plays the role of a model). But Moses does only provide a generic and not a typed model interface: It misses a pragmatical bridge. For example, to access a list of a Sudoku’s cells, one would write “V(G, ‘Cell’)” in Moses (“V” stands for vertices and “G” for the whole graph) and simply “Cell” with our approach. Future Work. As we mentioned, there are several approaches to describe operational semantics. Most of them come with their own description languages, which could be integrated into EProvide 2.0. Due to their formal foundations and popularity, we are particularly concerned with integrating graph transformations. Furthermore, we are working on testing the equivalence of operational semantics. Diﬀerent descriptions work on the same type of data structure, namely EMF models. Thus, we have a comparable basis for doing a bisimulation of some example language instance with diﬀerent semantics descriptions. To backtrack over runtime states of non-deterministic languages, we could use EProvide 2.0’s step-back support [8]. Acknowledgement. We thank Markus Scheidgen, Falko Theisselmann, Stephan Weißleder, and the reviewers for comments on preliminary versions of this paper. Terje Gjøsæter and Andreas Prinz explained us their Sudoku example. J¨ urgen Großmann explained us Continous TTCN-3. Roozbeh Farahbod helped us integrating the CoreASM interpreter. This work is supported by grants from the Deutsche Forschungsgemeinschaft, Graduiertenkolleg METRIK (GRK 1324/1).

References 1. Schieferdecker, I., Hartman, A. (eds.): ECMDA-FA 2008. LNCS, vol. 5095. Springer, Heidelberg (2008) 2. Alanen, M., Porres, I.: Diﬀerence and union of models. In: Stevens, P., Whittle, J., Booch, G. (eds.) UML 2003. LNCS, vol. 2863, pp. 2–17. Springer, Heidelberg (2003)

358

D.A. Sadilek and G. Wachsmuth

3. Alanen, M., Porres, I.: A relation between context-free grammars and Meta Object Facility. Technical report, TUCS (2004) 4. Chen, K., Sztipanovits, J., Abdelwalhed, S., Jackson, E.: Semantic anchoring with model transformations. In: Hartman, A., Kreische, D. (eds.) ECMDA-FA 2005. LNCS, vol. 3748, pp. 115–129. Springer, Heidelberg (2005) 5. Esser, R., Janneck, J.: Moses: A tool suite for visual modeling of discrete-event systems. In: HCC 2001, pp. 272–279. IEEE Computer Society Press, Los Alamitos (2001) 6. Farahbod, R., Gervasi, V., Gl¨ asser, U.: CoreASM: an extensible ASM execution engine. Fundamenta Informaticae 77(1-2), 71–103 (2007) 7. Gjøsæter, T., Isfeldt, I.F., Prinz, A.: Sudoku – a language description case study. In: Gaˇsevi´c, D., L¨ ammel, R., Van Wyk, E. (eds.) SLE 2009. LNCS, vol. 5452, pp. 305–321. Springer, Heidelberg (2009) 8. Hartmann, T., Sadilek, D.A.: Undoing operational steps of domain-speciﬁc modeling languages. In: Proceedings of the 8th OOPSLA Workshop on Domain-Speciﬁc Modeling (DSM 2008), University of Alabama at Birmingham (2008) 9. Kurtev, I., B´ezivin, J., Aksit, M.: Technological spaces: An initial appraisal. In: CoopIS, DOA 2002 Federated Conferences, Industrial track (2002) 10. Miller, S.G.: SISC: A complete scheme interpreter in Java. Technical report, Indiana University (January 2002) 11. Muller, P., Fleurey, F., J´ez´equel, J.: Weaving executability into object-oriented meta-languages. In: Briand, L.C., Williams, C. (eds.) MoDELS 2005. LNCS, vol. 3713, pp. 264–278. Springer, Heidelberg (2005) 12. Object Management Group. Meta Object Facility 2.0 Query/View/Transformation Speciﬁcation (2007) 13. Plotkin, G.D.: A Structural Approach to Operational Semantics. Technical Report DAIMI FN-19, University of Aarhus (1981) 14. Ruscio, D.D., Jouault, F., Kurtev, I., B´ezivin, J., Pierantonio, A.: Extending AMMA for Supporting Dynamic Semantics Speciﬁcations of DSLs. Technical Report 06.02, LINA Laboratoire D’Informatique De Nantes Atlantique (April 2006) 15. Sadilek, D.A., Wachsmuth, G.: Prototyping visual interpreters and debuggers for domain-speciﬁc modelling languages. In: Schieferdecker, I., Hartman, A. (eds.) ECMDA-FA 2008. LNCS, vol. 5095, pp. 63–78. Springer, Heidelberg (2008) 16. Scheidgen, M.: Textual modelling embedded into graphical modelling. In: Schieferdecker, I., Hartman, A. (eds.) ECMDA-FA 2008. LNCS, vol. 5095, pp. 153–168. Springer, Heidelberg (2008) 17. Schieferdecker, I., Großmann, J.: Testing hybrid control systems: an overview on cont. ttcn-3. Int. J. Softw. Tools Technol. Transf. 10(4), 383–400 (2008) 18. St¨ orrle, H.: A prolog-based approach to representing and querying software engineering models. In: Proceedings of the VLL 2007 workshop on Visual Languages and Logic. CEUR-WS, vol. 274, pp. 85–96 (2007) 19. Wachsmuth, G.: Modelling the operational semantics of domain-speciﬁc modelling languages. In: L¨ ammel, R., Visser, J., Saraiva, J. (eds.) GTTSE 2007. LNCS, vol. 5235, pp. 506–520. Springer, Heidelberg (2007) 20. Wielemaker, J.: SWI-Prolog 5.6 Reference Manual (2008), http://www.swi-prolog.org 21. Wimmer, M., Kramler, G.: Bridging grammarware and modelware. In: Bruel, J.-M. (ed.) MoDELS 2005. LNCS, vol. 3844, pp. 159–168. Springer, Heidelberg (2006)

Automatic Generation of Integrated Formal Models Corresponding to UML System Models Helen Treharne, Edward Turner1 , Richard F. Paige, and Dimitrios S. Kolovos2 1

Department of Computing, University of Surrey {H.Treharne,Edward.Turner}@surrey.ac.uk 2 Department of Computer Science, University of York {paige,dkolovos}@cs.york.ac.uk

Abstract. This paper presents a complex model transformation that takes two related UML models as input (class diagrams and state machines) and outputs an integrated formal model. The transformation is achieved using the Epsilon model management framework and has involved the deﬁnition of new CSP and B metamodels. The target model is a CSP || B speciﬁcation (a fusion of CSP and B) that can be used to analyse the source UML. Through a comparative study we conclude that the code written using Epsilon is more concise than the equivalent model-to-text transformation achieved using the iUML toolset. Keywords: Model transformation, Epsilon, UML, CSP, B.

1

Introduction

Model transformations are at the heart of Model-Driven Engineering (MDE) [1]. Transformations of various kinds (e.g., model-to-model, model-to-text, updatein-place) support MDE processes, allow incremental elaboration of models, and support interoperability between tools and languages. Model transformation also enables analysis of models, through transformation of system models (e.g., in UML, SysML, or domain-speciﬁc languages) to languages more amenable to rigorous analysis. There are powerful formal languages and tools to support rigorous analysis, ranging from ﬁnite-state model checkers, to automated or semi-automated theorem provers. Examples of such languages and tools include CSP [2], B [3], and Alloy [4]. However, these languages are not as widely used by engineers as less mathematical languages such as UML or SysML. Model transformation provides systematic mechanisms to bridge the gap between well-known system modelling languages, and languages well suited to formal analysis. We provide a speciﬁc application of model transformation to bridge the gap between UML modelling and formal modelling languages. There has been previous work on using model transformation to generate formal models, which we discuss in Section 5. This paper contributes a novel application of model transformation to support formal analysis, and does so M. Oriol and B. Meyer (Eds.): TOOLS EUROPE 2009, LNBIP 33, pp. 357–367, 2009. c Springer-Verlag Berlin Heidelberg 2009

358

H. Treharne et al.

in a novel way. The application of model transformation is novel because it simultaneously takes two types of UML diagrams as input, and produces formal models in an integrated language as output. The integrated language is a fusion of CSP and B. The context in which this transformation has been developed is also novel: it has been developed in the context of a safety-critical systems development project, where the model transformation that has been developed has to be shown to provide improvements over the traditional means by which formal models have been produced from UML models. Indeed, we are able to show improvements via use of state-of-the-art model transformation languages and tools and we discuss this in more detail in the paper. The model transformation framework that we use is part of Epsilon [5], which is a platform of integrated task-speciﬁc model management languages; we use the languages that focus on model-to-model (M2M) transformation (ETL) [6], and model-to-text (M2T) transformation (EGL) [7]. They both build on a common OCL-based model navigation and manipulation language (EOL). We will demonstrate how ETL can be used to support transformations between UML and CSP || B [8] models. To achieve this we deﬁne two metamodels; one for CSP and one for B. The reason for having two models, and not one, is that CSP and B are diﬀerent languages and we wanted to retain their separation at the meta-level. This will mean that the metamodels can be re-used in the future as a basis for other transformations. Separation of concerns is an underlying principle of the CSP || B modelling approach. A CSP || B model comprises a separate CSP speciﬁcation and a B speciﬁcation; together they describe the behavioural and data aspects of UML state machines and class diagrams. Their separation enables us to use the existing formal analysis tools of Fdr [9] and ProB [10] when analysing the CSP and B models respectively. We demonstrate how EGL is used to generate machine readable CSP and B scripts that can be executed in their respective analysis tools. Our previous work [11] has targeted transformations speciﬁcally from Executable UML (xUML) [12], which is currently our industrial collaborator’s favoured UML modelling language. The code generator we previously implemented was developed using the iUML tool-set [13], which uses a single, low-level language called ASL [14]. ASL is used to deﬁne all model-management tasks, including UML model actions. We deﬁned a suite of M2T transformations, and the output was CSP || B. In this paper, we produce a similar style of CSP || B using transformations in Epsilon (derived from our previous work) and we highlight that our new approach requires a signiﬁcantly reduced amount of code. The paper continues in Section 2 with an overview of the CSP || B approach; the transformations and metamodels are presented in Section 3. Our experiences of using Epsilon is summarised in Section 4 and we conclude with a comparison of related work and future directions.

2

CSP || B

CSP B is a formal methodology that combines the event-based CSP notation [2] with the state-based B notation [3] to aid the speciﬁcation of systems

Automatic Generation of Integrated Formal Models

359

containing both complex ﬂows of control and structured data. Thus, CSP || B is suited to specifying UML state machines and data-rich class diagrams. CSP process expressions can be constructed from a range of operators. Those used in this paper include external choice (2), local deﬁnitions (let . . . within, enabling the deﬁnition of processes of processes) and event sequencing, denoted preﬁx (→). The set of all events for a process is denoted its alphabet. B speciﬁcations are structured using machines. Each machine contains some variables describing the objects of interest, and operations to manipulate the state. The machine invariant declares state predicates, and speciﬁes what must be preserved by the execution of operations. Functions and relations are used to model complex state. A B operation takes the form PRE P THEN S END where P is a predicate and S represents the statements that update variables. In CSP B, the events of a CSP process trigger operation calls of a B machine, and we therefore refer to CSP processes as controllers. Structured events are used to pass values between a controller and a B machine, in both directions. For example, the event ev !x ?y, denotes a communication over the channel, ev , which outputs (!) x and binds y to an input value (?), and this event corresponds to a B operation ev that accepts x as input, and outputs y 1 . We combine collections of process/machine pairs using the architecture identiﬁed in [15], where collections of controllers represent the behaviour of instances of classes, and the machines record and update the instances’ attributes.

3

Generating CSP||B from UML

The generation of the formal model comprises two main phases, and has required the deﬁnition of new CSP and B metamodels. In the ﬁrst phase, we use ETL to perform an M2M transformation from the source UML model into one instance of both the CSP model and the B model. These newly created models provide the source models for an EGL M2T transformation, which traverses the meta-classes and prints them appropriately to both a CSP ﬁle and a B ﬁle. 3.1

Transformations to CSP

The abstract syntax for the CSP scripts is given in Fig. 1. As a basis for its development, we referred to the compact CSP metamodel by Bisztray et al. [16], who make use of it to verify behavioural equivalences between refactored UML systems comprising component, structure and activity diagrams. Indeed, it is possible to see similarities between the two metamodels, e.g., the Event class. There are however signiﬁcant diﬀerences. A notable diﬀerence is that they model CSP conforming to Hoare’s deﬁnition of the language [2], which states that every process P has its own, intrinsic alphabet, αP . Moreover, only a single parallel operator is deﬁned where P Q is a process comprising the synchronous execution of any event, ev ∈ αP ∩ αQ , and the concurrent execution of all other 1

It is not necessary for a corresponding CSP event/B operation pair to include parameters.

360

H. Treharne et al.

Fig. 1. CSP metamodel

events in P and Q . In terms of machine readable CSP, called CSP-M, which is the syntax required by Fdr, the authors model Hoare’s parallel using the interface parallel operator: P [| αP ∩ αQ |]Q . We have chosen to model directly a wide range of CSP-M notions not represented in [17], to allow for ﬂexibility in our current/future transformations. Therefore, we also support the deﬁnition of CSP alphabetised parallel, of the form P [αP || αQ ]Q , which also exhibits synchronous execution of any event, ev ∈ αP ∩ αQ , but which refuses the execution of events not speciﬁed in the alphabets. In fact, our experience is that alphabetised parallel improves the clarity of CSP-M, making it more simple for users to experiment and construct new CSP reﬁnements assertions. Another major diﬀerence is that we choose to represent the data that can reside in CSP, e.g., using the classes Datatype and Set. Thus, instead of having an Event class that represents all notions of events, our deﬁnition is more ﬁnegrained. We model all event parameters and their types, e.g., the event ev !a?b corresponds to an instance of Event for the Channel called ev , which has two EventParameters, a and b. Modelling this aspect is especially important for our use which requires communication of CSP events and parameters and corresponding B machine operations parameters.

Automatic Generation of Integrated Formal Models

Fig. 2. File class diagram

361

Fig. 3. File state chart

File SCTRL(self ) = let File Created ENTRY = File Created STATE File Created STATE = remove!self ? !File initialisedSignal → File Idle ENTRY File Idle ENTRY = File Idle STATE File Idle STATE = remove!self ? !File writeSignal → File Writing ENTRY 2 remove!self ? !File selfCheckSignal → File Idle ENTRY File Writing ENTRY = File Writing STATE File Writing STATE = . . . within File Created STATE Fig. 4. Generated CSP for the File state chart

Note that Fig. 1 shows only the main classes and association labels of the metamodel necessary for the purposes of this paper. There also exist classes that model all variations of the Choice operator (internal, external and asymmetric choice), namespaces, process interrupts, Skip, Stop, sequential composition, process assertions, renaming and the derivatives of Concurrent processes: interleaved/indexed and alphabetised/interface parallel. Figures 2 and 3 provide a source model for our transformations. The class diagram contains a File class with the attributes inUse and size, and the state machine represents the states of this ﬁle: Created, Idle and Writing. Fig. 4 presents part of the CSP script generated by the ETL and EGL transformation for the File class and state chart. The format of this CSP follows the style we used in our xUML-to-CSP || B code generator [11]. The key components of the script include deﬁnitions for modelling class instances, and channels that enable attribute manipulations/queries, operation calls, object creations/deletions and signal processing. The dynamic behaviour of each instance of each class is deﬁned through a repeatable set of process equations. The File SCTRL(self ) process represents the behaviour of the File state chart, where the self parameter is a File instance handle. This localised process (a let . . . within CSP construct) comprises a pair of process equations for each state. The ﬁrst process, e.g., File Idle ENTRY , models the state’s entry actions, none of which are deﬁned in our toy example. The second process captures the processing of signals that have arrived at a state, and its subsequent transition(s) to successor

362

H. Treharne et al.

states. For example, File Idle STATE includes the deterministic choice (2) of two events modelling the removal of two signals from the objects’ conceptual signal queue [12]. These events are followed by the entry state process of their respective successor states. Finally, observe that File SCTRL(self ) begins in the File Created STATE process equation, as speciﬁed by the terminal, ‘within’ clause; this information is derived from the initial state of the state chart. Listing 1 presents the core ETL rule used in our transformation workﬂow that produces CSP notation such as that given in Fig. 4. The StateMachine2CTRL Process rule is responsible for transforming a UML state machine into a localised (let . . . within) process.This primarily involves the creation of new instances of ProcessID, ProcessAssignment and ProcessParameterList. In lines 6 to 8, we link together the ProcessAssignment and ProcessID, and the ProcessID and ProcessParameterList. Line 9 sets the name of the process, e.g., File SCTRL, and lines 10 through to 13 add the controller process parameter ‘self’. We create a localised process (line 14), and on line 15, we iterate through each state of the state machine and apply the rule Sm2ProcPair. Each application of Sm2ProcPair returns a pair of ProcessAssignments, which are added to the localised process; these new instances represent the ENTRY and STATE processes described previously. Finally, we compute the initial state of the class diagram using the getStartState operation and we link the new localised process to the original ProcessAssignment, pa. 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16

17 18 19 20

rule StateMachine2CTRLProcess transform sm : StateMachine to p : CSP!ProcessID, pa : CSP!ProcessAssignment, ppl : CSP!ProcessParameterList { pa.procID := p; ppl.size := 1; p.paramList := ppl; p.name := sm.name+‘_SCTRL’; var procParam : new CSP!ProcessParameter; procParam.name := ‘self’; ppl.first := procParam; ppl.item.add(procParam); var root : new CSP!LocalisedProcess; for(state in StateMachine!State.all) { root.process := root.process + state.equivalents(‘Sm2ProcPair’). select(st | st.isKindOf(CSP!ProcessAssignment)); } root.first := getStartState(sm); pa.procExpr := root; } Listing 1. CSP transformation rule

Automatic Generation of Integrated Formal Models

363

Fig. 5. B metamodel

3.2

Transformations to B

The abstract syntax for the B we use is depicted in Fig. 5. The main classes deﬁning entities of a B machine are Machine, Operation, Variable and Expression. The Expression class is used as a basis from which speciﬁc B expressions are derived, such as Ifs, and Actions (e.g., assignments). To our knowledge, this metamodel is a novel contribution to the existing literature. Fig. 6 presents the B machine generated by the ETL and EGL transformation for the File class. We represent the set of all possible instance handles of the class by the B set, fileIH , and we deﬁne a subset of this, fileObj , to capture the ﬁle instances already created in the system. Functions are used to represent attributes, e.g., size ∈ fileObj → INTEGER, and we also generate simple operations to query or modify their values. Operations are also deﬁned to support the dynamic creation and deletion of objects, e.g., file create. Other user deﬁned operations of a class are transformed to skeleton B operations that specify the typing of input variables. We also create a single B machine to store the relationships of the class diagram; however, Fig. 5 does not include any. Listing 2 presents the core ETL rule used in our transformation workﬂow that produces B notation, such as that given in Fig. 6. The Class2Machine rule is responsible for transforming a UML class to a newly created B machine, Machine instance. First, a guard is speciﬁed to constrain the application of the rule to only non-UML (primitive) classes. The procedure sets the name of the machine to that of the source class, File, and creates for it a Set representing all instances of the class. The operation on line 7 getInstanceHandleSetName() is used to format the name of the set correctly; fileIH. Lines 9 to 11 create the variable representing the active objects in the system, where getActiveInstanceHandleSetName() is used to set its name to fileObj. A Predicate is then created to deﬁne the type of fileObj, on line 12. This predicate is set as a machine invariant, and we

364

H. Treharne et al.

MACHINE File SEES Bool TYPE SETS fileIH = { fifoIH0 , fifoIH1 } VARIABLES fileObj , inUse , size INVARIANT fileObj ⊆ fileIH ∧ inUse ∈ fileObj → BOOL size ∈ fileObj → INTEGER INITIALISATION fileObj := ∅ inUse := ∅ size := ∅

OPERATIONS file create ( ih ) = PRE ih ∈ fileIH ∧ ih ∈ llObj THEN fileObj := fileObj ∪ { ih } inUse ( ih ) := false END val ←− file get inUse ( ih ) = PRE ih ∈ llObj THEN val := inUse ( ih ) END file set inUse ( ih , val ) = PRE ih ∈ llObj ∧ val ∈ BOOL THEN inUse ( ih ) := val END /* Also defined: file delete, file get size, file set size */ END

; ; ;

Fig. 6. Generated B machine for the File class

also reference it from the variable to assert that this variable indeed has a type deﬁnition deﬁned. We call the attribute2variable operation so that we can identify for each attribute of the class (line 16) a new Variable and Predicate specifying its type. We also create a ‘get’ and ‘set’ operation for querying and manipulating the value of the attribute. 1 2 3 4 5 6 7 8 9 10 11 12 13

14 15 16 17 18 19 20 21 22 23

rule Class2Machine transform c : SourceUML!Class to mch : B!Machine { guard : ensureNonUmlClass(c) mch.name := c.name; var ihSet : new B!‘‘Set’’; ihSet.name := mch.name.getInstanceHandleSetName(); mch.sets.add(ihSet); var activeIHSet : new B!Variable; activeIHSet.name := mch.name.getActiveInstanceHandleSetName(); mch.variable.add(activeIHSet); var activeIHSet_type : new B!Predicate; activeIHSet_type.expression := activeIHSet.name+’ <: ’+ihSet.name; mch.invariant.add(activeIHSet_type); activeIHSet.type := activeIHSet_type; for(attr in c.ownedAttribute) { var var_preds := attribute2variable(c, attr); mch.variable.add(var_preds.at(0)); mch.invariant.add(var_preds.at(1)); mch.operation.add(attribute2getOperation(c, attr)); mch.operation.add(attribute2setOperation(c, attr)); } } Listing 2. Core B transformation rule

Automatic Generation of Integrated Formal Models

4

365

Experiences with iUML and Epsilon

In this section, we summarise brieﬂy our experiences of MDD using iUML, and subsequently migrating to a composition of M2M and M2T transformations using the Epsilon framework. Epsilon oﬀers speciﬁc languages, e.g., ETL and EGL, and syntax suitable for task-speciﬁc transformations. The result of these features is a reduction in the amount code that needs to be written to traverse the metamodels. For example, the declarative nature of an ETL rule that transforms a State to a ProcessAssignment, presented in Figure 1, was not as straightforward in iUML. In iUML, several statements were required in an ASL operation, corresponding to an ETL rule, just to identify all State instances and create a new ProcessAssignment, before any body of the rule is deﬁned. To give metrics, our Epsilon implementation of the code generator contains approximately 1600 lines of code, whereas, the iUML equivalent used approximately 5000 lines. During the development of the transformations, e.g., when adding new CSP and B operators in the metamodel, we found that it was incremental to add new rules in Epsilon, and these often had little impact on existing rules. Conversely, augmenting our iUML transformation with new operations required further changes to existing operations, e.g., to ensure they are invoked.

5

Related Work

There already exists a signiﬁcant body of work relating UML and formal methods. As previously introduced in Section 3.1, the work of Bisztray et al. [17] is particularly relevant since they have developed a metamodel for a subset of CSP, which we used as a basis for the development of our own CSP metamodel. Snook and Butler’s UML to B transformation tool, UML-B, is relevant since their target language is also B. However, they use B not only to represent system data, but also to provide control ﬂow via operation guards, instead of CSP. Their underlying metamodel is a combination of UML and Event-B concepts which limits re-usability. The emphasis of UML-B is to provide a graphical interface to Event-B rather than analyzing the integrity of a UML model. Another work that is closely related is the UML2Alloy tool [18], which translates a subset of UML with OCL constraints to an Alloy speciﬁcation, using transformations deﬁned in SiTra [19]. The generated scripts are subject to an automatic analysis using the Alloy Analyzer [4], to provide validation of the speciﬁed OCL constraints. Our approach diﬀers since we support the dynamic behaviour of objects captured within state charts, in addition to class diagrams.

6

Conclusion and Future Work

In this paper we have presented our approach that uses M2M and M2T transformations, deﬁned using the Epsilon framework, to automate the generation of a CSP B speciﬁcation for a UML system comprising a class diagram and a

366

H. Treharne et al.

number of state charts. A CSP metamodel has been described, capturing some of the CSP features modelled originally in [17], and we have added support for representing a large range of additional, commonly used CSP concepts. Our metamodel for the B notation has also been presented. The core ETL transformation rules involved in the generation of a CSP || B system have been described, and we have given an example of the kinds of output that is produced by our transformations. Subsequently, we noted that our experiences showed that there is a signiﬁcant saving in code when using dedicated model-management languages. The state machines in our iUML transformations are very expressive since states can contain actions described using ASL. Nonetheless, iUML is not widely used. We intend to extend our Epsilon code generator to transform OCL constraints so that more complex state machines can be modelled. This will enable us to investigate whether a CSP || B analysis can be applied more generally. Acknowledgments. The authors are grateful to Steve Schneider, James Sharp and Neil Evans for discussions on this work.

References 1. Jouault, F., Allilaire, F., B´ezivin, J., Kurtev, I.: Atl: A model transformation tool. Sci. Comput. Program. 72(1-2), 31–39 (2008) 2. Hoare, C.A.R.: Communicating Sequential Processes. Prentice-Hall, Englewood Cliﬀs (1985) 3. Abrial, J.R.: The B Book: Assigning programs to meanings. Cambridge University Press, New York (1996) 4. Software Design Group at MIT: Alloy Analyser 4.1.2 (2008) 5. Kolovos, D.S., Paige, R.F.: The Epsilon model management platform (2009) 6. Kolovos, D.S., Paige, R.F., Polack, F.A.C.: The Epsilon Transformation Language. In: Vallecillo, A., Gray, J., Pierantonio, A. (eds.) ICMT 2008. LNCS, vol. 5063, pp. 46–60. Springer, Heidelberg (2008) 7. Rose, L.M., Paige, R.F., Kolovos, D.S., Polack, F.: The Epsilon Generation Language. In: Schieferdecker, I., Hartman, A. (eds.) ECMDA-FA 2008. LNCS, vol. 5095, pp. 1–16. Springer, Heidelberg (2008) 8. Schneider, S., Treharne, H.: CSP theorems for communicating B machines. Formal Asp. Comput. 17(4), 390–422 (2005) 9. Formal Systems Oxford: FDR 2.83 manual (2007) 10. Leuschel, M., Butler, M.: ProB: A Model Checker for B. In: ESA 1994. LNCS, pp. 855–874. Springer, Heidelberg (2003) 11. Turner, E., Treharne, H., Evans, N., Schneider, S.: Automatic generation of CSP || B skeletons from xUML models. In: Fitzgerald, J.S., Haxthausen, A.E., Yenigun, H. (eds.) ICTAC 2008. LNCS, vol. 5160, pp. 364–379. Springer, Heidelberg (2008) 12. Mellor, S.J., Balcer, M.J.: Executable UML, A Foundation for Model-Driven Architecture. Addison-Wesley, Reading (2002) 13. Carter, K.: Intelligent UML 2.30 modeller and simulator, http://www.kc.com 14. Wilkie, I., King, A., Clarke, M., Weaver, C., Raistrick, C., Francis, P.: UML ASL Reference Guide (ASL language level 2.5). Kennedy Carter Ltd. (2003)

Automatic Generation of Integrated Formal Models

367

15. Evans, N., Treharne, H., Laleau, R., Frappier, M.: Applying CSP B to information systems. Software and System Modeling 7(1), 85–102 (2008) 16. Bisztray, D., Heckel, R., Ehrig, H.: Veriﬁcation of architectural refactoring rules. Technical report, University of Leicester (2008) 17. Bisztray, D., Heckel, R., Ehrig, H.: Veriﬁcation of architectural refactorings by rule extraction. In: Fiadeiro, J.L., Inverardi, P. (eds.) FASE 2008. LNCS, vol. 4961, pp. 347–361. Springer, Heidelberg (2008) 18. Anastasakis, K., Bordbar, B., Georg, G., Ray, I.: UML2Alloy: A challenging model transformation. In: Engels, G., Opdyke, B., Schmidt, D.C., Weil, F. (eds.) MODELS 2007. LNCS, vol. 4735, pp. 436–450. Springer, Heidelberg (2007) 19. Akehurst, D.H., Bordbar, B., Evans, M.J., Howells, W.G.J., McDonald-Maier, K.D.: SiTra: Simple Transformations in Java. In: Nierstrasz, O., Whittle, J., Harel, D., Reggio, G. (eds.) MoDELS 2006. LNCS, vol. 4199, pp. 351–364. Springer, Heidelberg (2006)

Author Index

Apel, Sven 175 Apvrille, Ludovic

308

Batory, Don 175 Bebenita, Michael 77 Berbers, Yolande 116 Bergel, Alexandre 252 Blair, Lynne 18 Boronat, Artur 96 Brosch, Petra 1 Bruni, Camillo 328

Knorreck, Daniel 308 Kolovos, Dimitrios S. 357 Kuhlemann, Martin 175 Lamo, Yngve 37 Lienhard, Adrian 272 Meseguer, Jos´e 96 Meyer, Bertrand 195 Muller, Alexis 215 M¨ uller, Peter 195

Calcagno, Cristiano 195 Camilleri, Andrew 18 Caron, Olivier 215 Carr´e, Bernard 215 Cassou, Damien 252 Chang, Mason 77 Conejero, Jos´e M. 136 Coulson, Geoﬀrey 18

Nierstrasz, Oscar 272 Nordio, Martin 195

D’Hondt, Theo 231 De Meuter, Wolfgang 231 Deridder, Dirk 157 Ducasse, St´ephane 252

Rossini, Alessandro Rutle, Adrian 37

Fierz, Julien 272 Figueiredo, Eduardo Franz, Michael 77

136

Gal, Andreas 77 Garcia, Alessandro 136 Gonzalez Boix, Elisa 231 Hanenberg, Stefan 57 Hern´ andez, Juan 136 Hovsepyan, Aram 116 Jim´enez, Camilo 157 Joosen, Wouter 116 Jurado, Elena 136 Kappel, Gerti 1 Kargl, Horst 1 K¨ astner, Christian Klose, Karl 289

175

Ostermann, Klaus

289

Pacalet, Renaud 308 Paige, Richard F. 357 Pollet, Damien 252 37

Sadilek, Daniel A. 348 S´ anchez, Mario 157 Seidl, Martina 1 Stein, Dominik 57 Stricker, Vanessa 57 Treharne, Helen 357 Trujillo, Salvador 175 Turner, Edward 357 Vallejos, Jorge 231 Van Baelen, Stefan 116 Van Cutsem, Tom 231 Vanwormhoudt, Gilles 215 Verwaest, Toon 328 Villalobos, Jorge 157 Vitek, Jan 17 Wachsmuth, Guido 348 Wimmer, Manuel 1 Wolter, Uwe 37

Objects, Components, Models and Patterns: 46th International Conference, TOOLS EUROPE 2008, Zurich, Switzerland, June 30-July 4, 2008, Proceedings (Lecture Notes in Business Information Processing)

Objects, Components, Models, Patterns - TOOLS 2011

Business Information Systems: 12th International Conference, BIS 2009, Poznan, Poland, April 27-29, 2009, Proceedings (Lecture Notes in Business Information Processing)

Objects, Models, Components, Patterns: 48th International Conference, TOOLS 2010, Málaga, Spain, June 28 - July 2, 2010, Proceedings (Lecture Notes in ... Programming and Software Engineering)

Enterprise Information Systems: 11th International Conference, ICEIS 2009, Milan, Italy, May 6-10, 2009, Proceedings (Lecture Notes in Business Information Processing)

Software Business: First International Conference, ICSOB 2010, Jyväskylä, Finland, June 21-23, 2010, Proceedings (Lecture Notes in Business Information Processing)

Enterprise, Business-Process and Information Systems Modeling: 10th International Workshop, BPMDS 2009, and 14th International Conference, EMMSAD 2009, ... Notes in Business Information Processing)

Logic, Language, Information and Computation: 16th International Workshop, WoLLIC 2009, Tokyo, Japan, June 21-24, 2009, Proceedings (Lecture Notes in ... Lecture Notes in Artificial Intelligence)

E-Technologies: Innovation in an Open World: 4th International Conference, MCETECH 2009, Ottawa, Canada, May 4-6, 2009, Proceedings (Lecture Notes in Business Information Processing)

Proceedings of International Science Education Conference 2009

Web Information Systems and Technologies: 5th International Conference, WEBIST 2009, Lisbon, Portugal, March 23-26, 2009, Revised Selected Papers (Lecture Notes in Business Information Processing)

Hybrid Artificial Intelligence Systems: 4th International Conference, HAIS 2009, Salamanca, Spain, June 10-12, 2009, Proceedings (Lecture Notes in ... Lecture Notes in Artificial Intelligence)

Reliable Software Technologies - Ada-Europe 2009: 14th Ada-Europe International Conference, Brest, France, June 8-12, 2009, Proceedings (Lecture Notes ... Programming and Software Engineering)

Harvard Business Review - June 2009

Hybrid Artificial Intelligence Systems: 4th International Conference, HAIS 2009, Salamanca, Spain, June 10-12, 2009, Proceedings (Lecture Notes in ... Lecture Notes in Artificial Intelligence)

Harvard Business Review - June 2009

Advances in Nonlinear Speech Processing: International Conference on Nonlinear Speech Processing, NOLISP 2009, Vic, Spain, June 25-27, 2009, Revised ... Lecture Notes in Artificial Intelligence)

Advances in Nonlinear Speech Processing: International Conference on Nonlinear Speech Processing, NOLISP 2009, Vic, Spain, June 25-27, 2009, Revised Selected ... Lecture Notes in Artificial Intelligence)

Entertainment Computing -- ICEC 2009: 8th International Conference, ICEC 2009, Paris, France, September 3-5, 2009, Proceedings (Lecture Notes in ... Applications, incl. Internet Web, and HCI)

Advances in Enterprise Engineering III: 5th International Workshop, CIAO! 2009, and 5th International Workshop, EOMAS 2009, held at CAiSE 2009, Amsterdam, ... Notes in Business Information Processing)

Text, Speech and Dialogue: 12th International Conference, TSD 2009, Pilsen, Czech Republic, September 13-17, 2009. Proceedings (Lecture Notes in ... Lecture Notes in Artificial Intelligence)

Advanced Data Mining and Applications: 5th International Conference, ADMA 2009, Chengdu, China, August 17-19, 2009, Proceedings (Lecture Notes in ... Lecture Notes in Artificial Intelligence)

Pattern Recognition in Bioinformatics: 4th IAPR International Conference, PRIB 2009, Sheffield, UK, September 7-9, 2009, Proceedings (Lecture Notes in ... Science Lecture Notes in Bioinformatics)

Intelligent Robotics and Applications: Second International Conference, ICIRA 2009, Singapore, December 16-18, 2009, Proceedings (Lecture Notes in ... Lecture Notes in Artificial Intelligence)

Scale Space and Variational Methods in Computer Vision: Second International Conference, SSVM 2009, Voss, Norway, June 1-5, 2009. Proceedings (Lecture Notes in Computer Science)

Perspectives in Business Informatics Research. BIR 2011 Proceedings (Lecture Notes in Business Information Processing)

Theory and Practice of Model Transformations: Second International Conference, ICMT 2009, Zürich, Switzerland, June 29-30, 2009, Proceedings (Lecture ... Programming and Software Engineering)

Scale Space and Variational Methods in Computer Vision: Second International Conference, SSVM 2009, Voss, Norway, June 1-5, 2009. Proceedings (Lecture Notes in Computer Science)

Interoperating Geographic Information Systems: Second International Conference, INTEROP'99, Zurich, Switzerland, March 10-12, 1999 Proceedings

Intelligent Virtual Agents: 9th International Conference, IVA 2009 Amsterdam, The Netherlands, September 14-16, 2009 Proceedings (Lecture Notes in ... Lecture Notes in Artificial Intelligence)

Objects, Components, Models and Patterns: 47th International Conference, TOOLS EUROPE 2009, Zurich, Switzerland, June 29-July 3, 2009, Proceedings (Lecture Notes in Business Information Processing)

Objects, Components, Models and Patterns: 46th International Conference, TOOLS EUROPE 2008, Zurich, Switzerland, June 30-July 4, 2008, Proceedings (Lecture Notes in Business Information Processing)

Objects, Components, Models, Patterns - TOOLS 2011

Business Information Systems: 12th International Conference, BIS 2009, Poznan, Poland, April 27-29, 2009, Proceedings (Lecture Notes in Business Information Processing)

Objects, Models, Components, Patterns: 48th International Conference, TOOLS 2010, Málaga, Spain, June 28 - July 2, 2010, Proceedings (Lecture Notes in ... Programming and Software Engineering)

Enterprise Information Systems: 11th International Conference, ICEIS 2009, Milan, Italy, May 6-10, 2009, Proceedings (Lecture Notes in Business Information Processing)

Software Business: First International Conference, ICSOB 2010, Jyväskylä, Finland, June 21-23, 2010, Proceedings (Lecture Notes in Business Information Processing)

Enterprise, Business-Process and Information Systems Modeling: 10th International Workshop, BPMDS 2009, and 14th International Conference, EMMSAD 2009, ... Notes in Business Information Processing)

Logic, Language, Information and Computation: 16th International Workshop, WoLLIC 2009, Tokyo, Japan, June 21-24, 2009, Proceedings (Lecture Notes in ... Lecture Notes in Artificial Intelligence)

E-Technologies: Innovation in an Open World: 4th International Conference, MCETECH 2009, Ottawa, Canada, May 4-6, 2009, Proceedings (Lecture Notes in Business Information Processing)

Proceedings of International Science Education Conference 2009

Web Information Systems and Technologies: 5th International Conference, WEBIST 2009, Lisbon, Portugal, March 23-26, 2009, Revised Selected Papers (Lecture Notes in Business Information Processing)

Hybrid Artificial Intelligence Systems: 4th International Conference, HAIS 2009, Salamanca, Spain, June 10-12, 2009, Proceedings (Lecture Notes in ... Lecture Notes in Artificial Intelligence)

Reliable Software Technologies - Ada-Europe 2009: 14th Ada-Europe International Conference, Brest, France, June 8-12, 2009, Proceedings (Lecture Notes ... Programming and Software Engineering)

Harvard Business Review - June 2009

Hybrid Artificial Intelligence Systems: 4th International Conference, HAIS 2009, Salamanca, Spain, June 10-12, 2009, Proceedings (Lecture Notes in ... Lecture Notes in Artificial Intelligence)

Harvard Business Review - June 2009

Advances in Nonlinear Speech Processing: International Conference on Nonlinear Speech Processing, NOLISP 2009, Vic, Spain, June 25-27, 2009, Revised ... Lecture Notes in Artificial Intelligence)

Advances in Nonlinear Speech Processing: International Conference on Nonlinear Speech Processing, NOLISP 2009, Vic, Spain, June 25-27, 2009, Revised Selected ... Lecture Notes in Artificial Intelligence)

Entertainment Computing -- ICEC 2009: 8th International Conference, ICEC 2009, Paris, France, September 3-5, 2009, Proceedings (Lecture Notes in ... Applications, incl. Internet Web, and HCI)

Advances in Enterprise Engineering III: 5th International Workshop, CIAO! 2009, and 5th International Workshop, EOMAS 2009, held at CAiSE 2009, Amsterdam, ... Notes in Business Information Processing)

Text, Speech and Dialogue: 12th International Conference, TSD 2009, Pilsen, Czech Republic, September 13-17, 2009. Proceedings (Lecture Notes in ... Lecture Notes in Artificial Intelligence)

Advanced Data Mining and Applications: 5th International Conference, ADMA 2009, Chengdu, China, August 17-19, 2009, Proceedings (Lecture Notes in ... Lecture Notes in Artificial Intelligence)

Pattern Recognition in Bioinformatics: 4th IAPR International Conference, PRIB 2009, Sheffield, UK, September 7-9, 2009, Proceedings (Lecture Notes in ... Science Lecture Notes in Bioinformatics)

Intelligent Robotics and Applications: Second International Conference, ICIRA 2009, Singapore, December 16-18, 2009, Proceedings (Lecture Notes in ... Lecture Notes in Artificial Intelligence)

Scale Space and Variational Methods in Computer Vision: Second International Conference, SSVM 2009, Voss, Norway, June 1-5, 2009. Proceedings (Lecture Notes in Computer Science)

Perspectives in Business Informatics Research. BIR 2011 Proceedings (Lecture Notes in Business Information Processing)

Theory and Practice of Model Transformations: Second International Conference, ICMT 2009, Zürich, Switzerland, June 29-30, 2009, Proceedings (Lecture ... Programming and Software Engineering)

Scale Space and Variational Methods in Computer Vision: Second International Conference, SSVM 2009, Voss, Norway, June 1-5, 2009. Proceedings (Lecture Notes in Computer Science)

Interoperating Geographic Information Systems: Second International Conference, INTEROP'99, Zurich, Switzerland, March 10-12, 1999 Proceedings

Intelligent Virtual Agents: 9th International Conference, IVA 2009 Amsterdam, The Netherlands, September 14-16, 2009 Proceedings (Lecture Notes in ... Lecture Notes in Artificial Intelligence)

Recommend Documents