Lecture Notes in Computer Science Commenced Publication in 1973 Founding and Former Series Editors: Gerhard Goos, Juris Hartmanis, and Jan van Leeuwen
Editorial Board David Hutchison Lancaster University, UK Takeo Kanade Carnegie Mellon University, Pittsburgh, PA, USA Josef Kittler University of Surrey, Guildford, UK Jon M. Kleinberg Cornell University, Ithaca, NY, USA Alfred Kobsa University of California, Irvine, CA, USA Friedemann Mattern ETH Zurich, Switzerland John C. Mitchell Stanford University, CA, USA Moni Naor Weizmann Institute of Science, Rehovot, Israel Oscar Nierstrasz University of Bern, Switzerland C. Pandu Rangan Indian Institute of Technology, Madras, India Bernhard Steffen TU Dortmund University, Germany Madhu Sudan Microsoft Research, Cambridge, MA, USA Demetri Terzopoulos University of California, Los Angeles, CA, USA Doug Tygar University of California, Berkeley, CA, USA Gerhard Weikum Max Planck Institute for Informatics, Saarbruecken, Germany
6520
Roland Kaschek Lois Delcambre (Eds.)
The Evolution of Conceptual Modeling From a Historical Perspective towards the Future of Conceptual Modeling
13
Volume Editors Roland Kaschek Gymnasium Gerresheim Am Poth 60, 40625 Düsseldorf, Germany E-mail:
[email protected] Lois Delcambre Portland State University Maseeh College of Engineering and Computer Science Computer Science Department P.O. Box 751, Portland, OR, 97207-0751, USA E-mail:
[email protected]
Library of Congress Control Number: 2010941310 CR Subject Classification (1998): H.2-4, I.2.3-4, F.4.1, H.1, J.1, D.2, C.2 LNCS Sublibrary: SL 3 – Information Systems and Application, incl. Internet/Web and HCI ISSN ISBN-10 ISBN-13
0302-9743 3-642-17504-X Springer Berlin Heidelberg New York 978-3-642-17504-6 Springer Berlin Heidelberg New York
This work is subject to copyright. All rights are reserved, whether the whole or part of the material is concerned, specifically the rights of translation, reprinting, re-use of illustrations, recitation, broadcasting, reproduction on microfilms or in any other way, and storage in data banks. Duplication of this publication or parts thereof is permitted only under the provisions of the German Copyright Law of September 9, 1965, in its current version, and permission for use must always be obtained from Springer. Violations are liable to prosecution under the German Copyright Law. springer.com © Springer-Verlag Berlin Heidelberg 2011 Printed in Germany Typesetting: Camera-ready by author, data conversion by Scientific Publishing Services, Chennai, India Printed on acid-free paper 06/3180
Preface
The origin of this volume is the Dagstuhl Seminar on The Evolution of Conceptual Modeling (Schloss Dagstuhl, April 2008) organized by Roland Kaschek with Lois Delcambre and Heinrich C. Mayr as co-organizers. Their motivation for organizing the seminar was that they felt the field of conceptual modeling, though clearly very important for computing, is not very well organized, developed, or recognized. Although a number of researchers, instructors and teachers have a serious interest in conceptual modeling and often include conceptual modeling topics in a wide range of courses, in a typical curriculum, conceptual modeling may only be covered in a brief or peripheral manner, e.g., as part of another course. The objectives for the seminar were: (1) to look into conceptual modeling from a historical perspective with a view toward the future of conceptual modeling and (2) to achieve a better understanding of conceptual modeling issues in several different domains of discourse using a perspective that goes beyond individual (modeling) projects. Note that participation in a Dagstuhl Seminar is by invitation only and permission to conduct a Dagstuhl Seminar is granted by a scientific advisory board with the University of Saarbr¨ ucken, Germany, based on the seminar proposal as well as the list of invitees. The seminar, attended by 33 experts from the field of conceptual modeling, consisted primarily of the following presentations:1 • Valeria De Antonellis: Conceptual Modeling for Service-Oriented, Collaborative Systems • Karen C. Davis: Data Warehousing Modeling and Its Evolution • Lois M. L. Delcambre: Multiple Conceptual Models in Superimposed Information Systems • David W. Embley: Turning the Web into a Web of Knowledge • Nicola Guarino: Ontological Foundations of Conceptual Modeling • Sven Hartmann: Conceptual Modeling and Natural Language Grammar • Brian Henderson-Sellers: Meta-modeling • Wolfgang Hesse: Evolution in Software Engineering from Using Models to Ontologies • Dietmar Jannach: Error Recovery in Business Processes • Klaus P. Jantke: Conceptual Modeling for Computer Games • Roland Kaschek: Sociological Turn in Conceptual Modeling • Christian Kop: Templates and Glossaries in Conceptual Modeling • Gunther Kreuzberger: Entertainment Engineering for Digital Games 1
Please note that some of these speakers were presenting cooperatively produced work. We were not able to invite all authors to attend the seminar.
VI
Preface
• Heinrich C. Mayr: Extreme Non-programming • Oscar Pastor: Conceptual Modeling Techniques Used in Human Genome Engineering • Klaus-Dieter Schewe: Specifying Data-Centric Services with Abstract State Machines • Michael Schrefl: Ontological Multi-level Modeling • Vladimir A. Shekhovtsov: Using Simulation for Quality Requirements Elicitation • Stefano Spaccapietra: Modeling Scientific Hypotheses • Markus Stumptner: Modeling Web Service Composition • Yuzuru Tanaka: A Memetics View on Web Evolution • Bernhard Thalheim: A Token Model of Process Semantics • Susan D. Urban: A Historical Account of Semantic and Current Data Models • Gottfried Vossen and Gunnar Thies: Issues of Conceptual Modeling for Web 2.0 • Tatyana Welzer: Evaluation of Conceptual Modeling • Roel Wieringa: Conceptual Modeling of Social Domains as Opposed to Physical Domains Near the end of the seminar, the participants decided to produce this edited volume, based in large part on presentations from the seminar. With this volume we wish to contribute to increasing the role of conceptual modeling as a discipline in research, teaching, and practice. Our process was as follows. The participants2 in the seminar were invited to contribute a chapter to this volume. In addition to the quality assurance provided by the Dagstuhl process (resulting in the initial invitation of experts), the submitted papers were carefully reviewed in a peer review process, in the first round. In the second round, we, the editors, carefully reviewed all resubmitted papers. We are very pleased to present 14 chapters in this volume, offering significant coverage of the topics from the seminar. The book is organized into three sections. The first, “The Evolution of Conceptual Modeling Techniques,” includes six papers, namely: a discussion of real-world semantics of conceptual models (Roel Weiringa), a survey of the use of templates in domain modeling (Christian Kop and Heinrich C. Mayr), an analysis of the use of integrity constraints in conceptual database models (Bernhard Thalheim), a survey of methods for modeling abstraction at multiple levels (Bernd Neumayr, Michael Schrefl, and Bernhard Thalheim), a discussion of multi-level conceptual models (Brian Henderson-Sellers), and an account of the evolution of modeling software quality in conceptual models (Vladimir A. Shekhovtsov).
2
In a few cases, colleagues who were invited to participate in the seminar but were not able to attend were invited to contribute chapters to this volume. Also, based on their role as editors, Roland Kaschek and Lois M. L. Delcambre decided not to submit chapters to be considered for the book.
Preface
VII
The second section,“The Extension of Conceptual Modeling” includes the following papers: an account of how Web pages can be (nearly automatically) processed to produce a “Web of knowledge” (David W. Embley, Stephen W. Liddle, and Cui Tao), a description of a technique to produce “semantic overlays” in a peer-to-peer system (Devis Bianchini, Valeria De Antonellis, and Michele Melchiori), a survey and new proposal for dynamic Web service composition using conceptual modeling (Georg Grossmann, Rajesh Thiagarajan, Michael Schrefl, and Markus Stumptner), the presentation of a new approach to dynamic constraint checking, exception handling, and recovery in contemporary distributed applications using rules (Susan D. Urban, Le Gao, Rajiv Shrestha, and Andrew Courter), a discussion of past, current, and future approaches to exception handling in Web service processes (Dietmar Jannach, and Alexander Gut), and the presentation of a framework for model-based tool integration based on conceptual modeling techniques (Gerti Kappel, Manuel Wimmer, Werner Retschitzegger, and Wieland Schwinger). The third and final section of this volume, “New Directions for Conceptual Modeling,” includes two papers: the presentation of a new conceptual data model for scientific models and hypotheses (Fabio Porto and Stefano Spaccapietra) and the description and suggestions for further enhancement of the application of conceptual modeling techniques, namely, model-based software engineering, to new applications – with a particular focus on the interpretation of the human genome (Oscar Pastor, Ana M. Levin, Matilde Celma, Juan Carlos Casamayor, Aremy Virrueta, and Luis E. Eraso).
October 2010
Lois M.L. Delcambre Roland Kaschek
Introduction Issues of a Conceptual Modeling Agenda3
This introduction is intended to present to this volume’s audience some of the reasoning that led to organizing the Dagstuhl seminar this volume originates from. It aims at providing general insights into the field of conceptual modeling that are not necessarily discussed in any of this volume’s chapters but nevertheless aid in conceptualizing the inner structure and coherence of the field. The evolution of mankind can be considered as the evolution of how human beings know, what they know, what they use their knowledge for, and the knowledge media they use. Conceptual modeling is a rather recent approach to creating knowledge. It has emerged in response to the computer revolution which started in the middle of the twentieth century. Computers in the meantime have turned out to be a major new knowledge media. Conceptual modeling is an answer to the difficulties experienced throughout the development of computer applications and aims at creating an effective, reasonably priced, and sharable knowledge about using computers in business. It turns out, however, that conceptual modeling has the potential to exceed the boundaries of business and computer use. Humans share knowledge by developing a culture, i.e., a system of codified behaviors such that knowledge can be constituted as a system of meaningful differences. Knowledge can then be structured on a micro-level according to logical schemes such as “A is true” and “if A is true then B is true” and the like. On higher levels, humans structure knowledge into models and more or less basic and more or less confirmed theories. We do so because that way checking knowledge is reduced to checking the logic it arises from, the premises of that logic, and intended use the knowledge. Structuring knowledge, moreover, simplifies the process of knowledge acquisition and sharing as the knowledge may be broken down into small, manageable pieces. The knowledge acquired and maintained that way is not necessarily the only kind of knowledge that needs to be or can be considered. Other kinds of knowledge may be addressed by what we call intuition or revelation. The mechanisms or techniques to share them may be different from the mechanisms that play a role for modeling. Modeling is the use of a substitute rather than the original, i.e., the entity one is actually interested in. Investigating a substitute rather than an original may be a good thing to do because the original is inaccessible, unavailable or too expensive for study, or nonexistent. The structure of the modeling situation is peculiar because it is often difficult to establish the model’s suitability for the intended purpose. In particular if the original is natural and the model is an
3
I thank Lois Delcambre for helping me to improve the readability of this text.
X
Preface
artifact then an analytical justification of the model’s suitability might be out of reach. Models may even be used if an analytical justification is known to be nonexistent. What is needed is the empirical evidence (or in some cases only the prospect of it) of the model’s suitability. In monetary terms, the justification of a model’s suitability may be expressed as: using the model is cheaper than not using it. It goes without saying that the related cost estimations are by no means a simple task. Since modeling is the use of a substitute rather than the original it creates two classes of knowledge: generic and case-specific. The generic knowledge is encoded in models and the case-specific knowledge is used for deciding which model to use, when and how. Throughout the young history of computing the importance of models, their creation and use has continuously increased. This is why in this book we discuss a number of more or less fundamental aspects of models and modeling. In this book, however, we are not concerned with all kinds of modeling. Rather, we ignore bodily and physical modeling. We focus on what has come to be known as conceptual modeling. Bodily models are material objects used to study aspects of the original. For example, bodily models have been used intensively in the industry constructing ships, trains, cars, aircraft, or spacecraft. This kind of modeling also has a long history in architecture. For bodily models, it is often important that they are significantly smaller than their original, much easier to deal with, much cheaper to produce, retire or similar. On the other hand, physical models are models that are encoded in a language with key terms that refer to characteristics of a given computer or computational device such as a PDP/11, an IBM 360, 390, a Wintel PC, a Mac or similar. The difference between physical and conceptual modeling is the language used for encoding the model. While in physical modeling a low-level computer jargon is used, in conceptual modeling a high-level jargon is used. That high-level jargon tends to focus on the discourse domain to which the problem to be solved with a computer application belongs rather than to the computer itself. Conceptual modeling thus brings about a situation in which humans can use computers as a tool in problem-solving language games4 without having to understand computers. Usually one can rely on IT professionals to keep the computers functioning reliably and with satisfactory performance. Human knowledge is commonly classified as being either procedural (“know how”) or declarative (“know what”). As is well known, the distinction between these two kinds of knowledge is a fundamental one. For example, the largest part of any human individual’s knowledge is procedural, and acquiring proce-
4
With the use of Wittgenstein’s term, language game, I want to recall two things: First, when people say something, what they mean with their utterance usually strongly depends on what they do while talking, i.e., the context of the actual utterance. Second, when an adult talks to someone else they are usually not entirely free to say whatever they might want. Rather, there are types of situations in which certain utterances are expected of them and others are considered as inappropriate. To me this implies that in many situations a pattern of codified behavior applies that at least partially gives meaning to what people say.
Preface
XI
dural knowledge on average takes much less time than acquiring declarative knowledge. Conceptual modeling links declarative knowledge and procedural knowledge. From a business point of view, conceptual modeling thus is a key activity because it enables working out the formalized declarative knowledge that is required for implementing the computer applications that are needed to back up the current or intended business practices. Obviously the link between computer applications and business practices can also be employed the opposite way: technology can be used to enable new business practices. Today’s computing landscape is very different from the computing landscape of the 1940s. Back then, mathematicians, physicists, and engineers were the most prominent computer users. Moreover, the relative importance that is attributed to conceptual modeling nowadays was not attributed to it at the begining of the computing age. The metaphor of “evolution” is applicable to computing as a whole and to software development in particular because a trend can be perceived, a trend toward more abstract and hardware-independent descriptions of computational processes and structures. That trend, however, is not the result of an individual agent’s decision. Rather it results from all those forces impacting the whole area of computing and in particular from its status as a major new branch of business. In the middle of the twentieth century when modern computers and computer science entered the stage, computers actually were already quite old. Charles Babbage had designed mechanical computers between 1840 and 1850. In doing that he extended the work of the French loom maker Joseph-Marie Jacquard, who in 1805 presented a program-controlled loom. Other sources of mechanical computing seem to date back to much earlier in history as is evidenced by the early European mechanical calculators such as the 1623 calculator built by Wilhelm Schickard and the Antikythera mechanism, the construction of which has been dated back to about 100 B.C. Construction and practical use of analog computers is commonly dated back much further in history. Examples of such early computers are the Goseck observatory (about 5000 B.C.), Stonehenge (about 3000 B.C.) and the Nebra Sky Disk (about 2000 B.C.). While in the nineteenth century the driving forces might not have been sufficient for triggering a largescale interest in computers and computing, the situation was quite different in the middle of the twentieth century. The war on Nazi Germany, the unfolding cold war, and the capability to make money using computers changed the situation. At the end of the 1970s the observation was made that in software engineering, database engineering, and artificial intelligence, differences in the development of computer applications were beginning to show that were less a consequence of the subject than of the arbitrary differences between these fields. It was then natural to try to overcome aspects of overspecialization by focusing on what was common to these sub-disciplines of computing, i.e., conceptual modeling. More recently further stimuli of conceptual modeling have started impacting the field: First, successful software systems have achieved a lifetime beyond the lifetime of their hardware or software platform and thus require to have
XII
Preface
specifications independent of these platforms. Second, the variety of problems that can be solved without paying attention to characteristics of the basic hardware and software has increased significantly and so high-level specifications make more sense. Third, new branches of computing have emerged such as gaming, social networks, simulators, or Web systems. It is desirable for these new branches to exploit application specification expertise from the more traditional branches of computing. From an abstract point of view, during conceptual modeling such as ER, UML, system, Petri net, state chart modeling or similar, a network of concepts is worked out. The concepts in this network mutually define each other. This means that the related concepts together create a language that constitutes a domain of discourse and that within that discourse each of the concepts has a unique role. In this way conceptual modeling is very similar to mathematics. Three interesting differences between the two, however, should be stated. First, in mathematics, there is a much stricter adherence to a given standard modeling language (i.e., the Zermelo-Fraenkel axiomatic system explicating the set concept) than in conceptual modeling, where we find in use many more or less different basic modeling languages. Second, in conceptual modeling, the precise description of computational processes and data structures is enforced by the need to establish program control of the computer, i.e., the computational agent. In mathematics, the computational agent was initially and still is a human individual. Programmed control of that agent has not yet achieved a mature state. In computing, moreover, some of the computational processes and structures have turned out to be useful on levels abstracting from the hardware that executes programs. Third, in mathematics, the basic modeling language is formally defined (by the axiomatic system). In conceptual modeling, however, this is not the case. In computing, formalization is important but is often considered later, namely, when specified solutions need to be implemented on a computer. It should be noted finally that the idea of describing “real-world” domains as nets of concepts was put forward as early as 1928 by Rudolf Carnap in his book Der logische Aufbau der Welt and thus has a relevant prehistory outside computing. Probably the most important point about conceptual models is that they are utterances specifically tailored for a particular audience, a given purpose, a particular technology to be used and are often valid only for a particular period of time. One can define the quality of a model as its suitability for the intended audience and usage circumstances. Models are subject to classification with regard to occurrence in language games and speech acts within such games. For example, early during systems development, requirements engineers talk to anticipated stakeholders to find out what they want, do not want, should want, and should not want. This purpose turns requirements engineering into a language game with (in principle) specific rules. It would, for example, not be that important if the requirements initially were imprecise, incomplete, or inconsistent and in that case not even implementable. What counts is that, finally, it will be figured out what can or should be accomplished given the policy, resources, time and budget constraints of a project.
Preface
XIII
Since models are utterances they are also signs. Like any other sign they represent for an audience a cultural unit, i.e., a thing that within a given culture can meaningfully be referred to. By means of the sign, its user refers to the cultural unit. Different from arbitrary signs, the models are supposed to be processed and that way enable the sign user to create information about the cultural unit. The distinction between sign and model to some extent is arbitrary and there is an overlap, as signs in general need to be processed for the sign user to be capable of referring to the cultural unit. Moreover, signs may also give more information about the cultural unit than the one of its mere existence. For example, a traffic sign may not only indicate that a crossing is ahead but also tell the distance between the current location and that crossing. However, the key functionality of the sign is the mere reference to the cultural unit, while the key feature of a model is to create further information about it. In the case where a sign is a model, its cultural unit obviously is the model’s original. A modeler can use a number of different relationships between a model and its original. As early as the Middle Ages, with regard to models, a descriptive reference mode was distinguish from a prescriptive one. Roel Wieringa, in his PhD thesis, has suggested using what he has called the “direction of fit,” for showing the difference between these two relationships. I use the term mode or reference mode to denote the kind of relationship with which a modeler refers from a model to its original. For example, if a model is used in the descriptive mode, then a non-tolerable difference between a model and its original suggests changing the model. If, however, a model is used in a prescriptive mode then a non-tolerable difference between that model and its original suggests changing the original. Design models are typically used in a prescriptive mode while analysis models are typically used in a descriptive mode. Further reference modes have been identified for better understanding further kinds of modeling. For example in software process modeling and ontologies, the idealizing and the constituting reference modes are used, respectively. The former mode means that differences between a software process model and its instantiation are tolerated only if they are due to the deviation of the actual development conditions from the ideal ones. Otherwise corrective action has to be considered that brings the actual behavior closer to the ideal one. The constitutive reference mode, however, means that the original, for purposes of technical discourse, is created by the model. As a cultural unit, of course, the original must predate its use as a constitutive model. The kind of corrective action to be taken in the presence of intolerable differences between a model and the original depends on the employed reference mode. It therefore is one of the key tasks of conceptual modeling to identify the prime reference modes and work out appropriate techniques for effective and efficient resolution of such differences. It is, moreover, quite possible that a model user with a given model refers to a variety of originals, maybe even employing different reference modes. Therefore, such a modeler might want to use a number of versions of that model so that appropriate changes can be made without confusion. Typically, for example, throughout the software development process, an analysis model will be turned into a design model. Similarly, a model created
XIV
Preface
from a legacy application in a reverse engineering step might be turned into a prescriptive model when a reimplementation of that application becomes necessary. These complexities of the model–original relation play a role regardless of the particular philosophy the model’s creator might follow, as in many cases the model users would not even know or share the modeler’s more basic views. In this text I have tried to abstain from taking any strong position toward a particular worldview. Obviously any such worldview may have far-reaching impact on any stance toward modeling that one may take. This preface, however, is not the place to go into more related detail. With the requirements for a system (at least in a preliminary version) being elicited, developers start the system design in which a system architecture and prototypical implementation of required functionality of required quality are the issues. Any given system under development (SUD) is therefore subject to speech acts in different language games. While in requirements engineering stakeholders say that they want such and such a SUD, in systems design the developers demonstrate that they can build an SUD with such and such properties under the given constraints. It is well-known that the two mentioned language games may have to be carried out several times and may even take turns. I am, however, not going into further detail here. Rather, I only want to establish that there are a number of language games played in systems development and that a given SUD will occur in a variety of speech acts within these language games. From a holistic view on software development, one cannot, once and for all, give precedence to one of the occurring language games over all others. Therefore formal semantics of modeling languages and thus models cannot, once and for all, be considered as more important than what has been called apriori semantics of models. There are further language games to be considered with typical speech acts and uses of models in them such as system use, system maintenance, system retirement, and system deployment. Conceptual modeling naturally extends to these language games. It is well known that models often are abstractions from their original. This means that some of the original’s characteristics are ignored and do not make it into the model characteristics. The model therefore in some way is a simplification of the original. However, often the model can be more specific than its original. The model often has characteristics that are not characteristics of the original. These characteristics are frequent chosen such that the information about the original can be obtained in the intended way. For example, in mathematics models are frequently built such that an equation plays a prominent role. In computing, ER models are built such that a database easily can be defined. Similarly, state charts are worked out because we know how to create (or even generate) code from them. Obviously in “reality” there are usually neither equations, database structures, nor state charts. Therefore modeling and abstraction are very different processes. In modeling, frequently a “guiding metaphor” is used to work out the model. It sort of “pollutes” the conceptualization of the original in a systematic way so that the resulting model can be processed in the intended way. Modeling not only often ignores something, i.e., simplifies or leaves
Preface
XV
out unimportant details; it also often adds something. What it adds, moreover, can be quite problematic (at least for some model users). For example, the concept of limit in many mathematical models from calculus is used when it comes to figuring out the value a variable eventually takes. Such a variable value may quantify any parameter of a technical or natural process. Obtaining the related limit typically presupposes the ability to check out the parameter’s value for points in time arbitrarily far into the future or to let the parameter’s value grow beyond any boundary. Both presuppositions are, however, inconsistent with beliefs held by many regarding characteristics of the universe. For example, many regard the universe as finite in time and only consisting of finite number of particles. A model thus not only might have more characteristics than its original. Some of these surplus characteristics might contradict basic views of a modeler. If one considers design models a little close, then one finds that things actually are even more complicated than that. For example, a conceptual database model, at the time the database it describes does not even exist, is more detailed than that database. The database thus could be considered as an abstraction of its model. However, after the database has been implemented things change. The database is more detailed than its conceptual model and thus the former may be considered as an abstraction of the latter. Obviously, models used in the prescriptive mode in architecture also rely on this inversion of the abstraction relationship. Problems of this kind certainly may be worrying, however, as rumor has it: “All models are wrong. Some are useful.”5 While further interesting things could be said about all, this introduction needs to be concluded and I cannot go into more related detail. I hope that you will enjoy reading this book. Roland Kaschek
5
For this and other nice aphorisms see, e.g., http://www.chebucto.ns.ca/Science/AIMET/models/.
Organization
List of Reviewers Valeria De Antonellis Devis Bianchini Shawn Bowers Lois M. L. Delcambre J¨ org Desel David W. Embley Ulrich Frank Georg Grossmann Brian Henderson-Sellers Wolfgang Hesse Dietmar Jannach Gerti Kappel Roland Kaschek Christian Kop
Bernd Neumayr Andreas Oberweis Christine Parent Oscar Pastor Fabio Porto Michael Schrefl Vladimir A. Shekhovtsov Stefano Spaccapietra Markus Stumptner James Terwilliger Bernhard Thalheim Susan D. Urban Roel Wieringa
Table of Contents
Section I: The Evolution of Conceptual Modeling Techniques Real-World Semantics of Conceptual Models . . . . . . . . . . . . . . . . . . . . . . . . Roel Wieringa
1
Templates in Domain Modeling – A Survey . . . . . . . . . . . . . . . . . . . . . . . . . Christian Kop and Heinrich C. Mayr
21
Integrity Constraints in (Conceptual) Database Models . . . . . . . . . . . . . . . Bernhard Thalheim
42
Modeling Techniques for Multi-level Abstraction . . . . . . . . . . . . . . . . . . . . . Bernd Neumayr, Michael Schrefl, and Bernhard Thalheim
68
Random Thoughts on Multi-level Conceptual Modelling . . . . . . . . . . . . . . Brian Henderson-Sellers
93
On the Evolution of Quality Conceptualization Techniques . . . . . . . . . . . . Vladimir A. Shekhovtsov
117
Section II: Extensions of Conceptual Modeling A Web of Knowledge: A Conceptual-Modeling Perspective . . . . . . . . . . . . David W. Embley, Stephen W. Liddle, and Cui Tao Service-Based Semantic Collaboration in Networked Systems: A Conceptual Perspective . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Devis Bianchini, Valeria De Antonellis, and Michele Melchiori Conceptual Modeling Approaches for Dynamic Web Service Composition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Georg Grossmann, Rajesh Thiagarajan, Michael Schrefl, and Markus Stumptner The Dynamics of Process Modeling: New Directions for the Use of Events and Rules in Service-Oriented Computing . . . . . . . . . . . . . . . . . . . . Susan D. Urban, Le Gao, Rajiv Shrestha, and Andrew Courter Exception Handling in Web Service Processes . . . . . . . . . . . . . . . . . . . . . . . . Dietmar Jannach and Alexander Gut
137
161
180
205
225
XX
Table of Contents
Leveraging Model-Based Tool Integration by Conceptual Modeling Techniques . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Gerti Kappel, Manuel Wimmer, Werner Retschitzegger, and Wieland Schwinger
254
Section III: New Directions for Conceptual Modeling Data Model for Scientific Models and Hypotheses . . . . . . . . . . . . . . . . . . . . Fabio Porto and Stefano Spaccapietra Model-Based Engineering Applied to the Interpretation of the Human Genome . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Oscar Pastor, Ana M. Levin, Matilde Celma, Juan Carlos Casamayor, Aremy Virrueta, and Luis E. Eraso Author Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
285
306
331
Real-World Semantics of Conceptual Models Roel Wieringa University of Twente, Enschede, The Netherlands
[email protected] www.cs.utwente.nl/~roelw
Abstract. Conceptual modelling is the addition of more real-world semantics to the computations performed by a computer. It is argued that in a proper engineering approach to computing, three kinds of conceptual modelling need to be distinguished, (1) modelling a software solution, (2) modelling the domain in which it operates, and (3) modelling the impact of the software solution on the domain. Nearly 40 years of research in conceptual modelling has yielded a wealth of concepts and notations for conceptual modelling of the second variety, which we call domainoriented. A summary and framework of these results are presented. Conceptual modelling of the third variety, which we call impact-oriented, has been developed less. A framework for this is provided too, and promising directions for impact-oriented conceptual modelling are identified.
1
Introduction
Conceptual modelling as a discipline in computer science started in the 1970s with landmark papers by Abrial [1] and Chen [2] on data semantics. The goal of conceptual modelling is to describe the set of concepts used in a domain in order to define a conceptual schema for the memory that a software system should have of that domain [3, page 11]. This contains at least two subgoals, namely to specify the conceptual schema of an information system and to describe the concepts used in a domain. Any conceptual modelling notation can be used for either of these two subgoals but, as will be argued in section 2, the resulting conceptual models will be different, to the extent that a correct model of the domain can be an incorrect model of a software system and vice versa. A conceptual model of a software system is a specification of a proposed solution to some problem in a domain. Executable conceptual modelling, for example, focusses on a conceptual modelling of software systems to be realized [4,5]. But when a solution is proposed, it must be justified. This justification must show that the proposed solution, when inserted in a problem domain, will produce effects that satisfy stakeholder goals to some acceptable extent. Analysis of this justification in section 2 motivates a distinction between two kinds of conceptual modelling, one that we will call domain-oriented and focusses on entities, communication and and behavior in the domain, and one which we call impactoriented and focusses on the impact of the software solution on the domain. A review of the results of nearly 40 years of research in conceptual modelling Kaschek/Delcambre (Eds.): The Evolution of Conceptual Modeling, LNCS 6520, pp. 1–20, 2011. c Springer-Verlag Berlin Heidelberg 2011
2
R. Wieringa
(section 3) reveals a mature set of insights into domain-oriented conceptual modelling, that are summarized in section 4. Impact-oriented conceptual modelling has received less attention, and in section 5 we identify promising directions for further conceptual modelling research in this area.
2 2.1
Conceptual Modelling as Domain Modelling The Engineering Argument
Computers, as all other artifacts, are designed to help people achieve their goals, and do this by offering useful interactions with their environment. The crucial difference with all other artifacts is that computers are programmable and therefore real-world problem solving by computer means designing programs that are executed by computers, rather than designing the computers themselves. Nevertheless, the logic of engineering software for human goals is the same as the logic of any other engineering activity, and consists of the following reasoning pattern: (1) Problem domain D & solution S lead to effects E, (2) Effects E satisfy stakeholder goals G to some acceptable extent. For example, a cruise control (S) embedded in a car (D) leads to effects (maintaining speed) that satisfies a goal of the driver (easy driving). And an order information system (S) embedded in an organization with customers and suppliers (D) leads to efficient order processing (E), which satisfies the business goal of operational excellence (G). The engineering argument is the logic of industrial product design as analyzed by Roozenburg and Eekels [6]. For example, interaction of a user and a water tap (D) with a kettle (S) triggers a mechanism in the kettle, namely heating, by which the water in the kettle heats up (E), which satisfies the intention of the user to boil water (G) [7]. In software engineering, Gunter et al. [8] have identified the same reasoning pattern. The solution S is then the specification of a software program; the domain D consists of other software, people or hardware devices, and the interaction of these two should have properties that satisfy goals G. 2.2
Domain Models versus Software Specifications
The engineering argument makes a clear distinction between a solution and the domain in which it is inserted. This distinction is trivial if the solution S is a physical system, but it can be very confusing when S involves software. For example, we may describe a conceptual model of books and borrowers where one book cannot be borrowed by more than one person at the same time (Fig. 1). (The following examples are taken from my book [9], where they are discussed in detail.) In the entity models of figure 1 and 2, we use a snapshot interpretation, meaning that the model represents an arbitrary state of the world (not a history of the world) [9, page 80]. This means that Fig. 1 says that in any state of the
Real-World Semantics of Conceptual Models
3
world a book has a loan relationship to at most one person and that a person can have any number of loan relationships (including 0) to books. Viewed as a conceptual model of the domain, this is a description, and depending on the physical and legal properties of the domain, it could be violated. For example, the information analyst could discover that in a particular library, a book can be borrowed by organizations, and so she would have to correct her model by generalizing Person to include legal as well as biological persons.
Fig. 1. A simple conceptual model of snapshots of a domain
Viewed as a conceptual model of a database solution, by contrast, the diagram in Fig. 1 is a specification, not a description. In this case it can be violated too, because there are possible states of the database in which there may be any number of instances of the Loan relationship between the same document and same person. But our response to these violations would to correct the database, not the conceptual model, because the database should be a correct implementation of the model. The direction of fit between a specification and an implementation is opposite to that between a conceptual model and its subject domain [10,11]. Suppose now that we have a historical database, which accumulates Loan relationship instances, but never deletes them once created. In such a database any number of instances of Loan can exist, each representing a real-world Loan instance that existed at a different time. When specifying the conceptual structure of such a database, we must allow that many historical Loan instances are represented in the database at the same time, as shown in Fig. 2, still using a snapshot interpretation. As a model of the domain this is false, as a book cannot be borrowed by several persons at the same time; but as a model of a historical database it is correct.
Fig. 2. A simple conceptual specification of snapshots of a historical database
Fig. 3 clarifies the situation. A software system, such as a database, contains a model of the domain, and the domain and its model are described by different conceptual models. In our historical database example, Fig. 1 is a conceptual model (CM) of the domain, Fig. 2 is a CM of the database, which contains a historical model of the domain.
4
R. Wieringa CM of the model
CM of the domain
specifies
describes
Software system Model of the domain
represents
Domain
Fig. 3. Conceptual models may describe structures in the domain or may specify conceptual structures of a model in a computer, that in turn represents the domain
Fig. 3 reveals a possible confusion in our terminology concerning conceptual modelling. Conceptual models are actually descriptions of conceptual structures in a domain, as on the right-hand side of Fig. 3, or specifications of conceptual structures to be implemented in software, as on the left-hand side of Fig. 3. (Gonzales-Perez & Henderson-Sellers [11] refer to this as backward and forward looking models, respectively, but the distinction is an old one in the philosophy of language [10].) These descriptions or specifications may be written down in the form of text or diagrams. A model inside a software system, by contrast, is a set of symbol occurrences, physically realized in a computer, that represents a domain [12]. The simplest of such models is a single symbol occurrence in the software, called a surrogate by Hall [13], that represents a single entity in the domain. I will follow general practice and continue to speak of conceptual models, but we should be careful to remember that conceptual models of the domain are actually descriptions of the domain. The model of the domain inside the software system in Fig. 3 is a model in the classical sense that it a duplicates some, but not necessarily all, properties of its subject, just as a scale model duplicates some but not all properties of its subject [14]. We can use the model to answer questions about the subject. To distinguish this kind of model from a conceptual model, we will call it a representation model. Software systems that interact with a problem domain in order to achieve desired effects, i.e. achieve human goals, must contain a representation model of at least some properties of the problem domain in order to interpret stimuli and determine responses [9]. The representation model may be historical, in which case it constitutes a memory of past domain states, or it may be a snapshot, in which case at all times it represents a single recent state of the domain. Object-oriented software is based on the design principle that software contains a representation model of the domain, as in Fig. 3, but sometimes objectoriented methodologists tend to confuse the distinction between domain and software solution. For example, in the Fusion method this confusion was elevated to a central design principle [15]: After a class diagram of the domain was drawn, the analyst drew a rectangle enclosing part of the diagram and declared that this “was” the software solution. What was intended is that the software solution would contain a representation model of this part of the domain, in
Real-World Semantics of Conceptual Models
5
the same way as the object-oriented method avant la lettre Jackson System Development (JSD) recommended building up a software system from an initial representation model of the domain [16]. But when Fusion recommended declaring part of the domain class diagram to be the software solution, it left a gaping hole in the domain, and introduced confusion about which objects, existence constraints and cardinality constraints needed to be specified for this softwareand-domain model. Figures 1 to 3 illustrate how this confusion can arise.
3
Conceptual Modelling Notations Since the 1970s
Since the 1970s and arguably even earlier [17], conceptual modelling research has aimed at defining the semantics of a computation in terms of the problem to be solved rather than in terms of the infrastructure that will run the computation. In terms of the engineering argument, at first this amounted to providing a more abstract specifications of the software solution S. In the 1970s, several people made the move to conceptual modelling of the domain D, and this is where we start of our mini-history of conceptual modelling. We classify the notations into three groups, entity-oriented, communication-oriented and behavior-oriented. As elaborated in detail elsewhere [18], this means that we view real-world domains as consisting of entities that interact through communication channels and whose behavior consists of ordering of these interactions in time. This framework has proven to be general enough to classify, compare, disassemble and reassemble structured and object-oriented specification methods known in the later 1990s [19,9]. Here, we apply it to notations for conceptual modelling of the domain. 3.1
Entity-Oriented Approaches
In databases, a significant abstraction from computational structures took place in 1970, when Codd introduced the relational model as an abstraction from pointer structures [20]. This introduced a logical view on data from which storage structures had been abstracted; what remained were variables, called attributes, and cartesian products of these variables, called relations. A few years later, the move to conceptual modelling of the domain rather than of the software solution was made by Abrial [1] and, very influentially, by Chen, who modelled the domain in terms of entities, attributes and relationships [2], usually referred to as the entity-relationship (ER) approach. These developments matched with the three-level view of databases described by the ANSI-SPARC in a reference architecture [21], in which the storage structure view including the pointer structures is called “physical”, the relational view is called “logical” and the user views defined on top of this are called ”external”. This model contains two confusions (Fig. 4): First, there is nothing physical about the “physical” database view. Pointers and other storage structures are abstractions from a physical computer, and all ANSI-SPARC views are conceptual models in the broad sense that they express conceptual structures in
6
R. Wieringa
a computer. In this chapter a physical domain is a domain that can be described by physical measures such as meters, kilograms, seconds, ampere etc. This is to be distinguished from software domains, which consist of symbols and symbol manipulations, and which may contain many abstraction layers, of which the ANSI “physical” and logical layers are only two. Second, the ANSI-SPARC logical view is also often called the ”conceptual” view [22, page 184] and then defined as the collection of objects of interest in the domain, as in the approaches of Abrial and Chen. This confuses software models with domain models. The ANSI-SPARC models describe conceptual storage or data structures in a computer, and these must be distinguished from conceptual structures in a domain of interest to stakeholders of the system. Fig. 4 distinguishes conceptual software and data models from conceptual domain models, and adds the physical computer as a domain of interest. This allows us to think about the ways in which the physical computer impacts the domain.
Fig. 4. The three-level ANSI-SPARC model replaced by one in which software systems are distinguished from domains, and the real physical computer is included
Numerous extensions to the ER notation have been defined, notably extensions of cardinality constraints [23,24] extensions that deal with time and history [25,26], extensions with aggregation structures [27,28,29,30,31] and, more recently, sets [32], and extensions with taxonomic structures [27,33,34,35]. Temporal extensions of the ER notation offer ways of expressing constraints on how entities, their relationships and their cardinality constraints can evolve over time. Historical extensions allow one to express constraints on the history of entities or relationships. Extensions with aggregation allow the expression of various encapsulation and aggregation structures, and extensions with taxonomic structures allow the description of various static or changeable subclass relationships. Much of this research took place in the 1980s and by the end of that decade surveys started to appear [36,37,38]. By the early 1990s, ER approaches had stabilized and textbooks on ER modelling started to appear [39]. In parallel with the development of the ER approach, an approach was developed variously referred to as “Nijssen’s Information Analysis Method”, “Natural language Information Analysis Method” (both with acronym NIAM), binary
Real-World Semantics of Conceptual Models
7
information modelling and Object-Role Modeling (ORM) [40,41]. The core idea of NIAM/ORM is to describe a domain in elementary sentences of subject–verb– object structure and represent this as a set of binary relationships. NIAM/ORM recognizes taxonomic structures too. A third approach to modelling taxonomic domain structures arose in knowledge representation, using notations motivated from linguistics such as semantic networks [42,43]. This line of research has always been closely linked to various logic specification languages and, today, ontology languages based on description logic have been developed to describe the semantics of data found on the World Wide Web. Ontology languages allow one to define conceptual models of domain concepts and their taxonomic relations and they facilitate automated reasoning about taxonomic structures [44]. Analysis and design of these languages tends to ignore the entity modelling approaches listed above but uses results from philosophical logic and formal logic [45,46]. I will not describe these developments in detail but simply point out that all approaches mentioned here view the domain as a collection of actual and possible entities that may have properties and that play roles with respect to each other. Entity-oriented approaches make a number of important ontological assumptions about the domain: – They all assume that we can classify parts of the domain into discrete entities. – Classes can have taxonomic relationships with respect to each other. – In addition, entities play roles with respect to each other, and they can be counted independently of the roles they are playing. – Many approaches define a notion of aggregation, where one entity can be a part of another, and some distinguish several kinds of aggregation, such as set composition or part-whole relationships. 3.2
Communication-Oriented Approaches
The 1970s also saw the rise of communication-oriented conceptual modelling approaches. Just as in the case of databases, this started with an abstraction from the computing infrastructure. Structured programming was a significant step towards structuring software according to the problem to be solved rather than according to the computing infrastructure that would run the program [47]. However, the “problem to be solved” was still the computation to be performed. In the second half of the 1970s an additional layer of abstraction was added by structured analysis, which contained techniques to model the tasks and data flows needed to solve a problem independently from the logic of problem-solving steps. The notation introduced to represent these tasks was the data flow diagram. Several versions were proposed, including those of Ross [48,49], DeMarco [50], Gane & Sarson [51] and Yourdon & Constantine [52]. In the 1980s, techniques to represent control were added, but we will discuss these in the next section on behavior-oriented approaches. Here, two remarks about structured analysis are relevant.
8
R. Wieringa
First, structured analysis still maintained a focus on the solution (“the problem to be solved”) rather than the domain with which the solution would interact, but it removed attention from the steps to be performed. Structured analysis modelled the solution as a set of tasks that, by communicating with each other and with the domain, would solve the problem. Secondly, the domain was then truncated to the entities with which the solution would interact, which were then called “external entities”. So the focus was on communication within and across the interface of the solution, not on communication in the domain. In the 1980s this changed by the introduction of such methods as essential systems analysis [53] and information engineering [54], in which organizational tasks performed in the domain were represented and redesigned by means of data flow modelling. The central idea is that a conceptual model of the flow of information through an organization would show opportunities for improving the structure of the operational tasks, the business processes, that made the information flow this way. Next to this system-oriented approach to communication modelling, a more people-oriented communication modelling approach of the domain arose in the early 1970s in Scandinavia, where Langefors, Lundeberg and others proposed the ISAC method [55,56,57]. This now nearly-forgotten method started with the identification of stakeholders in the domain, catalogued and analyzed their problems, proposed a mitigation of these problems by improving business processes, and then used IT to implement these improvements. The domain model made in ISAC shows tasks performed in the domain and the information flow between tasks, just like data flow modelling. Unlike structured analysis, however, (1) ISAC made a communication model of stakeholders in the domain rather than of the software only, (2) it modelled problems to be solved, and (3) it did not truncate the domain model once it turned to software specification. ISAC was an information system development method, in which the domain was an organization, and it had a strong focus on analyzing human communication in the domain. In the 1980s a number of researchers, many again from Scandinavia, proposed analyzing not only the information flow but the speech acts performed by stakeholders in the domain [58,59,60,61]. A speech act is not just a flow of information, but it is a flow of information produced by human actions with a specific purpose, such as to request, to inform, to command, to declare, to express, to commit, etc. [62]. Analyzing speech acts in the domain with a view to identifying opportunities for supporting these acts by IT came to be known as the language/action perspective on information analysis [63]. Software tools for supporting speech acts have not caught on, and the language/action perspective has never become mainstream in IS research. However, there is still an active research community working on it [64]. We will return to the language/action approach when we discuss conceptual modelling of the impact of software systems in a social domain (section 5). Viewed from the point of view of conceptual modelling of a domain, the important difference between communication-oriented and entity-oriented approaches is that entity-oriented approaches are concerned with what stakeholders in the
Real-World Semantics of Conceptual Models
9
domain communicate about, whereas communication-oriented approaches focus on who they talk with (information flow) and for the achievement of which goal (speech act). Communication-oriented approaches make a number of important ontological assumptions about a domain and about the interactions of a software system with a domain. – From a communication-oriented perspective, the domain consists of actors that perform speech acts by which they exchange information with each other to achieve certain purposes; from an entity-oriented point of view, this information will be about entities that are classified and can be counted, and that can have aggregation and functional relationships to each other. – If a software system is to communicate with such a domain, it should contain a representation model, not only of the entities communicated about, but also of the actors communicated with. This gives us a two kinds of links with the domain, as represented in Fig. 5. Henceforth, we will refer to entities in the domain that can communicate (“talk”) as actors. Actors can be people or other software systems.
Fig. 5. A software system that interacts with a domain by talking with some actors in the domain and by containing a representation model of some of the entities talked about
3.3
Behavior-Oriented Approaches
In the 1980s structured analysis was extended by Ward & Mellor [65] and Hatley & Pirbhai [66] with notations for behavior to represent actual or desired behavior in domains to be controlled by software. This introduced a third group of notations in conceptual modelling that are behavior-oriented. Where communicationoriented notations represent who is interacting with whom, behavior-oriented notations allow representation of the temporal order in which these interactions take place. In structured analysis, the state transition diagrams and tables developed by Mealy and Moore in the 1940s to represent discrete behavior were used. In the 1980s, Harel combined and extended the Mealy and Moore notations into a very powerful notation called statecharts, that allows specification of complex reactive behavior [67,68]. This could be behavior of a software solution, or behavior of entities in the domain.
10
R. Wieringa
In the early 1980s Jackson proposed a behavior-oriented system development method (JSD) [16]. JSD starts with making a conceptual model of domain entities and events, and allocates events to the entities involved in them. It then models the software solution by specifying a conceptual model (i.e. a specification of the conceptual structure) of a representation model in the software, called an initial system model, which is the model of the domain in the software system in Fig. 3. The software specification is then elaborated by adding components to register, respond, and possibly control domain events according to the desired functions of the system. In the 1990s this approach was adopted and partly formalized in the object-oriented design method Syntropy [69]. All of these methods described behavior of domain entities by means of some form of state transition diagram or table. There is another class of notations, that is used to describe tasks to be performed by people: Workflow modelling notations like Petri nets [70,71], event-process chains [72], UML activity diagrams and the Business Process Modelling Notation (BPMN). There is nothing in these notations that prevents them from being used to model software solutions, but here we consider them as notations to describe the behavior of actors and entities in the domain. This also means that these notations are also used to describe speech acts (register, request, order, etc.) performed by people. Behavior-oriented approaches make a number of important ontological assumptions about the domain. – The domain consists of sequences, choices, and parallel occurrences of events and actions in time. Some notations allow the expression of the real time at which events and actions occur. – Some notations also allow the expression of information flow, such as the transfer of data objects from one task to another. Those notations then allow expression of communication in addition to behavior. – In addition, state transition diagrams usually allow the expression of reactive behavior, which tells us how events that occur to an entity cause the entity to respond by performing some action. For example, a description of a sensor may tell us that it produces a signal (effect) when it is blocked (event), and a specification of a business process may state that a confirmation must be sent (effect) when an insurance claim is received (cause). – Communication and reactive behavior are examples of causal behavior. Causality goes beyond simple temporal ordering, because events and actions that are temporally ordered need not be causally connected. Causality includes a kind of internal necessity which says that, all other things being equal, if the cause would not have occurred, the effect would not have occurred either [73]. The ceteris paribus clause “other things being equal” indicates that many factors may interact to produce an effect and if any one of those changes, the effect of any of the others may change too. Causal interaction is the second way software systems interact with a problem domain, in addition to communication by speech act (Fig. 6).
Real-World Semantics of Conceptual Models
11
Fig. 6. A software system that interacts with a domain by talking with some actors in the domain and causally interacting with entities in the domain
4
Real-World Semantics: Domain Structure
We have seen that entity-oriented approaches are concerned with what actors in the domain communicate about, communication-oriented approaches focus on who they talk with (information flow) and for the achievement of which goal (speech act), and behavior-oriented approaches express how entities and actors behave in order to achieve these goals. For example, in the domain of insurance claim handling, the claim handling workflow specifies how actors (damage experts, claim handling personnel) jointly perform a process that involves some entities (the claim itself and the damaged object) to make a decision about the insurance claim (goal). Fig. 7 provides a simple image of this. The claim handling process consists of activities listed in one column of the table, and each activity involves one or more rows of the table. For example, a damage expert (actor) may assess the damaged object (entity), and an evaluator (actor) may decide that the claim (entity) is accepted. One activity may be split into parts, one part for each entity involved in the activity. Each cell then lists the activities that an entity or actor is involved in within one business process. Finally, the row of an entity or actor contains all activities of all processes that this entity is involved in and the analyst may decide to express the life cycle of this entity or actor in a state transition diagram. For example, we may define the life cycle of an insurance claim or of a policy owner by means of a state transition diagram.
Fig. 7. Domain processes consist of the interaction of entities and actors in the domain. Each row lists the events and actions an entity or actor is involved in. Each column lists the events and actions that occur in a domain process.
12
R. Wieringa
In the previous sections we have learned the following about the real-world semantics of notations to express different aspects of domain structure. – Classification and counting. ER diagrams allow the analyst to represent the classes of entities that can exist in a domain. This includes the entities that can perform speech acts (actors), for entity notations do not distinguish this subcategory of entities. ER notations also allow the expression of taxonomic relationships between these classes, and of cardinality properties of these classes, such as how many instances of a class can exist at the same time or how many entities of one class are play a role with respect to an entity of another class. – Aggregation. Where classification is a relationship between classes, aggregation is a relationship between entities in which one entity somehow encapsulates another. Within the entity approach different kinds of aggregation have been recognized. Fig. 7 shows one important kind of aggregation, the aggregation of the domain out of constituent entities, but the situation may be more complex with intermediate levels of aggregation and with aggregations of different kinds. – Communication. Information about entities or actors is passed between actors by means of speech acts. Notations derived from data flow diagrams in structured analysis or information engineering all represent the flow of data between actors but they do not give more details about the speech acts performed by these actors. – Behavior. Activities in the domain are ordered in time, and they may have causal or logical dependencies. This is expressed in behavioral notations. In terms of Fig. 7, behavioral notations are used to express processes across the domain, such as business processes, workflows, or production control processes, or to express the life cycles of entities or actors. These structures have been analyzed at length elsewhere [74,9] and I will not discuss them in more detail in this high-level overview.
5
Real-World Semantics: Solution Impact
To attempt to solve a problem by computer is to insert a proposed automated solution as an additional actor in the problem domain (Fig. 8). The solution then interacts with other domain entities and actors, which should impact domain processes in such as way that the goals stated in the engineering argument are achieved. Impact-oriented conceptual modelling consists of building a conceptual model of how this happens. To understand this, it is useful to distinguish three kinds of domains.
Real-World Semantics of Conceptual Models
13
Fig. 8. The solution has been introduced in the domain as an additional actor (last row of the table). This impacts some domain processes, who now should contribute to the problem solving goal.
5.1
Physical, Social and Digital Domains
A physical domain is one described in terms of time, space, energy and mass measured by kilograms, meters, second, Amperes, etc. Physical domains can be described by borrowing terms from the physical sciences, such as physics, chemistry, biology, geology, etc. Note that this is a very restrictive view of physical domains. Roughly, a physical domain is a domain seen by sensors or keyboards, not by people. A social domain consists of social constructs such as money, commercial transactions, value, business processes, goals, job roles, responsibility, accountability, etc. The characteristic feature of a social domain is that it contains people who have a shared conceptual model of this domain. Many domain entities and events, such as organizations, job roles and money, are social constructions that would not exist of there were no people who share a conceptual model of these entities and events [75]. At the interface of physical and social domains, and part of both, are digital domains. A digital domain consists of symbols and their physical occurrences. Digital domains are physical, because symbol occurrences are physical: Paper, ink, magnetic disk states, signals travelling through a wire, etc. are physical. But at the same time symbol occurrences have a meaning for people, and this meaning is defined by a convention chosen by a group of people. The meaning of symbols can always be defined in a dictionary, which is part of a social world of shared concepts, but it may also be implicit, not described, but nevertheless shared. Examples of dictionaries are a natural language dictionary, a programming language definition, and a standard for exchanging data between companies. A characteristic feature of dictionaries are that from a physical point of view they are social conventions. The relation between a physical symbol
14
R. Wieringa
occurrence and its meaning is a social convention that from a physical point of view is arbitrary and could have been defined differently. This sets apart a photo from a JPG file describing the photo. People who see the photo do not interpret it according to an artificial convention but see whatever is depicted on the photo by means of a physical (biological) process. By contrast, a JPG file describing the photo is a (highly structured) symbol occurrence that must be interpreted according to an artificial convention, namely the JPEG standard. So the JPG file is part of the digital domain. Digital entities have been called lexical entities in NIAM [40], which emphasizes the presence of a dictionary, and workpieces by Jackson [12], which emphasizes the fact that they can manipulated. The popular term for them is virtual entities, which emphasizes the fact that they are simulated on a computer. In this chapter I use the term “digital entity”. The implementation of a digital domain in a software system is actually executable conceptual modelling, which is out of the scope of this chapter. However, describing the domain to be simulated is within the scope of this chapter. In terms of the first part of the engineering argument (1) domain D & solution S lead to effects E, simulation is the production of desired effects. For example, people (D) who want to view a photo (E) use viewing software (S) to do so. The effect (E) is created by interaction of the user (D) with the viewing software (S). And if a person (D) wants to edit a document (E) then he or she uses an editor (S) to do so. This justifies the description of the domain to be simulated independently from any software solution. This is exactly what is done in Jackson’s problem frame approach [12]. We will call a simulated domain an effect domain to indicate its role in the engineering argument. In some cases, the effect of inserting a software system in some domains is the creation of an effect domain, which can be physical (e.g. photos) or digital (e.g. documents). In other cases, the effect of inserting a software system into a physical domain is the imposition of desired behavior on an existing domain, e.g. a production process. And in yet other cases, the effect of inserting a software system into a social domain can be the creation of new possibilities for action, such as the answering of queries about the domain. 5.2
Physical Modelling
In physical domains, such as in production control or in medical applications, the solution interacts with the problem domain by means of causal interactions. In addition to the communicative interaction shown earlier (Fig. 5) we now have a causal interaction, that we elaborate as shown in Fig. 9. Physical events occur in the domain, which may lead to a chain of causes and effects to a physical stimulus at the interface of the system. Conversely, the system may respond to such a stimulus, or to some communication received from an actor, which leads through a causal chain of events to a change in the domain that we will call an action.
Real-World Semantics of Conceptual Models
15
Fig. 9. A software system that interacts with a domain by communicating with actors and causally interacting with entities. It contains a theory about the domain that is used to interpret stimuli and predict the effect of responses.
From a physical point of view the system now not only needs a model of the domain but also a theory: When it receives a stimulus it needs to explain which event caused it, and when it produces a response it needs to predict what action will then occur. Explanation and prediction are functions of scientific theories. For example, an embedded controller of a copier should use a theory to predict the angular speed of a rubber roller when paper is passing over it. The theory needed by the system, even if it is very simple, has the same structure as that of a scientific theory, namely a set of concepts and a set of empirically valid statistical or causal relationships among these concepts [76,77]. This means that there are two kinds of physical modelling activities for physical domains: – Conceptual modelling, which is the definition, analysis and formalization of a set of concepts in a domain. In the philosophy of science defined concepts are often called constructs or variables. – Empirical modelling, which is the description and validation of statistical or causal relationships among concepts. For example, suppose that a continuous process control system contains tubes with valves that can be opened and closed. To make a physical model of such a domain we need to make (1) a conceptual model that defines what a tube is, how different tubes are distinguished, what a valve is, in what states it can be etc. and (2) to make an empirical model of the behavior of these entities, such as the impact of fluid pressure on the processes of opening and closing the valve, fluid leakage through valves, failure behavior of valves, the effect of temperature, etc. This opens up interesting new avenues for conceptual modelling research, including not only the definition of constructs and measurement procedures but also the description of domain theories in a way that can be used for producing the engineering argument. This is domain-specific research because, for example, process control and car control will contain different theories of their domains[78,79]. In addition, there is also domain-independent research to be done, such as in the investigation of ways to operationalize and measure the constructs that we define in conceptual models, such as has also been done for software properties [80], and in the development and study of languages to represent dynamic systems [81].
16
5.3
R. Wieringa
Performative Modelling
In social domains, software systems can create effects by performing speech acts: answering questions, posing questions, giving commands, making commitments, etc. When inserted in a social domain, software systems become actors just as people are actors in this domain. This also means that a software system can perform declarative speech acts, which are speech acts that declare an event or state to exist by saying that it exists [82,83,84]. If done in the proper situation and according to the proper procedures as defined in a shared conceptual model, then these acts are successful. For example, in the Netherlands, software systems have been used to declare someone eligible for military service or for a housing subsidy. Using a software system to perform speech acts requires that the conceptual model of the impact of the software system in the domain be shared with people in the domain. If no one in the domain knows or believes that the output of a software system is an answer to a question, or is a command, or a declaration, then the software system is not really part of the domain and it cannot play its role, and there is a real sense in which its output in that case is not an answer, command or declaration. As pointed out before, there is a small but active community of researchers investigating software systems from a language/action perspective, but so far this has not had an impact on mainstream information systems development practices. An explanation for this is that practitioners do not experience a big problem in specifying speech acts to be performed by software systems. An interesting avenue for research here is not to develop yet another notation for speech act modelling, but to find out empirically what actually is the structure of automated speech acts in practice, how they are in fact defined, and how the speech acts are embedded in a social domain. For example, the evolution of language in a social domain tends to be frozen when part of the language gets automated in software systems [85] and it is interesting and relevant to know how people actually deal with this.
6
Summary and Conclusions
The system engineering argument justifies a solution by showing how it creates desired effects in a domain. Conceptual modelling research has focussed on the concepts and notations for modelling entities, communication and behavior in the domain but has neglected modelling the impact of the software solution on the domain that is to create the desired effects. Performative modelling research has gone some way in this direction but more empirical work needs to be done in this area, and this needs to be extended with research in physical modelling. Impact-oriented conceptual modelling contains some challenging research questions that deserve more attention than they have received so far. These questions include the study of techniques for operationalization and measurement in physical domains, the description of causal and statistical relationships in conceptual models of physical domains, and the way people deal with frozen conceptual models in social domains.
Real-World Semantics of Conceptual Models
17
Acknowledgments. This chapter benefited from constructive and insightful comments by the reviewers.
References 1. Abrial, J.R.: Data semantics. In: IFIP Working Conference Data Base Management, pp. 1–60. North-Holland, Amsterdam (1974) 2. Chen, P.S.: The entity-relationship model – Toward a unified view of data. ACM Transactions on Database Systems 1, 9–36 (1976) 3. Oliv´e, A.: Conceptual modeling of information systems. Springer, Heidelberg (2007) 4. Mellor, S., Balcer, M.: Executable UML: A Foundation for Model-Driven Architecture. Addison-Wesley, Reading (2002) 5. Pastor, O., Molina, J.C.: Model-Driven Architecture in Practice: A Software Production Environment Based on Conceptual Modeling. Springer, Heidelberg (2007) 6. Roozenburg, N., Eekels, J.: Product design: Fundamentals and Methods. Wiley, Chichester (1995) 7. Roozenburg, N.: On the pattern of reasoning in innovative design. Design Studies 14(1), 4–18 (1993) 8. Gunter, C., Gunter, E., Jackson, M., Zave, P.: A reference model for requirements and specifications. IEEE Software 17(3), 37–43 (2000) 9. Wieringa, R.: Design Methods for Reactive Systems: Yourdon, Statemate and the UML. Morgan Kaufmann, San Francisco (2003) 10. Wieringa, R., Meyer, J.J.C., Weigand, H.: Specifying dynamic and deontic integrity constraints. Data and Knowledge Engineering 4, 157–189 (1989) 11. Gonzalez-Perez, C., Henderson-Sellers, B.: Modelling software development methodologies: A conceptual foundation. The Journal of Systems and Software 80, 1778–1796 (2007) 12. Jackson, M.: Problem Frames: Analysing and Structuring Software Development Problems. Addison-Wesley, Reading (2000) 13. Hall, P., Owlett, J., Todd, S.: Relations and entities. In: Nijssen, G. (ed.) Modelling in Database Management Systems, pp. 201–220. North-Holland, Amsterdam (1976) 14. Apostel, L.: Towards a formal study of models in the non-formal sciences. In: Freudenthal, H. (ed.) The Concept and Role of the Model in the Mathematical and the Natural and Social Sciences, pp. 1–37. Reidel, Dordrecht (1961) 15. Coleman, D., Arnold, P., Bodoff, S., Dollin, C., Gilchrist, H., Hayes, F., Jeremaes, P.: Object-Oriented Development: The FUSION Method. Prentice-Hall, Englewood Cliffs (1994) 16. Jackson, M.: System Development. Prentice-Hall, Englewood Cliffs (1983) 17. Young, J., Kent, H.: Abstract formulation of data processing problems. The Journal of Industrial Engineering 9(6), 471–479 (1958) 18. Wieringa, R.: Requirements Engineering: Frameworks for Understanding. Wiley (1996) Also available at http://www.cs.utwente/nl/~ roelw/REFU/all.pdf 19. Wieringa, R.: A survey of structured and object-oriented software specification methods and techniques. ACM Computing Surveys 30(4), 459–527 (1998) 20. Codd, E.: A relational model of data for large shared data banks. Communications of the ACM 13, 377–387 (1970) 21. ANSI: Ansi/x3/sparc study group on DBMS’s interim report. SIGMOD FDT Bulletin of ACM 7(2) (1975)
18
R. Wieringa
22. Tsichritzis, D., Klug, A. (eds.): The ANSI/X3/SPARC DBMS framework report of the study group on database management systems. Information Systems 3, 173–191 (1979) 23. Thalheim, B.: Fundamentals of cardinality constraints. In: Pernul, G., Tjoa, A.M. (eds.) ER 1992. LNCS, vol. 645, pp. 7–23. Springer, Heidelberg (1992) 24. Liddle, S., Embley, D., Woodfield, S.: Cardinality constraints in semantic data models. Data & Knowledge Engineering 11(3), 235–270 (1993) 25. Elmasri, R., El-Assal, I., Kouramajian, V.: Semantics of temporal data in an extended er model. In: Kangalasso, H. (ed.) Entity-Relationship Approach: The Core of Conceptual Modelling, pp. 239–254. Elsevier, Amsterdam (1991) 26. Tansel, A., Clifford, J., Gadia, S., Jajodia, S., Segev, A., Snodgrass, R. (eds.): Temporal Databases. Benjamin/Cummings (1993) 27. Smith, J.M., Smith, D.: Database abstractions: Aggregation and generalization. ACM Transactions on Database Systems 2, 105–133 (1977) 28. Storey, V.: Meronymic relationships. Journal of Database Administration 2(3), 22– 35 (1991) 29. Motschnig-Pitrik, R.: The semantics of parts versus aggregates in data/knowledge modeling. In: Rolland, C., Cauvet, C., Bodart, F. (eds.) CAiSE 1993. LNCS, vol. 685, pp. 352–373. Springer, Heidelberg (1993) 30. Henderson-Sellers, B., Barbier, F.: Black and white diamonds. In: France, R.B. (ed.) UML 1999. LNCS, vol. 1723, pp. 550–565. Springer, Heidelberg (1999) 31. Opdahl, A., Henderson-Sellers, B., Barbier, F.: Ontological analysis of whole–part relationships in OO–models. Information and Software Technology 43, 387–399 (2001) (Erratum on page 577) 32. Ram, S., Khatri, V.: A comprehensive framework for modeling set-based business rules during conceptual database design. Information Systems 30(2), 89–118 (2005) 33. Mylopoulos, J., Bernstein, P., Wong, H.: A language facility for designing databaseintensive applications. ACM Transactions on Database Systems 5, 185–207 (1980) 34. Borgida, A., Mylopoulos, J., Wong, H.: Generalization/specialization as a basis for software specification. In: Brodie, M., Mylopoulos, J., Schmidt, J. (eds.) On Conceptual Modelling, pp. 87–114. Springer, Heidelberg (1984) 35. Wieringa, R., de Jonge, W., Spruit, P.: Using dynamic classes and role classes to model object migration. Theory and Practice of Object Systems 1(1), 61–83 (1995) 36. Urban, S., Delcambre, L.: An analysis of the structural, dynamic, and temporal aspects of semantic data models. In: Proceedings of the International Conference on Data Engineering, pp. 382–387 (1986) 37. Hull, R., King, R.: Semantic database modeling: Survey, applications, and research issues. ACM Computing Surveys 19(187), 201–260 38. Peckham, J., Maryanski, F.: Semantic data models. ACM Computing Surveys 20, 153–189 (1988) 39. Batini, C., Ceri, S., Navathe, S.: Conceptual Database Design: An EntityRelationship Approach. Benjamin/Cummings (1992) 40. Nijssen, G., Halpin, T.: Conceptual Schema and Relational Database Design. Prentice-Hall, Englewood Cliffs (1989) 41. Vermeir, D., Nijssen, G.M.: A procedure to define the object type structure of a conceptual schema. Information Systems 7(4), 329–336 (1982) 42. Brachman, R.: What IS-A is and isn’t: an analysis of taxonomic links in semantic networks. Computer 16(10), 30–36 (1983) 43. Woods, W.: What’s in a link: Foundations for semantics networks. In: Bobrow, D., Collins, A. (eds.) Representation ans Understanding: Studies in Cognitive Science, pp. 35–82. Academic Press, London (1975)
Real-World Semantics of Conceptual Models
19
44. W3C: OWL Web Ontology Language Overview (2004) 45. De Almeida Falbo, R., Guizzardi, G., Duarte, K.: An ontological approach to domain engineering. In: Proceedings of the 14th International Conference on Software Engineering and Knowledge Engineering (SEKE 2002), pp. 351–358. ACM Press, New York (2002) 46. Guizzardi, G.: Modal aspects of object types and part-whole relations and the de re/de dicto distinction. In: Krogstie, J., Opdahl, A.L., Sindre, G. (eds.) CAiSE 2007 and WES 2007. LNCS, vol. 4495, pp. 5–20. Springer, Heidelberg (2007) 47. Dahl, O.J., Dijkstra, E.W., Hoare, C.A.R.: Structured Programming. Academic Press, London (1972) 48. Ross, D.T.: Structured analysis (SA): A language for communicating ideas. IEEE Transactions on Software Engineering SE-3(1), 16–34 (1977) 49. Ross, D.T., Schoman, K.E.: Structured analysis for requirements definition. IEEE Transactions on Software Engineering SE-3(5), 6–15 (1977) 50. DeMarco, T.: Structured Analysis and System Specification. Yourdon Press/Prentice-Hall (1978) 51. Gane, C., Sarson, T.: Structured Systems Analysis: Tools and Techniques. PrenticeHall, Englewood Cliffs (1979) 52. Yourdon, E., Constantine, L.L.: Structured Design: Fundamentals of a Discipline of Computer Program and Systems Design. Prentice-Hall, Englewood Cliffs (1979) 53. McMenamin, S.M., Palmer, J.F.: Essential Systems Analysis. Yourdon Press/Prentice Hall (1984) 54. Martin, J.: Information Engineering. Prentice-Hall, Englewood Cliffs (1989), Three volumes 55. Langefors, B.: Information systems theory. Information Systems 2, 207–219 (1977) 56. Lundeberg, M., Goldkuhl, G., Nilsson, A.: A systematic approach to information systems development - I. Introduction. Information Systems 4, 1–12 (1979) 57. Lundeberg, M., Goldkuhl, G., Nilsson, A.: A systematic approach to information systems development - II. Problem and data oriented methodology. Information Systems 4, 93–118 (1979) 58. Flores, F., Ludlow, J.: Doing and speaking in the office. In: Fick, G., Sprague, R. (eds.) Decision Support Systems: Issues and Challenges, pp. 95–118. Pergamon Press, Oxford (1980) 59. Goldkuhl, G., Lyytinen, K.: A language action view on information systems. In: Ginzberg, M., Ross, C. (eds.) Proceedings of the Third International Conference on Information Systems, Ann Arbor (1982) 60. Lehtinen, E., Lyytinen, K.: Action based model of information systems. Information Systems 11, 299–317 (1986) 61. Auram¨ aki, E., Lehtinen, E., Lyytinen, K.: A speech-act-based office modeling approach. ACM Transactions on Office Information Systems, 126–152 (1988) 62. Searle, J.: Speech Acts. An Essay in the Philosophy of Language. Cambridge University Press, Cambridge (1969) 63. Winograd, T.: A language/action perspective on the design of cooperative work. Human-Computer Interaction 3(1), 3–30 (1987–1988) 64. Weigand, H.: Two decades of the language-action perspective: Introduction. Communications of the ACM 49(5), 44–46 (2006) 65. Ward, P.T., Mellor, S.J.: Structured Development for Real-Time Systems. PrenticeHall/Yourdon Press (1985), Three volumes 66. Hatley, D., Pirbhai, I.: Strategies for Real-Time System Specification. Dorset House (1987)
20
R. Wieringa
67. Harel, D.: Statecharts: a visual formalism for complex systems. Science of Computer Programming 8, 231–274 (1987); Preliminary version appeared as Technical Report CS 84-05, The Weizmann Institute of Science, Rehovot, Israel (February 1984) 68. Harel, D.: On visual formalisms. Communications of the ACM 31, 514–530 (1988) 69. Cook, S., Daniels, J.: Designing Object Systems: Object-Oriented Modelling with Syntropy. Prentice-Hall, Englewood Cliffs (1994) 70. van der Aalst, W., van Hee, K.: Business process redesign: a Petri-net-based approach. Computers in Industry 29(1-2), 15–26 (1996) 71. van der Aalst, W., van Hee, K.: Workflow Management: Models, Methods, and Systems. MIT Press, Cambridge (2004) 72. Scheer, A.W.: Business Process Reengineering: Reference Models for Industrial Enterprises. Springer, Heidelberg (1994) 73. Shadish, W., Cook, T., Campbell, D.: Experimental and Quasi-experimental Designs for Generalized Causal Inference. Houghton Mifflin Company (2002) 74. Wieringa, R.: Postmodern software design with NYAM: Not yet another method. In: Broy, M. (ed.) RTSE 1997. LNCS, vol. 1526, pp. 69–94. Springer, Heidelberg (1998) 75. Searle, J.: The Construction of Social reality. The Free Press, New York (1995) 76. Gregor, S.: The nature of theory in information systems. MIS Quarterly 30(3), 611–642 (2006) 77. Kaplan, A.: The Conduct of Inquiry. Methodology for Behavioral Science. Transaction Publishers (1998); First edition 1964 by Chandler Publishers 78. Jackson, M.: Specialising in software engineering. IEEE Software 16(6), 119–121 (1999) 79. Zave, P.: Formal methods are research, not development. Computer 29(4), 26–27 (1996) 80. Fenton, N., Pfleeger, S.: Software Metrics: A Rigorousn and Practical Approach, 2nd edn. Thomson (1997) 81. Sterman, J.: Business Dynamics: Systems Thinking and Modeling for a Complex World. McGraw-Hill, New York (2000) 82. Kimbrough, S., Lee, R., Ness, D.: Performative, informative and emotive systems: The first piece of the PIE. In: Maggi, L., King, J., Kraenens, K. (eds.) Proceedings of the Fifth Conference on Information Systems, pp. 141–148 (1983) 83. Kimbrough, S.: On representation schemes for promising electronically. Decision Support Systems 6, 99–121 (1990) 84. Wieringa, R.: Three roles of conceptual models in information system design and use. In: Falkenberg, E., Lindgreen, P. (eds.) Information System Concepts: An In-Depth Analysis, pp. 31–51. North-Holland, Amsterdam (1989) 85. Hoppenbrouwers, S.: Freezing Language: Conceptualisation processes across ICT supported organisations. PhD thesis, University of Nijmegen (2003), http://osiris.cs.kun.nl/iris/web-docs/research/index.html
Templates in Domain Modeling – A Survey Christian Kop and Heinrich C. Mayr Alpen-Adria-Universität Klagenfurt, Department of Applied Informatics / Application Engineering Austria {chris,mayr}@ifit.uni-klu.ac.at
Abstract. Conceptual modeling is often strongly related to a graphical language. Since the 80s, template-based approaches have also been proposed. However, they seldom reached the same popularity as graphical approaches. Nevertheless, template-based approaches are also important for collecting and representing information. This chapter will give a survey of approaches that used and use templates. It will be shown, how these templates are used and why their role is important. Keywords: template-based approaches, conceptual modeling, predesign.
1 Introduction Conceptual modeling is often strongly related to using a graphical language. The Unified Modeling Language (e.g., UML) and other languages like ER, ORM, Data Flows, etc. are good examples of this relationship. Many parts of UML are diagrams for specific purposes during software engineering (e.g., class diagrams, activity diagrams, use cases, state charts, object interaction diagrams). Class diagrams help to model the data structure of information systems. Use cases model the functionality the information system must provide to its users. Finally activity diagrams, state charts and object interaction diagrams help to model behavior (i.e., behavior of the system, components or single objects). Class Diagrams, use case and activity diagrams are used in the late stage of requirements engineering (domain modeling). Class diagrams, state charts, object interaction diagrams are typically used during the design phase in software engineering. Although graphical languages like UML are very popular, they only show parts of information needed to develop a system. For more information, template-based approaches have been also proposed. However, they were mostly seen as an “add on” to graphical approaches. This chapter will give a survey of approaches that used templates which are not based on a graphical (diagrammatic) representation. It will show how these templates are used and why their role is important. In order to do this, the chapter will be structured as follows. Section 2 gives an introduction into the several categories of templates. Notions are be defined and a categorization is made. Section 3 firstly provides some thoughts why graphical approaches are so popular, but afterwards it also discusses why template-based Kaschek/Delcambre (Eds.): The Evolution of Conceptual Modeling, LNCS 6520, pp. 21–41, 2011. © Springer-Verlag Berlin Heidelberg 2011
22
C. Kop and H.C. Mayr
approaches are important and thus should not be ignored. Section 4 gives a survey on existing template-based approaches. The next section provides the description and historical summary of a special approach which was developed by the authors (Section 5). Section 6 concludes with a visionary idea and scenario of how templatebased approaches can be integrated into the whole software development cycle in the future. The chapter is summarized in Section 7.
2 Types of Templates Before starting with the description and history of template-based approaches, it is necessary to define what is meant here by the term template. A template here is any kind of textual information which follows a structure. This means, it is possible to know the meaning of some information according to its position within the structure. Templates can be further divided by their purpose and usage into forms and controlled language sentences pattern. A form is a template that has predefined information and empty slots for each of the predefined information. These empty slots have to be filled out. The process of filling the empty slots is supported by the predefined information. In other words according to the predefined information to which an empty slot belongs, the reader of the form can conclude about the “semantic” of the slot’s content and knows what to do with it (i.e., what kind of information must be entered and/or checked). A form can be further refined to attribute-value-pair lists, tables, matrices, cubes, n-dimensional tables and glossaries. If the form is provided in a tabular format with two columns (attribute, value) and in each row the attribute column is filled with predefined information and the value column can be filled out, then it is an attribute-value-pair list. If the form information is repeated several times and the predefined slots can be extracted and used as column header information because they remain invariant in this repetition then the term table will be used. If a second dimension is added to the table, then it is called a matrix. Further dimensions can be added. Then this becomes a cube (3 dimensions) or a multidimensional table, in general (n-dimensions). In this case the header information is only invariant with respect to a certain dimension. Cubes and Multidimensional Tables are only mentioned for the sake of completeness and they will not be mentioned further. They are not commonly used as a mean for communication in Requirements and Software Engineering since two dimensions can be easily described and visualized on a sheet of paper or in standard software products (e.g., Word, PowerPoint, Excel etc.). The term glossary is commonly defined as follows. A glossary is a list of terms together with their definition. From the point of view of a representation, a glossary consists at least of two columns, the column with the term and the column with the definition of terms. Hence a glossary is a table. Beyond this, a glossary has a primary column which contains the terms which must be described. The rows in the other columns support the description of these elements.
Templates in Domain Modeling – A Survey
23
A controlled natural language sentence pattern is a template in which all information is given, but the semantic of a certain kind of information can be derived from its position within the structure (i.e., the grammar). Each sentence follows this very restrictive grammar. Controlled languages are: “Subsets of natural languages whose grammars and dictionaries have been restricted in order to reduce or eliminate both ambiguity and complexity”1 Although the grammar in a controlled language is restrictive, they have a main benefit. The information expressed as a sentence is unambiguous. Furthermore, according to the syntactical structure of a sentence a writer of a controlled language sentence knows which correct combination of word categories he must use in order to get a well formed sentence. For instance, if the pattern is followed that a well formed sentence should have the form subject, predicate object he will certainly not forget the subject or object. Furthermore if he knows that a verb needs an agent (someone/something that initiates execution of a process) he will not forget and the model becomes much more complete. Furthermore, templates can be divided according to their level of abstraction into • templates representing the schema level and • templates representing the instance level. 2.1 Templates for the Schema (Model) Level In a form that describes the schema, the predefined information belongs to the metamodel. Each of the empty slots belongs to a certain schema (model) element. Figure 1 shows some examples for a general form and an attribute value pair list.
precondition
precondition main flow exceptional flow
main flow
postcondition
exceptional flow
postcondition
Fig. 1. Example for a form and an attribute-value pair list
1
See: http://sites.google.com/site/controllednaturallanguage/ (last access: Sept., 27 2010)
24
C. Kop and H.C. Mayr
A controlled natural language sentence might be “The customer orders many products”. Based on the grammar conclusions can be made about certain words in the context of modeling (e.g., nouns become classes or attributes, verbs become relationships, verbs with an agent become candidates for actions in scenario steps etc.). 2.2 Templates for the Instance Level Templates can also be used on an instance level. Instead of specifying the meta-model in the predefined information and the schema in the empty slots, the predefined information is on the schema level and the empty slots must contain the instances of this schema. The previously mentioned notions form, table, matrix and glossary can be found here as well. Typical examples for forms, tables and matrixes are user interfaces, Excel-Sheets, forms for invoices, tabular reports etc. They are also useful for design since the predefined information is useful for the construction of the schema. The information which has to be entered into the empty slots must be managed by the schema and therefore is useful for concrete test cases.
First name Last name Address
Phone Fax e-Mail
Fig. 2. Example of a form at the instance level
Controlled language pattern can also provide information on the instance level. In this case, instead of common nouns, word categories like proper nouns and numbers are used in combination with common individual nouns. These proper nouns and numbers represent the instance level.
3 Graphical Modeling Languages vs. Template-Based Approaches Before talking about template-based approaches, graphical approaches will be discussed.
Templates in Domain Modeling – A Survey
25
3.1 An Advocacy for Graphical Modeling Languages? There are a number of reasons why graphical approaches are so popular: • Humans are visual beings. They perceive and explore the world through their eyes. Other senses - like the sense to hear, taste, feel, and smell something - are often used secondarily. • Pictures, graphical symbols (e.g., cave paintings), statues and totem-poles were the first models of mankind. • Pictures and graphical representations can provide us a spatial overview and an overall description of a physical thing, a phenomenon or a problem. This is why it is often said that a “picture is worth a thousand words”. Due to the good visual comprehension of human, they can better “read” a fact from an overall picture than from a sequence of letters (e.g., the Latin alphabet). • Pictures and graphical representations (if it does not belong to graphical art) is often made exactly just for the purpose to offer a summary (e.g., bar charts in business domains). • A conceptual model in informatics is often compared with a blue print for the construction of a material object (e.g., house, car, bridge etc.). To be trained to think in that way during the education also has an influence on the skills to read and interpret certain model representations. • Finally, it is often argued that according to the above mentioned advantages, these graphical representations are a good basis for later steps. Even model to model transformation is possible. The class diagrams strongly used in model driven architecture (MDA) are representatives for that argument. 3.2 Limitations of Graphical Modeling Languages According to the above subsection, a graphical representation for a modeling system offers advantages. However, a more detailed look to graphical representation techniques also shows that they are not optimal in every situation and for every reader. In order to read or interpret the graphical description, it is necessary that the “reader” understands the semantics of notions. If a person does not understand an underlying notion (e.g., the general idea of a class, an attribute or association) then a graphical representation (e.g., a UML class diagram) is useless for a reader who does not have the necessary skills. If the “reader” does not know that * means “many” in a detailed description of an association, he will not understand in which way the concepts are related to each other. The same happens if we see today a cave painting which was created ten thousand years ago. Although we see different kinds of painted animals we do not understand anymore what the meaning of the complete picture was. We can only make vague assumptions or guesses. Even if the modeling concepts of the graphical representation are semantically understandable, there is still a question concerning the number of concepts visible in the graphic or picture. If there are only a dozen, then the human being of course can
26
C. Kop and H.C. Mayr
realize and interpret all of them at once. However, if there are many (e.g., hundreds) of them, the reader will get lost and the graphical representation looses its ability to be an overview which can be interpreted immediately. Imagine, for example, a class diagram with 100 classes or a Use Case diagram with 100 use cases and actors related to these use cases. In such cases, human beings must introduce abstraction mechanisms (clustering of classes, use cases) to structure the picture. The different kinds of notions represented in a graphical design are the second problem. If there are too many, the reader may get confused. Imagine for example a class diagram that shows everything: classes, attributes, associations, visibilities, data types of attributes, multiplicities, roles, stereotypes of classes and associations, comments on all modeling concepts and much more. Such a class diagram will also likely become unreadable since the reader will not know on which concepts he has to focus. He will be overwhelmed with too much information which he cannot interpret at once. Conceptual modeling tools therefore often offer features to hide information (e.g., to hide attributes, roles, visibilities etc.). Beside the skills of the reader, often the intended purpose of usage has a great influence on the type of model that is used. If the overview character that is provided very well by a picture or graphic is of less importance, then the graphical representation may have less importance. Imagine for example a situation where a questionnaire or pattern is needed. Typically the structure of a questionnaire (i.e., pre defined questions) gives hints to the person who uses the questionnaire regarding which questions are important. Although many software engineers like to see themselves as architects of new systems, if they work in the field of requirements engineering, this “self description” is only conditionally valid. During the early phase of software engineering the relationship between a software (requirements) engineer and another stakeholder (e.g., end user) is like the relationship between a medical doctor and his patient. It is the responsibility of the doctor to extract the causes and later on find solutions for the causes. From the end user he will only get symptoms. Without questionnaires and patterns (at least in mind) he will not be able to find the causes. If requirements engineering experts do not use these questionnaires and patterns explicitly then this does not mean that they are never used since these experts already have internalized the questionnaires and can use them very flexibly, based on the state of communication with the end user. The final argument for graphical approaches (i.e., that they provide a good basis for subsequent steps) is not always true. Consider for instance a use case diagram. The core of the diagram consists of use cases, actors, relationships between use cases, and relationships between use cases and actors. With these information alone transformation to a subsequent step is not possible. It is a template-based approach – the uses case description – that allows a transformation (at least manually). Only with information about preconditions, exceptional flows, main flow and other parts of the use case description, the designer gets an imagination of the details and hence an understanding about the behavior of the future software system. In general a graphical representation is not responsible for the success of a model transformation. The underlying modeling notions of the source and target model are responsible for the success of a transformation. If a match between the notions of the source and target model is possible, then also a transformation is possible.
Templates in Domain Modeling – A Survey
27
3.3 Why Template-Based Approaches? According to Section 3.1 it seems that template-based approaches cannot compete with graphical approaches and graphical representation are better for human beings. According to Section 3.2 graphical approaches have limitations. Hence, templatebased approaches and graphical approaches can complement each other and there is no reason why graphical approaches alone should dominate. The next figure shows that a template in the form of a glossary can be used as a questionnaire. The columns represent hints for questions which must be asked if a cell of a column and row is empty.
ID
Name
120
ICD10
123
SSN
201
hospital
Do we have to consider synonyms for this notion – which ones?
Can you give me some examples for this notion?
Can you define/describe „ICD10“?
Description
?
Examples
?
Synonyms
ValueConstraint
?
?
Quantity description
?
Social Security No. of a patient
Is there any restriction on the values of the examples?
How many examples can exist (minimum, maximum, average)?
Fig. 3. A KCPM Glossary as a questionnaire
If these and other information would appear in a class diagram, then it would be confusing. Therefore, a glossary can complement a diagram in such a way that it provides the involved stakeholders with information that might be necessary for software development. Use cases are another example for such a symbiosis. The use case diagram is a nice picture to describe a certain aspect of the future software, however the most essential information are hidden in the use case descriptions. Thus the relationship between some graphical approaches and template approaches can be seen as an “iceberg”. Only a small percentage of the information appears graphically. The rest, which is represented by templates, is hidden under the “water”. 3.4 Template-Based versus Diagrams – Comparison Studies In the previous subsections pros and cons for diagram-based and template-based modeling languages were mentioned. In this subsection some studies found in literature are presented. These results are only exemplary for specific kinds of templates.
28
C. Kop and H.C. Mayr The visible (graphically represented) information, e.g., use cases, actors, classes, associations. attributes …
The hidden (not graphically represented) information, e.g., use case descriptions, visibilities of attributes, comments on classes …
Fig. 4. Iceberg of schema information
A study [44] was presented which compared textual use cases (TUC) and sequence diagrams (SSD) in the domain of safety system analysis. Textual use cases were tables with the following columns: user actions, system response, threats and mitigation. In each row of the table a piece of text described the user action, a possible system response as well as the threats and mitigations. For the system’s sequence diagram (SSD) the UML notation was used. The textual use case (TUC) was used for interaction between the system and the user. The SSD was used for the interaction between the system and the user and also for internal system communication. The main feature of safety systems is to guarantee safety and avoid failures. During analysis of such systems it is necessary to detect risky situations (failures which might occur). For the comparison of the two presentation techniques, two groups of people (one in Norway, the other in France) were asked to find failures. Statistical methods (t-test, proportional test) were used for answering the following research questions: Is a Textual Use Case (TUC), in general, better than SSD for identifying failure modes? If it is better, are there problem areas where SSD is better? If TUC is better, are there specific failure modes where SSD is better? According to the tests, the following were found: TUC is better to identify failure modes related to functionality of the system and the operator. However, for failure modes which occur within the system, SSD is better. Another study [9] focused on the comparison of tabular representation and bar charts in order to represent production statistics or income statistics. In this experiment people with two different educations (business and engineering) were asked to answer questions concerning income or production based on tabular and graphical representations (bar charts). The time which was needed to get the answer and the number of correct and incorrect answers were measured. The following results were found. There is no difference between two educations concerning the
Templates in Domain Modeling – A Survey
29
time which is needed to get the answer. If the questions become more complex, the time grows to answer it. Independent of their education however, all persons working with tables could answer the question much faster. Concerning the right answers however, there was an educational difference. People with an engineering background were much better using the graphical representation. On the other hand, people with a business background were better at using the tabular representation. Another study [4] came to the conclusion that graphical approaches are better for displaying instructions. A flow of instructions was presented in five formats: flowchart, logical tree, yes/no tree, decision table and list. Furthermore a control panel in a current state was presented. In total, the control panel was able to have eight states. The subjects who did the experiment had to find out the instruction path which leads to a certain state. This was done by pressing one of the eight button (each button represented a state) in each of the formats. According to the representation, the buttons were located at the end of a path (flow chart) beneath a leaf (decision tree) or in a row (in the decision tables). The number of correctly pressed buttons was measured. The results showed that subjects who worked with decision trees make much more errors. Furthermore, it took longer to find the solutions if decision tables and lists are used instead of the three other graphical representations (flow chart, logical tree, yes/no tree). However it also turned out that subjects who worked with a certain kind of format also preferred this format after switching to another format. The only exceptions were subjects who used lists. To summarize, the pros and cons of these different representations are still debated. The same study [9] which focused on graphs and tables for statistical information, listed in their related work many other results which reported the superiority of graphic representation and studies which reported the reverse.
4 A Historical Survey of Important Template-Based Approaches The main stream of techniques belongs to graphical methodologies. In the 70s and 80s methods like SADT, dataflow diagrams, entity relationship diagrams were used to model different aspects of a software system. The latest representative, UML covers static (class diagrams) as well as dynamic aspects (sequence diagrams, state charts, activity diagrams and use cases) of a system using graphical models. 4.1 Forms, Tables, Matrices, Glossaries Apart from graphical representations, template-based approaches were also used since the 70. Parnas [39] used tabular representations for the description of software functions. The rows and columns represented value ranges of two possible input parameters x and y. The cells described the result depending on the value range combinations of the two input parameters. In the 80s, the DATA ID approach [7] used glossaries as a central concept in their methodology. The generation of glossaries was embedded in an engineering process. Firstly the universe of discourse was divided into organizational units. A characteristic of an organizational unit is the homogenous usage of terms (i.e., no homonyms or synonyms exist in an organizational unit). Secondly users and their tasks which must be provided by the information system were identified. A matrix
30
C. Kop and H.C. Mayr
was generated which showed which user can tell information about which tasks. This matrix was the basis for interviews. The designers asked each user details about his tasks. The results of the interviews were written down as requirements statements. These requirements statements were categorized into data, operation and event statements. Finally the information in each statement was more refined and collected into entries of data, event and operation glossaries. The glossaries were the basis for traditional conceptual schemas (ER diagrams and Petri nets). KCPM adopted these glossaries as a modeling methodology. A detailed survey of KCPM is given in Section 5. In the 80s and in the 90s, object-oriented analysis and design were the buzz words of software engineering. Graphical languages like the Object Modeling Technique (OMT), Booch’s method, Jacobson’s Object Oriented Software Engineering (OOSE) became very popular. In the mid nineties, the previously mentioned languages were merged to form UML (Unified modeling language). UML once again was a graphbased modeling language (with some exceptions e.g., OCL). Though object oriented graphical languages gave good spatial overview, one deficiency was detected. What is the best granularity for a class? How large should a class be? Does an end user have the same understanding of the size and features of a class as the designer has? Do all designers share the same understanding of a certain class? It is interesting that these questions were not answered with the graphical opportunities given in OMT, the Booch method or UML. Instead Class Responsibility Cards were introduced in [3] and [51] as a mean to promote better communication between several stakeholders. For each class, an easily understandable template was generated. This template consists of the class name and a list of responsibilities. A responsibility is a short natural language description (i.e., one short sentence) of what the class offers to it’s environment. A responsibility is not a class method. It is only a general description of something that will be handled by one or more class methods. Together with these templates, guidelines were given (e.g., how many responsibilities a class should have in the best case). According to the number of responsibilities a certain class has, the stakeholders made decision if a class should be divided into more fine-grained classes or not. Although the graphical representation of use cases introduced in OOSE were adopted in UML and became very popular, there was a need to complement them with a template [10]. Cockburn added additional, very important information to use cases and also explained how use case descriptions should be applied during requirements engineering. Hence, someone interested in the detailed model could learn much more from use case descriptions than they could learn from the graphical representation of a use case. A use case description is a form that has the following items which must be filled out: Use case name, preconditions for the use case, primary actors, secondary actors, main (normal flow), alternative flows, exceptional flows, post conditions. Today use case descriptions are widely accepted as templates which store detailed information for a certain use case. The latest approach using form templates is NDT [15], [16]. NDT classifies requirements in storage, actor, functional, interaction and non functional requirements. For each of these requirement types, NDT provides a special template
Templates in Domain Modeling – A Survey
31
that has to be filled out during requirements elicitation. Particularly if there is a certain storage requirement (e.g., customer) or functional requirement (e.g., DVD rental) then a template is made for it. The two templates are typical examples of storage requirements and functional requirements, respectively. A template for functional interaction is similar to a use case description. Table 1. Template for a NDT storage requirement (SR) according to [15] SR-01 Description Specific Data
Customer Natural person who has the right to rent DVDs. Name and description Nature Customer id: identification String for the customer Name: the field stores the String customer’s name Address: the field stores the String postal address
Table 2. Template for a NDT functional requirement (FR) according to [15] FR-01 Description Actors Normal sequence
Exception
rent DVD Renting process of a DVD which is offered and executed by a clerk in the DVD rental store. Clerk AC-01: Clerk Step Action 1 The system provides possibilities to search for DVDs 2 The clerk searches for a certain DVD 3 The clerk enters the Customer ID 4 The clerk selects DVD and mark it as rented for customer with given Customer ID Step Action 2 DVD not available or all the requested DVDs are rented – restart with 1 or stop. 3 Customer ID not valid – stop the process 3 Customer is not yet a registered customer, continue with FR02 (customer registration)
4.2 Forms on the Instance Level On the instance level, forms were used as input for schema design [2]. They took forms used in offices and analyzed their structure. They identified several types of structures which can appear on a form (e.g., parametric text, structured frame, descriptive parts). In combination with their previously mentioned glossary approach (see DATA ID in Section 4.1) they proposed a design consisting of three steps. • Form analysis and area design, • Form design, • Interschema integration.
32
C. Kop and H.C. Mayr
In the first step, the concepts which appear in certain areas in the form and their relationships to other concepts are examined. This knowledge is stored in glossaries. During area design, knowledge about the form areas is taken to map each area to a corresponding conceptual schema. During the second step, the derived schemata are integrated for each form. Instead of several schemata which belong to the several areas of one form only one integrated schema for the whole form remains. The last step is another integration step. The schemata for the several forms are once more integrated to an overall first draft conceptual schema. Whereas the analysis of forms was done manually [2], other work [11] presented a computer supported tool which derives a conceptual schema from a user interface. Further works on user interfaces (forms) and their influence on database schemas were made [14] [47]. Forms and user interfaces were used as a user centered mean to query the database. 4.3 Controlled Language Sentences Though, controlled natural language now is a buzzword, the idea behind it was introduced in the 80s. Synonymous terms for controlled natural language are sentence patterns and structured language. For a better understanding of his Entity Relationship (ER) approach, Chen [8] proposed 11 heuristics to map sentences to an equivalent ER schema. He used 11 structured English sentences and showed on the basis of their structure how the elements of such a sentence can be mapped to elements of the ER schema (i.e., entity types, relationship types or attributes). Other research results (e.g., [5], [6], [30], [36], [37], [46], [48]) complemented and refined these initial results. Some of them provided automatic transformation from controlled language sentences by using parsers. In [12] the functional grammar defined in [13] was used as the linguistic basis. Dik’s verb patterns describe semantic roles which nouns can play in combination with a certain verb. The result of these studies was the Color-X model [49]. It is a graphical approach, but it is based on linguistic templates. In [19] the term controlled language is explicitly used. There a controlled language is applied for paraphrasing of ontologies. In [32] an artificial controlled language (“Orthosprache” / “Normsprache”) was developed. Natural language was the basis as well as logic. The artificial language only contains words which are needed to describe the domain (e.g., nouns and verbs). Words like prepositions and articles do not appear in this language. All notions of such a language must be well defined. For instance if the term customer is used then all stakeholders know what is meant. The same holds if another term e.g. “to order” (customer order products), is used. During requirements analysis, the main aim of the language is to support the deeper understanding of semantics of concepts (“Fachbegriffe”) which are used in a certain domain. Since controlled language avoids ambiguities, it is not only used as a basis for conceptual modeling but also during requirements engineering. It is the task of the requirements engineer to break down complex and ambiguous sentences to controlled language requirements statements [38].
Templates in Domain Modeling – A Survey
33
4.4 Controlled Language Sentences on the Instance Level A typical representative for a controlled language on the instance level is NIAM [31]. Structured language generation and analysis were the proposed procedure to get a graphical schema. Once again templates were important and were seen as a support. The idea was that the stakeholders should describe the domain with simple example facts (e.g., The student with student id XYZ visits the course with course number ABC; The course ABC can be visited by the student with student id XYZ). From the set of facts, a graphical model consisting of object types (e.g., student, course) with connected roles (student visits; course can be visited) can be generated. This model evolved and is now called ORM [21].
5 The Story of KCPM KCPM (Klagenfurt Conceptual Predesign Model) is a template-based modeling language that was initiated and developed in the Application Engineering research group at Klagenfurt University. 5.1 The Beginnings The idea to use glossaries was proposed by one of the authors according to his practical experiences [28]. The DATA ID approach, which was published a few years earlier, was introduced as an effective approach to communicate with the end users. 5.2 Research Three master theses examined different aspects of the DATA ID approach. One focused [17] on the transformation rules between glossaries and natural language sentences. Another thesis [40] extended the model of the DATA ID by improving the concept of event type (later called co operation type). The last one [23] focused on the structural and functional aspects (thing types, operation types). The DATA ID data glossaries were refined to thing glossaries (later called thing type glossaries). Also the DATA ID operation glossary was refined (later called operation type glossary). Since the representation was intended to be applied to any modeling concept of the new approach, also a connection (type) glossary was introduced. This master thesis also combined the glossary representation approach of DATAID with the fact-oriented approach used in NIAM. Therefore an algorithm was developed which maps thing types and connection types into entity types, value types, attributes and relationships of an entity relationship model. During that thesis also the name and acronym (KCPM) was born: KCPM = Klagenfurt Conceptual Predesign Model. With this name it is pointed out that the approach must be applied in between requirements analysis and conceptual modeling. It is a model that helps the software engineer to ask the right questions during requirements engineering but also has the advantage that the conceptual model can be generated easily from these working results. It is “conceptual” since there must be an agreement upon the language notions. Since it supports the development of the final conceptual design model it was called a predesign model.
34
C. Kop and H.C. Mayr
A few years later a PhD thesis [24] started with the aim to integrate the results of the first research studies. The outcome was a first, core, lean model based on a small set of notions namely: • Thing types (concepts): Notions/Object types which are important for the domain. • Connection types: Relationships between thing types. • Operation types: Services provided by the future system or components of the system. • Cooperation types with pre- and post conditions: Behavior that triggers operation types. Another aim of the PhD thesis was to build bridges between KCPM and natural language sentences. Therefore, KCPM was embedded into the NIBA2 project. NIBA was a German acronym for natural language based requirements analysis. Computer linguists developed a grammar model for natural languages sentences called NTMS (Naturalness Theoretical Morphosyntax) [18], [27]. The NTMS was taken as the basis to examine the relationships between KCPM glossaries and natural language text. There is still ongoing research on the KCPM topic. Meanwhile there is a PhD project that examines the possibilities of schema integration using the modeling notions of KCPM [50]. Another PhD project focuses on the questions of user centered visualization of web services. Template-based visualization is seen as one good possibility [20]. There are also research connections to other research groups. In 2006 a study started with the aim to map between KCPM thing types and connection types to CBEADS smart business objects [26]. Two master thesis [29],[52] focused on different mapping aspects from KCPM to the OLIVANOVA conceptual model [33], [34] and [35]. Another master thesis [41] analyzed the usage of thing types and connection types for teaching modeling in informatics education at a high school. There is ongoing research with the University of Kharkiv on aspectual and qualitative predesign [22],[42]. Furthermore KCPM is adopted as an ontology representation in software engineering [1] in a joint project with the University of Marburg/Lahn. 5.3 Practical Studies The research was also complemented by practical studies. After the first master theses on this topic were finished, a student working in a local publishing company used the approach to collect their requirements. It was demonstrated, that a domain expert could validate the requirements very well using the collected information in the glossaries. In another master thesis [45], a further practice study with KCPM was done. The primary goal of this master thesis was to investigate possible usage of KCPM in the domain of business process modeling. Since the student already worked in a software development enterprise which needed this information, the student was told to ask the involved stakeholders in which situations glossaries can be preferred over graphical representations. This study pointed out that glossaries are very well understood and preferred in situations where a listing is necessary, or information can be presented as check lists. Hence thing type glossaries, connection type glossaries as well as 2
NIBA was funded by the Klaus Tschira Stiftung Heidelberg.
Templates in Domain Modeling – A Survey
35
operation type glossaries were seen as the best candidates for a glossary representation (i.e., list of terms = thing type glossary, list of relationships = connection type glossaries, list of services of the system = operation type glossaries). The interviewed persons only had problems with cooperation type glossaries because of their complexity. A cooperation type glossary contains the set of operation types together with their preconditions and postconditions. The persons argued that a graphical representation is more suitable. This study was a very interesting hint for further research. Other practical experiences were gathered in a medical domain. The content of this requirements engineering project were data driven. New software was necessary to manage the data, to get statistics from the database and for decision support of the users. We found that thing type and operation type glossaries could be applied very successfully for the collection of all the requirements. To summarize, parts of the KCPM approach could be applied in several practical projects. The results were twofold. On the one hand it could be shown that there is a need for template approaches. On the other hand, the feedback and experiences from the practical studies were used to further improve the approach. 5.4 Beyond Glossaries – Current State of KCPM The arguments for glossaries and graphical approaches in Section 2 and the practical studies of Section 5.3 showed that glossaries are important but graphical approaches must always be considered. It would be a mistake to ignore graphical approaches. The reasons are simple: • The different skills of different users must be always considered. Some of them like templates and others like graphical approaches. • The situation (purpose) is always changing in a typical requirements engineering project. At one point in time, the requirements engineer must act like a medical doctor extracting the causes from the symptoms of the patients (end user). In the very next moment, he must give an overview to the end user or he needs the overview for his own better understanding. Because of these reasons, in one study [25] the main research goal was not to discuss why and in which situation glossaries are better rather to think of how combining different kinds of representations. Particularly: How could a graphical representation be combined with template representations? The conclusion was: Graphical and template-based representations must be seen as equally valuable views within in a toolset.
6 Conclusion and Future Vision 6.1 Conclusion A recent paper [43] describes how to switch among different aspects of a model (e.g., the dynamic aspects, static aspects etc.) within a meta-model. However, this was presented once again for a graphical representation. Particularly, based on the metamodel a tool provides a view with a class diagram and a view with the use case
36
C. Kop and H.C. Mayr
diagram, the meta-model guarantees consistency between the views. In other words, a use case is not longer seen as an independent model within UML where relationships to classes or other UML concepts are not checked. The idea to switch between different aspects of a model consistently must be combined with different representation techniques. It must not only be possible to switch between classes and use cases but also to switch between a graphic representation and a template-based representation. 6.2 Templates in SW Development – Future Vision In order to describe our view of the future, firstly we comment on the past - in particular something on the evolution of programming languages. At the beginning of software development, assembler languages were used to program the computer. In these assembler languages important and very commonly used combinations of low level operations were clustered into macros. However these languages had the disadvantage that they were close to the machine. The developers had to keep in mind to which memory address and registers they have to store values. Third generation languages like Cobol and Fortran were the first step towards a more human readable programming language. These programming languages were used in specific domains. Cobol was developed and used for implementing business solutions. Fortran was mainly used in mathematical domains. During those days scientists also tried to understand what is essential for good programming. The solutions were programming languages for structured programming (e.g., ALGOL, Pascal etc.). Further style guides (e.g., how to decompose large software into units that can communicate with each other without side effects) lead to further evolutions of 3rd generation programming languages to module and object based languages and then to object oriented languages (e.g., Modula-2, Modula-3, Smalltalk, C++, Java etc.). The idea to make programming languages more user-understandable was also realized in the 4th and 5th generation languages. These languages were developed for certain kind of applications (e.g., SQL for database queries, LISP and Prolog mainly for problemsolving situations). These languages were more human readable since they focused on the “WHAT” and not on the “HOW”. Using these languages the user had to specify WHAT he needed and not how it should be executed. This was once again achieved by hiding some internals (e.g., in SQL the user need not know the access paths to records in a database; in Prolog he need not know the technical details about backtracking but can rely on the Prolog interpreter to fire rules based on the available facts). If we summarize this, then it can be learn learned that during the evolution of programming languages, complexity was hidden and style guides as well as patterns were introduced. During the evolution of 3rd generation programming languages the goal always was to “transform” 3rd generation language code to efficient machine readable code in a 2nd or even 1st generation programming language. With model driven architecture (MDA), 3rd generation programming languages became the final target of transformation processes. Nowadays, MDA is based on the idea that every platform independent model (PIM) will be extended with platform specific features. Doing this, the PIM is transformed to a platform specific model (PSM) which itself can be the PIM for the next transformation step. What is done now in MDA is an
Templates in Domain Modeling – A Survey
37
evolution like the evolution of program languages in the past. Scientists came to an overall understanding about good programming. Therefore programming languages are now the target of transformation. Model driven architecture can be also seen as a next step to make modeling more human understandable. During the evolution of the programming languages, the main focus was to exempt the languages from machinespecific pieces of code. During model driven architecture one of the ideas is that pictures often represent more than 1000 lines of code. Once again certain kinds of macros are introduced. Classes can be specified with its attributes but there is no need to specify their constructors and their get- and set-methods. These methods are automatically derived from the class specifications. Combining this information with the knowledge about templates described in this paper, one vision of the future might be the following. Scientists will come to a better understanding about model driven architectures, based on style guides, design and architectural patterns. They will improve the quality of model driven architecture. At the end we will get a common understanding what a PIM needs in order to be of good quality. Having this, the question will not be any longer to get executable software or a completed source code according to a certain specification. Instead the focus will be on how to get a specification (the first PIM) from requirements. Templates will support this new construction idea. Instead of trying to get a graphical model, requirements engineers will behave like medical doctors to collect all the necessary information and to generate the target graphical model. Such a working step will focus on extracting structured requirements specifications from unstructured requirements specifications. Templates will play an important role within this step. As an intermediate result, for quality checking the stakeholders will mainly work with • • •
template (e.g., glossary) entries and only if really necessary with a graphical model that represents either the final PIM or an intermediate version.
On any of these intermediate results the stakeholders will be able to make corrections. End user and designer together will be able to check the templates. This will also be a further step towards human readability of models. The human reader is now of another kind. It is no longer a technically experienced user but also an application domain expert with little technical knowledge. Thus, in the future modeling a software system will be like going through a specific checklist for this software.
7 Summary This chapter presented a survey on template-based conceptual modeling approaches. Historical approaches were presented. Although some of them are mainly known as graphical modeling techniques, they are based on templates (e.g., forms, glossaries and in most cases linguistic templates). The aim of this chapter was to create awareness, that the usage of graphical representation is not always the best solution. It
38
C. Kop and H.C. Mayr
strongly depends on the stakeholder’s skills and situation. In certain situations, a template is better suited than a graphical representation. However the best usage of graphical and template-based techniques is always a situation depended combination of these techniques. Acknowledgments. The authors would like to thank Dr. h.c. Klaus Tschira for his support of the NIBA project. Without this support the extensive research on this interesting topic would not have been possible. Furthermore, we thank all the colleagues and students which have worked in the area of conceptual predesign. Finally we would thank the reviewers for their helpful hints and comments with which it was possible to improve this chapter.
References 1. Bachmann, A., Russ, A., Vöhringer, J., Hesse, W., Mayr, H.C., Kop, C.: OBSE - an Approach to Ontology-based Software Engineering in the Practice. In: Reichert, M., Strecker, S., Turowski, K. (eds.) Proceedings of the 2nd International Workshop on Enterprise Modeling and Information Systems Architectures. GI Lecture Notes in Informatics (LNI), vol. 119, pp. 129–142. Köllen Verlag (2007) 2. Batini, C., Demo, B., Di Leva, A.: A Methodology for conceptual design of offices data bases. Information Systems 9(2-3), 251–264 (1984) 3. Beck, K., Cunningham, W.: A Laboratory For Teaching Object-Oriented Thinking. In: Conference Proceedings on Object-oriented programming systems, languages and applications, pp. 1–6. ACM Press, New York (1989) 4. Boekelder, A., Steehouder, M.: Selecting and Switching: Some Advantages of Diagrams Over Tables and Lists for Presenting Instructions. IEEE Transaction on Professional Communication 41(4), 229–241 (1998) 5. Buchholz, E., Cyriaks, H., Düsterhöft, A., Mehlan, H., Thalheim, B.: Applying a Natural Language Dialog Tool for Designing Databases. In: Proc. International Workshop on Applications of Natural Language to Databases (NLDB 1995), pp. 119–133 (1995) 6. Buchholz, E., Düsterhöft, A., Thalheim, B.: Capturing Information on Behaviour with the RADD-NLI: A Linguistic and Knowledge Based Approach. In: Riet, v.d., Burg, R.P., Vos, A.J. (eds.) Proceedings of the 2nd Int. Workshop on Applications of Natural Language to Information Systems (NLDB 1996), pp. 185–192. IOS Press, Amsterdam (1996) 7. Ceri, S. (ed.): Methodology and Tools for Database Design. North Holland Publ. Comp., Amsterdam (1983) 8. Chen, P.P.: English Sentence Structure and Entity Relationship Diagrams. Int. Journal of Information Sciences 29, 127–149 (1983) 9. Coll, R.A., Coll, J.H., Thakur, G.: Graphs and Tables a Four-Factor Experiment. Communications of the ACM 37(4), 77–86 (1994) 10. Cockburn, A.: Writing Effective Use Cases. Addison Wesley Publ. Comp., Reading (2000) 11. Choobineh, J., Mannino, M.V., Tseng, V.P.: A form-based approach for Database Analysis and Design. Communication of the ACM 35(2), 108–120 (1992) 12. Dignum, F., Kemme, F., Kreuzen, W., Weigand, H., van de Riet, R.P.: Constraint modelling using a conceptual prototyping language. Data & Knowledge Engineerng 2, 213–254 (1987)
Templates in Domain Modeling – A Survey
39
13. Dik, S.: Functional Grammar. North Holland Publ. Company, Amsterdam (1978) 14. Embley, D.W.: NFQL: The Natural Forms Query Language. ACM Transactions on Database Systems 14(2), 168–211 (1989) 15. Escalona, M.J., Reina, A.M., Torres, J., Mejías, M.: NDT a methodology to deal with the navigation aspect at the requirements phase. In: OOPSLA Workshop: Aspect-Oriented Requirements Engineering and Architecture Design (2004) 16. Escalona, M.J., Koch, N.: Metamodeling the Requirements of Web Systems. In: Proceedings of the 2nd Int. Conf. Web Information Systems and Technologies (WebIST 2006). Lection Notes in Business Information Processing (LNBIP), vol. 1, pp. 267–280. Springer, Heidelberg (2006) 17. Felderer, A.: Zur Tabellarisierung natürlichsprachlicher Anforderungsbeschreibungen. Diplomathesis, Universität Klagenfurt (1992) 18. Fliedl, G.: Natürlichkeitstheoretische Morphosyntax – Aspekte der Theorie und Implementierung. Gunter Narr Verlag, Tübingen (1999) 19. Fuchs, N.E., Höfler, S., Kaljurand, K., Rinaldi, F., Schneider, G.: Attempto Controlled English: A Knowledge Representation Language Readable by Humans and Machines. In: Eisinger, N., Maluszynski, J. (eds.) Reasoning Web. LNCS, vol. 3564, pp. 213–250. Springer, Heidelberg (2005) 20. Gälle, D., Kop, C., Mayr, H.C.: A Uniform Web Service Description Representation for Different Readers. In: Berntzen, L., Smedberg, A. (eds.) Proceedings of the second International Conference on the Digital Society (ICDS 2008), pp. 123–128 (2008) 21. Halpin, T., Bloesch, A.: Data modelling in UML and ORM: a comparison. Journal of Database Management 10(4), 4–13 (1999) 22. Kaschek, R., Kop, C., Shekhovtsov, V.A., Mayr, H.C.: Towards simulation-based quality requirements elicitation: A position paper. In: Rolland, C. (ed.) REFSQ 2008. LNCS, vol. 5025, pp. 135–140. Springer, Heidelberg (2008) 23. Kop, C.: Herleitung von EERM+ Schemata aus Zusammenhangsverzeichnissen, erweiterten Ding- und Operationsverzeichnissen. Diplomathesis, Universität Klagenfurt (1993) 24. Kop, C.: Rechnergestützte Katalogisierung von Anforderungsspezifikationen und deren Transformation in ein konzeptuelles Modell. Doctoral thesis, Universität Klagenfurt (2002) 25. Kop, C.: Visualizing Conceptual Schemas with their Sources and Progress. International Journal on Advances in Software 2(2,3), 245–258 (2009), http://www.iariajournals.org/software/ (last access September 27, 2010) 26. Liang, X., Ginige, A.: Smart Business Object - A New Approach to Model Business Objects for Web Applications. In: ICSOFT 2006, pp. 30–39 (2006) 27. Mayerthaler, W., Fiedl, G., Winkler, C.: Lexikon der Natürlichskeitstheoretischen Morphosyntax. Stauffenburg Verlag, Tübingen (1998) 28. Mayr, H.C., Dittrich, K.R., Lockemann, P.C.: Datenbankentwurf. In: Lockeman, P.C., Schmidt, J.W. (eds.) Datenbank-Handbuch, pp. 486–552. Springer, Heidelberg (1987) 29. Michael, J.: Connecting the dynamic part of KCPM with the OlivaNova Modeler, Masterthesis, Universität Klagenfurt (2010) 30. Moreno, A., van de Riet, R.P.: Justification of the equivalence between Linguistic and Conceptual Patterns for the Object Model. In: Proc. 3rd Int. Workshop on Application of Natural Language to Information Systems, pp. 57–74 (1997) 31. Nijssen, G.M., Halpin, T.: Conceptual Schema and Relational Database Design – A fact oriented approach. Prentice Hall Publ. Comp., Englewood Cliffs (1989)
40
C. Kop and H.C. Mayr
32. Ortner, E.: Methodenneutraler Fachentwurf. B.G. Teubner Verlagsgesellschaft, Stuttgart, Leibzig (1997) 33. Pastor, O., Molina, J.C., Iborra, E.: Automated Production of Fully Functional Applications with OlivaNova Model Execution. ERCIM News (57), 62–64 (2004) 34. Pastor, O., Gomez, J., Insfran, E., Pelechano, V.: The OO-method approach for information systems modeling: from object-oriented conceptual modeling to automated programming. Information Systems 26(7), 507–534 (2001) 35. Pelechano, V., Pastor, O., Insfran, E.: Automated code generation of dynamic specializations: an approach based on design patterns and formal techniques. Data and Knowledge Engineering 40(3), 315–353 (2002) 36. Rolland, C., Ben Achour, C.: Guiding the Construction of textual use case specifications. Data & Knowledge Engineering Journal 25(1-2), 125–160 (1998) 37. Rolland, C.: An Information System Methodology Supported by an Expert Design Tool. In: Pirow, P.C., Duffy, N.M., Ford, J.C. (eds.) Proceedings of the IFIP TC8 International Symposium on Information Systems, pp. 189–201. North Holland Publ. Company, Amsterdam (1987) 38. Rupp, C.: Requirements Engineering und Management. Hanser Verlag (2004) 39. Ryszard, J., Parnas, D.L., Zucker, J.: Tabular Representations in Relational Documents. In: Hoffman, D., Weiss, D.M. (eds.) Software Fundamentials – Collected Papers by David Parnas, pp. 71–85. Addison Wesley Publishing Comp., Reading (2001) 40. Schnattler, M.: Herleitung von Ereignisschemata aus erweiterten Operations- und Ereignisverzeichnissen. Diploma thesis, Universität Klagenfurt (1992) 41. Schein, M.: Moderne Modellierungskonzepte der Informatik in der Schulpraxis, Diplomathesis, Universität Klagenfurt (2010) 42. Shekhovtsov, V., Kostanyan, A., Gritskov, E., Litvinenko, Y.: Tool Supported Aspectual Predesign. In: Karagianis, D., Mayr, H.C. (eds.) 5th International Conference on Information Systems Technology and its Applications (ISTA 2006). LNI, vol. P-84, pp. 153–165. Köllen Verlag (2006) 43. Sinz, E.: Tool-Unterstützung für die SOM-Methodik: Anforderungen und Lösungsstrategien. In: Presentation slides: 1st International Open Models Workshop, Klagenfurt (March 2010) 44. Stålhane, T., Sindre, G., du Bousquet, L.: Comparing Safety Analysis Based on Sequence Diagrams and Textual Use Cases. In: Pernici, B. (ed.) CAiSE 2010. LNCS, vol. 6051, pp. 165–179. Springer, Heidelberg (2010) 45. Stark, M.: Geschäftsprozessmodellierung im konzeptuellen Vorentwurf, Diplomathesis, Univeristät Klagenfurt (2000) 46. Tjoa, A.M., Berger, L.: Transformation of Requirement Specification Expressed in Natural Language into an EER Model. In: Elmasri, R.A., Kouramajian, B., Thalheim, B. (eds.) Proc. 12th International Conference on Entity Relationship Approach, pp. 127–149. Springer, New York (1991) 47. Terwillinger, J.F., Delcambre, L.M., Logan, J.: Queyring through a user interface. Data & Knowledge Engineering 63, 774–794 (2007) 48. Vadera, S., Meziane, V.: From English to Formal Specifications. The Computer Journal 37(9), 753–763 (1994) 49. Van de Riet, R.: Mokum for Correctness by Design in Relation to MDA, In: Kaschek R., Kop, Ch., Steinberger, C., Fliedl, G. (eds.) Information Systems and e-Business Technologies. Lecture Notes in Business Information Processing (LNBIP), vol. 5, pp. 352–364 (2008)
Templates in Domain Modeling – A Survey
41
50. Vöhringer, J., Mayr, H.C.: Integration of schemas on the pre-design level using the KCPM-approach. In: Nilsson, A.G., Gustas, R., Wojtkowski, W.G., Wojtkowski, W., Wrycza, S., Zupancic, J. (eds.) Advances in Information Systems Bridging the Gap between Academia & Industry, pp. 623–634. Springer, Heidelberg (2006) 51. Wirfs-Brock, R., Wilkerson, B.: Object-oriented design: a responsibility-driven approach. In: Conference proceedings on Object Oriented Programming Systems Languages and Applications, pp. 71–75. ACM Press, New York (1989) 52. Yevdoshenko, N.: A Proposal for Model Driven Development Life Cycle: From KCPM Requirements Specifications to OLIVANOVA Conceptual Model, Master Thesis, Universität Klagenfurt (2006)
Integrity Constraints in (Conceptual) Database Models Bernhard Thalheim Department of Computer Science and Applied Mathematics Christian Albrechts University Kiel, Olshausenstr. 40, 24098 Kiel, Germany
[email protected] Abstract. The theory of integrity constraints has led to a large body of knowledge and to many applications such as optimisation of schema behaviour through normalisation to schemata which are easier to maintain, as automatic enforcement of database integrity by triggering or other procedural techniques, as quality improvement of data in databases, and as query recompilations due to the knowledge of validity of constraints in a database. Integrity constraints are often misunderstood, are given in the wrong database context or within the wrong database models, often combine a number of very different facets of semantics in databases and are difficult to specify. At the same time, pearls and hidden treasure of the theory of integrity constraints have been already forgotten. We develop a unifying approach to specification and treatment of integrity constraints. We survey some pearls, nuggets and lessons learned with semantic models.
1 Revisiting Database Constraint Theory Structures and constraints are intertwined and coexisting fundamental concepts in modern database systems. Databases must contain trusted data that are correct and coherent. User want to trust that right answers are given to their queries. The golden rule must be thus that no modification operation to data in a database can cause conflicts with constraints. 1.1 Omnipresence of Database Constraints Database constraints have been introduced for an explicit definition of semantics. Typically a database type is given by a structure and a set of integrity constraints that restrict classes for a type to those for which the constraints are valid. This type theory is a specific form of abstract data type theory. Modification and retrieval operations are assumed to be canonically given for any database type. Similarly, a database schema is specified by database types and schema integrity constraints. Only canonical operations are given. Therefore, integrity constraints play a prominent role during database specification. Constraints also allow one to overcome limitations of the modelling language. For instance, the relational model is a model of low expressivity and thus enhanced by a very large set of different (> 95) integrity constraint classes. Semantical modelling languages such as object-relational (or entity-relationship) languages allow to express some of these constraints through constructs. At present we know at least five application fields of database constraints theory: (1) normalization for a more efficient storage, search and modification; (2) reduction of relations to subsets with the same information together with the semantic constraints; Kaschek/Delcambre (Eds.): The Evolution of Conceptual Modeling, LNCS 6520, pp. 42–67, 2011. c Springer-Verlag Berlin Heidelberg 2011
Integrity Constraints in (Conceptual) Database Models
43
(3) utilization of dependencies for deriving new relations from basic relations in the view concept or in so-called deductive databases; (4) verification of dependencies for a more powerful and user-friendly, nearly natural language design of databases; (5) transformation of queries into more efficient search strategies. A large number of structural and dynamical database constraints have been introduced in the past. We must however acknowledge that a fully fledged theory of database constraints is not yet existing. 1.2 Separation of Integrity Constraints by Their Use and Usage There are several classifications for integrity constraints: – either utilization characteristics are used for classification into domain constraints, key and functional dependencies, referential integrity constraints, join dependencies etc. – or their specific format of the formulas is used for classification into tuple-generating dependencies, equality-generating dependencies, existence constraints, single-table constraints, singleton-tuple constraints, etc. These characterizations are useful whenever constraints are formally defined. Their practical utility is, however, more restricted. Another characterization approach has been used in [Tha00] by relating constraints to the phase of database modelling into design, structural, semantic and representational constraints. We may combine the three approaches by clustering constraints according to their structural properties into – constraints expressing identification or partial identification of values by other values, – constraints stating relative independence of values in a class or within an object, – constraints stating existence (or non-existence) of values in an object, or values in groups of objects, or objects in a class, and – constraints expressing redundancy of values or objects. At the same time we may distinguish constraints according to their utilization in the design process. They might be meaningful at the level of the user, or at the level of the conceptual schema or at the level of the implementation. The following table shows this characterization. Partial identifi- Relative independence Existence depen- Redundancy depencation dency dency identification structure no null elementary facts
Business user level Conceptual functional, multivalued, hierarchilevel equality gener- cal, join dependencies, ating exclusion dependency, tuple generating, horizontal decomposition Implemen- key, uniqueness, decomposition, stored tation level trigger, check procedures, trigger
null-value-free, inclusion constraint, union constraints, exclusion constraint numerical, cardinality constraint no null, stored pro- referential integrity, cedures, trigger surrogate, container
44
B. Thalheim
1.3 Quality Criteria for Constraint Sets Database systems aim in automatic support of quality. There are a number of quality criteria that have classically been considered in many textbooks and papers. Structural quality criteria are structural completeness, satisfiability of the schema, liveness of the database, applicability of automatic control features, explicit exception handling, applicability of operations, executability of operations and framing consistency procedures. The first three conditions are well discussed in the database literature. Automatically generated tests and control conditions are still an open research field. Operations are currently mainly applied based on the transaction approach, i.e., forcing a rollback after problems have been detected. Exception handling and execution control use the same approach. The framing or ramification problem is not yet solved. It requires a separation within a database into data that are not affected by a change and into data that are under potential change. A typical example of non-framed executions are trigger avalanches. Quality control must also consider the abstraction level of the stakeholder involved. Integrity constraints may be ambiguous or may be based on context or ellipses. We therefore need an explicit statement of the abstraction level. For instance, join dependencies are a specific vehicle for structuring the database. They are not used by the requirements engineer. There are however specification constraints at the requirements level that must be mapped to the internal levels. 1.4 Optimisation of Behaviour through Normalisation of Database Structuring Normalisation has been developed as a vehicle for performance improvement of database systems. It addresses at least seven different targets: (A) Redundancy becomes problematic whenever additional actions are required for consistency management of data that are stored within different objects. (B) Blocking of management due to the information capacity of the schema. For instance, the insertion anomaly occurs since units of storage such as schema types do not support insertion of partial information. (C) Information loss after database modification occurs whenever data are eagerly deleted despite the importance of parts of it. The deletion anomaly is observed whenever facts are deleted together with the objects where they are contained despite its importance for the application. (D) Evolution sensitivity and instability of the database whenever changes are applied to the database. (E) Different abstractions are used for the database schema at the same time. For instance, views, derived attributes, logs are stored together with the basic data that are used to derive these values. (F) Performance problems can also be solved through restructuring. Typical performance problems considered are caused by integrity constraint maintenance. Update anomalies have been considered as a prominent example of a performance problem since singleton fact operations resulted in complex bulk operations. Performance problems are however also caused by architectures chosen for the application, by specific behaviour of the application, by retrieval requirements, by generation and maintenance of supporting structures such as indexes, etc. The last set of performance problems is often resolved by denormalisation, i.e., by intentional
Integrity Constraints in (Conceptual) Database Models
45
acceptance of another normalisation. Denormalisation may decrease complexity of retrieval and maintenance operations, may avoid additional join operations and may prepare special derived data for support of repeating computations. It allows us to consider semantically meaningful units instead of normalised structures. Index management is optimised. Denormalisation increases however complexity of some database operations, leads to redundancy problems, may result in inflexibility against evolutionary changes. (G) Expensive maintenance, operating and modification of databases often occurs due to consistency maintenance. Parallel execution of transactions may result in deadlocks. As far as we know there is not yet any theory that integrates the six targets of normalisation. Moreover, (A), (C) and (G) are considered to be the primary issues. 1.5 Outline of the Paper The paper aims to demonstrate the power of constraint theory for advanced database models. There is no textbook that provides a modern view onto the achievements of this theory. Many very useful results are forgotten, neglected or suppressed. We may claim that this state leads to new database models that ignore the achievements of the past and require the reinvention of the wheel or result in less elegant model languages. Some of the difficulties with constraints are based on their limitations in expressivity, are based on their restricted binding to a class, are due to their combinatorial complexity, are due to their partial axiomatisability, or are due to their practical treatment and enforcement. We start first with a systematisation of constraint handling in Section 2. It is surprising that some of the main problems of database theory remains open after almost two decades of research [Tha87]. Some few of them are reviewed in Section 3. At the same time, database theory has also resulted in jewels and background knowledge that here become folklore. We review some of the background results in Section 4 and apply them in Section 5. Since this paper does not intend to survey all papers in the area we refer the reader to [Tha00] for all definitions of notions.
2 Systematic Constraints Deployment In current practice constraints are declared at the conceptual level. These constraints are transformed to constraints at the logical level and then enhanced by control declarations or mapped to procedural representation. The enforcement and the application are left to the DBMS ... and result in a good number of problems. Moreover, structures and constraints are not handled in a holistic manner. We claim that this approach can be improved by an explicit framework and by a deeper consideration of concerns. 2.1 The Constraints Handling Framework Our framework for constraint handling is based on a four level model: 1. The specification level is used for description of the constraints with their schema context. The description consists of a specification of the constraint property, the
46
B. Thalheim
validation, and the policies for evaluation. It can be extended by specific policies for the various development methods such as agile development, by transformations of constraint properties to others, and by associations among constraints. Finally we may derive constraints for the application of the constraint. 2. The control or technical level treats the application of the constraint. It is based on the constraint property portfolio. We develop techniques and methods for applying constraint checks and derive the constraint evaluation plan. 3. The application or technology level handles the management of constraint handling within software projects based on the technology of development. 4. The establishment or organisational level is based on a methodology and may be supported by a constraint maintenance system. Classical research has mainly considered the first level. Implementation approaches have been tackling the second level by explicit specification of policies such as cascade, restrict, no action, defaultify and nullify. Level three is typically handled in an implicit form by some coherence among triggers and transactions. This four-level framework for constraint management can be extended by level five that provides facilities for handling satisfaction of constraints and for predicting changes of satisfaction whenever software is evolving. Level six integrates the constraint management into the optimisation of the software development process. Level seven uses experiences gained for innovation and adaptation of other processes and products that have not yet reached this maturity. Constraint can be oriented towards products of the database and software development or towards the processes of the database and software development. This distinction is reflected by a distinction into product-oriented constraints and process-oriented constraints. The constraints of the first kind are static one, the constraints of the second kind are transitional or dynamic ones. The second kind must be applied to basic or generic activities used within development processes. Most of these processes are refinement steps. Therefore, we may distinguish between faithful (or constraintpreserving) refinement activities and those which must be extended by constraint maintenance or support activities. 2.2 Implicit Model-Inherent Integrity Constraints Each structure used in a database schema is also based on a set of implicit modelinherent integrity constraints: Component-construction constraints are based on existence, cardinality and inclusion of components. These constraints must be considered in the translation and implication process. Identification constraints are implicitly used for the set constructor. Each object either does not belong to a set or belongs only once to the set. Sets are based on simple generic functions. The identification property may be, however, only representable through automorphism groups [BT99]. We shall later see that value-representability or weak-value representability lead to controllable structuring.
Integrity Constraints in (Conceptual) Database Models
47
Acyclicity and finiteness of structuring supports axiomatisation and definition of the algebra. It must, however, be explicitly specified. Constraints such as cardinality constraints may be based on potentially infinite cycles. Superficial structuring leads to representation of constraints through structures. In this case, implication of constraints is difficult to characterize. 2.3 Separation of Concerns Let us discuss three overlooked issues for constraints: (1) Some constraints may combine too many different meanings at the same time. This overburdening limits reasoning on constraint sets. (2) Constraints might have very different meaning for different stakeholders of an application. These different meanings may not be harmonically given within a singleton constraint. (3) Finally, some constraints are much better adopted to some database model languages and unnatural for others. Kinds of Functional Dependencies. Functional dependencies are the most well-known class of database constraints and commonly accepted. They are one of the most important class of equality-generating constraints. Given a type R and substructures X, Y of R. The functional dependency R : X −→ Y is valid in RC if o|Y = o |Y whenever o|X = o |X for any two objects o, o from RC . The following axiomatization is correct and complete for functional dependencies in the EER [HHLS03] for substructures X, Y, Z of the EER type R: Axioms: Rules:
R : X R Y −→ Y R : X −→ Y R : X −→ X R Y
R : X −→ Y, R : Y −→ Z R : X −→ Z
R : X −→ Y R : R [X] −→ R [Y ] The type R denotes a component type of R if R is a relationship type of order i and R is of order i − 1. Functional dependencies carry at least four different and interwoven meanings. The notion of the functional dependency is thus overloaded. It combines different properties that should be separated: Explicit declaration of partial identification: Functional dependencies are typically explicitly declaring a functional association among components of types. The left Ident hand attributes uniquely identify right side attributes, i.e., X −→ Y . Identification can either be based on surrogate or on natural attributes. Tight functional coupling: Functional dependencies may also be numerical constraints [Tha00]. Numerical constraints restrict the number of Y -values for each X-value. N um We denote such constraints by i.e., X −→ Y . Another denotation is based on cardinality constraints. Semantic constraint specific for the given application: Constraints may be stronger than observed in usual life since the application has a limited scope and allows us to strengthen the constraint. In this case, constraints restrict the application only to those
48
B. Thalheim
cases in which the left side has only one associated right side value despite that this Sem restriction may not be valid for any application. We denote this case by X −→ Y . Semantical unit with functional coupling: Semantical units are those reducts of a type that are essential in the given application. Their components cannot be separated without loosing their meaning. Semantical units may have their inner structure. This structure tightly couples dependent object parts to those that are Unit determining them. We denote this coupling by X −→ Y . Structural association among units: Semantical units may allow a separation of concerns for certain elements. Their separation supports a more flexible treatment while requiring that the dependent part cannot exist without the determining part. If Struct this dependence is functional we may represent such by the constraint X −→ Y . Example 1. Let us consider an abstract example with the functional dependencies {A} −→ {B, C} and {B} −→ {D}. TA A
C
(1,1)
TA .to.TB
B
TB D
The functional dependency {A} −→ {C} declares an explicit and direct dependency among the attributes. The functional dependency {A} −→ {B} is an inter-type constraint and leaves the scope of type TA . These two dependencies are completely different and need different support mechanisms. The five cases may be observed for the following instantiations of this example:
Explicit declaration Tight coupling Semantic constraint Semantical unit Structural association
instantiation TB = StudyProgram , B = ProgramCode, D = ProgramName TA = Student, TA .to.TB = MajorProgram, TB = StudyProgram TB = StudyProgram , B = ProgramCode, D = ResponsibleProfessor TB = StudyProgram , B = ProgramCode, D = ProgramDegree TA = Student, TB = RegisteredStudent, A = PersonID, B = StudentNr
Let us consider, for instance, that the type TB represents a study program, and the attributes B represent a program code, D the program name. The functional dependency {B} −→ {D} is an explicit key dependency. If the attribute D represents the professors that are responsible program then we might assume or not assume {B} −→ {D}. This depends on the given application. Therefore the dependency is a semantical constraint. From the other side, if the attribute D represents the program degree then this degree cannot be separated from the values for the program code. In this case the attributes form a semantical unit. Furthermore, in most universities a student is registered for one and only one major degree. The student must have a major program. In this case, the two types Student and StudyProgram are thus tightly coupled. Finally, we can separate the student personal data from the student university data. The person identification number is structurally associated to the student number.
Integrity Constraints in (Conceptual) Database Models
49
This separation results also in a different meaning of derivation rules. For instance, the augmentation rule has a different meaning depending on the kind of functional dependency: Trivilisation of identification
Ident
R : X −→ Y R Y Ident
R : X R X −→ Y
Adaptation of semantics scope
Sem
R : X −→ Y R Y Sem
R : X R X −→ Y
We may also use the Armstrong axiomatisation for derivation of functional dependencies of a given kind. The following two rules are examples of refined rules: Ident
Sem
R : X −→ Y, R : Y −→ Z Ident
R : X −→ Z
Sem
Ident
R : X −→ Y, R : Y −→ Z Sem
R : X −→ Z
Kinds of Key Dependencies. A key dependency or simply key X is a functional dependency R : X −→ R . A key is called minimal if none of its proper substructures forms a key. The set of all minimal keys of R is denoted by Keys(R). We notice that this set type E with n atomic attributes may may be very large. For instance, an entity n have nn minimal keys which is roughly c2√n . 2 The key concept reflects a variety of descriptive meanings depending of the level of consideration. Let us distinguish between external, conceptual and internal levels. The meaning of the key concept can be based on the uniqueness property (each object is unique with this key in the database), the identification property (each object can be identified by its key values), the existence property (for each object there exist key values), the integrity property (the database satisfies a key constraint), and the accessibility property (the properties of an object can be accessed through the key values of the object). These meaning are not equally important at all levels: – At the language definition level, the uniqueness concept is used to express the key property. The identifier, existence, integrity and accessibility functionalities are not considered. – The uniqueness concept is inherited by the external level and expressed mainly via the identification property. The existence property plays a secondary role. The integrity and accessibility concepts are not mainly under consideration. – At the conceptual level, the uniqueness concept of the language definition level is inherited by the identification and the integrity property. In order to express the identification concept, the surrogate concept is introduced. The key dependency is the integrity constraint usually used to express the key property. – At the internal level, the uniqueness concept is completely covered by the identification concept. From the external level the existence and uniqueness concepts are inherited and used in implementation restrictions like the requirement of invariant values in keys. The internal levels also inherit the integrity concept of the conceptual level and uses this concept for the accessibility property of the key. These descriptive meanings of the key concept are pictured in Figure 1. Beside the descriptive meanings we may also use implementation concepts such as pointers and pragmatic meanings such as naming.
50
B. Thalheim language definition level
uniqueness external level
identification
~
conceptual level identification (surrogate)
U integrity constraint (key dependency)
internal level
j identification z accessibility - invariance
existence
Fig. 1. Descriptive meanings of the key concept depending on the levels
Kinds of Multivalued Dependencies. It is often claimed that multivalued dependencies are difficult to model, to teach, to learn and to handle. They are introduced in almost any database book. The classical introduction is based on the tuple-generating definition of the multivalued dependency X → → Y |Z . It requires that whenever two tuples have the same value on the left-hand side then there also exists a tuple in the relation class which matches to the first tuple by the left-hand side and the first element of the right-hand side and which matches to the second tuple by the left-hand side and the second element of the right-hand side. This definition has the clarity of a mathematical definition and the problematic treatment often observed for other mathematical constructs. There are however five other definitions for multivalued dependencies that are easier to understand, to handle, and to represent. We may group these definitions as follows: Derivation constraint: The classical definition is based on the above mentioned tuple generating property. Another definition uses the construction approach for hinged Cartesian products and binds the validity of the multivalued dependency to the equality on query expressions (σX=x (RC ))[Y ∪ Z] = (σX=x (RC ))[Y ] × (σX=x (RC ))[Z] is valid for Y, Z with Y ∩ Z = ∅ for all x ∈ RC [X] . Structure constraint: The structure-oriented definition can be given through – the decomposition of the original class into two classes, i.e., the original class can be reconstructed by applying the natural join to the projection of RC to X ∪ Y and X ∪ Z respectively or – the potential structuring by a nested relation class νY (νZ (RC ))
X {Y } {Z} A1 ... Ak Ak+1 ... Am Am+1 ... An ... ... ... ... ... ... ... ... ...
or
– the potential restructuring in an ER model in the following form Independence concept: The original definition [Cad76] uses relative independence of Y -components from Z-components in a relation class, i.e., the values of the X-
Integrity Constraints in (Conceptual) Database Models
Y
XY
-
X
(1,n)
(1,n)
XZ
-
51
Z
components determine the values of Y-components of objects in RC independently from the values of Z-components, ( (σX=x (RC ))[Y ] = (σ(X=x)∧(Z=z) (RC ))[Y ] for all x-values x ∈ RC [X] and all z-values z ∈ RC [Z]). The generation definition is certainly the most mathematical and the least useful among the definitions. The construction definition provides a facility for checking validity. Both definitions also demonstrate that the relational model language is not appropriate for reasoning on these constraints. The independence notion is the simplest for modelling and for considering. The structure definitions are certainly the best way for the understanding of these constraints. The ER approach is the best suited model language for multivalued dependencies. Example 2. Let us consider a relationship type EmployeeAssociation defined on the entity types: StaffMember, DependentPerson, Project, Supplier, Product with the following multivalued dependencies: { StaffMember } → → { Department, DependentPerson }|{ Project, Product, Supplier } { StaffMember } → → { DependentPerson }|{ Department, Project, Product, Supplier } { Project } → → { StaffMember, Department, DependentPerson }|{ Product, Supplier } { Product } → → { Department, StaffMember, DependentPerson, Project }|{ Supplier } .
The ER model allows us to derive a new structuring by the decomposition of the relationship type. We obtain the schema in Figure 2 for which a new type ‘Working’ reflects the separation of concern to PP roduct, Department Staf f Member and the independence of suppliers and projects according to the multivalued dependencies.
Department
- StaffMember
Working
Dependent
0..1 0..1 Project of Product
-
?
Product
Supplier of Product
Fig. 2. The decomposition of the relationship type based on multivalued dependencies
3 Difficult Unsolved Problems 3.1 Global versus Local Normalisation Normalisation is typically carried out on the basis of one database type. This type is normalised (e.g. decomposed, split or reduced) according to a set of integrity constraints.
52
B. Thalheim
The association and the influence of this normalisation to other types is typically neglected. Therefore, normalisation is typically local. Local normalisation of a singleton database type is well reflected in most database books (e.g., [AHV95, Bis95, Leo92, Yan86]) and publications, most database courses, and in actual database practice. It is considered as one of the pearls of database research and known to almost everybody who knows database technology. The provenance and acknowledgement is based on the facility it provides: keeping as much as possible locally and globally supporting only those processes that are inherently global. Both independence concepts of databases (conceptual independence and implementation independence) are based on localisation. Local normalisation of database structures aims in derivation of such structures of databases that can easily be supported by the DBMS. In the past DBMS have been supporting keys, domain constraints and key-based inclusion constraints. Therefore, it is a goal to derive another equivalent schema to the given one which has an equivalent but supportable set of integrity constraints. This approach can be understood as a procedural approach to optimisation of database structuring depending on the platform for implementation. Normalisation is typically considered to be vertical normalisation. Deductive normalisation and horizontal normalisation are alternatives to vertical normalisation. Horizontal normalisation [PBGG89] is based on selection and union. Horizontal normalisation uses selections based on predicates α1 , ..., αn which n may be pairwise exclusive (αi → ¬αj , i = j) and cover the truth value ½ (( i=1 αi ) → ½). Horizontal normalisation also allows us to separate the part of a set for which a dependency is valid from the part that invalidates a dependency. For instance1 , αX−→Y = (o ∈ RC → ¬∃o ∈ RC (o[X] = o [X]∧o[Y ] = o [Y ])) separates those objects in RC for which the functional dependency is valid from those which invalidate the functional dependency. Deductive normalisation [Tha91] is based on reduction and extended selection. Deductive normalization reduces relations to those elements that cannot be generated from the other elements by generation rules. It is the most storage effective and the best computational method for normalisation as long as the tuple-generating dependency used for decomposition is acyclic. Horizontal and deductive normalisation methods have not yet received a support from the database systems vendors. Local normalisation must however take into account these three kinds of normalisation. Global normalisation aims in normalisation of the schema as a whole. It must take into account the three kinds of local normalisation. Global normalisation has not got an appropriate attention in research despite the interest in implementations. Therefore, a systematic treatment of this normalisation has not yet been given in the literature. 3.2 Optimal Synthesis Algorithm Classical normalisation algorithms usually start from a given set of dependencies and generate one final normalisation of the given schema to a number of schemata in a target normal form. These normalisation suffer from three drawbacks: 1
We use the symbol R for type or class specification and denote the class of elements of the type by RC . Tuples (in the case of object-relational models) or elements of RC are denoted by o. X −→ Y is a functional dependency.
Integrity Constraints in (Conceptual) Database Models
53
Deterministic solution: All known normalisation algorithms produce a different output in dependence of the order of the attributes and of constraints if there are several minimal covers for the same set of integrity constraints. Which order is the best must be determined through trial and error methods. Normalisation is neither confluent nor Church-Rosser. Optimal solution: Since the resulting normalisation depends on the order, the result might not be the optimal normalisation. At present, normalisation is not optimised against behaviour of the database for the given application. Completeness of specification: Normalisation algorithms are very sensitive to incompleteness of constraint specification. Since the number of independent functional dependencies in a relation schema with n attributes is exponential in n the constraint acquisition becomes the major bottleneck for normalisation. It is typically not achievable to collect all constraints that are valid for a certain database. Imagine to collect about 500 independent functional dependencies for a relation schema that has 11 attributes. Unnatural structural clusters: Most synthesis algorithms group all attributes that can be implied from a set of attributes. This collection procedure by left sides of functional dependencies unifies different concerns that are represented by functional dependencies based on the implication rule X → Y , X → Z |= X → Y ∪ Z . The meaning in the application and the treatment in reality of these two functional dependencies might however be completely different. 3.3 Treatment of Almost Valid Constraints Constraints may be valid to a large extent and invalid for exceptional cases. The classical approach to constraints handling proposes to use a holistic constraint that covers both cases. In most cases, this approach results in weakening the original constraint so that the exceptional cases are covered as well. There are however approaches that allow one to handle this case: – Conditional constraints [PBGG89, Tha91] bind the validity of constraints to those cases in which the conditions are valid 2 . – Deontic constraints [Tha00] separate between the normal case and the holistic case. Specific deontic constraints are the exception handling constraints in [BT07]. – State-dependent dynamic constraint sets [Tha00] bind the validity of constraints to general states of the database. 3.4 Consideration of Constraint Sets Instead of Singleton and Isolated Constraints Classically, database constraints are developed one by one. Finally, a constraint set occurs that is difficult to comprehend, to revise, to handle, to extend and to maintain. Moreover, constraints of different kinds cannot be considered together. This impotence is partially caused by non-axiomatisability, e.g. functional and inclusion dependencies. It is also caused by the complexity of constraint sets. Cardinality constraints in 2
These constraints have been reinvented for XML and are also called conditional functional dependencies [BFGM08].
54
B. Thalheim
ER models use an overloaded concept and thus do not have an axiomatisation. They combine a lower bound and an upper bound. For instance, a cardinality constraints card(R, R ) = (1, 1) represents at the same time a functional dependency and an inclusion dependency. It is well known [LL99], that an axiomatisation exists for functional dependencies and inclusion dependencies if functional dependencies and inclusion dependencies do not interact. If they are however inter-twinned then there is no Hilberttype axiomatisation [Tha91]. Moreover, cardinality constraints are specified locally but have global effects [Tha00]. It is thus a nightmare or at least an intellectual challenge to understand a large constraint set. We introduce in Section 4.2 a solution that allows one to handle constraint sets at least for sets of functional dependencies. 3.5 The Expressive Power and Language Relativity Database models are based on a database modelling language that has been selected for specification. Languages may however also restrict modelling. Language expressiveness is an important criterion in language design and it is somewhat different from any of the classical criteria for ‘good’ languages such as availability, understandability, efficiency, verifiability, and portability. The principle of language relativity postulates that languages have their own deficiency, i.e., some concepts of other languages may not be adequately represented in the language that has been chosen. A database modelling language is expressive for a developer or programmer if (s)he can identify a direct representation of concepts with which (s)he reasons about the given application. This language relativity leads to problematic or inadequate models or limits the representation of things. It is not well understood. Typical examples are existence constraints investigated below.
4 Background Theories and Results Research on integrity constraints has resulted in a rich body of knowledge that is known to specialists but has not found its way to practice, to educational books, to implementations, and to a foundation of database culture. At the same time, some results are considered to be so obvious that they are not mentioned in literature nowadays but used at the same time based on cultural background. This section aims in drawing attention to those pearls. Background theorems are currently known among people, are old statements whose authorship is difficult to detect, are combinations and abstractions of a bunch of theorems and form some commonly accepted culture. They are popular and neglected at the same time. 4.1 The Power of Simplicity Functional and multivalued dependencies are a very powerful and simple class of integrity constraints. It is well-known that their reasoning and deduction can be based on propositional formulas [Tha91]. At the same time, a simple and powerful axiomatisation can be given for simple dependencies and their negations. As an example we give an axiomatisation for functional and negated functional dependencies:
Integrity Constraints in (Conceptual) Database Models
55
Axioms XY → Y
Rules (1) (4)
X −→ Y X −→ Y , Y −→ Z X −→ Y , X −→ / Z (2) (3) XV W −→ Y V X −→ Z Y −→ / Z
X −→ / Y XZ −→ / YZ X −→ Z , X −→ / YZ Y −→ Z , X −→ / Z (5) (6) (7) X −→ / YZ XZ −→ / Y X −→ / Y X −→ / Y
4.2 The Power of Graphical Reasoning FD Graphs. Functional dependencies are typically treated in a linear fashion. Graphical representation can however be used for reasoning on constraints as well. It is a folklore approach that deserves more attention. Example 3. Given the set UR = {A, B, C, D, E} of attributes and the set σR = {A −→ E, B −→ E, CE −→ D} of functional dependencies. This set can be represented by the following graph: This graph is easier to survey, simpler for reasoning and A
*
B
E
CE
- D
C
contains the full information on σR . FD graphs can be axiomatised. This axiomatisation mainly follows the rule system of the Armstrong axiomatisation except that right hand side attributes are limited to singleton attribute sets. This axiomatisation [DMT04] provides the first axiomatisation for FD graphs (Y denotes a set; A, B, C denote attributes): (S)
Y →B YC →B
(T )
Y → A, Y A → B Y →B
(P )
Y C → B Y → B
Y → A, Y → B Y A → B, Y → B (R) ()¬(Y → B, Y → B) Y A → B Y → A () is a formalization of ’→’ being the negation of ’→’, i.e., ¬() can be deduced starting with contradictory sets of constraints. (Q)
Theorem 1. [DMT07] The ST implication system over graphs of functional dependencies with rules (S) and (T) and no axioms is sound and complete for functional dependencies. The PQRST implication system over graphs of functional and negated functional dependencies with all the presented rules and the symbolic axiom (), which is used for indicating contradiction, is sound and complete for functional dependencies and their negations if they are not conflicting.
56
B. Thalheim
At the same time reasoning on graphs is more appropriate also for normalisation. Yang [Yan86] has discussed a reduction algorithm that allows a simple generation of canonical covers and of minimal covers for a set of integrity constraints. We can also use graphs of functional dependencies for detection of normal forms that are better than those generated by classical synthesis algorithms. Synthesis algorithms blindly collect all dependencies with the same left side at the final step. Using graphical reasoning we may collect only those dependencies that are not covered by any other group. Example 4. Given: the set UR = {A, B, D, F, G, I} of attributes and the set σR = {A −→ IG, D −→ F G, IAB −→ D, IF −→ AG} of functional dependencies. This set can be represented by the graph in the left picture. The graph in the right picture
I IF
* G 6
* A
- D
ABI F
I
* G 6
* A
IF
- D
AB F
displays its reduction to a minimal cover. Classical synthesis algorithms generate the following relation schemata: R1 R2 R3 R4
= ({A, G, I}, {A −→ GI}) = ({A, F, I}, {A −→ I, F I −→ A}) = ({A, B, D}, {AB −→ D}) = ({D, F, G}, {D −→ F G})
This normalisation is however not minimal. Instead of R1 we can use R1 = ({A, G}, {A −→ G}). The relation schema R1 is obtained through the relation schema R2 in combination with R1 . R2 is not in BCNF. It cannot be split into two relation schemata. The graph in the picture directly allows us to detect this better normalisation since the dependency A −→ I is covered by R2 . This example illustrates the power of graphical reasoning. Normalisation is typically driven by goals such as being dependency preserving, providing the basis for key-based integrity maintenance (BCNF) or being redundancy-free (3NF) at least at the attribute level or avoiding maintenance problems (e.g., insert anomaly) or avoiding data losses (e.g., delete anomaly) or occurrence of performance traps (e.g., update anomaly). It might be extended to such criteria as being redundancy-free at the value level. Textbooks typically dont’t mention that normalisation also introduces additional constraints. Example 5. The last example results in pairwise inclusion constraints such as the multiple value occurrence bindings R1 [A] ⊆⊇ R2 [A] ⊆⊇ R3 [A]. This inclusion dependency set adds an overhead to integrity maintenance. The functional dependencies are
Integrity Constraints in (Conceptual) Database Models
57
directly represented through key dependencies. At the same time, any change to Avalues in one of the relations must be harmonised with changes to A-values in the other relations. FD Set Graphs. Demetrovics, et al., [DMT07] introduce graphical reasoning for sets of constraints. We may represent validity and invalidity of functional dependencies by a graph. Let us illustrate this approach for relational schemata with three attributes. Visualisations may be based on more complex figures, e.g. a hexagon for six attributes. To do so we distinguish two kinds of functional dependencies and display them as follows in Figure 3: Functional dependencies of the form {A} → {B, C} can be decomposed to canonical functional dependencies {A} → {B} and {A} → {C}. They are represented by endpoints of binary edges in the triangle representation. Functional dependencies with two-element left-hand sides {A, B} → {C} cannot be decomposed. They are represented in the triangle on the node relating their right side to the corner. (1)
C
A
(2)
B
C
A
(3)
B
C
A
× × × B
Fig. 3. Examples of the triangle representation. From left to right: (1) The functional dependency {A} → {B} (filled circle at B side (right side of the FD) of the AB edge) and the implied functional dependency {A, C} → {B} (circle around node B in the ABC triangle). (2) The functional dependencies {A} → {B}, {B} → {C} and their implied functional dependencies {A} → {C}, {A, B} → {C}, {A, C} → {B}. (3) The negated functional dependency {A, C} → {B} (crossed filled circle around node B in the ABC triangle) and the implied negated functional dependencies {A} → {B} (crossed circle at B side of the AB edge) and {C} → {B}.
This notation has the advantage that we are supported for reasoning on potential validity of constraints as well. We may represent also candidates for excluded functional dependencies by crossed circles for the case that we know that the corresponding functional dependency is not valid in applications or by small circles for the case that we do not know whether the functional dependency holds or does not hold. Since the ST implication system is sound and complete for non-trivial canonical functional dependencies, rules (S) and (T) can be used for deriving all implied functional dependencies given an initial set. Moreover, the PQRST implication system forms a sound and complete system for both positive and negative (excluded) non-trivial singleton functional constraints, rules (P), (Q) and (R) can be applied as complements of rules (S) and (T) when excluded functional constraints are taken into account. These rules can be interpreted in terms of the graphical representation as well. A deduction step using one of them deals with a node of a higher-dimension object (e.g., a triangle as a two-dimensional object with one of its three vertices) and one or two of its borders (with one dimension lower, e.g., edges of the same triangle as onedimensional objects). Graphical versions of rules are shown on Figure 4 for the triangle
58
B. Thalheim
Fig. 4. Graphical versions of rules (P), (Q), (R), (S) and (T) in terms of the triangle representation. The small black arrows indicate support (necessary context) while the large grey arrows show the implication effects.
representation (case Y = {C}). The large grey arrows indicate the implication effect of each rule. Rule (S) is a simple extension rule and rule (T) can be called the “rotation rule” or “reduction rule”. We may call the left-hand side of a functional dependency the determinant of it and the right-hand side the determinate. Rule (S) can be used to extend the determinant of a dependency resulting in another dependency with one dimension higher, while rule (T) is used for rotation, that is, to replace the determinate of a functional dependency by the support of another functional dependency with one dimension higher (the small black arrow at B indicates support of AC → B). Another possible way to interpret rule (T) is for reduction of the determinant of a higher-dimensional dependency by omitting an attribute if a dependency holds among the attributes of the determinant. For excluded functional constraints, rule (Q) acts as the extension rule (i.e., it needs the support of a positive constraint, i.e., functional dependency) and (R) as the rotation rule (which needs a positive support too). These two rules can also be viewed as negations of rule (T). Rule (P) is the reduction rule for excluded functional constraints, with the opposite effect of rule (Q) (but without the need of support). Rule (Q) is also viewed as the negation of rule (S). 4.3 The Power of Equivalence Class Reasoning Equality Set Characterisation of Functional Dependencies. Validity of functional dependencies in relation classes can also be characterised on the basis of equality sets. Given a relation schema R = (UR , ΣR ) and a relation class RC . Let ERC (A)) = { {t ∈ RC |t =A t } | t ∈ RC } for A ∈ UR and ERC (X)) = { {t ∈ RC |t =X t } | t ∈ RC } for X ⊆ UR for predicates t =A t and t =X t which are true iff t and t have equal A- or X-values, respectively.
Integrity Constraints in (Conceptual) Database Models
59
Lemma 1. Given a relation schema R = (UR , ΣR ) and a relation class RC . RC |= X −→ Y for a functional dependency defined on R iff ERC (X) ⊆ ERC (Y ) . We observe a number of properties for equality sets: (Ru 1) If E1 ⊆ E3 and E2 ⊆ E3 then E1 ∪ E2 ⊆ E3 . (Ru 2) If E1 ∩ E4 ⊆ E2 and E3,1 ∪ ... ∪ E3,k ⊆ E4 then (E3,1 ∪ ... ∪ E3,k ) ∩ E1 ⊆ E2 . (Ru 3) If E1 ⊆ E2 and E2 ⊆ E3 then E1 ⊆ E3 . (Ru 4) If E1 ⊆ E2 and E1 ⊆ E3 then E1 ⊆ E2 ∩ E3 . Equality sets form equivalence relations for tuples of a relation class. They are therefore idempotent, associative for both ∪ and ∩, right-⊆-monotone for ∪ (i.e., E1 ⊆ E1 ∪ E2 ) and left-⊆-monotone for ∩ (i.e., E1 ∩ E2 ⊆ E1 ), and commutative for ∩ and ∪. The equality set system thus forms a lattice for any relation class. The lattice is however not distributive since the following two containments cannot be strengthened to equalities: (E1 ∩ E2 ) ∪ E3 ⊆ (E1 ∪ E3 ) ∩ (E2 ∪ E3 ) (E1 ∪ E2 ) ∩ E3 ⊆ (E1 ∩ E3 ) ∪ (E2 ∩ E3 ) We may now consider a lattice L(RC ) of equality sets of a relation class RC . Each relation class RC on R may have its lattice. These lattices may be incomparable. A lattice L(M, ∩, ∪, , ⊥, ≤) is called sub-cylindric if (Ru 1), (Ru 2), (Ru 3) and (Ru 4) are valid for elements from M. Lattice can be ordered by an embedding relationship . A lattice L1 is smaller than another lattice L2 if a homomorphism from L1 to L2 exist that is a congruence for the lattice operations. Let us consider lattices with M ⊆ 2UR for a relation schema R, ⊥ = ∅, and = UR . The element X denotes the equivalence relation for a subset X of UR . We call theses lattice UR -lattices. Let L(M, ∩, ∪, , ⊥, ≤) be the smallest sub-cylindric lattice that obeys X ≤ Y in LR if ΣR |= X −→ Y . The existence proof is based on the finiteness property of UR . We denote this lattice by LR . We observe a characterisation property of LR . Its proof is straightforward and omitted. Proposition 1. Given a relation schema R. LR L(RC ) for any relation class RC on R. Operations Defined on Singleton Relation Schemata and Validity of Functional Dependencies. One of the main background theorems for functional dependencies is the following statement [Mai83, Tha91]: Theorem 2. Functional dependencies are preserved for join operations, are selectioninvariant, are intersection-invariant, are restricted union-invariant, are not complement-invariant and can be checked on two-tuple relation classes. It is well-known that functional dependencies are not preserved for the projection operation. It is an open question whether there exists a sufficient and necessary condition for extensions of relation classes defined on a projection to the full class. One solution to this problem refers to the existence of a constant complement [AHV95].
60
B. Thalheim
Example 6. Given the set UR = {A, B, C, D, E} of attributes and the set σR = {A −→ E, B −→ E, CE −→ D}, the subset X = {A, B, C, D} of UR . The projec∗ tion of ΣR to X is Σ = {AC −→ D, BC −→ D}. We detect [Mai83] that any projection πX (RC ) of a class RC on R obeys the following curious dependency ∀.(R(a1 , b1 , c1 , d1 ) ∧ R(a2 , b2 , c1 , d2 ) ∧ R(a1 , b2 , c2 , d3 ) ⇒ d1 = d2 ). This dependency does however not characterise those relations in SAT ((X, Σ )) that have an extension in SAT (R). This claim can not be used for the characterization of those relations in SAT ((X, Σ )) for which an extension in SAT (R) exists. For instance, the relation class C RX = {t1 = (1, 3, 6, 9), t2 = (1, 4, 7, 10), t3 = (2, 4, 8, 11), t4 = (2, 5, 6, 12)} obeys the curious dependency but has no extensions in SAT (R). None of the triples matches to the equality condition of the premise. There is another solution that is simpler than the constant complement property. Let us denote the restriction of the UR -lattice L to X by L|X . It is not difficult to prove the following lemma. Lemma 2. Given a relation class RC . Then LR |X L(πX (RC )) is valid for the projection of the UR -lattice. This property is obvious. We thus omit the proof. It turns out that this property is also sufficient. Example 7. Let us first prove the non-extensionality of the relation given above. The relation R obeys Σ since ERCX (A) = {{t1, t2}, {t3, t4}}, ERCX (B) = {{t1}, {t2, t3}, {t4}}, ERCX (C) = {{t1, t4}, {t2}, {t3}}, and ERCX (D) = {{t1}, {t2}, {t3}, {t4}} and ERCX (A) ∩ ERCX (C) ⊆ ERCX (D) , ERCX (B) ∩ ERCX (C) ⊆ ERCX (D). But this relation violates ERCX (A) ∪ ERCX (B)) ∩ ERCX (C) ⊆ ERCX (D). To show that C by E-values ei for each tuple ti (1 ≤ i ≤ 4). Since ERC (A)∪ERC (B) ⊆ we extend RX ERC (E) for any relation class on R we conclude e1 = e2 = e3 = e4 . The requirement ERC (E) ∩ ERC (C) ⊆ ERC (D) is however not fulfilled. Therefore, this relation does not have any extension in SAT (R). The contradiction in the example above has been constructed on the basis of the subset property (E(A) ∪ E(B)) ∩ E(C) ⊆ E(D) for the equality sets. This property can also be represented by a formula that allows us to check the projection: ∀.(R(a1 , b1 , c1 , d1 ) ∧ R(a1 , b2 , c2 , d2 ) ∧ P (a2 , b2 , c3 , d3 ) ∧ R(a2 , b3 , c1 , d4 ) ⇒ d1 = d4 ) . Theorem 3. Given a relation schema R. LR L(RC ) for any relation class RC on R. The proof is based on the completion method used in the example above and the tableau reasoning for equalising introduced values. Corollary 1. For any projection of a relation schema to a subset of its attributes, a set of formulas exist that allows us to check whether a relation on the projection attributes has an extension to the full set of attributes.
Integrity Constraints in (Conceptual) Database Models
61
Compound FD Sets. We can use the equality set approach for an introduction of a new class of equality generating dependencies that characterise those relation classes which have an extension. A set expression on UR is defined by the closure of the recursive construction: (0) Any subset X of UR is a set expression on UR . (1) The expressions S S and S S are set expression on UR for set expressions S, S on UR . A compound functional dependency is a statement S −→ X for a set expression S on UR and X ⊆ UR . C It is valid in a relation class RC if ES ⊆ EY for the equality sets on UR . Compound functional dependencies can be derived from a set of functional dependencies using rules similar to (Ru 1) - (Ru 4). X −→ Y, Z −→ Y X Z −→ Y
X ∪ X −→ Y X X −→ Y
S1 Y −→ Z, S3,1 ... S3,k −→ Y X (S3,1 ... S3,k ) −→ Z
Let us denote the set of all implied compound functional dependencies of a set Σ of functional dependencies by CF D(Σ). The following observations are obvious: Proposition 2. (1) For a given set UR of attributes and a set of functional dependencies ΣR the following relationship is valid: A relation class RC satisfies ΣR iff it satisfies CF D(ΣR ). (2) For any set ΣR of functional dependencies there exists an equivalent set of compound functional dependencies with not more than |UR | elements. The last statement uses the fact that compound functional dependency sets have an equivalent set of compound functional dependencies which right hand side set is a singleton set. It is a simple exercise to show that the following set of derivation rules is sound and complete for compound functional dependencies: (S1 S2 ) (S3 S4 ) −→ Z (S1 S3 ) (S1 S4 ) (S2 S3 ) (S2 S4 ) −→ Z S1 −→ Y, S2 −→ Y S1 S2 −→ Y S −→ Y, Y −→ Z S −→ Z
S1 −→ Y S1 S2 −→ Y
S1 −→ Y S1 −→ Y, S2 −→ Y S1 −→ Y ∪ Z S1 −→ Y
S −→ Y, S −→ Z S −→ Y ∪ Z
Let CF D(ΣR )+ the closure of the set of compound functional dependencies according to the rules above. We can now define the projection πX (S −→ Y ) of a compound functional dependency S −→ Y to a subset X of UR by simply removing those components from the set expression that are not supersets of X and by computing the interaction of X and Y for the right hand side set. We can now summarise the main result of this subsection:
62
B. Thalheim
Theorem 4. Given a relation schema R = (UR , ΣR ), the set of compound functional C dependencies CF D(ΣR )+ and a subset X of UR . A relation RX on X has an extension C + C to a relation R iff the set πX (CF D(ΣR ) ) is valid in RX . This theorem allows us to conclude another interesting property of Boyce-CoddNormal-Forms that is a folklore statement and well-known to people working in practice: Corollary 2. If the relation schema R is in BCNF then any view (or projection) relation class of this schema can be extended to a relation class of the schema R. We note that 3NF schemata that cannot be decomposed into BCNF schemata can often be decomposed may be normalised by applying horizontal normalisation along the additional attribute or by splitting the implied attribute into one that is functionally equivalent to the implied and one that is implied by the key dependency. Example 8. The relation schema R2 in Example 5 can be sliced along F by horizontal decomposition according to the equality set ERC2 (F ). Another resolution of the nontransformability into two BCNF schemata is the split of the attribute A into A1 , A2 with the dependencies IF −→ A2 , A1 −→ I, I −→ A1 .
5 Application of the Background Theories and Results 5.1 BV Constraints and Their Axiomatisation Integrity constraints can be specified based on the B(eeri-)V(ardi)-frame, i.e., by an implication with a formula for premises and a formula for the implication. BV-constraints do not lead to rigid limitation of expressibility. If structuring is hierarchic then BVconstraints can be specified within the first-order predicate logic. We may introduce a variety of different classes of integrity constraints defined: Equality-generating constraints allow to generate for a set of objects from one class or from several classes equalities among these objects or components of these objects. Object-generating constraints require the existence of another object set for a set of objects satisfying the premises. A class C of integrity constraints is called Hilbert-implication-closed if it can be axiomatised by a finite set of bounded derivation rules and a finite set of axioms. It is well-known that the set of join dependencies is not Hilbert-implication-closed for relational structuring. However, an axiomatisation exists with an unbounded rule, i.e., a rule with potentially infinite premises. Since constraint sets may be arbitrary we might ask in which cases an axiomatisation exists. The derivation operator Γ of a deductive system Γ and the implication operator |= may be understood as closure operators Φ, i.e., (0) (i) (+)
Φ0 (Σ) = Σ Φi+1 (Σ) = {α ∈ C ∩ Φ(Φi (Σ))} Φ∗ (Σ) = limi→∞ Φi (Σ)
for any subset Σ from a class C of constraints.
Integrity Constraints in (Conceptual) Database Models
63
The closure operator Φ is called compact for a class C if the property α ∈ Φ∗ (Σ) implies the existence of a finite subset Σ of Σ such that α ∈ Φ∗ (Σ ). It is called closed of Φ∗ (Φ∗ (Σ)) = Φ∗ (Σ) for any Σ ⊆ C. The closure operator is called monotone if Φ∗ (Σ) ⊆ Φ∗ (Σ ∪ Σ ). The operator is reflexive if α ∈ Φ∗ (Σ ∪ {α}) for all formulas and subsets from C. Observation 1. The implication operator Φ∗|= is reflexive, monotone, closed and compact if and only if there exists a deductive system Γ such that ΦΓ and Φ|= are equivalent. If Φ|= additionally has the inference property, the deduction property and is generalization invariant then Φ∗Γ (∅) = Φ∗|= (∅) . If the deduction property fails then the axiomatisation by a deductive system may be based on some obscure rules similar to those for the axiomatisation of PROLOG. The structuring of the extended ER model [Tha00] allows us to deduct a number of properties. As an example we consider the axiomatisation of constraints generalizing those discussed in [Tha91]. We observe first that implication in the hierarchical predicate logic is reflexive, monotone, compact and closed. Let us consider classes of BV-constraints in HERM which form a cylindric algebra [Tsa89]. The order of constraints by Φ|= possibly can be based on the order of of premises and conclusions. In this case the constraint set forms a pair algebra. Observation 2. Cylindric classes are pair algebras. Examples of cylindric classes are the class of functional dependencies, the classes of Hungarian functional dependencies [Tha91], the class of inclusion dependencies and the class of multivalued dependencies. Further, the n-class of all ≥ n-functional dependencies X → Y whose left side contains at least n components and the class of rigid ≤ n-inclusion dependencies T1 [X] ⊆ T2 [X] whose component list contains at most n components form a cylindric constraint set. Usually, union does not preserve cylindric sets. Observation 3. Cylindric constraint classes are axiomatised by reflexivity axioms, augmentation and transition rules. The proof of this statement is based on Section 4.3. If an axiomatisation leads to reflexivity, augmentation and transitivity then union and decomposition rules can be deducted by the other rules. Transitivity may have to consider the specific influence of premises, e.g., transitivity for full multivalued dependencies is based on the root reduction rule [Tha91]. 5.2 Optional, Complex and Null-‘Valued’ Structures Often structures include also optional components. XML schemata and XML DTD allow such kind of open-world-schematalogy. Their design methodology must be different from what we have learned in semantic and object-relational models.
64
B. Thalheim
Let us denote the set of all components of a set O of objects by compon(O) and the set of all optional components of O by componopt (O). Similarly we denote the set of all components used in a constraint α by compon(α). Validity of constraints is either based on strong semantics requiring validity for all object sets independent of whether componopt (O) ∩ compon(O) = ∅ or on weak semantics requiring validity for constraints only for those object sets O for which componopt (O) ∩ compon(O) = ∅ . Classical validity is based on weak semantics which has a severe disadvantage: Observation 4. Weak semantics leads to non-additivity of constraints for object sets O with componopt (O) = ∅ , i.e., it is not true in general that O |= {α1 , ...., αm } is valid if and only if O |= {αi } for each constraint in {α1 , ...., αm } . Observation 5. Strong semantics leads to non-reflexiveness or non-transitivity of constraints for object sets O with componopt (O) = ∅ , i.e., O |= α → α for some constraints α or the validity of O |= α → β and O |= β → γ does not imply O |= α → γ . Constructors used for construction of more complex types are often used for convenience and representing a different structuring. A typical example is the application of the list constructor with the meaning of representing sets. In this case we must add a list-to-set axiom [Tha00] ∀t ∈ compon(o)∀i, j(type(o.i) = type(o.j) = t ⇒ value(o.i) = value(o.j)) . This axiom is often overlooked and not considered. Observation 6. Semantics for structures defined by the list constructor and representing set must be extended by list-to-set axiom. Since attributes are also constructed on the basis of constructors from base types we may ask whether this construction affects the definition of constraints and the axiomatisability. This question is open for most of the constraints. It has, however, been shown [Lin03] that keys and functional dependencies have a similar treatment as in the relational case. Substructures are, however, more complex and represented by the Brouwerian algebra of subcomponents. An example of an existence declaration is the NULL ‘value’. NULL is a special marker used in SQL to indicate that a value for an attribute of an object does not exist in the database. Its aim is a representation of “missing information and inapplicable information”. The NULL marker is one of the most overloaded concepts in database technology. [Tha00] shows that 20 different kinds of null markers might be considered for specification of a database schema. Since this existence declaration is interpreted as a value in SQL, NULL has led to controversies and debates because of its treatment by 3-valued logics, of its special usage requirements in SQL joins, and the special handling required by aggregate functions and SQL grouping operators. The results are: – Unnatural schemata: Values can be missing due to failing generation or can be intentional missing. A value may be not existent or unknown. A value existence depends also on the thing that is represented by an object.
Integrity Constraints in (Conceptual) Database Models
65
– Computational problems: In the presence of NULL markers, the behaviour of predicates in SQL depends on the DBMS. Predicates such as (NOT) IN, (NOT) EXISTS, aggregation functions, and NOT show an anomalous behaviour. Boolean connectives are evaluated to UNKNOWN in the case of NULL markers. – Misuse of logics: SQL follows the semantics of predicate logic also in the case of missing values and thus cause a number of problems: The interpretation of null ‘values’ is not easy to adapt to a particular intuition; the meaning of relational operations when applied to both null values and certain data values is not well-defined; and the treatment of integrity constraints in databases containing null values is not completely defined. NULL markers can however be explicitly incorporated into SQL [Tha10]. Alternatively, we need to overcome the schema deficiency and to define away NULLs from relational schema whenever possible. NULL ‘values’ are representing additional semantical information within a tuple that must be taken into consideration. 5.3 General Normalisation and Optimisation Based on the axiomatisations discussed above we may introduce a general vertical decomposition form: Given a schema structuring S = (ER, ΣS ). A vertical decomposition of S is given as a mapping τ from S to S which is defined by projection functions. The decomposition is lossless if a query q on S can be defined such that for each db on S the equality q(τ (db)) = db is valid. Let further Σ be the set of those constraints from Φ|= (Σ) which are entirely defined on the structures in S . A decomposition based on projection is called C-constraint preserving if Σ ⊆ Φ|= (Σ ) . A classical example of vertical decompositions are decompositions of relations to relations in the third normal form. We may now introduce a general class of C-decomposition algorithms: Construct basic elements which are undecomposable. Derive maximal elements by backward propagation of augmentation. Reduce redundancy in the constraint set by backward propagation of transitivity. Derive a left-right graph by associating conclusions of a constraint with the premise of another constraint. Combine all minimal left sides of constraints which are not bound by another constraint to a group. Derive projections based on all groups in the graph. The first step of the decomposition algorithm is only introduced for convenience. This algorithm is a generalization of the classical synthesis algorithm. Observation 7. The C-decomposition algorithm leads to C-constraint preserving decomposition if the class C is cylindric.
66
B. Thalheim
6 Summary and Conclusion This paper revisits and restructures the theory of integrity constraints. Integrity constraints are widely used for normalisation of database structures, for derivation of integrity maintenance procedures, for prohibition of value inconsistencies in databases and for computational support of database operations. They led to a large body of knowledge that is not entirely covered by surveys, textbooks or monographs. We start with a reconsideration of classical problems and detect that we need a systematic treatment of integrity constraints. Therefore, a framework for integrity constraint handling must be developed. Implicit integrity constraints form an essential part of constraint specification since structures and constraints are intertwined. Although constraint theory is very rich3 , some of the main research issues are still open research problems. The most important problem is the global normalisation problem. It is related to revision of synthesis algorithms and to treatment of almostvalid constraints. Constraint specification typically leads to a specification of sets of constraints from various constraint classes. Research has either considered constraint classes in isolation or interference of entire constraint classes. The set aspect is not yet considered. The constraint literature has also led to combined results and pearls. We discuss in this paper four pearls: simplicity of reasoning, graphical reasoning based on visualisation techniques, reasoning based on equivalence classes, sophisticated specification of constraints based on explicit consideration of their meaning. These pearls allow a general theory of reasoning for constraints in arbitrary database models, provide a treatment of optional structures and support intelligent optimisation and normalisation approaches.
References [AHV95] [BFGM08]
[Bis95] [BT99] [BT07] [Cad76]
[DMT04]
3
Abiteboul, S., Hull, R., Vianu, V.: Foundations of databases. Addison-Wesley, Reading (1995) Bravo, L., Fan, W., Geerts, F., Ma, S.: Increasing the expressivity of conditional functional dependencies without extra complexity. In: ICDE, pp. 516–525. IEEE, Los Alamitos (2008) Biskup, J.: Foundations of information systems. Vieweg, Wiesbaden (1995) (in German) Beeri, C., Thalheim, B.: Identification as a primitive of database models. In: Proc. FoMLaDO 1998, pp. 19–36. Kluwer, London (1999) Berztiss, A., Thalheim, B.: Exceptions in information systems. In: Digital Libaries: Advanced Methods and Technologies, RCDL 2007, pp. 284–295 (2007) Cadiou, J.-M.: On semantic issues in the relational model of data. In: Mazurkiewicz, A. (ed.) MFCS 1976. LNCS, vol. 45, pp. 23–38. Springer, Heidelberg (1976) Demetrovics, J., Moln´ar, A., Thalheim, B.: Graphical reasoning for sets of functional dependencies. In: Atzeni, P., Chu, W., Lu, H., Zhou, S., Ling, T.-W. (eds.) ER 2004. LNCS, vol. 3288, pp. 166–179. Springer, Heidelberg (2004)
More than 40 pages of references in [Tha00] are directly related to constraints.
Integrity Constraints in (Conceptual) Database Models [DMT07]
[HHLS03]
[Leo92] [Lin03]
[LL99] [Mai83] [PBGG89] [Tha87] [Tha91] [Tha00] [Tha10] [Tsa89] [Yan86]
67
Demetrovics, J., Molnar, A., Thalheim, B.: Graphical axiomatisation of sets of functional dependencies in relational databases. In: Alkalmazott Matematikai Lapok, vol. 24, pp. 223–264 (2007) Hartmann, S., Hoffmann, A., Link, S., Schewe, K.-D.: Axiomatizing functional dependencies in the higher-order entity-relationship model. Inf. Process. Lett. 87(3), 133–137 (2003) Leonard, M.: Database design theory. MacMillan, Houndsmills (1992) Link, S.: Consistency enforcement in databases. In: Bertossi, L., Katona, G.O.H., Schewe, K.-D., Thalheim, B. (eds.) Semantics in Databases 2001. LNCS, vol. 2582, pp. 139–159. Springer, Heidelberg (2003) Levene, M., Loizou, G.: A guided tour of relational databases and beyond. Springer, Berlin (1999) Maier, D.: The theory of relational databases. Computer Science Press, Rockville (1983) Paredaens, J., De Bra, P., Gyssens, M., Van Gucht, D.: The structure of the relational database model. Springer, Berlin (1989) Thalheim, B.: Open problems in relational database theory. Bull. EATCS 32, 336– 337 (1987) Thalheim, B.: Dependencies in relational databases. Teubner, Leipzig (1991) Thalheim, B.: Entity-relationship modeling – Foundations of database technology. Springer, Berlin (2000) Thalheim, B.: NULL value algebras and logics. In: Proc. EJC 2010, Jyv¨askyl¨a, pp. 365–376 (2010) Tsalenko, M.S.: Modeling of semantics for databases, Nauka, Moscov (1989) (in Russian) Yang, C.-C.: Relational Databases. Prentice-Hall, Englewood Cliffs (1986)
Modeling Techniques for Multi-level Abstraction Bernd Neumayr1 , Michael Schrefl1 , and Bernhard Thalheim2 1
Department of Business Informatics - Data & Knowledge Engineering Johannes Kepler University Linz, Altenberger Straße 69, 4040 Linz, Austria 2 Department of Computer Science and Applied Mathematics Christian Albrechts University Kiel, Olshausenstr. 40, 24098 Kiel, Germany
Abstract. Employing multi-level abstraction in modeling refers to representing objects at multiple levels of one or more abstraction hierarchies, mainly classification, aggregation and generalization. Multiple representation, however, leads to accidental complexity, complicating modeling and extension. Several modeling techniques, like powertypes, deep instantiation, materialization, m-objects, HERM, and the component model may be used to reduce unnecessary complexity with multilevel abstraction. This chapter compares these modeling techniques using four comparison criteria: (1) compactness (modular and redundancyfree models), (2) query flexibility (number and kind of pre-defined entry points for querying), (3) heterogeneous level-hierarchies, and (4) multiple relationship-abstractions (such as between relationship occurrence and relationship type). Keywords: Multi-Level Modeling, Ontological Metamodeling, Deep Instantiation, Materialization, Powertypes, M-Objects.
1
Introduction
Modeling domain objects at multiple levels of abstraction has received growing attention over the last years. IT-systems cover increasingly broader domains with different applications considering the same entities from different abstraction perspectives. Consider the IT-system of a car manufacturer that supports marketing of and sales of cars. Marketing considers cars from the perspective of car models being instances of a class CarModel, one such instance being for example Porsche911CarreraS. Sales considers cars from the perspective of physical cars, each sold to an individual customer and being instance of some car model such as Porsche911CarreraS. The same object Porsche911CarreraS is once considered as instance and once as class. More generally, cars can be considered not only at two levels but at multiple levels of abstraction, such as brands (e.g., Porsche), models (e.g., Porsche911CarreraS), and physical cars (e.g., myPorsche911CarreraS). Modeling multi-level abstraction has been a research issue in the database and software engineering communities. These communities have introduced different Kaschek/Delcambre (Eds.): The Evolution of Conceptual Modeling, LNCS 6520, pp. 68–92, 2011. c Springer-Verlag Berlin Heidelberg 2011
Multi-level Abstraction
69
but related basic terms for the representation of the application domain vs. for the set of modeling constructs used to build this representation. The database community speaks of database (schema and instance) vs. data model, the software engineering community of ontological vs. linguistic modeling [4]. In this chapter we use the notions of domain model vs. modeling language. For example, such a domain model may be schema and instance of a data warehouse [21]. Several modeling techniques and modeling language constructs have been proposed in recent years to represent a domain concept (e.g, car) with descriptions at multiple abstraction levels (e.g., brand, model, physical entity) and the interplay between these levels. Research in this direction has been motivated by reducing accidental complexity of domain models, ‘that occurs due to mismatches between a problem and the technology used to represent the problem’ [5]. Complexity has to be seen with regard to the domain model and the modeling language. Domain models become more complex if multiple, simple modeling language constructs have to be employed together with explicit constraints to represent some domain concept, such as car with its abstraction levels and their interplay. Frequently, the complexity of a domain model can be reduced if the modeling language provides an appropriate complex language construct that can be employed to directly represent some domain concept. Thereby the complex language construct inherently captures integrity constraints that otherwise have to be stated explicitly. We later introduce several comparison criteria for modeling techniques that address accidental complexity from a wider and more in-depth perspective. Multi-level abstraction is not only associated with modeling at multiple classification levels, but more generally, also with levels in generalization and aggregation hierarchies. Existing modeling techniques focus on different aspects of multi-level abstraction. Ontological metamodeling with potency-based deep instantiation [3,15] provides for multiple levels of classification whereby an object at one level can describe the common properties for objects at each instantiationlevel beneath that level. A powertype [22,12,10] has subclasses of a given class as its instances and allows to describe common properties of these subclasses. Similar, but in our view more related to aggregation, materialization [9,24,8] relates a class of categories (e.g., car models) and a class of more concrete objects (e.g., physical cars). A multi-level object (m-object ) [19] encapsulates the different levels of abstractions that relate to a single domain concept; a multi-level relationship (m-relationship) [19] links two m-objects at multiple levels. The higher-order entity relationship model (HERM) [28] extends the classical entityrelationship model by complex attributes, relationship types of higher order, and cluster types. The component model [29] enhances the HERM by encapsulation based on component schemata. Components may have various ports and views for exchange. Ports are associated through harnesses. In this chapter we present a sample problem and comparison criteria for multilevel abstraction techniques. We will model the sample problem using alternative modeling techniques and highlight strengths and weaknesses of each approach with regard to our comparison criteria. Note, however, that each approach has
70
B. Neumayr, M. Schrefl, and B. Thalheim
been developed with a different focus in mind and that the approaches may complement one another. A related comparison of various modeling techniques to reduce accidental complexity in domain models is given by Atkinson and K¨ uhne [5]. An insightful comparison between powertypes and potency-based deep instantiation is given by Gonzalez-Perez and Henderson-Sellers in the related work section of [10]. These comparisons, however, do not consider all comparison criteria used in this chapter, such as heterogeneous level-hierarchies, query flexibility and multiple relationship-abstractions. The remainder of this chapter is organized as follows: Section 2 defines requirements and comparison criteria for multi-level modeling based on a sample problem. Sections 3 shows how to model the sample problem with each approach and evaluates how each approach fulfills our requirements. Section 4 gives a summary of our comparison and concludes the chapter mentioning application areas and future work.
2
Sample Problem and Comparison Criteria
In this section, we present our running example (taken from [19]), model it using plain UML, and introduce comparison criteria for multi-level abstraction techniques. Example 1 (Sample Problem). The product catalog of an online store is described at three levels of abstraction: physical entity, model, and category. Each product category has associated a tax rate, each product model has a list price. Book editions, i.e., objects at level model that belong to product category book, additionally have an author. Our company keeps track of physical entities of products (e.g., copies of book HarryPotter4), which are identified by their serial number. In addition to books, our company starts to sell cars. For selling cars, it introduces car as a new product category. Cars differ from books in that they are described at an additional level, namely brand, and in that they have additional attributes: maximum speed at level product model and mileage at level physical entity. As our sample online store specializes on selling cars of brand Porsche 911, it wants to be able to register physical entities of this car brand at the Porsche 911 club. Our company further keeps track of companies that produce these products. Companies are likewise described at multiple levels: industrial sector, enterprise, and factory. Producers of cars belong to a specific industrial sector, namely car manufacturer. To track quality problems, our company also associates with each physical entity of product category car the factory at which it was produced. Thereby, this factory must belong to the enterprise, which produces the car model. Using traditional semantic data modeling, e.g., using UML, such multi-level domains can be modeled by representing objects in different abstraction hierarchies, classification, aggregation and generalization. In our reference UML
Multi-level Abstraction
71
model (see Fig. 1) the three-level product catalog is modeled by arranging classes (ProductCategory, ProductModel, ProductPhysicalEntity) and their instances ({Car, Book}, {Porsche911CarreraS, Porsche911GT3, HarryPotter4}, {myPorsche911CarreraS, myCopyOfHP4}, respectively) in an aggregation hierarchy, where each aggregation level represents one level of abstraction (product category, product model, and physical entity).
Product Catalog
ProductCategory - taxRate : Integer
Product Model
ProductCatalog - desc : String
Product Category
Classes
Products: ProductCatalog - desc = 'Our Products'
Singleton BookCategory
ProductModel - listPrice : Integer
Product PhysicalEntity
BookTitle - author : String
CarPhysicalEntity - mileage : Integer
BookPhysicalEntity Product Catalog
Car:CarCategory - taxRate = 20
Singleton CarCategory
CarModel - maxSpeed : Integer
ProductPhysical Entity - serialNr : String
Instances
Product Category
Book:BookCategory - taxRate = 15
Singleton Porsche911CarreraS Model
Porsche911CarreraS: Porsche911CarreraSModel - listPrice = 91.838 - maxSpeed = 293 km/h
Singleton Porsche911GT3 Model
Porsche911GT3: Porsche911GT3Model - listPrice = 108.083 - maxSpeed = 310 km/h
Singleton HarryPotter4 BookTitle
Porsche911CarreraS PhysicalEntity Porsche911GT3 PhysicalEntity HP4-PhysicalEntity
HarryPotter4: HarryPotter4BookTitle - listPrice = 11.50 - author = 'J.K.Rowling' myPorsche911CarreraS: Porsche911CarreraS PhysicalEntity - serialNr = 'C333333' - mileage = 100000 myCopyOfHP4: HP4-PhysicalEntity - serialNr = 'A121212'
Product Model
Fig. 1. Product catalog modeled in plain UML, using generalization, instantiation and aggregation (relationships and level brand are omitted)
Objects at the same level in different subhierarchies may differ from each other. For example, product models have a listprice, but car models (i.e., objects at level model belonging to car, an object at level category) have an additional attribute maxSpeed. Furthermore, physical entities of car additionally have a mileage. In our reference UML model (see Fig. 1) this heterogeneity between objects on the same level is modeled by specializing classes that are part of the aggregation hierarchy below ProductCatalog, i.e. ProductCategory, ProductModel, PhysicalEntity, to corresponding classes, CarCategory, CarModel, CarPhysicalEntity, that describe common properties of objects belonging to product category car at the respective level. Note, that singleton classes like CarCategory are necessary to refine aggregation, e.g., to define that each instance of CarModel belongs to product category Car.
72
B. Neumayr, M. Schrefl, and B. Thalheim
Furthermore, we often need to access attributes defined at a higher level of abstraction, for example, to query the taxRate applicable to myPorsche911CarreraS, defined at its product category, car. Concerning our reference UML model we could navigate from myPorsche911CarreraS to taxRate at car by following the aggregation relationships myPorsche911CarreraS.ProductModel.ProductCategory.taxRate (note, that in order to follow links we use dot-notation and, in the absence of explicit role-names, the names of the classes at the other end of the aggregation relationship). Various modeling techniques —powertypes, potency-based deep instantiation, materialization, m-objects, HERM, and the component model— can be used to reduce the complexity of such multi-level models. Their suitability for multi-level abstraction may be benchmarked against the following comparison criteria. 1. Compactness: A domain model is compact if it is modular and redundancyfree. Modularity means that all model elements that represent a domain concept at different levels of abstraction (e.g., product category car in its instance role as well as in its class roles at levels product model and physical entity) are tied together and can be used and updated as a self-contained part of the model. The common structure of elements at lower levels of abstraction can be specialized without interfering with the rest of the model (e.g., introducing attribute maxSpeed only for car models but not for product models in general). A domain model contains redundancy if information is represented more than once1 . Our reference UML model (see Fig. 1) is neither modular nor redundancyfree: Domain concept car is represented by four separate model elements which are not tied together to a self-contained part of the model, namely by instance Car (labelled Car:CarCategory in Fig. 1) and by classes CarCategory, CarModel, and CarPhysicalEntity. Furthermore, the hierarchic relation between product and car is represented redundantly as aggregation between instances Car and Products and as generalizations between CarCategory and ProductCategory, between CarModel and ProductModel, and between CarPhysicalEntity and ProductPhysicalEntity. 2. Query flexibility: A domain model supports query flexibility if it provides several pre-defined entry points for querying, such as class names or qualified identifiers to refer to various sets of objects. In the context of multi-level abstraction we check whether one may easily refer to the set of objects at one abstraction level that belong to a certain domain concept. To test this criterion, we query (1) all objects belonging to product category car at level product model, (2) all objects belonging to product category car at level physical entity, and (3) all objects belonging to level product physical entity. In our reference UML model these entry points for queries are available through the classes (1) CarModel, (2) CarPhysicalEntity, and (3) ProductPhysicalEntity. 1
Note that domain models of cooperating agents may be redundant; such independent, distributed systems are not addressed here.
Multi-level Abstraction
73
3. Heterogeneous Level-Hierarchies: A modeling technique supports heterogeneous level-hierarchies if an additional abstraction level can be introduced for some domain concept without the need to change the set of abstraction levels for other domain concepts. To evaluate this criterion, we will introduce an additional abstraction level car brand for product category car between levels product category and product model and describe one car brand, namely Porsche 911. If this extension can be made without affecting the existing model, i.e, definitions of other product categories, e.g., book, or generally of products, the modeling technique is considered to support heterogeneous level-hierarchies. Concerning our reference UML domain model the respective extension could be made by introducing a class CarBrand as a component of class CarCategory and as an aggregate class of CarModel.
ProductCategory 1
IndustrialSector
context ProductModel inv: self.enterprise.industrialSector->forall( x | self. productCategory.industrialSector->exists( y | x = y) )
* ProductModel 1
producedBy
producedBy
context ProductPhysicalEntity inv: self.factory.enterprise->forall( x | self. productModel.enterprise->exists( y | x = y) )
* ProductPhysicalEntity
Car : ProductCategory
Porsche911CarreraS : ProductModel myPorsche911CarreraS : ProductPhysicalEntity
producedBy
producedBy
producedBy
producedBy
1
* Enterprise 1
* Factory
CarManufacturer : IndustrialSector PorscheLtd : Enterprise
PorscheZuffenhausen : Factory
Fig. 2. Relationships producedBy at different levels of abstraction modeled with UML and OCL
4. Multiple relationship-abstractions: A modeling technique supports multiple relationship-abstractions if it does not only support modeling of domain objects at multiple levels of abstraction, but equally supports multi-level abstraction of relationships. We are interested in modeling constructs that can be employed to simplify querying relationships at multiple levels and carry implicit constraints that are suitable for generalization, classification, and aggregation of relationships at multiple levels. Consider, for example, that products are produced by factories. Our sample product catalog describes products at three levels of abstraction (physical entity, model and category) and it describes the factories in which products are produced at three levels as well (factory, enterprise, and industrial sector). To test the support for multiple relationship-abstraction, we check how a modeling
74
B. Neumayr, M. Schrefl, and B. Thalheim
technique supports the abstraction of relationship produced by between physical entity and factory to the relationship produced by between product model and enterprise, and the abstraction of the latter to relationship produced by between product category and industrial sector. Support for multiple relationship-abstraction can further be characterized by (a) (Meta-)Classification of Relationships, i.e., relationship occurences into relationship classes and relationship classes into relationship metaclasses. (b) Adressability/Queryability of Relationships and (Meta-)Relationship Classes at different levels (c) Specialization/Instantiation of Domain and Range either explicitly through using (meta-)classes or implicitly through constraints. UML does not provide modeling constructs for multi-level abstraction of relationships; we employ (see Fig. 2) OCL to express extensional constraints that are imposed from a higher to a lower level of abstraction of the producedBy relationship. This only covers (c) specialization of domain and range.
3
Techniques Supporting Multi-level Abstraction
First, we give an overview of how modeling techniques for multi-level abstraction evolved. Then we present and compare several modeling techniques in detail. Note that specific modeling patterns in software engineering also relate to multilevel abstraction, e.g., the type-object pattern. We do not discuss this line of work and refer the interested reader to [5]. The 1970’s and 1980’s saw the emergence of semantic data models [26,13], introducing the abstraction principles of classification (collecting objects with common properties to object types), generalization (abstracting common properties of types into supertypes), and aggregation (abstracting component objects into an aggregate object). Classification recursively applied leads to metaclasses [14]. While Smalltalk supported metaclasses with the restriction that each metaclass has a single class as instance, the Common Lisp Object System (CLOS) provided for metaclasses with multiple classes as instances. Telos and Concept Base supported unbounded classification hierarchies. VODAK introduced the possibility that metaclasses describe not only their instances (which are classes) but also the instances of these classes, a principle which has later been extended to unbounded levels and coined Deep Instantiation [3]. It is described in Section 3.1. Combining classification and generalization, Cardelli [6] introduced the notion of a powertype that represents the metaclass for all subclasses of some class. Later, Odell [22] allowed one class to have several powertypes, each collecting some subclasses of the class. Powertypes are discussed in Section 3.2. Similarly, but in our view more related to aggregation, Pirotte and others [9,24] proposed the abstraction principle of Materialization that ‘relates a class of categories (e.g., models of cars) and a class of more concrete objects (e.g., individual cars)’ [8]. It is discussed in Section 3.3. Most recently, Neumayr et al.[19], introduced multi-level concretization hierarchies not only for objects but also for relationships. This approach is discussed in Section 3.4.
Multi-level Abstraction
75
Various extensions of the entity-relationship model [7] may be also employed to develop domain models capturing multi-level abstraction. As representatives we discuss the Higher-Order Entity Relationship Model (HERM) and the Component Model in Sections 3.5 and 3.6. 3.1
Potency-Based Deep Instantiation
Deep instantiation refers to meta modeling in which an object at some (meta-)level can describe the common properties for objects at each instantiation-level beneath that level. Metamodeling through meta classes that provide for deep instantiation has first been put forward in the area of databases by the VODAK system [14]. VODAK associated several types with an object, the own-type describing the object itself, the instance-type describing objects at one classification level below, and the instance-instance-type describing objects at two classification levels below. In this way, meta classes in VODAK allow to describe not only common properties of instances but also common properties of instances of instances. Later, Atkinson and K¨ uhne [3,4] introduced an approach which we call ontological metamodeling with potency-based deep instantiation. Rather than using different types for each level as in VODAK, attribute definitions within a clabject 2 are annotated with a potency, where this potency denotes the instance level, i.e., attributes with potency 0 describe the clabject itself, attributes with potency 1 describe instances of the clabject, attributes with potency 2 describe instances of instances of the clabject, and so forth. In this way, potency-based deep instantiation supports unbounded classification levels. K¨ uhne also introduced DeepJava [15] for programming with potency-based deep instantiation. Our sample product catalog (see Example 1) consisting of abstraction levels physical entity, product model, and product category, is modeled (see Fig. 3) by representing objects of level physical entity as objects at classification level 0 (individual level). Each physical entity (e.g., myCopyOfHP4 and myPorsche911CarreraS) is modeled as some instance of a product model (e.g., HarryPotter4 and Porsche911CarreraS). Each product model is a clabject at classification level 1 (class level), i.e., is a class of individuals at level 0 as well as an instance of some product category (e.g., Book, Car) at level 2 (metaclass level). Each product category is again a clabject, i.e., a class for objects at classification level 1 and instance of ProductCategory at classification level 3 (metametaclass level). To define that physical entities of product category car additionally have a mileage and car models additionally have a maximum speed, attributes maxSpeed and mileage are introduced with potency 1 and 2, respectively, at clabject Car. To access attributes defined at a higher level (upward navigation) it is possible to follow the instance-of relationships with method type(). The 2
Atkinson [2] coined clabject (short for class/object) for model elements playing both an instance role as well as a class role (also referred to as object facet and class facet, respectively).
76
B. Neumayr, M. Schrefl, and B. Thalheim Level 3 (Metametaclasses)
ProductCategory - taxRate1 : Integer - listPrice2 : Float - serialNr3 : String
Book - taxRate0 = 15 - author1 : String
HarryPotter4 - listPrice0 = 11.50 - author0 = 'J.K.Rowling'
myCopyOfHP4 - serialNr0 = 'A121212'
Car - taxRate0 = 20 - maxSpeed1 : Integer - mileage2 : Integer
Porsche911GT3 - listPrice0 = 108.083 - maxSpeed0 = 310 km/h
Level 2 (Metaclasses)
Porsche911CarreraS - listPrice0 = 91.838 - maxSpeed0 = 293 km/h
Level 1 (Classes)
myPorsche911CarreraS - serialNr0 = 'C333333' - mileage0 = 100000
Level 0 (Individuals)
Fig. 3. Product catalog modeled with potency-based deep instantiation
path from myPorsche911CarreraS to taxRate defined at Car is denoted as: myPorsche911CarreraS.type().type().taxRate. We evaluate this modeling technique against our comparison criteria: 1. Compactness: Potency-based deep instantiation supports modular, redundancy-free modeling. All information concerning all levels of one domain category can be described local to one clabject. For example, all information about domain concept car is encapsulated in a single model element Car which is related to ProductCategory only once (see Fig. 3). 2. Query flexibility: To our knowledge, extensions of classes are not maintained by DeepJava. But to maintain and retrieve the different member sets of our sample queries should be easy in principle, given the set theoretic interpretation of deep instantiation [16]. Every clabject with potency p > 0 can be viewed as having p member sets, one for each level beneath, e.g., Car = {{Porsche911GT3, Porsche911CarreraS},{{},{myPorsche911CarreraS}}}. 3. Heterogeneous Level-Hierarchies: Additional abstraction levels for some domain concept cannot be introduced without requiring global model changes. Consider introducing a new classification level (e.g., to represent that car models are classified by car brands, such as Porsche911) below level n (e.g., classification level 2 containing clabjects such as Car) and above level n-1 (e.g., classification level 1 containing clabjects such as Porsche911CarreraS). Such an additional classification level requires the following global changes: (1) For each clabject at a classification level above the new classification level (e.g., ProductCategory and Book) potencies of attributes that affect objects at classification levels below the new classification level have to be changed (e.g., listPrice and serialNr in clabject ProductCategory and author in clabject Book). (2) For each clabject at level n, a (dummy) clabject at the new classification level has to be introduced, e.g., BookBrandDummy,
Multi-level Abstraction
77
and each clabject at level n-1 has to be re-classified to be instanceof the respective clabject at the new classification level (HarryPotter4 is instance-of BookBrandDummy). Furthermore, (3) upward navigation paths have to be changed: the navigation from myPorsche911CarreraS to its taxRate changes from myPorsche911CarreraS.type().type().taxRate to myPorsche911CarreraS.type().type().type().taxRate. 4. Multiple relationship-abstractions: The constraints modeled in Fig. 2 can be implemented in DeepJava using type parameters [15], which capture aspect (c) Specialization/Instantiation of Range. More recently, Gutheil et al. [11] discussed connectors (relationships) between clabjects in multi-level modeling environments, focusing on their graphical rendering, not discussing querying different class roles of such connectors at different levels. 3.2
Powertypes
The powertype pattern consists of a partitioned type and a powertype. Every instance of the powertype is a subclass of the partitioned type. Such model elements —playing both an instance role as well as a class role— are again referred to as clabjects. For example (see Fig. 4), ProductCategory is powertype of partitioned type ProductModel. Clabject car plays two roles, namely as instance of ProductCategory and as subclass of ProductModel, represented by Car:ProductCategory and CarModel, respectively. Cascaded Powertypes - Basic Pattern. In order to model our sample threelevel product catalog one can use a cascaded setup of powertypes. For example (see Fig. 4), ProductCategory is powertype of ProductModel, which is, in turn, powertype of ProductPhysicalEntity. Thus, each instance of ProductCategory, such as car, is also considered as a subclass of ProductModel, such as CarModel. In turn, each instance of CarModel, such as Porsche911CarreraS, is also considered as a subclass of ProductPhysicalEntity, such as Porsche911CarreraSPhysicalEntity. We evaluate this modeling technique against our comparison criteria: 1. Compactness: The basic pattern for cascaded powertypes (see Fig. 4) partly supports modularity but may lead to redundancies. Domain concept car is represented by a clabject playing a class role (CarModel) and an instance role (Car:ProductCategory). This improves compactness as compared to our reference UML domain model. However, since the class role of car is the class of all car models, there is no class which has all physical cars as instances. Thus, it is necessary to define attribute mileage redundantly, both at Porsche911CarreraSPhysicalEntity as well as at Porsche911GT3PhysicalEntity. 2. Query flexibility: Querying powertypes is not explicitly treated in the literature we are aware of Pre-defined entry points for our “benchmmark queries” can partly be provided by the explicitly represented class roles: (1) All carmodels: members of the class role of clabject car (CarModel). (2) Physical entities of car: no predefined entry point (3) All physical entities of products: members of ProductPhysicalEntity.
78
B. Neumayr, M. Schrefl, and B. Thalheim
isClassifiedAs
ProductCategory - taxRate : Integer
isClassifiedAs
ProductModel - listPrice : Integer
ProductPhysical Entity - serialNr : String
Car:ProductCategory - taxRate = 20 CarModel - maxSpeed : Integer
Book:ProductCategory - taxRate = 15 BookTitle - author : String
Porsche911CarreraS: CarModel - listPrice = 91.838 - maxSpeed = 293 km/h Porsche911CarreraS PhysicalEntity - mileage : String Porsche911GT3: CarModel - listPrice = 108.083 - maxSpeed = 310 km/h
myPorsche911CarreraS: Porsche911CarreraS PhysicalEntity - serialNr = 'C333333' - mileage = 100000
Porsche911GT3 PhysicalEntity - mileage : String HarryPotter4:BookTitle - listPrice = 11.50 - author = 'J.K.Rowling' HP4-PhysicalEntity
myCopyOfHP4: HP4-PhysicalEntity - serialNr = 'A121212'
Fig. 4. Product catalog modeled with cascaded powertypes - basic pattern. Notation as proposed by Henderson-Sellers and Gonzalez-Perez [12].
3. Heterogeneous Level-Hierarchies: The basic cascaded powertype pattern does not support heterogeneous level-hierarchies. 4. Multiple relationship-abstractions: The literature we are aware of does not introduce special techniques for modeling multiple relationship-abstraction with powertypes. The modeling approach shown in Fig. 2, using plain UML and OCL, can be used together with powertypes, which only captures (c) specialization of domain and range. Cascaded Powertypes - Extended Pattern. To overcome the deficiencies of the simple cascaded powertype pattern one can extend it by introducing further subclasses when instantiating a powertype and by specializing relationships which connect partitioned types with powertypes. For example, when instantiating powertype ProductCategory by car, one can additionally introduce a subclass of ProductPhysicalEntity, namely CarPhysicalEntity, which can be used for defining common structure of physical cars as well as an entry-point for querying. The relationship between partitioned type ProductPhysicalEntity and powertype ProductModel is specialized between partitioned type CarPhysicalEntity and its powertype CarModel (which is the class role of clabject car). We evaluate this modeling technique against our comparison criteria: 1. Compactness: Using the extended powertype pattern we end up with a model that is only partly modular and contains redundancies. Using powertypes, representing domain concept car requires to define two separate model elements, namely class CarPhysicalEntity as well as a clabject which consists
Multi-level Abstraction CarBrand - marketLaunch : Date isClassifiedAs
Porsche911:CarBrand - marketLaunch= 1964 Car:ProductCategory - taxRate = 20 CarModel - maxSpeed : Integer
ProductPhysical Entity - serialNr : String
isClassifiedAs
Book:ProductCategory - taxRate = 15 BookTitle - author : String
isClassifiedAs
isClassifiedAs
ProductModel - listPrice : Integer
Porsche911Model isClassifiedAs
isClassifiedAs
ProductCategory - taxRate : Integer
79
Porsche911CarreraS: CarModel - listPrice = 91.838 - maxSpeed = 293 km/h Porsche911CarreraS PhysicalEntity
Porsche911 PhysicalEntity - porsche911Club : Boolean
Porsche911GT3: CarModel - listPrice = 108.083 - maxSpeed = 310 km/h
myPorsche911CarreraS: Porsche911CarreraS PhysicalEntity - serialNr = 'C333333' - mileage = 100000 - porsche911club = true
Porsche911GT3 PhysicalEntity CarPhysicalEntity - mileage : Integer
BookPhysicalEntity
HarryPotter4:BookTitle - listPrice = 11.50 - author = 'J.K.Rowling' HP4-PhysicalEntity
myCopyOfHP4: HP4-PhysicalEntity - serialNr = 'C333333'
Fig. 5. Cascaded powertypes - extended pattern
of instance role (Car) and class role (CarModel). The hierarchic relationship between domain concepts car and Product has to be modeled redundantly by generalization between CarPhysicalEntity and ProductPhysicalEntity, and by classification between Car and ProductCategory (with generalization between classes CarModel and ProductModel being implicit). 2. Query flexibility: Querying powertypes is not explicitly treated in the literature we are aware of. But pre-defined entry points can be easily provided for our “benchmmark queries”. (1) All car-models: members of car’s class role CarModel. (2) Physical entities of car: members of class CarPhysicalEntity. (3) All physical entities of products: members of ProductPhysicalEntity. 3. Heterogeneous Level-Hierarchies: It is possible to introduce an additional level in one sub-hierarchy (e.g., for product category car) without affecting other sub-hierarchies (e.g., product category book). An additional level brand between car category and car model is achieved as follows: One introduces class CarBrand as powertype of CarModel, whereby CarModel forms a clabject with instance Car as before. Then, a particular car brand such as Porsche911 is modeled as an instance of CarBrand, i.e., as a clabject playing an instance role (Porsche911) and a class role (Porsche911Model), where the latter is subclass of CarModel. Furthermore, to describe common properties (e.g., attribute porsche911Club) of objects belonging to car brand Porsche911 at level physical entity, class Porsche911PhysicalEntity is introduced as subclass of CarPhysicalEntity. 4. Multiple relationship-abstractions: The literature we are aware of does not introduce special techniques for modeling multiple relationship-abstraction with powertypes. But the modeling approach shown in Fig. 2, using plain UML and OCL, can be used together with powertypes. This can only capture aspect (c) Specialization of Domain and Range.
80
B. Neumayr, M. Schrefl, and B. Thalheim
3.3
Materialization
In conceptual modeling, materialization3 is a generic relationship type used to relate a class cc of more concrete objects and a class ca of more abstract objects (e.g., ProductModel materializes ProductCategory). At the instance level, each instance oc of the more concrete class materializes exactly one instance oa of the more abstract class (e.g., Porsche911CarreraS materializes Car). Additionally, materialization provides for powerful attribute propagation mechanisms: Object oc both inherits attribute values from object oa (shared values) and instantiates attributes defined at class cc as well as attributes introduced at object oa . In this line, similar to clabjects, oa plays both the role of an object as well as the role of a class. ProductCategory - taxRate : Integer (T1) - modelAttr (T3) - physEntAttr (T1-T3)
model
1 model materialization
materializes
* materialization ProductModel - listPrice : Integer - physEntAttr2 (T3) 1 model
Book:ProductCategory - taxRate = 15 - modelAttr = {author} - physEntAttr = {}
Car:ProductCategory - taxRate = 20 - modelAttr = {maxSpeed} - physEntAttr = {mileage}
-
Porsche911CarreraS: ProductModel listPrice = 91838 maxSpeed = '293 km/h' physEntAttr2 = {} /taxRate = 20 /physEntAttr = {mileage}
-
model materialization
Porsche911GT3: ProductModel listPrice = 108083 maxSpeed = '310 km/h' physEntAttr2 = {} /taxRate = 20 /physEntAttr = {mileage}
materialization
-
model materializes
* materialization ProductPhysicalEntity - serialNr : String
model
materialization
myPorsche911CarreraS: ProductPhysicalEntity - serialNr = 'C333333' - mileage = 100000 - /listPrice = 91838 - /maxSpeed = 293 km/h - /taxrate = 20
HarryPotter4: ProductModel listPrice = 11.50 author = 'J.K.Rowling' physEntAttr2 = {} /taxRate = 15 /physEntAttr = {}
materialization
-
myCopyOfHP4: ProductPhysicalEntity serialNr = 'A121212' listPrice = 11.50 /author = 'J.K.Rowling' /taxrate = 15
Fig. 6. Product catalog modeled with materialization (using a respective UMLnotation by Oliv´e [23]) and applying composite attribute propagation mechanism (T1T3) to mimic deep instantiation
Our sample three-level product catalog (see Example 1) can be modeled by consecutively applying materialization relationships. Class ProductPhysicalEntity materializes class ProductModel which materializes class ProductCategory. Each class represents one level of abstraction (or, in this context, one level of materialization). Definition of shared attribute values at the more abstract object oa that are propagated to the more concrete object oc , and definition of attributes at oa , that are instantiated at oc , are facilitated by a set of attribute propagation mechanisms, introduced by Pirotte et al. [24] and further described by Dahchour et al. [8]. Type 1 propagation (T1) simply propagates values of attributes (monoor multivalued) from oa to oc , e.g., taxRate=20 from Car to Porsche911CarreraS. 3
Not to be mixed up with materialization of database views (materialized views).
Multi-level Abstraction
81
For better readability, we have marked propagated T1-attributes with a “/”character, such as /taxRate=20 at object Porsche911CarreraS. Type 2 propagation (T2) allows to define at ca an attribute, instantiated at oa with a set of possible values. This attribute is instantiated at oc with one value (T2mono) or a set of values (T2multi) from the set of possible values defined at oa (not considered in our example). Type 3 propagation (T3) allows to define a multi-valued attribute at ca , instantiated with a set of values at oa . Each such attribute value at oa is transformed to an attribute in oc and instantiated there. For example, class ProductCategory defines an attribute modelAttr with T3 that is instantiated at object Car with {maxSpeed}, and maxSpeed is instantiated at object Porsche911CarreraS (see Fig. 6) Attribute values defined at objects at higher materialization levels (e.g., taxRate = 20 at Car) can be accessed at objects at lower materialization levels (e.g., myPorsche911CarreraS) either by traversal of materialization links (myPorsche911CarreraS.model.model.taxRate) or, if taxRate is propagated using T1, by directly accessing the propagated attribute value, e.g., myPorsche911CarreraS.taxRate. Composing attribute propagation types, as described in [8], allows one to define attributes that have impact on two materialization levels below and a combination of propagation modes T1 and T3 can be used to mimic potencybased instantiation. The combination of propagation modes T1 and T3 has to the best of our knowledge not been considered in literature, but it can be used as follows to mimic potency-based instantiation. An abstract class ca defines an attribute with T1-T3 propagation. The attribute is instantiated with a set of values at instance oa of class ca and, due to propagation mode T1 (which is first applied), propagated to every materialization oc of abstract object oa . The attribute is propagated further in T3 mode to every materialization occ of oc , i.e., occ receives an attribute for every value of that attribute at oc and instantiates it. In this way, attributes with T1-T3 resemble attributes with potency 2 (see Section 3.1) and attributes with T1-T1-T3 resemble attributes with potency 3, and so forth. We illustrate this approach by the following example. As described in our sample problem statement, product models and physical entities belonging to product category car differ from other product models and physical entities, in that they have attributes maxSpeed and mileage, respectively. To be able to define these peculiarities, attribute modelAttr(T3) and physEntAttr(T1-T3) are introduced at class ProductCategory. As described in the previous paragraph, these attributes serve as placeholders for attributes introduced in instances of ProductCategory. At Car, modelAttr is instantiated with {maxSpeed} to define that there is an attribute maxSpeed in every materialization of Car, i.e. in Porsche911CarreraS and in Porsche911GT3. Furthermore, attribute physEntAttr is instantiated with {mileage} to define that there is an attribute mileage in every materialization of a materialization of Car, i.e. in myPorsche911CarreraS. Alternatively, one could explicitly specialize classes ProductModel and ProductPhysicalEntity by subclasses CarModel and CarPhysicalEntity,
82
B. Neumayr, M. Schrefl, and B. Thalheim
respectively. We do not further consider this alternative, because subclassing of classes which are part of materialization hierarchies is not considered in the literature on materialization and is somewhat counter-intuitive to the idea of materialization. Another powerful composite attribute propagation type, T3-T2, is discussed in [8], but is not needed to model our sample product catalog. We evaluate this modeling technique against our comparison criteria: 1. Compactness: Materialization allows modular and redundancy-free models. All peculiarities of domain concept car, including those concerning objects at level product model and physical entity, can be described local to one model element, namely Car. Thus, materialization effectively reduces accidental complexity. 2. Query flexibility: Materialization hierarchies (as in [8]) provide a pre-defined entry point for all objects at a specific materialization level by the class representing this materialization level, e.g., all objects at level physical entity can be addressed via class ProductPhysicalEntity. Relevant literature does not consider pre-defined query entry points that refer to all objects of one materialization level that directly or indirectly materialize some abstract object. However, one should in principle be able to retrieve such sets of objects by queries like (in SQL-Syntax): SELECT * FROM ProductModel p WHERE p.model = Car or to access all physical cars: SELECT * FROM ProductPhysicalEntity p WHERE p.model.model = Car . 3. Heterogeneous Level-Hierarchies: Literature on materialization does not discuss introducing additional materialization levels local to a specific object in the materialization hierarchy, like level brand local to Car. Since the use of T1-T3-attribute propagation mimics potency-based deep instantiation, additional levels would lead to similar problems as when using potency-based deep instantiation, i.e., lead to global model changes. Using materialization, propagation modes of attributes need to be adapted analogously as described above for potency values of potency-based deep instantiation. 4. Multi-level relationships: Pirotte et al. introduce attribute propagation. Since attributes can be regarded as (directed) relationships, T2 and T3 attribute propagation are a restricted form of multi-level relationships and partly capture (a) (meta-)classification of relationships and capture (c) specialization of range but lacks (b) Adressability/Queryability of Relationships at different levels. They also briefly mention materialization of relationships but do not discuss materialization of relationships in detail. They basically consider a relationship as a class that can be materialized like any other class. Pirotte et al. [24] also sketch the idea of materialization of aggregation, e.g., a product model is composed of various parts (bill of materials) and this bill of materials can be materialized to the level of physical entities. This idea could be elaborated and extended for materialization of relationships in order to support multiple relationship-abstractions.
Multi-level Abstraction
3.4
83
Multi-level Objects
In [19] we introduce multi-level objects (m-objects) and multi-level relationships (m-relationships) for the concretization of objects and relationships along multiple levels of abstraction. The basic ideas of this approach, which builds on the previous approaches discussed above, are (i) to define and describe the different levels of abstraction that relate to a single domain concept (e.g., the level descriptions, catalog, category, model, physicalEntity that relate to single domain concept, product catalog) and to encapsulate them into a single m-object (e.g., Product), and (ii) to combine aspects of the different abstraction hierarchies (aggregation, generalization, and classification) into a single concretization hierarchy. An m-object encapsulates and arranges abstraction levels in a linear order from the most abstract to the most concrete one. Thereby, it describes itself and the common properties of the objects at each level of the concretization hierarchy beneath itself. An m-object specifies concrete values for the properties of the top-level. This top-level describes the m-object itself. All other levels describe common properties of m-objects beneath itself. An m-object can concretize another m-object, which is referred to as its parent. A concretization relationship between two m-objects comprises aggregation, classification, and generalization. This does not mean that one m-object is at the same time component of, instance of and subclass of another m-object as a whole. Rather, a concretization relationship between two m-objects, such as between Car and Product in Fig. 7, is to be interpreted in a multi-faceted way as follows: (1) Aggregation: product category car (represented by the top level of m-object Car) is part of our product catalog (represented by the top-level of m-object Product). (2) Classification: the top-level of Car is instance of the second-top level (category) of m-object Product, it instantiates attribute taxRate by assigning a value to it. (3) Generalization: m-object Car specializes m-object Product by introducing a new level brand and by adding attribute maxSpeed to level model and attribute mileage to level physical entity. The non-top levels model and physical entity of m-object Car are regarded as subclasses of the corresponding levels of m-object Product. M-relationships are analogous to m-objects in that they describe relationships between m-objects at multiple levels of abstraction. M-relationships are bi-directional (in the context of data warehousing [21] we also provide for nary m-relationships with attributes). To facilitate referencing objects involved in binary relationships, we take, however, a (potentially) arbitrary directional perspective by considering one object in the source role and the other object in the target role of the relationship. Each m-relationship links a source m-object with a target m-object. Additionally, it connects one or more pairs of levels of the source and the target. These connections between source- and target-levels constitute the abstraction levels of an m-relationship. They define that m-objects at source-level are related with m-objects at target-level. We note that the generic roles source and target, which we introduced here for simplicity, may be easily replaced by relationship-specific role names.
84
B. Neumayr, M. Schrefl, and B. Thalheim
Product : catalog - desc:String = ‚Our Products’
- taxRate : Integer <model> - listPrice : Float - serialNr Book : category - taxRate = 15 <model> - author : String
Product Category
Car : category - taxRate = 20 - marketLaunch : date <model> - maxSpeed : Integer - mileage : Integer Porsche911 : brand - marketLaunch = 1964 <model> - porsche911club :boolean
HarryPotter4 : model - listPrice = 11.50 - author = ‚J.K.Rowling’
Porsche911CarreraS : model - listPrice = 91.838 - maxSpeed = 293 km/h
myCopyOfHarryPotter4 : physicalEntity - serialNr = ‚A121212’
myPorsche911CarreraS : physicalEntity - serialNr = ‚C333333' - mileage = 100000 - porsche911club = true
Car Brand
Porsche911GT3 : model - listPrice = 108.083 - maxSpeed = 310 km/h
Product Model
Product Physical Entity
Fig. 7. Product catalog modeled with m-objects (inherited attributes not shown)
Like m-objects, m-relationships can be concretized. Concretizing an mrelationship means to substitute its source or its target for a direct or indirect concretization. The concretizes-relationship between two m-relationships expresses instantiation and/or specialization, depending on the levels one looks at. Consider m-relationship producedBy between m-objects Product and Company in Fig. 8, it specifies multiple source- and target-levels. The relationship expresses that product categories are produced by industrial sectors, product models by enterprises and physical entities by factories. This m-relationship is concretized by producedBy between Car and CarManufacturer, which in turn is further concretized between Porsche911CarreraS and PorscheLtd, which in turn is further concretized between myPorsche911CarreraS and PorscheZuffenhausen. We evaluate this modeling technique against our comparison criteria: 1. Compactness: M-objects allow modular and redundancy-free modeling by encapsulating all information concerning one domain concept into one mobject. For example, all information concerning domain concept car is encapsulated into one m-object, Car.
Multi-level Abstraction
Product <model>
producedBy
Company <enterprise>
Car : category <model>
producedBy
CarManufacturer : industrialSector <enterprise>
Porsche911CarreraS : model
producedBy
PorscheLtd : enterprise
myPorsche911CarreraS : physicalEntity
producedBy
PorscheZuffenhausen : factory
85
Porsche911 : brand <model>
Fig. 8. Product catalog modeled with m-objects and their m-relationships [19]
2. Query flexibility: Pre-defined, qualified identifiers can be used to refer to the set of m-objects at a specific level that belong to a specific m-object. The set of all m-objects that are descendants of a given m-object at some specified level is identified by qualifying the name of the m-object by that level. E.g., (1) to refer to all models of product category Car one writes Carmodel, (2) to refer to the physical cars of product category car: CarphysicalEntity, and (3) to refer to all physical entities of products: ProductphysicalEntity. (See [18] for details). 3. Heterogeneous Level-Hierarchies: Hierarchies of m-objects are easy to extend by an additional level (e.g., brand) in one sub-hierarchy (e.g., the subhierarchy rooted by car) without affecting other sub-hierarchies (e.g., the sub-hierarchy rooted by book). Fig. 7 already shows level brand with mobject car. To add this level to cars, but not to books, requires no changes above m-object car or in sibling hierarchies of m-objects (such as book). 4. Multiple relationship abstractions: M-relationships between m-objects are like m-objects described at multiple levels, associating one or more pairs of levels of the linked m-objects. M-relationships are interpreted in a multifaceted way, once as relationship occurrence (or sometimes also called relationship instance or link) and once as relationship class (or sometimes also called relationship type or association), or also as meta-relationship class. They take on these multiple roles depending on what pairs of linked levels one considers and on whether one considers the linked m-objects in an instance or a class-role. As m-relationships may also be concretized along the concretization hierarchy of linked m-objects, they support multiple relationship abstractions.
86
B. Neumayr, M. Schrefl, and B. Thalheim
M-Relationships cover (a) (Meta-)Classification of Relationships, (b) Adressability/Queryability of Relationships at different levels, and (c) Specialization/Instantiation of Domain and Range. 3.5
The Higher-Order Entity Relationship Model
The higher-order entity-relationship model (HERM ) [28] is a high-level data model that builds on the classical entity-relationship model [7] and subsumes most other extended ER models. HERM does not provide special modeling constructs for multi-level abstraction, but we will discuss in the following to what extent our sample problem can be represented by two alternative modeling approaches, through use of a specialization schema and an overlay schema (both explained below) with the general purpose modeling constructs of HERM. One of the distinguishing features of HERM are higher-order relationship types. While a (first-order) relationship type associates one or more entity types (called in HERM: ’components of the relationship type’), a higher-order relationship type also associates relationship types. Also note that in HERM, subtypes are typically modeled by unary relationship types, i.e., relationship types having only one component, e.g., entity type Product is specialized by unary relationship type Car. We first discuss the modeling approach using a specialization schema (see Fig. 9). Domain concepts at the highest abstraction level are modeled as entity types and are specialized by domain concepts at lower abstraction levels which are modeled as unary relationship types. For example, entity type Product is specialized by unary relationship types Car and Book, which are, in turn, specialized by unary relationship types Porsche911 and HarryPotter4, respectively. Relationship types myPorsche911CarreraS and myCopyOfHarryPotter4 define singleton classes (note that singleton classes are not typical for HERM). Specialization may be applied by adding properties, by providing values for properties, or by both (e.g., Book for Product with value assignment for taxRate and adding property author). HERM does not provide modeling constructs for defining abstraction levels; names of abstraction levels (e.g., Product Model in Fig. 9) are rather depicted as annotations for better readability. In order to define that an attribute semantically belongs to a certain abstraction level, like that listPrice belongs to level product model, and that it has to be instantiated at this level, one would additionally have to use explicit constraints. We note that this is a general shortcoming of the ER approach and not peculiar to HERM. Alternatively, one could model the sample product catalog by multiple specialization hierarchies, each rooted in an entity type representing an abstraction level (analogous to our reference UML solution in Fig. 1). HERM requires that each type has a key that may be composed of attributes of the type and attributes of the related types. Relationship types inherit identification from components but inherited identification may be overriden by a key that replaces the inherited key, e.g., ISBN is the identification for type Book.
Multi-level Abstraction
87
H
maxSpeed=310km/h listPrice=108.083 HH Porsche911GT3 H H porsche911club porsche911club=true
H H HH mileage=100000 HH H HH myPorsche911H Porsche911 H H Car Porsche911 HH HHCarreraS HHCarreraS HHH maxSpeed listPrice serialNr HmarketLaunch H =91.838 H =293km/h maxSpeed
mileageHtaxRate=20
ID
desc
taxRate
serialNr=’C33333333’
marketLaunch=1964
Product
I @
author HHH HHBook H
listPrice @
ISBN
product catalog
HH HHH myCopyOf H HarryPotter4 HHarryPotter4 H HH HH serialNr=’A121212’ listPrice=11.50
author=J.K. Rowling
taxRate=15
product category
product brand
product physical entity
product model
Fig. 9. Selected parts of the product catalog modeled with HERM (specialization schema)
We now discuss the modeling approach using an overlay schema (see Fig. 10). Abstraction levels are modeled as entity types (e.g., Catalog, Category, Brand, Model) or unary relationship types (e.g., Entity). The order of abstraction levels is modeled by binary relationship types (e.g., Brand to Model). This overlay schema combines the global schema of the product catalog with specialized schemas of categories car and book and of brand Porsche911. For this purpose, it uses optional attributes which may either be populated or not. The population is constrained by the following null-value constraints: [author]0 binds the population of values for authors in entity type Model to those models whose category kind is Book, [mileage]0 binds the population of values for mileage in relationship type Entity to those physical entities whose category kind is Car, and [clubAccepted]0 binds the population of values for clubs in relationship type Entity to those physical entities whose category kind is Car and whose brand allows club membership. ID
[marketLaunch]0 Brand
H HH Brand to HH Model (0,1) H
[club]0 Catalog ID
Model
[author]0
HH Catalog H to HHCategory H
[clubAccepted]0
ID
listPrice
6
H EntityHH HH H [mileage]0 desc
[maxSpeed]0 (1,1)
HH Category H - of HHModel H
-
Category
taxRate kind ID
desc
Fig. 10. Selected parts of the product catalog modeled with HERM (overlay schema)
88
B. Neumayr, M. Schrefl, and B. Thalheim
We evaluate this modeling technique against our comparison criteria: 1. Compactness: In general, HERM allows a very compact and redundancy-free representation of schemata. However, using specialization types (see Fig. 9) one has to use explicit constraints in order to define that an attribute has to be instantiated at a certain level. The overlay style provides a rather compact way of modeling application domains with many different specializations. It does, however, also not completely fulfill the modularity criterion. 2. Query flexibility: In the specialization schema, abstraction levels are not explicitly modeled, therefore it is not possible to query all objects that belong to a certain abstraction level, e.g., all product models. This is possible in the overlay schema. HERM supports declaration of extended views [28], view towers, and visual queries [30]. It can be combined with aggregation and abstraction functions. The queries and the views must, however, be based on the schema specification. 3. Heterogeneous level-hierarchies: Extended ER models support multi-level hierarchies if the modeling style supports it. The overlay schema in Fig. 10 is not prone to the introduction of additional abstractions whereas the specialization model in Fig. 9 supports multiple and flexible abstraction. 4. Multiple relationship-abstractions: Multiple relationship-abstractions are well supported as long as the relationships among hierarchies are homogeneous. The missing Brand hierarchy layer and changing semantical constraints among hierarchies cause problems. HERM allows one to build relationship types on relationship types and to introduce cluster types. The component diagram in Fig. 11 can be unnested through the introduction of a relationship type ProducedBy. The specialization of this type is not well supported within HERM. 3.6
The Component Model
The component model [29] enhances HERM by the explicit introduction of an encapsulation based on component schemata, views defined for component schemata with or without update propagation through these views, and ports to components that allow an exchange of data among components based on the view assigned to the port. Components may themselves consist of components. Views of inner components may be visible also to outer components. For instance, the Car component in Fig. 11 has its own HERM schema. It has only one view for the outside world that consists of all attributes of this component and the inherited attributes of its inner components. Components may have various ports and views for exchange. Ports are associated through harnesses. Harness producedBy associates products and companies. The harness has a specialisation hierarchy that follows the specialisation hierarchies in component Product and in component Company. It has been proven in [25] that any HERM schema can be transformed into componentized schemata that separate different concerns.
Multi-level Abstraction producedBy
'
◦ desc ◦ taxRate ◦ listPrice ◦ serialNr ' Car $ ◦ maxSpeed ◦ taxRate=20 '◦ marketLaunch ◦ mileage $ Porsche911 ' $
Product Catalog
'
◦ author
Book ◦ taxRate=15 ' HarryPotter4 author = ◦ ’J.K. Rowling’ ◦ listprice = 11.50 '
myCopy OfHP4
Porsche911CarreraS marketLaunch ◦ = 1964 ◦ maxSpeed = 293km/h ◦ porsche911club ◦' listPrice = 91.838 $ $ myPorsche 911CarreraS
serialNr = ’C33333333’ mileage = ◦ 100000 porsche911club ◦ = true ◦
serialNr = ◦ ’A121212’
& & & &
89
& % & % & % &
'
Porsche 911GT3
$ ' $ $Company ' $ Car$ Manufacturer '
PorscheLtd.
$'
$
Porsche Zuffenhausen
listPrice = 108.083 maxSpeed = ◦ 310km/h ◦
% % &
$
% & % & % & % &
% % % %
Fig. 11. Product catalog modeled with components
We evaluate this modeling technique against our comparison criteria: 1. Compactness: The component model inherits the advantages and the quality properties of HERM. It provides additional facilities for zooming out and zooming in depending on the intended level of detail. Components have their ports and views assigned to ports. These ports are connected through harnesses. The harness producedBy has been sketched in Fig. 11. It associates catalog products with companies, car manufacturers with cars, and ‘my Porsche’ to Zuffenhausen. As with HERM, explicit constraints have to be used to define at which level an attribute has to be instantiated. For example marketLaunch and mileage are both introduced at Car. To model that marketLaunch has to be instantiated by objects at level brand, e.g., Porsche911, and mileage has to be instantiated by objects at level physical entity, e.g., myPorsche911CarreraS, one has to use explicit constraints (not shown in Fig. 11). 2. Query flexibility: Query flexibility is similar to the query flexibility of HERM. Visual SQL supports the definition of views. Therefore, any query that can be expressed through Visual SQL can also be defined for component schemata. 3. Heterogeneous level-hierarchies: Handling of heterogeneous level-hierarchies is not restricted to the modeling style for component schemata. Any new level within a hierarchy can be introduced as long as the separation of components within a hierarchy level is based on exclusion constraints for each sub-component. For instance, we assume that Car and Book specify components for exclusive classes of products. 4. Multi-level relationships: The problem of handling multiple and heterogeneous relationships among components is resolved through the introduction of harnesses. Any kind of specialisation and generalisation can be defined also for harness schemata.
90
4
B. Neumayr, M. Schrefl, and B. Thalheim
Conclusion
We discussed the evolution of modeling techniques for multi-level abstraction and analyzed six modeling techniques in detail. The results are summarized in Fig. 12. We wish to stress that our evaluation only addresses the suitability of these techniques for multi-level abstraction as discussed in Section 2, but does not judge their overall quality. Compactness Query Heterogeneous Flexibility Levels Deep Instantiation + + − Powertypes (Simple) ∼ ∼ − Powertypes (Ext.) ∼ + + Materialization + ∼ − M-Objects + + + HERM ∼ −2 /+3 +2 /−3 Component Model ∼ ∼ + Legend: (+) full support (∼) partial support (−) no support 1) using OCL 2) specialization schema, 3) overlay schema
Relationship Abstraction ∼ −/∼1 −/∼1 ∼ + ∼ +
Fig. 12. Summary: evaluation of multi-level abstraction techniques
The discussed techniques provide to some extent support for describing multilevel abstraction, but only m-objects and m-relationships met the requirements benchmarked by our comparison criteria. We want to highlight that the running example was rather simple (among others, we did not consider multi-level abstraction of composite objects) and that in practice to-be represented domains tend to be much more complex. Thus, modeling with multi-level abstraction in the general case is much harder than discussed in this chapter. None of the discussed modeling techniques is capable of fully supporting multi-level abstraction in the general case. But, there are specific complex modeling problems that are well supported by some of the techniques and that could not be concisely modeled otherwise. For example, m-objects and m-relationships, in an extended form [21], may be used to model heterogeneous dimension hierarchies and cubes in data warehouses. This has been demonstrated by a proof-of-concept prototype [27] implemented on top of Oracle DB. Multi-level abstraction is also of interest in the realm of ontologies and the Semantic Web for representing domains where the borderline between individuals and classes is not clear cut [17]. We showed [20] how m-objects and mrelationships can be mapped to the decidable fragment of the web ontology language OWL extended with closed-world integrity constraints. It remains, however, open to future work to evaluate this approach in practice. Aschauer et al. [1] showed how clabjects (based on the ideas of Atkinson and K¨ uhne) can be applied for modeling industrial automation systems in practice. They found that deep instantiation is well applicable for their modeling problem
Multi-level Abstraction
91
if extended by ’gradual instantiation’ (i.e., introducing shared and default values for properties at higher levels). Although first approaches to multi-level abstraction date back 20 years and more, modeling techniques for multi-level abstraction are still in their infancy and are not appropriate for concisely modeling multi-level abstraction in the general case. To overcome these limitations, future work on multi-level abstraction should address the following open issues: – Employment of multi-level abstraction to other modeling primitives beyond properties and simple relationships, such as Composite Objects and Roles, and to common modeling languages, such as UML. – Query Languages for multi-level models, especially for extracting two-level models or retrieving multi-level sub-models. – Multi-Level Constraints, especially in the context of relationships (e.g., cardinality constraints) or for keys (multi-level identifiability).
References 1. Aschauer, T., Dauenhauer, G., Pree, W.: Multi-level Modeling for Industrial Automation Systems. In: EUROMICRO-SEAA, pp. 490–496 (2009) 2. Atkinson, C.: Meta-Modeling for Distributed Object Environments. In: EDOC (1997) 3. Atkinson, C., Auletta, V.: The Essence of Multilevel Metamodeling. In: Gogolla, M., Kobryn, C. (eds.) UML 2001. LNCS, vol. 2185, pp. 19–33. Springer, Heidelberg (2001) 4. Atkinson, C., K¨ uhne, T.: Model-Driven Development: A Metamodeling Foundation. IEEE Software 20(5), 36–41 (2003) 5. Atkinson, C., K¨ uhne, T.: Reducing accidental complexity in domain models. Software and System Modeling 7(3), 345–359 (2008) 6. Cardelli, L.: Structural Subtyping and the Notion of Power Type. In: POPL, pp. 70–79 (1988) 7. Chen, P.P.: The Entity-Relationship Model - Toward a Unified View of Data. ACM Trans. Database Syst. 1(1), 9–36 (1976) 8. Dahchour, M., Pirotte, A., Zim´ anyi, E.: Materialization and Its Metaclass Implementation. IEEE Trans. Knowl. Data Eng. 14(5), 1078–1094 (2002) 9. Goldstein, R.C., Storey, V.C.: Materialization. IEEE Trans. Knowl. Data Eng. 6(5), 835–842 (1994) 10. Gonzalez-Perez, C., Henderson-Sellers, B.: A powertype-based metamodelling framework. Software and System Modeling 5(1), 72–90 (2006) 11. Gutheil, M., Kennel, B., Atkinson, C.: A Systematic Approach to Connectors in a Multi-level Modeling Environment. In: Busch, C., Ober, I., Bruel, J.-M., Uhl, A., V¨ olter, M. (eds.) MODELS 2008. LNCS, vol. 5301, pp. 843–857. Springer, Heidelberg (2008) 12. Henderson-Sellers, B., Gonzalez-Perez, C.: Connecting Powertypes and Stereotypes. Journal of Object Technology 4, 83–96 (2005) 13. Hull, R., King, R.: Semantic database modeling: survey, applications, and research issues. ACM Comput. Surv. 19(3), 201–260 (1987) 14. Klas, W., Schrefl, M.: Metaclasses and Their Application - Data Model Tailoring and Database Integration. Springer, Heidelberg (1995)
92
B. Neumayr, M. Schrefl, and B. Thalheim
15. K¨ uhne, T., Schreiber, D.: Can programming be liberated from the two-level style: multi-level programming with deepjava. In: OOPSLA, pp. 229–244 (2007) 16. K¨ uhne, T., Steimann, F.: Tiefe Charakterisierung. In: Modellierung, pp. 109–119 (2004) 17. Motik, B.: On the Properties of Metamodeling in OWL. J. Log. Comput. 17(4), 617–637 (2007) 18. Neumayr, B.: Multi-Level Modeling with M-Objects and M-Relationships. Ph.D. thesis, Johannes Kepler Universit¨ at Linz (2010), http://www.dke.jku.at/research/publications/PT1002.pdf 19. Neumayr, B., Gr¨ un, K., Schrefl, M.: Multi-Level Domain Modeling with M-Objects and M-Relationships. In: Link, S., Kirchberg, M. (eds.) APCCM. CRPIT, vol. 96, pp. 107–116. ACS, Wellington (2009) 20. Neumayr, B., Schrefl, M.: Multi-level Conceptual Modeling and OWL. In: Heuser, C.A., Pernul, G. (eds.) ER 2009 Workshops. LNCS, vol. 5833, pp. 189–199. Springer, Heidelberg (2009) 21. Neumayr, B., Schrefl, M., Thalheim, B.: Hetero-Homogeneous Hierarchies in Data Warehouses. In: Link, S., Ghose, A. (eds.) APCCM. CRPIT, vol. 110, pp. 61–70. ACS, Brisbane (2010) 22. Odell, J.J.: Power Types. In: Advanced Object-Oriented Analysis & Design Using UML, pp. 23–32. Cambridge University Press, Cambridge (1998); also published as: James Odell: Power Types. JOOP 7(2), 8–12 (1994) 23. Oliv´e, A.: Conceptual Modeling of Information Systems. Springer, Heidelberg (2007) 24. Pirotte, A., Zim´ anyi, E., Massart, D., Yakusheva, T.: Materialization: A Powerful and Ubiquitous Abstraction Pattern. In: VLDB, pp. 630–641, 0605 (1994) 25. Schewe, K.D., Thalheim, B.: Component-driven engineering of database applications. In: APCCM, pp. 105–114 (2006) 26. Schrefl, M., Tjoa, A.M., Wagner, R.: Comparison-Criteria for Semantic Data Models. In: ICDE, pp. 120–125 (1984) 27. Sch¨ utz, C.: Extending data warehouses with hetero-homogeneous dimension hierarchies and cubes – A proof-of-concept prototype in Oracle. Master’s thesis, Johannes Kepler Universit¨ at Linz (2010), http://www.dke.jku.at/research/publications/MT1002.pdf 28. Thalheim, B.: Entity-Relationship Modeling: Foundations of Database Technology. Springer, Heidelberg (2000) 29. Thalheim, B.: Component development and construction for database design. Data Knowl. Eng. 54(1), 77–95 (2005) 30. Thalheim, B.: Visual SQL (2008), http://www.informatik.uni-kiel.de/en/is/miscellaneous/visualsql/
Random Thoughts on Multi-level Conceptual Modelling Brian Henderson-Sellers School of Software Faculty of Engineering and Information Technology University of Technology, Sydney P.O. Box 123, Broadway, NSW 2007, Australia [email protected]
Abstract. Conceptual modelling has a lengthy history. In the context of software engineering, over the last three decades, conceptual modelling has moved from largely being used for data representation, especially in databases, to applications in metamodelling, ontologies, method engineering, standards and business process management. Here, I reflect, somewhat randomly, upon some of these issues and possible future application/research areas for conceptual modelling. Of prime importance must be the synergistic exploration of model quality, a topic poorly represented in the conceptual modelling literature. Keywords: Conceptual modelling, metamodelling, ontologies, standards, methodology, quality.
1 Introduction Modelling has many guises e.g. physical modelling for constructing cars or testing the flight capability of aircraft, for evaluating the stress for dams and reservoirs before construction e.g. [1], for weather forecasting, for representing a business information system e.g. [2], for mathematical modelling of ecosystems e.g. [3] or for representing the more ephemeral characteristics of music e.g. [4]. In information systems and computing in general, many of these modelling approaches are, of course, irrelevant; rather, the subset of modelling known as “conceptual modelling” has been highlighted within the community [5]. In reality, conceptual modelling is just modelling but applied to cognitive artefacts, particular designs for software systems in our context. Conceptual modelling typically uses graphical representations (ER, data modelling, object-oriented modelling languages and so on) as compared to mathematical modelling using partial differential equations, the more likely norm in the physical sciences. In this paper, I discuss some current issues, questions and concerns related to conceptual modelling. Since modelling relies heavily on the notion of abstraction [6], this is discussed first – in Section 2, where several kinds of mapping are highlighted. Abstraction leads naturally to the recognition that modelling may be useful on several cognitive “levels”. In Section 3, multi-level frameworks relevant to IT are explained, introducing specifically the notion of “strict metamodelling” and its problems in modern-day usage together with the mathematically based powertype, which is used in Kaschek/Delcambre (Eds.): The Evolution of Conceptual Modeling, LNCS 6520, pp. 93–116, 2011. © Springer-Verlag Berlin Heidelberg 2011
94
B. Henderson-Sellers
some recent metamodelling standards. The ideas of metamodels are closely related to those of ontologies as discussed in the conceptual modelling and software engineering literature over the last decade (Section 4). How ontological thinking is merged with metamodelling approaches is discussed in detail. All of these discussions lead to the obvious question: “How do we assess model quality?” Some initial observations are found in Section 5 followed by a short evaluation of some of the application areas for conceptual modelling (Section 6). We end with a short indication of some possible future research directions (Section 7).
2 Abstraction Models all have one thing in common: they are representations of conceptualizations of the real world, called here generically the System Under Study or SUS (Figure 1(a)). The conceptualization creates a cognitive or mental model that is said to be an abstraction of the (part) of reality under study e.g. [7-10], which simplifies our conception of the real domain (the SUS) by removing (unnecessary) detail [11,12]. The elements of the cognitive model are then represented in some fashion e.g. using a graphical notation, a natural language, a mathematical equation – which may itself add more detail peculiar to the model of the SUS (see the TI abstraction of Guichiglia and Walsh [13]). This is named the plenty property by Kaschek [14]. Although the link from Conceptualization to Model (Figure 1) is called the representational mapping in ontology (e.g. [8]), many modellers in software engineering and information systems label the “shortcut” (the horizontal “refers-to” relationship in Figure 1(a)) from Model to SUS as “represents” (Figure 1(b)). This can be formulated [9,15,16] as a mapping α given by α=τoπ
(1)
where π represents the abstraction mapping (called “projection” in [9]) and τ the representation mapping of Figure 1(a).
(a)
represents
Model
depicts
abstracts
refers to
Conceptualization
(b)
Conceptualization
SUS
Model
abstracts
represents
Fig. 1. Ogden and Richards’ [17] triangle applied to modelling
SUS
Random Thoughts on Multi-level Conceptual Modelling
95
However, although we acknowledge this two-stage process from SUS to model via conceptualization as being a more appropriate description of the modelling process, clearly differentiating between abstraction and representation, for the remainder of our discussion of the literature, we will adopt the more prevalent view that “a model represents the SUS” (Figure 1(b)). Abstraction is thus crucial for modelling. Informally, Giunchiglia and Walsh [13] propose that 1) abstraction maps a representation of the problem to a new, more “abstract” representation. (This is the essence of modelling); 2) by preserving relevant, desirable properties, the abstraction process allows one to solve the original problem; 3) by throwing away details, the result of the abstraction process provides a simpler problem to be solved than the original; and proceed to formalize the idea of an abstraction function, α, from a notion1 S to a notion A where (e.g. [18]) α: S ⇒ A
(2)
Formally, an abstraction can be defined as a mapping between two formal systems, which may be similar or dissimilar (e.g. [19]). Here, a formal system, Σ, is a set of formulae, Θ, written in a language Λ i.e. Σ = (Λ,Θ)
(3)
Giunchiglia and Walsh [13] then equate the set of formulae, Θ, with the language, Λ. Then, an abstraction is the mapping between two formal systems, Σ = (Λ). The definition (which addresses property 1 above) given in [13] is: f : Σ 1 ⇒ Σ2
(4)
where the mapping is between a pair of formal systems (Σ1, Σ2) with languages Λ1 and Λ2 respectively and there is an effective total function, fΛ, that maps “ground language” Λ1 to “abstract language” Λ2 i.e. fΛ : Λ1 → Λ2
(5)
Equation (5) is given in [11] as the basic definition of abstraction. These authors note that this mapping needs to be both total and surjective so that, respectively, for each element of Λ1, fΛ is defined. Giunchiglia and Walsh [13] introduce three kinds of abstraction, which they call TD, TI and TC (where a TC abstraction is both a TD and a TI abstraction). These are picture in Figure 2. In a TD abstraction, some elements of the base situation are deleted; TI abstractions are when the abstract language has additional axioms or inference rules; and TC abstractions are effectively bijective mappings. 1
Although we do not discuss notions here, we follow Kashek [18] in assuming models to be implicitly systems of notions.
96
B. Henderson-Sellers Base language
Abstract language
TD
TI
TC
Fig. 2. Three kinds of abstraction of Giunchiglia and Walsh [13]
Keet [12] proposes a set of abstractions similar to the TC, TD and TI abstractions of [13] but more relevant to conceptual modelling and ontologies as follows: R-abs: An abstraction is said to be an Rabs abstraction if and only if the more detailed type φ abstracts into its related parent type ψ (6) This makes an isomorphic mapping F-abs: An abstraction is said to be an Fabs abstraction if and only if a more detailed theory T0 (with φ1…φn entity types, relations and constraints among them) abstracts into a single related parent type, ψ, in T1 that is distinct from (7) any φi This makes F-abs an on to or surjective morphism D-abs: An abstraction is said to be a Dabs abstraction if and only if a more detailed theory T0 with φ1…φn entity types, relations and constraints abstracts into theory T1 with φ′1…φ′n entity types, relations and constraints, and with m
Random Thoughts on Multi-level Conceptual Modelling
97
As noted above, one aim of modelling is to provide assistance with the original problem e.g. [10,11]. This may be to gain understanding of an existing condition or to forecast some future condition. These two modes of models are often referred to as descriptive and prescriptive [10] or as backward-looking and forward-looking (see [20] for further discussion on this topic). Other modes found in information systems are given [14] as prognostic (a forecast mode), idealizing (as when a prescribed process is only an idealization of the original) and comprehensive (including quality characteristics – see also Section 5 below). There are several different mechanisms that can be employed to simplify an entity and thus obtain an abstracted one e.g. [11]. The source and resultant entities, depending on the mechanism used, may be concept or a non-concept entities. Some abstraction mechanisms maintain the typing level, i.e. produce an entity of the same type (concept or non-concept) as the source entity. Examples of such abstraction mechanisms are granularity and synopsis, which can operate on any kind of entities, plus generality, which operates only on concepts. They are essentially identity-preserving mappings. Other mechanisms, on the other hand, increment the typing level, i.e. produce a concept entity that is one level above the source entity i.e. the source and target are of different types e.g. [21] – and therefore may need different languages for their description. Classification/Categorization and Typing belong to this category. These are non-identity preserving mappings. (An interesting comparison between these two mechanisms of classification and generalization is given in [22]). There are also parallels with the proposals of Kühne [9,23] of type models and token models. Type models rely on classification to create a type hierarchy. Classification is of course linked to instantiation (in fact, its inverse). Kühne argues that applying a type model twice results in a metamodel. In contrast, token models are those that rely on granularity abstraction (rather than classification abstraction). Thus, a fine grained map is a token model of the system under study (SUS) (the actual topography) and a coarse grained map is both a model of the fine-grained map and also a model of the SUS (Figure 3).
Key
name
object direction of mapping
Fig. 3. Set of models that do not fulfil the criteria for metamodelling but rather illustrate varying granularities (after [9, figure 1, © 2006]) With kind permission of Springer Science+Business Media
As noted above, abstraction, as a means of simplifying, can be supported by a range of mechanisms, including generalization (set/subset), instantiation (type/instance) and aggregation (whole/part). Thus, we find in the literature discussion of abstraction and
98
B. Henderson-Sellers
“abstraction levels” where the languages Λ1 and Λ2 are the same (as would be the case for generalization and aggregation) or different (type/instance). When we discuss modelling language and multiple levels (Section 3), this can be seen immediately to lead to potential ambiguities.
Fig. 4. Examples of isotypical, prototypical and metatypical interpretive mappings. A circle is shown to have an isotypical mapping. The shaded square on the model is an example (prototype) for any square in the SUS. The declaration of “pentagon” in the model is metatypically mapped to all possible pentagons in the SUS (after [24]).
Another viewpoint is provided by [24] in an analysis of the representation relationship and the abstraction mechanism discussed above. They call these interpretive mappings, which link an individual entity within the SUS and its surrogate homomorphic entity inside the model. These authors identify three types of interpretive mappings (Figure 4), which are all representational mappings (Figure 1(a)): • • •
Isotypical mappings are those that map one model entity to one SUS entity. The model entity straightforwardly represents the SUS entity. It relates to the inverse of the R-abs of Keet [12] – Equation 6; Prototypical mappings are those that map one model entity to a set of SUS entities given by example, i.e. the model entity exemplifies the kind of SUS entities that can be mapped to it; Metatypical mappings are those that map one model entity to a set of SUS entities given declaratively, i.e. the model entity is a description;
noting that the OMG uses a metatypical mapping as its representational relationship. Although many authors and users think of conceptual modelling as being concerned with how to draw diagrams to represent their systems or their systems-to-be, for such a diagram to be of quality, the notation must have well-defined syntax and coupled semantics (e.g. [25]), often provided in the form of a metamodel or possibly an ontology [26]. The metamodel typically defines the syntax of a “modelling language” with its semantics defined in natural language or a simplified logic. We can say that a metamodel is “a model of models” e.g. [27]; in other words, a metamodel is itself a model – and must therefore be defined by a modelling language: the metametamodel; and so on. Practicality requires closure and this is usually done at this metameta-level. A natural consequence of models is therefore seen as a set of multiple domains, sometimes called layers or levels, which are linked by the form of a “representation” relationship [28,29] or an interpretive mapping [24]. It is the detailed form of both the multi-domain architecture and the representation relationship that has
Random Thoughts on Multi-level Conceptual Modelling
99
provided a significant research focus over the last decade and is discussed briefly in the next section.
3 Multiple Levels As noted in Section 2, abstraction can lead to the concept of multiple modelling languages and multiple “levels” or “domains” with one of a set of possible representational relationships chosen to link adjacent levels.
M3 (MOF) Metametamodel instance_of
M2 Metamodel instance_of
M1 Model instance_of
M0 Data Fig. 5. The four layer architecture advocated by, inter alia, the Object Management Group and based on [35] (after [36]) © Pearson Education Limited
In traditional object-oriented (OO) technology, linked as it is to set theory and category theory e.g. [30], this is often regarded as an allocation of the “set” and the members of that set to two different metalevels, as is done for instance in the OMG 4level architecture (Figure 5), which is stated to use strict metamodelling, defined in [31,32] as a kind of modelling across multiple layers that uses only “instance-of” relationships between layers – equivalent to the mathematical representation of an individual as a member of a set (Figure 6). This particular kind of representational relationship is in fact the classification abstraction and has gained superiority in the OMG/UML/MOF/MDA world over other kinds of abstraction. Consequently, there are many e.g. [33,34] that use the nomenclature of “abstraction levels” for the four layers of Figure 5, stating that each layer is “at a higher level of abstraction” than its predecessor. However, the analysis of strict metamodelling has more recently uncovered problems. For example, while the type/instance relationship is useful across two layers, when extended to three layers, there is a problem since an instance of a type cannot be further instantiated [20] because instantiation (which is an intransitive relationship) here refers to the selection of one individual from a set of individuals i.e. an instance is an indivisible individual (Figure 6). Another problem is highlighted in [37] in terms of what these authors call ontological versus linguistic metamodelling. Detailed
100
B. Henderson-Sellers
analysis of these two kinds of metamodelling led these authors to making several proposals including two-dimensional representations of architectures to replace the onedimensional one of Figure 5 [38]. These and other metamodel problems are discussed briefly in [39] and in more detail in [40]. set x
x x
x x
x
x x
x x x
x
x x
individual members of the set
Fig. 6. A set is a collection of individual elements (called members). A set S is defined by S ={si}, i=1 to n.
To obviate some of these concerns (highlighted in the previous paragraph, especially the intransitivity of the instance-of relationship) caused by the use of strict metamodelling and the architecture of Figure 5, Gonzalez-Perez and HendersonSellers [41] proposed the use of powertypes [42] for the representational link between domains together with a revised architecture focussed not on an instantiation relationship (as Figure 5) but on the domains of interest to real software development professionals. Figure 7 shows the three domains thus defined: Endeavour (including actual projects), Model (designs, tools, methodologies, handbooks) and Metamodel (modelling language): an architecture used recently in the International Standard ISO/IEC 24744 [43]. The use of the powertype relationship (Figure 8) introduces some interesting possibilities since it permits some of the attributes defined in a (standard) metamodel to be instantiated to actual values in the Method Domain whilst others are passed unchanged through to the Endeavour Domain before they are given actual values. Figure 8 shows one such example in which the Team entity has both a class facet (derived from the OrganizationalGrouping class in the metamodel) and an object facet (derived from the OrganizationalGroupingKind class). The two classes in the Metamodel Domain (OrganizationalGrouping and OrganizationalGroupingKind) are linked together via a classification relationship.
4 Ontologies and Metamodels The use of ontologies in software engineering, although over a decade old e.g. [45], has been increasingly introduced into discussions on methodology and conceptual modelling e.g. [46-50]. Indeed, one could view an ontology as a particular kind of model e.g. [14,51]. However, the relationship between ontologies, metamodels and conceptual models is not clear [52] – Atkinson et al. [53] compare ontologies with models whereas Devedzic [54] contrasts them with metamodels. Ruiz and Hilera [55, p64] also note the confusion between the terms metamodel and ontology. They suggest that this is because they are often depicted with the same language. They also
Random Thoughts on Multi-level Conceptual Modelling
101
suggest that there may be differences based on focus in that metamodels seek to improve similar but different models whereas an ontology does this for knowledge models. They also argue that an ontology is descriptive (of the problem domain) whereas a metamodel is prescriptive, belonging to the solution domain. However, this seems something of a “red herring” since, as [20] points out, prescriptive and descriptive are terms used to describe the mode of usage of a model rather than being a mandated characteristic of a “model”.
endeavour
method methodologies
assessment
quality
tools
WorkUnit
metamodel Activity
Task
Technique *
*
Fig. 7. Three layer architecture of recent standards (after [44])
Organizational Grouping Kind Intra-Action: Boolean
Relationship is is-instance-of
Relationship is classification Team Intra-Action = YES
Organizational Grouping GroupMemberNames
Relationship is generalization
GroupMemberNames
My team (Brian, Cesar, Atif)
Fig. 8. A powertype representation for Team. This method fragment inherits from OrganizationalGrouping and thus has an attribute GroupMemberNames, which is not allocated actual values until it is instantiated (in the Endeavour Domain: see Figure 7), and also has an instantiation relationship to a second class in the metamodel (OrganizationalGroupingKind). The attributes on this metalevel class are allocated values in the Method Domain – here Intra-Action = YES.
102
B. Henderson-Sellers Design Solution Domain Prescriptive (Specifications)
Analysis Problem Domain Descriptive
Metametamodels (specification concepts)
M3 metametamodel level
«instance-of» «instance-of»
Metamodels (languages, language concepts)
Upper ontologies (description concepts)
M2 metamodel level
«instance-of»
M1 model level Domain ontologies
Models
«described by»
M0 object level
Real-world objects «described by»
«instance-of»
Software objects
Fig. 9. Proposed integration of an ontological view with a four-layer metamodel architecture (after [57, figure 9.7, © 2006]) With kind permission of Springer Science+Business Media])
Venn diagram-style representation
Activity class (set partition)
ActivityKind class (set membership) Design x
x x
x x
x
Code x x
x x x
x
x
xDesign x Code
x x Test
Test
UML-style representation Activity
Design
Test
generalization
ActivityKind
Design
is-instance-of
Fig. 10. Comparison of set partition and set membership using (upper) Venn diagrams and (lower) UML-style class notation
Whatever the alignment between models, metamodels and ontologies, it is clear that, similar to the arguments in Section 3 above for just models and metamodels, there is a need for a multi-level model architecture e.g. [56] in which all three entities can co-exist. For instance, that proposed in [57] is based on Figure 5 in that it attempts to
Random Thoughts on Multi-level Conceptual Modelling
103
add the ontology construct into the OMG strict metamodelling hierarchy. As can be seen in Figure 9, these authors find themselves unable to accomplish this without breaking the rules of strict metamodelling (which permits only instance-of relationships between levels) as mandated within the OMG architecture. Instead, they introduce a generalization relationship2 (which represents set partition rather than the set membership of instantiation: see Figure 10) between domain ontologies at the OMG M1 level and upper ontologies (M2). This combination of generalization and instantiation (in Figure 9) is suggestive of the use of powertypes as used in [43], leading to the tentative proposal shown here in Figure 11. Here, the powertype pattern is between levels M1 and M2 as extracted into Figure 12 for clarity. Whether this pattern can be substantiated remains a topic for future research. Design Solution Domain Prescriptive (Specifications)
Analysis Problem Domain Descriptive
Metametamodels (specification concepts)
M3 metametamodel level
«instance-of» «instance-of»
M2 metamodel level
Upper ontologies (description concepts)
«classified by»
Metamodels (languages, language concepts)
«instance-of»
M1 model level Domain ontologies
Models
«described by»
M0 object level
«instance-of»
Real-world objects «described by»
«instance-of»
Software objects
Fig. 11. Introduction of a powertype into the pyramid structure of Aßmann et al. [57] shown in Fig. 7
Upper ontologies
«classified-by»
Metamodels
«instance-of»
Domain ontologies
Fig. 12. Powertype embodied in Figure 11 2
Although they later replaced this by a “described-by” relationship.
104
B. Henderson-Sellers
Such a “hybrid” architecture may help to resolve both the internal contradiction of the Ontology-based software Development Environment (ODE) approach of [58] (discussed in [52]) and the architecture proposal in [57] as demonstrated in Figure 9. Although offered as a first attempt at such an integration, the full analysis of this potentially interesting architectural model to unite ontologies, conceptual models and metamodels is outside the scope of this paper and must remain a topic for future research. The discussion so far has been focussed on underlying theoretical issues relating to abstraction, multi-level architectures, ontologies and metamodels. The common thread is that of a “model” as being representative of a SUS (Figure 1(b)) whilst recognizing that this is a shortcut for an abstraction mapping followed by a representational mapping as depicted in Figure 1(a). Kaschek [14] also raised the issue of the comprehensive mode for all model types wherein the quality is evaluated on the basis of a number of quality aspects or quality characteristics, these characteristics being quantifiable. Overall quality is then given by the vector of the values for each characteristic, ensuring that the measurement scales are compatible. In the next section, we consider briefly some of the quality issues most pertinent to models, metamodels and ontologies.
5 Model Quality – Some Observations Having established that both models and metamodels (and indeed metanmodels) in computing can be regarded as conceptual models (Section 3), we must now ask what the quality of such models may be – recognizing that quality is not unambiguously defined in the context of conceptual modelling [59]. Some of the concerns raised in earlier work, mostly focussed on models documented using UML [34,60] include: • Models that are clearly describing a real-life domain, such as banking or pets, being described incorrectly as metamodels. These are often readily identified since they use UML activity diagrams with generic naming; • Incorrect and untenable semantics for whole-part relationships in UML e.g. [61,62]. These relate to contradictions in the OMG documents, multiple definitions (as whole-part or as non-whole-part ownership) and to a partitioning that creates both overlaps and true subsets being excluded from all definitions (of aggregation and composition); • Incorrect definitions of UML stereotypes [63] leading to much misuse. Unfortunately, such misuse, originally discussed in [64], has recently spread from the object-oriented literature to the agent-oriented literature e.g. [65]; • Use of UML modelling in which the metamodel diagram is clearly inconsistent or contains multiple, overlapping subtypes (e.g. Figure 13 in which the reader may not realize that a “tactical guideline” must also be either a “chunk” or a “not-chunk” – and vice versa). If used for modelling purposes, these should, at the least, be identified by the application of a discriminant [66,67] – although it should be noted that the appearance of a discriminant on a conceptual model is not readily implementable and often indicates a design that needs further quality improvement.
Random Thoughts on Multi-level Conceptual Modelling
Strategic Guideline
Tactical Guideline
1
Guideline
105
Simple Guideline
1..*
is based on
1..*
Product Part
represented by
1
Chunk
Not-Chunk
Fig. 13. A simple example of a published metamodel, expressed using UML’s notation, in which OO/UML subtyping has been used in poorly by creating two subtyping ranges: an arguably complete partition between three guidelines (suggested missing discriminant of “purpose”) together with a clearly complete partition based on chunk-ness (suggested missing discriminant of “internal structure’ (after [39]).
The quality of conceptual models, especially those that are standardized metamodels, is an important factor in ensuring successful use of contemporary design and development practices in industry worldwide. Regrettably, the quality of some current metamodel standards is sub-standard. Inconsistencies and ambiguities occur in the widely used OMG [34] and ISO [60] UML standard for depicting conceptual models [26,68,69]. Even the four-layer architecture itself, especially when applied to methodologies rather than more straightforward modelling languages, is found to be problematic, as noted above (see full discussion in [40]), in that a “M2” metamodel (Figure 5) cannot be used to support the “M0” level elements. Identifying appropriate metrics to apply to quantify the “quality” of such models and metamodels is not easy. Although complexity metrics abound in the literature, few are well validated, both theoretically and empirically. Furthermore, metrics by themselves have little relevance to software quality improvement – unless underpinned by an appropriate conceptual model [70]. The linking of readily measured internal characteristics of software to the much more useful external characteristics remains a challenge. One problem is the propensity of researchers to invent a metric without consideration of how, or even if, it may be useful e.g. [71]. Indeed, a better guiding framework is that of the GQM (Goal-Question-Metric) approach [72] – as was applied, for instance, in the empirical study of [73]. While such observations can be useful, what is needed are objective software engineering metrics that can be applied to assess the quality of conceptual models. Some early work on complexity metrics (which is but one of the contributing factors to quality) was summarized in [70]. Such metrics are reasonably well understood although still poorly validated in practice. Typical measures are of coupling and cohesion [74], fan-in/fan-out [75], inheritance e.g. [76] and reusability [77]. Consistency in terminology of software measurement is urged in [78,79] in terms of proposals for an ontology as a first step in terminological harmonization.
106
B. Henderson-Sellers
Other recent compendium works include [80] and [81]. There are also many individual papers attempting to introduce quality assessment into conceptual modelling although these are as yet far from definitive in terms of a large-scale and accepted body of software engineering metrics that can be applied by conceptual modellers. Interesting recent examples include [82-84]. Quality in broad terms can also be analyzed in terms of syntactic, semantic and aesthetic issues [85,86] and was, interestingly, the topic of a call for papers issued in March 2008 for a special issue of the international journal Information and Software Technology – published in December 2009 (volume 51, issue 12). In addition, a metamodel is not just a model (and can therefore be assessed with any model-focussed metrics); it is also a language [20,87,88] since it defines the syntax and often some or all of the semantics necessary for a language specification. This second aspect of a metamodel will likely need different quality metrics, metrics that have not yet been investigated (see [88] for further discussion on this interesting challenge). Measures of the dynamic aspects of software development are much harder. ISO 15504 [89] offers some means to measure actual processes as they are enacted on projects. This is accomplished in terms of the organizational “capability”. Other simple checks are possible; for example, checking that all task outputs are needed elsewhere as inputs to other tasks, which itself leads to the notion of a work product pool approach ([90] and Figure 14) in which a software development methodology is created by analyzing each work product and looking at its precursors, starting with the final, delivered software application.
Class Model
e
Requirements Spec ification
Software System
F
Use Case Specification
i
Reusabl e Asset Pool
Key: e=externally provided ii=internally available F=final
Fig. 14. Small example of the application of the work product pool approach to software development. [Note that in this diagram, boxes represent work products and arrows represent necessity (such as “a class model is necessary to develop the software system”).] (after [90] Copyright Elsevier, 2008)
Any metrics/measures that are derived must be mathematically sound, usually being based upon measurement theory (see, e.g. [91]) and must have appropriate mathematical qualities (some of these are outlined in [92] although some are disputed). If the measure is theoretically sound, then it can be evaluated for utilization in some prediction chain. Fenton [93] outlines how a simple metric, which is typically a count of some internal feature (such as lines of code, decision points) of the target
Random Thoughts on Multi-level Conceptual Modelling
107
software unit, can be linked, via a model, to an external feature of interest – these external features usually being called the “-ilities” since they include features such as maintainability, reliability etc. External features are not, in general, immediately deducible from internal features unless an appropriate conceptual model linking them is determined. Typically, some empirical analysis is needed here – although simple statistics, especially correlation analysis, is rarely sufficient. Fenton [71] notes the poor quality of much of the metrics literature. Again a combination of modelling and reflective quality evaluation is critical to the success of this sub-discipline of software engineering.
6 Application of Conceptual Models Application areas of conceptual models (be it modelling languages or metamodels and ontologies) are multifarious. Obvious examples are object technology, componentbased approaches, agents, entity-relationship approaches and services; methodologies, processes and capability assessment; workflow and business process modelling (BPM). Furthermore, conceptual models, and especially metamodels, are critical to the success of software drawing tools, Computer-Aided Software Engineering (CASE) tools and Computer-Aided Method Engineering (CAME) tools. In this section, we examine just two broad areas of the application of conceptual models. In Section 6.1 we briefly consider the use of object technology, agent technology and method engineering – all areas of current research and practical experience within software engineering. In Section 6.2 we look at the parallel domain of business process modelling and especially the cross-overs between business issues and IT issues for conceptual models. 6.1 Objects, Agents and Method Engineering Applications of Modelling Within software engineering, conceptual models may be useful to describe work products, processes or roles played by people. A well-known example in object modelling is the Unified Modeling Language (UML) [34,60]. From a quality point of view, however, UML has many flaws (see Section 5). Conceptual modelling is also used in agent modelling languages such as INEGENIAS’s underpinning metamodel [94], the FAML metamodel [95,96] and the work of Azaiez et al. [97]. In the field of methodologies and processes, conceptual modelling also has a long history with the current culmination in the methodology metamodel captured as SEMDM (Software Engineering Metamodel for Development Methodologies) [43]. A combination of work product, process and people/roles3 lays the foundation for the use of conceptual models in building full, industry-strength methodologies using a technique known as situational method engineering (SME) [99-103]. SME uses metamodels (conceptual models of work products and of static descriptions of process models) from which to generate method fragments that are the elements from which the situationally specific methodology is composed. That very construction is also 3
This third element, roles, is less well researched but interesting models have been proposed, for instance in [98] or as part of FAML [96].
108
B. Henderson-Sellers
modelled as a process i.e. another application of conceptual modelling. Three such approaches are contrasted in [104] – although much still remains to be done. Furthermore, conceptual models now play a key role in international standards, from both specialized bodies like OMG and FIPA as well as international organizations like ISO. These standardization efforts have resulted in (a) widely used modelling languages, such as the Unified Modeling Language [60] that are applicable to depicting and communicating the conceptual framework of information systems work products and (b) underpinning metamodels such as CDIF [105] and SEMDM [43]. All of these metamodel-based standards are used to generate model elements that together compose the information systems model, while themselves being defined using a conceptual modelling approach. 6.2 Extending IT Modelling into Business Process Management Modelling of business processes also requires the application of conceptual modelling. The close connection between (IT) process models and processes executed in the business domain has led to an interesting convergence. For example, commencing in 2001, the Business Process Modeling Notation (BPMN) was developed within the business community (Business Process Management Initiative: http://bpmi.org). However, more recently, the responsibility for standardization of BPMN has been taken over by the Object Management Group [106], a group specifically set up to address standards in object technology and not business. A second link results from the fact that IT underpins information systems and at the same time information systems underpin business and business processes. These links are likely to be strengthened with the suggestion (OMG members’ emails: Oct and Nov 2008) that BPMN should be recast as a UML profile. Indeed, a major element of BPMN has strong similarities to the Activity Diagram of UML. A parallel OMG initiative seeking synergy between business processes and IT is in terms of the Business Process Definition Metamodel [107]. The challenges facing the BPM (Business Process Management) community are discussed in [108,109] including concerns about lack of standards, weaknesses in process specification and lack of methodology – all potential topics of interest to conceptual modellers. A second area that links the use of conceptual models in both business and IT is to be found in the work of the Workflow Management Coalition (WfMC) who offer a standard workflow model to depict the steps in a business process. Indeed, as pointed out in [110, p8], a workflow may be construed as a packaging of a number of services. This establishes another link since [111] suggests that method fragments should be recast as method services. The only other direct link between business process management (BPM) and situational method engineering (SME) that has been identified is that of [112] who argue in favour of adaptation of metamodels for business process modelling with appropriately conformant business fragments and business chunks i.e. the merger of traditional SME with BPM.
7 Future Research Opportunities Conceptual modelling, although originating largely for data representation, especially in databases, has matured far beyond that single usage. In software engineering, conceptual
Random Thoughts on Multi-level Conceptual Modelling
109
models underpin many research and industry software developments using various technologies such as objects, agents and services. Ontologies provide a further integration challenge in the software engineering domain. Whilst metamodelling is growing in its presence in the literature, there are still several unsolved issues (as discussed here). In addition, as a topic for future research, there is the challenge of aligning (or perhaps integrating) the broad range of metamodels that now exist [113]. The context of model-driven engineering (MDE) and the role of megamodels [27,114] and their possible contributions to further clarify the relationship between linguistic and ontological metamodelling [37] are of increasing interest. Methodology construction uses many aspects of conceptual models, both in terms of the influencing characteristics and the construction process. A useful approach here, but one that has yet to find large-scale industry adoption (and hence further applied research opportunities), is that of situational method engineering. While such approaches rely on metamodels, at present these are static and cannot be modified without significant effort. A future research challenge is that of SMME (Situational Metamodel Engineering [115]) in which means to modify the metamodels themselves are utilized to adapt published metamodel standards to specific method engineering contexts. A similar approach is found in [116] in which the large UML metamodel is “pruned” to an “effective metamodel”. For all these applications, quality is paramount and has not been satisfactorily explored. Preliminary attempts, such as those of [116-121], for example, can be used to formulate new research programmes of value both to the acquisition of knowledge (basic research) and its industrial application. Linked to the notion of quality is that of formality insofar as several authors seek an underpinning theory for conceptual modelling [20,29,114,122-126] – a topic that will surely be further developed in the near future. Finally, it must be remembered that if standards are released that are not of adequate (i.e. high) quality, there is a danger they will be ignored in toto, even though the deficiencies are proportionally small. Conceptual modelling itself may be discarded if software development teams become disillusioned by poor quality supporting products – standards and tools built to effect those standards.
Acknowledgements I wish to thank Cesar Gonzalez-Perez for his comments on an early version of this paper. This is contribution 09/03 of the Centre for Object Technology Applications and Research within the Centre for Human Centred Technology Design at the University of Technology, Sydney.
References 1. Henderson-Sellers, B.: Reservoirs, p. 128. Macmillan, Basingstoke (1979) 2. Edwards, J.M., Henderson-Sellers, B.: A graphical notation for object-oriented analysis and design. J. Object-Oriented Programming 5(9), 53–74 (1993)
110
B. Henderson-Sellers
3. Henderson-Sellers, B.: Towards a conceptual model of randomness. Ecological Modelling 85, 303–308 (1996) 4. Henderson-Sellers, B., Cooper, D.: Has classical music a fractal nature? - a re-analysis. Computers and the Humanities 27, 43–506 (1993) 5. Brodie, M.L.: John Mylopoulos: sewing seeds of conceptual modelling. In: Borgida, A.T., Chaudhri, V.K., Giorgini, P., Yu, E.S. (eds.) Conceptual Modeling: Foundations and Applications. LNCS, vol. 5600, pp. 1–9. Springer, Heidelberg (2009) 6. Roussopoulos, N., Karagiannis, D.: Conceptual modeling: past, present and the continuum of the future. In: Borgida, A.T., Chaudhri, V.K., Giorgini, P., Yu, E.S. (eds.) Conceptual Modeling: Foundations and Applications. LNCS, vol. 5600, pp. 139–152. Springer, Heidelberg (2009) 7. Stachowiak, H.: Allgemeine Modelltheorie. Springer, Vienna (1973) 8. Guizzardi, G.: Ontological Foundations for Structural Conceptual Models, Enschede, The Netherlands (2005) 9. Kühne, T.: Matters of (meta-) modelling. Software and Systems Modeling 5, 369–385 (2006) 10. Ludewig, J.: Models in software engineering – an introduction. Software and Systems Modeling 2, 5–14 (2003) 11. Ghidini, C., Giunchiglia, F.: A semantics for abstraction. In: Lopez de Mantaras, R., Saitta, L. (eds.) Procs. ECAI 2004, pp. 343–352. IOS Press, Amsterdam (2004) 12. Keet, C.M.: Enhancing comprehension of ontologies and conceptual models through abstractions. In: Basili, R., Pazienza, M.T. (eds.) AI*IA 2007. LNCS (LNAI), vol. 4733, pp. 813–821. Springer, Heidelberg (2007) 13. Giunchiglia, F., Walsh, T.: A theory of abstraction. Artificial Intelligence 57(2-3), 323– 390 (1992) 14. Kaschek, R.: Modelling Ontology Use for Information Systems. In: Althoff, K.-D., Dengel, A.R., Bergmann, R., Nick, M., Roth-Berghofer, T.R. (eds.) WM 2005. LNCS (LNAI), vol. 3782, pp. 609–622. Springer, Heidelberg (2005) 15. Hesse, W.: More matters on (meta-)modelling: remarks on Thomas Kühne’s “matters”. Softw. Syst. Model. 5, 387–394 (2006) 16. Kühne, T.: Clarifying matters of (meta-)modeling: an author’s reply. Softw. Syst. Model. 5, 395–401 (2006) 17. Ogden, C.K., Richards, I.A.: The Meaning of Meaning. Harcourt, Brace and World, New York (1923) 18. Kaschek, R.: A little theory of abstraction. In: Rumpe, B., Hesse, W. (eds.) Proceedings of Modellierung 2004. LNI, vol. 45, pp. 75–92. GI, Bonn (2004) 19. Mani, I.: A theory of granularity and its application to problems of polysemy and underspecification of meaning. In: Cohn, A.G., Schubert, L.K., Shapiro, S.C. (eds.) Principles of Knowledge Representation and Reasoning: Proceedings of the Sixth International Conference (KR 1998), pp. 245–257. Morgan Kaufmann, San Mateo (1998) 20. Gonzalez-Perez, C., Henderson-Sellers, B.: Modelling software development methodologies: a conceptual foundation. J. Systems Software 80(11), 1778–1796 (2007) 21. Machado, E.P., Traina Jr., C., Araujo, M.R.B.: Classification abstraction: An intrinsic element in database systems. In: Yakhno, T. (ed.) ADVIS 2000. LNCS, vol. 1909, pp. 57–70. Springer, Heidelberg (2000) 22. Kühne, T.: Contrasting classification with generalisation. In: Kirchberg, M., Link, S. (eds.) Procs. Sixth Asia-Pacific Conference on Conceptual Modelling (APCCM 2009). CRPIT, vol. 96, pp. 71–78 (2009)
Random Thoughts on Multi-level Conceptual Modelling
111
23. Kühne, T.: What is a model? In: Bézivin, J., Heckel, R. (eds.) Procs. Dagstuhl Seminar 04101, Language Engineer-ing for Model-Driven Software Development (2004), http://drops.dagstuhl.de/portals/index.php/semnr=04101 24. Gonzalez-Perez, C., Henderson-Sellers, B.: A representation-theoretical analysis of the OMG modelling suite. In: Fujita, H., Mejri, M. (eds.) New Trends in Software Methodologies, Tools and Techniques, pp. 252–262. IOS Press, Amsterdam (2005) 25. Harel, D., Rumpe, B.: Meaningful modeling: what’s the semantics of “semantics”? IEEE Computer 37(10), 64–72 (2004) 26. Laarman, A., Kurtev, I.: Ontological Metamodeling with Explicit Instantiation. In: van den Brand, M., Gašević, D., Gray, J. (eds.) SLE 2009. LNCS, vol. 5969, pp. 174–183. Springer, Heidelberg (2010) 27. Favre, J.-M.: Foundations of model (driven) (reverse) engineering: models. Episode I: Stories of The Fidus Papyrus and of The Solarus. In: Bézivin, J., Hockel, R. (eds.) Procs. Dagstuhl Seminar 04101 Language Engineering for Model-Driven Software Development (2005) ISSN 1862-4405 28. Seidewitz, E.: What models mean. IEEE Software 20, 26–31 (2003) 29. Muller, P.-A., Fondement, F., Baudry, B.: Modeling modeling. In: Schürr, A., Selic, B. (eds.) MODELS 2009. LNCS, vol. 5795, pp. 2–16. Springer, Heidelberg (2009) 30. Whitmire, S.A.: Object Oriented Design Measurement, p. 452. John Wiley & Sons, Inc., New York (1997) 31. Atkinson, C.: Metamodelling for distributed object environments. In: First International Enterprise Distributed Object Computing Workshop (EDOC 1997), Brisbane, Australia (1997) 32. Atkinson, C.: Supporting and Applying the UML Conceptual Framework. In: Bézivin, J., Muller, P.-A. (eds.) UML 1998. LNCS, vol. 1618, pp. 21–36. Springer, Heidelberg (1999) 33. Jørgensen, K.A.: Modelling on multiple abstraction levels. In: Procs. 7th Workshop on Product Structuring – Product Platform Development, March 24-25. Chalmers University of Technology, Göteborg (2004) 34. OMG, Unified Modeling Language: Superstructure. Version 2.1.1, OMG document formal/07-02-03 (2007) 35. ANSI: Information Resource Dictionary System (IRDS). American National Standards Institute, New York (1989) 36. Henderson-Sellers, B., Unhelkar, B.: OPEN Modeling with UML, p. 245. AddisonWesley, Reading (2000) 37. Atkinson, C., Kühne, T.: Model-driven development: a metamodelling foundation. IEEE Software 20(5), 36–41 (2003) 38. Atkinson, C., Kühne, T.: Concepts for Comparing Modeling Tool Architectures. In: Briand, L.C., Williams, C. (eds.) MoDELS 2005. LNCS, vol. 3713, pp. 398–413. Springer, Heidelberg (2005) 39. Henderson-Sellers, B.: On the challenges of correctly using metamodels in method engineering. In: Fujita, H., Pisanelli, D. (eds.) New Trends in Software Methodologies, Tools and Techniques, Proceedings of the Sixth SoMeT_ 2007, pp. 3–35. IOS Press, Amsterdam (2007) 40. Gonzalez-Perez, C., Henderson-Sellers, B.: Metamodelling for Software Engineering, p. 210. J. Wiley & Sons, Chichester (2008) 41. Gonzalez-Perez, C., Henderson-Sellers, B.: A powertype-based metamodelling framework. Software and Systems Modeling 5(1), 72–90 (2006) 42. Odell, J.J.: Power types. Journal of Object-Oriented Programming 7(2), 8–12 (1994)
112
B. Henderson-Sellers
43. ISO/IEC: Software Engineering - Metamodel for Development Methodologies, ISO/IEC 24744. International Organization for Standardization/International Electrotechnical Commission, Geneva (2007) 44. Henderson-Sellers, B.: Method engineering: theory and practice. In: Karagiannis, D., Mayr, H.C. (eds.) 5th International Conference Information Systems Technology and its Applications, ISTA 2006, Klagenfurt, Austria, May 30-31. LNI, vol. P-84, pp. 13–23. Gesellschaft für Informatik, Bonn (2006) 45. Guarino, N.: Formal ontology and information systems. In: Int. Conf. on Formal Ontology in Information Systems - FOIS 1998, Trento, Italy (1998) 46. Wyssusek, B., Klaus, H.: Ontological foundations of information systems analysis and design: extending the scope of the discussion. In: Green, P., Rosemann, M. (eds.) Business Systems Analysis with Ontologies, pp. 322–344. IGI Group Publishing, Hershey (2005) 47. Green, P., Rosemann, M.: Business Systems Analysis with Ontologies. IGI Group Publishing, Hershey (2005) 48. Calero, C., Ruiz, F., Piattini, M. (eds.): Ontologies in Software Engineering and Software Technology, p. 339. Springer, Berlin (2006) 49. Hesse, W.: Engineers discovering the “real world” - from model-driven to ontologybased software engineering. In: Kaschek, R., Kop, C., Steinberger, C., Fliedl, G. (eds.) UNISCON 2008. LNBIP, vol. 5, pp. 136–147. Springer, Berlin (2008) 50. Hesse, W.: From conceptual models to ontologies – a software engineering approach, paper presented at Dagstuhl Seminar on Conceptual Modelling, April 27-30 (2008), preprint on conference website: http://drops.dagstuhl.de/opus/volltexte/2008/1598 51. Gruber, T.R.: A translation approach to portable ontology specifications. Knowledge Acquisition 5(2), 199–220 (1993) 52. Henderson-Sellers, B.: Bridging metamodels and ontologies in software engineering (2010) (submitted for publication) 53. Atkinson, C., Gutheil, M., Kiko, K.: On the relationship of ontologies and models, in Meta-Modelling and Ontologies. In: Proceedings of the 2nd Workshop on MetaModelling, WoMM 2006. LNI, vol. P-96, pp. 47–60. Gesellschaft für Informatik, Bonn (2006) 54. Devedzic, V.: Understanding ontological engineering. Comms. ACM 45(4), 136–144 (2002) 55. Ruiz, F., Hilera, J.R.: Using ontologies in software engineering and technology. In: Calero, C., Ruiz, F., Piattini, M. (eds.) Ontologies for Software Engineering and Software Technology, pp. 49–102. Springer, Berlin (2006) 56. Neumayr, B., Schrefl, M.: Comparison criteria for ontological multi-level modelling, paper presented at Dagstuhl Seminar on Conceptual Modelling, April 27-30 (2008) (preprint on conference website) 57. Aßmann, U., Zschaler, S., Wagner, G.: Ontologies, meta-models, and the model-driven paradigm. In: Calero, C., Ruiz, F., Piattini, M. (eds.) Ontologies for Software Engineering and Software Technology, pp. 239–273. Springer, Berlin (2006) 58. Falbo, R.A., Ruy, F.B., Moro, R.D.: Using ontologies to add semantics to a software engineering environment. In: Procs. SEKE 2005, Skokie, IL, USA (2005) 59. Unhelkar, B.: Verification and Validation for Quality of UML 2.0 Models. J. Wiley and Sons, Chichester (2005)
Random Thoughts on Multi-level Conceptual Modelling
113
60. ISO/IEC: Unified Modeling Language (UML) Version 1.4.2. ISO/IEC 19501. International Organization for Standardization/International Electrotechnical Commission, Geneva (2005) 61. Barbier, F., Henderson-Sellers, B., Le Parc-Lacayrelle, A., Bruel, J.-M.: Formal-ization of the whole-part relationship in the Unified Modeling Language. IEEE Trans. Software Eng. 29(5), 459–470 (2003) 62. Barbier, F., Aretxandieta, X.: State-based composition in UML 2. Int. J. Software Eng. and Knowledge Eng. 18(7), 1–25 (2008) 63. Henderson-Sellers, B., Gonzalez-Perez, C.: Uses and abuses of the stereotype mechanism in UML 1.x and 2.0. In: Wang, J., Whittle, J., Harel, D., Reggio, G. (eds.) MoDELS 2006. LNCS, vol. 4199, pp. 16–26. Springer, Heidelberg (2006) 64. Atkinson, C., Kühne, T., Henderson-Sellers, B.: Systematic stereotype usage. Software and System Modelling 2(3), 153–163 (2003) 65. Fuentes-Fernandez, R., Gomez-Sanz, J.J., Pavon, J.: Integration in agent-oriented development. Int. J. Agent-Oriented Software Eng. 1(1), 2–27 (2007) 66. McGregor, J.D., Korson, T.: Supporting dimensions of classification in object-oriented design. J. Obj.-Oriented Programming 5(9), 25–30 (1993) 67. Henderson-Sellers, B., Edwards, J.M.: BOOKTWO of Object-Oriented Knowledge: The Working Object, p. 594. Prentice-Hall, Englewood Cliffs (1994) 68. Opdahl, A.L., Henderson-Sellers, B.: Ontological evaluation of the UML using the Bunge-Wand-Weber model. Software Syst. Model. 1(1), 43–67 (2002) 69. Moody, D., van Hillegersberg, J.: Evaluating the visual syntax of UML: An analysis of the cognitive effectiveness of the UML family of diagrams. In: Gašević, D., Lämmel, R., Van Wyk, E. (eds.) SLE 2008. LNCS, vol. 5452, pp. 16–34. Springer, Heidelberg (2009) 70. Henderson-Sellers, B.: Object-Oriented Metrics. In: Measures of Complexity, p. 234. Prentice Hall, NJ (1996) 71. Fenton, N.: Software measurement: a necessary scientific basis. IEEE Trans. Soft-ware Eng. 20, 199–206 (1994) 72. Basili, V.R., Rombach, H.D.: The TAME project: towards improvement-orientated software environments. IEEE Trans. Software Eng. 14(6), 758–773 (1988) 73. Nugroho, A., Chaudron, M.R.V.: Evaluating the impact of UML modeling on software quality: An industrial case study. In: Schürr, A., Selic, B. (eds.) MODELS 2009. LNCS, vol. 5795, pp. 181–195. Springer, Heidelberg (2009) 74. Stevens, W.P., Myers, G.J., Constantine, L.L.: Structured design. IBM Syst. J. 13(2), 115–139 (1974) 75. Henry, S., Kafura, D.: Software structure metrics based on information flow. IEEE Trans. Software Eng. 7(5), 510–518 (1981) 76. Chidamber, S., Kemerer, C.: A metrics suite for object-oriented design. IEEE Trans. Software Eng. 20(6), 476–493 (1994) 77. Yap, L.-M., Henderson-Sellers, B.: A semantic model for inheritance in object-oriented systems. In: Procs. ASWEC 1993, pp. 28–35. IREE, Sydney (1993) 78. Martin, M., de los, A., Olsina, L.: Towards an ontology for software metrics and indicators as the foundation for a cataloguing web system. In: Procs. First Conf. Latin American Web Congress, pp. 103–113. IEEE Computer Society, Los Alamitos (2003) 79. Garcia, F., Bertoa, M.F., Calero, C., Vallecillo, A., Ruiz, F., Piattini, M., Genero, M.: Towards a consistent terminology for software measurement. Inf. Software Technol. 48, 631–644 (2006) 80. Genero, M., Piattini, M., Calero, C. (eds.): Metrics for Software Conceptual Models. Imperial College Press, London (2005)
114
B. Henderson-Sellers
81. Lanza, M., Marinescu, R.: Object-Oriented Metrics in Practice, p. 205. Springer, Berlin (2006) 82. Aggarwal, K.K., Singh, Y., Kaur, A., Malhotra, R.: Software design metrics for objectoriented software. J. Obj. Technol. 6(1), 121–138 (2007) 83. Aggarwal, K.K., Singh, Y., Kaur, A., Malhotra, R.: Investigating effect of design metrics on fault proneness in object-oriented systems. J. Obj. Technol. 6(10), 127–141 (2007) 84. Franch, X.: A Method for the Definition of Metrics over i* Models. In: van Eck, P., Gordijn, J., Wieringa, R. (eds.) CAiSE 2009. LNCS, vol. 5565, pp. 201–215. Springer, Heidelberg (2009) 85. Unhelkar, B., Henderson-Sellers, B.: Applying syntax, semantics and aesthetic checks to verifying and validating the quality of UML models. In: Procs. IRMA 2005. Idea Group, Hershey (2005) 86. Du Bois, B., Lange, C.F.J., Demeyer, S., Chaudron, M.R.V.: A Qualitative Investigation of UML Modeling Conventions. In: Auletta, V. (ed.) MoDELS 2006. LNCS, vol. 4364, pp. 91–100. Springer, Heidelberg (2007) 87. Gonzalez-Perez, C., Henderson-Sellers, B.: Metamodelling for Software Engineering, p. 210. J. Wiley & Sons, Chichester (2008) 88. Bertoa, M.F., Vallecillo, A.: Quality attributes for software metamodels. In: Procs. QAOOSE 2010, Malaga, Spain, July 2 (2010) 89. ISO/IEC: Software Process Assessment - Part 1: Concepts and Vocabulary. ISO/IEC 15504-1: International Standards Organization/International Electrotechnical Commission, Geneva (2004) 90. Gonzalez-Perez, C., Henderson-Sellers, B.: Methodology enactment using a work product pool approach. J. Systems and Software 81(8), 1288–1305 (2008) 91. Zuse, H.: Software Complexity: Measures and Methods, p. 605. Walter de Gruyter, Berlin (1994) 92. Weyuker, E.: Evaluating software complexity measures. IEEE Trans. Software Eng. 14(9), 1357–1365 (1988) 93. Fenton, N.: Software Metrics: A Rigorous Approach, p. 337. Chapman and Hall, London (1991) 94. Pavon, J., Gomez-Sanz, J., Fuentes, R.: The INGENIAS methodology and tools. In: Henderson-Sellers, B., Giorgini, P. (eds.) Agent-Oriented Methodologies, pp. 236–276. IDEA Group Publishing, Hershey (2005) 95. Beydoun, G., Gonzalez-Perez, C., Henderson-Sellers, B., Low, G.: Developing and Evaluating a Generic Metamodel for MAS Work Products. In: Garcia, A., Choren, R., Lucena, C., Giorgini, P., Holvoet, T., Romanovsky, A. (eds.) SELMAS 2005. LNCS, vol. 3914, pp. 126–142. Springer, Heidelberg (2006) 96. Beydoun, G., Low, G., Henderson-Sellers, B., Mouratidis, H., Gomez-Sanz, J., Pavon, J., Gonzalez-Perez, C.: FAML: a generic metamodel for MAS development. IEEE Trans. Software Eng. 35(6), 841–863 (2009) 97. Azaiez, S., Huget, M.-P., Oquendo, F.: An approach for multi-agent metamodelling. Multiagent and Grid Systems 2(4), 435–454 (2006) 98. Odell, J.J., Van Dyke Parunak, H., Fleischer, M.: Modeling agent organizations using roles. Software and Systems Modeling 2, 76–81 (2003) 99. Kumar, K., Welke, R.J.: Methodology engineering: a proposal for situation-specific methodology construction. In: Cotterman, W.W., Senn, J.A. (eds.) Challenges and Strategies for Research in Systems Development, pp. 257–269. John Wiley & Sons, Chichester (1992)
Random Thoughts on Multi-level Conceptual Modelling
115
100. Brinkkemper, S.: Method engineering: engineering of information systems devel-opment methods and tools. Information and Software Technology 38(4), 275–280 (1996) 101. Henderson-Sellers, B.: Method engineering for OO system development. Communications of the ACM 46(10), 73–78 (2003) 102. Henderson-Sellers, B., Ralyte, J.: Approaches to situational method engineering. Journal of Universal Computer Science 16(3), 424–478 (2010) 103. Ralyté, J., Brinkkemper, S., Henderson-Sellers, B. (eds.): Situational Method Engineering: Fundamentals and Experiences. Proceedings of the IFIP WG 8.1 Working Conference, Geneva, Switzerland, September 12-14. IFIP Series, vol. 244, p. 380. Springer, Berlin (2007) 104. Seidita, V., Ralyté, J., Henderson-Sellers, B., Cossentino, M., Arni-Bloch, N.: A comparison of deontic matrices, maps and activity diagrams for the construction of situational methods. In: Eder, J., Tomassen, S.L., Opdahl, A.L., Sindre, G. (eds.) CAiSE 2007 Forum Proceedings, pp. 85–88 (2007) 105. ISO/IEC: CDIF Framework. ISO/IEC 15474, International Organization for Standardization, Geneva (1998) 106. OMG: BPMN Fundamentals, OMG document bei/04-11-05.pdf (2005) 107. Bock, C.: Introduction to the Business Process Definition Metamodel, OMG doc. BPDM_08-06-32.pdf (2008) 108. Bandara, W., Indulska, M., Chong, S., Sadiq, S.: Major issues in business proc-ess management: an expert perspective, BP Trends, pp. 1–8 (October 2007) 109. Indulska, M., Recker, J., Rosemann, M., Green, P.: Business process modeling: Current issues and future challenges. In: van Eck, P., Gordijn, J., Wieringa, R. (eds.) CAiSE 2009. LNCS, vol. 5565, pp. 501–514. Springer, Heidelberg (2009) 110. Graham, I.: Requirements Modelling and Specification for Service Oriented Architecture, p. 301. J. Wiley and Sons, Chichester (2008) 111. Deneckère, R., Iacovelli, A., Kornyshova, E., Souveyet, C.: From method frag-ments to method services. In: Halpin, T., Proper, E., Krogstie, J., Franch, X., Hunt, E., Coletta, R. (eds.) Proceedings of the 13th International Workshop on Exploring Modeling Methods for Systems Analysis and Design (EMMSAD 208) held in conjunction with the CAiSE 2008 Conference, Montpellier, France, June 16-17. CEUR Workshop Proceedings, vol. 337, pp. 80–96 (2008), http://www.ceur-ws.org 112. Saidani, O., Nurcan, S.: Meta-model tailoring for situation-aware business process modelling. In: Procs MoDISE-EUS 2008, CEUR Proceedings, vol. 341 (2008), http://ceur-ws.org 113. Henderson-Sellers, B., Qureshi, M.A., Gonzalez-Perez, C.: Towards an interop-erable metamodel suite: complexity assessment. In: Parsons, J., et al. (eds.) ER 2010. LNCS, vol. 6412, pp. 275–288. Springer, Heidelberg (2010) 114. Gašević, D., Kaviani, N., Hatala, M.: On metamodeling in megamodels. In: Engels, G., Opdyke, B., Schmidt, D.C., Weil, F. (eds.) MODELS 2007. LNCS, vol. 4735, pp. 91– 105. Springer, Heidelberg (2007) 115. Hug, C., Front, A., Rieu, D., Henderson-Sellers, B.: A method to build information systems engineering process metamodels. J. Systems Software 82(10), 1730–1742 116. Sen, S., Moha, N., Baudry, B., Jézéquel, J.-M.: Meta-model pruning. In: Schürr, A., Selic, B. (eds.) MODELS 2009. LNCS, vol. 5795, pp. 32–46. Springer, Heidelberg (2009) 117. Unhelkar, B., Henderson-Sellers, B.: Modelling spaces and the UML, Innova-tions Through Information Technology. In: Khosrow-Pour, M. (ed.) Procs. IRMA 2004, pp. 892–895. Idea Group Publishing, Hershey (2004)
116
B. Henderson-Sellers
118. Du Bois, B., Lange, C.F.J., Demeyer, S., Chaudron, M.R.V.: A Qualitative Investigation of UML Modeling Conventions. In: Auletta, V. (ed.) MoDELS 2006. LNCS, vol. 4364, pp. 91–100. Springer, Heidelberg (2007) 119. Shekhovtsov, V.A.: On conceptualization of quality, paper presented at Dagstuhl Seminar on Conceptual Modelling, April 27-30 (2008) (preprint on conference website) 120. Moody, D.L.: Theoretical and practical issues in evaluating the quality of conceptual models: current state and future directions. Data & Knowledge Eng. 55, 243–276 (2005) 121. Dupuy-Chessa, S.: Quality in ubiquitous information system design. In: Procs. Third Int. Conf. on Research Challenges in Information Science (RCIS 2009), pp. 343–352. IEEE Computer Society Press, Los Alamitos (2009) 122. Thalheim, B.: Towards a Theory of Conceptual Modelling. In: Heuser, C.A., Pernul, G. (eds.) ER 2009 Workshops. LNCS, vol. 5833, pp. 45–54. Springer, Heidelberg (2009) 123. Jackson, M.: Some Notes on Models and Modelling. In: Borgida, A.T., Chaudhri, V.K., Giorgini, P., Yu, E.S. (eds.) Conceptual Modeling: Foundations and Applications. LNCS, vol. 5600, pp. 68–81. Springer, Heidelberg (2009) 124. Favre, J.-M.: Towards a basic theory to model model-driven engineering. In: Procs. WISME 2004, a workshop at UML 2004, Lisbon, Portugal (2004), http://megaplanet.org/jean-marie-favre/papers/ TowardsABasicTheoryToModelModelDrivenEngineering.pdf (accessed May 13, 2010) 125. Hoppenbrouwers, S.J.B.A., Proper, H.A(E.), van der Weide, T.P.: A Fundamental View on the Process of Conceptual Modeling. In: Delcambre, L.M.L., Kop, C., Mayr, H.C., Mylopoulos, J., Pastor, Ó. (eds.) ER 2005. LNCS, vol. 3716, pp. 128–143. Springer, Heidelberg (2005) 126. Siau, K., Tan, X.: Improving the quality of conceptual modeling using cognitive mapping techniques. Data & Knowledge Eng. 55, 343–365 (2005)
On the Evolution of Quality Conceptualization Techniques Vladimir A. Shekhovtsov Department of Computer-Aided Management Systems, National Technical University “Kharkiv Polytechnic Institute”, Ukraine [email protected]
Abstract. We investigate the notion of software product quality from the point of view of its integration into the modeling activities on the same level of abstraction as traditional functional models (a conceptualization of quality). We pay special attention to the evolution of the approaches for obtaining this conceptualization through the history of conceptual modeling, propose their classification according to common attributes and outline their distinguishing features. Based on the proposed classification, we outline a way of establishing an evaluation framework for quality conceptualizations aiming at supporting the choice of a conceptualization solution best suited for the problem at hand. Keywords: software product quality, conceptual modeling, quality conceptualization, quality model, quality ontology, quality evaluation.
1 Introduction Conceptual modeling is often viewed as an activity related to capturing the knowledge about the desired system functionality (as quoted from [89], “the conceptual schema of an information system is the specification of its functional requirements.”) This view, however, can lead to the problem of quality-unaware conceptual modeling: − looking at the problem domain to obtain its conceptual model only from the point of view of the functionality of the system-to-be restricts the analyst and can be the source of mistakes due to the fact that quality has to be introduced into the existing system model later on the development lifecycle; − these mistakes are often difficult to find until the later stages of the software process, because many quality-related issues become only evident when the system is put into use; on the other hand, being parts of early decisions they can be difficult and costly to fix [25, 49, 56]. To overcome the above problem, it seems natural to represent the quality concepts in a way compatible with conceptual modeling notions (define a conceptualization of quality) and make this representation connected to the conceptual schema of the desired system’s functionality (i.e. reflecting the software quality in conceptual models). Researchers proposed different techniques to conceptualize the product quality, in particular, quality modeling [20, 28, 32] and metamodeling [17, 18, 38], quality ontological engineering [13, 31, 57] etc. Kaschek/Delcambre (Eds.): The Evolution of Conceptual Modeling, LNCS 6520, pp. 117–136, 2011. © Springer-Verlag Berlin Heidelberg 2011
118
V.A. Shekhovtsov
While this abundance of methods allows the analyst to enjoy an excellent flexibility when choosing a conceptualization technique, it also has its downside, because this choice can be confusing. The problem is that the current research literature lacks a unified classification of the quality conceptualization techniques (in particular, we are not aware of any comparative treatment of quality modeling and quality ontology engineering). As a result, it is difficult for analyst to decide which quality conceptualization solution is better suited for the problem at hand. It is unfortunate to observe this situation in contrast to extensive attention paid to e.g. the classification and unification of the approaches to the problem of handling the quality of conceptual models [75, 84]. In this paper, we aim at overcoming this problem by elaborating a detailed classification of the existing quality conceptualization techniques together with a view of their evolution along the proposed classification dimensions. This classification can serve as a foundation for the evaluation framework for quality conceptualizations as these dimensions could underlie the quality criteria for evaluation. Section 2 continues by introducing the concept of quality to be used in this paper. In Section 3, we present a classification of quality conceptualization techniques based on the dimensions of their evolution. In Section 4, we outline the way of establishing the evaluation framework based on this classification. Section 5 describes the related work; in Section 6, we make conclusions, and outline future research directions.
2 A Concept of Quality Before elaborating the classification of quality conceptualization techniques (QCT), it is necessary to define the concept of quality to be used in this paper as we plan to build this classification, in part, upon the properties of this concept. We need to find a common ground here, as different researchers often rely on definitions of quality referring to quite different things [67]: adherence to requirements, fitness for a particular purpose, user satisfaction etc. We believe that it can be done based on the recent efforts of establishing the notion of software quality and its usage in terms of the formal ontology [78]. In this paper, we rely on this body of work, represented by Descriptive Ontology for Linguistic and Cognitive Engineering (DOLCE) [79] and, more specifically, by Core Ontology for Requirements Engineering (CORE) [57]. Quality Concept in DOLCE and CORE. We follow the approach of the authors of CORE in adopting the definition of quality from DOLCE to the case of software product quality. In doing so we agree that Quality (as a whole) is a concept embracing the set of particular qualities [79, pp. 16-18] i.e. perceivable and measurable entities that characterize particular individuals (such as functional units of the system-to-be) and are related to other qualities (reflecting e.g. a hierarchical order or quality interdependencies). Every quality is accompanied with a conceptual space [46] which is a collection of quality dimensions defining how the individuals are positioned (measured) according to this quality; it is called quality space in DOLCE. Qualities (such as the response time for the particular operation) are distinguished from their values (such as 1 sec); these values are called quales and describe positions of individuals in a particular quality space.
On the Evolution of Quality Conceptualization Techniques
119
To clarify the properties of qualities as perceivable entities we turn to CORE that categorizes ways of perceiving quality by stakeholders as a part of communicated information. This categorization is based on the type of the speech act used by stakeholders [57, p. 181]. Directive acts (ordering something to be done) define goals: − precise (usually quantitative) quality constraints if the accompanying quality space is shared between all stakeholders so everybody agree how to measure this quality e.g. “the response time of X must be below 0,5 sec”; − vague qualitative softgoals in a NFR framework sense [26] if there is no such agreement so the perception of this quality is stakeholder-specific e.g. “the performance of X must be good”; Other kinds of speech acts define, in particular, (soft or hard) domain assumptions (descriptions of something existing in a domain e.g. “the average response time of X is 0,4 sec”) and evaluations (comparative statements about something in a domain e.g. “the response time of X in case of Y is 30% better than in case of Z”). Our Definition of Quality. For now, we can distinguish the following properties of the particular quality qc in a DOLCE sense (i.e. the quality characteristic in a sense of ISO/IEC 25010 standard [55]): − − − −
measurability (availability of the quality space for qc); characteristics of the quality space of qc (such as its structure or degree of sharing); characteristics of the relationships between qc and other qualities; characteristics of the relationships between qc and the functional units of the system-to-be characterized by this quality; − characteristics of the relationships between qc and stakeholders referring to this quality in their description of the system-to-be. To formalize this view in context of all available qualities, we describe the quality of the software system as
Q = QC , QR, QU , QF
(1)
where − QC is a set of qualities (quality characteristics) in a sense of DOLCE where every qc ∈ QC is accompanied by a shared or private quality space with particular characteristics; − QR is a set of relationships among particular qualities (such as those defining a hierarchy of qualities or interdependencies between particular qualities such as reliability and performance): qr (QC ′ ⊆ QC ) ∈ QR , where qr is a particular relationship of this kind, QC’ is a set of qualities involved into this relationship; − QU is a set of relationships among qualities and stakeholders of the system-to-be such as those defining stakeholder speech acts involving the particular quality (directive acts to define goals etc) and connecting private quality spaces to stakeholders: qu (QC ′ ⊆ QC ,U ′ ⊆ U ) ∈ QU where qu is a particular relationship of this kind, QC’ is a set of involved qualities, U’ is a set of involved stakeholders, U is a set of all available stakeholders;
120
V.A. Shekhovtsov
− QF is a set of relationships among qualities and functional entities defined by a particular architecture of the system-to-be characterized by these qualities: qf (QC ′ ⊆ QC , F ′ ⊆ F ) ∈ QF where qf is a particular relationship of this kind, QC’ is a set of involved qualities, F’ is a set of involved functional entities, F is a set of all available functional entities. In the following section, we will use this definition to establish a classification framework for quality conceptualization techniques.
3 Quality Conceptualization Techniques: Dimensions of Evolution We start from the definition of a conceptualization of quality to be used throughout this paper: a conceptualization of quality is a representation of the system quality on the same level of abstraction as the conceptual model of the functionality of this system. Currently such conceptualizations are obtained using various quality conceptualization techniques. The purpose of this section is to establish a classification of these techniques and to look at their evolution through the history of conceptual modeling. We find a set of dimensions or “axes” for the treatment of quality to evolve along and use these dimensions as a foundation for the proposed classification. Categories for QCT Evolution Dimensions. Based on the definition of quality presented in Section 2, we distinguish five categories of QCT evolution dimensions (covering 14 dimensions in total).
First category reflects the properties of the conceptualization process itself: 1. Conceptualization process dimensions (2 in total) reflect the properties of the process of performing QCT (such as QCT ability to support model-based or ontological view on quality, and to represent quality on different levels of abstraction). Next three categories group dimensions related to a QCT’s ability to support software quality defined according to (1): 2. Quality properties support dimensions (3 in total) reflect a QCT’s ability to support different properties of the QC set, including QR relationships (such as the way of organizing the set of qualities, the way of organizing quality spaces, the support for dependencies between qualities etc); 3. Quality perception support dimensions (2 in total) reflect a QCT’s ability to support the properties of the QU set describing quality as a perceivable entity (such as the ability to reflect shared and separate quality spaces, the support for quality constraints and softgoals etc); 4. Quality usage support dimensions (3 in total) reflect a QCT’s ability to support the properties of the QF set to model such software process artifacts as quality requirements or attributes (such as the support of qf relationships themselves, the availability of particular techniques for their description etc.).
On the Evolution of Quality Conceptualization Techniques
121
Last category covers dimensions related to QCT applicability: 5. Applicability dimensions (4 in total) reflect a QCT’s applicability for the particular classes of problems (such as the stage of a software lifecycle it can be used for, if it is applicable for the particular application domain or software category etc); Next, we will look in detail at the particular dimensions belonging to these categories. 3.1 Conceptualization Process Dimensions
These dimensions reflect the capabilities of the QCT process. We start from two general dimensions reflecting model-ontology dichotomy and the supported abstraction level. 3.1.1 Conceptualization Space Purpose: to reflect the problem-solution space dichotomy; following [7, 90] we define its two possible scale values (Table 1): Table 1. Conceptualization space scale Value quality modeling technique (QMT) quality ontology engineering technique (QOT)
Meaning Means of expressing software quality in a solution space (prescriptively defining the system-to-be) under “closed-world” assumption (everything not explicitly described is assumed non-existent) Means of expressing software quality in a problem space (describing the real-world problem domain addressed by the system under development) under “open-world” assumption (everything not explicitly described is assumed unknown)
Examples [15, 37, 55, 58], reviews are in [20, 32]
[31, 40, 59], no reviews except [78] covering formal techniques not restricted to software quality
Notes on evolution. QMT are considerably older than QOT: first quality models were introduced in mid-70s [15, 80], whereas the earliest works describing ontological representation of quality appeared in mid-90s. To our knowledge, the term “quality ontology” was first used in a paper [66] describing an ontology aimed at “a logical formalization of quality knowledge”; it showed the quality as a set of concepts supporting predicting and assessing the quality for the software system. Later, [14] proposed the domain taxonomy (defined by means of ontological approach) aimed at helping to describe the conflicts in requirements; it also covered the quality concept. For many years, QMT significantly outnumbered QOT, but recently the evolution trend started to reflect the growing need for deeper ontological foundations for quality conceptualization e.g. based on cognitive theory [46, 79, 87]. 3.1.2 Abstraction Level Purpose: to reflect that quality conceptualization solutions can be described on different levels of abstraction [112]; we consider abstraction levels for QMT and QOT separately as they treat such levels differently (despite some similarities).
Abstraction levels for quality models. We start from describing the abstraction level dimension of the QMT evolution according to the notion of a modeling meta-pyramid [7] where the solution belonging to a particular level defines the model for a language used
122
V.A. Shekhovtsov
to define the solutions on the level below (i.e. a metamodel). We restrict the pyramid to three levels (the scale values in Table 2 correspond to these levels): − M1 level – for the models of some phenomenon; in our case a quality model directly describes the phenomenon of quality as used in a system-to-be; − M2 level – for the metamodels (models of languages used for describing the M1level models); in our case a quality metamodel is used to define quality models; − M3 level – for the meta-metamodels (models of languages used for describing the metamodels); here a meta-metamodel can be used to define quality metamodels. Table 2. Abstraction level scale (QMT) Value concrete
metadescriptive
meta-metadescriptive
Meaning Examples QMT solution belongs to M1 level; e.g. it is a taxon- early QMT [15, 80] omy of quality characteristics with more general standards [54, 55], ones on the top level and more concrete ones on the etc. levels below, often not changeable (fixed model QMT [41]) and described by example without metainformation. Criticized due to the arbitrary choice of quality characteristics and the lack of connection to measurable quality attributes at the bottom level [32, 67] QMT solution belongs to the M2 (meta-) level (a metamodel) implicit quality metamodel: a metamodel not docu- [37, 53] menting this fact (introduced “bottom-up” e.g. via custom or mixed model QMT [41] allowing modification of the model structure and often omitting quality characteristics from the model descriptions) explicit quality metamodel: introduced “top-down” generic metamodels: with an explicit purpose of describing the set of pos- [17, 18, 58, 106] sible quality models; industry examples are UML UML profiles: quality profiles adding support for new modeling [1, 6, 95, 109] concepts to UML via an extension of the UML metamodel QMT belongs to the M3 (meta-meta-) level; no solu- generic: MOF-based tions on this level explicitly addressing the represen- [17, 18], formal tation of quality metamodels notions [58]
Abstraction levels for ontologies. For QOT, this dimension reflects the level of abstraction of the concepts described with a quality ontology. It differs from the QMT abstraction level as it refers to the concepts being described – not the description approach (all ontologies of different levels can be defined using the same ontology language). It has the same values as for QMT but with different semantics (Table 3). Notes on evolution. An evolution of QCT along this dimension is not linear. For QMT, it mostly follows the path up the meta-pyramid from M1 to M2 levels [32, 57, p. 232], but particular M1-level solutions appear if no reuse of models is necessary. For QOT, the first known quality ontology [14] was of concrete kind, upper-level quality ontologies have been introduced later via generalization and are used more frequently now, though some concrete and mixed ontologies are available as well.
On the Evolution of Quality Conceptualization Techniques
123
Table 3. Abstraction level scale (QOT) Value concrete
metadescriptive
meta-metadescriptive
Meaning QOT defines quality ontology directly describing realworld phenomenon of quality (via particulars; with concepts for performance, reliability etc) QOT defines quality upper ontology describing the concepts (universals) for concrete quality ontologies (e.g. by defining concepts for quality characteristic, quality metric, describing their relationships etc.) QOT reflects the ontology language; we follow [7] in that it such language is a modeling language described on M3 level.
Examples [3, 14]
[13, 40, 57, 76]
OWL [76], formal notations [57, 79]
3.2 Quality Properties Support Dimensions
These dimensions correspond to the degree of completeness of reflecting properties of the QC set of qualities (1) and the QR relationships among the elements of this set. We distinguish dimensions reflecting QC structure, measurability of its elements, and dependencies inside QC (omitting some others e.g. prioritization support). For brevity, we list only few qualitative values per dimension; later we will show that to perform QCT evaluation based on this classification, it will be necessary to quantify these values to make the dimensions measurable. 3.2.1 Structural Complexity Purpose: to reflect the organization of QC as defined by internal relationships qr ∈ QR (Table 4). Table 4. Structural complexity scale Value single quality set taxonomy graph
Meaning QCT describes a single quality characteristic; often such techniques conceptualize the domain related to this characteristic QCT does not describe any structure for QC
QCT describes a hierarchy of qc ∈ QC with more general characteristics at higher level QCT deviates from pure taxonomy in QC structure e.g. by taking into account overlapping characteristics
Examples security [39, 106] etc. reliability [50, 116] etc. performance [74, 109] [57, 66] Early QCT [15, 80], standards [53, 54] etc. [16, 37]
Notes on evolution. The treatment of this issue has evolved into taking into account overlapping quality characteristics. This evolution has been driven by the increasing level of understanding of the real-world notion of quality, which goes beyond seeing it as a simple taxonomy. Older pure-taxonomy solutions are still widespread especially as they are parts of a standard e.g., ISO/IEC 9126.
124
V.A. Shekhovtsov
3.2.2 Quality Measurability Purpose: to reflect the degree of supplementing qc ∈ QC with quality metrics i.e. the availability of the shared quality spaces [57] connected to these characteristics (Table 5). Table 5. Quality measurability scale Value none basic
complete
Meaning Purely qualitative QCT; they are usually resulted from establishing top-level quality characteristics first QCT includes only generic support for quality metrics (e.g. by establish the “metric” concept but not elaborating the concepts describing the way of calculating these metrics); usually this is true for the solutions that describe the quality but are not intended for its evaluation or prediction QCT includes more extensive support for quality metrics (e.g. conceptualizing the way of calculating their values)
Examples Early QCT [15, 80] etc. [18, 40, 99]
[13, 81, 107]
Notes on evolution. Initially, QCTs were qualitative in nature; the evolution of their treatment resulted in supplementing the conceptualizations with extensive sets of quantitative metrics. Quality model standards [54, 55] also include metrics. Currently, there are some arguments in favor of the prohibition of using purely qualitative models at least for some problem areas, see the discussion in [32, 42, 56]. 3.2.3 Quality Dependencies Support Purpose: to reflect the degree of taking into account the interdependencies among quality characteristics defined by relationships qr ∈ QR (Table 6). Table 6. Quality dependencies support scale Value none basic
complete
Meaning No support for interdependencies QCT deals with interdependencies among quality characteristics (e.g. how they influence each other) in qualitative fashion (i.e. via such values as “supports” or “neutral”) QCT includes quantitative and qualitative QR treatment
Examples [15] [13, 26, 99]
Jureta et al. [58]
Notes on evolution. The support for interdependencies inside QC is not widely available in QCT. Only recently a solution [58] appeared with their complete support. 3.3 Quality Perception Support Dimensions
These dimensions are related to the fact that different stakeholders perceive quality differently. Their values reflect the properties of the QU set (QU-relationships) (1). 3.3.1 Stakeholder Dependency Purpose: to reflect a QCT’s ability to support QU-relationships (Table 7).
On the Evolution of Quality Conceptualization Techniques
125
Table 7. Stakeholder dependency scale Value none basic
complete
Meaning No QU-relationships: single-person view on quality (also implies “none” value for all other dimensions from this category) QU-relationships are supported but without private quality spaces (conceptualizations include the notion of stakeholder, but do not allow per-stakeholder metrics) QU-relationships are accompanied by private quality spaces (allowing per-stakeholder measuring of quality)
Examples [40, 54] etc. [48, 65]
[25, 26, 57]
Notes on evolution. The earliest QMT did not support QU-relationships at all. Later evolution led to including this support [48] and soon to accompanying it with private quality spaces via introducing the concept of softgoal in the NFR framework [26]. 3.3.2 Speech Mode Support Purpose: to reflect a QCT’s ability to support different speech modes in QUrelationships; here we group values by the type of the artefact produced (Table 8). Table 8. Speech mode support scale Value partial
complete
Meaning QCT supports a subset of the available speech modes: goals: directive mode assumptions/attributes: declarative/assertive mode evaluations: expressive mode QCT supports all speech modes
Examples [25, 61, 117] etc. [62] [24, 107] etc. [57]
Notes on evolution. First supported speech mode was a directive one used to define goals (formulated as quality constraints [14, 71] or softgoals [26]). Later, e.g. after introducing evaluation-related QCT [107] (see Section 3.5.1) the support for other modes started to appear; a complete solution is now only available in CORE [57]. 3.4 Quality Usage Support Dimensions
These dimensions are related to the usage of the quality concept to model quality requirements, quality attributes, or other concepts related to the quality of the systemto-be. They reflect a QCT’s ability to support the properties of the QF set (1) i.e. the relationships between a conceptualization of quality and a conceptual model of the system functionality (QF-relationships). The challenge here lies in a fact that currently QF-relationships are mostly treated implicitly, their descriptions are obscured by the descriptions of the particular requirement- or attribute-related modeling solutions; as a result, we know of no attempts for categorizing them or investigating their evolution. 3.4.1 Usage Support Purpose: to reflect an objective of using a quality conceptualization via QFrelationships (Table 9).
126
V.A. Shekhovtsov Table 9. Usage support scale
Value none present
Meaning No support for QF-relationships (also implies “none” value for all other dimensions from this category) QCT reflects the relationships aimed at modeling the quality requirements for the system-to-be or the quality attributes for the implemented system
Examples [15, 40, 54, 80] etc. requirements [3, 25, 57,, 61, 82] etc. attributes [62, 85, 111] etc.
Notes on evolution. The earliest QMT did not support QF-relationships; they just described the quality phenomenon and were not concerned about its usage. Later, the general trend was to first include and then extend this support in both QMT and QOT. 3.4.2 Usage Explicitness Purpose: to reflect a degree of explicitness in describing the usage of a conceptualization via qf ∈ QF (Table 10). Table 10. Usage explicitness scale Value explicit implicit
Meaning QCT reflects the relationships defined explicitly e.g. as associations between quality and functionally-related concepts QCT reflects the relationships defined implicitly e.g. by specifying the criteria for participation
Examples [3, 13, 42, ch. 18, 61] etc. [29, 68, 83]
Implicit solutions employ aspect-oriented techniques [98] by defining criteria (pointcuts in aspect-oriented terminology) describing sets of functionality-related concepts (e.g. classes or methods) to be involved into relationships with particular qualityrelated concepts. In this case it is necessary to express such criteria at an appropriate level of abstraction [38, 104] e.g. using semantic notions [22, 102]. Notes on evolution. Initially, QF-relationships were of explicit kind, recently the techniques for expressing them implicitly started to appear, but the former are still in a wider use and there are doubts [61] if the introduced complexity is justifiable. 3.4.3 Usage Target Purpose: to reflect the properties of the system-to-be functional units used in QFrelationships (Table 11). Table 11. Usage target scale Value static dynamic all
Meaning QF-relationships depend on a static decomposition of the connected model (e.g. by targeting classes) QF-relationships depend on a behavior of the system-to-be reflected in its model (e.g. by targeting activities, events or states) QF-relationships target both static elements and behavior
Examples [42, ch. 18, 117] [23, 33, 95] [1, 83]
Notes on evolution. Currently we observe the move from static to dynamic relationships as more knowledge about the relations between quality and behaviour is gained.
On the Evolution of Quality Conceptualization Techniques
127
3.5 Applicability Dimensions
These dimensions are related to the external view on the quality conceptualization by describing a QCT’s applicability for the particular classes of problems. 3.5.1 Application Goal Purpose: to reflect the goal of applying the solution produced by this QCT, we follow [32, 112] in defining its values (Table 12). Table 12. Application goal scale [32, 112], “all” value is also available Value definition assessment prediction
Meaning QCT defines the notion of quality QCT is used for quality assessment QCT is used for quality prediction
Examples [15, 54, 79] etc. review is in [107] [94, 114] etc.
Notes on evolution. Early QCT were mostly of a “definition” kind. Now, techniques of all three kinds are being actively developed in parallel (with researchers often unaware of each other’s work [32]) with definition models being the most widespread. 3.5.2 Process Stage Dependency Purpose: to reflects the stage(s) of the software process the particular QCT to be applied for [112] (Table 13). Table 13. Process stage dependency scale Value none stagespecific
integrated
Meaning QCT is generic enough to be applied at different software process stages QCT is specific for a particular process stage(s) requirements engineering architectural design software implementation QCT produces integrated quality conceptualizations to be used in MDA-like environment
Examples standards [54, 55], etc. [36, 45, 57] etc. [2, 29, 70, 108] etc. [44] [5, 68] etc.
Notes on evolution. Early QCT were mostly of a stage-independent kind; currently stage-specific solutions outnumber the former due to a wider field of application (with requirement engineering stage as the most frequent target). Very few QCT target software implementation. The trend is also toward integrated MDA-based techniques. 3.5.3 Application Category Dependency Purpose: to reflect the classification of the software solutions QCT to be applied for (Table 14). Notes on evolution: early QCT were category-independent, over time, the average degree of category dependency tends to increase alongside the amount of gained knowledge about the quality of the particular software classes.
128
V.A. Shekhovtsov Table 14. Application category dependency scale
Value categoryspecific
none
Meaning Examples QCT produces conceptualization suited for a particular software category(ies) service-oriented systems [45, 58, 65, 73, 100] etc. real-time systems [6, 43, 109] component software [4, 19, 20, 24] etc. web-based systems [18, 51, 77, 86] etc. QCT does not target any particular software standard-related QCT category (category-independent) [17, 28, 54, 55, 105], etc.
3.5.4 Domain Dependency Purpose: to reflect the degree of domain-dependency possessed by the QCT (Table 15). Notes on evolution: early QCT were domain-independent. Currently, solutions of this kind are still most widely available, domain-specific solutions are not often published in software engineering literature; recently, domain-customizable solutions started to appear instead. Table 15. Domain dependency scale Value domainspecific
Meaning QCT produces ad-hoc conceptualization specific for the particular application domain
domaincustomizable none
QCT allows to develop domain-dependent conceptualizations by customizing a generic “template” QCT produces conceptualization not specific to any particular domain (domain-independent)
Examples e-government [76] e-commerce [12, 103] education [9, 47, 92] [21, 40, 64, 101] all except above
4 Towards a QCT Evaluation Framework In this section, we discuss the steps that could be taken to turn the proposed set of classification dimensions into the set of evaluation criteria to be used for selecting the best QCT for the problem at hand. Full coverage of this issue will be the target for subsequent publications. 4.1 Evaluating Quality of Quality Conceptualizations
To make the proposed set of QCT evolution dimensions a part of a QCT evaluation/comparison framework we need to take into account the fact that our proposed dimension scale values are just qualitative “tags” established for marking the evolution milestones. The problem with using such values to organize QCT comparison and evaluation is that quality spaces [46, 57] associated with our dimensions are rudimentary (the choice of values is arbitrary, there is no order or distance defined etc). We see several ways to overcome this limitation. First of all, it is possible to drop quality spaces altogether by establishing so-called “descriptive mode” comparison framework such as the one proposed by Babar et al. for software architecture evaluation methods [8]. Such framework defines only dimensions; for every QCT under evaluation, it is necessary to elaborate narrative descriptions of its positions related to
On the Evolution of Quality Conceptualization Techniques
129
these dimensions. For particular QCT, such descriptions can go into great detail (see the description of ATAM according to the Babar et al. framework [63]). In addition, for the specific classes of projects, it is possible to describe the most suitable QCT position w.r.t. every dimension. This way, one would document a set of useful (though informal) guidelines for selecting QCT suitable to the problem at hand. Other approaches involve quantifying the dimensions turning them into quality evaluation dimensions to establish QCT quality model of the assessment type (“quality of quality” model - QM2) and applying this model to the problem at hand. Several dimension quantification techniques can be used to aid in solving this problem. 1. Following bottom-up quality modeling approach [37] by defining measurable QCT quality-bearing properties and connecting our dimensions to these properties. To do so, we can adapt the metrics from e.g. general evaluation frameworks for conceptual models [75, 84, 91] and ontologies [30, 88]. For example, one can take the quality metamodel defined by the particular QMT and calculate the completeness of its coverage of the QU-relationships by modifying the technique from [35]. Next, we connect this property (and others) to the “stakeholder dependency” dimension by using the weighting approach of [10, 100] or by other means. 2. Quantifying the dimensions by means of multiple criteria decision making (MCDM) technique e.g. Analytic Hierarchy Process (AHP) [2, 12, 72]. This way, the positions of the particular QCT (decision alternatives) related to a particular dimension are compared using AHP pairwise comparison technique (how better is QCTA to QCTB w.r.t. stakeholder dependency?); as a result, the values for these relative positions are calculated for every dimension. This usually involves experts. 3. Applying fuzzy logic to the problem by converting dimensions into linguistic variables and establishing appropriate membership functions. After the dimensions are quantified (we obtained the figures for the positions of QCT alternatives), we can again apply a MCDM technique (e.g. AHP) to take into account the relative importance of the dimensions (quality criteria) for the problem at hand (a generic procedure of this kind is described e.g. in [60]). 4.2 Engineering Quality Conceptualizations for Situational Methods
It seems feasible to integrate the customizable (or selectable) QCTs as method components into the situational method engineering (SME) framework [52] aimed at establishing the situational conceptual modeling process [11, 110] tailored to the problem at hand. Such components could be called QCT method chunks [93], it is also possible to turn them into QCT method services according to a method-as-a-service idea [34, 96]. QCT evaluation framework defined as outlined above could be integrated into this process as a means of selecting the appropriate chunks in a way similar to described in [69]; in method-as-a-service case it could be also feasible to treat QCT qualities as QoS and employ QoS-based service selection techniques (e.g. [115]). The idea of introducing quality definitions into the software process by means of SME was first proposed by Saeki [97] who presented a SME framework aimed at embedding metrics into the software process activities to make produced development artifacts measurable. This approach automates assessing the quality of conceptual models (such as class diagrams complexity and readability) or quality characteristics
130
V.A. Shekhovtsov
of the system-to-be that could be measured at development time based on these artifacts (such as measuring modifiability based on attributes of the use case diagram). Our goal is to facilitate extending the existing methods with QCT method chunks/services aimed at dealing with quality in conceptual models such as making available pre-selected quality conceptualizations suitable for the problem at hand.
5 Related Work Despite the fact that QCT received a lot of attention from the conceptual modeling research community, we know of no attempts to review all types of these techniques (including both quality models and quality ontologies) using a unified set of classification/evolution dimensions. Available reviews of quality models are not numerous [20, 27, 32]. The closest to ours is the approach by Wagner at al [113] which proposed six classification dimensions to some degree similar to ours (purpose, view, attribute, phase, technique, and abstractness), they, however, did not explore this issue in detail as it was not the main purpose of that work, in particular, no evolution of these techniques was investigated. There are also QCT reviews limited to particular evolution dimensions. For example, numerous reviews of quality requirement conceptualizations [25, 61, 108] are, from the point of view of this paper, detailed treatments of just a subset for a speech mode dimension possessing “goal” (directive) value. Another example is the review by Tian [107] dedicated to evaluation models. Quality ontologies received even less attention. Actually, we know of no attempts to perform dedicated review of these techniques except some limited reviews in the “related work” sections of the papers dedicated to particular techniques [56, 57, 59]. We need, however, to mention a paper [78] reviewing different approaches of representing quality by means of formal ontology without explicit connections to software.
6 Conclusions and Future Work In this paper, we proposed an extended set of evolution dimensions for quality conceptualization techniques which can also serve as criteria for their classification; such classification could help with a confusion related to the abundance of QCT and lacking clear definitions of the limits of their applicability. We based these dimensions on a QCT’s ability to represent software quality defined according to its treatment in CORE [57] and DOLCE [79] ontologies, on the degree of support for stakeholder perception of quality and the connections to the functional components of the systemto-be, and on a QCT’s applicability to the particular classes of problems. We outlined the ways of establishing a QCT evaluation framework based on the proposed classification criteria. This framework could help to resolve the problem of selecting QCT best suited for the project at hand. We plan to apply our classification framework to all QCT known to us and make the results of this classification available to public; we expect that some adjustments to the set of dimensions will need to be made in the course of such application. On the other hand, we plan to elaborate the evaluation framework and QCT method chunks/services outlined in Section 4.
On the Evolution of Quality Conceptualization Techniques
131
References 1. Aagedal, J., de Miguel, M.A., Fafournoux, E., Lund, M.S., Stolen, K.: UML Profile for Modeling Quality of Service and Fault Tolerance Characteristics and Mechanisms, Technical Report TR 2004-06-01. OMG (2004) 2. Al-Naeem, T., Gorton, I., Babar, M.A., Rabhi, F.A., Benatallah, B.: A quality-driven systematic approach for architecting distributed software applications. In: Roman, G.-C., Griswold, W.G., Nuseibeh, B. (eds.) Proc. 27th International Conference on Software Engineering (ICSE 2005), pp. 244–253. ACM, New York (2006) 3. Al Balushi, T.H., Sampaio, P.R.F., Dabhi, D., Loucopoulos, P.: ElicitO: A Quality Ontology-Guided NFR Elicitation Tool. In: Sawyer, P., Heymans, P. (eds.) REFSQ 2007. LNCS, vol. 4542, pp. 306–319. Springer, Heidelberg (2007) 4. Alvaro, A., de Almeida, S.: A software component quality framework. ACM SIGSOFT Software Engineering Notes 35, 1–18 (2010) 5. Ameller, D., Gutierrez, F., Cabot, J.: Dealing with non-functional requirements in modeldriven development. Report ESSI-TR-10-05. Universitat Politècnica de Catalunya (2010) 6. Apvrille, L., Courtiat, J.-P., Lohr, C., de Saqui-Sannes, P.: TURTLE: A Real-Time UML Profile Supported by Formal Validation Toolkit. IEEE Transactions on Software Engineering 30, 473–487 (2004) 7. Aßmann, U., Zschaler, S.: Ontologies, Meta-models, and the Model-Driven Paradigm. In: Calero, C., Ruiz, F., Piattini, M. (eds.) Ontologies for Software Engineering and Software Technology, pp. 255–279. Springer, Heidelberg (2006) 8. Babar, M.A., Zhu, L., Jeffery, R.: A Framework for Classifying and Comparing Software Architecture Evaluation Methods. In: Proc. 15th Australian Software Engineering Conference (ASWEC 2004), pp. 309–318. IEEE Press, New York (2004) 9. Bajnaid, N., Cogan, B., Al-Nuaim, H.: Software quality ontology for teaching: a development methodology’s issues. In: Proc. 5th International Innovations in Information Technology (IIT 2008), pp. 352–356 (2008) 10. Bansiya, J., Davis, C.G.: A hierarchical model for object-oriented design quality assessment. IEEE Transactions on Software Engineering 28, 4–17 (2002) 11. Becker, J., Janiesch, C., Pfeiffer, D., Seidel, S.: Evolutionary method engineering: towards a method for the analysis and conception of management information systems. In: Proc. 12th Americas Conference on Information Systems (AMCIS 2006), pp. 3686–3697 (2006) 12. Behkamal, B., Kahani, M., Akbari, M.K.: Customizing ISO 9126 quality model for evaluation of B2B applications. Information and Software Technology 51, 599–609 (2009) 13. Bertoa, M., Vallecillo, A., Garc¡a, F.: An Ontology for Software Measurement. In: Calero, C., Ruiz, F., Piattini, M. (eds.) Ontologies for Software Engineering and Software Technology, pp. 175–196. Springer, Heidelberg (2006) 14. Boehm, B., In, H.: Identifying Quality-Requirements Conflicts. IEEE Software 13, 25–35 (1996) 15. Boehm, B.W., Brown, J.R., Kaspar, H., Lipow, M., MacLeod, G.J., Merritt, M.J.: Characteristics of Software Quality. North Holland, New York (1978) 16. Buglione, L., Kececi, N., Abran, A.: An Integrated Graphical Assessment for Managing Software Product Quality. In: Proc. 12th International Software Quality Conference (ICSQ 2002), Ottawa, Canada (2002) 17. Burgués, X., Franch, X., Ribó, J.M.: A MOF-Compliant Approach to Software Quality Modeling. In: Delcambre, L.M.L., Kop, C., Mayr, H.C., Mylopoulos, J., Pastor, Ó. (eds.) ER 2005. LNCS, vol. 3716, pp. 176–191. Springer, Heidelberg (2005)
132
V.A. Shekhovtsov
18. Cachero, C., Calero, C., Poels, G.: Metamodeling the Quality of the Web Development Process’ Intermediate Artifacts. In: Baresi, L., Fraternali, P., Houben, G.-J. (eds.) ICWE 2007. LNCS, vol. 4607, pp. 74–89. Springer, Heidelberg (2007) 19. Carvalho, F., Meira, S.R.L., Xavier, E., Eulino, J.: An Embedded Software Component Maturity Model. In: Choi, B. (ed.) Proc. 9th International Conference on Quality Software (QSIC 2009), pp. 426–431. IEEE Press, New York (2009) 20. Carvallo, J.P.: Systematic Construction of Quality Models for COTS-Based Systems. PhD Thesis. Universitat Politècnica de Catalunya, Barcelona (2005) 21. Carvallo, J.P., Franch, X., Quer, C.: Building and Using Quality Models for Complex Software Domains. Technical Report LSI-03-28-R. Universitat Politècnica de Catalunya, Barcelona (2007) 22. Cazzola, W., Jezequel, J.M., Rashid, A.: Semantic join point models: Motivations, notions and requirements. In: Proc. Software Engineering Properties of Languages and Aspect Technologies Workshop (SPLAT 2006) (2006) 23. Charfi, A., Müller, H., Mezini, M.: Aspect-Oriented Business Process Modeling with AO4BPMN. In: Kühne, T., Selic, B., Gervais, M.-P., Terrier, F. (eds.) ECMFA 2010. LNCS, vol. 6138, pp. 48–61. Springer, Heidelberg (2010) 24. Choi, Y., Lee, S., Song, H., Park, J., Kim, S.H.: Practical S/W Component Quality Evaluation Model. In: Proc. 10th IEEE International Conference on Advanced Communication Technology (ICACT 2008), pp. 259–264. IEEE Press, New York (2008) 25. Chung, L., do Prado Leite, J.: On Non-Functional Requirements in Software Engineering. In: Borgida, A.T., Chaudhri, V.K., Giorgini, P., Yu, E.S. (eds.) Conceptual Modeling: Foundations and Applications. LNCS, vol. 5600, pp. 363–379. Springer, Heidelberg (2009) 26. Chung, L., Nixon, B.A., Yu, E., Mylopoulos, J.: Non-Functional Requirements in Software Engineering. Kluwer Academic Publishers, Boston (1999) 27. Cote, M., Suryn, W., Georgiadou, E.: Software Quality Model Requirements for Software Quality Engineering. In: Proc. 14th International Conference on Software Quality Management (SQM 2006), pp. 31–50 (2006) 28. Cote, M.A., Suryn, W., Georgiadou, E.: In search for a widely applicable and accepted software quality model for software quality engineering. Software Quality Journal 15, 401–416 (2007) 29. Dai, L., Cooper, K.: Modeling and analysis of non-functional requirements as aspects in a UML based architecture design. In: Proc. 6th ACIS International Conference on Software Engineering, Artificial Intelligence, Networking and Parallel/Distributed Computing (SNPD 2005), pp. 178–183. IEEE Press, New York (2005) 30. Davies, I., Green, P., Milton, S., Rosemann, M.: Analyzing and comparing ontologies with meta-models. In: Krogstie, J., Halpin, T., Siau, K. (eds.) Information Modeling Methods and Methodologies, pp. 1–16. Idea Group, Hershey (2005) 31. de Boer, R.C., van Vliet, H.: QuOnt: an ontology for the reuse of quality criteria. In: Proc. ICSE Workshop on Sharing and Reusing Architectural Knowledge (SHARK 2009), pp. 57–64. IEEE Press, New York (2009) 32. Deissenboeck, F., Juergens, E., Lochmann, K., Wagner, S.: Software quality models: Purposes, usage scenarios and requirements. In: Proc. 7th International Workshop on Software Quality (WoSQ 2009), pp. 9–14. IEEE Press, New York (2009) 33. Deissenboeck, F., Wagner, S., Pizka, M., Teuchert, S., Girard, J.F.: An activity-based quality model for maintainability. In: Proc. IEEE International Conference on Software Maintenance (ICSM 2007), pp. 184–193. IEEE Press, New York (2007) 34. Deneckere, R., Iacovelli, A., Kornyshova, E., Souveyet, C.: From Method Fragments to Method Services. In: Halpin, T., Krogstie, J., Proper, E. (eds.) Proc. 13th Intl Workshop on Evaluation of Modeling Methods in Systems Analysis and Design (EMMSAD 2008). CEUR-WS, vol. 337, pp. 80–96 (2008)
On the Evolution of Quality Conceptualization Techniques
133
35. Dieste, O., Genero, M., Juristo, N., Moreno, A.M.: A Proposal of A Measure Of Completeness For Conceptual Models. In: Piattini, M., Genero, M., Calero, C. (eds.) Metrics for Software Conceptual Models, pp. 19–57. Imperial College Press, London (2005) 36. Dinkel, M., Baumgarten, U.: Modeling nonfunctional requirements: a basis for dynamic systems management. ACM Software Engineering Notes 30, 1–8 (2005) 37. Dromey, R.G.: Cornering the Chimera. IEEE Software 13, 33–43 (1996) 38. Eichberg, M., Awasthi, P., Ostermann, K.: Pointcuts as functional queries. In: Chin, W.N. (ed.) APLAS 2004. LNCS, vol. 3302, pp. 366–381. Springer, Heidelberg (2004) 39. Ekelhart, A., Fenz, S., Klemen, M., Weippl, E.: Security Ontology: Simulating Threats to Corporate Assets. In: Bagchi, A., Atluri, V. (eds.) ICISS 2006. LNCS, vol. 4332, pp. 249–259. Springer, Heidelberg (2006) 40. Falbo, R.A., Guizzardi, G., Duarte, K.C.: An ontological approach to domain engineering. In: Proc. 14th International Conference on Software Engineering and Knowledge Engineering (SEKE 2002), pp. 351–358. ACM Press, New York (2002) 41. Fenton, N., Pfleeger, S.L.: Software metrics: a rigorous and practical approach. PWS Publishing, Boston (1997) 42. Firesmith, D.G., Capell, P., Hammons, C.B., Latimer, D.W., Merendino, T.: The Method Framework for Engineering System Architectures. Auerbach Pubs, Boca Raton (2008) 43. Flake, S., Müller, W.: A UML profile for real-time constraints with the OCL. In: Li, J., Hussmann, H., Cook, S. (eds.) UML 2002. LNCS, vol. 2460, pp. 179–195. Springer, Heidelberg (2002) 44. Fuentes, L., Sanchez, P.: Towards executable aspect-oriented UML models. In: Proc. AOM Workshop at AOSD 2007, pp. 28–34. ACM, New York (2007) 45. Galster, M., Bucherer, E.: A taxonomy for identifying and specifying non-functional requirements in service-oriented development. In: Proc. IEEE Congress on Services (SERVICES 2008) - Part I, pp. 345–352. IEEE Press, New York (2008) 46. Gärdenfors, P.: Conceptual Spaces: A Geometry of Thought. MIT Press, Cambridge (2000) 47. Gasparini, I., Lichtnow, D., Pimenta, M.S., de Oliveira, J.P.M.: Quality Ontology for Recommendation in an Adaptive Educational System. In: Proc. International Conference on Intelligent Networking and Collaborative Systems (INCOS 2009), pp. 329–334. IEEE Press, New York (2009) 48. Gilb, T.: Principles of Software Engineering Management. Addison Wesley, Reading (1988) 49. Glinz, M.: On Non-Functional Requirements. In: Proc. 15th IEEE International Conference on Requirements Engineering (RE 2007), pp. 21–26. IEEE Press, New York (2007) 50. Grassi, V., Mirandola, R., Sabetta, A.: From design to analysis models: a kernel language for performance and reliability analysis of component-based systems. In: Proc. 5th International Workshop on Software and Performance (WOSP 2005), pp. 25–36. ACM Press, New York (2005) 51. Hakkarainen, S., Strasunskas, D., Hella, L., Tuxen, S.: Choosing appropriate method guidelines for web-ontology building. In: Delcambre, L.M.L., Kop, C., Mayr, H.C., Mylopoulos, J., Pastor, Ó. (eds.) ER 2005. LNCS, vol. 3716, pp. 270–287. Springer, Heidelberg (2005) 52. Henderson-Sellers, B., Ralyte, J.: Situational Method Engineering: State-of-the-Art Review. Journal of Universal Computer Science 16, 424–478 (2010) 53. IEEE 1061-1998: IEEE Standard for Software Quality Metrics Methodology. IEEE Press, New York (1998) 54. ISO/IEC 9126-1, Software Engineering – Product Quality – Part 1:Quality model. International Organization for Standardization (2001) 55. ISO/IEC FCD 25010: Systems and software engineering – System and software product Quality Requirements and Evaluation (SQuaRE) – System and software quality models. International Organization for Standardization (2010)
134
V.A. Shekhovtsov
56. Jureta, I.: Essays in Information Management: Contributions to the Modeling and Analysis of Quality in Information Systems Engineering, PhD Thesis. Department of Business Administration. University of Namur (2008) 57. Jureta, I., Mylopoulos, J., Faulkner, S.: A core ontology for requirements. Applied Ontology 4, 169–244 (2009) 58. Jureta, I.J., Herssens, C., Faulkner, S.: A Comprehensive Quality Model for ServiceOriented Systems. Software Quality Journal 17, 65–98 (2009) 59. Kabilan, V., Johannesson, P., Ruohomaa, S., Moen, P., Herrmann, A., Ehlfeldt, R.M., Weigand, H.: Introducing the Common Non-Functional Ontology. In: Gonçalves, R.J., Müller, J.P., Mertins, K., Zelm, M. (eds.) Enterprise Interoperability II, pp. 633–645. Springer, Heidelberg (2007) 60. Kaschek, R., Pavlov, R., Shekhovtsov, V., Zlatkin, S.: Characterization and tool supported selection of business process modeling methodologies. In: Abramowicz, W., Mayr, H.C. (eds.) Technologies for Business Information Systems, pp. 25–37. Springer, Heidelberg (2007) 61. Kassab, M., Ormandjieva, O., Daneva, M.: An Ontology Based approach to NonFunctional Requirements Conceptualization. In: Proc. 4th International Conference on Software Engineering Advances (SEA 2009), pp. 299–308. IEEE Press, New York (2009) 62. Kayed, A., Hirzalla, N., Samhan, A.A., Alfayoumi, M.: Towards an Ontology for Software Product Quality Attributes. In: Proc. 4th International Conference on Internet and Web Applications and Services (ICIW 2009), pp. 200–204. IEEE Press, New York (2009) 63. Kazman, R., Bass, L., Klein, M., Lattance, T., Northrop, L.: A Basis for Analyzing Software Architecture Analysis Methods. Software Quality Journal 13, 329–355 (2005) 64. Khomh, F., Gueheneuc, Y.G.: DEQUALITE: Building Design-based Software Quality Models. In: Proc. SPAQU 2008 (2008) 65. Kim, E., Lee, Y.: Quality Model for Web Services 2.0. OASIS (2005) 66. Kim, H., Fox, M.S., Gruninger, M.: An Ontology of Quality for Enterprise Modelling. In: Proc. ETICE 1995, pp. 105–116. IEEE Press, New York (1995) 67. Kitchenham, B., Pfleeger, S.L.: Software quality: the elusive target. IEEE Software 13, 12–21 (1996) 68. Koellmann, C., Kutvonen, L., Linington, P., Solberg, A.: An aspect-oriented approach to manage QoS dependability dimensions in model driven development. In: Proc. 3rd International Workshop on Model-Driven Enterprise Information Systems (MDEIS 2007), pp. 85–94. INSTICC Press (2007) 69. Kornyshova, E., Deneckere, R., Salinesi, C.: Method Chunks Selection by Multicriteria Techniques: an Extension of the Assembly-based Approach. In: Ralyte, J., Brinkkemper, S., Henderson-Sellers, B. (eds.) Situational Method Engineering: Fundamentals and Experiences, pp. 64–78. Springer, Heidelberg (2007) 70. Krechetov, I., Tekinerdogan, B., Garcia, A., Chavez, C., Kulesza, U.: Towards an Integrated Aspect-Oriented Modeling Approach for Software Architecture Design. In: Proc. AOM Workshop at AOSD 2006 (2006) 71. Krogstie, J.: Integrating the understanding of quality in requirements specification and conceptual modeling. ACM SIGSOFT Software Engineering Notes 23, 86–91 (1998) 72. Kumar, A., Grover, P.S., Kumar, R.: A quantitative evaluation of aspect-oriented software quality model (AOSQUAMO). ACM SIGSOFT Software Engineering Notes 34, 1–9 (2009) 73. Lee, J.Y., Lee, J.W., Du Wan Cheun, S.D.K.: A Quality Model for Evaluating Software-as-aService in Cloud Computing. In: Proc. SERMA 2009, pp. 261–266. IEEE, New York (2009) 74. Lera, I., Sancho, P.P., Juiz, C., Puigjaner, R., Zottl, J., Haring, G.: Performance assessment of intelligent distributed systems through software performance ontology engineering (SPOE). Software Quality Journal 15, 53–67 (2007)
On the Evolution of Quality Conceptualization Techniques
135
75. Lindland, O.I., Sindre, G., Solvberg, A.: Understanding quality in conceptual modeling. IEEE Software 11, 42–49 (1994) 76. Magoutas, B., Halaris, C., Mentzas, G.: An Ontology for the Multi-Perspective Evaluation of Quality in E-government Services. In: Wimmer, M.A., Scholl, J., Grönlund, Å. (eds.) EGOV. LNCS, vol. 4656, pp. 318–329. Springer, Heidelberg (2007) 77. Malak, G., Badri, L., Badri, M., Sahraoui, H.: Towards a multidimensional model for web-based applications quality assessment. In: Bauknecht, K., Bichler, M., Pröll, B. (eds.) EC-Web 2004. LNCS, vol. 3182, pp. 316–327. Springer, Heidelberg (2004) 78. Masolo, C., Borgo, S.: Qualities in formal ontology. In: Hitzler, P., Lutz, C., Stumme, G. (eds.) Proc. Workshop on Foundational Aspects of Ontologies (FOnt 2005), pp. 2–16 (2005) 79. Masolo, C., Borgo, S., Gangemi, A., Guarino, N., Oltramari, A.: The WonderWeb Library of Foundational Ontologies. WonderWeb Deliverable D18. Ontology Library (final). ISTC-CNR, Trento (2003) 80. McCall, J.A., Richards, P.K., Walters, G.F.: Factors in Software Quality. NTIS (1977) 81. McQuillan, J.A., Power, J.F.: Towards the re-usability of software metric definitions at the meta level. In: ECOOP PhD Workshop (2006) 82. Mead, N.R., Hough, E.D., Stehney, T.R.: Security Quality Requirements Engineering (SQUARE) Methodology. Software Engineering Institute, Stanford (2005) 83. Meier, S., Reinhard, T., Seybold, C., Glinz, M.: Aspect-Oriented Modeling with Integrated Object Models. In: Modellierung 2006. LNI, vol. 82, pp. 129–144. GI, Bonn (2006) 84. Moody, D.L.: Theoretical and practical issues in evaluating the quality of conceptual models: current state and future directions. Data & Knowledge Engineering 55, 243–276 (2005) 85. Morasca, S.: A probability-based approach for measuring external attributes of software artifacts. In: Proc. ESEM 2009, pp. 44–55. IEEE Press, New York (2009) 86. Nam, J.: Web portal quality. In: Proc. SOLI 2009, pp. 163–168. IEEE Press, New York (2009) 87. Neuhaus, F., Grenon, P., Smith, B.: A formal theory of substances, qualities, and universals. In: Varzi, A., Vieu, L. (eds.) FOIS 2004, pp. 49–59. IOS Press, Amsterdam (2004) 88. Obrst, L., Ceusters, W., Mani, I., Ray, S., Smith, B.: The Evaluation of Ontologies Toward Improved Semantic Interoperability. In: Baker, C.J.O., Cheung, K.-H. (eds.) Semantic Web: Revolutionizing Knowledge Discovery in the Life Sciences, pp. 139–158. Springer, Heidelberg (2007) 89. Olive, A.: Conceptual Modeling of Information Systems. Springer, Heidelberg (2007) 90. Pastor, O., Molina, J.C.: Model-Driven Architecture in Practice. Springer, Heidelberg (2007) 91. Piattini, M., Genero, M., Poels, G., Nelson, J.: Towards a Framework for Conceptual Modelling Quality. In: Piattini, M., Genero, M., Calero, C. (eds.) Metrics for Software Conceptual Models, pp. 1–18. Imperial College Press, London (2005) 92. Plaza, I., Igual, R., Marcuello, J.J., Sanchez, S., Arcega, F.: Proposal of a Quality Model for Educational Software. In: Proc. EAEEIE 2009, pp. 1–6 (2009) 93. Ralyte, J., Deneckere, R., Rolland, C.: Towards a generic model for situational method engineering. In: Eder, J., Missikoff, M. (eds.) CAiSE 2003. LNCS, vol. 2681, pp. 95– 110. Springer, Heidelberg (2003) 94. Riaz, M., Mendes, E., Tempero, E.: A systematic review of software maintainability prediction and metrics. In: Proc. ESEM 2009, pp. 367–377. IEEE Press, New York (2009) 95. Rodríguez, A., Fernández-Medina, E., Piattini, M.: Capturing Security Requirements in Business Processes Through a UML 2.0 Activity Diagrams Profile. In: Roddick, J., Benjamins, V.R., Si-said Cherfi, S., Chiang, R., Claramunt, C., Elmasri, R.A., Grandi, F., Han, H., Hepp, M., Lytras, M.D., Mišić, V.B., Poels, G., Song, I.-Y., Trujillo, J., Vangenot, C. (eds.) ER Workshops 2006. LNCS, vol. 4231, pp. 32–42. Springer, Heidelberg (2006)
136
V.A. Shekhovtsov
96. Rolland, C.: Method engineering: towards methods as services. In: Wang, Q., Pfahl, D., Raffo, D.M. (eds.) ICSP 2008. LNCS, vol. 5007, pp. 10–11. Springer, Heidelberg (2008) 97. Saeki, M.: Embedding metrics into information systems development methods: An application of method engineering technique. In: Eder, J., Missikoff, M. (eds.) CAiSE 2003. LNCS, vol. 2681, pp. 374–389. Springer, Heidelberg (2003) 98. Schauerhuber, A., Schwinger, W., Kapsammer, E., Retschitzegger, W., Wimmer, M.: Towards a common reference architecture for aspect-oriented modeling. In: Proc. AOM Workshop at AOSD 2006. ACM, New York (2006) 99. Shekhovtsov, V.A., Kop, C., Mayr, H.C.: Capturing the Semantics of Quality Requirements into an Intermediate Predesign Model. In: Hesse, W., Oberweis, A. (eds.): SIGSAND-EUROPE 2008 Symposium. LNI, vol. P-129, pp. 25–37. GI, Bonn (2008) 100. Shim, B., Choue, S., Kim, S., Park, S.: A Design Quality Model for Service-Oriented Architecture. In: Proc. APSEC 2008, pp. 403–410 (2008) 101. Sibisi, M., van Waveren, C.C.: A process framework for customising software quality models. In: Proc. AFRICON 2007, pp. 1–8 (2007) 102. Soeldner, G., Kapitza, R., Schober, S.: AOCI: ontology-based pointcuts. In: Proc. 8th Workshop on Aspects, Components, and Patterns for Infrastructure Software, pp. 25–30. ACM, New York (2009) 103. Stefani, A., Xenos, M.: E-commerce system quality assessment using a model based on ISO 9126 and Belief Networks. Software Quality Journal 16, 107–129 (2008) 104. Stein, D., Hanenberg, S., Unland, R.: Query Models. In: Baar, T., Strohmeier, A., Moreira, A., Mellor, S.J. (eds.) UML 2004. LNCS, vol. 3273, pp. 98–112. Springer, Heidelberg (2004) 105. Suryn, W., Abran, A., Laporte, C.: An integrated life cycle quality model for general public market software products. In: Proc. BSI 2004, pp. 5–7 (2004) 106. Susi, A., Perini, A., Mylopoulos, J.: The Tropos Metamodel and its Use. Informatica 29, 401–408 (2005) 107. Tian, J.: Quality-Evaluation Models and Measurements. IEEE Software 21, 84–91 (2004) 108. Tsadimas, A., Nikolaidou, M., Anagnostopoulos, D.: Handling non-functional requirements in Information System Architecture Design. In: Proc. SEA 2009, pp. 59–64. IEEE Press, New York (2009) 109. UML Profile for Schedulability, Performance, and Time, version 1.0. OMG (2003) 110. van de Weerd, I., Brinkkemper, S.: Meta-modeling for situational analysis and design methods. In: Handbook of Research on Modern Systems Analysis and Design Technologies and Applications, pp. 35–54. IGI Global, Hershey-New York (2009) 111. Wada, H., Suzuki, J., Oba, K.: Modeling Non-Functional Aspects in Service Oriented Architecture. In: Proc. SCC 2006, pp. 222–229. IEEE Press, New York (2006) 112. Wagner, S., Deissenboeck, F.: An integrated approach to quality modelling. In: Proc. WoSQ 2007. ACM, New York (2007) 113. Wagner, S., Lochmann, K., Winter, S., Goeb, A., Klaes, M.: Quality models in practice: A preliminary analysis. In: Proc. ESEM 2009, pp. 464–467. IEEE Press, New York (2009) 114. Wu, C., Lin, H.L.: Integrating fuzzy theory and hierarchy concepts to evaluate software quality. Software Quality Journal 16, 263–276 (2008) 115. Zeng, L., Benatallah, B., Ngu, A.H.H., Dumas, M., Kalagnanam, J., Chang, H.: QoSaware Middleware for Web Services Composition. IEEE Transactions on Software Engineering 30, 311–327 (2004) 116. Zhou, J., Niemelä, E., Evesti, A.: Ontology-Based Software Reliability Modelling. In: Proc. SSVM 2007, pp. 17–31 (2007) 117. Zhu, L., Gorton, I.: UML Profiles for Design Decisions and Non-Functional Requirements. In: Proc. ICSE Workshop on Sharing and Reusing Architectural Knowledge (SHARK 2007). IEEE Press, New York (2007)
A Web of Knowledge: A Conceptual-Modeling Perspective David W. Embley1, , Stephen W. Liddle2 , and Cui Tao1, 1
Department of Computer Science Department of Information Systems Brigham Young University, Provo, Utah 84602, U.S.A. 2
Abstract. The current web is a web of linked pages. Frustrated users search for facts by guessing which keywords or keyword phrases might lead them to pages where they can find facts. Can we make it possible for users to search directly for facts? Equivalently, can we turn the web into a web of facts (instead of a web of pages containing facts)? Ultimately, can the web be a knowledgebase—a web of knowledge—that can provide direct answers to factual questions and also provide the confidence necessary to make those answers believable? We answer these questions by showing how to superimpose a web of data over the web of pages, resulting in a web of knowledge. Our research group at Brigham Young University has been working on this challenge for more than a decade. Our solution, which is based on conceptual modeling, calls for turning raw symbols contained in web pages into knowledge and making this knowledge accessible via the web. The particulars of our solution show ways to overcome three impeding challenges: (1) automatic (or near automatic) creation of ontologies, (2) automatic (or near automatic) annotation of web pages with respect to these ontologies, and (3) simple but accurate query specification, usable without specialized training. Meeting these basic challenges can simplify knowledge-web content creation and access to the point that the vision of a web of knowledge can become a reality. Throughout, we show that conceptual modeling plays a key role in actualizing these ideas. Keywords: web of knowledge, web of data, semantic web, ontology learning, information extraction, free-form query specification.
1
Introduction
To think about turning raw data into accessible knowledge in a Web of Knowledge (WoK ), we first ask some fundamental questions: What is data? What are facts? What is knowledge? How does one reason and know? Philosophers have sought answers to these questions for millennia; and although we do not pretend to be able to contribute to Philosophy, we can use their ideas about ontology, epistemology, and logic to guide us in how to build a WoK.
Supported in part by the National Science Foundation under Grant #0414644.
Kaschek/Delcambre (Eds.): The Evolution of Conceptual Modeling, LNCS 6520, pp. 137–160, 2011. c Springer-Verlag Berlin Heidelberg 2011
138
D.W. Embley, S.W. Liddle, and C. Tao
– Ontology is the study of existence. It asks: “What exists?” In our quest to build a WoK, we must find computational solutions the question: “What concepts, relationships, and constraints exist?” We answer computationally, saying that we can declare a formal conceptual model for some domain of knowledge that captures the relevant concepts along with the relationships among these concepts and the constraints over these concepts and relationships.1 – Epistemology is the study of the nature of knowledge. It asks: “What is knowledge?” and “How is knowledge acquired?” To build a WoK, we provide computational answers to “What is digitally stored knowledge?” and “How does raw data become algorithmically accessible knowledge?” Our answer is to turn raw data into knowledge by populating conceptual models—by embedding facts in the concepts and relationships in accord with constraints. – Logic comprises principles and criteria of valid inference. It asks: “What is known?” and “What can be inferred?” In the computational context of a WoK, it can answer the question: “What are the known facts (both given and implied)?” We ground our conceptual model in a description logic—a decidable fragment of first-order logic [5]. To make this logic practical for non-logicians, we must and do add a query generator whose input consists of ordinary free-form textual expressions or ordinary fill-in-the-blank query forms. As we explain later in this paper, both query modes fundamentally depend on conceptual models. To actualize these ideas, we present a way to turn raw symbols contained in web pages (or other source documents) into computational knowledge and to make this knowledge accessible by average web users. The key research problems are: (1) How do we make ontology creation—conceptual-model creation—easy enough to be usable by typical human knowledge workers? (2) How do we make epistemology—content annotation with respect to an ontology—easy enough to require little, if any, training for human annotators? (3) How do we make logic— query specification—easy enough for web users to apply without specific training in writing queries in sophisticated query languages? Not only do these activities need to be easy enough, they also have to be good enough. Without a resolution to these problems, the barrier to WoK content creation and usage will remain too high, and the envisioned WoK will remain elusive. Our paper addresses these challenges and contributes by showing how answers can enable a WoK. We draw upon our extensive body of published work and, as a main thread, highlight the role of conceptual modeling. In Section 2 we provide an overview of our vision of how to superimpose a web of knowledge over a web of pages. In Section 3 we give the details of two types of tools: (1) tools to create and populate ontologies for a WoK and (2) tools to query populated WoK ontologies. 1
Purists argue that conceptual models are not ontologies [16,17,24]. We agree that when conceptual models play their traditional role to aid in database schema design, they typically are not ontologies. But when they are used to answer “What exists?” and thus when they formally capture the concepts, relationships, and constraints that exist in a domain, they are ontologies.
A Web of Knowledge: A Conceptual-Modeling Perspective
139
We explain why believe these tools provide the simplicity and flexibility needed to make them practical for typical knowledge workers. Additionally, we explain how these tools point to future work that can expand the ability of knowledge workers to successfully generate, populate, and query ontologies. We conclude in Section 4 by reiterating the key role that conceptual modeling plays in enabling the superposition of a web of knowledge over the current web of pages.
2
From a Web of Pages to a Web of Knowledge
We use two examples to show how to turn a web page into a page of queriable data. Figure 1 shows part of three ordinary, human-readable web pages about cars for sale. The facts in these pages are obvious: e.g., a ’93 NISSAN is for sale; it is sweet cherry red, has air conditioning, and sells for $900. Figure 2, which is about genes and molecular biology, shows a second example. Facts on this page are much less obvious to ordinary readers, but a specialist can see a myriad of facts. For example, gene cdk-4 has genetic position X:12.68 +/- 0.009 cM and genomic position X:13518824.13515774bp. Users would like to be able to query the facts on these pages directly: “Find me a red Nissan for under $5000; a 1990 or newer with less than 120K miles on it.” Or, “Tell me the genomic location of cdk-4.” We cannot, however, directly access these facts with the current structure of the web. Our approach makes these facts visible from outside the page and directly accessible to query engines (as opposed to search engines). 2.1
From Symbols to Knowledge — Ontological and Epistemological Tools
To make facts available to query engines, we first map out a guiding pathway to turn raw symbols into knowledge. Symbols are characters and character-string instances (e.g., $, mileage, red, Gene, Protein, WP:CE18608). Data builds on symbols by adding conceptual meta-tags (e.g., Price: $900, Color: red, Protein: WP:CE18608). Conceptualized data groups data tagged with conceptual identifiers into the framework of a conceptual model (e.g., an ordinary, extended ER model, although in our work the conceptual models we use are fact-oriented, like ORM [18]). We have knowledge when we populate a conceptual model with correct2 conceptualized data. To specify ontological descriptions, we need a conceptual-modeling language. We use OSM [12], which lets us classify and describe things that exist as object sets (of things) and relationships among these things as relationship sets. OSM also supports constraints of various kinds over object and relationship sets, and generalization/specialization hierarchies for is-a-related object sets. In principle, 2
Correct is interesting. How do we know whether conceptualized data is correct? Humans struggle to know; machines may never know. For the WoK we are contemplating, we rely on evidence and provenance by always linking conceptualized data back to its original source—the human-readable web page from which it was extracted.
140
D.W. Embley, S.W. Liddle, and C. Tao
Fig. 1. Sample Car Ads Web Pages
it does not matter which conceptual-modeling language we use, but for a WoK as we envision it, the language should have certain characteristics: – The conceptual-modeling language should have a direct correspondence to predicate calculus—the simpler and more direct the correspondence, the better. Fact-oriented conceptual models (e.g., OSM [12] and ORM [18]), which model both entities and attributes as objects, have a fairly direct correspondence to predicate calculus. Each object set represents a one-place predicate and each n-ary relationship set represents an n-place predicate. Populating object and relationship sets directly with values yields the facts for a firstorder interpretation (which, incidentally, is why these types of conceptual models are called “fact-oriented”). – Every constraint representable in the chosen conceptual-modeling language should be expressible as a well-formed formula of the first-order language induced by the representation of the object and relationship sets of the language. (Constraint translations for both OSM [13] and ORM [19] are examples.)
A Web of Knowledge: A Conceptual-Modeling Perspective
141
Fig. 2. Sample Molecular-Biology Web Page
– To facilitate reasoning, the conceptual-modeling language should be merely an alternate representation for a first-order language, as are fact-oriented conceptual-modeling languages whose constraints are expressible as wellformed first-order expressions. To keep reasoning both tractable and decidable, appropriate limitations are necessary.3 An appropriately limited version of OSM satisfies these requirements better (in our opinion) than the more traditional conceptual-modeling languages such as extended ER or UML and is more easily tailored to the needs of WoK development than ORM, a more widely known fact-oriented language. OSM, in general, has the power of predicate calculus, but the limited version we typically use is essentially an ALCN description logic [5]. Thus, OSM has the formal properties required (1) to ontologically represent concepts and their interrelationships, (2) to epistemologically represent knowledge as predicate-calculus 3
Adjusting the conceptual-modeling language for our WoK vision as researchers come to better understand the decidability and tractability issues of description logics may be appropriate.
142
D.W. Embley, S.W. Liddle, and C. Tao
theories in terms of formal interpretations, and (3) to logically establish inference rules and to query over base interpretations and inference rules. Further, in its limited version, OSM has reasonable processing efficiency. At a minimum, one necessary tool is an editor that allows users to construct and populate conceptual models. Building and populating ontologies by hand, however, becomes a bottleneck in the process of turning data into knowledge [7]. Thus, we seek an answer to this question: Can we automatically construct and populate an ontology for a domain of knowledge from raw source domain information? If so, how? If not fully automatically, can we at least semi-automatically construct domain knowledge and do so with most of the burden of construction shifted to the machine? Attempts to extract ontologies from natural-language text documents have been largely unsuccessful [7]. The unbounded freedom in natural-language expression makes synthesizing conceptualizations extremely difficult. Attempts to extract ontologies from semi-structured documents, such as the ones in Figures 1 and 2, although challenging, appear promising [26]. To the extent we can algorithmically discover meta-information and tie it to data values, we can synthesize conceptualizations. The degree of semi-structuredness is a rough measure of how amenable a document is to automatic conceptualization. With algorithmic conceptualization in mind, we approach the problem of automatic and semi-automatic ontology construction by building tools that use the web pages we wish to turn into knowledge as sources to help us construct the ontology. This works by recognizing that the data is formatted in a particular way (e.g., the table in the OnlineAthens ads in Figure 1) and by using reverse-engineering techniques to construct a conceptual model (e.g., to discover that P rice, Y ear, M ake, and M odel in the table in Figure 1 are concepts for a car-sales conceptual model or to discover that Genetic Position and Genomic Position in Figure 2 are alternate ways to specify locations for a gene ontology). Since we cannot fully count on these automatic ontology-building tools, we also provide a way to build ontologies that leverages the idea of an ordinary form. People generally know how to design forms with single- and multiple-entry blanks. And we know how to algorithmically turn form-like nested structures into conceptual models [1]. Ontological descriptions, however, are not enough for our envisioned web of knowledge. We also need a way to link raw facts in web pages with ontological descriptions. We need epistemological tools as well as ontological tools. A way to link actual facts with an ontology is to annotate a web page with respect to that ontology. We annotate a data value in a web page with respect to an ontology by mapping it to an object set in an OSM ontology. Likewise, we annotate related pairs of values (and, more generally, related n-tuples of values) in a web page by mapping them to a relationship set in an ontology. Although it is possible to annotate a web page with respect to an ontology by hand, this is likely to be too tedious and time consuming to be practical for most
A Web of Knowledge: A Conceptual-Modeling Perspective
143
applications.4 We therefore augment OSM ontologies with instance recognizers (such an ontology is called an extraction ontology). Instance recognizers contain regular expressions that recognize common textual items such as dates, times, prices, and telephone numbers. They can also contain lexicons that match with items such as car makes and models or protein names and functions. Much can and has been said about how to build and use these instance recognizers embedded within extraction ontologies [9,11]. Although we can use extraction ontologies to automate the annotation of web pages, building instance recognizers for OSM extraction ontologies is laborious. We therefore seek better ways to do automatic annotation without having to build many specialized recognizers by hand. We first observe that for many common data items such as ordinary numbers, dates, time, currency, and percentages, we can expend the effort to create good recognizers, and we can store them in a library and use them liberally in many extraction ontologies. Even so, however, many needed recognizers would not likely be in the library. From our examples (Figures 1 and 2), we would not likely have recognizers for makes, models, and features of cars; nor for proteins, amino acids, and gene locations. For these types of items, we seek alternative ways to build recognizers. We observe that many of these types of data items come from regular patterns in semi-structured pages. Columns in tables, like the makes and models in the table in Figure 1 or like the proteins and amino acids in the table in Figure 2, are common, as are lists of items, either in a single column or delimiter-separated like features in car ads. We should be able to classify and annotate values in semi-structured documents. Indeed, we observe that if we can extract ontological knowledge from semi-structured web pages, we should at the same time be able to also extract epistemological knowledge. For example, if we can recognize the table structures in Figure 2 well enough to derive the meta-knowledge necessary for an ontology, we should be able to link the data associated with the metaknowledge (e.g., category labels in tables) to the ontological concepts to which they belong. This maps the data to an ontology and thus annotates the data in the web page. Furthermore, as we annotate data such as car makes and models and protein names and amino acids, we can keep the values we find in lexicons and thus automatically build and continuously improve instance recognizers for extraction ontologies. 2.2
Querying Knowledge — Logic Tools
After building tools to turn raw symbols in web pages into knowledge, we next need to provide appropriate query capabilities. Given that we have data on a web page annotated with respect to an ontology, we can immediately generate subject-predicate-object triples in RDF [29], the W3C standard for representing 4
Although tedious, we do foresee hand-annotation as a viable way to create annotated content. Moreover, we can also annotate images like some commercial enterprises do (e.g., [15]), but with respect to ontologies so that they can be queried in the envisioned WoK. Likewise, sound bites and videos, and in general all multi-media objects, can be annotated with respect to ontologies.
144
red
D.W. Embley, S.W. Liddle, and C. Tao
Nissan for under $5000 ; a 1990 or newer with less than 120K
miles on it
Fig. 3. Free-Form Query
ontological data. We can then directly use the SPARQL query language [30], also a W3C standard, to write and execute queries over this RDF data. In addition, from OSM ontologies, we can also immediately generate OWL ontologies [28], another W3C standard. This further enhances our ability to reason with the data since we can write and execute inference rules over OWL ontologies (e.g., in Pellet [23], an open source OWL description-logic reasoner). If everyone could write SPARQL queries, we would basically have what we need to enable users to search the envisioned WoK. However, since we target users who are not trained in formal query specification, we should not expect these users to learn SPARQL or any other formal query language. Instead, we should provide a query system in which users can pose queries in their own terms. Figure 3 shows a free-form query for a tool we have built [2,27]. The key to making these free-form queries work is not natural-language processing (at least not in the usual sense of natural-language processing), but rather is the application of extraction ontologies to the queries themselves. This lets us align user queries with ontologies and thus with facts in annotated web pages. In essence, these free-form queries are keyword queries over both the instances and the concepts in populated ontologies (as opposed to keyword queries over web pages). In addition, the system uses keywords and instance recognizers to identify implied operators and operands for operations. As Figure 3 shows, the query engine highlights words, values, phrases, and operations it recognizes (e.g., the context keyword miles, the value red, and the operation under applied to the value $5000 ). The highlighting provides feedback to users, letting them know which words, values, phrases, and operations the search engine recognizes. Anyone can readily pose free-form queries. To be successful, however, users have to guess which keywords, values, and constraint expressions might be available in an extraction ontology for the domain of interest. This is similar to users having to guess keywords and values for current search-engine queries. Since arbitrary free-form queries may not always be successful, we also provide a form query language, based on the ontology, that allows a user to fill out a form and submit it as a query in much the same way users currently pose queries by filling in forms currently on the web. Interestingly, these query forms are automatically derivable from domain ontologies, and thus need not be specified by developers. Instead of reverse-engineering a form to create an ontological structure, we can forward-engineer (derive) forms from the ontology and use them as a naturalforms query language (e.g., [10]). Finally, we must make all of this scale globally. It is not enough to be able to return answers to user queries and to point them to a source page showing that the answer accurately reflects the facts as recorded on the pages. We also have to return answers in real time. Users are not patient enough to tolerate long delays in receiving answers to what they perceive as a simple question. To this
A Web of Knowledge: A Conceptual-Modeling Perspective
145
end, we must define and build tools to process queries quickly. First, we cache all pages we annotate.5 Second, we have defined semantic indexing [3], with which we expect to be able to quickly find applicable ontologies for user queries, which may be the key bottleneck in the process. To pursue this further, we are inclined to follow the lead of search engines—to fully exploit massive parallelism and expansive data stores. In our research environment, however, we will not be able to build a full-scale system; we can, however, build prototypes that show the way and provide technical answers to make it work.
3
WoK Tools
Although we have not yet built the full set of tools we envision, we have developed some prototypes. In Section 3.1 we briefly describe our ontological/epistemological tools, and in Section 3.2 we briefly describe our query tools. In doing so, we emphasize again the central role of conceptual modeling. 3.1
Ontology/Epistemology Creation Tools
Automatic ontology/epistemology creation requires source documents that embody both the conceptual structure of the knowledge as well as the facts embedded in the structure. Automatic creation of a populated ontology then becomes a reverse-engineering process. Researchers in the conceptual-modeling community have worked on reverse-engineering structured data into conceptual models for many years (e.g., [4,8,20]). Web pages, however, normally include neither relational tables nor any other standard database structure. Can we extend these reverse-engineering techniques to data laid out in arbitrary, human-readable forms and tables, or to data laid out as human-readable semi-structured data, or even to unstructured human-readable data? The answer appears to be “yes”, but the task becomes increasingly more difficult as the structure of the input becomes increasingly less structured. When data is structured in well-known ways, we know which symbols represent meta-data and which represent data, and we know how the meta-data relates to the data. When data is structured in less well-known ways, distinguishing data from meta-data and relating data and meta-data become more challenging. Even when data, meta-data, and their interrelationships are known, the reverse-engineering process is not necessarily straightforward. Reverse mappings are typically not unique, so selecting among the plausible solutions can also be an issue. In our ontology/epistemology creation tools, we exploit two well-known and commonly-used information structures: forms and tables. In both cases we take a particular approach that limits the difficulty of the reverse-engineering process. For forms, we choose to stay within the bounds of well understood form fields in 5
Indeed we must, for otherwise, we cannot guarantee that our provenance links will be correct. This implies, by the way, that for sites whose pages change often, we must have a fully automatic way to re-annotate pages from the site.
146
D.W. Embley, S.W. Liddle, and C. Tao
everyday use, and we establish in advance how we will reverse-engineer each type of form field as well as each combination of form fields. We call our form-based tool FOCIH (Form-based Ontology Creation and Information Harvesting). For tables, we consider only HTML tables, and, in particular, only HTML tables in sibling pages—machine-generated pages each laid out in the same way. We call our table-interpretation tool TISP (Table Interpretation in Sibling Pages), and we augment its name to TISP++ when we refer to an extension of TISP that builds and populates ontologies from interpreted tables. We now describe FOCIH and TISP/TISP++. FOCIH FOCIH is a tool that lets users specify ontologies without having to know any conceptual-modeling language or ontology language [25]. We observe that forms are a natural way for humans to collect information. As an everyday activity, people create forms and ask others to fill in forms so that specified information can be gathered. Believing that users can specify and fill in forms, we let users create their own forms to describe information they wish to harvest. Once defined, users can fill in forms from web pages by copy and paste. From these user actions, FOCIH generates an ontology and annotates the web page with respect to the ontology. Further, if the web page is machine-generated and has sibling pages, FOCIH is able to harvest the specified information from all the sibling pages, usually without further user intervention. FOCIH’s form-creation mode provides users with a way to define different kinds of form features. FOCIH has five basic form-field elements from which users can choose: single-label/single-value, single-label/multiple-value, multiplelabel/multiple-value, mutually-exclusive choice, and non-exclusive choice. Figure 4 shows an example of form creation in our prototype implementation.6 To create a form, users click on form-field icons to say which type of form field they want and then click on the pencil icon and type in a name to provide a label for the form field. By clicking on the plus icon, users can extend the number of columns in a multiple-label/multiple-value form field and can extend choice elements to have has many choices as desired. In Figure 4 the only plus-icon appears with Accessory, the last choice for Feature. Clicking on the plus-icon would allow a user to add another feature category, for example, for safety features or anti-theft features. Finally, observe in Figure 4 that form-field icons appear inside every elementary form field; this lets users construct forms with form fields nested as deeply as desired. As an example, suppose we wish to buy a car. We can let FOCIH harvest the information we wish to consider and then query the harvested information to find cars we may want to buy. We let Car be the title for the base form as Figure 4 shows. For each car we want to harvest the Year, Make, Model, and Mileage. Since each car has only one year, make, model, and mileage, we choose 6
We note that our implementation is only a research prototype. A more intuitive interface can be built; our focus in the prototype is form-creation functionality leading to an ontology, plus automated information harvesting.
A Web of Knowledge: A Conceptual-Modeling Perspective
147
Fig. 4. A Sample Form
single-value form fields for each as Figure 4 shows. We also want car colors; since a car may have more than one color, we let Color be a single-label/multiple-value form field. A car may have several features of interest, which we wish to think of as Body features, Engine features, and Accessory features. To accommodate these specializations of Feature, we add an exclusive choice field—“exclusive” because any particular feature can only be in one of the specializations. Further, because for each feature category we can have several features in the category, we nest a single-label/multiple-value form field in each. Of course, we also want to know how much the owner expects the buyer to pay for the car, so we add a Price single-value form field at the end. Figure 4 shows the resulting form. FOCIH’s form-fill-in mode lets users browse to a web page they wish to annotate and copy and paste values into form fields. A user highlights values in the web page and then clicks on the form field to fill in a value. Figure 5 shows the price $6,990 highlighted and entered in the Price form field. To add several
148
D.W. Embley, S.W. Liddle, and C. Tao
values to a multiple-value field, a user adds each to the field one at a time. The values “4 Cylinder”, “Gasoline”, and “Automatic” for example are all Engine features. To concatenate values that may be separate such as “Altima SL” and “Black Interior” in Figure 5, a user adds subsequent components by clicking on the plus icon instead of the pencil icon. From the filled-in form, FOCIH can generate both a conceptual model, eventually to be represented as an OWL ontology, and an annotation document, eventually to be represented as RDF triples. Further, FOCIH also records the annotation information: (1) paths to leaf nodes in the DOM tree of an HTML page containing each value and, for concatenated values, each value component; (2) for each value the most specific instance recognizer from the data-frame library (e.g., string, number, year, make, model, color); and (3) enough left, right, and delimiter context within each leaf node to identify the value or values within the DOM-tree node. This enables FOCIH to harvest the same information from all machine-generated sibling pages from the same web site. Details about our implementation of FOCIH are elsewhere [25]. Since we are focusing in this paper on the role of conceptual modeling in our WoK tool set, be briefly explain here how FOCIH generates a conceptual model from a form specification. Figure 6 graphically shows the result of converting the form in Figure 4 to a conceptual model. We limit our explanation here to the generation of the features in this conceptual model.
Fig. 5. A Filled in Form with a Source Data Page
A Web of Knowledge: A Conceptual-Modeling Perspective
Make
Model
Year
Car
149
Mileage
Feature
Price Color
Body
Accessory Engine
Fig. 6. Graphical View of the FOCIH-Generated Ontology
A form title (e.g., Car in Figure 4) becomes a non-lexical object set (e.g., Car in a solid box Figure 6). Object identifiers represent values in non-lexical object sets (i.e., for each car, we generate an OID). A single-value field becomes a lexical object set functionally dependent on the object set of its enclosing form field. Thus, Year, Make, Model, Mileage, and Price are all lexical object sets (enclosed in dashed boxes, as opposed to solid boxes for non-lexical object sets), and each depends functionally on Car, the enclosing form field. As Figure 6 shows, we denote a functional relationship set with an arrow connecting a domain object set on the tail side to a range object set on the head side. A single-label/multiple-value field like Color in Figure 4 becomes a lexical object set linked non-functionally to the object set of its enclosing form field as Figure 6 shows. Because Feature is a choice form field, it is the generalization object set of a generalization/specialization. Its specializations are single-label/multiplevalue form field, which implies the possibility of several features for each car and thus that the connecting relationship set between Car and the root generalization Feature should be many-many. Since no nested form field appears inside Feature itself (i.e., nested form fields only appear inside its specializations), all the values are in the specializations and therefore the union of the values in the specializations constitutes the values in the generalization. Hence, along with the choice being exclusive, this implies a partition constraint—the specialization object sets are mutually exclusive and their union constitutes all the values. The symbol in the triangle in Figure 6 asserts that the specialization object sets Body, Engine, and Accessory partition the generalization object set Feature. Figure 6 faithfully represents all form-implied object sets, relationship sets, and implied constraints over the values in these object and relationship sets. We note, however, that form specification, as provided in FOCIH, fails to provide some constraints. FOCIH provides no way to specify reverse cardinality constraints. We do not know from the form alone, for example, whether any single-value field or any combination of single-value fields constitutes a key for Car. FOCIH also fails to provide for mandatory/optional constraints. The mileage, for example, may not be listed in some car ads, and some car ads list no features. It is the form specification, not the underlying conceptual-modeling language that imposes these limitations. Potentially, we could augment the set of form features to allow a form creator to specify additional constraints—but
150
D.W. Embley, S.W. Liddle, and C. Tao
at the expense of complicating form specification and likely losing the intuitive correspondence between FOCIH forms and forms in common usage. Thus, in the interest of maintaining simplicity, we do not add additional notation to FOCIH forms to capture these constraints or to capture other more advanced features of conceptual models. We do point out, however, that for some constraints, we can observe the data and adjust to what we see as we harvest information. For example, if we obtain enough evidence to conclude that a value in a form field uniquely identifies an object, we can infer that the form-field values are keys for the objects identified. We also point out that for the purpose of annotating web pages, neither reverse cardinality constraints nor optional/mandatory constraints matter very much, if at all. With the conceptual model generated and the data harvested from web pages with respect to the conceptual model, it is straightforward to generate an OWL ontology and RDF triples. In addition to the data, the RDF triples include annotation information: references to source pages and to each data item or data-item component within the source pages. TISP/TISP++ TISP is a tool that interprets tables in sibling pages [25]. To interpret a table is to properly associate table category labels with table data values. Using Figure 2 as an example, we see that Identification, Location, and Function are labels for the large rectangular table. Inside the right cell of the first row is another table with headers IDs, NCBI KOGs, Species, etc. Nested inside of this cell are two tables, the first with labels CGC name, Sequence name, Other name(s), WB Gene ID, Version and the second with labels Gene Model, Status, Nucleotides (coding/transcript), Protein, and Amino Acids. Most of the rest of the text in the large rectangular table comprises the data values. We associate labels with data values by observing the table structure. A cell in a table associates with its header label (or labels in the case of multi-dimensional tables). For nested tables, we trace the sequence of labels from data cell to the outermost label. Thus, for example, the associated label for the sequence-name value F18H3.5 is the sequence of labels Identification, IDs, and Sequence name. Although automatic table interpretation can be complex, if we have another page, such as the one in Figure 7, that has essentially the same structure, the system can usually obtain enough information about the structure to make automatic interpretation possible. We call pages that are from the same web site and have similar structures sibling pages.7 The two pages in Figures 2 and 7 are sibling pages. They have the same basic structure, with the same top banners that appear in all the pages from this web site, with the same table title (Gene Summary for some particular gene), and a table that contains information about the gene. Corresponding tables in sibling pages are called sibling tables. If we compare the two large tables in the main part of the sibling pages, we can see 7
Hidden-web pages are usually generated dynamically from a pre-defined templates in response to submitted queries; therefore they are usually sibling pages. Quite often hidden-web pages display their data in tables.
A Web of Knowledge: A Conceptual-Modeling Perspective
151
Fig. 7. Sibling Page
that the first columns of each table are exactly the same. If we look at the cells under the Identification label in the two tables, both contain another table with two columns. In both cases, the first column contains identical labels IDs, NCBI KOGs, ..., Gene Model Remarks. Further, the tables under Identification.IDs also have identical header rows. The data rows, however, vary considerably. Generally speaking, we can look for commonalities to find labels and look for variations to find data values. Although we look for commonalities to find labels and look for variations to find data values, we must be careful about being too strict. Sometimes there are additional or missing label-value pairs. The two nested tables whose first column header is Gene Model in Figures 2 and 7 do not share exactly the same structure. The table in Figure 2 has five columns and three rows, while the table in Figure 7 has six columns and two rows. Although they have these differences, we can still identify the structure pattern by comparing them. The top rows in the two tables are very similar. Observe that the table in Figure 7 only has
152
D.W. Embley, S.W. Liddle, and C. Tao
an additional Swissprot column inserted between the Protein and Amino Acids columns. It is still not difficult, however, to tell that the top rows are rows for labels.8 Given that we can interpret a table—find labels and values and properly associate them—our next task is to infer the general structure pattern of the table. Does the table have its labels across the top—as does the OnlineAthens table in Figure 1? Or, are the labels row headers—as are the labels in the table in the “2003 Nissan Altima” page in Figure 1? Or, does the table have both row and column headers? Or, is the table even more complex, having for example, tables nested inside one another such as the tables in Figures 2 and 7 or tables with labels that are implied or are in a tree structure? As implemented, TISP works only with tables that have labels as row headers, column headers, or both row and column headers where the row headers can be treated as values. Additionally, our TISP implementation works with tables having these structure patterns when they are nested inside one another.9 Observe that the structure patterns TISP can process are also structure patterns for FOCIH forms. We can therefore immediately generate an OSM ontology in the same way we generate an OSM ontology for FOCIH. Further, based on the TISP interpretation of the tables, we can also immediately populate the ontology and thus annotate the data. We call these additions to TISP, TISP++ . Figure 8 shows part of the ontology TISP++ generates for the sibling tables in Figures 2 and 7. Observe, for example, that nested under Location are three single-label/single-value location attributes—Genetic Position, Genomic Position, and Genomic Environs, the first two of which have values and the last of which has none. Thus, from the Location object set in Figure 8 emanate three functional relationship sets to these attributes, the last of which has optional participation. The non-functional relationship set between Identification and Gene models arises because a multiple-value form field—the nested table with labels Gene Model, ..., Amino Acids—appears as the only field nested in its outer structure, Gene models. This is similar to the nesting of the multiple-value form fields nested in Feature in Figure 4. The relationship between Identification and IDs, however, is functional because the (degenerate) table nested under IDs has only one data row in all known tables, and TISP++ therefore treats the degenerate table as a sequence of single-label/single-value form fields.10 Because of the isomorphic relationship between TISP tables and FOCIH forms, it is also possible to directly generate FOCIH forms. This leads to the 8
9 10
In our implemented TISP prototype, in addition to discovering the structure pattern for a web site, we also dynamically adjust the pattern if the system encounters a table that varies from the pattern. For example, if we had not seen the extra Swissprot column in our initial pair of sibling pages, TISP would add Swissprot as a possible label for the table when encountering it. Processing more complex structure patterns requires semantic enrichment procedures [21], which rely on semantic resources such as WordNet [14]. As TISP++ processes the sibling pages of a site, it may observe that in other sibling tables, the table nested under IDs is not degenerate. In this case, it would adjust the ontology, making the relationship from Identification to IDs non-functional.
A Web of Knowledge: A Conceptual-Modeling Perspective
153
possibility that users can enhance the TISP++ -generated ontology. Users may, for example, wish to rename the system-chosen name W ormBase with Gene—a more meaningful name for the root element of the ontology. Users may also wish to make the relationship set from IDs to Other names non-functional and show FOCIH, by example, how to recognize the list of other names, so that the names XO136 and NM 077855 in Figure 2 would be picked up out of the data cell and stored as individual names. With or without any adjustments to the TISP++ -generated ontology, we are able to generate an OWL ontology and RDF triples. Again, as with FOCIH, we also have all the annotation information we need about the pages and the data items within the pages in a generated RDF file. 3.2
Query Tools
Given a generated file of RDF triples, we are immediately able to query the file using SPARQL. For typical, untrained users, however, we need a better way to query a WoK. Like current queries to web search engines, WoK queries will likely migrate to free-form text. Further, the free-form text is likely to be cryptic, keyword-based, and non-grammatical; an example is the query in Figure 3. How can we accept this kind query as input and produce a SPARQL query as output? We base our approach on extraction ontologies. The essence of the idea is to (1) extract constants, keywords, and keyword phrases in a free-form query; (2) find the ontology that matches best; and (3) embed the query in the ontology yielding (3a) a join over the relationship-set paths connecting identified concepts, (3b) a selection over identified constants modified by identified operators, and
WormBase ...
Identification
... IDs
NCBI KOGs
Location
... GeneModels
Genetic Position ...
...
...
CGC name
Sequence name
Other names
Gene Model
Genomic Environs
Genomic Position
Swissprot
Fig. 8. Graphical View of the TISP++ -Generated Ontology
154
D.W. Embley, S.W. Liddle, and C. Tao
(3c) a projection on mentioned concepts. Both AskOntos [27] and SerFR [2] implement this basic approach to free-form query processing.11 As a key feature of extraction ontologies, the concepts each have an associated data frame. A data frame describes information about a concept—its external and internal representations, its contextual keywords or phrases that may indicate the presence of an instance of the concept, operations that convert between internal and external representations, and other manipulation operations that can apply to instances of the concept along with contextual keywords or phrases that indicate the applicability of an operation. Figure 9 shows sample (partial) data frames for the concepts Price and Make for the ontology in Figure 6. As Figure 9 shows, we use regular expressions to capture external representations. The Price data frame, for example, captures instances such as “$4500” and “17,900”. A data frame’s context keywords are also regular expressions. The Price data frame in Figure 9, for example, includes context keywords such as “asking” and “negotiable”. In the context of one of these keywords in a car ad, if a number appears, it is likely that this number is a price. The operations of a data frame can manipulate a concept’s instances. For example, the Price data frame includes the operation LessThan that takes two instances of Price and returns a Boolean. The context keywords of an operation indicate an operation’s applicability; context keywords such as “less than” and “<”, for example, apply to the LessThan operation. Sometimes external representations are best described by lexicons. These lexicons are also regular expressions—simple lists of possible external representations—and can be used in place of or in combination with other regular expressions. In Figure 9, CarMake.lexicon is a lexicon of car makes, which would include, for example, “Toyota”, “Honda”, and “Nissan” and potentially also misspellings (e.g. “Volkswagon”) and abbreviations (e.g. “Chev” and “Chevy”). We can apply an extraction ontology to obtain a structured representation of the unstructured information in a relevant document. For example, given that we have added data frames to the ontology in Figure 6, making it an extraction ontology, and given a car ad such as the first Nissan ad in the City Weekly page in Figure 1: ’93 NISSAN Model XE, $900, Air Conditioning, new tires, sweet cherry red. For listings call 1-800-749-8104 ext. V896.
we can extract “’93” as the Year, “NISSAN” as the Make, “XE” as the Model, “$900” as the Price, “red” as the Color, and both “Air Conditioning” and “new tires” as Accessorys. As part of the extraction, the conversion routines in the data frames convert these extracted values to canonical internal representations, so that, for example, “’93” becomes the integer 1993 and “$900” becomes the integer 900. Now, consider the sample query in Figure 3 and assume that the ontology in Figure 6 is an extraction ontology—i.e. is augmented with data frames as 11
SerFR builds on AskOntos by expanding its options to recognize more complex operations and to provide for advanced query specification and resolution. Further, SerFR can also link directly to web services.
A Web of Knowledge: A Conceptual-Modeling Perspective
155
Price internal representation: Integer external representation: \$?(\d+ | \d?\d?\d,\d\d\d) context keywords: price | asking | obo | neg(\.|otiable) | ... ... LessThan(p1: Price, p2: Price) returns (Boolean) context keywords: less than | < | or less | fewer | ... ... end Make external representation: CarMake.lexicon ... end Fig. 9. Sample Data Frames
illustrated in Figure 9. When we apply this extraction ontology to the query, the extraction ontology recognizes the highlighted strings in the query (see Figure 3). It recognizes “red” as a possible value for the Color object set, “Nissan” for Model, “$5000” for Price, “1990” for Year, and, with the aid of the recognized keyword “miles”, “120K” as a Mileage. The conversion routines in the data frames normalize all these values, making the numbers integers, converting “red” to its RGB value, and standardizing car makes to have initial capital letters (no change to the string “Nissan” in this case). The extraction ontology also recognizes “under” as the less-than operator for Price, “or newer” as the greaterthan operator for Year, and “less than” as the less-than operator for Mileage. From this extracted information along with the known ontological structure of the data provided by the generated OWL ontology and the data itself provided by the RDF triples, it is straightforward to generate a SPARQL query. In essence, the query searches for cars that satisfy the following constraints: Year ≥ 1990 Make = ‘Nissan’ Mileage ≤ 120000 ColorWithinRange(255,0,0) Price ≤ 5000 Because we are processing queries under an open- rather than a closed-world assumption, we generate the SPARQL query with OPTIONAL clauses allowing it to return cars that, for example, have no color specified or no mileage specified so long as they meet the requirements for the fields where values do appear.
4
Conclusion
This work presents a grand vision of a “Web of Knowledge” (a “WoK”)—a vision that others share [6]. A major barrier to realizing this vision is the overwhelming
156
D.W. Embley, S.W. Liddle, and C. Tao
amount of human effort that appears to be required both for creating and querying WoK content. To surmount this barrier, we have described ontological and epistemological creation tools to significantly reduce or totally eliminate the barrier to creating WoK content, and we have described a query tool usable by anyone. FOCIH allows users with no training to specify simple ontologies and to annotate web pages with respect to these ontologies. FOCIH can also annotate and harvest specified information from all sibling pages of an initial hand-annotated web page. TISP/TISP++ uses sibling tables to interpret tables and from interpreted tables to generate ontologies and to annotate the information in these interpreted tables with respect to these generated ontologies. TISP/TISP++ is fully automatic, but is limited to information captured in sibling tables. AskOntos and SerFR provide for free-form query processing. And, although SerFR supports an advanced query interface for sophisticated users, less sophisticated users (most people) can only pose conjunctive queries and must limit their vocabulary to words, phrases, and symbols recognized by data frames associated with ontologies. Conceptual modeling plays a key role in actualizing these ideas. An ontology is a conceptualization of a real-world domain in terms of object sets, relationship sets, generalizations, specializations, and constraints over these conceptualizations. Indeed an ontology can be thought of as a conceptual model grounded formally in a logic system. Automatic and semi-automatic ontology generation from data-rich, semi-structured web pages is akin to reverse engineering structured data into conceptual models—a task that has traditionally been associated with the conceptual-modeling community. Automatic and semi-automatic annotation of web pages can proceed bottom-up, occurring as a by-product of ontology generation via reverse engineering. Or annotation can proceed top-down, coming from extraction ontologies in which instance recognizers attached to conceptual object sets and relationship sets extract data on web pages with respect to conceptual models comprising these object and relationship sets. In either case, conceptual modeling plays the role of organizing this knowledge. For query processing, conceptual models grounded in description logics form a template to which free-form queries can be matched to yield formal queries to be processed by standard query engines. Opportunities for future work abound. And many of these opportunities are best approached through conceptual modeling. Related to FOCIH, we see the following: – Many OWL ontologies and ontologies in other structured forms already exist. We should be able to reverse-engineer them into FOCIH forms, which would immediately allow users to annotate web pages with respect to these existing ontologies. Users should also be able to alter these reverse-engineered forms and thus tailor them to suit their needs. Tailoring can also be a joint venture, which provides means for cooperative knowledge definition. – Structured data repositories also already exist. We should also be able to reverse-engineer them into FOCIH forms. Further, we should be able to capture and annotate their data as well.
A Web of Knowledge: A Conceptual-Modeling Perspective
157
– Given an extraction ontology, in addition to being able to reverse-engineer it into a FOCIH form, we should also be able to use the extraction ontology to automatically do the initial form fill-in. A user could correct any annotation mistakes the augmented FOCIH system might make and complete the form fill-in for any data items it might miss. Information harvesting could then proceed with minimal user intervention—ideally, none at all. Related to TISP/TISP++ , we see the following: – TISP only interprets HTML tables. In principle, the ideas apply to all welldefined tables, including Microsoft Word tables, Excel tables, and PDF tables, in which vast amounts of data appear. – The idea of using sibling tables to identify category labels, data, and the relationships between category labels and data can extend beyond tables to semi-structured sibling pages in general. We should also be able to identify and interpret lists and patterned layout by sibling-page comparison. – Because TISP works with HTML tables, which typically have simple label structures, it does very little to semantically enrich the tables it interprets. By considering semantic lexicons such as WordNet [14] and other semantic resources such as a data-frame library, it is possible to considerably enrich interpreted tables by identifying generalizations, specializations, and aggregations within label structures and discovering constraints and interrelationships among data items not initially apparent in the data and meta-data of an interpreted table [21]. Related to free-form queries, we see the following: – Often, even when users are unable to find the “right” vocabulary for making requests, a WoK system should be able to find an appropriate ontology. This ontology provides context for an interaction between system and user. We could exploit this context by exposing the vocabulary of the ontology and thereby allowing users to find the “right” words with which to ask their questions. It is also possible to generate, on the fly, a standard query form based on the ontology. Users should be able to fill in this form, as users do for typical HTML forms, to pose their queries. These forms would include drop-down selection lists for form entries with a short list of possibilities, range queries for ordered types, and facilities for more complex queries such as disjunctive queries and queries with negation. – We can further explore advanced query specification. We could, for example, explore the use of natural-language processing techniques for generating logic statements [22]. – It is possible to linguistically ground reasoning rules so that they too can play a role in free-form query processing. Further, then, in addition to providing provenance for extracted facts by allowing users to click on results to see original sources, we can also provide provenance for inferred facts by exposing and explaining reasoning chains and showing how they eventually resolve into extracted facts.
158
D.W. Embley, S.W. Liddle, and C. Tao
Related to a data-frame library, we see the following: – As FOCIH and TISP (or any other annotation technique) runs, we can enhance data frames. Data frames commonly have lexicons, and as new values are annotated, they can be added to these lexicons. For example, as new makes and models of cars become annotated, the envisioned WoK system can automatically add them to existing make and model lexicons. – As FOCIH and TISP (or any other ontology-generation technique) creates ontologies, the envisioned WoK system could extract reusable knowledge components and store them in the data-frame library. For example, it is common to informally see car models as concatenations of what is technically the model and what is technically called the trim. (See Figure 1, which shows the model and trim in accord with these technical terms.) To accommodate both, a molecular-size knowledge component can describe Model as the aggregate concatenation of Model and Trim. In general, having a large collection of these molecular-size components in the data-frame library would aid in semantic enrichment and instance recognition. We are implementing our WoK prototype with a MySQL backend database and programs written in Java, PHP, and JavaScript. Currently, we have completed an initial implementation of FOCIH, TISP, and TISP++ as described here, and we have implemented them so that they run smoothly together with other tools such as OWL ontology generators, RDF instance generators, and a SPARQL query engine. We have also separately implemented prototypes for AskOntos and SerFR, and we have integrated a basic AskOntos query engine into our WoK prototype. In addition, we have established a data-frame library, but we have only begun to populate it with useful atomic and molecular-size data frames. Earlier versions of our extraction ontologies have been separately implemented. Currently, we are upgrading and integrating these prototypes into our WoK prototype. We have accomplished much, but, as always, there is much more to do.
References 1. Al-Kamha, R.: Conceptual XML for Systems Analysis. PhD dissertation, Brigham Young University, Department of Computer Science (June 2007) 2. Al-Muhammed, M.J.: Ontology Aware Software Service Agents: Meeting Ordinary User Needs on the Semantic Web. PhD dissertation, Brigham Young University, Provo, Utah (August 2007) 3. Al-Muhammed, M.J., Embley, D.W., Liddle, S.W., Tijerino, Y.: Bringing web principles to services: Ontology-based web services. In: Proceedings of the Fourth International Workshop on Semantic Web for Services and Processes (SWSP 2007), Salt Lake City, Utah, pp. 73–80 (2007) 4. Alhajj, R.: Extracting the extended entity-relationship model from a legacy relational database. Information Systems 28(6), 597–618 (2003) 5. Baader, F., Nutt, W.: Basic description logics. In: Baader, F., Calvanese, D., McGuinness, D., Nardi, D., Patel-Schneider, P. (eds.) The Description Logic Handbook, ch. 2, pp. 43–95. Cambridge University Press, Cambridge (2003)
A Web of Knowledge: A Conceptual-Modeling Perspective
159
6. Berners-Lee, T.: Future of the world wide web, March 1 (2007), Testimony of Sir Timothy Berners-Lee Before the United States House of Representatives Committee on Energy and Commerce Subcommittee on Telecommunications and the Internet, http://dig.csail.mit.edu/2007/03/01-ushouse-future-of-the-web 7. Buitelaar, P., Cimiano, P., Loos, B.: Preface. In: Proceedings of the 2nd Workshop on Ontology Learning and Population: Bridging the Gap Between Text and Knowledge (COLING-ACL 2006), Sydney, Australia (July 2006) 8. Chiang, R.H.L., Barron, T.M., Storey, V.C.: Reverse engineering of relational databases: Extraction of an EER model from a relational database. Data and Knolwedge Engineering 12(1), 107–142 (1994) 9. Ding, Y., Embley, D.W., Liddle, S.W.: Automatic creation and simplified querying of semantic web content: An approach based on information-extraction ontologies. In: Mizoguchi, R., Shi, Z.-Z., Giunchiglia, F. (eds.) ASWC 2006. LNCS, vol. 4185, pp. 400–414. Springer, Heidelberg (2006) 10. Embley, D.W.: NFQL: The natural forms query language. ACM Transactions on Database Systems 14(2), 168–211 (1989) 11. Embley, D.W., Campbell, D.M., Jiang, Y.S., Liddle, S.W., Lonsdale, D.W., Ng, Y.-K., Smith, R.D.: Conceptual-model-based data extraction from multiple-record web pages. Data & Knowledge Engineering 31(3), 227–251 (1999) 12. Embley, D.W., Kurtz, B.D., Woodfield, S.N.: Object-oriented Systems Analysis: A Model-Driven Approach. Prentice-Hall, Englewood Cliffs (1992) 13. Embley, D.W., Zitzelberger, A.: Theoretical foundations for enabling a web of knowledge. In: Proceedings of the Sixth International Symposium on Foundations of Information and Knowledge Systems (FoIKS 2010), Sophia, Bulgaria, pp. 211– 229 (February 2010) 14. Fellbaum, C.: WordNet: An Electronic Lexical Database. MIT Press, Cambridge (1998) 15. Footnote.com (2007), http://www.footnote.com 16. Gruber, T.R.: A translation approach to portable ontology specifications. Knowledge Acquisition 5(2), 199–220 (1993) 17. Guarino, N.: Formal ontologies and information systems. In: Guarino, N. (ed.) Proceedings of the First International Conference on Formal Ontology in Information Systems (FOIS 1998), Trento, Italy, pp. 3–15 (June 1998) 18. Halpin, T.: Conceptual Schema & Relational Database Design, 2nd edn. Prentice Hall of Australia Pty. Ltd., Sydney (1995) 19. Jarrar, M.: Towards automated reasoning on ORM schemes. In: Parent, C., Schewe, K.-D., Storey, V.C., Thalheim, B. (eds.) ER 2007. LNCS, vol. 4801, pp. 181–197. Springer, Heidelberg (2007) 20. Lammari, N., Comyn-Wattiau, I., Akoka, J.: Extracting generalization hierarchies from relational databases: A reverse engineering approach. Data & Knowledge Engineering 63(2), 568–589 (2007) 21. Lynn, S.: Automating mini-ontology generation from canonical tables. Master’s thesis, Department of Computer Science, Brigham Young University, Provo, Utah (2008) 22. Rus, V.: A first evaluation of logic form identification systems. In: Mihalcea, R., Edmonds, P. (eds.) Senseval-3: Third International Workshop on the Evaluation of Systems for the Semantic Analysis of Text, Barcelona, Spain, pp. 37–40 (March 2004) 23. Sirin, E., Parsia, B., Grau, B.C., Kalyanpur, A., Katz, Y.: A practical OWL-DL reasoner. Journal of Web Semantics 5(2), 51–53 (2007)
160
D.W. Embley, S.W. Liddle, and C. Tao
24. Smith, B.: Ontology. In: Floridi, L. (ed.) Blackwell Guide to the Philosophy of Computing and Information, pp. 155–166. Blackwell, Oxford (2003) 25. Tao, C.: Ontology Generation, Information Harvesting and Semantic Annotation for Machine-Generated Web Pages. PhD dissertation, Brigham Young University, Department of Computer Science (December 2008) 26. Tijerino, Y.A., Embley, D.W., Lonsdale, D.W., Ding, Y., Nagy, G.: Toward ontology generation from tables. World Wide Web: Internet and Web Information Systems 8(3), 261–285 (2005) 27. Vickers, M.: Ontology-based free-form query processing for the semantic web. Master’s thesis, Brigham Young University, Provo, Utah (June 2006) 28. OWL Web Ontology Language Reference Manual. W3C (World Wide Web Consortium), http://www.w3.org/TR/owl-ref 29. Resource Description Framework (RDF). W3C (World Wide Web Consortium), http://www.w3.org/RDF 30. SPARQL Query Language for RDF. W3C (World Wide Web Consortium), http://www.w3.org/TR/rdf-sparql-query
Service-Based Semantic Collaboration in Networked Systems: A Conceptual Perspective Devis Bianchini, Valeria De Antonellis, and Michele Melchiori Universit` a di Brescia Dip. di Ingegneria dell’Informazione Via Branze, 38 - 25123 Brescia, Italy {bianchin,deantone,melchior}@ing.unibs.it
Abstract. In a peer-to-peer (P2P) scenario, modern collaborative systems are characterized by networked independent partners that dynamically cooperate by sharing available resources, both data and services. Partners may join and leave the P2P network at any moment; in fact, they loosely connect one to another, looking for parties that can provide relevant knowledge with respect to a given target. They cooperate without any a-priori knowledge about each other. Furthermore, there is no single, shared, global view of the available, distributed resources. In this scenario, to support effective collaboration by resource sharing, new advanced tools are required for semantic representation and discovery of distributed resources. In this paper, we focus on service-oriented collaborative systems and propose a conceptual framework to enable the construction of a semantic overlay for dynamic service discovery and sharing, built on the top of a P2P network. Keywords: Service semantic overlay, P2P semantic collaboration, peer knowledge infrastructure, semantic interoperability, semantic routing.
1
Introduction
In a peer-to-peer (P2P) scenario, modern collaborative systems are characterized by networked independent partners that dynamically need to cooperate by sharing available resources, both data and services. For example, consider a P2P network where various organizations are strongly motivated to extend their knowledge about potential partners available in the web and to collaborate with them. The main purpose of the collaboration is to share and access the huge number of available resources over the network. Typically, the collaboration among partners takes place in a dynamic way. Partners have their own knowledge and loosely connect one to another (in general, only when there is a need for knowledge sharing). Partners can join and leave the P2P network at any moment and the collaboration constraints may frequently change. Taking into account the dynamic nature of the collaboration and the multi-knowledge scenario, where each peer refers to its own knowledge representation, it becomes impossible and unrealistic to have a single, global view of the distributed resources. Therefore, a global agreement for knowledge Kaschek/Delcambre (Eds.): The Evolution of Conceptual Modeling, LNCS 6520, pp. 161–179, 2011. c Springer-Verlag Berlin Heidelberg 2011
162
D. Bianchini, V. De Antonellis, and M. Melchiori
sharing must be reached dynamically and can only emerge from local interactions between collaborative partners [1,30]. In recent literature, with the advent of the Semantic Web, the use of ontologies for knowledge sharing has been widely suggested. In particular, ontology-based techniques have been defined for data and service sharing and discovery. In fact, ontologies provide the benefits of formal specifications and inference capabilities apt to improve semantic resource description and matchmaking. Ontologies and Semantic Web technologies are adopted as key solutions to actually define the fundamental components of semantic collaborative platforms. Meaningful examples of such components are semantics- and context-driven data and service discovery [7,9,20]. In a P2P scenario, additional difficulties in knowledge sharing must be considered, due to: (i) the highly dynamic nature of peer collaboration; (ii) the lack of any agreed-upon global view; (iii) the necessity of modeling different forms of collaboration between peers abstracting from technical details on the underlying network communication and setup mechanisms. 1.1
Contribution
In this chapter, we describe P2P Semantic Driven Service Discovery (P2PSDSD), a conceptual framework to enable the construction of a service semantic overlay (SSO) for service discovery and sharing in service-oriented collaborative systems built on top of a P2P network. The framework is constituted by: (i) a conceptual model of the semantic overlay; (ii) a set of semantic matching techniques for the creation and the evolution of the semantic overlay; (iii) a collaboration model with different policies to support interaction among peers. The semantic overlay is built over a network of collaborative peers: each peer maintains its own semantic service descriptions, and semantic links towards similar service descriptions belonging to different peers. Semantic links are dynamically established and maintained as a consequence of local interactions between peers. The set of semantic service descriptions and semantic links across peers in the network constitutes the semantic overlay. In fact, the semantic overlay can be seen as a continuously evolving global view over the peers that provide similar services and constitute synergetic service centres in a given domain. In contrast to our earlier work [9], here we consider the P2P environment and its featuring aspects related to the dynamic conditions of collaboration, and the possibility of reaching a global agreement for knowledge sharing only through local interactions between partners. Original contributions of this chapter concern: the knowledge model of each P2P-SDSD peer; the definition of the semantic overlay as a dynamic global view across collaborative peers; the dynamic collaboration model. The chapter is organized as follows: Section 2 presents the related work; Section 3 describes an application scenario; Section 4 describes the semantic overlay modeling framework; Section 5 discusses the techniques to build and maintain the semantic overlay; Section 6 illustrates the collaboration model and the different policies based on the semantic overlay; Section 7 presents concluding remarks.
Service-Based Semantic Collaboration in Networked Systems
2
163
Related Work
Research areas relevant to the topic of this chapter include the use of ontologies for semantic service descriptions as well as techniques and tools for ontology mapping and semantic-driven service discovery. 2.1
Ontology-Based Service Description
Ontology-based techniques and tools are exploited for semantic service descriptions and for for semantic service discovery. In particular, for semantic service description, OWL-S [13] and WSMO [10] provide the most general frameworks. OWL-S uses the OWL language to describe services by means of three key elements: (i) a service profile, to describe the functionalities of the service, what it requires from the user and what it provides, that is, the input/output parameters, preconditions, results and service category; (ii) a service model, to give a detailed description of how the service works and how it has to be invoked; (iii) a service grounding, to map the abstract functional interface of the service into the concrete implementation and to provide details about how to interact with the service by means of message exchanges. The Web Service Modeling Ontology (WSMO) provides a formal language (Web Service Modeling Language WSML [24]) to semantically describe services through four core elements: (i) ontologies, to provide the terminology used by other WSMO elements in terms of machine-processable formal definitions; (ii) Web Services, to describe non functional properties, capabilities and interfaces; (iii) goals, to represent the user’s requirements in terms of expected outputs and effects; and (iv) mediators, to deal with interoperability problems between different WSMO elements; a WSMO mediator serves as a third party component connecting heterogeneous elements. Both OWL-S and WSMO require a high expressiveness of the description language with consequent computational complexity in reasoning. In the proposed approach, service descriptions are given in terms of functional interfaces (operations and I/Os). Ontologies are used to provide semantics to these descriptions, by means of annotation, and to organize them on the basis of semantic relationships. 2.2
Ontology Mapping
Ontology mapping or alignment, has gained more and more attention as the number of available, independently developed ontologies increased [28]. Ontology mapping is defined as a set of correspondences between elements of ontologies [12]. Correspondences can be equivalence relationships, subclass or superclass relationships, transformation rules. Some approaches for mapping discovery between two ontologies use, as intermediary, other ontologies, that are developed with the explicit goal of providing the basis for future semantic integration. A general upper ontology is agreed upon
164
D. Bianchini, V. De Antonellis, and M. Melchiori
by developers, who then specialize this general ontology with concepts, properties and semantic relationships for specific application domains. Examples such as the Suggested Upper Merged Ontology (SUMO) [26] and DOLCE [17] have been developed as formal foundational ontologies containing high-level concepts that can be specialized. Other approaches use a reference ontology or terminology that helps in aligning the ontologies. For example, S-Match [18] uses WordNet as the terminology to bridge the different ontologies. Zhang and Bodenreider used UMLS1 to verify automatically generated mappings between two anatomy ontologies [33]. In both cases, lexical-matching techniques are used to map concepts from the source ontologies. More complex approaches directly compare elements of the structure of ontologies to be mapped. QOM [15] and HMatch [11] use a range of structural features such as concept labels, domains and ranges of properties, relations between classes to find similar concepts in source ontologies. Prompt [27] is an ontology mapping tool that uses a mixture of lexical and structural features, as well as input from the user during an interactive merging session to find mappings. GLUE [14] and FOAM [14] employ machine-learning techniques to find mappings. Gligorov and colleagues [19] perform approximate mapping by decomposing a concept definition into a set of sub-definitions and by determining how many of these sub-definitions map another concept that they are trying to match. In our approach we use WordNet as a reference terminology for mapping concepts of different peer ontologies. 2.3
P2P Semantic-Driven Service Discovery
P2P semantic-driven service discovery has attracted much attention from the Web services and Semantic Web areas and relies on several efforts in related research fields, such as data integration and emergent semantics in P2P environments [1,5,20] to go beyond limitations of centralized service-oriented architectures. P2P semantic-driven service discovery often extends work on semantic service discovery [21,22,23] to distributed environments. In METEOR-S [31] service descriptions are kept in UDDI Registries semantically enhanced with local domain ontologies. A centralized registries ontology is used to classify peer registries. During the discovery process, the registries ontology is browsed to find the proper registry for request submission. WSPDS [3] describes a P2P network where peers refer to global DAML ontologies to provide service semantics and links with other peers based on similarity between services they provide. When a request is submitted to a peer, it searches for local matching results and, at the same time, it forwards the request to all the known peers where services similar to the request are stored. The forwarding strategy is therefore independent of the local matching results. In another effort [4], a P2P-based system to support efficient access to data resident in e-catalogs is provided. Scalability issues are solved by organizing the information space in communities that are inter-related using peer relationships, 1
http://www.nlm.nih.gov/research/umls/.
Service-Based Semantic Collaboration in Networked Systems
165
defined as mappings between ontologies. Selection of relevant peers/communities to which user queries must be forwarded is based on a query rewriting algorithm. In these approaches, the main limitations are due either to the assumption of a common ontology [3,31] or to the lack of strategies apt to prune the set of peers to which to forward the request [4]. In P2P-SDSD, we aim to overcome both these limitations. In P2P-SDSD, specifically, peers are not committed to use the same ontology. As a consequence, a priori agreement among the peers is not required and there is no need for special peers storing centralized data structures, thus increasing the robustness of the approach. Moreover, with respect to forwarding service requests over the P2P network, we propose a collaboration model, with different forwarding policies, based on the semantic overlay.
3
Application Scenario
The P2P-SDSD approach assumes that peers need, or are interested, to augment their common knowledge and provide a larger offering of services to their users. This requirement is well suited for collaboration scenarios, e.g., collaborative non-profit organizations. The approach described in this chapter has been developed mainly in the ESTEEM (Emergent Semantics and cooperaTion in multiknowledgE EnvironMents) project [8], where a comprehensive platform for data and service discovery in P2P systems is proposed, with advanced solutions for trust and quality-based data management, P2P infrastructure definition, query processing and dynamic service discovery in a context-aware scenario. In particular, in this project we have considered the healthcare domain where peer organizations (laboratories, drug-stores, research centres, hospitals) provide useful services (i.e, drug ordering, diagnosis services, on-line consultation). Consider a doctor working in a small hospital, during his/her daily job, who has to treat a patient for anaphylactic shock treatment in presence of complex clinical conditions with concurrent diseases. The doctor searches for new suitable drugs, but specific analysis is required to figure out possible side effects or interactions between different active principles administered to the patient. To establish a correct diagnosis, the doctor needs specialized laboratory analysis services and to consult healthcare centres and/or specialists. After a diagnosis is established, the doctor needs drug ordering services to deliver the suitable medicines not available in his/her small hospital. To retrieve information and services that may help in formulating correct diagnoses and choose adequate treatments, the doctor could join with his/her PDA, laptop or PC a network of healthcare organizations (laboratories, drug-stores, research centres, hospitals) and other doctors (e.g., specialists in immunological diseases), where he/she could obtain or share expertise and find useful healthcare services. In the healthcare domain, different vocabularies and ontologies exist (such as MeSH [25], SNOMED-CT [29], GALEN [16]) and are adopted by different healthcare organizations, raising interoperability issues. Examples in the following section will refer to the healthcare scenario.
166
3.1
D. Bianchini, V. De Antonellis, and M. Melchiori
Overview of the P2P Collaborative Network
We consider a P2P collaborative network where any peer can play three different roles: (1) it can store semantic service descriptions and references to corresponding deployments (broker); (2) it can publish on any broker the semantic service description of a new service (provider); (3) it can look for a service (requester). To become member of the collaborative network a peer PX must register at the network access point. After the registration, PX may connect and see the IP’s of other connected peers. If PX is a provider, it can publish its services on a broker PY , then it waits for a request to serve. If PX is a requester, it can send a service request to a broker PY , then it waits for the answer. Finally, if PX is a broker, it can contact a broker PY to receive a list of brokers known by PY . Brokers are in charge of establishing and maintaining the semantic links to other brokers on the basis of similarity between semantic service descriptions they store. Semantic service descriptions and semantic links constitute the service semantic overlay (SSO) over the network. In the rest of the chapter, we will explain how the semantic overlay is built and maintained and how collaboration policies can be defined on it.
4
The Semantic Overlay Modeling Framework
In this section, the overall vision of the knowledge infrastructure over the P2P network is described. 4.1
Modeling the Peer Knowledge
In the P2P-SDSD approach, independent peers cooperate in a common domain by sharing services without any a priori reciprocal knowledge. In such a scenario, no centralized authorities are defined to provide a comprehensive view of the shared services in the collaborative network. Each peer refers to its own ontology. Ontologies are used for semantic annotation of service descriptions, to obtain a formal conceptualization and to enable semantic matching [7]. The ontological knowledge is represented and stored using the OWL-DL language. Any broker is endorsed with an UDDI Registry and a peer knowledge infrastructure. In the UDDI Registry, implementations of services are registered with their URL. The peer knowledge infrastructure is constituted by local and network knowledge as shown by the UML class diagram in Figure 1. Local Knowledge. Services are advertised on brokers through the description of their interface, in terms of operations and related Input/Output parameters with semantic annotations by means of ontological concepts. The resulting semantic service descriptions are expressed according to the WSDL-S language [2]) and associated to their implementations through the tModel mechanism in the UDDI registry. Semantic service descriptions are part of the local knowledge infrastructure of any broker, which includes:
Service-Based Semantic Collaboration in Networked Systems
167
Fig. 1. Peer knowledge infrastructure
– Service Category Taxonomy (SCT), extracted from available standard taxonomies (e.g., like general service taxonomies UNSPSC, NAiCS or domain specific ones) to conceptualize service categories. Semantic service descriptions are associated to SCT categories; note that since Service Category Taxonomy contains high level categories from widely used classifications, we assume that in the network the same SCT is used by each broker. – Peer Ontology, that provides the definitions of concepts used to annotate operations and I/O parameters and the definitions of semantic relationships between concepts; note that, in general, different brokers may use different ontologies. As regards the Peer Ontology (see Figure 2), it is constituted by: (i) a Service Functionality Ontology (SFO), that provides semantics on the concepts used to express service functionalities (operations); (ii) a Service Message Ontology (SMO), that provides semantics on the concepts used to express input and output messages (parameters) of services. Concepts in the Peer Ontology are related through formal semantic relationships (e.g., subclass-of or equivalent-to). In the considered multi-knowledge environment, different ontologies on different peers could refer to the same concept using different terms, that can be synonyms or related by other terminological relationships (e.g., broader/narrower terms). To bridge the gap between slightly different terminologies, in P2P-SDSD, each Peer Ontology is extended with a thesaurus of terms related by terminological relationships to the names of concepts in the Peer Ontology. A thesaurus is formed by a set of terms and a set of terminological relationships between them. The considered types of terminological relationships are syn for synonyms, bt for broader terms (resp., nt for narrower terms) and rt for other term relationships. The thesaurus is built from the WordNet lexical system2 and manages also 2
WordNet: http://wordnet.princeton.edu.
168
D. Bianchini, V. De Antonellis, and M. Melchiori
composite terms, e.g., getTherapy, as explained in [7]. In this way, it is possible to enlarge matchmaking capabilities when looking for correspondences between elements in semantic service descriptions in presence of different ontologies as happens in the P2P environment. In fact, terminological knowledge is used to extend the traditional Description Logics subsumption test. A detailed explanation of how the thesaurus is built and used has been presented in the cited work [7].
Fig. 2. An example of local knowledge infrastructure
Example 1. Figure 2 shows a portion of a peer ontological equipment for the healthcare reference scenario. In particular, consider the PosologyServ service in figure. The service is categorized as a Drug Consultation Service, that is a Consultation Service, which in turn is a Healthcare Service, according to the NAiCS taxonomy. The getInfo operation is annotated with getDrugInformation concept from the SFO, to conceptualize the action of provisioning information about a particular drug. Service input and output parameters are associated to concepts in the SMO, for example, PharmaceuticalDrug is defined as a Pharmacologic Substance. Since different organizations could refer to other terminologies, the thesaurus is exploited to provide terms related to SMO concept names by means of terminological relationships. For example, Dosage and Dose are two synonyms, used in two distinct terminologies, to denote the quantity of substance to be administered.
Service-Based Semantic Collaboration in Networked Systems
169
Fig. 3. An example of network knowledge infrastructure
Network Knowledge. The network knowledge of a broker is modeled by a set of semantic links to brokers which store similar semantic service descriptions and called semantic neighbors). Precisely, a semantic link relates a locally published semantic service description with a similar description published on a semantic neighbor. Semantic links are established by applying semantic matchmaking techniques and are labeled by matching information, as explained in the Section 5.1. Each broker maintains a representation of its network knowledge, as shown in Figure 3. In this example, the network knowledge describes a semantic link from the healthcare service to a semantic neighbor PY . The network knowledge description is associated to the service thorough the tModel mechanism. The set of all dynamic semantic links across peers in the networks is part of the dynamic semantic overlay. 4.2
Ontology-Based Matchmaking Techniques for Semantic Overlay Creation and Evolution
To compare semantic service descriptions and define semantic links, we use a hybrid matchmaking model defined in our earlier work [7], supported by the COMPAT architecture [6]. In the hybrid matchmaking, a deductive matchmaking model is combined with a similarity-based model to compare services on the basis of their functional interface. The SCT, Peer Ontology and thesaurus are exploited by the matchmaker to identify matching services. The deductive matchmaking model is based on a subsumption checking () between concepts used to annotate operations and I/O parameters in the compared service descriptions. The subsumption checking is based on the Peer
170
D. Bianchini, V. De Antonellis, and M. Melchiori
Ontology and thesaurus, and it is performed by a DL reasoner. The deductive model is used to qualify the kind of match MatchType(S1,S2 ) between two semantic service descriptions S1 and S2 . We have provided a formal definitions of the following kinds of match [7]. S2 Exact S1 to denote that S1 and S2 have the same capabilities, that is, for each operation in S1 there exists an operation in S2 that has: (i) an equivalent name; (ii) equivalent output parameters; (iii) equivalent input parameters. S2 Extends S1 to denote that S2 offers at least the same capabilities of S1 , that is, for each operation in S1 there exists an operation in S2 that has: (i) an equivalent or more specific operation name; (ii) an equivalent or more specific output parameter for each output parameter of the S1 operation; (iii) a set of input parameters, each of them is equivalent or more generic than an input parameter of the S1 operation; the rationale behind the Extends match is that S2 at least fulfills the functionalities of S1 if it provides all the S1 outputs, but, on the other hand, S1 must be able to provide all the inputs needed for the execution of S2 . S2 Intersects S1 to denote that S1 and S2 have some common capabilities, that is, there exist an operation in S2 and an operation in S1 such that: (i) their names are related in any generalization hierarchy; (ii) there exists at least a pair of I/O parameters, one from S1 and one from S2 , that are related in any generalization hierarchy. S2 Mismatch S1 otherwise. In case of partial match (S1 Extends|Intersects S2 ), a similarity-based matching model is used to quantify the degree of match GSim(S1 , S2 ) between semantic service descriptions. This evaluation is performed through coefficients properly defined to deal with service functional interfaces shown in Table 1. Exact and Extends matches correspond to the case GSim(S1 , S2 ) = 1.0, while Mismatch corresponds to the case GSim(S1 , S2 ) = 0.0. GSim() is obtained as a weighted sum of the normalized Entity-based similarity coefficient N ormESim() and the Functionality-based similarity coefficient N ormF Sim(). N ormESim() evaluates the similarity of I/O parameters of the whole services to measure how much they are based on the same information. Inputs and outputs are considered separately. If both S1 and S2 have no inputs, the first term of ESim() formula in Table 1 is set to 1. Similarly, the second term of F Sim() formula is set to 1. For instance, this is the case of a service with just one operation that returns the time. N ormF Sim() compares pairs of operations together with their corresponding I/O parameters to measure how much the two services perform the same functionalities. The similarity between two operations is computed through the OpSim() coefficient, that takes into account the affinity between names of operations, similarity between names of their input parameters and similarity between
Service-Based Semantic Collaboration in Networked Systems
171
Table 1. Similarity coefficients between two semantic service descriptions S1 and S2 Entity-based similarity
ESim(S1 , S2 ) =
2·Atot (INS ,INS ) 1 2 |INS |+|INS | 1 2
+
2·Atot (OU TS ,OU TS ) 1 2 |OU TS |+|OU TS | 1 2
∈ [0, 2]
INS1 , INS2 - sets of input parameter names of S1 and S2 OU TS1 , OU TS2 - sets of output parameter names of S1 and S2 Atot (INS1 , INS2 ) = N A(iniS , injS ) j 1 ∈INS ini ∈INS ,in 2 S2 S 2 1 1 Atot (OU TS1 , OU TS2 ) = N A(outiS , outjS ) j i out
S1
1
∈OU TS ∈OU TS ,out S2 2 1
2
Operation similarity
OpSim(opiS , opjS ) = N A(name opiS , name opjS ) + 1
2
1
2
[0, 3] j i INS , INS 1
2
i ,IN j ) 2·Atot (INS S2 1 j i | |+|IN |IN S2 S1
+
i ,OU T j ) 2·Atot (OU TS S2 1 j i | |+|OU T |OU T S2 S1
∈
- sets of input parameter names of the i-th operation of S1 and the j-th operation of S2
OU TSi , OU TSj - sets of output parameter names of the i-th operation of S1 and the j-th operation of 1 2 S2 Functionality-based similarity
F Sim(S1 , S1 ) =
2·
j i i,j OpSim(opS1 ,opS2 ) |OP (S1 )|+|OP (S2 )|
∈ [0, 3]
OP (S1 ), OP (S2 ) - sets of operations of S1 and S2 Global similarity GSim(S1 , S2 ) = w1 · N ormESim(S1 , S2 ) + w2 · N ormF Sim(S1 , S2 ) ∈ [0, 1] w1 , w2 - weights introduced to assess the relevance of each kind of similarity (w1 ∈ [0, 1] and w2 = 1−w1 ) N ormESim(), N ormF Sim() - ESim() and F Sim() normalized to the range [0, 1]
names of their output parameters. The building block of these coefficients is the Name Affinity coefficient N A(n1 , n2 ). The arguments n1 and n2 can be either names of concepts related by sumbsumption or equivalence relations in the peer ontology, or terms in the thesaurus related to ontological concepts by means of synonymy (syn), broader/narrower term (bt/nt) and related term (nt) relationships. A weight σ ∈ (0, 1] is associated to each kind of relationship: σ = 1.0 for equivalence or synonymy, σ = 0.8 for subsumption or BT/NT relationships, σ = 0.5 for the other kinds of relationships; n1 and n2 can be related by a chain of weighted relationships. We define N A(n1 , n2 ) between n1 and n2 as the product of weights associated to that chain of relationships. Two services are similar (denoted with S1 ≈S2 ) when MatchType(S1,S2 ) is not Mismatch and Sim(S1 , S2 )≥δ, where δ is a similarity threshold experimentally set. Service categories are initially exploited to filter out not matching services: if a subsumption or an equivalence relationship, based on the SCT, is not identified
172
D. Bianchini, V. De Antonellis, and M. Melchiori
between at least one category for S1 and one category for S2 , the hybrid matchmaking model is not applied at all and the kind of match is directly set to Mismatch. A detailed presentation of the hybrid matchmaking model and related experimental results is given in [7].
5
The Collaboration Model
In this section, the dynamic collaboration process based on local interactions and the emergence of the service semantic overlay as an aggregation of local semantic agreements are described. 5.1
Emergent Semantic Overlay from Local Interactions
To establish inter-peer semantic links, we define a probe collaboration model. – A broker PX , to find brokers with semantically related services, sends a probe service request for each service SiX to be shared with other peers. The probe service request contains the interface of SiX and the IP address of PX . – A broker PY that receives the probe service request matches it against its own abstract services SjY by applying the matchmaking techniques. For each comparison the MatchType and the similarity value are computed. PY then replies to PX with the list of SjY such that SiX ≈SjY , together with the MatchType and the similarity value. – an inter-peer semantic link islPX →PY (SiX , SjY ) is established on PX for each SjY and PY is recognized as a semantic neighbor of PX . Inter-peer semantic links are labeled by the match type and the similarity degree. In other words, we can state that a local semantic agreement has been reached. Example 2. Figure 4 shows an example of a service semantic overlay as viewed by a peer PX . Peer PX recognizes PY as its semantic neighbor through a sequence of local interactions between PX and PY . In fact, to find semantic neighbors, the peer PX sends a probe service request for each SiX of its services to other peers connected through the logical network overlay, namely the P2P network connections. The probe service request contains the interface of SiX and the IP address of PX . The recipient peers match the probe service request against their own semantic service interfaces by applying the matching techniques and obtain for each comparison the MatchType and the similarity value. If the MatchType is not Mismatch and the similarity value exceeds the threshold δ, an inter-peer semantic link can be established. In the example, the peer PY finds a semantic service interface matching the request (in this example, S1Y ) and replies to PX sending only S1Y together with the MatchType and the similarity value. PX establishes an inter-peer semantic link, denoted by islPX →PY (S3X , S1Y ), and PY is recognized as a semantic neighbor of PX .
Service-Based Semantic Collaboration in Networked Systems
173
Fig. 4. A portion of service semantic overlay
5.2
Service Semantic Overlay Evolution
Semantic links are established and updated when registered peers join or leave the P2P network and when services are published/removed from peers. Peer knowledge evolution occurs at the service semantic overlay through the probe collaboration model, but it is initiated at the logical network overlay. The logical network overlay collects all the peers participating to the collaborative network. In order to guarantee up-to-date connections between peers an Overlay Management Protocol (OMP) is used, which defines specific procedures to join, leave and modify the logical overlay. Given the inner dynamism of P2P networks, in our framework a shuffling-based OMP is chosen in order to allow more effective information diffusion among peers [32]. This kind of OMP arranges the logical overlay as a graph in which each peer is directly connected to a small portion of the whole peer population, that is, its logical neighbors. The shuffling protocol is quite simple: peer PX keeps changing the set of its logical neighbors by randomly contacting one of its current neighbors; then PX and the contacted peer exchange some of their logical neighbors. The obtained result is an inexpensive overlay membership management, in the sense that any joining or leaving of peers is quickly and efficiently detected without overloading the network. As shown in the evolution statechart in Figure 5, after PX joins the collaborative network at the logical network overlay, through the probe collaboration model PX sends a probe service request to each new logical neighbor PY . If PX does not receive any reply, no inter-peer semantic links are established and PX remains connected only at the logical overlay (state 2). Otherwise, PX establishes
174
D. Bianchini, V. De Antonellis, and M. Melchiori
Fig. 5. UML statechart describing the logical network overlay and the service semantic overlay evolution
inter-peer semantic links with its semantic neighbors, to which it is connected at the service semantic overlay (state 3). The probe procedure is activated only when PX receives IP addresses of new logical neighbors through the shuffling mechanism or when a new service is supplied on PX . Probe service requests are sent according to a Time-To-Live (TTL) mechanism to avoid network overload. Collaborative peers that cannot be reached from PX , due to the TTL value, can be eventually reached thanks to the shuffling-based OMP. PX recognizes that a semantic neighbor PY is disconnected if PY has not been answering the service requests sent by PX for more than a given period of time. In this case, the inter-peer semantic links towards semantic service descriptions provided by PY are removed from PX . If PY is not available for a shorter period, the semantic links are not removed to avoid to loose this kind of information in case of temporarily disconnections. If a new semantic service description SiX is added due to the publication of a new service on PX , a probe service request is sent to semantic neighbors of PX and inter-peer semantic links are established on the basis of obtained answers. Finally, if a semantic service description is removed, the inter-peer semantic links starting from it are removed without the need of any propagation of messages on the network. In this way, the service semantic overlay evolves through on-the-fly local interactions between collaborative peers.
Service-Based Semantic Collaboration in Networked Systems
6
175
Navigating the Service Semantic Overlay for Service Discovery
Service discovery allows for finding required functionalities in the collaborative network. A service request SR is formulated by specifying one or more service categories and the desired functional interface. The request is sent to a broker PX , to which the requester is connected, where SR is processed in two steps. Step 1 - Local search. The broker PX searches for abstract services in its own registry according to the specified categories for SR and applies the hybrid matchmaking model to build a list M S(SR ) of matching semantic service descriptions. Formally: M S(SR ) = {S1X , GSim1 , mt1 , . . . SnX , GSimn , mtn }
(1)
where: GSimi = GSim(SR , SiX ) ≥ δ, mti =MatchType(SR, SiX ) ∈ {Exact, Extends, Intersects}. Step 2 - Distributed search. After performing local search on peer PX , local matching services SiX ∈ M S(SR ) are considered as starting points to select the best semantic neighbors to which the service request can be forwarded. The PX network knowledge is investigated to select the semantic neighbors that are related to services SiX ∈ M S(SR ) through inter-peer semantic links. A list SN of semantic neighbors is returned, where each sn ∈ SN is defined as SN = nsn , {S1X sn , mt1 sn , GSim1 sn , . . . SkX sn , mtk sn , GSimk sn }
(2)
where nsn is the identifier for the semantic neighbor sn (i.e., its IP address), S1X sn , . . . SkX sn are the semantic service descriptions in M S(SR ) that are sources of an inter-peer semantic link from PX towards sn, mtj sn and GSimj sn , with j = 1, . . . k, are the kind of match and the similarity degree labelling the j-th inter-peer semantic link, respectively. The neighbors which receive the forwarded request repeat the two-steps discovery process. Forwarding Policies. To forward service requests among brokers during distributed search, policies are defined for navigating the service semantic overlay, based on: – a stop condition, that establishes when to forward the request: (a) the request is always forwarded to the other brokers or (b) the request is forwarded only if particular conditions (see below) hold in M S(SR ); in any case, the forwarding policy limits the number of allowed propagations of the request by means of a TTL (Time-To-Live) mechanism; each time a broker forwards a request, the TTL is decreased by 1; when the TTL=0, the request cannot be further propagated; – a selection function, that establishes to which brokers the request must be forwarded: (a) the request is forwarded to a subset of semantic neighbors;
176
D. Bianchini, V. De Antonellis, and M. Melchiori
(b) the request is forwarded to all the semantic neighbors of the current broker; (c) the request is forwarded to a set of brokers connected at the logical network overlay. We distinguished among two policies: a minimal and an exhaustive policy. According to the minimal policy, search over the service semantic overlay stops when matching services which fully satisfy the service request have been found. Formally, if ∃SiX ∈M S(SR ) such that SiX Exact | Extends SR , it is not necessary to forward the service request to the semantic neighbors of PX , since concrete implementations of SiX already satisfy the request. Otherwise, the list SN of semantic neighbors is investigated to find remote services that potentially offer additional functionalities with respect to local services. A semantic neighbor sn ∈ SN is considered for service request forwarding if ∃SiX sn such that mti sn is not Exact. In fact, if for all SiX sn the corresponding mti sn is Exact, this means that sn does not provide services with additional functionalities with respect to those already provided by services SiX ∈ M S(SR ) on peer PX . According to the exhaustive policy, the request is always forwarded to the other collaborative peers on the network. This policy follows the same rules of the minimal one, but the search does not stop when matching services that fully satisfy the request are found: the request SR is forwarded to semantic neighbors to find other services that could present, for example, better non functional features (not discussed in this chapter). In any case, both the forwarding policies contain the TTL mechanism. We formally define a forwarding policy Ψ as a 4-uple: Ψ = SR , T T L, ϕ, {λ()}
(3)
where: (i) SR is the service request; (ii) T T L is the Time-To-Live value; (iii) ϕ is the stop condition; (iv) λ() is the selection function. The generic definition of forwarding policy Ψ can be specialized into the minimal policy Ψminimal as follows: Ψminimal = SR , T T L, ϕ0, {λ0 (), λ1 (), λ1 ()} (4) where: ϕ0 = (T T L≤0) ∨ ϕa0 ∨ ϕb0 ϕa0
= {∃SiX ∈M S(SR ) s.t. mti = (Exact|Extends)} sn ϕb0 = {∀sn∈SN, ∀Sisn X ⇒ (mti = Exact)}
(5)
The stop condition ϕ0 is the conjunction of three terms: (i) the first one depends on the TTL value; (ii) the second one, ϕa0 , depends on the results of local search, and (iii) the third one, ϕb0 , depends on the matching information labelling the inter-peer semantic links towards the semantic neighbors. Also the selection function λ() can be instantiated in different ways. If the request SR must be forwarded only to the semantic neighbors sn∈SN , we set: λ0 () = SN
(6)
Service-Based Semantic Collaboration in Networked Systems
177
If the request is forwarded to all the semantic neighbors of the current broker, independently from the results in M S(SR ), we set: λ1 () = {PY |PY ∈SN (χ)}
(7)
where SN (χ) represents the set of semantic neighbors of the current broker χ. If the request is forwarded to the brokers connected with the current one at the logical network overlay, we set: λ2 () = {PY |PY ∈LN (χ)}
(8)
where χ represents the set of logical neighbors of χ. In the definition of a forwarding policy Ψ , more selection functions can be included and executed in sequence, as shown in Ψminimal policy. If a selection function produces an empty set, the next one is considered. For example, in the Ψminimal policy, if no matching services have been found locally (and therefore λ0 () = SN = ∅), the current broker forwards SR to the set of all its semantic neighbors (obtained through λ1 () function). If also λ1 () = ∅, then the current broker forwards SR to the set of its logical neighbors (obtained through the λ2 () function). Similarly, the exhaustive policy Ψexhaustive is defined as follows: Ψexhaustive = SR , T T L, ϕ1, {λ0 (), λ1 (), λ2 ()}
(9)
where the stop condition ϕ1 is uniquely based on the TTL value: ϕ1 = (T T L≤0)
(10)
and the selection functions are the same as for the minimal policy.
7
Conclusions
In this chapter, we presented the P2P-based Semantic Driven Service Discovery (P2P-SDSD) framework to enable cooperation and communication based on a semantic overlay that organizes semantically the P2P knowledge space. The framework is constituted by: (i) a conceptual model of the semantic overlay; (ii) a set of semantic matching techniques for the creation and the evolution of the semantic overlay; (iii) a collaboration model with different policies to support interaction among peers. The service semantic overlay is built over the logical network overlay and provides semantic links among peers that offer similar services. The semantic links enable effective similarity-based service search and optimization strategies are defined for request propagation over the P2P network. From a conceptual modeling perspective, the semantic overlay can be considered as a continuously evolving conceptual representation of the distributed peer knowledge. The semantic overlay is exploited to optimize the service discovery process and improve its efficacy keeping the generated network overload low. Preliminary experiments [9] have been performed to confirm the advantages derived from the exploitation of the semantic overlay if compared with traditional non semantic approaches.
178
D. Bianchini, V. De Antonellis, and M. Melchiori
References 1. Aberer, K., Cudre-Mauroux, P., Hauswirth, M.: The Chatty Web: Emergent Semantics Through Gossiping. In: Proc. of the 12th International World Wide Web Conference (WWW 2003), Budapest, Hungary, pp. 197–206 (2003) 2. Akkiraju, R., Farrell, J., Miller, J., Nagarajan, M., Schmidt, M., Sheth, A., Verma, K.: Web Service Semantics - WSDL-S. A joint UGA-IBM Technical Note, version 1.0 (April 2005), http://lsdis.cs.uga.edu/Projects/METEOR-S/WSDL-S 3. Banaei-Kashani, F., Chen, C., Shahabi, C.: WSPDS: Web Services Peer-to-Peer Discovery Service. In: Proc. of the 5th Int. Conference on Internet Computing (IC 2004), Las Vegas, Nevada, USA, pp. 733–743 (2004) 4. Benatallah, B., Hacid, M., Paik, H.Y., Rey, C., Toumani, F.: Towards semanticdriven, flexible and scalable framework for peering and querying e-catalog communities. Information Systems 31(4-5), 266–294 (2008) 5. Bernstein, P., Giunchiglia, F., Kementsietsidis, A., Mylopoulos, J., Serafini, L., Zaihrayeu, I.: Data Management for Peer-to-Peer Computing: A Vision. In: Proc. of the 5th International Workshop on the Web and Databases (WebDB 2002), Madison, Wisconsin, pp. 177–186 (2002) 6. Bianchini, D., De Antonellis, V., Melchiori, M.: An Ontology-based Architecture for Service Discovery and Advice System. In: Proc. of the IEEE DEXA Int. Workshop on Web Semantics (WebS 2005), Copenhagen, Denmark (2005) 7. Bianchini, D., De Antonellis, V., Melchiori, M.: Flexible Semantic-based Service Matchmaking and Discovery. World Wide Web 11, 227–251 (2008) 8. Bianchini, D., Montanelli, S., Aiello, C., Baldoni, R., Bolchini, C., Bonomi, S., Castano, S., Catarci, T., De Antonellis, V., Ferrara, A., Melchiori, M., Quintarelli, E., Scannapieco, M., Schreiber, F.A., Tanca, L.: Emergent semantics and cooperation in multi-knowledge communities: the esteem approach. World Wide Web 13, 3–31 (2010) 9. Bianchini, D., De Antonellis, V., Melchiori, M.: Service-based semantic search in p2p systems. In: IEEE Computer Society 7th European Conference on Web Services (ECOWS 2009), Eindhoven, The Netherlands, pp. 7–16 (2009) 10. Bussler, C., de Bruijn, J., Feier, C., Fensel, D., Keller, U., Lara, R., Lausen, H., Polleres, A., Roman, D., Stollberg, M.: Web Service Modeling Ontology. Applied Ontology 1, 77–106 (2005) 11. Castano, S., Ferrara, A., Montanelli, S.: Matching ontologies in open networked systems: techniques and applications. Journal on Data Semantics 5, 25–63 (2006) 12. Choi, N., Song, I., Han, H.: A survey on ontology mapping. SIGMOD Record 35(3), 34–41 (2006) 13. The OWL-S Coalition: OWL-S 1.1 release (November 2004) http://www.daml.org/services/owl-s/1.1/ 14. Doan, A., Madhavan, J., Domingos, P., Halevy, A.: Learning to map between ontologies on the semantic web. In: The Eleventh International WWW Conference (2002) 15. Ehrig, M., Staab, S.: QOM – quick ontology mapping. In: McIlraith, S.A., Plexousakis, D., van Harmelen, F. (eds.) ISWC 2004. LNCS, vol. 3298, pp. 683–697. Springer, Heidelberg (2004) 16. The OpenGALEN Foundation, http://www.opengalen.org/ 17. Gangemi, A., Guarino, N., Masolo, C., Oltramari, A.: Sweetening WordNet with DOLCE. AI Magazine 24(3), 13–24 (2003)
Service-Based Semantic Collaboration in Networked Systems
179
18. Giunchiglia, F., Shvaiko, P., Yatskevich, M.: S-match: an algorithm and an implementation of semantic matching. In: European Conference on Semantic Web (ESWC 2004) (2004) 19. Gligorov, R., Aleksovski, Z., ten Kate, W., van Harmelen, F.: Using Google distance to weight approximate ontology matches. In: Seventeenth World Wide Web Conference (WWW 2007) (2007) 20. Halevy, A., Ives, Z., Suciu, D., Tatarinov, I.: Schema Mediation in Peer Data Management Systems. In: Proc. of the 19th International Conference on Data Engineering (ICDE 2003), Bangalore, India (2003) 21. Horrocks, I., Li, L.: A Software Framework for Matchmaking Based on Semantic Web Technology. In: Wellman, M., Riedl, J. (eds.) Int. Journal of Electronic Commerce (IJEC), Special Issue on Semantic Web Services and Their Role in Enterprise Application Integration and E-Commerce (2004) 22. Keller, U., Lara, R., Lausen, H., Polleres, A., Fensel, D.: Automatic location of services. In: G´ omez-P´erez, A., Euzenat, J. (eds.) ESWC 2005. LNCS, vol. 3532, pp. 1–16. Springer, Heidelberg (2005) 23. Klusch, M., Kaufer, F.: Wsmo-mx: A hybrid semantic web service matchmaker. In: Web Intelligence and Agent Systems, vol. 7, pp. 23–42 (2009) 24. Lausen, H., de Bruijn, J., Polleres, A., Fensel, D.: WSML - a language framework for semantic web services. Position Paper for the W3C Rules Workshop, Washington DC, USA (April 2005) 25. Medical Subject Headings, http://www.nlm.nih.gov/mesh/ 26. Niles, I., Pease, A.: Towards a standard upper ontology. In: The 2nd International Conference on Formal Ontology in Information Systems (FOIS 2001) (2001) 27. Noy, N., Musen, M.: The PROMPT suite: interactive tools for ontology merging and mapping. International Journal of Human-Computer Studies 59(6), 983–1024 (2003) 28. Shvaiko, P., Euzenat, J.: A survey of schema-based matching approaches. Journal on Data Semantics 4, 146–171 (2005) 29. SNOMED CT: Systematized Nomenclature of Medicine-Clinical Terms, http://www.ihtsdo.org/snomed-ct/ 30. The ESTEEM Team: Emergent Semantics and Cooperation in Multi-Knowledge Environments: the ESTEEM Architecture. In: VLDB Int. Workshop on Semantic Data and Service Integration (SDSI, Vienna, Austria, 1–12 (2007) 31. Verma, K., Sivashanmugam, K., Sheth, A., Patil, A., Oundhakar, S., Miller, J.: METEOR-S WSDI: A Scalable Infrastructure of Registries for Semantic Publication and Discovery of Web Services. Journal of Information Technology and Management, Special Issue on Universal Global Integration 6(1), 17–39 (2005) 32. Voulgaris, S., Gavidia, D., van Steen, M.: CYCLON: Inexpensive Membership Management for Unstructured P2P Overlays. Journal of Network and Systems Management 13(2), 197–217 (2005) 33. Zhang, S., Bodenreider, O.: Alignment of multiple ontologies of anatomy: deriving indirect mappings from direct mappings to a reference. In: AMIA Annual Symposium (2005)
Conceptual Modeling Approaches for Dynamic Web Service Composition Georg Grossmann, Rajesh Thiagarajan, Michael Schrefl, and Markus Stumptner School of Computer and Information Science, University of South Australia, Mawson Lakes, SA 5095, Adelaide, Australia {firstname.lastname}@unisa.edu.au
Abstract. Service composition is a recent field that has seen a flurry of different approaches proposed towards the goal of flexible distributed heterogeneous interoperation of software systems. Usually they are based on the expectation that such systems must be derived from higher level models rather than be coded at low level. We survey the state-of-the-art of techniques for conceptual modeling of Web service composition from a broad, multi-field perspective that captures approaches from classical structure-oriented models over workflow languages to planning-based approaches. We describe how the related fields of model-driven development, conceptual modeling of business processes and workflows, semantic process descriptions through ontology, and service matching through constraint satisfaction can be utilized in a complementary way to support dynamic (i.e., runtime), instance-based selection and composition of Web services. Further we present an overview and a comparison of existing approaches for dynamic service composition.
1
Introduction
The dynamic interaction of the software systems that support modern business processes is among the most important challenges faced by organizations today. A key issue faced by the developers in such situations has always been that, inevitably, separately developed systems rely on disparate models. No matter how flexible the infrastructure is, it requires major effort to establish interoperability via the mutual adaptation of coexisting business processes. For example, the integration of legacy applications, or the incorporation of local “grassroots” solutions in individual branches of enterprises that often have thousands of applications and data repositories spread across the organization. In most cases nowadays, this integration task is addressed by manual analysis and laborious development of specific integration solutions. This problem, traditionally experienced in the context of interaction and distributed database systems, has become particularly poignant in recent times
This research was partially supported by the Australian Research Council (ARC) under grant DP0988961.
Kaschek/Delcambre (Eds.): The Evolution of Conceptual Modeling, LNCS 6520, pp. 180–204, 2011. c Springer-Verlag Berlin Heidelberg 2011
Conceptual Modeling Approaches for Dynamic Web Service Composition
181
with the explosive development of new styles of infrastructure that support significantly more flexible and dynamic interactions between applications. The socalled Service Oriented Computing (SOC) paradigm uses the idea of flexibly (possibly dynamically) assembling individual application components (possibly directly accessible via the Web) into complex processes. The underlying technology on which SOC is often assumed to be based is the technology of Web Services, using Web-based communication protocols and XML-based data formats and communication interface definitions, all of which are enjoying increasingly widespread usage. However, fully realizing the SOC dream requires overcoming the interoperability problem. In a recent issue of IEEE Computer [39] infrastructure support to integrate service-based applications into processes and to integrate processes with other processes has been identified as an open and important research issue to move a major step forward in SOC. The context of this article is the interoperation of several service processes that are dynamically composed into a task workflow to achieve a certain business goal. Semantic interoperation is the process of bridging and reconciling heterogeneities for combining a set of service processes, selected based on their capabilities, to meet a goal that in general cannot be met by using a single service. The composition of the services of the combined service processes needs to be planned and the composed services needs to be enacted. This is described by a workflow of itself, the task workflow [20]. Traditionally, application integration has focused specifically on the integration of data structures, reaching its culmination in the development of ontologybased methods [37]. However, it is now generally acknowledged that data integration alone is insufficient and representation of behavior is necessary [35,36]. Moreover, the major challenge ahead is to move beyond the composition of single services, to the integration of service processes (i.e., processes composed of individual services). Pre-planning and manual combination of service processes into complex structures is a significant bottleneck on the way to establish dynamic interactions and interoperability, an ability which is one of the main perceived advantages of the service-oriented approach over other paradigms. The goal of providing automated support for Web service discovery and composition has led to the development of advanced service description approaches, employing languages which permit defining not just inputs and outputs but also parts of the semantics of the service. In practice, however, even apparent front-runners can prove to be difficult to apply [42,6] and insufficient for describing certain complex functionality [17]. A fundamental decision concerns the selection of the underlying interoperability architecture and knowledge representation of service processes and the composite task workflow. The representation must be rich enough to capture the semantics of service process profiles in order to facilitate the detection and bridging of semantic heterogeneities, abstracting from syntactic details of Web service descriptions. The architecture should support different target
182
G. Grossmann et al.
implementations. An approach that was developed with this goal in mind is the so-called Model Driven Architecture (MDA). It offers a layered architecture that permits modeling of system artifacts in a platform independent fashion, with the goal of automatically mapping them later to a specific implementation platform. MDA representations typically (though not always) use variants or subsets of UML. UML has been used for modeling service workflows, e.g., [20], but again only at the structural level. UML does provide more powerful means of expression, such as the Object Constraint Language OCL. Example 1. We use the following business trip booking scenario as a motivating example to demonstrate strengths and weaknesses of existing composition approaches: r1 : Make a round trip airline booking from Adelaide to Vienna on either one of the airlines Deepblue or PurpleRed, with preferred departure date 4th of July and preferred return date 14th of July, and make an accompanying hotel reservation with StarHotels. r2 : Spend a maximum of 2, 800 AUD for the airline ticket. r3 : Select a hotel rated as 3-stars or higher, spend a maximum of 1, 900 for accommodation. r4 : The overall trip costs should not exceed 4, 600. If overall costs can be reduced by more than 10% percent, the travel itinerary may deviate from the preferred dates, with earliest departure 2nd of July, latest departure 6th of July, earliest return 12th of July, latest return 18th of July, and a minimum stay of 10 days and maximum stay of 14 days. In this article we describe a typical development life-cycle for dynamic service composition in Section 2 and classify related work according to the life-cycle steps in Section 3. In Section 4 we identify requirements for conceptual modeling approaches for the support of the life-cycle steps and evaluate existing work according to them. Based on the evaluation we identified future challenges which will be summarized in Section 5.
2
A Dynamic Service Composition Life-Cycle
Semantic Web service composition is the process of combining a number of Web services, selected based on their capabilities, to meet a goal that in general cannot be met by using a single service. Web service compositions are specialized business processes where every activity or operation in the process is carried out by a Web service. Previous works have examined a number of traditional paradigms such as planning [28,49,44], ontology-based matchmaking [34,18,29] and constraint-based systems [3,22] to automate various parts of the composition task. Alternatively, some approaches such as [20,55] have investigated the effectiveness of using MDA-based technologies to accomplish semantic Web tasks, e.g., a model-driven service composition approach was investigated in [20]. In this section we describe a life-cycle for the dynamic composition of Web services
ENACTMENT
CONFIGURATION
DESIGN
Conceptual Modeling Approaches for Dynamic Web Service Composition
Modelling Behaviour
183
Modelling Data
Modelling a task workflow with abstract activities
Add or change functional and non-functional requirements Re-design if no service candidate can be found given requirements
Configuration of Instances - Discovery of services instances - Service instance composition - Service instance selection
Configuration of Services - Discovery of service candidates - Service composition - Service mediation
Re-configuration: Search for alternative candidate if no candidate can be found or suddenly unavailable
Execution
Binding - Bind input values from user - Bind task workflow activities - Bind query tasks
Execution and monitoring of task workflow
Re-binding: New user requirements if no instance can be found
Fig. 1. Life-cycle for the dynamic composition of services
and use it as a starting point to classify different conceptual approaches and identify requirements on the application of conceptual models. The life-cycle for the dynamic composition of services consists of 7 steps which are performed iteratively. Figure 1 illustrates the flow between the steps which are performed in three phases, an design, configuration and enactment phase. The steps in the design phase consists of modeling behavior and data requirements of services using conceptual models. On the service provider side, Web services interface and behavior are modeled and on the service consumer side, the user requirements are specified by an abstract service model. The following configuration phase deals with the discovery, composition, selection and mediation of service candidates that confirm with the user requirements. In the third phase, the enactment phase, an execution engine binds the abstract service model with selected services, executes the service calls and monitors the calls. During execution, it handles compensation and replacement of failed service calls and performs auditing for quality assurance. In the remainder of this section we apply the presented service composition life cycle on the previously introduced Example 1. Outcome of the life cycle is the task workflow shown in Figure 3 that explicitly models the dynamic aspects of service and instance configuration. In the following sections we discuss the three phases in the life-cycle in more detail and illustrate them with the example introduced beforehand. We assume the availability of certain functionality, e.g., possibility to query service descriptions and query availability of service instances on the service provider side.
184
2.1
G. Grossmann et al.
Design Phase
The design phase consists of modeling and deployment of Web service specifications on the service provider side and modeling of requirement specifications on the consumer side. Both can be categorized into the specification of behavior and data, and their semantics. The examples below contain only the behavior description for simplicity. Modeling and Deployment of Web Services: For each offered service process its provider publishes a service process description, describing its properties by states and constraints, and its behavior by states and actions. Furthermore, it specifies a set of parametrized queries, referred to as inquires, that select all or a subset of the service instances available and that retrieve the current state of a specific service instance. Example 2. Let us consider the service process descriptions of two airlines, DeepBlue and PurpleRed. For simplicity, we assume that our two airlines offer only return tickets and quote only the cheapest price for each itinerary. For brevity, we also consider a simplified service process. Potential service instances (un-booked flight offers) of both airlines are identified by attributes fltNo, departureDate, retFltNo, retDate, and are described by the ticket price. A reserved or booked itinerary is identified by a reservationNo that is unique per airline. The reservation number is returned by the first action in a service process e.g, reserve with DeepBlue or book&pay with PurpleRed. Both airlines offer two inquires: obtainQuote (from, to, depDate, retDate), which return a set of quotes for the specified itinerary, and checkStatus(reservationNo) which returns for the current status of a reservation. Figures 2(a) and 2(b) depict the service process descriptions of airlines DeepBlue and PurpleRed, respectively. The figures show actions and associated state transitions. While airline DeepBlue offers the possibility to make reservations (reserve) and to cancel or confirm the reservation later, which results in a booking that is paid (pay) and used (checkIn), airline PurpleRed offers only instant ticket sales (book&pay). Once a ticket is sold it may be used (checkIn) or canceled (cancel). Canceled tickets may be entitled to a refund, subject to a deducted cancellation fee (collectRefund). The life cycle of the service process of our hotel group StarHotels consists of a single action only, makeReservation(quoteNo). The service provider StarHotels offers one inquiry service obtainQuote (fromDate, toDate) returning a set of quotes for the indicated period, each one identified by a unique quoteNo and described by the hotel name and dailyRate. While flight quotes will be in Australian Dollar, hotel quotes will be in Euro. Activities of both life cycles will be referenced consistently to their execution sequence in the task workflow shown in Figure 3. Modeling User Requirements: On the service consumer side, a process designer specifies the user requirements in a so-called task workflow. This workflow is used as the basis for the subsequent service composition steps and models the
Conceptual Modeling Approaches for Dynamic Web Service Composition
185
(a) Deepblue Airline Service Process (simplified)
(b) PurpleRed Airline Service Process (simplified)
Fig. 2. Service processes of user preferred airlines
life-cycle of the dynamic service composition explicitly from the user point of view. It may be extended or altered during the configuration or reconfiguration phase in case unexpected changes occur the workflow needs to be adapted to them in order to reach a goal. Two steps are included in the phase, which are modeling behavior and modeling data requirements. Example 3. The task workflow has two abstract service objects, flightItinerary and hotelBooking. Abstract service object flightItinerary may be bound to a service instance of either DeepBlue or PurpleRed, and abstract service object hotelBooking to a service instance of StarHotels. Each instance of the task workflow is described by the following attributes: fromCity, toCity, preferredDepDate, preferredRetDate, maxAirFare, minStars, maxHotelCosts, maxOverallCosts, earliestDep, latestDep, earliestReturn, latestReturn, miniumStay, maximumStay, deviateThreshold. The attributes are self explanatory, except deviateThreshold which indicates the percentage in savings which justify choosing an
alternative itinerary than that of the preferred dates. The binding of the abstract service objects to service instances is constrained by the collected user requirements. Fig. 3 depicts the life cycle of the task workflow of our business trip. An instance of the task workflow is created by a control activity collectTravelReq, which collects the travel requirements from the user and initializes the attributes of the workflow instance accordingly. Thereafter, three inquiry activities, obtainQuotes DB, obtainQuotes PR, obtainQuotes SH, to obtain quotes from service providers DeepBlue, PurpleRed and StarHotels, respectively, may be executed in any order. As quotes from DeepBlue are in Euro, they are converted by a subsequent bridging activity convertCurrency into Australian Dollar. Once these activities have been performed, the quotes are matched and selected by control activity composeTravelPlan. If a flight itinerary and a hotel quote could be found that meet the specified user requirements, the workflow can proceed with respective bookings. If the offers cannot meet the user requirements, the workflow is continued by following the k.o. link and continued with a control activity that recollects changed user requirements reCollectTravelReq. For better readability of the Figure, only states to which we explicitly refer to in our examples are named.
186
G. Grossmann et al.
The types of activities used in the example are control bridging, service usage, and inquiry tasks, and are listed at the bottom of Fig. 3. Also listed are the two types of special arcs used (k.o. and o.k. links). Control tasks are internal activities which collect information and execute the configuration algorithm. Bridging tasks deal with syntactical and semantical heterogeneities between two activities and are discussed in Section 2.2 below, service usage tasks represent calls of external services that are provided by partner services, and inquiry tasks query partners for the availability of service instances with certain properties. If a task is executed successfully then the orchestrator instance follows the o.k. link. If an exception occurs, e.g., due to unavailability of a service or service instances, then the instance follows the k.o. link.
""#
! !
!
%
$ !
! $
""#
Fig. 3. Task Workflow: Business Trip
Modeling Semantics: The specification of the Web service semantics is crucial for the configuration and enactment phase and needs to be formal in order to achieve automation support for service integration and execution. Semantics need to be defined for behavior and data. We concentrate on behavior in this section. We assume Petri net-like execution semantics for the service description where activities are mapped to transitions and pre- and postconditions to states. Petri nets are a well-established formalism and their simple execution semantics can be applied for the majority of process models and Web service orchestration specifications. The notion of workflow state is important to formally capture the current processing state of a workflow instance (which is referred to as life cycle state). A workflow state represents the current state of a workflow instance which may include multiple states and activities. As the outcome of activities in the workflow execution is not known in advance, e.g., the booking of a quoted hotel room may be successful or not, the different steps of the execution may be “guarded” which is expressed by a life cycle state of the workflow instance. When the modeling of available services on the provider side and abstract service requirements on the consumer side is finished, the life-cycle enters the configuration phase.
Conceptual Modeling Approaches for Dynamic Web Service Composition
2.2
187
Configuration Phase
The configuration phase deals with discovery, composition, selection, and mediation of services. These tasks need be performed on the schema as well as on the instance level which is represented as configuration of services and configuration of instances in Figure 1. In the following subsections we explain the tasks in more detail. Service Discovery: Discovery is the process of identifying service candidates and is divided in discovery on the schema and discovery on the instance level. On the schema level, the discovery step includes retrieving information from a service registry by searching service types and receiving service interfaces specifications which include specifications about behavior, functional and meta data describing data types and additional semantics. The service type request specifies the domain, e.g., “flight services” and may specify further details like “flight services between certain destinations”. In a subsequent matching step applied on the service registry, potential service candidates are queried. Matching includes matching of service type, behavior and functional requirements, and data semantics. Often requirements of the task workflow will not match perfectly with the descriptions of available services. In some cases it might be possible to overcome differences with bridging tasks that mediate between behavioral and/or semantical heterogeneous services as discussed below. If the matchmaking step identifies compatible services either with or without mediation then those services are returned as potential service candidates. Example 4. Service discovery on the schema level is not modeled explicitly in the task workflow shown in Figure 3 because we assume here that the user knows a priori that he wants to fly only with DeepBlue or PurpleRed and stay in a hotel offered by StarHotels and he knows that those services are available for the required travel destinations. The behavior of the task workflow is consistent with the behavior of the preferred services and therefore they are potential service candidates. However, the data semantics do not match entirely and need to be mediated as demonstrated in the example below. On the instance level, the discovery step retrieves information about available service instances from the service provider through the service interfaces returned by the discovery on the schema level. A matchmaking step on the provider side selects a subset of available instances based on user inputs and non-functional requirements like Quality of Service (QoS). If some instance matches are identified then they are returned as potential bindings to the user. Example 5. The service instance discovery is modeled explicitly by the three ”‘obtaining quotes”’ tasks that follow collectTravelReq. They obtain potential bindings of abstract service object flightItinerary listed in table in Figure 4(a). The set of potential bindings is determined by the result of executing in our task workflow the inquiry activities obtain DB quotes (followed by bridging activity convertCurrency) and obtain PR quotes for available flights in the period between 2nd of July
188
G. Grossmann et al.
(earliest departure) and 18th of July (latest return). The table in Figure 4(b) depicts the potential bindings of abstract service object hotelBooking. The set of potential bindings is determined by the result of executing in our task workflow inquiry activities obtainQuotes SH for available hotels in the period between 2nd of July (earliest departure) and 18th of July (latest return).
(a) Potential bindings for abstract ser- (b) Abstract service object hotelBooking, vice object flightItinerary, available after available after obtaining hotel quotes. obtaining airline quotes. Fig. 4. Potential bindings for preferred airlines and hotels
Service Mediation: Service mediation overcomes behavioral and semantical heterogeneities between service specifications. It may be provided as separate services that act as a mediator in form of an adapter between different services. We call tasks that bridge the gap between heterogeneous specifications bridging tasks. Information about which bridging tasks are available is essential for the service matching step. Depending on the type of discrepancy, bridging tasks are situated on different abstraction levels. On the data level, bridging tasks translate between different data types or semantically different data values. On the protocol level, they overcome different message structures defined by protocols such as ebXML and RosettaNet, and on the process level, they overcome differences in the expected interaction sequence of messages. Example 6. There is a currency mismatch between quotes returned by discovery task obtainQuotes DB and quotes required by the composition task composeTravelPlan where the discovery task returns quotes in Euro and the composition task requires quotes in Australian Dollars. This heterogeneity is mediated by a so-called bridging task convertCurrency which is modeled explicitly in the task workflow. Service Composition: Service composition deals with the combination of discovered services to reach a user defined goal. Like service discovery, composition is also performed in two stages, first on the service candidate level and then on the service instance level. On the service candidate level, the potential combinations of service candidates can be identified by matching their requirements queried from the service registry. Example 7. In our example, the user has preferences for flight services DeepBlue and PurpleRed and hotel reservation service StarHotels. Information about flight destinations of both airlines and hotel locations of StarHotels can be queried from the registry and then matched so possible combinations can be identified.
Conceptual Modeling Approaches for Dynamic Web Service Composition
189
On the service instance level, specific service instances which were discovered by matching user requirements on the service provider side are composed to form the lowest level of service processes. Example 8. Figure 5 depicts the tentative bindings of abstract service objects FlightItinerary and hotelBooking after service matching and composition and before the respective reservations or bookings have been made.
Fig. 5. Tentative bindings of abstract flight and abstract hotel after service matching
Service Selection: The last step in the service configuration phase is the selection of a possible service composition. If there are more than one combination possible then a ranking according to QoS or user interaction is required to select one solution. In our simplified example, there is only one possible combination of services identified beforehand. Example 9. Figure 6 depicts the workflow state and execution schedule after service configuration. The workflow is in life cycle state { travelPlanComposed }, which is the post state of control activity composeTravelPlan in our task workflow (note: this state is not named for better overall readability in Fig. 3). Two unguarded activities are scheduled in accordance with the task workflow description, the reservation of the flight itinerary (activity reserve DB) with service provider DeepBlue for the tentative binding of abstract service object flightItinerary and the booking (activity book SH) of the hotel with service provider StarHotels according to the tentative binding of abstract service object hotelBooking (cf. Figure 5). The third step of the execution schedule is guarded by life cycle state {hotelBookingFailed}, the ko-post state of activity book SH. I.e., this step will be executed only if the requested hotel booking fails once a quoted offer is no longer available. The fourth state will be explained later. step# 1 2 3 4
lcs
activity p.bindings reserve DB dept: (DP1,1/Jul), ret: (DP2,11/Jul) book hotel quoteId: 21 { hbf } cancel DB { fcld } rTp
clcs = { travelPlanComposed } rTp . . . recomposeTravelPlan, hbf . . . hotelBookingFailed fcld . . . flightCanceled
Fig. 6. Workflow state with execution schedule after service matching, i.e., after execution of control activity composeTravelPlan
After potential service candidates were selected and a composition plan was chosen the life-cycle enters the enactment phase which is the operational phase between service consumer and providers.
190
G. Grossmann et al.
2.3
Enactment Phase
In the enactment phase the binding, execution and monitoring of the execution takes place. The binding comprises binding of user inputs to the task workflow and the binding of service usage tasks directly or indirectly via adapter services to service providers. The execution is performed by an engine and the monitoring is performed by “guards”. Under certain circumstances, when an action fails and steps have to be retaken, existing objects may have to be wound up, i.e., finish their lifecycle according to their current condition, as part of the compensation activities for the problem. Example 10. Figure 7 depicts the execution stage of our workflow instance after the tentative hotel booking at UniLodge hotel from 2/Jul till 12/Jul (cf. Fig. 5) according to quote no. 21 failed due to the quoted offer no longer being available. The workflow instance has entered the k.o. post state flight cld (flight canceled) of activity book hotel. The workflow has entered the compensation phase. The flight reserved with airline DeepBlue needs to be canceled (activity cancel DB of step 1) and the travel plan needs to be re-composed. This re-tasking step is represented by control activity recomposeTravelPlan that is scheduled as step no. 2 in the workflow execution schedule after the flight has bee canceled. This control activity will invoke another match-making process and, thereupon, define another work flow execution schedule and enter state travelPlanComposed. If this match-making process fails, due to no flight and hotel quotes matching the user requirements, the work flow continues with another control activity, recollectTravelReq, in which the user changes his or her travel requirements or abandons the business trip altogether. step# lcs activity p.bindings 1 { hb failed } cancel DB quoteId: 21 2 { flight cld } rTp
clcs = { hb failed }
Fig. 7. Execution stage during compensation phase after tentative hotel booking found unavailable
In some cases, it may be useful to re-task a workflow, i.e., to determine new bindings for abstract service objects, while the compensation phase is still incomplete. In such a case, a wind-up copy of the task workflow instance is created that is a concurrently existing to the current workflow version of the process. The wind-up workflow may maintain a separate branch until it has reached the end of its lifecycle. A wind-up area of the task workflow is identified by a set of activities and states in the workflow description. The wind-up area is part of a task workflow that deals with the remaining execution of tasks referring to discarded bindings. It ensures that the task workflow finishes the interaction with partners properly. Example 11. In our business trip workflow, service usage activity collectRefund PR together with its prestate and its poststate constitutes a wind-up area.
Conceptual Modeling Approaches for Dynamic Web Service Composition
191
Notice that this area is not explicitly emphasized as such in Fig. 3. The wind-up process maintains the binding of abstract service object flightItinerary such that a refund can be obtained for the canceled flight booking (service usage activity collectRefund PR). Concurrently, the proper workflow process can be continued by recomposing the travel plan (control activity collectRefund PR) and, thus, associating another flight itinerary (service process instance) to the abstract service object flightItinerary. The next section presents a survey of existing approaches. They can be classified according to the abstraction levels of a standard model-driven approach such as the Model Driven Architecture. Conceptual approaches using notations such as UML and BPMN are generally assigned to the PIM level and XML-based specifications such as BPEL and SAWSDL are assigned to the PSM level.
3
Related Work
We begin with discussing earlier survey work. Then related work with a focus on a model-driven approach for the first two phases of the presented life-cycle is discussed. Within the design and configuration phase we focus on work in modeling orchestration and choreography, service descriptions, discovery, mediation, matching, and composition. 3.1
Surveys on Web Service Compositions
Automation of Web service composition has been studied in [46,40,15]. The surveys of Srivastava et al. [46] and Rao et al. [40] investigated a model-driven approach using workflow-based composition in combination with AI planning techniques. Srivastava et al. compared solutions using industry standards like the Web Service Description Language (WSDL) and Business Process Execution Language (BPEL) with solutions from the Semantic Web community using Resource Description Framework (RDF) and OWL-S with situation calculus. They show that available standards focus on the runtime perspective and their flexibility is limited due to the binding details at runtime. The planning approaches in the Semantic Web area focus on process-centric description of services and offer more flexibility because different choices of services are considered. Despite the advantage of planning approaches over industry standards, there are still many challenges to solve, e.g., the mapping of complex process languages to AI planning approaches, or the support of a dynamic search space where new objects are created during runtime. Rao et al. categorize approaches into workflow composition and AI planning approaches. Similar to Dustdar et al. [15], the authors distinguish between static and dynamic composition of Web services. A static approach means that an abstract process model is created first before the composition planning starts. On the other hand, the dynamic approach creates a process model and selects services automatically. This requires the specification of constraints.
192
G. Grossmann et al.
Two development life-cycles similar to the one shown Fig. 1 can be found in [47,27]. Kuropka et al. [27, p.16] describe a service delivery cycle and proposes the use of AI planning techniques for composition. This approach considers replanning but does not distinguish between planning on the schema and instance level. The life-cycle presented in [47] originates from the Semantic Web area and does not incorporate model-driven approaches. 3.2
Orchestration and Choreography
Two interaction types are mentioned in the Web service stack, orchestration and choreography. This distinction also exists in the related area of Business Process Management [53, Ch.4-5]: Process orchestration specifies the activities that are performed within a single organization, or in context of Web services, performed by a single Web service. This means it specifies the internal behavior of a Web service and its interaction with the outside world. Process choreography specifies dependencies between various process orchestrations and can therefore be regarded as the global view of a collaboration network. Weske et al. gives an overview of interaction patterns that can be applied in the design of a collaboration [7]. For supporting dynamic service composition, an approach must incorporates both interaction types. An orchestration interaction is necessary for the modeling of user requirements or task workflow and a choreography is necessary to specify interaction between task workflow and multiple external services where a service may be specified by another orchestration. Furthermore an interaction between external service may optimize the composition. For example, if a flight is delayed then the hotel is informed automatically so the hotel pick-up service can be changed accordingly. 3.3
Service Description Approaches
The service description is the base of any service-oriented paradigm. Each task that needs to be performed during the composition of Web services depends on the description of services because they contain all available information. Since Web services became an accepted standard in industry and research the development of adequate descriptions has progressed significantly and lead to further standardization efforts [5]. Apart from the interface description in Web Service Description Language (WSDL), additional specifications for behavior using BPEL and semantics becomes increasingly important for the automation of composition. Here, we focus on the semantic annotation of services. Description of Semantics: Describing the behavior was an important enrichment of service descriptions but the adequate description of underlying service semantics beyond the abstract behavior models is still a challenging problem. For automated discovery of services it is important that a search engine can retrieve the semantics of input and output parameters and is able to perform reasoning on it. Ontologies are currently mentioned often as a possible solution for those capabilities. They are a shared conceptualization of entities, written in a logic based representation language, and come with an executable calculus enabling
Conceptual Modeling Approaches for Dynamic Web Service Composition
193
querying and reasoning at run time [27]. So far there exists no agreed-upon standard ontology but research efforts share some commonalities apart from its definition: They are metadata based, use XML as serialization format, provide a query language [11,24] and provide high level concepts for their design. Most of the conceptual models for designing ontologies are graph based or use UML class diagrams [1,27]. Currently, some effort is invested in providing semantic annotations by extensions of existing standards. The most prominent candidates are WSDL-S, SAWSDL, OWL-S, and WSMO. WSDL-S is an extension of WSDL with elements to refer to semantic models defined elsewhere. Since web service developers are already familiar with WSDL, the developers of WSDL-S argue that simple extensions to WSDL to incorporate references to remote semantic models is more amenable to developers instead of comprehensive solutions such as OWL-S or WSMO. The minor extensions to WSDL can be relatively quickly incorporated into existing WSDL tools. SAWSDL is the latest WSDL-S specification that extends WSDL and XML Schema with provisions for semantic annotations. WSDL-S does not restrict the choice of languages that are used to develop semantic models because WSDL-S documents merely refer to the models. The choice of formalism is left to the developer. While flexible, the use of external (possibly different) formal languages brings in diversity, which obstructs seamless integration of services, especially in open infrastructures such as the web. OWL-S is a semantic web service description language that borrows ontological operators from OWL and provides a service upper ontology to describe web services and service processes in a standard manner. A service description is defined in three parts namely: service profile, service process and service grounding. The profile of a service is used to advertise the functionality that is offered. When applied to practical application scenarios, direct use of OWL-S is still subject to a number of restrictions due to the in-progress nature of the approach [6], with issues arising during profile matchmaking and process execution. As a result, approaches that offer service specification constructs combined with formal semantics are gaining prominence. Particularly, the Web Service Modeling Ontology (WSMO) and the combination of Unified Modeling Language (UML) and Object Constraints Language (OCL) are viewed as promising alternatives to OWL-S. Web Service Modeling Ontology (WSMO) [41] is a comprehensive semantic service description framework that is proposed as an alternative to OWL-S. WSMO comprises of four top level elements namely Ontologies, Web services, Goals and Mediators. WSMO, like OWL, is also based on First Order Logic but employs an object-oriented style of modeling based on frames. WSMO can be seen as a meta-level definition that specifies how services and associated knowledge have to be described. Any concrete WSMO compliant language can be used to physically define WSMO elements. WSMO prescribes the Web Service Modeling Language (WSML) [41] as a suitable candidate to actually represent WSMO elements. Unlike OWL-S, WSMO includes service and ontology specification
194
G. Grossmann et al.
formalisms with well-defined formal semantics, whereas OWL-S relies on OWL for ontology specifications and on languages such as SWRL and KIF for formal semantics. The next subsections deal with service discovery and matching, mediation and composition which are performed in the configuration phase. 3.4
Service Discovery
The improvement of service discovery went hand in hand with the development of service descriptions. Keyword-Based Search: In the beginning, the search for a particular service was simple and used only keyword-based search and data type comparisons applied on the available interface descriptions in a directory. Such directories are written in a standard language like the Universal Description Discovery and Integration (UDDI) language and provide an API for publishing and discovering information about services. Simple keyword-based search and syntactical comparison were not sufficient because underlying semantics were ignored and result often in false positives [38]. Domain specific frameworks like RosettaNet and ebXML tried to overcome this limitation by introducing a categorization of keywords with more detailed information about services. These efforts were a first step towards an ontology-driven discovery but their handicap lay in proprietary and domain specific coordination protocols. With the introduction of richer descriptions which includes behavioral descriptions and ontologies, more sophisticated types of discovery were considered in [24]: Subsumption-based discovery allows to compare hierarchical concepts of data types as typically used in an ontology, profile covering allows the reasoning on input and output matches, state-based discovery takes into preconditions, effects, and global requirements, and process-based querying queries a process ontology for covering behavioral descriptions. IO-Based Matchmaking: The Input and Output (IO) profile of a web service describes the properties of its interface, such as name, type and optionally a description of the parameters. IO-profile matchmaking has been addressed by Information Retrieval (IR) methods and type subsumption matching. Generally IR methods measure similarity between entities to retrieve suitable information. Similarity denotes the relatedness of two entities. A number of generic similarity measurement metrics such as the cosine similarity measure [30,12], Dice’s coefficient [50] and Jaccard’s index [21] have been defined to compute this similarity between entities in IR frameworks. Type subsumption approaches [25,20] select matches by determining the relationship between the IO profile of a request and the IO profile of a service. Logic-Based Matchmaking: In order to overcome the limitations of IO-based matchmaking, rigorous approaches based on logical reasoning are being proposed. In such a scheme every service is associated with a semantic service description of its functionality. A matchmaking request is also expressed as a
Conceptual Modeling Approaches for Dynamic Web Service Composition
195
semantic description of the requirements that the target service should satisfy. Typically these schemes restrict themselves to the family of languages known as Description Logics [52,51]. Since description-logic reasoning at the instance-level is inefficient [29], most such approaches map instance-level details of a service within its corresponding conceptual-level. These approaches perform matchmaking by computing the relationship between a conceptual service profile and a matchmaking request. The result of the conceptual matching process heavily depends on the level of abstraction that is used to write profiles and queries. Different notions of service matches have been proposed to partially overcome some of the difficulties [29]. A pure conceptual matchmaking approach is not scalable because creating concepts for every instance-level detail is impractical and unrealistic [19]. Therefore, conceptual matchmaking approaches remain unsatisfactory to a certain degree due to incompleteness and abstraction in service profiles. For example, that a service advertises the capability to sell books does not necessarily imply that it will be able to supply all book titles. Similarly, the same profile may or may not match a request for the more specific concept novel, depending on the matching algorithm used. System Design Languages: Constraint-based systems, which have been successfully applied to a number of large-scale industrial problems, are potentially useful in the Web service composition context. Previous works, in particular [3,22], have examined the feasibility of using constraint-based systems to address the Web service composition problem. While [3] provided a constraint-based workflow design based on input and output compatibilities, [22] provided an optimized approach to carry out simple checks on Web service attributes. The need for a matchmaking process, that filters Web service candidates based on the functionality a Web service offers, for use within semantic Web based tools has already been established. However, the previous constraint-based proposals do not consider semantics such as functionality of a Web service while performing candidate selection. While the replacement of abstract service specifications by concrete services in a model-driven approach offers a flexible design process, the approach is effective only if all the abstract service specifications are replaced (after discovery) with services that provide the required functionality. The composition process in [20] supports matchmaking based only on input and output relationships (termed syntactic matching). The matchmaking technique is likely to return a number of unintuitive false positives. For example, a request for a book inquiry service, which takes in a book ID (integer) and return availability information (string) will be matched with all the services that have integer input and string output irrespective of the functionality of the matched service. Semantic matchmaking is the process of discovering services based on the functionality they offer. The ineffectiveness of the syntactic matchmaking approaches led to the search for semantic matchmaking techniques. OCL was originally introduced as a means to represent additional semantics that are not easily captured through the various diagram notations in UML.
196
G. Grossmann et al.
OCL is a promising alternative to the traditional semantic Web formalisms for representing declarative service semantics such as service functionality [48]. 3.5
Service Mediation
As stated by Kashyap et al., Web service discovery and composition in the past often assumed the use of a common conceptual model for defining data, processes and protocols [24]. It is unlikely that all participants agree on a common representation and therefore the interoperability issues need to be bridged. One solution to this problem is ontology alignment with its origin in schema mapping as proposed by WSMO [41]. This may overcome syntactical differences such as different conceptual models and integration of heterogeneous data structures but it will not resolve heterogeneous behavior of services such as non-matching order of operations. Behaviour-based integration of services came up only recently because it requires a more complex approach compared to the mediation of static data. A common concept for behavior-based integration are re-usable behavior patterns which are applicable in different scenarios [7,14]. A grounding on a common “Modeling Space” is proposed in [13] and contextspecific algorithms are used in [16]. 3.6
Service Composition
The surveys of existing service composition techniques presented in Section 3.1 acknowledged the existence of two styles of solution to the composition problem: static and dynamic composition. Here we discuss only approaches for dynamic composition of services. Many dynamic composition approaches aim to compose services with little or no information about the target assembly. Dynamic composition is usually performed by assembling existing services together by exploiting details present in a goal and formal description of individual services. Relevant dynamic composition approaches are reviewed in this section. We focus here on two categories, planning based approaches and constraint-based approaches. In AI planning, an initial- and final state are provided along with constraints on actions that are available to agents. SHOP2 (HTN Planning): In Hierarchical Task Network (HTN) an abstract plan of the major tasks is computed first and these major tasks are decomposed into simpler tasks that can be performed by simple actions. The domainindependent HTN planner SHOP2 was used in [54,44]. Maestro: SAP Research extended the BPMN business process modeling tool Meastro with a planning-based composition module in [10]. It uses a planning scheme called FF planning presented in [23,43], where plans are synthesized through forward search guided by a heuristic-function. Haley (Decision-Theoretic Planning): In [56], a Markov Decision Process (MDP) is used to synthesize an optimal assembly of services that satisfy a goal.
Conceptual Modeling Approaches for Dynamic Web Service Composition
197
The MDP abstracts the classical planning problem to a problem where the effects of actions may be uncertain. Each plan branch is associated with a cost computed from the non-functional Quality-of-Service (QoS) attributes advertised by the candidate services. Graph Planning using Capability Semantic Structures(CSS): A composition approach that combines planning and data adaptation is presented in [8]. It adopts a HTN-based graph planning system that starts with the root node (the goal) and decomposes it into child nodes, where services are selected from Capability Semantic Structures (CSS) which groups services from a particular domain into a tree-like structure. This approach examines only input and output parameters of a service to assess it suitability. State Transition Systems (STS): A web service composition approach that combines behavior specifications of candidate services and service process is presented in [9,31]. This approach is based on the principles of State Transition Systems (STS) and planning as model-checking. Planning with Abstraction (ABS): The composition framework in [26] works by abstracting service specifications into abstract services and planning the abstracted services. Like STS, the aim is to compute a coordinator service that controls all possible behaviors of individual services. The approaches mentioned above belong to AI planning approaches. We discuss now constraint-based approaches which are proposed either as an alternative to AI planning approaches or to complement them. METEOR-S: METEOR-S [2] is a comprehensive service management framework that supports different phases of web service development including modeling, discovery and composition. Composition begins with a user pre-specifying the structure and kind of services that participate in the target assembly using the Abstract Process Designer tool. The composer seeks to bind tasks pre-specified in the abstract process with concrete services by converting requirements of a target assembly to a Constraint Satisfaction Problem (CSP), where services are predominantly filtered based on their Quality of Service (QoS) parameters. Configuration-Based Workflow Composition (COM): [3,4] studied the feasibility of automatically composing services using configuration techniques building on earlier work. An efficient technique that synthesizes service compositions by automatically combining a given set of candidate workflow instances while satisfying general composition constraints modeled in the form of a meta-model is presented. A workflow composition problem is defined with a constrained object meta-model, candidate service processes, and transformation services for mediation [4].
198
G. Grossmann et al.
Combining SHOP2 with Constraint Satisfaction (SHOP2CSP): SHOP2CSP [45] performs service composition in two phases: (1) a planning phase where the SHOP2 HTN planner is used to devise an abstract service composition, and (2) a CSP phase where the abstract composition is bound with a set of atomic one-shot services selected based on their non-functional properties such as QoS attributes. Generative Constraint Satisfaction Problem (GCSP): The GCSP approach works by modeling a service composition problem as a Generative Constraint Satisfaction Problem (GCSP) [33]. Service descriptions including interface specifications, domain knowledge, and composition problems (goals) are modeled in a generic manner using a declarative constraint-based formalism. The GCSP framework also incorporates an algorithm that simultaneously exploits generic descriptions of individual atomic services and service processes in a single framework. The algorithm works by instantiating generic service or service process components dynamically as required during the composition process. In the next section we describe requirements and comparison factors for the above mentioned approaches.
4
Requirements and Evaluation
The following set of evaluation factors is used to compare existing static and dynamic composition approaches. The factors are highlighted in bold and their abbreviations in parentheses can be found in comparison Table 1. Graphical notation (Graph.): Supporting graphical design notations is critical to the acceptance of composition frameworks and are advocated by modeldriven approaches [15]. We investigated if existing approaches either support the de-facto standard UML (U) or other (H) non-standard graphical notations. Matchmaking (Match.): Most of the static composition frameworks aim to solve the modeling issues and do not focus on employing a matchmaking technique to automatically select candidate services. We considered three matchmaking techniques in the comparison: (1) syntactic (X) approaches such as IO-based and non-functional properties-based matchmaking, (2) semantic (S) approaches that support a formal matchmaking approach based on service descriptions, and (3) instance-level (I) approaches that are capable of handling instance-level service details. Composition (Comp.): Three possible composition types could be identified, (1) atomic (A) indicating the composition of atomic services, (2) composite or process (P) indicating the composition of composite service processes, and (3) re-configuration (R) indicating the re-configuration (or re-planning) on different abstraction levels as discussed in Sections 2.3. Discovery (D) in the comparison table denotes that service discovery is supported. Tool (T) indicates that a tool was developed to perform the composition.
Conceptual Modeling Approaches for Dynamic Web Service Composition
199
Modeling comprehends the indication of support for five modeling aspects, (1) formal (F) indicates the existence of a formal language to model complex service descriptions and requirements, (2) model-driven (M) indicates the support for the notion of model-driven development, (3) life-cycle (L) indicates that the life-cycle phases are modeled explicitly in a process similar to the task workflow shown in Fig. 3, (4) evolution (E) indicates the support for evolutionary models, i.e., self-adaptive models that are able to change during the execution similar to the presented task workflow, and (5) orchestration (O) indicates the support for specifying behavioral user requirements in form of an orchestrator. In the last category structural, the approaches were compared according to the support of three structural elements in a workflow, (1) control flow (C), (2) data flow (W), and supporting multiple instance (N) of a service or service process. Table 1 gives a comparative summary of existing composition approaches based on the factors described in this section. Table 1. Comparison of dynamic composition approaches Approaches SHOP2 Maestro Haley CSS STS ABS
Graph. U H − − − + − − − + − − − −
Match. X S I ++− ++− ++− +−− +−− +−−
A + + + + + +
Comp. P R − +− − − − − +− − + − + −
D T − + − − − −
+ + + + + +
Modeling F M L E O + − − − + + + − − + + +− − − − − + − − +− + + − − + + − − − −
Structural C W N + + + − + + − + + − + + + + − − + −
METEOR-S − − + + − + − − + + + + − − + − − COM + − + + − + + − + + + + +− − + + + SHOP2CSP − − + − − + − − − − +− − − − − + +− GCSP − − + + + + + +− +− + + + − − +− +− + + . . . supported, +− . . . partially supported, − . . . not supported.
+ − + +
Comparison Table 1 indicates that the COM and GCSP approaches support the majority of the factors. Both support the composition of processes and the discovery of new services, neither of which are not supported by the other approaches. The COM framework is the only approach which partially supports the life cycle of a composition similar to the presented task workflow in Section 2. No support for life cycles could be found in the other approaches. However, one limitation of COM is that it does not support multiple instances of candidate services and service processes. This problem can be circumvented by prespecifying the number of instances in the target assembly of the configurator prior to the composition, although it cannot be changed during runtime. The GCSP approach overcomes this limitation but does not support control flow patterns such as loops. GCSP is especially suited for dynamic scenarios where number, type or structure of services in the goal cannot be pre-specified.
200
G. Grossmann et al.
Both COM and GCSCP belong to the group of configuration-based approaches. The other approaches in that group are METEOR-S and SHOP2CSP. METEORS allows the user to specify behavioral requirements which is not supported by SHOP2CSP, however METEOR-S does not support the decomposition of nonatomic service processes. SHOP2CSP is able to de- liver an optimal service assembly due to a two phase refinement but, similar to METEOR-S, it only supports atomic services. Furthermore, it lacks semantic data reasoning capabilities and service discovery needs to be performed manually. Among AI planning approaches, the Maestro and the STS approach support more factors than the other approaches from that group. Maestro is the only one supporting service discovery. However, similar to METEOR-S and SHOP2CSP, it only investigates the composition of atomic services. In the STS project, the ASTRO composer was developed [32]. It includes tools to convert services specified in languages such as BPEL to the prescribed STS syntax. This approach suffers from the limitations that there is no service discovery mechanism, no provision for multiple service instances, and no rigorous semantic data reasoning. An increase in efficiency compared to STS is achieved by reducing the number of variables in the ABS approach. For example, ABS neglects output variables that do not determine the control flow of the coordinator. Other than that, ABS still is subject to similar restrictions as the other approaches. The other AI planning approaches examined here are SHOP2, Haley, and CSS. Unlike a classical planner, SHOP2 mandates the presence of a rich knowledge base that consists of primitive actions and composite activities. A strong assumption of this approach is that all complex service processes have to be decomposed into atomic services before the planning can begin. Also, only control flow but no data flow elements are supported. Like SHOP2 and Maestro, Hayley only considers atomic services and neglects the problem of composing composite services. An advantage of Hayley is that it supports Quality of Service (QoS) attributes. The CSS approach supports multiple level of abstractions of input and output parameters and can therefore deal with complex parameter types. However, it requires that data structures are modeled in a specific format (“Data Semantic Structures”) which may cause additional overhead due to an initial mapping from existing structures.
5
Conclusion
We started this article with describing a model-driven life-cycle for dynamic service composition. It consists of design, configuration and enactment phase which are performed iteratively. Its unique properties which distinguish it from other life-cycle proposals are an evolutionary process model concept called task workflow and the consideration of re-configuration on different abstraction levels. By capturing the life-cycle phases explicitly in the task workflow and modeling evolutionary tasks the dynamic composition approach becomes truly model-driven as proposed by the Model Driven Architecture. It allows the modeling of evolutionary tasks such as service discovery, mediation, and composition interleaved with standard operational tasks such as external service calls and can therefore
Conceptual Modeling Approaches for Dynamic Web Service Composition
201
represent decision making steps explicitly which are usually hidden in standard orchestration modeling. The conceptual model serves to describe schema level information and process templates that can be instantiated for the execution of actual services, a distinction that is routine in classic data modeling, but not normally made so far in service composition. By separating out the schema and instance levels, finer distinction between potential service capabilities in configuration and reconfiguration during runtime can be achieved. This provides a structured framework for describing how a process can be readjusted at runtime according to information that in real world scenarios is ultimately dependent on execution of particular service instances and therefore not available at static design time, such as the availability of seats on a particular flight at the moment of booking a whole itinerary that was previously planned. The life-cycle therefore enables dynamic composition of the service process under control of the process itself, including the ability to request and incorporate user feedback and decisions when adjusting the process to meet changing conditions imposed by the environment. In the second part a survey examines and classifies the plethora of different approaches to service composition according to the life-cycle steps with a focus on the design and configuration phase. We identified some requirements on conceptual models and composition heuristics to facilitate the dynamic composition. We then compared some state-of-the-art composition approaches with them as shown in Table 1. We found out that only a few approaches incorporate discovery and control flow. The majority considered control flow indirectly by data flow dependencies. From a modeling point of view almost all approaches proposes a formal framework but none of them supports a conceptual model that can be adapted during runtime or models the whole life-cycle explicitly. Only a few considered the re-configuration (or re-planning) of parts of a composition for the adaptation and none supported the re-configuration on different abstraction levels such as schema and instance level.
References 1. Acuna, C.J., Marcos, E.: Modeling Semantic Web Services: A Case Study. In: Proc. ICWE, pp. 32–39. ACM Press, New York (2006) 2. Aggarwal, R., Verma, K., Miller, J.A., Milnor, W.: Constraint Driven Web Service Composition in METEOR-S. In: Proc. IEEE Int’l Conference on Service Computing (SCC), Shanghai, China, pp. 23–30 (September 2004) 3. Albert, P., Henocque, L., Kleiner, M.: Configuration Based Workflow Composition. In: Proc. of the IEEE International Conference on Web Services (ICWS), Orlando, Florida, USA, pp. 285–292 (July 2005) 4. Albert, P., Henocque, L., Kleiner, M.: An end-to-end configuration-based framework for automatic sws composition. In: Proceedings of the 20th IEEE International Conference on Tools with Artificial Intelligence (ICTAI), Dayton, Ohio, USA, pp. 351–358 (November 2008) 5. Alonso, G., Casati, F., Kuno, H., Machiraju, V.: Web Services – Concepts, Architecturse and Applications. Springer, Heidelberg (2003)
202
G. Grossmann et al.
6. Balzer, S., Liebig, T., Wagner, M.: Pitfalls of OWL-S – A practical Semantic Web Use Case. In: Proc. ICSOC, pp. 289–298 (2004) 7. Barros, A., Dumas, M., ter Hofstede, A.H.M.: Service Interaction Patterns. In: van der Aalst, W.M.P., Benatallah, B., Casati, F., Curbera, F. (eds.) BPM 2005. LNCS, vol. 3649, pp. 302–318. Springer, Heidelberg (2005) 8. Beauche, S., Poizat, P.: Automated service composition with adaptive planning. In: Bouguettaya, A., Krueger, I., Margaria, T. (eds.) ICSOC 2008. LNCS, vol. 5364, pp. 530–537. Springer, Heidelberg (2008) 9. Bertoli, P., Pistore, M., Traverso, P.: Automated composition of Web services via planning in asynchronous domains. Artificial Intelligence 174(3-4), 316–361 (2010) 10. Born, M., Hoffmann, J., Kaczmarek, T., Kowalkiewicz, M., Markovic, I., Scicluna, J., Weber, I., Zhou, X.: Semantic Annotation and Composition of Business Processes with Maestro. In: Bechhofer, S., Hauswirth, M., Hoffmann, J., Koubarakis, M. (eds.) ESWC 2008. LNCS, vol. 5021, pp. 772–776. Springer, Heidelberg (2008) 11. Breitman, K.K., Casanova, M.A., Truszkowski, W.: Semantic Web – Concepts, Technologies and Applications. In: NASA Monographs in Systems and Software Engineering. Springer, Heidelberg (2007) 12. Dhyani, D., Ng, W.K., Bhowmick, S.S.: A survey of web metrics. ACM Comput. Surv. 34(4), 469–503 (2002) 13. Dietze, S., Benn, N., Domingue, J., Conconi, A., Cattaneo, F.: Two-Fold Service Matchmaking - Applying Ontology Mapping for Semantic Web Service Discovery. In: ASWC, Shanghai, China, pp. 246–260 (December 2009) 14. Dustdar, S., Hoffmann, T.: Interaction pattern detection in process oriented information systems. DKE 62, 138–155 (2007) 15. Dustdar, S., Schreiner, W.: A survey on web services composition. IJWGS 1(1), 1–30 (2005) 16. Elgedawy, I., Tari, Z., Thom, J.A.: Correctness-aware high-level functional matching approaches for semantic Web services. TWEB 2(2) (2008) 17. Felfernig, A., Friedrich, G., Jannach, D., Stumptner, M., Zanker, M.: Transforming UML Domain Descriptions into Configuration Knowledge Bases. In: Knowledge Transformation for the Semantic Web, pp. 154–168. IOS Press, Amsterdam (2003) 18. G´ omez-P´erez, A., Gonz´ alez-Cabero, R., Lama, M.: ODE SWS: A Framework for Designing and Composing Semantic Web Services. IEEE Intelligent Systems 19(4), 24–31 (2004) 19. Grimm, S., Motik, B., Preist, C.: Matching Semantic Service Descriptions with Local Closed-World Reasoning. In: Sure, Y., Domingue, J. (eds.) ESWC 2006. LNCS, vol. 4011, pp. 575–589. Springer, Heidelberg (2006) 20. Grønmo, R., Jaeger, M.C.: Model-Driven Semantic Web Service Composition. In: Proc. APSEC, pp. 79–86. IEEE Press, Los Alamitos (2005) 21. Hamers, L., et al.: Similarity measures in scientometric research: The Jaccard index versus Salton’s cosine formula. Inf. Process. Manage. 25(3), 315–318 (1989) 22. Ben Hassine, A., Matsubara, S., Ishida, T.: A Constraint-Based Approach to Horizontal Web Service Composition. In: Cruz, I., Decker, S., Allemang, D., Preist, C., Schwabe, D., Mika, P., Uschold, M., Aroyo, L.M. (eds.) ISWC 2006. LNCS, vol. 4273, pp. 130–143. Springer, Heidelberg (2006) 23. Hoffmann, J., Weber, I., Scicluna, J., Kaczmarek, T., Ankolekar, A.: Combining scalability and expressivity in the automatic composition of semantic web services. In: ICWE, pp. 98–107 (2008) 24. Kashyap, V., Bussler, C., Moran, M.: The Semantic Web – Semantics for Data and Services on the Web. Springer, Heidelberg (2008)
Conceptual Modeling Approaches for Dynamic Web Service Composition
203
25. Kawamura, T., De Blasio, J.-A., Hasegawa, T., Paolucci, M., Sycara, K.: Public Deployment of Semantic Service Matchmaker with UDDI Business Registry. In: McIlraith, S.A., Plexousakis, D., van Harmelen, F. (eds.) ISWC 2004. LNCS, vol. 3298, pp. 752–766. Springer, Heidelberg (2004) 26. Kil, H., Nam, W., Lee, D.: Efficient Abstraction and Refinement for Behavioral Description Based Web Service Composition. In: Proceedings of the International Joint Conference on Artificial Intelligence, Pasadena, California, USA, pp. 1740– 1745 (July 2009) 27. Kuropka, D., Tr¨ oger, P., Staab, S., Weske, M. (eds.): Semantic Service Provisioning. CAT-SWS-1. Springer, Heidelberg (2008) 28. L´ecu´e, F., L´eger, A.: Semantic Web Service Composition Based on a Closed World Assumption. In: Proc. ECOWS, pp. 233–242. IEEE Press, Los Alamitos (2006) 29. Li, L., Horrocks, I.: A Software Framework for Matchmaking Based on Semantic Web Technology. In: Proc. of WWW Conference, pp. 331–339. ACM Press, New York (2003) 30. Manning, C., Raghavan, P., Sch¨ utze, H.: Introduction to Information Retrieval. Cambridge University Press, Cambridge (2008) 31. Marconi, A., Pistore, M.: Synthesis and composition of web services. In: Bernardo, M., Padovani, L., Zavattaro, G. (eds.) SFM 2009. LNCS, vol. 5569, pp. 89–157. Springer, Heidelberg (2009) 32. Marconi, A., Pistore, M., Traverso, P.: Automated Composition of Web Services: the ASTRO Approach. IEEE Data Eng. Bull. 31(3), 23–26 (2008) 33. Mayer, W., Thiagarajan, R., Stumptner, M.: Service Composition As Generative Constraint Satisfaction. In: Proc. ICWS (2009) 34. McIlraith, S., Son, T.C.: Adapting Golog for the Composition of Semantic Web Services. In: Proc. KR, pp. 482–493 (2002) 35. Moser, S., Martens, A., H¨ abich, M., M¨ ulle, J.: A Hybrid Approach for Generating Compatible WS-BPEL Partner Processes. In: Dustdar, S., Fiadeiro, J.L., Sheth, A.P. (eds.) BPM 2006. LNCS, vol. 4102, pp. 458–464. Springer, Heidelberg (2006) 36. Nezhad, H.R.M., Benatallah, B., Martens, A., Curbera, F., Casati, F.: Semiautomated adaptation of service interactions. In: Proc.of WWW Conference, pp. 993–1002. ACM Press, New York (2007) 37. Noy, N.F.: Semantic Integration: A Survey Of Ontology-Based Approaches. ACM SIGMOD Record 33(4) (2004) 38. Paolucci, M., Kawamura, T., Payne, T.R., Sycara, K.: Importing the Semantic Web in UDDI. In: Bussler, C.J., McIlraith, S.A., Orlowska, M.E., Pernici, B., Yang, J. (eds.) CAiSE 2002 and WES 2002. LNCS, vol. 2512, pp. 225–236. Springer, Heidelberg (2002) 39. Papazoglou, M.P., Traverso, P., Dustdar, S., Leymann, F.: Service-Oriented Computing: State of the Art and Research Challenges. IEEE Computer 40(11), 38–45 (2007) 40. Rao, J., Su, X.: A Survey of Automated Web Service Composition Methods. In: Cardoso, J., Sheth, A.P. (eds.) SWSWPC 2004. LNCS, vol. 3387, pp. 43–54. Springer, Heidelberg (2005) 41. Roman, D., Keller, U., Lausen, H., de Bruijn, J., Lara, R., Stollberg, M., Polleres, A., Feier, C., Bussler, C., Fensel, D.: Web Service Modeling Ontology. Applied Ontology 1(1), 77–106 (2005) 42. Sabou, M., Richards, D., van Splunter, S.: An experience report on using DAML-S. In: Proc. of WWW Workshop on E-Services and the Semantic Web (ESSW) (2003)
204
G. Grossmann et al.
43. Sirbu, A., Hoffmann, J.: Towards Scalable Web Service Composition with Partial Matches. In: Proc. of the IEEE International Conference on Web Services (ICWS), pp. 29–36 (2008) 44. Sirin, E., Parsia, B., Wu, D., Hendler, J., Nau, D.: HTN Planning for Web Service Composition Using SHOP2. Journal of Web Semantics 1(4), 377–396 (2004) 45. Song, X., Dou, W., Song, W.: Workflow Framework for Intelligent Service Composition. In: Workshop Proceedings at the Grid and Pervasive Computing Conference, Geneva, Switzerland (May 2009) 46. Srivastava, B., Koehler, J.: Web service composition - current solutions and open problems. In: ICAPS 2003 Workshop on Planning for Web Services (2003) 47. Stollberg, M., Haller, A.: Semantic Web Services Tutorial. In: ICWS (2005) 48. Thiagarajan, R., Stumptner, M.: A Native Ontology Approach for Semantic Service Description. In: Proc. of the Australasian Ontology Workshop (AOW). CRPIT Series, vol. 72, pp. 85–90. ACS (2006) 49. Trainotti, M., et al.: ASTRO: Supporting composition and execution of web services. In: Benatallah, B., Casati, F., Traverso, P. (eds.) ICSOC 2005. LNCS, vol. 3826, pp. 495–501. Springer, Heidelberg (2005) 50. van Rijsbergen, C.J.: Information Retrieval, Butterworth (1979) 51. Vitvar, T., Kopeck´ y, J., Viskova, J., Fensel, D.: WSMO-lite annotations for web services. In: Bechhofer, S., Hauswirth, M., Hoffmann, J., Koubarakis, M. (eds.) ESWC 2008. LNCS, vol. 5021, pp. 674–689. Springer, Heidelberg (2008) 52. Wang, H., Li, Z., Yang, B., Xia, H.: A Context-Aware Service Matchmaking Method Using Description Logic. In: Proc. IEEE Asia-Pacific Service Computing Conference (APSCC), Tsukuba Science City, Japan, pp. 26–32 (December 2007) 53. Weske, M.: Business Process Management: Concepts, Languages, Architectures. Springer, Heidelberg (2007) 54. Wu, D., Parsia, B., Sirin, E., Hendler, J., Nau, D.S.: Automating DAML-S web services composition using SHOP2. In: Fensel, D., Sycara, K., Mylopoulos, J. (eds.) ISWC 2003. LNCS, vol. 2870, pp. 195–210. Springer, Heidelberg (2003) 55. Yu, B., Zhang, C., Zhao, Y.: Transform from Models to Service Description Based on MDA. In: Proc IEEE APSCC, pp. 605–608. IEEE Press, Los Alamitos (2006) 56. Zhao, H., Doshi, P.: A hierarchical framework for logical composition of web services. Service Oriented Computing and Applications 3(4), 285–306 (2009)
The Dynamics of Process Modeling: New Directions for the Use of Events and Rules in Service-Oriented Computing Susan D. Urban, Le Gao, Rajiv Shrestha, and Andrew Courter Texas Tech University Edward E. Whitaker Jr. College of Engineering Department of Computer Science Lubbock, TX 79409 {susan.urban,le.gao}@ttu.edu
Abstract. The introduction of service-oriented computing has created a more dynamic environment for the composition of software applications, where processes are affected by events and data changes and also pose data consistency issues that must be considered in application design and development. This chapter addresses the need to develop a more effective means to model the dynamic aspects of processes in contemporary, distributed applications, especially in the context of concurrently executing processes that access shared data and cannot enforce traditional transaction properties. After an assessment of current tools for process modeling, we outline four approaches for the use of events and rules to support dynamic behavior associated with constraint checking, exception handling, and recovery. The specific techniques include the use of integration rules, assurance points, application exception rules, and invariants. The chapter concludes with a discussion of future research directions for the integrated modeling of events, rules, and processes. Keywords: service composition, event and rule processing, integration rules, application exception rules, assurance points, invariant conditions, dynamic process modeling.
1
Introduction
The advent of Web Services and service-oriented computing has significantly changed software development practices and data access patterns for distributed computing environments, creating the ability to develop processes that are composed of service executions. These processes are often collaborative in nature, involving long-running activities based on loosely-coupled, multi-platform, service-based architectures. This new software development paradigm makes the concept of virtual organizations a reality, better supporting enterprise-toenterprise business processes and data exchange. Service-oriented computing, however, also poses new challenges for software process modeling. In particular, processes must be flexible enough to respond to the different types of change Kaschek/Delcambre (Eds.): The Evolution of Conceptual Modeling, LNCS 6520, pp. 205–224, 2011. c Springer-Verlag Berlin Heidelberg 2011
206
S.D. Urban et al.
that can occur during execution. Change occurs, for example, when exceptional conditions arise in an application, such as a customer canceling or changing an order, or a warehouse discovering damaged shipments. Change also occurs when a process fails and needs to be recovered in a manner that maintains consistency for the failed process as well as for other processes that access shared data with the failed process. Processes must also be capable of executing in environments that no longer support traditional transactional properties but also guarantee correctness and consistency of execution. The ability to respond to change and guarantee consistency requires not only a flexible execution environment, but also techniques that support the modeling of a process’s ability to correctly respond to events and failures that affect the normal flow of execution. In this chapter, we first summarize existing techniques for modeling the dynamics of processes. We then introduce additional considerations for the use of events and rules in process modeling, especially in the context of serviceoriented computing. In particular, this chapter illustrates the use of integration rules, invariant rules, and application exception rules together with the concept of assurance points to model the more dynamic nature of service-oriented computing. Integration rules are similar to the use of events and rules to control process flow [11,12,20,38]. They are different, however, in that events are raised before and after the execution of services to trigger integration rules that test control logic that is orthogonal to the main procedural specification of process flow. Assurance points (APs) enhance the use of integration rules, providing checkpoints that are placed at critical locations in the flow of a process. An AP is used to store execution data that is passed as parameters to integration rules that check pre and post conditions and invoke additional execution logic. APs are also used as intermediate rollback points to support compensation, retry, and contingent procedures in an attempt to maximize forward recovery. Whereas integration rules can be used to check data conditions at certain points in process execution, invariants provide a stronger way to monitor data conditions that must hold over a certain period of time during the execution of a process, especially when data items cannot be locked over the span of multiple service executions. Invariants are activated with a starting AP, deactivated with an ending AP, and monitor the data of the invariant condition in between APs using a concept known as Delta-Enabled Grid Services (DEGS) [2]. An invariant therefore allows a process to declare data conditions that are critical to the execution of the process, but to allow multiple processes to access the same data in an optimistic fashion. When critical data conditions are violated, as detected by the DEGS capability, recovery conditions can be invoked. Finally, application exception rules provide a way to interrupt the execution of a process in response to exceptional conditions and to respond to exceptions in different ways depending on the state of the executing process as determined by assurance points. Application exception rules can also be combined with a data dependency analysis procedure associated with the use of DEGS to provide a way to help a process determine how its own recovery or forward execution can be affected by the failure and recovery of other processes that are accessing
The Dynamics of Process Modeling
207
shared data [46]. This is especially important for maintaining data consistency in environments that cannot provide traditional transaction processing guarantees. In the sections that follow, we first outline past work in the area of process modeling with a specific focus on the use of events, rules, and exception handling to provide dynamic behavior. We then provide motivation for integration rules, assurance points, invariants, and application exception rules in the context of decentralized process execution agents (PEXAs) that we have developed as part of our research on service-oriented computing. We then elaborate on assurance points and the rule functionality of our research. The chapter concludes with a summary and discussion of future research for modeling methodologies, modeling tools, and execution environments that support events and rules.
2
Conceptual Modeling of Business Processes
As described by Lu and Sadiq [26], most modeling techniques can be categorized as either graph-based techniques or rule-based techniques. The following subsections summarize graph and rule-based techniques, with a focus on support for dynamic capabilities. 2.1
Graph-Based Modeling Techniques
In graph-based modeling techniques such as BPMN [43], UML [14], and EPC [36], a business process is described by a graph notation in which activities are represented as nodes, and control flow and data dependencies between activities as arcs or arrows. BPMN. The Business Process Modeling Notation (BPMN V1.0) was introduced by the Business Process Management Initiative in 2004 [43]. The objective of BPMN is to provide a graphical model that can depict business processes and can be understood by both users and developers. Flow objects include symbols to represent events, activities, and gateways (i.e., decision points). Flow objects are connected to each other via connecting objects that represent sequence flow, message flow, and association. A process always starts from an event and ends in an event. All other events inside the process are called intermediate events and can be part of the normal flow or attached to the boundary of an activity. An attached event indicates that the activity to which the event is attached should be interrupted when the event is triggered. The attached event can trigger either another activity or sub-process. Typically, error handling, exception handling, and compensation are triggered by the attached event. To detail a business process, swim lanes and artifacts can be used. Swim lanes are used to either horizontally or vertically group a process into subgroups by rules, such as grouping processes by departments in a company business process. Artifacts provide additional information in a business process to make a model more readable, such as text descriptions attached to an activity. BPMN (V2.0 beta 1) [3] was released in 2009. In BPMN 2.0, the most important update is standardized execution semantics which provide execution
208
S.D. Urban et al.
semantics for all BPMN elements based on token flows. A choreography model is also supported in BPMN 2.0. Other significant changes include 1) a data object supporting assignments for activity; 2) updated gateways supporting exclusive/parallel event-based flow; 3) event-subprocesses used to handle event ocurrences in the bounding subprocess; 4) a call activity type that can call another process or a global task; and 5) escalation events for transferring control to the next higher level of responsibility. Mapping tools that can convert BPMN to executable languages, where the translation is enhanced with the execution semantics of BPMN 2.0. For example, the Business Process Execution Language (BPEL) [1] is used for executing business processes that are composed of Web Services. A well-known, open-source mapping tool is BPMN2BPEL [31]. UML. The Unified Modeling Language (UML) is a general-purpose modeling language with widespread use in software engineering. UML provides a set of graphical modeling notations to model a system. An activity diagram describes a business process in terms of control flow. A state diagram represents a business process using a finite number of states. UML also provides sequence diagrams that emphasize interactions between objects. In UML 2.0, new notations have been added to activity diagrams to provide support for the specification of pre and post conditions, events and actions, time triggers, time events, and exceptions. These notations provide more dynamic support to process modeling in UML. Researchers have also proposed process modeling enhancements to UML. For example, the work in [32] proposes a framework that supports exception handling using UML state charts. In [15], the authors present a method that can handle exceptions in sequence diagrams. EPC. The Event-driven Process Chain (EPC) method was developed within the framework of the Architecture of Integrated Information Systems (ARIS) in the early 1990s. The merit of EPC is that it provides an easy-to-understand notation. OR, AND, and XOR nodes are used to depict logical operations in the process flow. The main elements of a process description include events, functions, organization, and material (or resource) objects. The EPC does not have specific notation support for exception processing. Instead, EPC uses the logical operations to specify the handling of events and exceptional conditions. Recent work has modified the EPC notation to provide better support for process modeling. For example, in [29], yEPC provides a cancellation notation to model either an activity or a scope cancellation process. Other Related Methods. FlowMake is presented by Sadiq and Orlowska in [35]. FlowMake models a workflow using a graphical language, including workflow constraints that can be used to verify the syntactic correctness of a graphical workflow model. Reichert and Dadam [34] present a formal foundation for the support of dynamic structural changes of running workflow instances. ADEPTflex is a graph-based modeling methodology that supports users in modifying the structure of a running workflow, while maintaining its correctness and consistency [34]. YAWL [42] is designed based on Petri nets for the specification of control flow. ActivityFlow [24] provides a uniform workflow specification
The Dynamics of Process Modeling
209
interface to describe different types of workflows and helps to increase the flexibility of workflow processes in accommodating changes. ActivityFlow also allows reasoning about correctness and security of complex workflow activities independently from their underlying implementation mechanisms. An advantage of graph-based languages is that they are based on formal graph foundations that have rich mathematical properties. The visual capabilities also enhance process design for users and designers. The disadvantage is that graphbased modeling methods are not agile for dynamic runtime issues, requiring the use of specialized notations that can cause the model to become more complex. 2.2
Rule-Based Modeling Techniques
In a rule-based modeling approach, business rules are defined as statements about guidelines and restrictions that are used to model and control the flow of a process [16]. More recently, rules are used together with agent technology to provide more dynamic ways of handling processes. Use of Rules in Workflow and Service Composition: Active databases extend traditional database technology with the ability to monitor and react to circumstances that are of interest to an application through the use of EventCondition-Action (ECA) rules [44]. The work of Dayal et al. [9] was one of the first projects to use ECA rules to dynamically specify control flow and data flow in a workflow. In the CREW project [21], ECA rules are used to implement control flow. The TrigSFlow [23] model uses active rules to decide activity ordering, agent selection, and worklist management. Database representation of workflows [17] uses Event-ConditionMessage rules to specify workflows and utilize database logging and recovery facilities to enhance the fault-tolerance of the workflow application. Migrating workflows[7] provide dynamics in workflow instances. A migrating workflow transfers its code (specification) and its execution state to a site, negotiates a service to be executed, receives the results, and moves on to the next site [7]. ECA rules are used to specify the workflow control. Active rules also provide a solution for exception handling in workflow systems. ADOME [5] and WIDE [4] are commercial workflow systems that use active rules in exception handling. Rules are also used in workflow systems to respond to ad-hoc events that have predefined actions. Other workflow projects that use active rules are described in [6,13]. Active rules have been used to generate data exchange policies at acquaintance time among peer databases [22]. Agent-Based Techniques. Agent technology has been introduced to model business processes. Agents are autonomous, self-contained and capable of making independent decisions, taking actions to fulfill design goals and to model elements in a business process. Agents also support dynamic and automatic workflow adaptations, thus providing flexibility for unexpected failures. ADEPT [18] is an agent-based system for designing and implementing processes. The process logic is defined by a service definition language, where agents have sufficient freedom to determine which alternative path should be executed at runtime. AgentWork [30] is a flexible workflow support system that provides better
210
S.D. Urban et al.
support for exception handling using an event monitoring agent, an adaptation agent, and a workflow monitoring agent. Events represent exceptional conditions, with rule conditions and actions used to correct the workflow. The adaptation agent performs adjustments to the implementation. The workflow monitoring agent checks the consistency of the workflow after adaptation implementation. If the workflow is inadequate, the workflow monitoring agent will re-estimate the error and invoke a re-adaptation of the workflow. Rule and agent-based modeling methods provide better support for flexibility and adaptability in process modeling. Rule-based methods support modifications at runtime much easier than graph-based methods. It is easy to modify a process model by rule-based methods, and, unlike graph-based methods, rule-based methods do not need new notations to express exception handling processes. Rule-based methods, however, can be difficult to use and understand.
3
Motivation for New Rule Functionality
As illustrated in the previous section, most process modeling techniques are aligned with either a procedural approach, specified as a flow graph, or a ruledriven approach, where events and rules are used to control the flow of execution. Rules provide a more dynamic way to respond to events that represent a need to change the normal flow of execution. The use of rules in process modeling is especially important considering the growing prevalence of complex events, event-driven applications, and business activity monitoring. In our view, a dynamic approach to process modeling for service-oriented environments requires a combination of graph and rule-based techniques, where graph-based techniques provide a means for specifying the main application logic and events are used to interrupt or branch off of the main flow of execution, triggering rules that check constraints, respond to exceptions, and initiate parallel activity. Events and rules should also play an increased role in supporting failure and recovery activity. Planning for failure and recovery should be an integral component of process modeling for service-oriented architectures, especially in the context of concurrently executing processes that access shared data and cannot enforce traditional transactional properties. Our research addresses consistency checking as well as failure and recovery issues for service-oriented environments through the use of integration rules, invariants, and application exception rules, used together with a checkpointing concept known as assurance points. To motivate the use of these concepts, consider a decentralized execution environment consisting of Process Execution Agents (PEXAs) as shown in Figure 1. In our research, a PEXA is responsible for monitoring the execution of different processes. As shown in Figure 1, PEXA 1 is responsible for the execution of P1 and P4, PEXA 2 is responsible for P2, and PEXA 3 is responsible for P3. Each process invokes services at various locations within the network. As shown in Figure 1, P1 invokes operation a at the site of PEXA 1, operation b and operation c at the site of PEXA 2, and operation d at the site of PEXA 3.
The Dynamics of Process Modeling
211
Figure 1 also illustrates that PEXAs are co-located with Delta-Enabled Grid Services (DEGS). A DEGS is a Grid Service that has been enhanced with an interface that stores the incremental data changes, or deltas, that are associated with service execution in the context of globally executing processes [2,41]. A DEGS uses an OGSA-DAI Grid Data Service for database interaction. The database captures deltas using capabilities provided by most commercial database systems. Our own implementation has experimented with the use of triggers as a delta capture mechanism, as well as the Oracle Streams capability [41]. Oracle Streams is a feature that monitors database redo logs for changes and publishes these changes to a queue to be used for replication or data sharing.
Source Database
Local Delta Schedule
DEGS Interface
Service Invocation P2P Communication
P1
Interruption
PEXA1
Pointer: Process Instance P4 Instance Details
Rule Invocation P1
P1
Process
AP1
. Operation_a .
Assurance Point
AP1 . . AP2 Operation_b Operation_c
External Event P2
AP3 . . AP4 . Operation_d .
Integration Rules {Pre|Post}
PEXA
Process Execution Agent
DEGS Interface
Delta-Enabled Grid Service Interface
Invariant Rules
Application Exception Rules P3
PEXA2
PEXA3
DEGS Interface
DEGS Interface
Source Database
Local Delta Schedule
Local Delta Schedule
Source Database
Fig. 1. Decentralized Process Execution Agents with Events and Rules
Deltas captured using DEGS are stored in a delta repository that is local to the service. Our past work has experimented with the creation of a centralized Process History Capture System (PHCS) that included deltas from all DEGSs in the environment and the process runtime context generated by the process execution engine. Deltas are dynamically merged using timestamps as they arrive in the PHCS to create a time-ordered schedule of data changes from the DEGS. The global delta object schedule is used to support recovery activities when process execution fails [47,46,45], where the global delta object schedule provides
212
S.D. Urban et al.
the basis for discovering data dependencies among processes. Our most recent work has transformed the global delta object schedule into a distributed schedule with a decentralized algorithm for discovering data dependencies [40,25]. As a result, each PEXA in Figure 1 has its own local delta object schedule. Given that processes can execute in an environment where decentralized PEXAs can monitor data changes and communicate about data dependencies among concurrently executing processes, our work is focused on how to enhance the ability to monitor data consistency in the failure and recovery process. To illustrate our approach, the expanded view of P1 in Figure 1 shows the use of assurance points (APs) and the different types of rules. As shown for P1, APs can be placed at strategic locations in a process, where an AP is a combined logical and physical checkpoint that can be used to store execution data, alter program flow, and support recovery activity. One use of an AP is to trigger integration rules as shown for AP1, where integration rules check pre and post conditions for service execution. By checking pre/post conditions, user-defined consistency constraints can validated, which is important since most service-oriented environments cannot rely on traditional notions of serializability to ensure the correctness of concurrently executing processes. Another use of an AP is to activate invariant rules. Invariants indicate conditions that must be true during process execution between two different APs. As shown for P1 in Figure 1, an invariant is monitored during the execution between AP2 and AP3, where the invariant represents a data condition that is critical to the correct execution of P1. P1, however, may not be able to lock the data associated with the invariant during the service executions between AP2 and AP3. Given that DEGS can be used to monitor data changes, P1 can activate the invariant condition, but still allow concurrent processes to access shared data. P1 can then be notified if data changes violate the invariant condition. For example, if P3 modifies data associated with the invariant of P1, P1 can re-evaluate the invariant condition and invoke recovery actions if needed. P1 also illustrates the use of application exception rules at AP4. A process should be capable of responding to external events that may affect execution flow. The response to the event, however, may depend on the current status of the process. For example, P1 may respond one way if the process has passed AP4, but may respond differently if the process is only at AP1. Application exception rules therefore provide a case-based structure that allows a process to use information about assurance points to provide greater flexibility in response to events. Furthermore, since PEXAs can communicate about data dependencies among concurrently executing processes, when a process Pj invokes recovery procedures in response to integration rules, invariant conditions, or application exception rules, event notifications can be sent through P2P communication to dependent processes that are controlled by other PEXAs. Application exception rules can be used by a process Pi to intercept such events, determine how the failure and recovery of Pj potentially affects the correctness conditions of Pi , and respond in different ways depending on the AP status of the process.
The Dynamics of Process Modeling
4
213
Assurance Point and Rule Functionality
This section provides a more detailed description of the capabilities outlined in Section 3. The first subsection elaborates on foundational work with integration rules. The following subsections then address assurance points, invariant rules, and application exception rules. 4.1
Dynamic Behavior with Integration Rules
Integration Rules (IRules) were originally defined in [38] to investigate the middle-tier, rule processing technology necessary for the use of declarative, active rules in the integration of Enterprise Java Beans (EJB) components. Several different subcomponents to the IRules language framework have been defined, including the Component Definition Language (CDL) for defining a global object model of components and their relationships [11], the IRules Scripting Language (ISL) for describing application transactions (a BPEL-like language), the Event Definition Language (EDL) for defining events, and the Integration Rule Language (IRL) for defining active rules [11,12,38]. In this section, we focus on IRL and the functionality that it provides for dynamically testing the correctness of process execution. The remaining subsections then show modifications to IRL for additional dynamic modeling capabilities that address exception handling and the consistency of concurrent processes. IRules are different from past work with the use of rules to control workflow in that they are integrated with the use of procedural specifications. Using IRules, the main logic of a process can be expressed using a modeling tool such as BPMN. It is assumed, however, that the start and end of a process generates application transaction events. The execution of individual services within a process also generates method events both before and after the execution of a service. IRules are then used to respond to application transaction events and method events, controlling rule actions together with the normal process flow using rule coupling modes from active database technology [44]. Integration rules can therefore be used to check pre and post conditions, to change the flow of execution, to spawn a new flow of execution that eventually joins the main flow, to defer a new flow of execution upon successful completion of the main flow, or to invoke a new, independent, parallel flow of execution in addition to the main flow. The structure of an integration rule is shown in Figure 2. Events are generated before and after the execution of individual services and their enclosing processes by wrappers that coordinate rule and component execution. Rule conditions and actions can be enhanced with ec (for event/condition) and ca (for condition/action) coupling modes [20]. For example, the immediate synchronous mode implies that the main flow of execution (i.e., the one that generated the event that triggered the rule) is halted while rule execution occurs. The immediate synchronous mode is therefore useful for checking preconditions before the execution of a service. The immediate asynchronous mode allows the main flow to continue during rule execution, with the rule executing in the same transactional framework as the process that triggered the rule. The deferred mode provides a
214
S.D. Urban et al.
way of triggering a rule that schedules the execution of a procedure at the end of the main procedural flow. The deferred mode is useful for scheduling the execution of a post condition that must be used to ensure data consistency at the end of a service execution. Alternatively, a post condition can be tested by triggering an integration rule with an immediate synchronous mode after the execution of a service. The decoupled mode is used to trigger the execution of a rule condition or action that executes in parallel with the main flow of execution as a separate transactional entity. The decoupled mode therefore provides a way to use rules for invoking procedures that involve business activity monitoring. create rule event condition action
ruleName eventName(eventParameters) [on componentName componentVariables] [ec coupling] rule condition specification [ca coupling] rule action
Fig. 2. Structure of an Integration Rule [20]
As an example, consider the integration rule for a stock application in Figure 3. The purpose of the stockSell rule is to initiate sellStock transactions when a price increase occurs. We only want to initiate such transactions, however, for pending orders where the NewPrice exceeds the desired selling price in the pending order. This situation implies that we need to compare the new price of the stock with the old price of the stock to determine if there was a price increase. This can only be done by examining the old and new values before the execution of the price change in the Stock component, thus illustrating the need for the beforeSetPrice event. The coupling mode on the rule condition is immediate, indicating that the check for a price increase should be performed as soon as the rule is triggered. The sellStock transactions on the appropriate pending orders, however, should only be executed after the completion of the setPrice method. As a result, the action part of the rule is deferred, meaning that the action will not be performed until the end of the outer-most transaction in which the rule was triggered. The full details of the integration rule execution model as originally used with EJB components can be found in [19,20,38]. With respect to dynamic process modeling, integration rules illustrate the manner in which rules can be used for more than just the interconnection of steps in a workflow. Integration rules work together with procedural, graph-based specifications and are particularly useful for 1) checking conditions that validate the correctness of service execution and 2) invoking business activity monitoring procedures that execute in parallel with business processes. 4.2
Dynamic Behavior with Assurance Points
In our current research, we have enhanced the use of integration rules using assurance points and recovery actions. An assurance point (AP) is defined as a
The Dynamics of Process Modeling
create rule event condition action
215
stockSell beforeSetPrice(NewPrice) on stock S immediate when NewPrice>S.price deferred from Pn in S.pendingTrades where S.price>=Pn.desiredPrice AND Pn.action=”sell” do sellStock(S,Pn);
Fig. 3. Integration Rule Example for a Stock Application [38]
process execution correctness guard as well as a potential rollback point during the recovery process [37,39]. Given that concurrent processes do not execute as traditional transactions in a service-oriented environment, inserting APs at critical points in a process is important for checking consistency constraints and potentially reducing the risk of failure or inconsistent data. An AP also serves as a milestone for backward and forward recovery activities. When failures occur, APs can be used as rollback points for backward recovery, rechecking pre-conditions relevant to forward recovery. The work in [37,39] has developed a prototype of APs using the Process Modeling Language (PML) described in [27,28]. An AP is defined as: AP = , where: - apID is the unique identifier of the AP - apParameters is a list of critical data items to be stored as part of the AP, - IRpre is an integration rule defining a pre-condition, - IRpost is an integration rule defining a post-condition, - IRcond is an integration rule defining additional application rules. In the above notation, * indicates 0 or more occurrences, while ? indicates zero or one optional occurrences. IRpre, IRpost, and IRcond are expressed in the integration rule format introduced in Figure 2, where the eventName is the name of the assurance point that triggers the rule. For IRpre and IRpost, a constraint C is always expressed in a negative form (not(C)). The action of a rule is invoked if the pre or post condition is not true, invoking a recovery action or an alternative execution path. If the specified action is a retry activity, then there is a possibility for the process to execute through the same pre or post condition a second time. In such a case, IRpre and IRpost rules support the specification of a second action to invoke a different recovery procedure the second time through. In its most basic form, the recovery action of an integration rule simply invokes an alternative process. Recovery actions can also be one of the following actions: - APRollback: APRollback is used when the entire process needs to compensate its way back to the start of the process. - APRetry: APRetry is used when a process needs to be backward recovered using compensation to a specific AP. The backward recovery process will go to the first AP reached as part of the compensation process. The pre-condition defined in the AP is re-checked before resuming the execution.
216
S.D. Urban et al.
- APCascadedContingency (APCC): APCC is a backward recovery process that searches backwards through the hierarchical nesting of processes to find a contingent procedure for a failed sub-process. During the APCC backward recovery process, when an AP is reached, the pre-condition defined in the AP is re-checked before invoking a contingent procedure for forward recovery. When the execution of a process reaches an AP, integration rules associated with the AP are invoked. The condition of an IRpost is evaluated first. If the postcondition is violated, the action invoked can be one of the pre-defined recovery actions as described above. If the post-condition is not violated, then an IRpre rule is evaluated before the next service execution. If the pre-condition is violated, one of the pre-defined recovery actions will be invoked. If the pre-condition is satisfied, the AP will check for any conditional rules (IRcond) that may exist. IRcond rules do not affect the normal flow of execution but provide a way to invoke parallel activity based on application requirements. Note that the expression of a pre-condition, post-condition or any additional condition is optional. As an example, consider a subset of an online shopping process, as shown in Figure 4, where two APs are inserted. Both APs have integration rules that must be checked when the process execution reaches the APs. The cop and top in the process indicate the compensation and contingency of the attached activity, respectively. AP1 is orderPlaced, which reflects that the customer has finished placing the shopping order. Before executing the payment activity, the pre-condition at AP1 is checked to guarantee that the store has enough goods in stock. Otherwise, the process invokes the backOrderPurchase process instead. Similarly, the CreditCardCharged AP2 after payment activity has a post-condition that further guarantees that the in-stock quantity must be in a reasonable status (not less than zero) after the decInventory operation. Otherwise, a recovery action APRetry must be invoked to recover the process back to AP1 and re-execute the payment activity. If the post-condition fails after re-execution, then APRollback will be invoked to abort the overall process. 4.3
Dynamic Behavior with Invariants
APs together with integration rules allow data consistency conditions to be checked at specific points in the execution of a process [37], using rule actions to invoke recovery procedures. In some applications, however, stronger condition checking techniques may be needed to monitor data consistency. As a result, an additional way to use rules together with APs is through the use of invariants. An invariant is a condition that must be true during process execution between two different APs. An invariant is designed for use in processes where 1) isolation of data changes in between service executions cannot be guaranteed (i.e., critical data items cannot be locked across multiple service executions), and 2) it is critical to monitor constraints for the data items that cannot be locked. The data monitoring functionality provided by our previous work with DEGS makes it possible to monitor invariant conditions. Invariants provide a stronger way of monitoring constraints and guaranteeing that a condition holds for a specific
The Dynamics of Process Modeling
217
Process
Operation
Add to cart
Select shipping method
Cop
Compensation
Top
Contingency Assurance Point
Payment information input
cop (AbortOrder)
Place an order
AP1
AP2
Rule Invocation
OrderPlaced (orderId, itemID, N)
cop(creditBack)
Charge credit card
top(eCheckPay)
Dec inventory
cop(incInventory)
CreditCardCharged (orderId, cardNumber, amount)
create rule QuantityCheck::pre event: OrderPlaced (orderId) condition: exists(select L.itemId from Inventory I, LineItem L where L.orderId=orderId and L.itemId=I.itemId and L.quantity>I.quantity) action: backOrderPurchase(orderId)
create rule QuantityCheck::post event: CreditCardCharged (orderId, cardNumber, amount) condition: exists(select L.itemId from Inventory I, LineItem L where L.orderId=orderId and L.itemId=I.itemId and I.quantity<0) action1: APRetry action2: APRollback
Fig. 4. Portion of a Process Illustrating Assurance Points and Integration Rules
duration of execution without the use of locking. A prototype of the invariant capability has been developed and demonstrated in [8]. Using the invariant technique, a process declares an invariant condition when it reaches a specific AP in the process execution, also declaring an ending AP for monitoring of the invariant condition. When a concurrent process modifies a data item of interest in an invariant condition, the process that activated the invariant is notified by a monitoring system built on top of Delta-Enabled Grid Services. If the invariant condition is violated during the specified execution period, the process can invoke the recovery procedures defined in Section 4.2. The strength of the invariant technique is that it provides a way to monitor data consistency in an environment where the coordinated locking of data items across multiple service executions is not possible. An invariant has an identifier, two AP specifications (APs as a starting AP and and APe as an ending AP), and optional parameters that are necessary in the condition specification. Once APs is reached, the invariant rule condition becomes active. The condition is specified as an SQL query. The condition is initially checked and the action is executed if the invariant condition is violated. If the invariant condition holds, the rule condition goes into monitoring mode using the DEGS capability. The condition monitoring continues until APe is reached or until the invariant condition is violated.
218
S.D. Urban et al.
As shown in Figure 5, when an invariant condition goes into monitoring mode, the data items of interest in the invariant condition are registered with a monitoring service. The monitoring service subscribes to the DEGSs that contain the relevant data items referenced in an invariant. The DEGSs of the environment will notify the service of any changes to the relevant data items by concurrent processes. Any deltas that are forwarded to the monitoring service will cause the invariant condition to be rechecked. As long as the condition still holds, then there is no interference among the concurrent process executions. If the condition is violated, then the recovery action of the invariant rule will be executed.
Process Specification
Process A . . . AP1 . . .
Process Specification
Process B . . .
Invariant data changed data monitoring
AP2 . . .
N
condition checked violated?
Update operations
Y
invoke recoveryAction
. . .
DEGS interface
Fig. 5. Semantics of an Invariant under Monitoring Mode
As a specific example, consider the invariant in Figure 6, where the LoanAmountMonitoring invariant is to be monitored between the LoanAppCreation AP (i.e., the starting AP for the monitoring process) and the LoanCompletion AP (i.e., the ending AP for the monitoring process). The process represents a loan approval process, where the process is creating a loan application for a customer at a bank that already has an account at that bank. Figure 6 shows an invariant that checks to make sure the loan applicant has a tenth of the requested loan amount in the account, where the amount and customerid are passed as parameters from the LoanAppCreation AP. The condition is expressed as an SQL query, preceded with the not exists clause. Therefore, according to the SQL condition defined, if the account balance does not meet the criteria, then the select condition will return no tuples, making the not exists clause true, which triggers recoveryAction1. If the query returns tuples that satisfy the SQL condition, then the process continues and the status of the SQL query is monitored using the DEGS capability and the invariant monitoring system. If the process reaches the LoanCompletion AP and the applicants account balance still meets the necessary criteria, then the process continues past the LoanCompletion AP, completing the loan application after deactivating the LoanAmountMonitoring invariant. If at anytime between the LoanAppCreation
The Dynamics of Process Modeling
219
AP and the LoanCompletion AP, the applicant’s account balance falls below the necessary criteria, the invariant monitoring system will notify the process, which will execute one of the recovery actions.
create rule event condition
recoveryAction1 recoveryAction2
LoanAmountMonitoring::inv LoanAppCreation(LoanCompletion, customId) (Not exists (select * from loan where loan.applicantID = ‘“+customId+”’ and loan.status=’pre-qualified’ and loan.amount < (select 10*balance from account where account.customId = ‘“+customId+”’)) APRetry APRollback
Fig. 6. An Invariant Example for Monitoring a Bank Balance for a Loan Approval Process
4.4
Dynamic Behavior with Application Exception Rules
Dynamic behavior can also be achieved by using rules to respond to exceptional conditions, where exceptions are communicated as events that interrupt the normal flow of execution. Whereas past work generally provides a fixed response to exceptions, our work with application exception rules, provides a more flexible way of using rules to respond to exceptions. We are currently developing the application exception rule functionality in [33]. As with integration rules and invariants, application exception rules are also associated with assurance points. An outline of a process with APs is shown in the leftmost column of Figure 7, where two different APs are defined. Each AP represents the fact that a process has passed certain critical points in the execution and that responding to an exception depends on the APs that have been passed for individual instances of a process. Application exception rules are then written to respond to exceptions according to the AP status of a process. As shown in the middle column of Figure 7, application exception rules have a case structure, defining recovery actions based on APs. When an exception occurs, application exception rules are triggered. The exception handling procedure to execute varies according to the AP status of the process, where recovery actions can query the execution state associated with the most previous AP. As shown in Figure 7, one instance of process A executes recoveryAction1 since the process has passed AP1 but not AP2. The other instance of process A executes recoveryAction2 since the process has passed AP2. For example, in an order processing application, if an order is canceled before the packing and shipment of the order, then the order processing is cancelled. If the order is cancelled after the shipment has occurred, the order processing might be cancelled with an additional restocking fee charged to the customer.
220
S.D. Urban et al.
3URFHVV 6SHFLILFDWLRQ 3URFHVV$ $3 $3
$SSOLFDWLRQ ([FHSWLRQ5XOH 2Q(YHQW% &DVH
,QVWDQFH 3URFHVV$ $3 ([HFXWLRQSRLQW
$3 $FWLRQ $3 $FWLRQ (QGFDVH
,QVWDQFH 3URFHVV$ $3 $3 ([HFXWLRQSRLQW
Fig. 7. The Use of Application Exception Rules
APs and application exception rules represent the fact that a response to an exception is not always a fixed action. The manner in which a process responds to an exception depends on the state of the process. Identifying exceptions that alter the execution path should be a routine aspect of process modeling. Application exception rules advocate that the identification of exceptions should be extended to also consider the critical execution points that may affect recovery actions, and that rules, together with supportive execution environments, should be designed to provide variability of response. A broader use of application exception rules is in the context of the decentralized execution environment with support for data dependency analysis as described in Section 3. Using the data monitoring capabilities of DEGS, we have developed a decentralized data dependency analysis algorithm [40,25] to enhance recovery activities for concurrent processes that execute with relaxed isolation properties. In particular, when one process fails, recovery activities in the form of compensation can occur. Compensating procedures, however, may make changes to the data that has been read and used by concurrent processes. Using DEGS together with the decentralized data dependency analysis algorithm, the failure and recovery of one process can include the identification of other dependent processes that may be executing within the decentralized environment. Events can then be used to interrupt the execution of dependent processes, with application exception rules providing a way to respond to such events in a flexible manner. We have experimented with this approach using process interference rules (PIRs) [46]. A PIR is written from the perspective of an executing process and is used to test user-defined correctness conditions to determine if a dependent process should continue running or invoke its own recovery procedures. We are currently integrating the PIR functionality into the concept of application exception rules to provide a more dynamic way to 1) recognize potential data inconsistency problems among concurrently executing processes and 2) use the event and rule functionality of application exception rules to interrupt dependent processes, test data consistency conditions, and invoke recovery procedures as needed.
The Dynamics of Process Modeling
5
221
Summary and Future Directions
This chapter has outlined several non-traditional uses of rules for supporting dynamic behavior in service-oriented environments. The advantage of the techniques presented is that they integrate the use of procedural and rule-based techniques for process modeling. As a proof of concept, prototypes have already been developed for integration rules, assurance points, the integration rule recovery actions, invariants, the process interference rule pre-cursor to application exceptions rules, and decentralized data dependency analysis [20,37,39,40,47,25]. A prototype of the more general use of application exception rules is currently under development. An interesting challenge lies in developing methodologies that are capable of supporting each rule paradigm in an integrated manner. Each rule type addresses a different dimension of dynamics for service-oriented environments. Methodologies are needed to define when and how the different rule forms are defined. Notational conventions are needed to enhance existing, graph-based approaches with notations that depict the way in which rules are used and integrated with procedural specifications. Guidelines are also needed to assist with the placement of assurance points, with the specification of the different types of rules, and with defining the conditions under which the rules are used. Execution environments are also needed to support the dynamic capabilities supported by integration rules, application exception rules, and process interference rules. Our own research is focused on the development of decentralized Process Execution Agents that communicate in a peer-to-peer manner to dynamically determine data dependencies among concurrently executing processes and to coordinate the execution of processes with the different rule forms outlined in this chapter. Acknowledgments. This research has been supported by NSF Grant No. CCF0820152 and NSF Grant No. IIS-9978217.
References 1. Alves, A., Arkin, A., Askary, S., Barreto, C., Bloch, B., Curbera, F., Ford, M., Goland, Y., Gu´ızar, A., Kartha, N., et al.: Web services business process execution language version 2.0. OASIS Standard, April 11 (2007), http://docs.oasis-open.org/wsbpel/2.0/OS/wsbpel-v2.0-OS.html 2. Blake, L.: Design and Implementation of Delta-Enabled Grid Services. M.S. Thesis, Deptment of Computer Science and Engineering, Arizona State Univ. (2005) 3. BPMN, O.: BPMN 2.0 beta 1 (2009), http://www.omg.org/cgi-bin/doc?dtc/09-08-14.pdf 4. Ceri, S., Grefen, P., Sanchez, G.: WIDE-a distributed architecture for workflow management. In: Proceedings of 7th Int. Workshop on Research Issues in Data Engineering, pp. 76–79 (1997) 5. Chiu, D., Li, Q., Karlapalem, K.: Exception handling with workflow evolution in ADOME-WFMS: a taxonomy and resolution techniques. ACM Siggroup Bulletin 20(3), 8 (1999)
222
S.D. Urban et al.
6. Cichocki, A.: Workflow and process automation: concepts and technology. Kluwer Academic Pub., Dordrecht (1998) 7. Cichocki, A., Rusinkiewicz, M.: Migrating workflows. NATO ASI series. Series F: Computer and system sciences, pp. 339–355 (1998) 8. Courter, A.: Supporting Data Consistency in Concurrent Process Execution with Assurance Points and Invariants. M.S. Thesis, Texas Tech University (2010) 9. Dayal, U., Hsu, M., Ladin, R.: A transactional model for long-running activities. In: Proceedings of the 17th International Conference on Very Large Data Bases, pp. 113–122. Citeseer (1991) 10. Desel, J.: Process modeling using petri nets. In: Process-Aware Information Systems: Bridging People and Software through Process Technology, pp. 147–177 (2005) 11. Dietrich, S., Patil, R., Sundermier, A., Urban, S.: Component adaptation for eventbased application integration using active rules. Journal of Systems and Software 79(12), 1725–1734 (2006) 12. Dietrich, S., Urban, S., Sundermier, A., Na, Y., Jin, Y., Kambhampati, S.: A language and framework for supporting an active approach to component-based software integration. Informatica-Ljubljana 25(4), 443–454 (2002) 13. Do˘ ga¸c, A.: Workflow management systems and interoperability. Springer, Heidelberg (1998) 14. Engels, G., F¨ orster, A., Heckel, R., Th¨ one, S.: Process modeling using UML. In: Process Aware Information Systems: Bridging People and Software Through Process Technology, pp. 85–118 (2005) 15. Halvorsen, O., Haugen, O.: Proposed notation for exception handling in UML 2 sequence diagrams. In: Software Engineering Conference, 2006, p. 10 (2006) 16. Herbst, H., Knolmayer, G., Myrach, T., Schlesinger, M.: The specification of business rules: A comparison of selected methodologies. In: Proceedings of the IFIP WG8, vol. 1, pp. 29–46. Citeseer (1994) 17. Jean, D., Cichock, A., Rusinkiewicz, M.: A database environment for workflow specification and execution. In: Proc. Intl. Symposium on Cooperative Database Systems Kyoto (1996) 18. Jennings, N., Faratin, P., Norman, T., O’Brien, P., Odgers, B., Alty, J.: Implementing a business process management system using ADEPT: A real-world case study. Applied Artificial Intelligence 14(5), 421–463 (2000) 19. Jin, Y., Urban, S., Dietrich, S.: A concurrent rule scheduling algorithm for active rules. Data & Knowledge Engineering 60(3), 530–546 (2007) 20. Jin, Y., Urban, S., Dietrich, S., Sundermier, A.: An Integration Rule Processing Algorithm and Execution Environment for Distributed Component Integration. Informatica-Ljubljana 30(2), 193 (2006) 21. Kamath, M., Ramamritham, K.: Failure handling and coordinated execution of concurrent workflows. In: Proceedings of the International Conference on Data Engineering, pp. 334–341. Citeseer (1998) 22. Kantere, V., Kiringa, I., Mylopoulos, J., Kementsietsidis, A., Arenas, M.: Coordinating peer databases using ECA rules. In: Aberer, K., Koubarakis, M., Kalogeraki, V. (eds.) VLDB 2003. LNCS, vol. 2944, pp. 108–122. Springer, Heidelberg (2004) 23. Kappel, G., Proll, B., Rausch-Schott, S., Retschitzegger, W.: TriGS/sub flow: Active object-oriented workflow management. In: Proc. of HICSS, p. 727. IEEE Computer Society, Los Alamitos (1995) 24. Liu, L., Pu, C.: Activity flow: Towards incremental specification and flexible coordination of workflow activities. In: Embley, D.W. (ed.) ER 1997. LNCS, vol. 1331, pp. 169–182. Springer, Heidelberg (1997)
The Dynamics of Process Modeling
223
25. Liu, Z.: Decentralized Data Dependency Analysis for Concurrent Process Execution. M.S. Thesis, Texas Tech University (2009) 26. Lu, R., Sadiq, W.: A survey of comparative business process modeling approaches. In: Abramowicz, W. (ed.) BIS 2007. LNCS, vol. 4439, pp. 82–94. Springer, Heidelberg (2007) 27. Ma, H.: The design and implementation of the GridPML: a process modeling language for the DeltaGrid. M.S. Thesis, Arizona State University (2005) 28. Ma, H., Urban, S., Xiao, Y., Dietrich, S.: GridPML: A Process Modeling Language and History Capture System for Grid Service Composition. In: Proceedings of the International Conference on e-Business Engineering (2005) 29. Mendling, J., Neumann, G., N¨ uttgens, M.: Yet another event-driven process chain. In: van der Aalst, W.M.P., Benatallah, B., Casati, F., Curbera, F. (eds.) BPM 2005. LNCS, vol. 3649, pp. 428–433. Springer, Heidelberg (2005) 30. M¨ uller, R., Greiner, U., Rahm, E.: AW: a workflow system supporting rule-based workflow adaptation. Data & Knowledge Engineering 51(2), 223–256 (2004) 31. Ouyang, C., Dumas, M., Aalst, W., Hofstede, A., Mendling, J.: From business process models to process-oriented software systems. ACM Transactions on Software Engineering and Methodology (TOSEM) 19(1), 2 (2009) 32. Pint´er, G., Majzik, I.: Modeling and analysis of exception handling by using UML statecharts. In: Guelfi, N., Reggio, G., Romanovsky, A. (eds.) FIDJI 2004. LNCS, vol. 3409, pp. 58–67. Springer, Heidelberg (2005) 33. Ramachandran, J.: Integrating Exception Handling and Data Dependency Analysis through Application Exception Rules. M.S. Thesis (in progress), Texas Tech University (to appear, 2011) 34. Reichert, M., Dadam, P.: ADEPT flex supporting dynamic changes of workflows without losing control. Journal of Intelligent Information Systems 10(2), 93–129 (1998) 35. Sadiq, W., Orlowska, M.: On capturing process requirements of workflow based business information systems. In: Proceedings of the 3rd International Conference on Business Information Systems (BIS 1999) (1999) 36. Scheer, A., Thomas, O., Adam, O.: Process modeling using event-driven process chains. In: Process-Aware Information Systems: Bridging People and Software Through Process Technology, pp. 119–145 (2005) 37. Shrestha, R.: Using Assurance Points and Integration Rules for Recovery in Service Composition. M.S. Thesis, Texas Tech University (2010) 38. Urban, S., Dietrich, S., Na, Y., Jin, Y., Saxena, S., Urban, S., Dietrich, S., Na, Y., Jin, Y.: The irules project: using active rules for the integration of distributed software components. In: Proceedings of the 9th IFIP 2.6 Working Conference on Database Semantics: Semantic Issues in E-Commerce Systems, Hong Kong, pp. 265–286. Citeseer (2001) 39. Urban, S., Gao, L.S., Courter, A.: Achieving Recovery in Service Composition with Assurance Points and Integration Rules. In: Meersman, R., Dillon, T.S., Herrero, P. (eds.) OTM 2010. LNCS, vol. 6426, pp. 428–437. Springer, Heidelberg (2010) 40. Urban, S., Liu, Z., Gao, L.: Decentralized data dependency analysis for concurrent process execution. In: 13th Enterprise Distributed Object Computing Conference Workshops, EDOCW 2009, pp. 74–83 (2009) 41. Urban, S., Xiao, Y., Blake, L., Dietrich, S.: Monitoring data dependencies in concurrent process execution through delta-enabled grid services. International Journal of Web and Grid Services 5(1), 85–106 (2009) 42. Van Der Aalst, W., Ter Hofstede, A.: YAWL: yet another workflow language. Information Systems 30(4), 245–275 (2005)
224
S.D. Urban et al.
43. White, S., et al.: Business Process Modeling Notation (BPMN) Version 1.0. Business Process Management Initiative, BPMI org (2004), http://www.bpmi.org/bpmi-downloads/BPMN-V1.0.pdf 44. Widom, J., Ceri, S.: Active database systems: Triggers and rules for advanced database processing. Morgan Kaufmann Pub., San Francisco (1996) 45. Xiao, Y., Urban, S.D., Liao, N.: The deltaGrid abstract execution model: Service composition and process interference handling. In: Embley, D.W., Oliv´e, A., Ram, S. (eds.) ER 2006. LNCS, vol. 4215, pp. 40–53. Springer, Heidelberg (2006) 46. Xiao, Y., Urban, S.: Using Data Dependencies to Support the Recovery of Concurrent Processes in a Service Composition Environment. In: Proceedings of the Cooperative Information Systems Conference (COOPIS), Monterrey, Mexico, pp. 139–156 (2008) 47. Xiao, Y., Urban, S.: The DeltaGrid Service Composition and Recovery Model. International Journal of Web Services Research 6(3), 35–66 (2009)
Exception Handling in Web Service Processes Dietmar Jannach and Alexander Gut Technische Universit¨ at Dortmund, 44221 Dortmund, Germany {firstname.lastname}@tu-dortmund.de
Abstract. Cross-company business processes are common in today’s networked economy and are nowadays often supported by process support systems that integrate the information systems of the different partners based on web service technology. In contrast to earlier Workflow Management Systems, which were often deployed in the controlled environment of a single company, the distributed nature of modern solutions make company-spanning web service processes more susceptible to failures. Therefore, it is desirable to augment the process models already at design time with error-handling behavior such that disrupted process instances can for instance be rolled back or completed on an alternative execution path, if, e.g., an individual service is not reachable. In this chapter, we will give an overview of past and current approaches as well as potential future works to exception handling in web service processes. We start with the concepts that were developed in the area of Workflow Management Systems, continue with error-handling techniques in state-of-the-art process modeling languages and finally give an outlook on future automated approaches to error recovery and repair.
1
Introduction
In the last quarter-century, many companies have successfully (re-)organized their business according to the process-oriented paradigm – as opposed to product- or function-oriented models – in order to increase efficiency and withstand competitive pressure. Alongside with this evolution, information systems that support this process-oriented way of organizing a business have been developed and are now well-established in industrial practice. Beside large and integrated Enterprise Resource Planning (ERP) systems, which comprise a wide range of cross-domain configurable processes, and specialized domain-specific solutions, Workflow Management Systems (WfMS) came up in the early 1990s as a means to support electronic business processes and accompanying document flows. One particularity of this type of Process Support Systems is that such systems are based on process models, which are not pre-implemented but can be designed according to the needs of the business. Thus, most Workflow Management Systems consist of a design-time component, which is used to graphically sketch the process flow (also called workflow schema), and a run-time component called workflow engine, whose main task is to execute the process flow by e.g., forwarding work items to the appropriate users. Kaschek/Delcambre (Eds.): The Evolution of Conceptual Modeling, LNCS 6520, pp. 225–253, 2011. c Springer-Verlag Berlin Heidelberg 2011
226
D. Jannach and A. Gut
A main assumption of very early WfMSs was that the supported processes do not go beyond company borders. Although other WfMS-external information systems or software solutions may be integrated, the engines were primarily designed to run in a “controlled environment” within the company’s organizational and IT infrastructure. Likewise, the problem of dealing with unexpected situations during process execution was not in the focus of the software tools and/or research in the early years. However, the need for a defined way of dealing with “broken” workflows is obvious as in reality, things not always work out as planned. In the real world, typical ways of dealing with problems during a process comprise, e.g., the cancelation or undoing of already performed steps or the execution of alternative, compensating actions. Early commercial WfMSs had no special, built-in means to model what should happen in case of an unexpected situation or upon a failure of an external tool. In these systems, the only option is to extend the process model (workflow schema) with explicit error-checking conditions and execution branches for expected problems. Typical measures to forward recovery of problems anticipated by the process designer might include re-doing a step (n times), canceling the process, skipping, retrying or postponing an activity and so on. However, even for very small processes, such an approach is not viable in practice because the resulting models become too complex when an additional execution branch is defined for every activity whose execution may fail. In fact, adding error recovery activities to the model in order to introduce fault tolerant behavior may paradoxically even lead to a reduced fault tolerance due to increased system complexity [HA00]. Therefore, researchers soon began to work on more elaborate approaches to augment the workflow models with additional error-handling constructs and engines with corresponding error recovery capabilities. The first proposed approaches were based on the concepts of transactions known from the field of database management systems, where the problem consists of ensuring a consistent system state even if one or the other step in the process fails, see [SR93] for an early work. Later on, in the late 1990s, companies became more and more connected based on Internet technology and emerging e-Business standards. Consequently, the problem of unexpected behavior during workflow execution became more and more important as workflows could now span different companies and integrate various autonomous information systems, which are out of the control of the main driving workflow engine1 . Note that from a technological perspective, application integration was a hot issue then, when commercial implementations of the CORBA2 specification reached some level of maturity. Beside transactions, at that time also the concept of exception handling – inspired by the exception handling constructs of object-oriented programming languages – was introduced to workflow models as a means to error recovery, see [HA00] for an example. 1
2
In fact, interface definitions for the integration of different workflow engines had already been included in the Workflow Reference Model developed by the Workflow Management Coalition around 1995; http://www.wfmc.org/ http://www.omg.org/corba/
Exception Handling in Web Service Processes
227
Today, in the late 2000s, the term “workflow management” is out of fashion. Cross-organizational electronic business process support, however, is more important than ever in today’s networked economy. Thus, the problem of unexpected errors has remained and only the technologies have changed. Instead of CORBA, XML-based document formats and web service technology are used today. Instead of proprietary notations of WfMS vendors, (various) standardized process modeling languages such as WS-BPEL3 are common for what is called “orchestrated web services” or web service processes. In this chapter we will first give an overview of how fault tolerant behavior, based e.g., on exception handling or transaction concepts, can be expressed in process models in state-of-the-art modeling languages. The considered modeling languages are the Business Process Modeling Language BPMN4 , WS-BPEL, and YAWL[Hof05]. With respect to transactions and co-operating web services, we will also shortly discuss the corresponding “choreography” web service standards. In the last sections, we will summarize current research efforts toward socalled “self-healing” web services based on extended process models and sketch how semantic annotations for services can serve as a basis for next-generation, automated recovery mechanisms.
2
Process Modeling Languages
Over the last two decades, several proposals for languages for business process modeling have been proposed by individual research groups or by industry consortia. While we could observe some convergence of the competing proposals in the last years, e.g., in the form of language mappings, there exists no “‘Lingua Franca” today that is consistently used across all different communities that have to do with business process modeling. In the following, we will discuss which error-handling mechanisms are supported by three popular languages. The languages are the (a) BPMN, which is based on informal semantics and graphical models only, (b) YAWL, as a language from academia with its origins in workflow modeling and (c) WS-BPEL, as today’s de-facto standard for modeling orchestrated web service processes. 2.1
BPMN
In 2002, the Business Process Management Initiative (BPMI) developed the Business Process Modeling Notation (BPMN) as a new graphical notation for modeling business processes. One of the particular design goals of the BPMN was that it should be easy to understand for all involved stakeholders, i.e., both for the business and the IT people. Later on, in 2005, the BPMI merged with the Object Management Group (OMG), an influential standardization body which also maintains the Unified Modeling Standard UML. Similar to UML, the BPMN specification describes the semantics of the graphical symbols that can be used 3 4
http://docs.oasis-open.org/wsbpel/2.0/OS/wsbpel-v2.0-OS.html http://www.omg.org/spec/BPMN/, current version is 1.2
228
D. Jannach and A. Gut
in the diagrams only in an informal way. Note that in the past, approaches that are based on informal semantics have been quite successful and are wide-spread in industry despite the problems that are caused by missing formal semantics.
Fig. 1. Process example from BPMN specification
Figure 1 shows a simple example process model from the BPMN 1.1. specification. The main modeling elements of the notation are activities (tasks), sequences, so-called gateways (for modeling decisions or parallel execution), loops and events such as incoming messages or – as in the example – a timer event. In particular, with the help of gateways and events, alternative execution paths for expected problems can be – at the price of increased model complexity – quite easily modeled in BPMN. However, BPMN also includes additional notational elements to model error handling behavior. A so-called “Exception Flow” can be seen in Figure 1, where the normal flow in the loop is interrupted when some time limit is exceeded. Note that although this is called an Exception Flow in BPMN, in this case it corresponds more to a construct for loop-termination or timeout treatment than to exception handling in programming languages. Figure 2, also taken from the specification, shows a process model that includes more built-in error handling modeling elements of BPMN. Error handling is based on transactions (modeled as sub-processes), error and exception events as well as on compensation actions. The intended business logic of the example from the travel planning domain is as follows. The process foresees that the buyer will only be charged, when both the flight and the hotel reservations are successful. If one of the bookings fails, the transaction should be rolled back by executing “undo” (cancel) actions for already completed tasks. In the model, each action within the transaction is therefore associated with a compensating task (marked with the fast rewind symbol). The scope of the transaction is graphically defined through the double lined border. The start of a rollback process is initiated when a “Cancel” event happens, which can be triggered
Exception Handling in Web Service Processes
229
Fig. 2. Error handling example from BPMN specification
by a “Cancel End Event” within the transaction or, as in this example, when a cancel message is received from the underlying transaction layer (e.g., WSTransaction5 or the Business Transaction Protocol6 ). If such an event happens, a “Cancel Intermediate Event” (cross in a circle) directs the execution flow to the activity for sending an unavailability notice. This action is executed when all cancelation actions have been completed. The process model in Figure 2 also contains an “Error Intermediate Event” (lightning in circle) for situations, in which “something went terribly wrong”. In this case, no compensation is initiated and the flow is directly diverted to the error handling activity. Beside these intuitive pre-defined and dedicated error-handling mechanisms, the concept of “ad-hoc” processes can in principle also be used to provide for flexible error handling in case of an exception. An ad-hoc process is defined as a group of activities that have no pre-defined execution order, i.e., only the actual performers (determined at run time) decide on the execution flow. Thus, one can design some pre-defined ways of reacting on problems and delay the decision of what to do until the problem actually occurs. Since processes are typically long-running and often require human interaction, this mechanism can be used to incorporate more flexibility in the recovery phase. This mechanism might be particularly helpful when it would be too complex to model all possible problem causes and corresponding reactions, when some information is not available in electronic form, or when human decision taking is required. Discussion. The error-handling concepts are very general and built upon ideas that have already been developed in the context of WfMS, see e.g., [EL97] or 5 6
http://www.oasis-open.org/committees/ws-tx http://www.oasis-open.org/committees/business-transaction/
230
D. Jannach and A. Gut
[EL96], who propose sub-processes as transactions, backward recovery and undoing of already performed steps. Workflow Recovery in the sense of a partial backtrack and continuation of another execution path [EL96] can also be specified with BPMN with the above-described concepts. Remember that BPMN is only a graphical notation that has no formal semantics and its design is based on the idea that also business people can design the process models. In that context one might ask whether it is realistic that business people are capable of understanding the exception handling concepts. For IT people, transactions, events and handler routines are known concepts but for business analysts, extensive training might be necessary to use the concepts (if the used drawing tool supports them anyway). For these reasons, syntactically incorrect BPMN diagrams can often be found in practice. If the enhancement of processes with error handling behavior is thus left to the IT department, a part of the value of a language that can be used by both sides is lost. In that context, the fact that BPMN has no formal semantics and can only be translated or mapped to other representations such as WS-BPEL or XPDL7 can lead to additional problems when tool vendors interpret the mapping itself or the details of transactions, events and intended error-handling behavior in different ways. A recent approach toward achieving compatible error-handling behavior across different products and overcoming the problem that fault-tolerant models soon become very complex is described by Combi et al. in [CDP08]. Their work is based on modeling so-called triggers with the high-level Chimera-Exception language [CCPP99] to define the exception handling rules. These triggers are then automatically mapped to XPDL definitions, i.e., a compiler produces enriched XPDL models from the annotated process models that for instance contain an additional exception handling lane. Although Combi et al. report some improvements with respect to portability of the process models, they also mention that most of today’s commercial WfMSs do not fully comply to the XPDL standard yet or rely on proprietary language extensions, which hampers cross-product portability, in particular of fault-tolerant process models. 2.2
Yet Another Workflow Language – YAWL
One of the main issues of notations such as the BPMN is that they have no precisely defined formal semantics, which easily leads to different interpretations of the same standard by the tool vendors. One intuitive way of modeling the control flow of a workflow process with precise formal semantics are Petri Nets. Since their development in the 1960s, the basic scheme has been extended with time-related concepts and with distinguishable tokens that carry data (Colored Petri Nets). However, as argued in [Hof05], even such high-level Petri nets have some limitations when it comes to workflow modeling. One of the problems is related to the locality of events, i.e., tokens are only enabled when their directly connected input places are loaded. On the other hand, they can also only affect their direct successors. This aspect of locality is problematic, in particular when a cancelation procedure has to be executed in case of an external event such as 7
A BPMN serialization format by the WfMC, see http://www.wfmc.org/xpdl.html
Exception Handling in Web Service Processes
231
a timeout: When no other modeling constructs are available, one has to model a so-called “vacuum cleaner” to remove tokens from different places in the web, which in turn leads again to complex models. As an answer to this and other problems of high-level Petri nets, ter Hofstede proposes a different formalism called YAWL[Hof05]. The design of this language is based on the analysis of the functionality of various system and the formulation of commonly occurring workflow patterns. YAWL formalism itself uses extended workflow nets, which consist of tasks (activities) and conditions (similar to Petri Net places). The workflow nets can also be structured hierarchically. In the context of error handling, the cancelation pattern corresponds to the situation where an (external) event should cause the cancelation of individual running or scheduled activity or of the whole process.
Fig. 3. YAWL Cancel activity pattern
Figure 3 shows the “Cancel Activity/Region” pattern from [Hof05]. YAWL notation has to be read as follows: When the cancel task is executed, which happens when one of the booking fails, all tokes within the region surrounded by the dashed lane (cancelation region) are removed, i.e., the other booking tasks are also cancelled. With the help of this pattern, complex constructs for token-removal in different parts of the graph can be avoided. The cancelation pattern can be combined with the event-handling mechanism of YAWL to model expected exceptions. Events are simply modeled as special tasks: timed tasks (denoted with a T) execute automatically after a certain waiting time, event tasks (denoted with an E) wait for an incoming (external) event. Figure 4 (also from [Hof05]) shows how events and timers can be combined with cancelation. In that example, a time out will cancel the waiting for the payment. Vice versa, upon receipt of a payment, the timer will be canceled. While with the help of these constructs expected problems can explicitly be modeled with decision tasks, they cannot be used to model exceptions in the sense of programming languages, where the control flow is diverted automatically to some error-handling activities as for example in BPMN. Therefore, Russel et al. in [RvdAtH06] extended their library of workflow patterns with a set of additional exception handling patterns and corresponding notational elements for YAWL. The following general types of exceptions were identified: Work Item Failure (work item cannot progress for some reason), Deadline Expiry, Resource Unavailability, External Trigger (also from other process instances), and Constraint Violation (based on explicitly modeled invariants).
232
D. Jannach and A. Gut
Fig. 4. YAWL Cancel activity with events and timer
The exception patterns are based on a combination of three considerations: – What is the state of the work item in the work item life cycle? – How will already started workflow instances be handled? Options are continuing the instances or removing the current or all work items. – Which remedial actions are taken? Options are rollback, compensation, or “no-action”. Based on this characterization of exceptions, the authors propose to introduce additional symbols to the notation and in order to model so-called exception handling strategies. These strategies imply some sort of control-flows that contain both error-handling tasks as well as standard YAWL activities. The main point however is that the exception handling behavior is modeled separately (in an own sub-diagram) and only linked to the normal control flow. In addition, different exception handling flows can be designed that depend on the type of the exception (i.e., a work-item failure may require other actions than a deadline expiry) or even on different data values of the current work item. Based on this separation, not only the mixture of business logic and error handling is avoided, it also allows for easier verification and maintenance of the resulting models. A different method for exception handling called “Exlets” was proposed in the context of YAWL by Adams et al. in [AtHvdAE07]. They base their method on the concept of Worklets [AtHEvdA06], which was devised as a new approach to introduce additional flexibility in the workflow models. Worklets are small workflow processes, which are designed to handle specific (sub-)tasks in the context of another process. The main idea is that the “main” workflow process is not seen as a rigid prescription of the series of actions to be completed but rather as a guideline. Depending on the current situation during workflow execution, an individual (atomic) task of the main process can be substituted by a Worklet, i.e., the process control is handed over to the small Worklet process. The selection of the appropriate Worklet is driven by a decision engine that evaluates contextual selection rules, which are modeled as “Ripple Down Rules”[CJ90]. One particular advantage of the method is that the set of available Worklets (i.e., the repertoire of possible actions) can be extended even at run-time allowing for incremental workflow evolution. Conceptually, “Exlets” are very similar to “Worklets” with respect to their basic nature, i.e., they are small work processes that are dynamically invoked
Exception Handling in Web Service Processes
233
during the execution of some parent process and their selection is driven by decision rules8 . The main difference is that – depending on the workflow model – the Worklet selection process is triggered only by specific tasks, while checks for Exlet execution are run for every case. Technically, the so-called Exception Service[AtHvdAE07] can be implemented as a separate (service-oriented) component and be invoked from different workflow execution engines using defined interfaces for data mapping. If the Exlet invokes another Worklet (that e.g., cancels or undoes other tasks), this Worklet can again be monitored by the Exception Service. Although there are no limitations with respect to what can be modeled in an Exlet, it is proposed that an exception engine should also support the above mentioned Exception Patterns[RvdAtH06]. The system described in [AtHvdAE07], which was implemented in the YAWL-environment, supports seven of ten defined exception types: constraints that are checked before and after item and case execution, timeouts and externally triggered exceptions. Other mechanisms, which depend stronger on the internals of the workflow engine such as item abort, resource unavailability, or events that occur during work item execution, were not implemented in the system. Discussion. The original YAWL method only supported the Cancelation Pattern as a way of dealing with problematic situations during process execution. Later on, Exception Patterns were proposed to enhance YAWL in this context, mainly on the conceptual level. Only the last addition to YAWL based on the Worklet extension provides comprehensive error-handling support, which allows for easy modeling of re-do activities and alternative execution paths. In addition, a further main advantage of the Worklet-based exception modeling technique in YAWL is that the normal flow and the “exception flow” can be designed and evolved – even at run-time of a case – in separate models. On the other hand, due to the loose coupling, ensuring that the exception handling behavior is correct and works as intended, may become more complex. 2.3
WS-BPEL
Today, the Web Services Business Process Execution Language, short WS-BPEL, is the de-facto standard for modeling “executable” workflow models that are based on web services. The language (called BPEL in the following) supersedes two earlier languages for programming in the large: The Web Service Flow Language (WSFL) by IBM and Microsoft’s XLANG and was submitted to OASIS for standardization by IBM, Microsoft, SAP, Siebel and BEA in 2003. The current version (WS-BPEL 2.0) was released by OASIS in 2007. In BPEL, the flow of activities in a business process – which are implemented as web services – are described with the help of an XML format since BPEL itself has no graphical notation. Therefore a BPEL document contains several elements such as the list of activities and the corresponding control flow elements (if-else etc.), definitions of code blocks (called scopes) and variables that store instance data. Beside these elements, which can also be found in many programming 8
The Exlet process may again invoke a Worklet process with compensation tasks.
234
D. Jannach and A. Gut
Fig. 5. BPEL fault handling and compensation activities
languages, incoming messages and (asynchronous) events and the corresponding handler definitions play an important role in BPEL. Since BPEL was designed as a language for orchestrating (distributed) web services that can be provided by different partners, the language design naturally includes different mechanisms for handling errors and developing fault-tolerant applications. The basic error handling mechanism in BPEL are Fault Handlers. Conceptually, BPEL fault handlers are similar to exception handling blocks in programming languages, i.e., the code in the fault handling block is executed in case of an error and the block can also re-throw an exception so that it can be handled in an outer block. In addition, different fault handlers can be defined for different types of problems and can be attached to the various (nested) parts of the process: error-handling routines can be defined for a single activity, a scope, or the process itself. Note that in contrast to normal, single-threaded programs, parallel execution of different tasks is not uncommon in web service processes. Before the errorhandling procedure begins, all running activities of the affected scope have to be terminated first, which is implicitly done by a fault handler. Optionally, a Termination Handler can be used for some cleanup activities. Beside graceful termination and notification, the cancelation of already performed activities is another core element of the error-handling features of BPEL. To this purpose, the language supports the definition of explicit Compensation Handlers, which contain a list of activities to be executed to undo things. Thus, when an explicit compensate activity is invoked in the fault handler, the compensation handler for the affected scope and already performed activities will be executed. Figure 5 shows an example of BPEL’s fault handling mechanism9 : The sales business process consists of two activities, which are organized in own scopes: 9
Adapted from K¨ onig and Baretto’s WS-BPEL symposium presentation (2007), www.oasis-open.org/events/symposium/2007/slides/WS-BPELWorkshop.ppt
Exception Handling in Web Service Processes
235
charging the customer’s credit card and (electronic) shipment of the goods. Let us assume that a problem occurred in the second step. The ship goods activity fails and control is handed to the local fault handler attached to the scope. This fault handler propagates (re-throws) the error to the surrounding scope, which happens to be already the main process. This fault handler first executes a notification activity and than initiates the compensation process, i.e., it invokes the compensation handler of the already executed activity charge credit card, which contains the compensating task refund customer. Beside the fault handler, also the event processing mechanism of BPEL can on principle be used to incorporate fault tolerant behavior in the process models. BPEL supports two types of events. Alarm events are raised when a user-defined timer goes off, i.e., they can be used to model expected timeout exceptions. Message events, on the other hand, are triggered by inbound messages. They can for instance be used to capture failure notifications of a connected system. Discussion. BPEL basically supports the catch-and-handle approach to model error-handling behavior and follows the principle of nested exception blocks and compensation handlers. Based on these principles, a reaction also to unexpected problems is possible and not all types of possible problems have to be explicitly modeled within the normal flow. What cannot easily be expressed in BPEL is the strategy of ignoring, but logging problems. In some situations, some errors should be caught and someone should be notified, but the process should continue as planned. Once the control is handed to the fault handler in BPEL, however, all running activities are terminated. The only exception in BPEL is the suppressJoinFailure setting, which allows to quietly ignore false join conditions. Also the “redo” scenario is not well-supported in BPEL. Although it is of course possible to redo activities in a scope by copying the corresponding part of the process into the compensation handler, this leads to an undesired duplication of the process logic and increased model complexity. Because of these limitations of BPEL with respect to error handling, Modafferi et al. in [MMP06] propose to extend BPEL engines with “self-healing” mechanisms for error recovery (SH-BPEL). Their approach is based on pre-processing the model and using additional annotations in the process models from which standard-compliant, but more fault-tolerant, BPEL-definitions can be generated that support re-doing of a scope, rollback to a defined (save)point in the process, or the choice of an alternative execution path. Finally note that that BPEL has no graphical notation. This fact may be problematic when stakeholders from the business side should model the intended error-handling behavior of the process, since the XML-based format and the underlying language principles are not easy to comprehend by non IT-experts. 2.4
Other “Programming in the Large” Modeling Languages
In industrial practice, many other (graphical) languages are in use for modeling business processes. However, many of them are designed for special purposes such as business process analysis, simulation and system design and will not be
236
D. Jannach and A. Gut
discussed in detail here, since they either are proprietary, have no defined formal semantics or are not actually designed for web service process execution. Examples are the ADONIS standard language (discussed for instance in [Gla08]) or Event-Driven Process Chains [STA05], notations which are used for process analysis and simulation purposes and do not have special constructs for exception handling. In some cases, UML activity diagrams are also used to model business processes. In recent versions, UML does support the concept of exception handling and introduces an own graphical symbol10 . While this is a feature that is probably inspired by BPEL, it is the only error-handling mechanism in UML Activity Diagrams known to the authors. Moreover, the missing formal semantics of UML leaves room for interpretations. Despite this fact, in [GM06], a first proposal was made to transform Activity Diagrams into executable XPDL documents. However, the above-mentioned exception handling notation is not supported in this approach. Finally, a notation for modeling exception handing in UML Sequence Diagrams was proposed in [HH06] but is not yet contained in the standards.
3
Choreography Models
Modern computerized business processes often need to integrate the services of different business partners in a single workflow, e.g., on the basis of web service and BPEL technology. The coordination model of BPEL is called “orchestration”, i.e., there is one partner that owns and steers the process and whose workflow engine invokes the partner systems. The other service co-ordination model is that of “choreography”. In this model, all services are equal (i.e., there is no dedicated coordinator) and only some distinguished aspects of a service are visible to the others. A choreography model therefore only contains these external service descriptions as well as agreements between them that may relate to sequencing, timed behavior, or transactional aspects. Early approaches toward web service choreography were based on Behavioral Interfaces, which describe the observable behavior of one single service in a choreography. The Web Service Choreography Interface (WSCI)11 is an example of such a language for describing the externally visible behavior of a web service. This XML-based language was submitted to W3C already in 2002. The proposal reached the status of a note and was not developed further. Later on, however, the main concepts of choreography were integrated into the Web Service Choreography Description Language (WS-CDL)12 . The design of WSCI demands that the observable behavior of services is defined declaratively through the description of the service interface and the temporal and logical constraints on the interaction with other services. The 10 11 12
OMG UML 2.1 Superstructure Specification, p. 361. http://www.w3.org/TR/wsci/ http://www.w3.org/TR/ws-cdl-10/
Exception Handling in Web Service Processes
237
descriptions themselves are not executable and it is left to the web service designer how the behavior is actually implemented. Technically, WSCI is based on an extension of the standard Web Service Description Language (WSDL)13 . While WSDL can only be used to describe the different messages of a service (including their technical characteristics), a WSCI document can further extend these definitions, e.g., with information about how newly arriving messages are related with previous messages. Beside message dependencies, WSCI also supports the description of the errorhandling behavior of a particular service. Possible errors are, e.g., WSDL error messages, messages with wrong format, timeouts and errors raised by the service itself. The following listing illustrates the timeout behavior of an online payment service and specifies that when the allowed time for entering the identification number has passed, a particular compensation service has to be executed in order to undo the transaction. <exception>
As can be seen in the example, WSCI supports similar error-handling concepts as “programming in the large” approaches such as BPEL, i.e., events, transactions and compensation. The difference is that no global view and coordination exists and the descriptions are only defining the behavior of a single service (e.g., which compensating actions are invoked in case of an exception) in a declarative manner. In contrast to WSCI, which only describes the behavioral interface of a single service, WS-CDL aims to extend this model and support a global (but not centrally coordinated) view on the choreography. Since 2005, WS-CDL has the status of a candidate recommendation at W3C. A description of a choreography in WS-CDL has in some respect a similar structure as XPDL or BPEL process definitions. It does not only describe the roles and relationships of the different partners in the process, but also the different activities that the partners can execute along with their associated control flow structures in the form of WorkUnits. This concept of WorkUnits can also be used to describe the error-handling behavior in the choreography. Exceptions that occur at run-time can be caught by so-called ExceptionBlocks and be handled in exception workunits. Discussion. Although WS-CDL can be used to describe similar aspects of a process as BPEL, BPMN or XPDL, the goal of the WS-CDL designers is to have descriptions that are even more independent from the actual implementation below. For example, the individual actions do not have to be web services and could also be Java or C# programs or executable BPEL processes. 13
http://www.w3.org/TR/wsdl
238
D. Jannach and A. Gut
While this independence from technical details is desirable in particular from an academic perspective, choreography approaches suffer from their limited acceptance in practical environments. Even the standardization efforts at W3C have come to a halt in the last years, probably because BPEL in the current version contains the concept of abstract processes, which supports the description of interfaces without requiring a specification of the concrete execution logic. Recently, approaches toward a further enhancement of BPEL (BPEL4Chor) with features required for choreography modeling have been proposed in [DKLW07]. Since BPEL is today’s de-facto standard for modeling web service processes, we expect that languages such as WS-CDL will be replaced in the future by choreography-enhanced process modeling languages.
4
Toward More Intelligent Error Handling Methods
The approaches discussed so far are mostly based on the throw-and-catch error handling pattern as well as on transaction rollbacks or compensation actions, which are explicitly modeled by the designer of the process. In practical applications, such approaches have their limitations in particular when it comes to the problem of recovery or repair, i.e., when the problem is to complete the process despite the presence of an error. If, for instance, one of the involved web services is not reachable, in many domains the option exists to use another service that provides the same functionality; another option in that situation could be to try a different set of activities to reach the same goal (think, e.g., of different payment services). With current approaches it is possible to model such recovery actions only through explicit alternative execution paths or compensation actions. Since, in reality, many or all activities of a process may fail and every single failure may require different recovery activities, the resulting process models soon become too complex. As a result, process designers will only model repair actions for a few of the possible error cases and will accept a rollback for the other situations. In the following, we will summarize recent approaches toward more intelligent error-handling techniques that support the precise identification of the problem, the automated discovery of alternative web services, or the dynamic computation of suitable recovery plans. 4.1
Error Diagnosis and Repair
In order to intelligently handle a problem during process execution, e.g., through the usage of an alternative service that provides the same functionality, one goal could be to find out in a first step, which of the activities could have caused the problem. Only at the first glance, this looks like an easy problem, which can be solved by modeling an exception handler for every single task in the process. Note, however, that a service can also raise an exception due to wrong input data calculated by an erroneous preceding activity. Thus, the goal of diagnosis techniques is to identify a set of already executed activities that could have potentially contributed to the observed problem.
Exception Handling in Web Service Processes
239
Most recent approaches toward advanced error analysis for workflows are based on Model-based Diagnosis(MBD) [Rei87] techniques. Figure 6 shows the basic principle of Model-Based Diagnosis, which was originally developed to diagnose physical systems such as electronic circuits but later on also used to debug software systems, see e.g., [FSW99], [Wot02], or [FFJS04].
Fig. 6. Principle of Model-based Diagnosis
As the name suggests, the approach is based on a model of the real-world system, which is often referred to as SD, the System Description. In an electronic circuit diagnosis scenario, SD contains the list of elements of the circuit (e.g., the different gates), their interconnections and also a description of the relevant physical laws. Based on this model, the system behavior can be simulated. In the domain of electronic circuits, we would for example test different inputs to the circuit and observe how the values are propagated to the output lines. Basically, the diagnosis approach consists of a comparison of the observed behavior of the real system with the expected behavior, which we can determine with the help of a simulation run. If there are discrepancies, we know that that there is some error in the real system (assuming that the model is correct and captures reality properly). The problem now is to find out, which of the components of the system causes the error and should be replaced. In MBDliterature, a set of components of SD that – if assumed to be faulty – explain the observed behavior of the system, is called a diagnosis. Typically, we are only interested in minimal diagnoses, i.e., we do not want to include components in the diagnosis that are not relevant for explaining the error. The set of (minimal) diagnoses for a given problem can for instance be calculated with Reiter’s Hitting Set algorithm[Rei87]. Figure 6 also indicates that finding a diagnosis is normally not the ultimate purpose of model-based problem analysis. Commonly, one is interested in repairing the system, e.g., by replacing individual components. Note that in typical problem settings there are usually several diagnosis candidates that can explain the observed faulty behavior of a process. The (automated) selection of the most probable or appropriate one for a given situation is an open and critical issue in many MBD-based approaches, because also the corresponding repair actions can be different for different diagnoses.
240
D. Jannach and A. Gut
Diagnosing BPEL Process through Dependency Analysis. In [YD07], Yan and Dague propose a method for monitoring and diagnosing web service processes based on the MBD paradigm. In order to apply MBD techniques, a formal model of the web service process has to be developed. Therefore, the existing process definition (given in the BPEL4WS language) is first transformed into a Discrete Event System, that is, into a synchronized automaton, where the states are defined by the values of the variables of the process. State transitions are caused by variable changes that are in turn triggered by actions such as variable assignments, message emissions or receptions. The model of non-abnormal behavior for every activity in usual MBD notation is: ¬ab(A) ∧ ¬ab(A.inputs) ⇒ ¬ab(A.outputs)
(1)
The non-abnormal (correct) model states that if activity A behaves correctly (i.e., is not abnormal ) and the inputs to A are also correct, then also the outputs of activity A, i.e., the resulting variables changes, are correct. Once an exception occurs during run-time execution, the diagnosis phase is initiated and the process starts comparing the observed behavior (given by the execution logs of the engine and the corresponding execution “trajectory”) with the expected behavior (as described by the automata and the given variable dependencies).14 At run-time, the diagnosis engine is called when an exception occurs. The subsequent diagnostic process is based on three pillars. – Dependency analysis: Similar to Program Slicing [Wei81], a software debugging technique, a (static) analysis is made of how variable values can be affected by activities. To that purpose, activities are viewed as functions that accept input variables (defined as global BPEL variables) and produce output variables. These relationships can be derived from BPEL’s correlation sets, which describe how arriving messages are related, and are used to model the non-abnormal behavior of an activity more precisely, i.e. ¬ab(A.input) ⇒ ¬ab(A.output), if U til(A, A.input, A.output)
(2)
The function Util(A,V1,V2) denotes that through activity A output variable V 2 is correlated with the input variable V 1. – Trajectory reconstruction: The execution trajectories are recovered from the BPEL logs by synchronizing the observations (events and exceptions in the log) with the system description, i.e., the formal execution model. Figure 7 shows a simple process model with parallel activity execution (left hand side) and two execution trajectories. One corresponds to a possible successful execution path; in the other, an exception is thrown before the process finishes. 14
Note that to that purpose the BPEL engine must provide additional logging support and e.g., record incoming and outgoing SOAP messages as well as events and exceptions.
Exception Handling in Web Service Processes
241
Fig. 7. Model and two execution trajectories
– Accountability analysis: In this step, the possible “culprits” (i.e., activities that may have caused the problem) of the exception are determined. The set of candidates is limited to those activities that have been already executed and to those who possibly have affected the input variables of the failed activity (which can also be the only responsible activity). This calculation is based on applying responsibility propagation rules through the given trajectory. If two or more exceptions happen simultaneously, the analysis is made in parallel and the resulting diagnoses are then combined. In case of sequential (chained) exceptions, only the first exception is analyzed. Discussion. In principle, the approach is very similar to the well-known Program Slicing technique (with some slight differences). Thus, it applies a proven technique for ordinary programs to the “programming in the large” world. A deeper discussion of the relationship between Model-based Diagnosis for Software Debugging and Program Slicing can be found in [Wot02], who applies an even more elaborate diagnosis technique for Java programs. One limitation of the approach by [YD07] is that the underlying BPEL engine must support the extended logging mechanisms, which are required to spot the problematic activity. In addition, the approach is also limited to certain types of failures. Time-out exceptions, for instance, which are probably caused by problems in a remote system, cannot be analyzed by the proposed technique. Diagnosing Workflow Processes from “First Principles”. In order to overcome the problems of the approach from [YD07], Friedrich and Ivanchenko in [FI08a] propose a more fundamental approach to model-based debugging of workflow executions. Their problem formalization of the workflow semantics is not based on a specific language such as BPEL, but rather on implementation independent Colored Petri Nets (CPN) and they support common workflow control-flow patterns such as sequences, splits, choices, merges and joins. Loops (cycles) in the models are, however, not yet supported by their approach. In order to properly model variable dependencies and process execution states, the authors extend CPNs with so-called (data) objects and call the models Workflow/Data-Petri-Nets (WFD). In addition to these data objects, the authors also make use of guard transitions (activities) that are similar to assertions in programming languages, i.e., at some given points during execution, certain Boolean expressions are checked. In the proposed formalism, a workflow exception therefore corresponds to a situation in which an assertion/guard expression evaluates to false. Together, all these mechanisms (data dependencies, guard transitions and the execution history of individual process instances) serve as a
242
D. Jannach and A. Gut
basis for identifying the potential sources of an exception more precisely in the WFD approach [FI08a]. In order to diagnose workflow executions from “first-principles”, the extended CPN model (WF) and the execution behavior of workflows are expressed in a logical representation, i.e., by a set of First-Order Logic sentences. These descriptions serve as a basis for a precise formal and consistency-based characterization of diagnoses of workflow executions which can be calculated with the help of Reiter’s Hitting Set algorithm [Rei87] and a theorem prover for consistency checking. In a later work, Friedrich et al. [FMS10] reformulate the workflow diagnosis problem using on a different logic-based formalism and encoding. Their approach includes different improvements over the basic model-based debugging method and supports diagnosis-repair settings in which activities are potentially re-executed by the execution engine in order to successfully complete a process despite the existence of a fault. In contrast to dependency-based approaches, their method for identifying incorrect activities in process execution has the advantage that it does not require precise behavioral activities descriptions (which are often not available in loosely coupled systems based on web services). Discussion. When compared with the work in [YD07], which represents more a Program Slicing than a Diagnosis technique, both share the idea of utilizing a formal model of workflow executions and the analysis of data dependencies. The approach of Friedrich and Ivanchenko aims to tackle the diagnosis problem in a more fundamental manner and to characterize diagnoses for workflow execution in general (and not dependent on dependency propagation). While there are some simplifying assumptions of their approach (e.g., loops are allowed in the model but can be dealt with using loop unfolding techniques), the authors show in [FMS10] that a variety of real-world problems can be modeled and solved with their formalism. In addition to the mentioned approach of [YD07], the theory developed in [FI08a] also lays the foundation for the automated computation of possible repair actions for problematic workflow instances, see [FI08b] and [FFM+ 10]. Distributed Diagnosis of Web Services. Yet another approach toward modelbased workflow problem diagnosis that goes beyond today’s standard throw-andcatch scheme, is proposed by Ardissono et al. in [ACG+ 05] and [AFG+ 06]. Their work is driven by the fact that web service processes often span several companies and that the individual elements of the global overall process can be seen as whole sub-processes that are executed at a partner system. Thus, a central point of problem analysis cannot easily be installed, because, e.g., the participating partners may not be willing to share all the internal information required for diagnosis to the coordinating partner. Therefore, the authors propose a model for distributed error handling as shown in Figure 8. It is assumed that every participant of the distributed web service process has a local diagnoser component, which uses the log information of the local web service to diagnose the reasons for the problem. Note however that a failure found in a service is possibly caused by an error in another service. Therefore, a global
Exception Handling in Web Service Processes
243
Fig. 8. Local and global diagnosers (adapted from [AFG+ 06])
diagnoser is introduced, which can combine the information of the individual local diagnosers into a global hypothesis about the reasons of the problem; the communication itself is based again on web service technology and a defined message exchange protocol. Overall, the main advantage of the approach is that the central diagnoser does not have to be aware of the internals of the other participating web services. Even more, the global component does not necessarily need to know in advance how the different services interact with each other. Discussion. Similar to the other works, in the approach of Ardissono et al., activities are viewed as components of a system that transform inputs to outputs and which can have the behavioral modes ok and ab(normal). In particular, every activity has three subcomponents that can fail as the authors distinguish between three types of variables: Variables, whose values should be simply forwarded, variables that are newly created through the activity and variables, whose values are changed by the component. In that respect, the model is a bit more detailed than the more simple input-output model of [YD07]. Details of how activity execution chains in the sub-processes are handled (e.g., based on trajectories as in [YD07]) are not given. On the level of the infrastructure, the approach from Ardissono et al. is based on the assumption that every local service, which is for instance implemented as orchestrated BPEL process, communicates with the local diagnoser. For that purpose, the local web service engine has to be extended and – in case of BPEL processes – the fault handlers of the model have to be modified, so that the diagnoser is appropriately informed whenever a local exception occurs. Overall, techniques from model-based diagnosis appear to be a promising way to deal with exceptions in web service processes, in particular as the basic mechanisms in many cases are to a large extent independent of the protocols and engines that are used in practical environments. What remains open in all discussed approaches is the question of computational costs for real-world web service processes. While it is known that the computation of diagnosis is in general a computationally complex problem, it would be interesting to know
244
D. Jannach and A. Gut
whether the proposed methods are applicable to process definitions of limited complexity that can be found in the real world. Repair Planning / Contingency Planning. Finding out which part of a web service process failed or causes a problem is helpful, when the goal is to exclude a certain service from being used in future process instances or when alternative services that provide the same functionality can dynamically be incorporated into the processes. In many cases, however, this repair scheme, which can be summarized as “find an alternative service and re-try” is not appropriate. It would therefore be desirable to have an agent who can determine dynamically whether a broken workflow instance can still be successfully completed, e.g., by executing a set of alternative activities. In other words, the agent needs to find a repair plan to complete the service or otherwise abort it. In [FI08b], Friedrich and Ivanchenko propose such a method for computing repair plans based on the workflow formalism by [RD98], in which a workflow is represented by a directed acyclic graph of control nodes and activity nodes together with a set of associated objects. A process that terminates normally results in a certain configuration of the output objects. When an error occurs, the problem is to find a repair plan (which is itself a workflow) that leads to a configuration at the endpoint that is equal to the configuration that would appear after normal process execution. The repair approach in [FI08b] is based on the usage of common repair actions such as (re-)execution (suitable, e.g., to deal with transient failures), activity substitution and compensation. In order to deal in particular with transient failures, the concept of time has to be incorporated into the repair model. The starting point for repair planning is therefore a data record that describes which activity failed at which point in time15 . Repair planning itself is based on a logical formalization of the workflow process model, the effects of the different repair actions, the time model and the workflow execution state. The authors of [FI08b] classify their repair planning procedure as “conditional planning with background theories”. The logical formalism was chosen in a way that it can be handled by the dlv disjunctive logic programming system [LPF+ 06]. How this system can be used for knowledge-based planning has first been shown by Eiter et al. in [EFL+ 04]. Later on an extension was developed to support (optimistic) conditional planning was later on developed by [vNEV07]. [FI08b] finally expand these ideas in a way guaranteeing that contingency plans always lead to successful workflow completions. Discussion. An evaluation of various automatically generated test cases of small to medium sized workflow processes indicates that the method is generally applicable to real-world repair problems. The examined processes consisted of up to a few dozen activities. Computation times on a standard desktop PC remained less than a half-minute for most of the test cases, which is acceptable, 15
Note that this information could be derived from a diagnosis step that precedes the repair phase.
Exception Handling in Web Service Processes
245
taking into account that web service processes are typically longer lasting and have no tight time constraints attached. However, what has not been discussed are limitations of the approach, such as the problem of handling loops in the process models. It can be assumed that repair plans for such processes cannot be computed with that method yet (compare the work of the same authors in [FI08a]). Another aspect, which may constrain the applicability of the method in real world concerns the question whether one can assume that an in-depth description of the precise effects of individual activities (web services) is available in practical settings. 4.2
Semantic Web Technology
Replacing a transiently or permanently failing service of a process chain with an alternative service that provides the same functionality was already mentioned as a possible repair action in previous sections. Recent technology requires that the set of alternatives is known in advance and is described explicitly in the process model, e.g., in the exception handler of an individual action. Still, it would be desirable that alternative services could automatically be discovered and used by an intelligent system in case of a problem. Consider, for instance, the problem of booking a travel arrangement automatically which involves various web services and activities such as flight or hotel reservations (compare [MSZ01]). If a particular online hotel booking service fails, the process should not be halted but the booking functionality of another e-tourism platform should be used instead. If we do not want to model the alternative services or the service discovery process explicitly, a more generic mechanism is required and the particular problem is to find a service (or a set of services) that provide the same – or at least a sufficiently similar – functionality as the failing service that caused the problem. Today’s standard technologies for describing web services such as WSDL allow us to model a service and its interface on the syntactical level, i.e., they can be used to list the set of known operations and their parameters. However, these technologies cannot be used to describe the semantics of the service, i.e., the effects of invoking the service. Therefore, in order to allow for automated discovery or – which is even more complex – the automatic composition of service chains to complete an otherwise failing process, mechanisms for precisely describing web service semantics are required. In a broader context, the idea of annotating web resources in a way that they can be located intelligently or combined by a computerized system, is the core vision of the Semantic Web [BLHL01]. In recent years, significant research efforts went into the realization of this vision and lead to the development of a the Web Ontology Language OWL adopted by W3C16 . Shared ontologies are the basis of the Semantic Web and define the set of concepts that can be used to annotate resources in the web, i.e., they help us to insure that the same terms and vocabulary is used and interoperability is 16
http://www.w3.org/2004/OWL/
246
D. Jannach and A. Gut
guaranteed. OWL itself is a formal, XML-based and machine-processible language that can be used to describe specific domain ontologies. An (OWL-based) ontology for the above mentioned travel-planning problem would for instance define the concepts “flight” or “hotel” and probably also describe that a “domestic flight” is a special subtype of a flight. In the context of web service discovery, we could therefore try to extend the existing syntactic WSDL descriptions with semantic annotations, e.g., use the agreed-upon term “Amsterdam hotel booking” to denote that our service provides that functionality. Whenever a service fails in a process chain with this annotation, an alternative service with the same or a comparable annotation – according to a common ontology – could be invoked, if the parameter lists are somehow compatible. If a simple replacement of a service is not sufficient and a different chain of service invocations is required to achieve the original goal, a different way of describing the web service semantics is required. Note that the computation of diagnoses and associated repair plans was also the goal of the approaches of [FI08b] or [YD07] described above. In these approaches, the required execution semantics were described in terms of the variable dependencies or the effects of activities on data objects. However, this representation mechanism is rather technically oriented and not well-suited for sharing execution semantics because modeling is done on the level of variables and their dependencies and not the level of a more abstract domain ontology. A more general and implementation-independent way of capturing web service semantics is to view a service as an action in a STRIPS-style [FN95] planning problem from the field of Artificial Intelligence (AI) and correspondingly model a service through its inputs, outputs, preconditions, and effects (IOPE), see also [MSZ01]. How such a technology can potentially help us to react on workflow exceptions more intelligently, will be described later on. Web Service Discovery. In this subsection we will first sketch the opportunities and open challenges of automated web service discovery, which – as mentioned above – can also be seen as a mechanism to finding alternatives for a transiently or permanently failing web service. UDDI (Universal Description, Discovery and Integration) is a standard that can be used for publishing and locating standard WSDL-based web services in a company-wide or global registry and was originally designed with the concept of service-brokering in mind. Consequently, “matchmaking” between service providers and consumers was a central functionality that should be provided by such an exchange platform. However, the mechanisms provided by UDDI in that context are rather limited and mainly consist of API-functions for locating a service by name, through keyword-matching or category-based search. Beside names and categorizations, web service descriptions can only be given in plain text or keywords. Automatic matchmaking is therefore not easily possible as the information can be incomplete, misleading, contain ambiguous words and so forth. Even the use of classification schemes such as UNSPSC17 may not 17
http://www.unspsc.org/
Exception Handling in Web Service Processes
247
be sufficient because different schemes may be used and these schemes are not compatible. In [TAH07], Tsetos et al. summarize the key innovations and additional components that are required for semantic web service discovery and also propose different possible architectures. The concept of Service Annotation Ontologies (SAO), which can be used to precisely specify the service capabilities, is at the core of their considerations. One proposal for a standardized language for describing web service semantics in this form is OWL-S which was submitted to W3C for standardization in 2004. Note that competing proposals such as WSDL-S or WSMO18 were submitted to W3C. In this chapter, we focus on OWL-S because for different reasons. First, OWL-S is designed in a way that the semantic annotations can contain concepts from domain-specific OWL-based ontologies. At the same time, OWL-S also allows us to “ground” an action and technically bind it to a standard WSDL- and SOAP-based web service. According to the proposal of [TAH07], an extended web service discovery architecture can make use of the existing UDDI infrastructure, but the registry entries should have pointers to the unambiguous semantic descriptions expressed in the annotation language, i.e., the “Service Advertisements” should contain semantic descriptions. If the “Service Requests” are also stated in the same way, i.e., when the desired service capabilities are described in a semantically precise way, advanced matchmaking algorithms can be employed to decide whether a given service matches the requirements. Since exact matches cannot always be expected, usually a measure for the “Degree of Match” (DoM) is introduced. Such a measure can be based on various aspects such as the overlap of concepts used in the advertisement and the request or the placement of the services in a classification hierarchy. Note that the DoM measure can not only be used for service discovery (in the sense of matching) but also for a subsequent ranking step, which is also common in such matchmaking architectures [KSF07]. Finally – besides sophisticated matchmaking algorithms – also modern ontology integration and mediation techniques can be utilized to align ontologies of different user communities; for an overview see, e.g., [ES07]. Technically, different architectures are possible. The semantic web service discovery system can either be centralized (e.g., as an extended UDDI service) or as a peer-to-peer system, in which every participant can act as a provider or requester of a service. For a centralized version, again different architectural choices are possible. In one proposal of [TAH07], the matching algorithms are themselves provided as services. Depending on the type of the request – keywordbased or semantics-based – the corresponding matching service and algorithm is selected dynamically. Web Service Composition and Re-planning. Once semantic descriptions of individual services are available, it may not only be desirable to locate a single 18
http://www.w3.org/Submission/OWL-S/, http://www.w3.org/Submission/WSDL-S/, http://www.w3.org/Submission/WSMO/
248
D. Jannach and A. Gut
service but probably also to find a chain of services to reach a certain process goal or – corresponding to the focus of this chapter – find a repair plan to recover from a workflow exception. This process of deriving execution chains automatically is called web service composition and is often based on AI planning technology, see, e.g., [SPW+ 04] or [HBP07]. Note that although the idea of deriving web service process chains automatically was the driving scenario in the beginning of Semantic Web Services research (see, e.g., the paper of McIlraith et al. in 2001 [MSZ01]), still no reports on practical business implementations can be found until today. One system, in which Semantic Web Services and AI planning were combined successfully in the domain of server-side multimedia adaptation is reported in [JLTH06]. In this work, the goal was to design an open and extensible architecture for a multimedia server which has two basic features: First, it should be capable of integrating multimedia adaptation services and tools of different vendors and provides. Second, the server should be able to combine the different services into an invocation chain that converts a given resource (such as a video) from the given format to the desired format if a single-step adaptation is not possible. In order to achieve these goals, the available adaptation operations (ranging from simple image resizing to complex content- and environment based video transformations) are modeled as Semantic Web Services (SWS) using OWL-S. An action description comprises the required in- and outputs as well as the preconditions for applying a transformation, i.e., the acceptable input formats, and the effects of the transformation that describe how the resource is changed after applying the operator. Thus, given the set of actions, a “start state”, which corresponds to a description of the resource to be transformed, and a “goal state”, that corresponds to a description of how the end user wants to consume the resource, any standard state-space planning engine can be used to calculate a corresponding adaptation chain. Note that in any domain in which Semantic Web Service technology should be exploited, the participants have to agree on a shared domain ontology determining, for example, how the different actions are described. The establishment of such an agreed-upon ontology often is a hampering factor when it comes to applications that should be open to a broad community of participants. In the domain of multimedia adaptation, this problem could be solved by interpreting the existing ISO/MPEG standards (MPEG-7, MPEG-21) as a domain ontology, as these standards exactly prescribe how the different features of a resource (such as color, size etc.) have to be described [JL07]. While the works described in [JLTH06] and [JL07] were not primarily designed for exception handling in web service processes, this example helps us to illustrate how SWS technology can serve as an enabler for more advanced errorrecovery techniques. By design, the system can handle additions and removals of individual services. As long as there exists a possible execution chain that uses the currently available services, the system will find it due to the soundness and completeness of the underlying planning algorithms. If a service transiently or
Exception Handling in Web Service Processes
249
permanently is not available, alternative services can be retrieved automatically based on the semantic annotations of their effects. Moreover, since the composition (and repair) problem is reduced to a wellunderstood AI planning problem, the known techniques for contingency planning [PC96] or re-planning can be applied, see e.g., [Pee05] for a replanning algorithm for web services. The most simple method would be to halt the execution chain when a problem is detected and generate a new plan that leads from the intermediate state to the original goal state. More elaborate replanning and rescheduling techniques that for instance aim to produce a new plan which has minimal deviations from the original plan are also possible, see [KJF07] for an example from the area of process scheduling. Discussion. Semantic Web technology can be seen as a promising base technology for future, more intelligent error-handling techniques in web service processes which are based on automated service discovery or dynamic service composition. However, several challenges remain in practical environments. First, although OWL today is the the de-facto standard for ontology representation, there exists no agreed-upon standard for describing web service semantics. The different proposals (OWL-S, WSMO, WSDL-S) have been submitted for standardization already in 2004 and 2005, but no “winner” has been determined yet. Beside this problem of standardizing the language, most applications domains suffer from the problem that no shared domain ontology exists or that several competing pseudo-standards exist. Consider again the example from the travel planning domain. While there exist several classification schemes such as UNSPSC, the sector is far from having a common ontology that unambiguously describes what the effects of a “flight reservation” or a “hotel booking” are. Thus, semantics-based error recovery is only possible if (a) the involved companies tightly co-operate and agree on a detailed shared ontology or (b) domainspecific standards such as the MPEG-7 specification in the context of the work in [JL07] already exist. Next, as mentioned in [Sri03], there are some fundamental differences between BPEL-based process models and the representation mechanism used in AI planning. BPEL processes describe a global picture of the flow of activities, which may contain complex control structures and so forth. With AI-planning techniques, one needs to model individual activities and think of appropriate preconditions and effects that are required to express rather simple precedence constraints. Another open issue is the question of how to model side-constraints that have to apply to the overall process, which cannot be easily captured by the IOPE-style action descriptions. There might be for example several online airline booking services available, which are basically exchangeable with respect to their functionality. Due to internal business rules, it might however only be allowed to use a certain subset of those booking services in certain situations. Finally, also questions of scalability of the planning and re-planning approaches to web service composition and the computation of repair plans have not been answered so far. Some more practical problems of web service composition (which
250
D. Jannach and A. Gut
are not even semantically-enriched ones) are also discussed in the “reality-check” paper of Lu et al. in [LYRS07].
5
Summary
In contrast to classical workflow processes, today’s web service processes require additional attention with respect to exception handling and recovery because of their inherent distributed and open nature. Problematic cases for instance include situations where a remote service is simply unreachable or where detailed information about a remote exception is simply unavailable. In this chapter, we have discussed various mechanisms to handle exceptions in web service processes. Today’s de-facto process modeling “standard” technology (BPEL and BPMN) is influenced by research in the area of earlier workflow systems and relies on the “catch-and-handle” exception handling pattern known from programming languages as well as on the concepts of transactions and compensating actions. In order to support modular and compact process models that do not mix the regular execution flow with error-handling procedures, some languages such as YAWL provide special model structuring mechanisms and the opportunity to dynamically change the execution flow at run-time. In the final sections, an outline of more intelligent error-handling mechanism of current web service process technology was given. Most recent approaches aim to avoid explicit error-handling models and exploit additional knowledge to calculate problem recovery actions automatically. These recent proposals include for example more advanced debugging aids based on Model-based Diagnosis as well as techniques for the computation of corresponding repair plans. In addition, the opportunities for automated web service discovery and re-planning as a special form of service composition that come along with the emergence of Semantic Web technology have been discussed in this chapter.
References [ACG+ 05]
[AFG+ 06]
[AtHEvdA06]
[AtHvdAE07]
[BLHL01]
Ardissono, L., Console, L., Goy, A., Petrone, G., Picardi, C., Segnan, M., Dupre, D.T.: Enhancing web services with diagnostic capabilities. In: Proc. 3rd Europ. Conf. on Web Services, V¨ axj¨ o, Sweden, p. 182 (2005) Ardissono, L., Furnari, R., Goy, A., Petrone, G., Segnan, M.: Fault tolerant web service orchestration by means of diagnosis. In: Gruhn, V., Oquendo, F. (eds.) EWSA 2006. LNCS, vol. 4344, pp. 2–16. Springer, Heidelberg (2006) Adams, M., ter Hofstede, A.H.M., Edmond, D., van der Aalst, W.M.P.: Worklets: A service-oriented implementation of dynamic flexibility in workflows. In: Proc. CooPIS 2006, pp. 291–308 (2006) Adams, M., ter Hofstede, A.H.M., van der Aalst, W.M.P., Edmond, D.: Dynamic, extensible and context-aware exception handling for workflow. In: Proc. CoopIS 2007, Algarve, Portugal (2007) Berners-Lee, T., Hendler, J., Lassila, O.: The semantic web. Scientific American, 34–43 (2001)
Exception Handling in Web Service Processes [CCPP99]
[CDP08]
[CJ90]
[DKLW07] [EFL+ 04]
[EL96]
[EL97] [ES07] [FFJS04] [FFM+ 10]
[FI08a]
[FI08b]
[FMS10]
[FN95]
[FSW99] [Gla08] [GM06]
[HA00]
251
Casati, F., Ceri, S., Paraboschi, S., Pozzi, G.: Specification and implementation of exceptions in workflow management systems. ACM Transactions on Database Systems 24(3), 405–451 (1999) Combi, C., Daniel, F., Pozzi, G.: XPDL Enabled Cross-Product Exception Handling for WfMSs. In: Fischer, L. (ed.) 2008 BPM and Workflow Handbook, Future Strategies, Inc, Florida, pp. 177–186 (2008) Compton, P., Jansen, R.: Knowledge in context: a strategy for expert system maintenance. In: Proc. 2nd Australian Joint Conference on Artificial Intelligence, pp. 292–306 (1990) Decker, G., Kopp, O., Leymann, F., Weske, M.: Bpel4chor: Extending bpel for modeling choreographies. In: ICWS 2007, Salt Lake City, Utah, pp. 296–303 (2007) Eiter, T., Faber, W., Leone, N., Pfeifer, G., Polleres, A.: A logic programming approach to knowledge-state planning: Semantics and complexity. ACM Transactions on Computational Logic 5(2), 206–263 (2004) Eder, J., Liebhart, W.: Workflow recovery. In: Proc. 1st IFCIS International Conference on Cooperative Information Systems, p. 124 (1996) Eder, J., Liebhart, W.: Workflow transactions. Workflow Handbook, pp. 157–163. John Wiley, Chichester (1997) Euzenat, J., Shvaiko, P.: Ontology matching. Springer, Heidelberg (2007) Felfernig, A., Friedrich, G., Jannach, D., Stumptner, M.: Consistencybased diagnosis of configuration knowledge bases. Artificial Intelligence 152(2), 213–234 (2004) Friedrich, G., Fugini, M., Mussi, E., Pernici, B., Tagni, G.: Exception handling for repair in service-based processes. IEEE Trans. Software Eng. 36(2), 198–215 (2010) Friedrich, G., Ivanchenko, V.: Diagnosis from first principles for workflow executions, Technical Report 2008/002. Institute of Applied Informatics, University Klagenfurt, Austria (2008) Friedrich, G., Ivanchenko, V.: Model-based repair of web service processes, Technical Report 2008/001. Institute of Applied Informatics, University Klagenfurt, Austria (2008) Friedrich, G., Mayer, W., Stumptner, M.: Diagnosing process trajectories under partially known behavior. In: Proc. ECAI 2010, Lisbon, Portugal, pp. 111–116 (2010) Fikes, R.E., Nilsson, N.J.: Strips: a new approach to the application of theorem proving to problem solving. In: Computation & Intelligence: Collected Readings, pp. 429–446. AAAI Press, Menlo Park (1995) Friedrich, G., Stumptner, M., Wotawa, F.: Model-based diagnosis of hardware designs. Artificial Intelligence 111(1-2), 3–39 (1999) Glassey, O.: A case study on process modelling - three questions and three techniques. Decision Support Systems 44(4), 842–853 (2008) Guelfi, N., Mammar, A.: A formal framework to generate XPDL specifications from UML activity diagrams. In: SAC 2006, pp. 1224–1231 (2006) Hagen, C., Alonso, G.: Exception handling in workflow management systems. IEEE Trans. Softw. Eng. 26(10), 943–958 (2000)
252
D. Jannach and A. Gut
[HBP07]
[HH06]
[Hof05] [JL07]
[JLTH06]
[KJF07]
[KSF07]
[LPF+ 06]
[LYRS07]
[MMP06]
[MSZ01] [PC96] [Pee05]
[RD98]
[Rei87] [RvdAtH06] [SPW+ 04]
Hoffmann, J., Bertoli, P., Pistore, M.: Web service composition as planning, revisited: In between background theories and initial state uncertainty. In: Proc. AAAI 2007, Vancouver, BC, pp. 1013–1018 (2007) Halvorsen, O., Haugen, O.: Proposed notation for exception handling in UML 2 sequence diagrams. In: Proc. 18th Australian Software Engineering Conference, Sydney, Australia, pp. 29–40 (2006) Ter Hofstede, A.H.M.: Yawl: yet another workflow language. Information Systems 30, 245–275 (2005) Jannach, D., Leopold, K.: Knowledge-based multimedia adaptation for ubiquitous multimedia consumption. Journal of Network and Computer Applications 30(3), 958–982 (2007) Jannach, D., Leopold, K., Timmerer, C., Hellwagner, H.: A knowledge-based framework for multimedia adaptation. Applied Intelligence 24(2), 109–125 (2006) Kuster, J., Jannach, D., Friedrich, G.: Handling alternative activities in resource-constrained project scheduling problems. In: Proceedings IJCAI 2007, Hyderabad, India, pp. 1960–1965 (2007) Kopeck´ y, J., Paslaru Bontas Simperl, E., Fensel, D.: Semantic web service offer discovery. In: Proc. Workshop on Service Matchmaking and Resource Retrieval in the Semantic Web (SMRR 2007) co-located with ISWC 2007 + ASWC 2007, Busan, South Korea (2007) Leone, N., Pfeifer, G., Faber, W., Eiter, T., Gottlob, G., Perri, S., Scarcello, F.: The dlv system for knowledge representation and reasoning. ACM Transactions on Computational Logic 7(3), 499–562 (2006) Lu, J., Yu, Y., Roy, D., Saha, D.: Web service composition: A reality check. In: Benatallah, B., Casati, F., Georgakopoulos, D., Bartolini, C., Sadiq, W., Godart, C. (eds.) WISE 2007. LNCS, vol. 4831, pp. 523–532. Springer, Heidelberg (2007) Modafferi, S., Mussi, E., Pernici, B.: SH-BPEL: a self-healing plug-in for WS-BPEL engines. In: Proc. Workshop on Middleware for Service Oriented Computing, pp. 48–53 (2006) McIlraith, S.A., Son, T.C., Zeng, H.: Semantic web services. IEEE Intelligent Systems 16(2), 46–53 (2001) Pryor, L., Collins, G.: Planning for contingencies: A decision-based approach. Journal of Artificial Intelligence Research 4, 287–339 (1996) Peer, J.: A pop-based replanning agent for automatic web service composition. In: G´ omez-P´erez, A., Euzenat, J. (eds.) ESWC 2005. LNCS, vol. 3532, pp. 47–61. Springer, Heidelberg (2005) Reichert, M., Dadam, P.: Adeptf lex - supporting dynamic changes of workflows without losing control. Journal of Intelligent Information Systems 10(2), 93–129 (1998) Reiter, R.: A theory of diagnosis from first principles. Artificial Intelligence 32(1), 57–95 (1987) Russell, N., van der Aalst, W., ter Hofstede, A.: Workflow exception patterns. In: Martinez, F.H., Pohl, K. (eds.) CAiSE 2006. LNCS, vol. 4001, pp. 288–302. Springer, Heidelberg (2006) Sirin, E., Parsia, B., Wu, D., Hendler, J., Nau, D.: HTN planning for web service composition using SHOP2. In: Proc. ISCW 2004, Sanibel Island, Florida, pp. 377–396 (October 2004)
Exception Handling in Web Service Processes [SR93] [Sri03]
[STA05]
[TAH07]
[vNEV07]
[Wei81] [Wot02] [YD07]
253
Sheth, A., Rusinkiewicz, M.: On transactional workflows. IEEE Data Engineering Bulletin 16, 3–34 (1993) Srivastava, B.: Web service composition - current solutions and open problems. In: Prof. ICAPS 2003 Workshop on Planning for Web Services, Trento, Italy, pp. 28–35 (2003) Scheer, A.W., Thomas, O., Adam, O.: Process modeling using eventdriven process chains. In: Dumas, M., van der Aalst, W.M.P., ter Hofstede, A.H.M. (eds.) Process-Aware Information Systems, pp. 119– 145. Wiley, Chichester (2005) Tsetsos, V., Anagnostopoulos, C., Hadjiefthymiades, S.: Semantic web service discovery: Methods, algorithms and tools. In: Cardoso, J. (ed.) Semantic Web Services: Theory, Tools and Applications, pp. 240–280. IDEA, USA (2007) van Nieuwenborgh, D., Eiter, T., Vermeir, D.: Conditional planning with external functions. In: Baral, C., Brewka, G., Schlipf, J. (eds.) LPNMR 2007. LNCS (LNAI), vol. 4483, pp. 214–227. Springer, Heidelberg (2007) Weiser, M.: Program slicing. In: Proc. 5th International Conference on Software Engineering, Piscataway, NJ, USA, pp. 439–449 (1981) Wotawa, F.: On the relationship between model-based debugging and program slicing. Artificial Intelligence 135(1-2), 125–143 (2002) Yan, Y., Dague, P.: Modeling and diagnosing orchestrated web service processes. In: Proc. ICWS 2007, Salt Lake City, Utah, pp. 51–59 (2007)
Leveraging Model-Based Tool Integration by Conceptual Modeling Techniques Gerti Kappel1 , Manuel Wimmer1 , Werner Retschitzegger2 , and Wieland Schwinger3 1
2
Business Informatics Group Vienna University of Technology, Austria {kappel,wimmer}@big.tuwien.ac.at Department of Business Information Systems University of Vienna, Austria [email protected] 3 Department of Telecooperation Johannes Kepler University Linz, Austria [email protected]
Abstract. In the context of model-based tool integration, model transformation languages are the first choice for realizing model exchange between heterogenous tools. However, the lack of a conceptual view on the integration problem and appropriate reuse mechanisms for already existing integration knowledge forces the developer to define model transformation code again and again for certain recurring integration problems in an implementation-oriented manner resulting in low productivity and maintainability of integration solutions. In this chapter, we summarize our work on a framework for modelbased tool integration which is based on well-established conceptual modeling techniques. It allows to design integration models on a conceptual level in terms of UML component diagrams. Not only the design-time is supported by conceptual models, but also the runtime, i.e., the execution of integration models, is represented by conceptual models in terms of Coloured Petri Nets. Furthermore, we show how reusable integration components for resolving structural metamodel heterogeneities, which are one of the most frequently recurring integration problems, can be implemented within our framework.
1
Introduction
With the rise of Model-Driven Engineering (MDE) [25] models become the main artifacts of the software development process. Hence, a multitude of modeling tools is available supporting different tasks, such as model creation, model simulation, model checking, model transformation, and code generation. Seamless exchange of models among different modeling tools increasingly becomes a crucial prerequisite for effective MDE. Due to lack of interoperability, however, it is often difficult to use tools in combination, thus the potential of MDE cannot be fully utilized. For achieving interoperability in terms of transparent model Kaschek/Delcambre (Eds.): The Evolution of Conceptual Modeling, LNCS 6520, pp. 254–284, 2011. c Springer-Verlag Berlin Heidelberg 2011
Leveraging Model-Based Tool Integration
255
exchange, current best practices (cf., e.g. [28]) comprise creating model transformations based on mappings between concepts of different tool metamodels, i.e., the metamodels describing the modeling languages supported by the tools. We have followed the aforementioned approach in various projects such as the ModelCVS1 project [13] focusing on the interoperability between legacy case tools (in particular CA’s AllFusion Gen) with UML tools and the MDWEnet2 project [29] trying to achieve interoperability between different tools and languages for web application modeling. The prevalent form of heterogeneity one has to cope with when creating such mappings between different metamodels is structural heterogeneity, a form of heterogeneity well-known in the area of federated and multi database systems [2,14]. In the realm of metamodeling structural heterogeneity means that semantically similar modeling concepts are defined with different metamodeling concepts leading to differently structured metamodels. Current model transformation languages, e.g., the Object Management Group (OMG) standard Query/View/Transformation (QVT) [21], provide no appropriate abstraction mechanisms or libraries for resolving recurring kinds of structural heterogeneities [18]. Thus, resolving structural heterogeneities requires to manually specify partly tricky model transformations again and again which simply will not scale up having also negative influence on understanding the transformation’s execution and on debugging. In this work, we summarize our work on a framework for realizing model-based tool integration which is based on well-established conceptual modeling techniques. In particular, the framework allows to build so-called metamodel bridges on a conceptual level by using UML component diagrams [19]. Furthermore, such a metamodel bridge allows the automatic transformation of models since for each mapping operator, which is expressed as a component, the operational semantics is specified on basis of Colored Petri Nets [7]. Colored Petri Nets provide a uniform formalism not only for representing the transformation logic together with the metamodels and the models themselves, but also for executing the transformations, thus facilitating understanding and debugging. To demonstrate the applicability of our approach we apply the proposed framework for defining a set of mapping operators subsumed in our mapping language called CAR. This mapping language is intended to resolve typical structural heterogeneities occurring between the core concepts usually used to define metamodels, i.e., class, attribute, and reference, as provided by the OMG standard Meta Object Facility (MOF) [20]. Finally, we present an inheritance mechanism for reusing applied mappings within a single metamodel bridge. The framework has been applied in several modeling tool integration projects where structural modeling languages as well as behavioral modeling languages have been integrated. The rest of the paper is structured as follows. In Section 2 we introduce our framework for defining mapping operators for establishing metamodel bridges. In Section 3, the mapping language CAR is presented for which an inheritance mechanism for reusing mappings within a single metamodel bridge is introduced 1 2
ModelCVS—Concurrent Versioning System for Models. MDWEnet—Model-Driven Web Engineering Network.
256
G. Kappel et al.
in Section 4. While an evaluation of the framework is presented in Section 5, Section 6 discusses related work. Finally, in Section 7 a conclusion and a discussion on future research issues is given.
2
Metamodel Bridging at a Glance
In this section, we describe the conceptual architecture of the proposed Metamodel Bridging Framework in a by-example manner. The proposed framework provides two views on the metamodel bridge, namely a mapping view and a transformation view as illustrated in Figure 1. On the mapping view level, the user defines mappings between elements of two metamodels (cf. M2 in Figure 1). Thereby a mapping expresses also a relationship between model elements, i.e., the instances of the metamodels [3]. In our approach, we define these mappings between metamodel elements with mapping operators standing for a processing entity encapsulating a certain kind of transformation logic. A mapping operator takes as input elements of the source model and produces as output semantically equivalent elements of the target model. Thus, it declaratively describes the semantic correspondences on a high-level of abstraction. A set of applied mapping operators defines the mapping from a left hand side (LHS) metamodel to a right hand side (RHS) metamodel further on subsumed as mapping model. For actually exchanging models between different tools, the mapping models have to be executed. Therefore, we propose, in addition to the mapping view, a transformation view which is capable of transforming models (cf. M1 in Figure 1) from the LHS to the RHS on basis of Colored Petri Nets [7]. 2.1
The Mapping View
For defining mapping operators and consequently also for building mapping models, we are using a subset of the UML 2 component diagram concepts. With this formalism, each mapping operator can be defined as a dedicated component, representing a modular part of the mapping model which encapsulates an arbitrary complex structure and behavior, providing well-defined interfaces to the environment. The resulting components are collected in a mapping operator library which can be seen as a domain-specific language for bridging metamodels. The user can apply the mapping operators expressed as components in a plug&play manner, i.e., only the connections to the provided and required interfaces have to be established manually. Our motivation for using UML 2 component diagrams for the mapping view is the following. First, many software engineers are likely to be familiar with the UML component diagram notation. Second, the provided and required interfaces which can be typed, enable the composition of mapping operators to resolve more complex structural heterogeneities. Third, the clear separation between black-box view (i.e., the component’s externals) and white-box view (i.e., the component’s internals) of components allows switching between a high-level mapping view
Leveraging Model-Based Tool Integration
LHS
M Mapp ingView
1
MappingModel
TargetMM
C2C
Class
M2
RHS
Bridging
SourceMM
name : String
EntityType A2A
ownedAttributes
name : String attributes
R2R
Attribute
Property name : String
name : String
C2C
A2A
Transform
2
Transform 4
SourcePlaces
EntityType 2
history
Traansforrmatio onVie ew
Class_name
2
TargetPlaces
C2C
Class
M1 1+M M2
Transform
TransformationLogic
2
A2A
EntityType_name
2
2
history
Class_ownedAttributes
R2R
EntityType attributes EntityType_attributes
2
2
history
Property
Attribute C2C
2
2
history
Property_name
Attribute_name
A2A
2
2
history
6 5 UMLModel
TokenModel o1
3
o3:Property name=“ssn”
name=“Student”
o4:Property name=“studentnr”
o2
7
TokenModel o1
ERModel
o2
“Student“
“Student“
8 o1:EntityType
“Professor“
o3
o esso “Professor“
o4 “studentnr“ studentnr
“ssn“
o3
o4 studentnr “studentnr“
Trransform
name=“Professor”
o2:Class
Transform m
M1
o1:Class
Export
Execution
Import
name=“Professor”
o3:Attribute name=“ssn”
“ssn“
Fig. 1. Metamodel Bridging Framework by-example
o2:EntityType name=“Student”
o4:Attribute name=“studentnr”
257
258
G. Kappel et al.
and a detailed transformation view, covering the operational semantics, i.e., the transformation logic, of an operator. Anatomy of a mapping operator. Each mapping operator (as for example shown in the mapping model of Figure 1) has input ports with required interfaces (left side of the component) as well as output ports with provided interfaces (right side of the component). Because each mapping operator has its own trace model, i.e., providing a log about which output elements have been produced from which input elements, an additional providedContext port with a corresponding interface is available on the bottom of each mapping operator. This port can be used by other operators to access the trace information for a specific element via requiredContext ports with corresponding interfaces on top of the operator. In the mapping view of Figure 1 (cf. step 1), an example is illustrated where a small part of the metamodel of the UML class diagram (cf. source metamodel) is mapped to a part of the metamodel of the Entity Relationship diagram (cf. target metamodel). In the mapping view, source metamodel elements have provided interfaces and target metamodel elements have required interfaces. This is due to the fact that in our scenario, models of the LHS are already available whereas models of the RHS must be created by the transformation, i.e., the elements of the LHS must be streamed to the RHS according to the mapping operators. Consequently, Class and Property of the source metamodel are mapped to EntityType and Attribute of the target metamodel with Class2Class (C2C ) operators, respectively. In addition, the C2C operator owns a providedContext port on the bottom of the component which shall be used by the requiredContext ports of the appropriate Attribute2Attribute (A2A) and Reference2Reference (R2R) operators to preserve validity of target models. In particular, with this mechanism it can be ensured that values of attributes are not transformed before their owning objects has been transformed and links as instances of references are not transformed before the corresponding source and target objects have been transformed. 2.2
The Transformation View
The transformation view is capable of executing the defined mapping models. For this, so called transformation nets [12,23,34] are used which are a special kind of Colored Petri Nets consisting of source places at the LHS and target places at the RHS. Transitions between the source and target places describe the transformation logic located in the bridging part of the transformation net as shown in Figure 1. Transformation nets provide a suitable formalism to represent the operational semantics of the mapping operators, i.e., the transformation logic defined in the white-box view of the component due to several reasons. First, they enable the execution of the transformation thereby generating the target model out of the source model, which favors also debugging of a mapping model. Second, it allows a homogeneous representation of all artefacts involved in a model transformation (i.e., models, metamodels, and transformation logic) by means of a simple
Leveraging Model-Based Tool Integration
259
formalism, thus being especially suited for gaining an understanding of the intricacies of a specific metamodel bridge. In the next paragraphs, we discuss rules for assembling metamodels, models, and mapping models into a single transformation net and how the transformation can actually be executed. Places represent Metamodels. First of all, places of a transformation net are used to represent the elements of the source and target metamodels (cf. step 2 in Figure 1). In this respect, we currently focus on the three major building blocks of metamodels (provided, e.g. by meta-metamodels such as MOF), namely class, attribute, and reference. In particular, classes are mapped onto one-colored places whereby the name of the class becomes the name of the place. The notation used to visually represent one-colored places is a circle or oval as traditionally used in Petri Nets. Attributes and references are represented by two-colored places, whereby the name of the containing class plus the name of the reference or of the attribute separated by an underscore becomes the name of the place (cf. e.g. Class name and Class ownedAttributes in Figure 1). To indicate that these places contain twocolored tokens, the border of two-colored places is double-lined. Tokens represent Models. The tokens of the transformation net are used to represent the source model which should be transformed according to the mapping model. Each element of the source model is expressed by a certain token, using its color as a means to represent the model element’s identity in terms of a String (cf. step 3 in Figure 1). In particular, for every object, a one-colored token is produced, whereby for every link as an instance of a reference, as well as for every value of an attribute, a two-colored token is produced. The fromColor for both tokens refers to the color of the token that corresponds with the containing object. The toColor is given by the color of the token that corresponds with the referenced target object or the primitive value, respectively. Notationally, a two-colored token consists of a ring (carrying the fromColor) surrounding an inner circle (depicting the toColor). Considering our example shown in Figure 1, the objects o1 to o4 of the UML model shown in the M1-layer are transformed into one-colored tokens. Each one-colored token represents an object identity, pointed out by the object name beneath the token. E.g., the tokens with the inner-color ”Student” and ”Professor” have the same outer-color as their containing objects and the token which represents the link between object o1 and o3 has the same outer-color as the token representing object o1 and the inner-color corresponds to the onecolored token representing object o3. Transitions represent Mapping Models. The mapping model is expressed by the transformation logic of the transformation net connecting the source and the target places (cf. Step 4 in Figure 1). In particular, the operational semantics of the mapping operators are described with transitions, whereby the behavior of a transition is described with the help of preconditions called query-tokens (LHS of a transition) and postconditions called generator-tokens (RHS of a transition). Query-tokens and generator-tokens can be seen as templates, simply visualized
260
G. Kappel et al.
as color patterns, describing a certain configuration of tokens. The pre-condition is fulfilled and the transitions fires, if the specified color pattern described by the query-tokens matches a configuration of available input tokens. In this case, the postcondition in terms of the generator-tokens produces the required output tokens representing in fact the necessary target model concepts. In the following, the most simple mapping operators used in our example are described, namely C2C, A2A, and R2R. C2C. The white-box view of the C2C operators as shown in the transformation view of Figure 1 ensures that each object instantiated from the class connected to the input port is streamed into the mapping operator, the transition matches a single token from the input port, and streams exactly the same token to the output port. This is expressed in the transition by using the most basic querytoken and generator-token combination, both having the same color pattern. In addition, every input and output token combination is saved in a history place representing the trace model which is connected to the providedContext port and can be used as trace information by other operators. A2A. The white-box view of the A2A operator is also illustrated in the bridging part of the transformation view in Figure 1. Two-colored tokens representing attribute values are streamed via the input port into the mapping operator. However, a two-colored token is only streamed to the output port if the owning object of the value has been already transformed by a C2C operator. This is ensured in that the transition uses the same color pattern for the one-colored query-token representing the owning object streamed from the requiredContext port and for the outer color of the two-valued query-token representing the containing object of the attribute value. Only, if a token configuration matches this pre-condition, the two-colored token is streamed via the generator-token to the output port. Again, the input tokens and the corresponding output tokens are stored in a history place which is connected to the providedContext port. R2R. The white-box view of the R2R operator shown in the transformation view of Figure 1 consists of three query-tokens, one two-colored query-token representing the link and two one-colored query-tokens for accessing trace information from C2C operators. The two-colored query-token must have the same inner and outer colors as provided by the C2C trace information, i.e., the source and target objects must be already transformed. When this precondition is satisfied by a token configuration, the two-colored token representing the link is streamed via the generator-token to the output port. Execution of the transformation logic. As soon as the metamodels are represented as places, which are furthermore marked with the respective colored tokens depicting the concepts of the source model (cf. step 5 in Figure 1), the transformation net can be started. Now, tokens are streamed from the source places over the transitions into the target places (cf. step 6 in Figure 1). Considering our running example, in a first step only the transitions of the C2C operators are able to fire due to the dependencies of the A2A and R2R operators. Hence, tokens from the places Class and Property are streamed to
Leveraging Model-Based Tool Integration
261
the appropriate places of the RHS and all combinations of the queried input and generated output tokens are stored in the trace model of the C2C operator. As soon as all necessary tokens are available in the trace model, dependent operators, i.e., the A2A and R2R operators, are also able to fire. Generation of the target model. After finishing the transformation, the tokens from the target places can be exported (cf. step 7 in Figure 1) and transformed back into instances of the RHS metamodel (cf. step 8 in Figure 1). In our example, the one-colored tokens o1 to o4 contained in the target places are transformed back into objects of type EntityType and Attribute. The twocolored tokens which represent attribute values, e.g., ”Professor” and ”Student”, are assigned to their containing objects, e.g., o1 and o2 whereas ”ssn” and ”studentnr” are assigned to o3 and o4. Finally, the two-colored tokens which represent links between objects are transformed back into links between o1 and o3, as well as between o2 and o4.
3
Mapping Operators of the CAR Mapping Language
3.1
Motivating Example
Based on experiences gained in various interoperability projects [10,11,35,36] it has been shown that although most meta-metamodels such as MOF offer only a core set of language concepts for defining metamodels, numerous structural heterogeneities occur when defining modeling languages. LHS-MM
c
Package
Package
name : String
name : String {id} d superClasses Cl *
Class
* Class name : String *
name : Stringg package : String {idref}
*
subClasses *
Generalization
* superClasses Cl
Attribute
*
name : String
Attribute name : String isID : Boolean minCard : Integer maxCard : Integer
RHS-MM
: kind
a
DesAtt b
IdAtt
1..1
Multiplicity minCard : Integer maxCard : Integer
Fig. 2. Structural Heterogeneities Between Metamodels - Example
As an example for structural metamodel heterogeneity consider the example shown in Figure 2. Two MOF-based metamodels represent semantically equivalent core concepts of the UML class diagram in different ways. Whereas the LHS metamodel uses only a small set of classes, the RHS metamodel employs a much larger set of classes thereby representing most of the UML concepts which are in the LHS metamodel implicitly defined as attributes or references explicitly as
262
G. Kappel et al.
first class citizens. More specifically, four structural metamodel heterogeneities can be found (cf. Subsection 3.3 - 3.6) which require mapping operators going beyond the simple one-to-one mappings provided by the mapping operators in Section 2. 3.2
CAR Mapping Language at a Glance
For resolving structural metamodel heterogeneities, we provide nine different core mapping operators as depicted in Figure 2. These nine mapping operators result from the possible combinations between the core concepts of metametamodels, namely class, attribute, and reference, which also led to the name of the CAR mapping language. These mapping operators are designed to be declarative and bi-directional and it is possible to derive executable transformations based on transformation nets. One important requirement for the CAR mapping language is that it should be possible to reconstruct the source models from the generated target models, i.e., any loss of information during transformation should be prevented. In Figure 3, the mapping operators are divided according to their functionality into the categories Copier, Peeler, and Linker which are explained in the following. Class
Attribute
Reference
Class
C2C
C2A
C2R
Att ib t Attribute
A2C
A2A
A2R
Reference
R2C
R2A
R2R
Legend
… Copier … Peeler … Linker
Fig. 3. CAR Mapping Operators and their categorization
Copier. The diagonal of the matrix in Figure 3 depicts the symmetric mapping operators of the CAR mapping language which have been already discussed in Section 2. The term symmetric means that the input and outport ports of the left side and the right side of the mapping operators are of the same type. This category is called Copier, because these mapping operators copy one element of the LHS model into the RHS model without any further manipulations. Peeler. This category consists of mapping operators which create new objects by ”peeling”3 them out of values or links. The A2C operator bridges heterogeneities which are resulting from the fact that a concept is expressed as an attribute in one metamodel and as a class in another metamodel. Analogously, a concept can be expressed on the LHS as a reference and on the RHS as a class which can be bridged by a R2C operator. Linker. The last category consists of mapping operators which either link two objects to each other out of value-based relationships (cf. A2R and R2A operator) 3
Note that the term ”peeling” is used since when looking at the white-box view the transformation of an attribute value into an object requires in fact to generate a one-colored token out of a two-colored token.
Leveraging Model-Based Tool Integration
263
or assign values or links to objects for providing the inverse variants of the A2C and R2C operators (cf. C2A and C2R operator). To resolve the structural heterogeneities depicted in Figure 2, in the following subsections the necessary mapping operators are discussed in detail, comprising besides a variation of the C2C operator mainly mapping operators falling into the above mentioned peeler and linker category. The solution for the integration example is shown in Figure 4 and Figure 5 on the mapping view and on the transformation view, respectively.
LHS
RHS
Bridging
SourceMM
MappingModel
TargetMM Package
C2C
Package
name : String
A2A
name : String A2R C2C
superClasses classes
Class *
A2A
name : String package : String
*
subClasses
Class name *: String St i
R2C
* *
Generalization superClasses
R2R
{inv: true} C2C
attributes ib
*
Attribute attributes
*
Attribute
name : String C2C
:kind
isID=false
name : String isID : Boolean minCard : Integer maxCard : Integer
A2A
C2C
1 1 1..1
Multiplicity minCard : Integer maxCard : Integer
DesAtt isID=ftrue
multiplicity
IdAtt
A2C
Fig. 4. Integration Example solved with CAR (Mapping View)
3.3
Conditional C2C Mapping Operator
Problem. In MOF-based metamodels, a property of a modeling concept can be expressed via a discriminator of an inheritance branch or with an additional attribute. An example for this kind of heterogeneity can be found in Figure 2(a), namely between Attribute.isID on the LHS and the subclasses of the class Attribute on the RHS. This heterogeneity is not resolvable with a primitive C2C operator per se, because one class on the LHS corresponds to several classes on the RHS whereby each mapping is only valid under a certain condition. On the model level, this means that a set of objects has to be splitted into several subsets based on the object’s attribute values.
264
G. Kappel et al.
LHS
Bridging
SourcePlaces Package
RHS
TransformationLogic
TargetPlaces Package
C2C
historyy
A2A
Package_name
Package_name historyy C2C
Class hi t history
Class
Class_name A2A
Class_name historyy
Package_classes
A2R
Class_package _p g
Generalization_subClasses
history R2C
Generalization
Class_superClasses
Generalization_superClasses R2R
history
Class_attributes Class_attributes history
C2C <>
Attribute
Attribute
C2C
DesAtt IdAtt
history
Attribute_name
isID=false s a se
C2C
Attribute_name history
Attribute_isID
isID=true
Attribute multiplicity Attribute_multiplicity
A2A
Multiplicity
Attribute_minCard history A2C
Multiplicity_minCard
Attribute_maxCard _ Multiplicity_maxCard history
Fig. 5. Integration Example solved with CAR (Transformation View)
Leveraging Model-Based Tool Integration
265
Solution. To cope with this kind of heterogeneity, the C2C operator has to be extended with the capability of splitting a set of objects into several subsets. For this we are annotating the C2C operator with OCL-based preconditions assigned to ports as depicted in Figure 6(a). These preconditions supplement the querytokens of the transitions by additionally allowing to specify constraints on the source model elements. The reason for introducing this additional mechanism is that the user should be able to configure the C2C operator without having to look into the white-box view of the operator, realizing its basic functionality. Example Application. In the example shown in Figure 4, we can apply two C2C mapping operators with OCL conditions, one for mapping Attribute to DesAtt with the precondition Attribute.isID = false, and one for mapping Attribute to IdAtt with the precondition Attribute.isID = true. In addition, this example shows a way how mappings can be reused within a mapping model by allowing inheritance between mappings. This mechanism allows to define certain mappings directly between superClasses and not for each subClass combination again and again (cf., e.g., the A2A mapping between the Attribute.name attributes), as described in more detail in Section 4. 3.4
A2C Mapping Operator
Problem. In Figure 2(b), the attributes minCard and maxCard, which are part of the class Attribute at the LHS, are at the RHS part of a dedicated class Multiplicity. Therefore, on the instance level, a mechanism is needed to ”peel” objects out of attribute values and to additionally take into account the structure of the LHS model in terms of the attribute’s owning class when building the RHS model, i.e., instances of the class Multiplicity must be connected the corresponding instances of class Attribute. Solution. The black-box view of the A2C mapping operator as illustrated in Figure 6(b) consists of one or more required interfaces for attributes on the LHS depending on how many attributes are contained by the additional class, and has in minimum three provided interfaces on the RHS. The first of these interfaces is used to mark the reference which is responsible to link the two target classes, the second is used to mark the class that should be instantiated, and the rest is used to link the attributes of the LHS to the RHS. Additionally, an A2C operator has a required interface to a C2C, because the source object is splitted into two target objects, thereby only one object is created by the A2C, the other has to be generated by a C2C operator which maps the LHS class to its corresponding target RHS class. The white-box view of the A2C operator shown in Figure 6(b) comprises a transition consisting of at least two query-tokens. The first query-token guarantees that the owningObject has been already transformed by a C2C operator. The other query-tokens are two-colored tokens representing the attribute values which have as fromColor the same color as the first query-token. The
266
G. Kappel et al.
object : Class
object : Class
C2C pre: OCL-Exp c2c
Black-Box View
Black-Box View
owningObject : c2c link2OwningObj : Reference value1 : Attribute
…
valueObject : Class
A2C
value1 : Attribute
…
valueN : Attribute
valueN : Attribute a2c
C2C object : Class
object : Class
History pre: OCL-Exp c2c
White-Box View
White-Box View
owningObject: c2c
A2C
link2OwningObj : Reference
value1 : Attribute
valueObject : Class
…
…
value1 : Attribute
…
valueN : Attribute
valueN : Attribute
History a2c
link2SourceObj : Reference link : Reference
R2C
link2TargetObj : Reference r2c sourceObject : c2c
targetObject : c2c
White-Box View
linkObject : Class
Black-Box View
sourceObject : c2c
targetObject : c2c
(b) A2C operator
R2C
link2SourceObj : Reference linkObject : Class
link : Reference
History
link2TargetObj : Reference
r2c
(c) R2C operator
targetObject : c2c
sourceObject : c2c keyValue : Attribute
link : Reference
A2R
keyRefValue : Attribute
a2r targetObject : c2c
sourceObject : c2c
White-Box View
Black-Box View
(a) Conditional C2C operator
A2R keyValue : Attribute keyRefValue : Attribute
link : Reference
History a2r
(d) A2R operator
Fig. 6. Black-box and white-box views of CAR mapping operators
post-condition of the transition consists of at least three generator-tokens. The second generator-token introduces a new color, i.e., this color is not used in the pre-condition part of the transition, and therefore, the generator-token produces a new object with a unique identity. The first generator-token is used for linking the newly created object appropriately into the target model and the other twocolored generator tokens are used to stream the values into the newly generated object by changing the fromColor of the input values. Example Application. In Figure 4, the attributes minCard and maxCard are mapped to attributes of the class Multiplicity. Furthermore, the reference between the classes Attribute and Multiplicity is marked by the A2C mapping as well as the class Multiplicity. To assure that the generated Multiplicity objects can be properly linked to Attribute objects, the A2C mapping is in the context of the C2C mapping between the Attribute classes.
Leveraging Model-Based Tool Integration
3.5
267
R2C Mapping Operator
Problem. In Figure 2(c), the reference superClasses of the LHS metamodel corresponds to the class Generalization of the RHS metamodel. This kind of heterogeneity requires an operator which is capable of ”peeling” an object out of a link and to additionally preserve the structure of the LHS in terms of the classes connected by the relationships at the RHS. Solution. The black-box view of the R2C mapping operator, as depicted in Figure 6(c), has one required interface on the left side for pointing to a reference. On the right side it has three provided interfaces, one for the class which stands for the concept expressed as reference on the LHS and two for selecting the references which are responsible to connect the object which has been peeled out of the link of the LHS into the RHS model. To determine the objects to which the peeled object should be linked, two additional required interfaces on the top of the R2C operator are needed for determining the corresponding objects of the source and target objects of the LHS. The white-box view of the R2C mapping operator, as illustrated in Figure 6(c), consists of a pre-condition comprising three query-tokens. The input link is connected to a two-colored query-token, the fromColor corresponds to the query-token standing for the source object and the toColor corresponds to a query-token standing for the target object. The post-condition of the transition introduces a new color and is therefore responsible to generate a new object. Furthermore, two links are produced by the other generator-tokens for linking the newly generated object with the corresponding source and target objects of the LHS. Example Application. In Figure 4, the reference superClasses in the LHS metamodel is mapped to the class Generalization by an R2C operator. In addition, the references subClasses and superClasses are selected for establishing an equivalent structure on the RHS as existing on the LHS. For actually determining the Class objects which should be connected via Generalization objects, the R2C operator has two dependencies to C2C mappings. This example can be seen as a special case, because the reference superClasses is a reflexive reference, therefore both requiredContext ports of the R2C operator point to the same C2C operator. 3.6
A2R Mapping Operator
Problem. The attribute vs. reference heterogeneity shown in Figure 2(d) resembles the well-known difference between value-based and reference-based relationships, i.e, corresponding attribute values in two objects can be used to ”simulate” links between two objects. Hence, if the attribute values in two objects are equal, a link ought to be established between them. Solution. For bridging the value-based vs. reference-based relationship heterogeneity, the A2R mapping operator as shown in Figure 6(d) provides on the LHS two interfaces, one for marking the keyValue attribute and another for marking
268
G. Kappel et al.
the keyRefValue attribute. On the RHS, the operator provides only one interface for marking the reference which corresponds to the keyValue/keyRefValue attribute combination. The white-box view of the operator comprises a transition which has four query-tokens. The first two ensure that the objects which are referencing each other on the LHS have been already transformed. The last two are the keyValue and keyRefValue query-tokens whereby the inner-color (representing the attribute values) is the same for both tokens. The generator-token of the transition produces one two-colored token by using the outer-color of the keyRefValue query-token as the outer-color and the outer-color of the keyValue query-token as the inner-color. Example Application. In Figure 4, the A2R operator is used to map the Package.name attribute as the key attribute and the Class.package attribute as the keyRef attribute of the LHS metamodel to the reference between Package and Class on the RHS metamodel. After presenting how common structural metamodel heterogeneities are resolved with CAR mapping operators, we proceed with discussing how mapping reuse within a single mapping model may be achieved. This is especially needed for large metamodels which consists of extensive class hierarchies such as the UML 2 metamodel. For example, as reported in [16], the number of single inheritance meta-classes in UML 2 is 209. Therefore, in the next section, an inheritance mechanism for CAR mapping operators for achieving reuse within a single mapping model is presented.
4
An Inheritance Mechanism for CAR Mappings
Why is an inheritance mechanism needed? Metamodeling languages such as MOF allow to define class hierarchies in metamodels through generalization which is frequently used in practice. Consequently, when one defines mappings between metamodels which heavily use generalizations leading to huge taxonomies, it should be possible to reuse previously defined mappings between general classes for mappings between subclasses. Required reuse mechanisms. To facilitate the definition of mapping models between MOF-based metamodels, reuse may be achieved in three ways: (1) Reuse of feature mappings: Mappings between features of superclasses may be defined once for the superclasses and may be inherited by mappings between subclasses which may define additional feature mappings between the subclasses. This reuse mechanism is comparable with code reuse through implementation inheritance supported by common object-oriented programming languages. (2) Reuse for indirect instances: The following mapping situation frequently occurs when two metamodels have to be integrated. Assume, we have on the LHS a superclass which has a multitude of subclasses. In contrast, on the RHS the class hierarchy on the LHS is collapsed into a single class inhibiting all features of the LHS class hierarchy at the RHS, which is equivalent to the LHS
Leveraging Model-Based Tool Integration
269
superclass and also to its subclasses. To avoid that for each subclass a mapping to the RHS class must be defined, it should be possible to apply the mapping between the LHS superclass and the RHS class also for indirect instances of the LHS superclass. (3) Refinement of mappings: A refinement mechanism is needed to define for certain subclasses specific mappings, which refine the mapping between the superclasses. For example, instances of a LHS subclass should be transformed into instances of a RHS subclass and not into instances of the RHS superclass. Of course, the feature mappings between the superclasses should be also applied for such submappings. In the following subsections, we elaborate on how the CAR mapping language is extended in order to provide these three reuse features. 4.1
Inheritance for C2C Mappings
For reusing existing feature mappings (first reuse mechanism), we introduce the possibility to define generalization relationships between C2C mappings. This means, the user defines general mappings between superclasses called supermappings and more specific mappings between subclasses called submappings which may be used to refine the supermappings (third reuse mechanism). As concrete syntax for generalization relationships between C2C mappings, we reuse the notation of UML generalization relationships between classes, i.e., a line with a hollow triangle as an arrowhead. Concerning the second reuse mechanism, it has to be noted that there are also cases in that the mentioned behavior of applying a supermapping for indirect instances is not desired. Sometimes it is required that only direct instances should be transformed and not indirect instances. Therefore, certain configuration parameters for mappings are required in order to express such integration details in the mapping model.
CAR R
superMappings
* C2C
* isAbstract: boolean isApplicable4Subclasses: boolean subMappings
MOF
LHSClass
1
1 Class
isAbstract: boolean superClasses
RHSClass subClasses
* *
Fig. 7. C2C Operator Extended with Generalization Relationships
We allow generalization relationships only for C2C mappings for inheriting feature mappings which are dependent on C2C mappings such as symmetric mappings (A2A, R2R), or asymmetric mappings (A2C, R2C, A2R, and their inverse operators). This is due to the fact that C2C operators are responsible for providing the context information for all other CAR mapping operators. The introduction of generalization relationships between C2C mappings
270
G. Kappel et al.
results in an extension of the C2C operator as shown in Figure 7. In this figure, the class C2C, representing the C2C mapping operator, is extended for defining generalization relationships by setting the references superMappings and subMappings accordingly. Furthermore, for allowing different kinds of supermappings, i.e., if a mapping is itself executable or if it is applicable for unmapped subclasses, two additional boolean attributes, namely C2C.isAbstract and C2C.isApplicable4SubClasses, are defined for the C2C class. These two attributes enable the user to configure the behavior of the mappings in more detail. One important constraint for generalization relationships between C2C operators is that if a generalization between two C2C operators is defined, the participating LHS classes of the supermappings and the submappings must be either in a generalization relationship or it must be actually the same class. Of course, the same constraint must hold on the RHS. These two constraints must be ensured, because the submappings inherit the feature mappings of the supermappings and therefore, the features of the superclasses must be also available on instances which are transformed according to the submappings. The OCL constraints shown in Listing 1.1 validate a mapping model with respect to the correct usage of generalization relationships between C2C mappings. Listing 1.1. Well-formedness Rules for C2C Generalizations context C2C inv : s e l f . su p e r M a p p i n g s −> f o r A l l ( supMap | supMap . LHSClass . s u b C l a s s e s −> u n i o n ( supMap . LHSClass ) −> c o n t a i n s ( s e l f . LHSClass ) ) ; inv : s e l f . su p e r M a p p i n g s −> f o r A l l ( supMap | supMap . RHSClass . s u b C l a s s e s −> u n i o n ( supMap . RHSClass ) −> c o n t a i n s ( s e l f . RHSClass ) ) ;
The default configuration of the C2C operator for each mapping situation specifies that the supermapping itself is executable and applicable for indirect instances. In order to give the user more possibilities to explicitly define other interpretations of supermappings, we furthermore allow three non-default supermapping configurations, thereby the first configuration allows to define abstract supermappings with the capability to be applied for indirect instances, and the other two configurations allow reuse of depending mappings of supermappings without applying the supermappings on indirect instances. For further explanations how to use generalization between C2C operators, we assume that the mapping problem is symmetric, i.e., the same generalization structure is available on the LHS and on the RHS, and that only single inheritance is used for defining the metamodels. In particular, we assume that on the LHS and on the RHS a superclass with various subclasses exists. For asymmetric mapping problems, i.e., one side has a taxonomy and the other has not, and integration scenarios where metamodels use multiple inheritance the interested reader is kindly referred to [32]. 4.2
Representing Inheritance within Transformation Nets
In this subsection we discuss how C2C generalization relationships influence the generation of transformation nets and consequently the execution of the
Leveraging Model-Based Tool Integration
271
transformation logic. An overall design goal is naturally to express new language concepts at the black-box view – such as mapping generalizations in this case – as far as possible by means of existing transformation net mechanisms. Basic Idea. When we take a closer look on supermappings with a standard configuration, we see that these mappings must provide the context, i.e., the trace model information, for all dependent mappings. This means, the supermappings must also provide context information about the transformation of indirect instances, e.g., for assigning attribute values of indirect instances when the attribute is contained by the superclass. Consequently, for a supermapping a transformation component is derived which contains the union of its own trace model for logging the transformation of direct instances of the superclass and the trace models of its submappings for logging the transformation of indirect instances. Therefore, the corresponding transformation components of the submappings are nested into the transformation component of the supermapping. For constructing the union of trace models of nested transformation components, each nested component gets an arc from its own trace model to the union trace model of the outer component. Mappings which depend on the supermapping are connected to the union trace model available on the outer component and mappings which are dependent on submappings are directly connected to the individual trace models of the nested components. Transformation Net
Mapping Model
C2C1 C2C1
history
C2C2.1 C2C3.1 C2C2.1
C2C2.2
C2C3.2
… C2C3.1
C2C3.2
…
C2C2.2
union(2.1, 3.1, 3.2, …)
union(1, 2.1,2.2)
Fig. 8. Representing Inheritance Structures with Nested Transformation Components
Figure 8 illustrates the derivation of generalization relationships into transformation net components. For describing the basic mapping rule how generalization relationships are represented in transformation nets, it is assumed that all mappings are concrete mappings and it is not considered if a mapping is applicable for subclasses or not. The mapping C2C1 of the mapping model shown on the LHS of Figure 8 is transformed into the outer component C2C1 , which consists of a transition for transforming direct instances and of two subcomponents C2C2.1 and C2C2.2 . In addition, the outer component provides a union trace model of the transformation components C2C1 , C2C2.1 , and C2C2.2 . Because
272
G. Kappel et al. pre: oclIsTypeOf(C1|C3)
C2C1 history
Config 1: C2C1.isAbstract = false C2C1.isApplicable4SubClasses = true
Rule1
C2C2
C2C1
pre: oclIsTypeOf(C3)
Config 2:
Mapping Model LHS_MM
C1
C3
C2
RHS_MM C2C1
C1‘ C1
C2C2
C2‘ C2
C2C1.isAbstract = true C2C1.isApplicable4SubClasses = true
history
Rule2
C2C2
C2C1
pre: oclIsTypeOf(C1)
Config 3: C2C1.isAbstract = false C2C1.isApplicable4SubClasses = false
history
Rule3
C2C1
Config 4: C2C1.isAbstract i Ab t t = true t C2C1.isApplicable4SubClasses = false
C2C2
R l 4 Rule4
C2C2
Fig. 9. Representing Supermapping Configurations in Transformation Components
the mapping C2C2.1 has two submappings, the corresponding transformation component has also two subcomponents C2C3.1 and C2C3.2 . In addition, the component C2C2.1 provides a union trace model of itself and the subcomponents C2C3.1 and C2C3.2 . Mapping Rules for Supermapping Configurations. In addition to the derivation of inheritance structures to nested transformation components, specific derivation rules for the configuration variants of the supermappings are needed to represent abstract and concrete mappings in transformation nets as well as the applicability of supermappings for subclasses. In particular, the following four rules, which are summarized in Figure 9, are sufficient to generate transformation nets for all possible supermapping configurations. The mapping model shown in Figure 9 is used as an example input mapping model for describing the mapping rules and comprises a mapping between the superclasses C1 and C1’ of the LHS and RHS metamodels and between the subclasses C2 and C2’, whereby the subclass C3 of the LHS remains unmapped. Rule 1 - Concrete/Applicable Supermapping: When a supermapping is concrete, a transition is available in the outer transformation component for transforming direct instances of the superclass and indirect instances for which no specific mappings are available. Because only direct and indirect instances of subclasses without specific mappings should be transformed by the transition of the outer component, an OCL condition is attached on the inputPort which leads to the transition in order to reject tokens for which more specific mappings are available. Such constraints can be defined with the OCL function oclIsTypeOf
Leveraging Model-Based Tool Integration
273
which gets as parameters the superclass and all subclasses for which no specific mappings have been defined in the mapping model (cf. OCL condition oclIsTypeOf(C1 |C3)). If there is a more specific mapping between subclasses, a nested component is produced and the tokens are not streamed via the superclass mapping, instead the subplace generated from the LHS subclass gets an additional arc which leads to a more specific transformation component. Rule 2 - Abstract/Applicable Supermapping: Although the supermapping is abstract, a transition resides directly in the outer component, which is not applicable for direct instances but for transforming all indirect instances for which no specific mapping has been applied (cf. OCL condition oclIsTypeOf(C3)). Rule 3 - Concrete/Non-Applicable Supermapping: If a supermapping is defined as concrete and non-applicable for unmapped subclasses then an outer component is produced which consists of a transition for transforming direct instances of the superclass (cf. OCL condition oclIsTypeOf(C1)). Rule 4 - Abstract/Non-Applicable Supermapping: When a supermapping is abstract and non-applicable for unmapped subclasses only the outer component is generated for providing a union trace model for its submappings. This is sufficient, because neither direct instances nor indirect instances have to be transformed by such a component. Design Alternatives. The following three design alternatives exist for transformation nets to model the applicability of the supermapping transition on subPlaces. First, we could extend the place modeling constructs with tags such as “superTransition is applicable”. However, the introduction of such a transformation net feature would violate our design goal that the transformation net formalism should not be changed. The second possibility is to generate for each unmapped class an additional arc from the corresponding source place to the outer component generated for the supermapping. This variant would lead to much more complicated transformation nets and to a huge amount of duplicated arcs, which simply does not pay off the information gain for the user. Therefore, we decided for a third variant, namely the usage of OCL constraints as explained for Rule 1 to 3. Example. To summarize this section, a concrete integration example, as shown in Figure 10, is discussed on the mapping view and on the transformation view. In the LHS metamodel, a class Person is specialized into Supplier, Employee, and Customer classes. The RHS metamodel consists also of a superclass Person, and of Client, Staff, and ShareHolder subclasses. Each LHS class can be mapped to a RHS class, except the class Supplier. Hence, the LHS class Person is mapped with a C2C mapping operator to the RHS class Person. The properties of this C2C are set to isAbstract=FALSE and Applicable4SubClasses=TRUE. Consequently, each instance of the LHS class Person is transformed into an instance of the RHS class Person, as well as each instance of a subclass which has no further refinement mapping is also transformed into an instance of the RHS Person class. For example, each instance of the class Supplier becomes an instance of the class Person. Additionally, the name attribute of the LHS class Person is
274
G. Kappel et al.
LHS
RHS
Bridging MappingModel
SourceMM
TargetMM
M Mapping gView
C2C
Person
Person
A2A
name:String
name:String C2C
Client
Customer cuNr:int
Supplier suNr:int
C2C
clNr:int
Employee p y
Staff
A2A
emNr:int
Share Holder
stNr:int
shNr:int
A2A
Transform SourcePlaces
Transform
Transform
TransformationLogic
TargetPlaces
C2C
Person Supplier
Employee
Person
history
Customer
Client
C2C
ShareHolder Staff
Transformation nView
history
C2C history
Person_name
Person_name
A2A
Employee_emNr
Staff_stNr history
A2A
Customer_cuNr
Client_clNr history
Supplier_suNr
ShareHolder_shNr
A2A
history
Fig. 10. Inheritance between C2C Mappings - Example
mapped by an A2A mapping operator to the name attribute of the RHS class Person. The subclasses Employee and Customer of the class Person on the LHS are mapped by C2C mappings to Staff and Client of the RHS, respectively. Additionally, the attributes of these classes, namely Customer.cuNr, Employee.emNr, and Client.clNr, Staff.stNr, are mapped by A2A mappings, respectively. Due to the fact that each of the subclasses inherit the attributes of the superclass – the attribute Person.name – the A2A mapping between the superclasses is also inherited by the C2C mappings by setting the superMappings reference to the C2C mapping which resides between the Person classes.
Leveraging Model-Based Tool Integration
275
The corresponding transformation net for the presented mapping model is depicted in the Transformation View of Figure 10. The Person classes become places which comprise for each subclass an inner place. As subclass places are nested in superclass places, the inheriting submappings are nested in the transformation component which corresponds to the supermapping. The outer transformation component, corresponding to the supermapping, contains a transition, because the isAbstract property of the C2C mapping is set to FALSE. Furthermore, due to the isApplicable4SubClasses property of the C2C mapping, which is set to TRUE, the outer transformation component of the transformation net owns an additional OCL constraint, namely oclTypeOf(Person|Supplier). Due to readability purposes, we refrain from displaying these features in Figure 10. Consequently, each direct instance of type Person from the LHS is transformed into an instance of class Person on the RHS. Furthermore, this OCL constraint ensures that each instance of subclasses of the class Person, which has no refined mapping, is also transformed by the supermapping into an instance of type Person on the RHS. The attribute Person.name can be transformed only if the containing instance which can be of type Person, Employee, or Customer has been already transformed. Consequently, the A2A transformation component for name values must be in the context of three C2C transformation components. This is achieved by the trace model provided by the black port in the middle of the bottom of the outer C2C transformation component. This trace model unifies the individual trace models of the C2C transformation components. The other A2A operators are connected to the gray ports which link directly to individual trace models of the nested components. This example shows the value of using an explicit notion of trace models together with a union of trace models for automatically synchronizing transformation net components. With the automatic synchronization provided by the Petri Net semantic, the streaming of dependent elements such as attribute values and links comes for free. Thus, no additional control structures and OCL constraints are required, and the derivation of transformation nets from mapping models is straightforward.
5
Evaluation
In this section, we discuss the applicability of our approach by (1) elaborating on how additional tool integration requirements, namely roundtrip capabilities, may be supported by our approach and by (2) comparing the efforts of building such roundtrip transformations using the proposed mapping approach and using model transformation languages. Roundtrip Transformations. In case modeling languages of two different tools are not entirely overlapping, i.e., some modeling concepts are available in one modeling language which cannot be expressed in the other modeling language, a transformation may be lossy. Thus, although transformations from tool A to tool B and vice versa are available, the initial model Ma of tool A may be
276
G. Kappel et al.
different from the roundtrip result Ma which is computed by translating Ma into Mb via the transformation Ta2b and the application of Tb2a on Mb to produce Ma . The main reason for not roundtripping transformations is the fact that bijective mappings are not always possible to establish between metamodels as for example reported in [27]. Here the challenging question arises, how to deal with such cases in order to ensure roundtripping transformations. In the ModelCVS project4 , besides other integration efforts, we have bridged the Domain Specific Language (DSL) for defining database schemas of the AllFusion Gen5 (AFG) modeling tool to the class diagram concepts of the UML Rational Software Modeler6 tool. Thereby, the first case study was to bridge structural modeling, i.e., the AFG Data Model with the UML Class Diagram. The first attempt was to bridge AFG with plain UML. The resulting bridge was not appropriate for using the tools in combination. Because, although we have defined for each AFG modeling concept a specific mapping to UML, a lot of information was lost during roundtrip or even though after the first step when moving from the AFG to UML. Table 1 summarizes the roundtrip scenario by depicting some model metrics for each step in the roundtrip. Table 1. Model Metrics for Data Model/Class Diagram RoundTrip (RT) Metrics Initial AFG Model UML Model AFG Model after RT Diff in % #Objects #Values #Links #Containment Links File Size
156 1099 44 155 32,8 KB
165 156 54 164 16 KB
156 156 36 155 14,6 KB
0 85,8 18,2 0 55,5
The main reason for the massive loss of information was the fact that on the attribute level only a minimal overlap between the languages exists. In most cases, only the name attribute of the modeling concepts may be bridged, but all platform specific attributes of the AFG modeling language such as database optimization information may not. When we take a look at the model metrics in Table 1, the initial AFG model and the generated UML model have nearly the same amount of objects and containment links, only some derived objects are additionally instantiated in the UML model. This means, the same model structure can be reproduced on the UML side. However, when we compare the amount of values, we see that a huge amount of information gets lost in the first step. In particular, when comparing the number of values of the initial AFG model and the resulting AFG model after roundtrip, 85,8 % of values are lost during the roundtrip. For links which are not containment links we see that more links exist in the generated UML model compared to the initial AFG model. This is due to the fact, that also derived links are generated for the aforementioned 4 5 6
http://www.modelcvs.org http://ca.com/us/products/product.aspx?ID=256 http://www-306.ibm.com/software/awdtools/modeler/swmodeler
Leveraging Model-Based Tool Integration
277
additionally derived objects. Therefore, even though we have more links and objects on the UML side, less information is expressed and some links cannot be reconstructed when transforming the UML models back to AFG models. Finally, the information loss has of course an effect on the file size, namely the resulting file after roundtrip has only half the size of the initial file. Table 2. Model Metrics for Data Model/Class Diagram Roundtrip Revisited Metrics Initial AFG Model UML Model AFG Model after RT Diff in % #Objects #Values #Links #Containment Links File Size #Annotations #Key/Value Pairs
156 1099 44 155 32,8 KB
165 156 54 164 58,5 KB
156 1099 44 155 32,8 KB
0 0 0 0 0
-
156 951
-
-
In Table 2, the model metrics are again presented, however, now for unmapped features (i.e., attributes and references) dedicated annotations are created on the UML side by using an enhanced transformation generation from mapping models tailored to preserving information during roundtrip. As one can see in the most right column of the table, no difference regarding the number of model elements between the initial AFG model and the AFG model after roundtrip exists. The information can be preserved by applying for each corresponding object an appropriate annotation allowing to use key/value pairs for saving attribute values which otherwise would be lost. This specific logic can be easily introduced in the transformation generation thanks to the declarative nature of the mapping models. Furthermore, a comparison of the initial AFG model and the resulting AFG model using the comparison facility of EMF Compare7 demonstrated that both were equivalent, thus no information has been lost during roundtrip. Metrics for Mapping Model and corresponding Model Transformations. In addition to the evaluation of the roundtrip capability of the generated bridge, the number of model elements of the manually created mapping model compared to the number of elements needed for the corresponding model transformations defined in the de-facto model transformation standard in Eclipse, namely the ATLAS Transformation Language (ATL) [8], is presented. This comparison should give an indication on how much effort requires the creation of the mapping model in contrast to building the integration artifacts manually from scratch by using a textual model transformation language such as ATL. Table 3 summarizes some metrics for the mapping model and for the corresponding ATL model transformations. We decided to use as metrics for the mapping model, first, the number of applied MOPs, and second, how many non-default property values (e.g., for setting inheritance relationships between MOPs) have to be set by the user, because this is exactly the work the user has 7
www.eclipse.org/modeling/emft
278
G. Kappel et al.
to do for creating the mapping model. For the transformation code we simply use lines of code (LOC) as metric, just to give an indication how much effort the manual implementation of the transformations for both directions (AFG2UML, UML2AFG) would cause. Table 3. Metrics Overview: Mapping Model vs. ATL Code Mapping Model Metrics Mapping Operator User Actions C2C A2A R2R Properties
6 1 7 7
Model Transformation Metrics ATL File Lines of Code AFG2UML
UML2AFG
120 overall 75% declarative 25% imperative 100 overall 86% declarative 14% imperative
For realizing model exchange between AFG and UML, in total 220 lines of ATL code are necessary, where most parts are declarative rules. However, for using annotations within the transformation for saving unmapped values, imperative code is needed, e.g., for applying annotations and for setting key/value pairs when moving from AFG to UML as well as for assigning key/value pairs as attribute values when we are going back from UML to AFG. Instead, using the presented mapping approach, we are able to develop the same bridge using only 14 MOPs and setting 7 properties of the applied MOPs from which the transformations in both directions are derived. For dealing with annotations for setting unmapped values, the generator for the transformations is capable of producing this dedicated logic without requiring the user to define mappings for this aspect. Only the generator has been extended with one additional generation rule which can be turned on/off as required. Compared to the manual authoring of model transformations where this aspect is intermingled with the rest of the code, the presented mapping approach allows for a faster and more systematic development of tool integrations.
6
Related Work
With respect to our approach of defining reusable mapping operators for resolving metamodel heterogeneities as a kind of mapping between metamodels we distinguish between three broad categories of related work: first, related work concerning our goal to design a framework for building reusable mapping operators in the field of MDE, and second, related work concerning our solution approach in the field of ontology integration. In addition, we elaborate on related approaches which employ Petri Nets as conceptual modeling language for defining model transformations. 6.1
Reusable Model Transformations
Generic Model Transformations. Typically model transformation languages, e.g., ATL [8] and QVT [21], allow to define transformation rules based on types
Leveraging Model-Based Tool Integration
279
defined as classes in the corresponding metamodels. Consequently, model transformations are not reusable and must be defined from scratch again and again with each transformation specification. One exception thereof is the approach of Varr´ o et al. [30] who propose a notion of specifying generic transformations within their VIATRA2 framework, which in fact resembles the concept of templates in C++ or generics in Java. VIATRA2 also provides a way to implement reusable model transformations, although it does not foster an easy to debug execution model as is the case with our proposed transformation nets. In addition, there exists no explicit mapping model between source and target metamodel which makes it cumbersome to reconstruct the correspondences between the metamodel elements based on the graph transformation rules, only. Transformation Patterns. Very similar to the idea of generic transformations is the definition of reusable idioms and design patterns for transformation rules described by Karsai et al. [1]. Instead of claiming to have generic model transformations, the authors propose the documentation and description of recurring problems in a general way. Thus, this approach solely targets the documentation of transformation patterns. Realization issues how these patterns could be implemented in a generic way remain open. Mappings for bridging metamodels. Another way of reuse can be achieved by the abstraction from model transformations to mappings as is done in our approach or by the ATLAS Model Weaver (AMW) [6]. AMW lets the user extend a generic so-called weaving metamodel, which allows the definition of simple correspondences between two metamodels. Through the extension of the weaving metamodel, one can define the abstract syntax of new weaving operators which roughly correspond to our mapping operators. The semantics of weaving operators are determined by a higher-order transformation that take a weaving model as input and generates model transformation code. Compared to our approach, the weaving models are compiled into low-level transformation code in terms of ATL which is in fact a mixture of declarative and imperative language constructs. Thus, it is difficult to debug a weaving model in terms of weaving operators, because they do not explicitly remain in the model transformation code. Furthermore, the abstraction of mapping operators from model transformations expressed in ATL seems more challenging compared to the abstraction from our proposed transformation net components. 6.2
Ontology Mapping for Bridging Structural Heterogeneities
In the field of ontology engineering, several approaches exist which make use of high-level languages for defining mappings between ontologies (cf. [9] for an overview). For example, in Maedche et al. [17], a framework called MAFRA for mapping two heterogeneous ontologies is proposed. Within this framework, the mapping ontology called Semantic Bridge Ontology usually provides different ways of linking concepts from the source ontology to the target ontology. In addition to the Semantic Bridge Ontology, MAFRA provides an execution platform for the defined mappings based on services whereby for each semantic
280
G. Kappel et al.
bridge type a specific service is available for executing the applied bridges. In [24], Scharffe et al. describe a library of so called Ontology Mediation Patterns which can be seen as a library of mapping patterns for integrating ontologies. Furthermore, the authors provide a mapping language which incorporates the established mapping patterns and they discuss useful tool support around the pattern library, e.g., for transforming ontology instances between different ontology schemas. The main difference to our approach is that ontology mapping approaches are based on Semantic Web standards, such as OWL and RDFS, and therefore contain mapping operators for typical description logic related mapping problems, e.g., union or intersection of classes. We are bridging metamodels expressed in MOF, a language which has only a partial overlap with OWL or RDFS, leading to different mapping problems. Furthermore, in contrast to the ontology mapping frameworks, we provide a framework allowing to build new mapping operators by using well-known modeling techniques not only for defining the syntax but also for the operational semantics of the operators. 6.3
Petri Nets and Model Transformations
The relatedness of Petri Nets and graph rewriting systems has also induced some impact in the field of model transformation. Especially in the area of graph transformations some work has been conducted that uses Petri nets to check formal properties of graph production rules. Thereby, the approach proposed in [31] translates individual graph rules into a place/transition net and checks for its termination. Another approach is described in [5], which applies a transition system for modeling the dynamic behavior of a metamodel. Compared to these two approaches, our intention to use Petri Nets is totally different. While these two approaches are using Petri Nets as a back-end for automatically analyzing properties of transformations by employing place/transition nets, we are using Colored Petri Nets as a front-end for debuggability and understandability of transformations. In particular, we are focussing on how to represent model transformations as Petri Nets in an intuitive manner. This also covers the compact representation of Petri Nets to eliminate the scalability problem of low-level Petri nets. Finally, we introduce a specific syntax for Petri Nets used for model transformations and integrate several high-level constructs, e.g., colors, inhibitor arcs, and pages, into our language. However, it has to be noted that the higher expressivity gained from high-level constructs comes with a negative impact on the analyzability of the Petri Nets.
7
Conclusion and Future Research Issues
In this paper we have presented a framework allowing the definition of mapping operators and their application for building metamodel bridges. Metamodel bridges are defined by the user on a high-level mapping view which represents the semantic correspondences between metamodel elements and are tested and
Leveraging Model-Based Tool Integration
281
executed on a more detailed transformation view which also comprises the transformation logic of the mapping operators. The close integration of these two views and the usage of models during the whole integration process further enhances the debugging of the defined mappings in terms of the mapping operators. The applicability of the framework has been demonstrated by implementing mapping operators for resolving structural metamodel heterogeneities8 . The proposed framework is expressive enough to define advanced composition mechanisms such as inheritance between mapping operators. This is achieved by the explicit notion of trace models for mapping operators in combination with the automatic synchronization provided by the Petri Net semantic. The presented framework has been applied in several modeling tool integration projects (for more details the interested reader is kindly referred to www.modelcvs.org and www.modeltransformation.net). It has to be mentioned that the presented approach is not only applicable for integrating structural modeling languages, but also for integrating behavioral modeling languages. For example, we have integrated the dialog flow modeling language of CA’s AllFusion Gen with UML state machines as well as with UML activity diagrams. The most interesting point of this case study was that we explored nearly the same metamodel heterogeneities as we explored when integration structural modeling languages. Thus, the presented metamodel heterogeneities seem to be modeling domain independent. The work presented in this chapter leaves several issues open for further research. In the following, we present four research issues that we believe are most important for the success of model-based tool integration. (1) Bridging Technical Spaces. Several modeling languages are not described with MOF-based metamodels as proposed by the OMG. Instead, text-based languages such as EBNF, DTD, or XML schema are employed. In order to use model-based integration frameworks which require MOF-based metamodels, converters are needed. In particular, not only the language definition must be converted into a MOF-based metamodel, also the models have to be transformed into instances conforming to the generated metamodels. This raises the question of how to produce such converters for bridging different technical spaces [15] with reasonable effort also in the light of the evolution of these languages. (2) Automatic Creation of Mapping Models. Another issue is the automatic creation of mapping models between two metamodels. With the rise of the semantic web and the emerging abundance of ontologies, several matching approaches and tools for automatically creating mapping models have been proposed, for an overview see [22,26]. The typical output of such tools are simple one-to-one correspondences. However, these correspondences cannot cope with structural heterogeneities between MOF-based metamodels as presented in this work. Therefore, a novel matching approach is needed which is capable to automatically generate mappings expressed with a more powerful mapping language. 8
For more details about the implementation, we kindly refer the interesting reader to www.modeltransformation.net
282
G. Kappel et al.
(3) Formal Verification of Mapping Models. As the correctness of the automatically generated target model fully depends on the correctness of the specified mapping model, formal underpinnings are required to enable verification of mapping models by proving certain properties like confluence and termination, to ease debugging of mapping models. The formal underpinning of CPNs enables simulation of mapping models and exploration of the state space, which shows all possible firing sequences of a CPN. In the future, it has to be determined how generally accepted behavioral properties, characterizing the nature of a certain CPN, e.g., with respect to confluence or termination, as well as custom functions, e.g., to check if a certain target model can be created with the given transformation logic, can be applied for interactive debugging and automatic verification of mapping models [33]. (4) The Role of Semantics. An open research problem in MDE is how to explicitly and formally specify the semantics of modeling languages. In the last decade several diverse approaches inspired from programming language engineering have been proposed. However, in contrast to syntax, currently there is no commonly approved or standardized approach as well as tool support for defining the semantics of modeling languages. Thus, often the semantics are only informally specified in terms of natural language—the most prominent example is UML—or the semantics are hard-coded in code generators, simulators, or interpreters. However, for bridging modeling tools, an explicit notion of semantics would be of paramount importance. For example, when integrating tools for modeling state machines, it can happen that the modeling tools provide completely the same syntax definition, but the execution of the same model in tool A can differ from the execution in tool B, as is for example reported in [4]. Thus, one of the biggest challenges in MDE is how to provide explicit semantic definitions for modeling languages and how to use these definitions to build and verify integration solutions.
References 1. Agrawal, A., Vizhanyo, A., Kalmar, Z., Shi, F., Narayanan, A., Karsai, G.: Reusable Idioms and Patterns in Graph Transformation Languages. In: Proceedings of the International Workshop on Graph-Based Tools (GraBaTs 2004) (2004) 2. Batini, C., Lenzerini, M., Navathe, S.B.: A Comparative Analysis of Methodologies for Database Schema Integration. ACM Computing Survey 18(4), 323–364 (1986) 3. Bernstein, P.A., Melnik, S.: Model management 2.0: manipulating richer mappings. In: Proceedings of the ACM SIGMOD International Conference on Management of Data, China (2007) 4. Crane, M.L., Dingel, J.: UML vs. classical vs. rhapsody statecharts: not all models are created equal. Software and System Modeling 6(4), 415–435 (2007) 5. de Lara, J., Vangheluwe, H.: Translating Model Simulators to Analysis Models. In: Fiadeiro, J.L., Inverardi, P. (eds.) FASE 2008. LNCS, vol. 4961, pp. 77–92. Springer, Heidelberg (2008) 6. Fabro, M.D.D., B´ezivin, J., Jouault, F., Breton, E., Gueltas, G.: AMW: a generic model weaver. In: Proceedings of the 1re Journe sur l’Ingnierie Dirige parles Modles, IDM 2005 (2005)
Leveraging Model-Based Tool Integration
283
7. Jensen, K.: Coloured Petri Nets: Basic Concepts, Analysis Methods and Practical Use. Springer, Heidelberg (1992) 8. Jouault, F., Kurtev, I.: Transforming Models with ATL. In: Bruel, J.-M. (ed.) MoDELS 2005. LNCS, vol. 3844, pp. 128–138. Springer, Heidelberg (2006) 9. Kalfoglou, Y., Schorlemmer, W.M.: Ontology mapping: The state of the art. In: Dagstuhl Seminar Proceedings: Semantic Interoperability and Integration (2005) 10. Kappel, G., Kapsammer, E., Kargl, H., Kramler, G., Reiter, T., Retschitzegger, W., Schwinger, W., Wimmer, M.: Lifting Metamodels to Ontologies: A Step to the Semantic Integration of Modeling Languages. In: Wang, J., Whittle, J., Harel, D., Reggio, G. (eds.) MoDELS 2006. LNCS, vol. 4199, pp. 528–542. Springer, Heidelberg (2006) 11. Kappel, G., Kargl, H., Kramler, G., Schauerhuber, A., Seidl, M., Strommer, M., Wimmer, M.: Matching Metamodels with Semantic Systems - An Experience Report. In: Workshop Proceedings of Datenbanksysteme in Business, Technologie und Web (BTW 2007) (2007) 12. Kappel, G., Kargl, H., Reiter, T., Retschitzegger, W., Schwinger, W., Strommer, M., Wimmer, M.: A Framework for Building Mapping Operators Resolving Structural Heterogeneities. In: Proceedings of 7th Int. Conf. on Information Systems Technology and its Applications (2008) 13. Kapsammer, E., Kargl, H., Kramler, G., Kappel, G., Reiter, T., Retschitzegger, W., Schwinger, W., Wimmer, M.: On Models and Ontologies - A Semantic Infrastructure Supporting Model Integration. In: Proceedings of Modellierung 2006 (2006) 14. Kashyap, V., Sheth, A.P.: Semantic and schematic similarities between database objects: A context-based approach. VLDB Journal 5(4), 276–304 (1996) 15. Kurtev, I., Aksit, M., B´ezivin, J.: Technical Spaces: An Initial Appraisal. In: Meersman, R., Tari, Z., et al. (eds.) CoopIS 2002, DOA 2002, and ODBASE 2002. LNCS, vol. 2519. Springer, Heidelberg (2002) 16. Ma, H., Shao, W.-Z., Zhang, L., Ma, Z.-Y., Jiang, Y.-B.: Applying OO Metrics to Assess UML Meta-models. In: Baar, T., Strohmeier, A., Moreira, A., Mellor, S.J. (eds.) UML 2004. LNCS, vol. 3273, pp. 12–26. Springer, Heidelberg (2004) 17. Maedche, A., Motik, B., Silva, N., Volz, R.: MAFRA – A MApping FRAmework for Distributed Ontologies. In: G´ omez-P´erez, A., Benjamins, V.R. (eds.) EKAW 2002. LNCS (LNAI), vol. 2473, p. 235. Springer, Heidelberg (2002) 18. Olsen, G.K., Aagedal, J., Oldevik, J.: Aspects of Reusable Model Transformations. In: Proceedings of the 1st European Workshop on Composition of Model Transformations (CMT 2006) (2006) 19. OMG: UML Superstructure Specification, version 2.0 formal/05-07-04 edition (2005) 20. OMG: Meta Object Facility (MOF) 2.0 Core Specification, version 2.0 formal/200601-01 edition (2006) 21. OMG: Meta Object Facility (MOF) 2.0 Query/View/Transformation Specification, version 1.0 formal/2008-04-03 edition (2008) 22. Rahm, E., Bernstein, P.A.: A survey of approaches to automatic schema matching. VLDB Journal 10(4), 334–350 (2001) 23. Reiter, T., Wimmer, M., Kargl, H.: Towards a runtime model based on colored Petri-nets for the execution of model transformations. In: 3rd Workshop on Models and Aspects, in conjunction with ECOOP 2007 (2007) 24. Scharffe, F., de Bruijn, J.: A language to specify mappings between ontologies. In: Proceedings of the 1st International Conference on Signal-Image Technology & Internet-Based Systems (SITIS 2005) (2005)
284
G. Kappel et al.
25. Schmidt, D.C.: Guest editor’s introduction: Model-driven engineering. IEEE Computer 39(2), 25–31 (2006) 26. Shvaiko, P., Euzenat, J.: A survey of schema-based matching approaches. In: Spaccapietra, S. (ed.) Journal on Data Semantics IV. LNCS, vol. 3730, pp. 146–171. Springer, Heidelberg (2005) 27. Stevens, P.: Bidirectional Model Transformations in QVT: Semantic Issues and Open Questions. In: Engels, G., Opdyke, B., Schmidt, D.C., Weil, F. (eds.) MODELS 2007. LNCS, vol. 4735, pp. 1–15. Springer, Heidelberg (2007) 28. Tratt, L.: Model transformations and tool integration. Software and System Modeling 4(2), 112–122 (2005) 29. Vallecillo, A., Koch, N., Cachero, C., Comai, S., Fraternali, P., Garrig´ os, I., G´ omez, J., Kappel, G., Knapp, A., Matera, M., Meli´ a, S., Moreno, N., Pr¨ oll, B., Reiter, T., Retschitzegger, W., Rivera, J.E., Schauerhuber, A., Schwinger, W., Wimmer, M., Zhang, G.: MDWEnet: A Practical Approach to Achieving Interoperability of Model-Driven Web Engineering Methods. In: Workshop Proceedings of 7th International Conference on Web Engineering (ICWE 2007) (2007) 30. Varr´ o, D., Pataricza, A.: Generic and Meta-transformations for Model Transformation Engineering. In: Baar, T., Strohmeier, A., Moreira, A., Mellor, S.J. (eds.) UML 2004. LNCS, vol. 3273, pp. 290–304. Springer, Heidelberg (2004) 31. Varr´ o, D., Varr´ o-Gyapay, S., Ehrig, H., Prange, U., Taentzer, G.: Termination Analysis of Model Transformations by Petri Nets. In: Corradini, A., Ehrig, H., Montanari, U., Ribeiro, L., Rozenberg, G. (eds.) ICGT 2006. LNCS, vol. 4178, pp. 260–274. Springer, Heidelberg (2006) 32. Wimmer, M.: From Mining to Mapping and Roundtrip Transformations - A Systematic Approach to Model-based Tool Integration. PhD thesis, Vienna University of Technology (2008) 33. Wimmer, M., Kappel, G., Kusel, A., Retschitzegger, W., Sch¨ onb¨ ock, J., Schwinger, W.: Right or Wrong? - Verification of Model Transformations using Colored Petri Nets. In: Proceedings of the 9th OOPSLA Workshop on Domain-Specific Modeling (DSM 2009) (2009) 34. Wimmer, M., Kusel, A., Reiter, T., Retschitzegger, W., Schwinger, W., Kappel, G.: Lost in Translation? Transformation Nets to the Rescue! In: Proceedings of 8th Int. Conf. on Information Systems Technology and its Applications (2009) 35. Wimmer, M., Schauerhuber, A., Schwinger, W., Kargl, H.: On the Integration of Web Modeling Languages: Preliminary Results and Future Challenges. In: Workshop Proceedings of 7th International Conference on Web Engineering (ICWE 2007) (2007) 36. Wimmer, M., Schauerhuber, A., Strommer, M., Schwinger, W., Kappel, G.: A Semi-automatic Approach for Bridging DSLs with UML. In: Workshop Proceedings of 7th OOPSLA Workshop on Domain-Specific Modeling (DSM 2007) (2007)
Data Model for Scientific Models and Hypotheses Fabio Porto1 and Stefano Spaccapietra2 1
LNCC – National Laboratory of Scientific Computing, Petr´ opolis, Brazil [email protected] 2 EPFL-IC – Database Laboratory, Lausanne, Switzerland [email protected]
Abstract. New instruments and techniques used in capturing scientific data are exponentially increasing the volume of data consumed by insilico research, which has been usually referred to as data deluge. Once captured, scientific data goes through a cleaning workflow before getting ready for analysis that will eventually confirm the scientist’s hypothesis. The whole process is, nevertheless, complex and takes the focus of the scientist’s attention away from his/her research and towards solving the complexity associated with managing computing products. Moreover, as the research evolves, references to previous results and workflows are needed as source of provenance data. Based on these observations, we claim that in-silico experiments must be supported by a hypotheses data model that describes the elements involved in a scientific exploration and supports hypotheses assessment. Adopting a data perspective to represent hypotheses allow high-level references to experiments and provides support for hypotheses evolution. The data model drives the proposal of a data management system that would support scientists in describing, running simulations and interpreting their results.
1
Introduction
The availability of important experimental and computational facilities nowadays allows many large-scale scientific projects to produce a never before observed amount of experimental and simulation data. This wealth of data needs to be structured and controlled in a way that readily makes sense to scientists, so that relevant knowledge may be extracted to contribute to the scientific investigation process. Current data management technologies are clearly unable to cope with scientists’ requirements [1]. In particular, traditional database technology, which has been designed for supporting business-oriented applications, is poorly suitable for scientific applications. Only a few scientific projects, such as the Sloan Digital Sky Survey, claim to successfully use relational databases [2]. More frequently, science applications require a complex data model [1] to represent meshes, sequences and other scientific abstractions, support for knowledge evolution, and close integration with Kaschek/Delcambre (Eds.): The Evolution of Conceptual Modeling, LNCS 6520, pp. 285–305, 2011. c Springer-Verlag Berlin Heidelberg 2011
286
F. Porto and S. Spaccapietra
simulators and scientific workflow engines. Globally, the list of scientific data management requirements is long and variable, but would also include support for intense world-wide collaboration, and reasoning with uncertain knowledge and uncertain models for an incremental elaboration of scientific knowledge. A promising approach to better scientific data management is based on the concept of scientific models. Building on the informal introduction to the concept given in [3], we define a scientific model as a formal representation that 1) synthesizes the understanding about a studied phenomenon, entity or process, and 2) allows scientists to eventually simulate the behaviour of the real world processes they investigate and compare the expected behaviour against experimental results. Both goals, understanding and simulating, build upon the formalization of the process at hand, e.g. as a set of mathematical formulae or as logic programs. Formalization, indeed, ensures unambiguous interpretation of what is being stated and precisely describes what the implementation must achieve in order to run experiments that will support or contradict their assumptions. The formalization stands at the core of the scientific model. The latter, however, includes much more than formal definitions. Its aim is to include all the knowledge that in one way or another is related to or has contributed to the elaboration and assessment of the formalization. This is a fundamental requisite for collaboration between different teams in the elaboration of the scientific model and for its reuse by other groups. Scientists indeed do not adopt a scientific result without examining how and by whom it has been established, what were the underlying hypotheses and supporting previous works, what were the experimental frameworks that allowed the results to be produced and assessed. It is this holistic view of scientific work that the scientific model concept intends to support. From a data management perspective, scientific models encompass all the information used and produced during a scientific exploration process. This information is contextualized by including a link to the corresponding scientific domain, bibliographic references, provenance data, and a description of the observed phenomenon that is represented by the formalization. Part of a scientific model is its computational model, which we define here as an implementation-oriented specification that translates the formalization into a computable process, including the details about the software and hardware resources. Experiments, essential to the scientific process, are run as defined by the computational model to simulate the behaviour of the studied phenomenon and its interaction with the interfacing environment, according to the specifications in the scientific model. The elaboration of the computational model is a complex but necessary task. It allows scientists to confront their simulation hypotheses with experimental results, reiterate in different experimental settings, and with possibly updated specifications reflecting changes in the hypotheses, eventually and hopefully to an accurate representation of the studied phenomenon. This retry and refine process, known as model tuning, includes both modifying programs (i.e. changing the behaviour) and fitting new input parameter and set-up values. In many cases when experimental data are available, regression analysis provides formal procedures
Data Model for Scientific Models and Hypotheses
287
for model fitting, i.e. the process of fitting the model to match with existing experimental results [4]. It is a fact that the complexity of specifying and running a computational model and managing all the resources produced during a scientific endeavour takes the scientist attention off the phenomenon being investigated onto implementation concerns [5]. State of the art approaches to computational simulation resort to scientific workflows languages, such as SCUFL [6] and BPEL [7], to specify workflows and to their corresponding running environments [8–10] for models evaluation. Tailored environments like MATLAB [11] and Neuron [12] can be used as support for small-scale, non data-intensive simulations. While some of these have attracted a large user community [13], they fail to provide an integrated scientific model environment as the one proposed in this paper, which aims at providing an integrated management of all the facets of the studied phenomenon, the derived models and data therein produced. Having said that, the scientific model management system proposed in this chapter does not substitute theses simulation tools; neither does it with respect to scientific workflow systems. Rather it may integrate them as part of an environment to support the scientists. This work intends to support scientists in specifying, running, analysing and sharing scientific models and data produced by running simulations. Adopting a data-oriented perspective for scientific model representation, we provide formal definitions for the scientific model concept and its components, as informally proposed in [3]. Simulations are expressed using a high-level query language enabling runtime optimizations. The simulation therein specified may refer to software implemented in traditional programming languages as C or Fortran, or using a simulation tool. The data model is grounded in XML and tightly related to domain ontologies that provide formal domain descriptions and uniform terminology. Our intention is to use the proposed model as the basis for a computational environment designed to help scientists manage the resources associated to insilico scientific exploration. Using such an environment, scientists may search for existing scientific models, specify and run simulations. The results therein produced generate complex data to be analyzed and metadata to base queries. A by-product of managing the metadata and data is the provenance information [14] gathered during the research exploration. Initially, a framework for modeling and simulating single scientific models is discussed. Next, the basic framework is extended in two directions. In one direction, support for complex scientific models is added. A complex scientific model integrates other scientific models (in full or in part) in its definition, leading to a compositional network of scientific models. Higher-level models can then be specified, e.g. through views, that export a unified representation of underlying scientific models. A second direction concerns the extension to support scientific hypothesis and its integration with scientific models. Finally, we sketch a software architecture for the computational environment to be built based on the proposed to which we coin the name of scientific model management system.
288
F. Porto and S. Spaccapietra
The remainder of this chapter is structured as follows. Section 2 discusses related work. Section 3 presents a running example extracted from the neuroscience domain to be explored throughout the chapter. Section 4 covers scientific model-driven modeling and Section 5 details the simulation language. Section 6 extends scientific models with hypotheses and Section 7 discusses some advanced model elements. Finally, Section 7 introduces a general architecture for a scientific model management system. The conclusion discusses our achievements so far and future work.
2
Related Work
Data and knowledge management supporting in-silico scientific research is a comprehensive topic that has appeared under the eScience or computational science label. It encompasses the semantic description of the scientific domain, the experiment evaluation through scientific workflow systems, and results analysis through a myriad of different techniques, among other in-silico related tasks. Given the broad class of application domains that may benefit from eScience related data management techniques, it has been postulated that there is little chance that a single solution would cover the diverse set of requirements coming from these domains [1]. The semantic description of scientific domains through ontologies [15] is one exception that has attracted the attention of the scientific community as a means to support collaboration through common conceptual agreement. In this line, GeneOntology [16] is probably the most well-known and successful example of practical adoption of ontologies in the scientific domain. Similarly, scientific workflows have become the de facto standard for expressing and running in-silico experiments, using execution environments, such as [8–10]. The problem of systematically capturing and managing provenance information for in-silico experiments has also attracted significant attention. According to [14], provenance information helps interpreting and understating results by examining the sequence of steps that led to a result. The analysis of provenance data helps with verifying that experiments were conducted according to acceptable procedures and eventually supports reproducibility of the experiment. Provenance information has been classified into two categories: prospective and retrospective [17]. The former refers to description of the computational tasks involved in the experiment, whereas the latter refers to all the information used and produced during the evaluation of the experiment. Some scientific workflow systems give support for gathering and publishing both prospective and retrospective provenance information, such as Vistrails [18] and REDUX [19]. The combination of prospective and retrospective information potentially supports the whole exploratory in-silico experiment with data and metadata. Querying provenance data retrieves experiment results and inform about experiment metadata. Conceptually, our work introduces hypotheses definition and its integration with experiments into prospective provenance. We believe that hypotheses enrich provenance information both qualitatively and quantitatively. Integrating scientific hypotheses in a data model is, however, not trivial.
Data Model for Scientific Models and Hypotheses
289
Hypotheses modeling was introduced in databases back in the 80’s [20]. In that context, one envisions a hypothetical database state, produced by delete and insert operations, and verifies whether queries are satisfied on that hypothetical state. This approach is, however, far from the semantics needed in the experimental settings that we are interested in. Closer to our objective is the logical model proposed in the context of the HyBow project [21, 22] for modeling hypotheses in the biology domain. Hypotheses (H) are represented as a set of first-order predicate calculus sentences with free quantifiers. In conjunction with an axiom set specified as rules that models known biological facts over the same universe, and experimental data, the knowledge base may contradict or validate some of the sentences in H, leaving the remaining ones as candidates for new discovery. As more experimental data is obtained and rules identified, discoveries become positive facts or are contradicted. In the case of contradictions, the rules that caused the problems must be identified and eliminated from the theory formed by H. The approach adopted by Hybrow supports hypotheses validation in the spirit of what we aim to represent, i.e. as a formal definition to be confronted with experimental results. According to what has been discussed above concerning modeling of hypotheses in a scientific framework, the Hybrow model does not offer any support for simulations. Critical is the observation that the adopted modeltheoretical approach for hypotheses validation is not sufficient for representing simulation based research. This is due to the fact that in a model-theoretical approach, the validation of hypotheses only considers the satisfiability of the logical implications defined in the model with respect to an interpretation. This is fine for qualitative assessment of hypotheses but it is insufficient for simulationbased research, in which validation is decided based on the quantitative analysis between the simulation results and the observations. In this context, our work aims at integrating hypotheses with the simulation environment bridging the gap between qualitative and quantitative representation.
3
A Neuroscience Scientific Model
In this section, we introduce a running example taken from scientific models developed for the neuroscience domain. According to Kandel [23], “the task of neural science is to explain behavior in terms of the activities of the brain”. Numerous computational neuroscience groups investigate scientific models that aim at explaining some specific behavior. A classical example from these scientific models is the “axon membrane action potential”, a model proposed by Hodgkin and Huxley [24] that describes how action potentials traverse the cell membrane. An action potential is a pulse-like wave of voltage that can traverse certain types of cell membranes, such as the membrane of the axon of a neuron. The model quantitatively computes the potential between the interior and the extracellular liquid, based on the flow of ions of sodium (N a+ ), potassium (K + ) and a leakage flow representing all other channel types. The formalization in the model is given by the following mathematical equation:
290
F. Porto and S. Spaccapietra
I = m3 h gN a(E − EN a ) + n4 gk (E − Ek ) + gL (E − EL )
(1)
In (1), gi,i={N a,K,L} is a time-dependent variable that represents the membrane conductance for sodium, potassium and leakage; Ei models the equilibrium potential for each ion channel; E is the membrane potential; and, n, m and h are parameters controlling the probability of the sodium or potassium gates to be opened. The total ionic current across the membrane of the cell is modeled by the variable I.
Fig. 1. Single Neuron Model using the Hodgkin&Huxley model domain ontology
This action potential model simulates the variation of voltage in the cell when applied to a neuron compartment. It has been used, for example, in the formulation of the scientific model of a single neuron. Fig. 1 shows the domain ontology that supports the conceptual representation of a single neuron with its various compartments and the membrane behaviour given by the Hodgkin&Huxley (HH) model. The Hodgkin&Huxley class in the ontology represents the corresponding model with the input and output parameters conforming to equation (1). From the scientific model specification, a computational neuroscientist will conceive programs that implement the behaviour defined by equation (1). Next, when running a simulation, the program receives input values for the parameters identified in equation (1) and produces the total ionic current across the membrane (variable I).
4
Data Models for Scientific Models
During in-silico scientific investigation, scientists explore, produce and register data and metadata describing a phenomenon and associated simulations. Different scientific products are generated and need to be managed. Scientific model metadata covers provenance as well as contextual and descriptive information that drive scientific model searching and querying. Similarly, computational model metadata are used as the basis for the automatic instantiation and evaluation of simulations and serve as context to qualify input and output data. To structure this wealth of information, we propose a data model for scientific models data and metadata that is presented in this section.
Data Model for Scientific Models and Hypotheses
4.1
291
The Observed Phenomenon
The starting point of an in-silico scientific investigation is the clear specification of the phenomenon one attempts to explain. The formal description of a phenomenon includes a domain ontology, setting the formal conceptual representation of the domain in which the phenomenon is inserted, a phenomenon title, and a informal textual description. A Phenomenon is represented in the data model as in equation (2), where P hid is the phenomenon identification and Oph refers to the domain ontology URL: P h =< P hid , Oph , T itle > 4.2
(2)
Scientific Model
A scientific model provides a comprehensive description of the scientists’ interpretation of the observed phenomenon. Of prime importance is a formal representation of the scientific phenomenon interpretation, possibly using mathematical formulae, and a reference to the phenomenon it attempts to explain. Once a scientific model is inserted into the same semantic domain as the phenomenon it refers to, the Phenomenon ontology covers its domain description. In addition, the scientific model (SM) description includes bibliography references and other metadata supporting model presentation. Formally, a scientific model is expressed as a sextuplet, as presented in (3). In equation (3), SMid is the scientific model identification and P Hid denotes the phenomenon identifier. The URL in OMF references the ontology describing the scientific model formalization. Finally, a list of bibliographic references, images and annotations are pointed by B, I, and A, respectively. SM =< SMd , P hid , OMF , B, I, A >
(3)
The Hodgkin&Huxley model, presented in Section 3, can be depicted as a scientific model. Its data view is illustrated in Fig. 2. It provides metadata to support basic search queries over a scientific model database and reasoning capabilities on the theory specified by the scientific model ontology. 4.3
Computational Model
The definition of the scientific model suggests an explanation for a phenomenon using some formal language. In order to run in-silico experiments, a scientist develops a computational representation of the scientific model. Although desirable, an automatic mapping from a formal description of a SM to its computational model is still not feasible, requiring engineering efforts from the scientific group. Nevertheless, once a computation model has been specified and built, an engine may read such specification and automatically instantiate an execution if input data is provided. The computational model (CM) description according to the proposed data model identifies required metadata for such automatic instantiation.
292
F. Porto and S. Spaccapietra
Fig. 2. The Hodgkin&Huxley scientific model representation
In this context, the Environmental and Domain ontologies contribute to disambiguate CM specifications. The Environmental ontology describes the execution environment associated to the CM, including: programming language specification, programming environmental parameters, libraries, documentation, input and output parameters, initialization procedure, and executing programs. Computational model input/output parameters are structured into two formats: XML and attribute-value pairs. Parameters that are read or produced by the simulation are structured into a XML document, whereas set-up values are expressed as attribute value pairs. In order to be able to capture output produced by the underlying program, an outputWrapper class must be provided that transforms the output into a set of attribute value pairs mapped into a XML structure. A CM is formally defined as a 7-tuplet CM =< CMid , SMid , XOE , XOP h , Mi , Mo , A >
(4)
In (4), CMid is the CM resource identification; SMid refers to the associated scientific model, XOE and XOP h are the XML serializations of the environmental ontology, and of the Phenomenon domain ontology, respectively1 . Mi and Mo are the mappings between the underlying program input and output parameters and the corresponding domain ontology properties (XML tree leave nodes). Finally, A corresponds to annotations identifying authoring information. Fig. 3 illustrates the representation of a computational model implementing the scientific model SM01 from Fig. 2. 4.4
Simulation
The previous elements of the model present metadata used in scientific model formalization and querying; they also support automatic instantiation of the 1
The serialization of ontologies into an XML structure follows a detailed technique not presented here. The main intuition is to form a tree structure having concepts as nodes and edges represent associations between concepts.
Data Model for Scientific Models and Hypotheses
293
Fig. 3. A Hodgkin-Huxley Computational Model
computational model. We now turn to the expression of simulations. Simulations are in-silico experiments run to assess the scientific model against the observed phenomenon. By analogy with databases, where users’ data is intentionally expressed in queries, we call the specification of a simulation a simulation query. Let us define a simulation database DB S = {VCM 1 , VCM 2 , ..., VCM m }, where VCM i , 1 ≤ i ≤ m, are n-ary data views on the computational model. Given a computational model CMi , a corresponding n-ary data view VCM i abstracts the software program behavior associated to the CM, by exposing its input and output parameters as data attributes and completely hiding its implementing programs. This is similar to modeling user-defined functions as relations in databases [25]. Consider, for instance, the data view below corresponding to the CM in Fig. 3: HodgkinHuxley(i : (m, n, h, gN a , gK , gL , EN a , EK , EL ), o : (I))
(5)
The HodgkinHuxley data view presents one attribute for each input/output CM parameter. Querying a data view requires binding attributes in the input parameter set (prefixed with i:) to input values and retrieving results associated to output parameters (prefixed with o:). In this context, a simulation query (S) interrogates a data view VCM i by providing binding values and obtaining results from output attributes. Section 5 formally presents the simulation query language.
5
Simulation Language
In-silico experiments are commonly expressed using workflow or some sort of scripting languages. We aim to leverage the expression of simulations by providing a high-level query language with which scientists may express a large class of workflows, notably those that can be modeled as directed acyclic graphs.
294
F. Porto and S. Spaccapietra
In this context, a simulation query is specified as an expression in nonrecursive Datalog [26] comprising a head and a body. The body is a boolean expression composed of a conjunction of predicates, whereas the head specifies a predicate holding variables containing the expected simulation results, necessarily appearing in one of the predicates in the body. Users interface with simulation queries by providing the input parameters and set-up values needed for the evaluation of the predicates, and getting in return the output values bound to the variables defined in the head. We start with presenting the syntax and semantics of simulation query predicates. 5.1
Simulation Predicate
A simulation query predicate is specified as: Si ((Vi , Wi ); (Xi , Xo ); (Ii , Oi ), IS )
(6)
In (6), Si labels the simulation query predicate and associates it to the corresponding VCM resource identification. Vi and Wi are the two sets of variables defined to refer to values provided as input or produced as output when running the underlying CM program. The set of input and output parameter’s values are provided by the XML documents Xi and Xo , respectively. Note that the associated CM definition specifies the schemas for Xi and Xo . For example, using the CM in Fig. 3, the Xi document can be obtained from the result of the XPath expression “/CM/DomainOntology” over the Hodgkin&Huxley CM element XOP h , and by filling its leaf nodes with the input values. Thus, /Neuron/Axon/Hogking-Huxley/m = 0.1 illustrates a possible value assignment for the input parameter m. Ii and Oi are the mappings defining the correspondence (see Definition 1) between the input and output variables in Vi and Wi and the input and output parameter values in Xi and Xo . Finally, IS represents simulation set-up parameters. Definition 1. Correspondence assertions in Ii and Oi are specified as $x = Path, where $x is a variable in {Vi ∪ Wi } and Path is an XPath [27] expression pointing to a data element in Xk , k={i,o}, whose leaf node is either an input parameter value or an output value. Having described the syntax for individual simulation predicates, we can proceed to define the semantics of body expressions. We start with defining the semantics of a single simulation predicate. The definition for complete body expressions follows. 5.2
Semantics of a Single Simulation Predicate
A single simulation predicate returns a boolean value of its evaluation according to the definition in Definition 2 below with respect to its syntax in (6) and the CM specification in (4).
Data Model for Scientific Models and Hypotheses
295
Definition 2. A simulation predicate Si evaluates to true iff given an Xi holding the set of parameter values input to the program implementing the corresponding CM, according to Mi , there exists a Xo whose leaf values are produced by the evaluation of the referenced program and that is built from the mappings in Mo . 5.3
Semantics of the Body of a Simulation Expression
More elaborate conjunctive expressions can be composed from single simulation predicates to form the body of a simulation. The semantics of a conjunction of simulation predicates in the body of a simulation is defined in Definition 3. Definition 3. A conjunction of simulation predicates s = s1 ∧ s2 ∧ ... ∧ sn , s is considered to hold true if the conjunctive expression on the right evaluates to true. Moreover, if more than one simulation predicates si and sj in s refer to the same variable, for instance $x, then they share a single associated value. In addition, the shared variable must hold a single binding to a value, either provided as input or produced as output by an underlying program computation. Note that the restriction regarding sharing variables among simulation predicates leads to data-dependency relationships, in which the simulation predicate holding the value associated to the shared variable shall precede in evaluation order the remaining simulation predicates sharing that particular variable. Moreover, variable sharing introduces a particular mode of value assignment to data elements in Xi , which replaces that of the node corresponding to its associated path. Finally, given a body that evaluates to true, then the head of the simulation identifies the variables in the query body whose values are returned as the simulation results, such that if K is the set of variables in the head, then K ⊆ (Vi ∪Wi ), for 1 ≤ i ≤ n, with n being the number of simulation predicates in the body. 5.4
Simulation Query
A simulation query combines the head and its body into a clause as illustrated in (7), according to definitions 1, 2 and 3. S(K) := S1 ((V1 , W1 ); (Xi1 , Xo1); (I1 , O1 ), IS1 )∧ S2 ((V2 , W2 ); (Xi2 , Xo2); (I2 , O1 ), IS2 )∧ ...∧ Sn ((Vn , Wn ); (Xin , Xon ); (In , On ), ISn )
(7)
An example of a simulation query is given in (8). This particular query returns the total ionic current across the membrane ($I) according to the parameter values specified in the input document HHCM 01I . As discussed before, the user must provide a mapping from each query variable to the corresponding data element of the domain ontology XML serialization document. In this example, the input and output XML documents, Xi and Xo , are illustrated by documents HHCM 01I and HHCM 01O , respectively, both of type Neuron.
296
F. Porto and S. Spaccapietra
S($I, $z) := CM01 (i : ($m, $h, $n, $gN a, $gK , $gL, $EN a , $EK , $EL ), o : ($I)); (HHCM 01I , HHCM 01O ); ($m = /N euron/Axon/Hodgkin − Huxley/m, ....2 , $I = /N euron/Axon/Hodgkin − Huxley/I)) ∧ CM023(($I, $z); (ACM 02I , ACM 02O ); ($z = /Analysis/result))
6
(8)
Hypothesis Modeling
Up to now, we have focused our discussion on scientific models and the entities that derive from them during a research process. An important missing aspect from a conceptual point of view is the expression of scientific hypotheses, which drives research by proposing an explanation for a studied phenomenon. Indeed, according to wikipedia [28], a scientific hypothesis is used as a tentative explanation of an observation, but which has not yet been fully tested by the prediction validation process for a scientific theory. A hypothesis is used in the scientific method to predict the results of further experiments, which will be used either to confirm or disprove it. A successfully-tested hypothesis achieves the status of a scientific theory. The explanation a scientific hypothesis conveys a relationship between the causal phenomena and the simulated one, namely, that the simulated phenomenon is caused by or produced under the conditions set by the causal phenomena. Thus, by running the simulations defined by the antecedents in the causal relationship, the scientist aims at creating hypothetical simulation of the studied phenomenon. In this context, the scientific hypothesis is an element of the scientific model data model that may replace a phenomenon. When computing a simulation based on a scientific hypothesis, i.e. according to the causal relationship it establishes, the output results may be compared against phenomenon observations to assess the quality of the hypothesis. The scientific hypotheses therein introduce a valuable contribution by bridging the gap between qualitative description of the phenomenon domain and the corresponding quantitative valuation obtained through simulations. Thus, in our modeling approach we aim at coming up with a representation of scientific hypotheses that may be used in qualitative ( i.e., ontological) assertions and with minimum tricks can be quantitatively confronted to phenomenon observations. Having said that, bridging this gap is a challenging job that we are starting to tackle and whose first results we report in this section. 6.1
Running Example
Let us consider a small variation on the scenario presented in Section 3. Suppose we want to feed a scientific visualization application with the temporal variation 2 3
The remaining mappings are not shown due to lack of space. The remaining mappings are not shown due to lack of space.
Data Model for Scientific Models and Hypotheses
297
on the value of the ionic current (I), assuming in other words, the ionic current is a function of time. The result is a time series showing the variation of the ionic current during an interval of time Δt. In addition, we will assume that independent scientific models are conceived to model the ionic current on each gate (i.e. sodium, potassium and leakage). In this revised scenario, the formulae in (1) can be re-written as: I = (1−d) m3 h(E − EN a )gN a (t)dt + (1−d) n4 (E − EK )gK (t)dt+ (9) (E − EL )gL (t)dt (1−d) In (9), d is the duration of the simulation and the membrane conductance gi is a function of the simulation time instant. The ionic current on each gate is modeled by a different scientific model, leading to the following computational models: ionicChannelN a, ionicChannelK and ionicChannelL . Given this new scenario, a scientist may formulate the following hypothesis concerning HodkingHuxley model: The total ionic current on a membrane compartment is a time function of the ionic current on the sodium, potassium and leakage channels. Under the scenario exposed above, the following entities have been identified: – phenomena: 1. total ionic current on a neural membrane compartment; 2. time dependent ionic current on a sodium ionic channel; 3. time dependent ionic current on a potassium ionic channel; 4. time dependent ionic current on a leakage ionic channel. – computational models: 1. ionicChannelN a 2. ionicChannelK 3. ionicChannelL – scientific hypothesis: The total ionic current on a membrane compartment is a time function of the ionic current on the sodium, potassium and leakage channels. 6.2
Hypotheses Model
In order to integrate scientific hypothesis with the scientific model data model, we formally define a Hypotheses Data Model (HDM). A HDM describes an experiment domain and is defined as HDM = {Ph , PhO , H , E , SM , CM , VCM , SQ , V }, where: – – – – – – – – –
Ph - is a set of phenomena, as defined in (1); PhO - is a set of phenomenon observations, as defined in Definition 4; H - is a set of hypothesis, as defined in Definition 5; E - experiments, as described in Definition 6; SM - is a set of scientific models, as defined in (3); CM - is a set of computational models, as defined in (4); VCM - computational model data views, as described in Section 7; SQ - is a simulation querie, is defined in (7); V - scientific model views.
298
F. Porto and S. Spaccapietra
Elements of HDM are n-ary relations with attributes defined in a set of concrete domains D, such as integers, strings, floats, etc. Definition 4. A phenomenon observation (PhO) is a temporal record of a phenomenon, quantitatively described by its attribute values. The specification in (1) for phenomenon observations is extended as: ph i (obid , date, V , U , A ), Ph i ∈ Ph . In PhO , ph i is a phenomenon observation set label, obid is an observation identifier, V =< V 1 , V 2 , ..., V k >, U =< a1 , a2 , ..., al >, with ai ,V j ∈ D m , D n , respectively, D m , D n ⊆ D , for all 1 ≤ i ≤ k, 1 ≤ j ≤ l. V represents a list of initial set-up values and U a list of phenomenon attributes. Finally, A is a list of annotations. Thus, a phenomenon observation models the observable entity in different states and time instants. The observed variation is recorded as PhO attribute values. In the example of Section 6.1, a scientist may register the following observations concerning totalIonicCurrent: totalIonicCurrent(ob1 , date, < m1 , n1 , h1 >, 0.001); totalIonicCurrent(ob2 , date, < m2 , n2 , h2 >, 0.002);
(10)
In the representations above (10), two instances of the phenomenon totalIonicCurrent are depicted. The observations include an identifier, the time instant of the observation, a set of initial state values, specifying the context on which the phenomenon was observed, and an attribute that quantitatively describes it. The latter serves as the basis for assessing hypotheses. Syntactically, we distinguish initial set-up values from observable attributes with an underline on the latter. Thus, in light of the example in Figure 6.1, we would have the following phenomena observation schemas involved in computing the ionic current in a membrane compartment: totalIonicCurrent (obid , date, IN A , IK , IL , totalIonic); ionicChannelN a (obid , date, t, m, h, IN A ); ionicChannelK (obid , date, t, n, IK ); ionicChannelL (obid , date, t, IL ). In the above schema, IN A , IK , IL represent the ionic current on the three gates of our running example. Phenomenon observations are the basis for the modeling activity. A scientist formulates hypotheses that may or may not be validated when compared to observations. Definition 5. A scientific hypothesis is the specification of a possible explanation for a given phenomenon. In this context, a scientific hypothesis is specified as H =< id, title, Ph id , F , SQ id >
(11)
Data Model for Scientific Models and Hypotheses
299
In (11), id refers to the hypothesis identifier; Ph id is the corresponsing phenomenon identifier; F is a comparison function, used in measuring the accuracy of hypotheses with respect to observations; and SQ id is an identifier for the simulation query that sets the causal relationship among the causing phenomena and the simulated phenomenon. Referring to our running example, the total ionic current phenomenon hypothesis schema would be defined as: totalIonicCurrent (hid , P htotalIonicCurrent , fi , SQtic );
Fig. 4. A scientific hypothesis
Observe that by providing the simulation query SQ tic in a hypothesis declaration, we are indicating that hypotheses are associated to a computational representation, expressed by the query pointed to by SQ id , and whose evaluation simulates the phenomenon identified by Ph id . Definition 6. An experiment (E) corresponds to the instantiation of a simulation, including the results obtained by running the simulation query associated with the hypothesis bound to input-values, and forms the basis for quantitative hypothesis validation. Experiments are formally specified as: ei (hj , date, pj , c, q, dist)
(12)
In (12), ei ∈ E ; hj is an experience identifier; pj = p1 , p2 , ..., pn , pk ∈ D k , D k ⊆ D , for all 1 ≤ k ≤ n, is the initial condition’s value set; c = (c1 , c2 , ..., cm ), cl ∈ D l ,D l ⊆ D for all 1 ≤ l ≤ m, is the set of attribute values used in validation against phenomenon observations; q is a set of phenomenon observations, which define the observations comparison set; and dist is a measure of distance, computed by hypothesis’ comparison function f between the experiment results and the explained phenomenon observations phj . Continuing with our running example, we have: Experiment: totalIonicCurrent (hi1 , < m1 , n1 , h1 >, 0.003, < ob1 , ob2 >, dist1 );
300
F. Porto and S. Spaccapietra
Simulation Query: totalIonicCurrent($totalIonic)= totalIonicCurrent ($IN A , $IK , $IL , $totalIonic) ∧ ionicChannelN a(t1 , m1 , h1 , $IN A )∧ionicChannelK (t1 , n1 , $IK )∧ ionicChannelL (t1 , $IL ) An experiment is planned to hold the same initial condition values as the phenomenon observation it attempts to reproduce. The results obtained by evaluating the computational models representing the causal phenomena are compared against phenomenon observation attribute values through hypotheses comparison function. We motivated the need for a scientific data model so as to permit knowledge evolution in sync with the investigation progress. Hypotheses foster evolution by allowing competing hypotheses to be individually specified and assessed against phenomenon observations. Similarly, by modifying a computational model, a new hypothesis may be specified. Finally, each instance of a hypothesis instance set represents a new scientific essay with a different input value set. Thus, hypotheses, experiments and competing hypotheses are three different dimensions through which scientists can register the evolution of the investigation while keeping provenance information about the progress of the research. The proposed model distinguishes three aspects of a scientific investigation: phenomenon experiment, simulation and formal representation. Table 1 presents a synthesis of the hypothesis data model classified according to these groups. Table 1. Main elements to be considered in the HDM Phenomenon
Phenomenon (Ph)
Simulation
Hypothesis (H) Scientific Model (SM)
Formal representaion
7
Phenomenon observation (PhO) Experiment (E)
Computational Model (CM)
Data View (VCM)
Simulation query (SQ)
Complex Scientific Models
In this section we briefly introduce complex scientific models representation to support the composition of computation models in the same vein as database views [29]. For this reason, we call the construct a simulation view (V). It allows simulation queries to be stored so that they can be re-executed later on or included in a more complex simulation. V provides users with an external perspective of a simulation through the set of input parameter values that configure the participants computational models. In addition, a simulation view establishes correspondences between the exported parameters and the ones specified on each simulation predicate taking part in the body of the simulation describing the view. A simulation view is expressed as follows:
Data Model for Scientific Models and Hypotheses Sv ((V, W ); (Xiv , Xov ); (Iv , Ov ); (ISv , Ms )) = S1 ((V1 , W1 ); (Xi1 , Xo1 ); (I1 , O1 ); IS1 )∧ S2 ((V2 , W2 ); (Xi2 , Xo2 ); (I2 , O2 ); IS2 )∧ ...∧ Sn ((Vn , Wn ); (Xin , Xon ); (In , On ); ISn )
301
(13)
The body of the simulation view is like the one in ordinary simulations, expressing a conjunction of simulation predicates. The difference appears in the head of the formula. Indeed, the latter exports an integrated view of the simulation predicates’ input and output parameters that appear in the body of the formula and that are specified in Xik and Xok , 1 ≤ k ≤ n. The two sets of correspondences, Ik and Ok , map the external view in Xiv , Xov to the corresponding parameters in the simulation predicates in the body, Xik , Xok . Thus, a correspondence assertion is expressed as Sv .path/dataelement ≡ Si .path/dataelement, where path is an XPath expression. In the same line as the input/output parameters, ISv expresses the uniform view of set-up parameter values appearing in the body of the formulae and Ms asserts the correspondences between the set-up data elements in ISv and those in the body.
8
Managing Scientific Models
A scientific model management system (SMMS) supports scientists in designing, creating and searching scientific model’s entities, and in managing the results of simulations and analyses. Fig. 5 depicts the main system functions structured into four layers. A user layer provides the interface for scientists to create and edit elements of the model and to request system services, such as running simulations, querying and reasoning. Users may query scientific model metadata as well as hypotheses data and simulation results. The metadata management layer stores scientific model metadata and supports metadata management services. In this work, scientific model metadata is based on a set of ontologies that guarantees uniform terminology among models and scientists. A transformation and selection service allows scientists to map ontology fragments to XML trees, which are then used in data model elements description. The catalog service manages metadata about scientific model and data, as well as supporting information such as ontologies, views (see section 5.5) and transformation rules. The service layer supports simulation evaluation, querying of simulation results and reasoning. We have extended the query processing system QEF [10] to cope with simulation queries evaluation. Finally, a data management layer supports distributed scientific models management and wrappers implementing complex data types, offering access to simulation results, such as graphs and time series. In this paper, the details of the architecture are not further explored. Similarly, the details regarding ontology management, transformation and alignment are left to future work.
302
F. Porto and S. Spaccapietra
Fig. 5. Scientific Model Management System Architecture
9
Conclusion
Managing in-silico simulations has become a major challenge for eScience applications. As science increasingly depends on computational resources to aid solving extremely complex questions, it becomes paramount to offer scientists mechanisms to manage the wealth of knowledge produced during a scientific endeavor. This chapter presented initial results aiming to contribute to this idea. We propose a data-centric semantic based data model with which scientists represent scientific hypotheses, scientific models and associated computational models. Scientific hypotheses are explanations of observable phenomena expressed through the results of computer simulations, which can be compared against phenomena observations. The model allows scientists to record the existing knowledge about an observable investigated phenomenon, including a formal mathematical interpretation of it, if one exists. Additionally, it intends to serve as the basis for the formal management of the scientific exploration products, as well as supporting models evolution and model sharing. Traditionally, scientific computation takes either a mathematical or computational view when modeling is discussed. Scientific workflows are the most practical example of its computational view. By taking a declarative perspective on scientific model we envisage various benefits. Firstly, a higher-level declarative language allows scientists to concentrate on the scientific questions he/she tries to answers saving precious time that otherwise would be expended on workflow definition and evaluation. Secondly, hypotheses are good candidates to bridge the gap between an ontological description of studied phenomena and the simulations that aim at explaining them. Finally, data views on scientific entities allow for querying and searching for models supporting scientific models sharing among different scientific groups.
Data Model for Scientific Models and Hypotheses
303
There are various opportunities for future work. In this chapter we havent introduced the ontological modeling for hypotheses. Different relationships among hypotheses may be expressed, such as: composition, similarity, use, etc. Investigating how to express these and other relationships among hypotheses and the phenomena, they attempt to explain, is an interesting topic. Furthermore, hypotheses evolve and eventually become a theory. It would be interesting to model and infer on such evolution. Taking into account hypotheses validation, the distance between a hypothesis instance and the referred phenomenon may be probabilistically interpreted. Thus, considering probabilistic inference models when running hypotheses formula needs to be investigated. Another path of research explores the use of provenance information to help on the investigation evolution. Considering data management on the results of simulations, there is a huge path to follow. Computational models usually deal with complex data types, such as meshes, grids, temporal series and etc. Integrating these data types into our model is future work. Besides, all the research work involving heterogeneous data are of relevance to allow the automatic communication between computational models involved in a simulation. We developed a first prototype system that implements the data model and the simulation query language on top of the QEF system. The system was designed in the context of scientific model management system architecture with a set of minimal services that scientists may expect from such an environment.
References 1. Stonebreaker, M., Becla, J., DeWitt, D., et al.: Requirements for Science Data Base and SciDB. In: Conference on Innovative Data Systems Research. CIDR, Asilomar, CA, USA (January 2009) 2. Szalay, A., Kunszt, P., Thakar, A., Gray, J., et al.: Designing and Mining MultiTerabyte Astronomy Archives: The Sloan Digital Sky Survey. In: ACM SIGMOD, Dallas, Tx, USA, pp. 451–462 (2000) 3. Hunter, J.: Scientific Models A User-oriented Approach to the Integration of Scientific Data and Digital Libraries. In: VALA 2006, Melbourne (February 2006) 4. Jaqaman, K., Danuser, G.: Linking data to models: data regression, Nature Reviews. Molecular Cell Biology (7), 813–819 (2006) 5. Silvert, W.: Modelling as a discipline. International Journal General Systems 30(3), 1–22 (2000) 6. http://www.gridworkflow.org/snips/gridworkflow/space/XScufl 7. Akram, A., Meredith, D., Allan, R.: Evaluation of BPEL for Scientific Workflows, Cluster Computing and the Grid. In: CCGRID, vol. 1, pp. 269–274 (May 2006) 8. Oinn, T., Greenwood, M., Addis, M.: Taverna: Lessons in creating a workflow environment for the life sciences. Concurrence and Computation: Practice & Experience 18(10), 1067–1100 (2006) 9. Altintas, I., Berkley, C., Jaeger, E., Jones, M., Ludascher, B., Kepler, M.: An Extensible System for Design and Execution of Scientific Workflows. In: Proceedings of the 16th Conference on Scientific and Statistics Database Management, pp. 423–424 (2004)
304
F. Porto and S. Spaccapietra
10. Porto, F., Tajmouati, O., Silva, V.F.V., Schulze, B., Ayres, F.M.: QEF Supporting Complex Query Applications. In: 7th Int. Symposium on Cluster Computing and the Grid, Rio de Janeiro, Brazil, pp. 846–851 (2007) 11. MATLAB, http://en.wikipedia.org/wiki/Matlab (last access 24/06/2008) 12. Neuron, http://www.neuron.yale.edu (last access 24/06/2008) 13. Roure, D., Goble, C., Stevens, R.: Designing the myExperiment Virtual Research Environment for the Social Sharing of Workflows. In: e-Science 2007 - Third IEEE Int. Conf. on e-Science and Grid Computing, Bangalore, India, December 10-13, pp. 603–610 (2007) 14. Freire, J., Koop, D., Santos, E., Silva, C.T.: Provenance for Computational Tasks: A Survey. Computing in Science and Engineering 10 I(3), 11–21 (2008) 15. Gruber, T.R.: Toward Principles for the Design of Ontologies Used for Knowledge Sharing. International Journal of Human-Computer Studies (1995) 16. http://www.geneontology.org/ 17. Clifford, B., et al.: Tracking Provenance in a Virtual Data Grid. Concurrency and Computation: Practice and Experience 20(5), 565–575 (2008) 18. Callahan, S.P., Freire, J., Santos, E., Scheidegger, C.E., Silva, C.T., Vo, H.T.: Vistrails, Visualization meets Data Management. In: Proceedings of ACM SIGMOD, Chicago, Illinois, June 26-29, pp. 745–747 (2006) 19. Barga, R.S., Digiampietri, L.A.: Automatic Capture and Efficient Storage of eScience Experiment Provenance. Concurrency and Computation: Practice and Experience 20(5), 419–429 (2008) 20. Bonner, A.J.: Hypothetical Datalog: Complexity and Expressibility. Theoretical Computer Science 76, 3–51 (1990) 21. Racunas, S.A., Shah, N.H., Albert, I., Fedoroff, N.V.: Hybrow: a prototype system for computer-aided hypothesis evaluation. Bioinformatics 20, suppl. 1, 257–264 (2004) 22. Racunas, S., Griffin, C., Shah, N.: A Finite Model Theory for Biological Hypotheses. In: Proc. of the 2004 IEEE Computational Sytems Bioinformatics Conferences (2004) 23. Kandel, E., Schwarts, J., Jessel, T.: Principles of NeuroScience, 4th edn. McGrawHill, New York (2000) 24. Hodgkin, A., Huxley, A.: A quantitative description of ion currents and its applications to conduction and excitation in nerve membranes. Journal of Physiology 117, 500–544 (1952) 25. Chauduri, S., Shim, K.: Query Optimization in the presence of Foreign Functions. In: Proc. of the 19th Very Large Database Conference, Dublin, Ireland, pp. 529–542 (1993) 26. Ullman, J.: Principles of Database and Knowledge-Base Systems, vol. 1. Computer Science Press, Rockville (1988) 27. http://www.w3.org/TR/xpath (last accessed 26/04/2008) 28. http://en.wikipedia.org/wiki/Scientific_hypothesis (last access 04/09/2009) 29. Elmasri, R., Navathe, S.: Fundamentals of Database Systems, 2nd edn. Benjamin/Cummings (1994) 30. http://lsids.sourceforge.net/ (last accessed 26/04/2008) 31. Grosof, B., Horrocks, I., Volz, R., Decker, S.: Description Logic Programs: Combining Logic Programs with Description Logic. In: Proc. WWW 2003, Budapest (May 2003)
Data Model for Scientific Models and Hypotheses
305
32. Christiansen, H., Andreasen, T.: A Practical Approach to Hypothetical Database Queries. In: Kifer, M., Voronkov, A., Freitag, B., Decker, H. (eds.) Dagstuhl Seminar 1997, DYNAMICS 1997, and ILPS-WS 1997. LNCS, vol. 1472, pp. 340–355. Springer, Heidelberg (1998) 33. Davidson, S., Freire, J.: Provenance and Scientific Workflows: Challenges and Opportunities. In: Proc. 2008 ACM SIGMOD Int’l. Conf. on Management of Data, Vancouver, CA, pp. 1345–1350 (2008)
Model-Based Engineering Applied to the Interpretation of the Human Genome Oscar Pastor, Ana M. Levin, Matilde Celma, Juan Carlos Casamayor, Aremy Virrueta, and Luis E. Eraso Research Center on Software Production Methods (ProS). Universidad Politecnica de Valencia. Camino Vera S/N. 46007, Valencia, Spain [email protected]
Abstract. In modern software engineering it is widely accepted that the use of Conceptual Modeling techniques provides an accurate description of the problem domain. Applying these techniques before developing their associated software representation (implementations) allows for the development of high quality software systems. The application of these ideas to new, challenging domains –as the one provided by the modern Genomics- is a fascinating task. In particular, this chapter shows how the complexity of human genome interpretation can be faced from a pure conceptual modeling perspective to describe and understand it more clearly and precisely. With that, we pretend to show that a conceptual schema of the human genome will allow us to better understand the functional and structural relations that exist between the genes and the DNA translation and transcription processes, intended to explain the protein synthesis. Genome, genes, alleles, genic mutations... all these concepts should be properly specified through the creation of the corresponding Conceptual Schema, and the result of these efforts is presented here. First, an initial conceptual schema is suggested. It includes a first version of the basic genomic notions intended to define those basic concepts that characterize the description of the Human Genome. A set of challenging concepts is detected: they refer to representations that require a more detailed specification. As the knowledge about the domain increases, the model evolution is properly introduced and justified, with the final intention of obtaining a stable, final version for the Conceptual Schema of the Human Genome. During this process, the more critical concepts are outlined, and the final decision adopted to model them adequately is discussed. Having such a Conceptual Schema enables the creation of a corresponding data base. This database could include the required contents needed to exploit bio-genomic information in the structured and precise way historically provided by the Database domains. That strategy is far from the current biological data source ontologies that are heterogeneous, imprecise and too often even inconsistent. Keywords: Modelling, Human Genome. Kaschek/Delcambre (Eds.): The Evolution of Conceptual Modeling, LNCS 6520, pp. 306–330, 2011. c Springer-Verlag Berlin Heidelberg 2011
Model-Based Engineering Applied to the Interpretation
1
307
Introduction
It is widely accepted that in modern Software Engineering the use of conceptual modeling techniques applied to software development creates higher quality systems [1]. This is due to the fact that description and understanding of the problem domain is done before the implementation occurs, thus the work is carried out at higher abstraction levels. Conceptual modeling techniques have been applied successfully to many different Information Systems domains from Business to medical applications. A well known example of that successful use is the one related with Organizational Systems [2]. The main idea is to provide a suitable conceptual framework, including possible, simple, clear and unambiguous definitions of the most fundamental concepts in the information system field and a suitable terminology for them. Notions as information and communication, organization and information system, etc.. are properly defined. The Information Systems (IS) experience –meaning where that all the efforts done by the IS in order to design, develop and manage Organizational Systems- has provided a relevant set of results, and it is still a domain where a huge amount of academic and industrial work is in progress. But what could be considered “next” in Conceptual Modeling? When exploring the world looking for new, challenging and suitable domains for the application of Conceptual Modeling techniques, one specific domain pops up that is surprisingly not included: Genomics, and in particular Human Genome proper interpretation. A main issue of this chapter is to show how well IS concepts and practices can work on this Human Genome domain. It is widely accepted that the existence of a Conceptual Model improves the efficiency of Information Systems since it helps to manage modularity and evolution [1,3]. Traditionally, Software Engineering applied to Genomics field has been more oriented to the design of powerful and efficient search algorithms, working in the solution space rather than in the problem space. The complexity of the domain added to the explainable lack of knowledge that software engineers experience in this field may explain the absence of Conceptual Modeling contributions in the Bioinformatics domain. We mean by that, that too often the Bioinformatics work is based on heterogeneous repositories of information that contain inconsistencies, redundancies, partial information, different representation of the same data. This type of problems are well-known and well-reported in the IS domain, and they have been intensively explored, especially in the database context, where a relevant set of sound solutions has been provided during years. With more than 1300 different databases currently reported to be used and exploited, it is easy to understand how hard searching relevant data to undertake any genomic-oriented activity is. Or how hard data analysis it is to perform successfully impossible. Data integration is essential to answer questions in a simple and effective way. To solve these problems, we are convinced that a Conceptual Schema of the Human Genome is strongly required, for facilitating global research among the various distinct research groups, by fixing a conceptual gamut from which researchers can draw, in order to ensure a ’standard dictionary
308
O. Pastor et al.
of concepts’. This can be achieved by fixing concepts, their properties, behavior and relations, and by disambiguating existing biological concepts. These are the reasons why we have created a multidisciplinary group in which software engineers with a strong IS background in Conceptual Modeling and biologists experts in Genomics worked close together in order to look for novel and effective solutions that illuminate the understanding of Human Genome. The result of this collaborative work has been the first ideas around a Conceptual Schema of the Human Genome [4]. Unifying the knowledge needed to understand the Human Genome is a difficult task for software engineers. The conceptualization of the relevant concepts of these domains implies that software engineers change the way things are done. A big effort is required to fix concepts definitions in a context where even the knowledge associated with basic concepts is continuously evolving and changing. As we will see, even fundamental notions as what defines a gene are discussed in depth when we want to have an exact definition. Often, the functional result of some partially unknown structure is just what characterizes that given structure. The subsequent conceptual challenge here is to model concepts taking into account the behavior of the domain elements. In this context, these elements are identified by their functional participation in the genome and not merely by their attributes and the relations between them. All the ideas mentioned above imply a new way of conceptualize domains, where model evolution is a need, and where we could speak about a “fuzzy conceptual modeling” strategy, because many parts of the domain that are being modeled, are being understood step by step and day by day. By showing how this conceptual discussion evolves before concluding how to model any particular genomic concept, we try to provide new insights to answer the question of how could the human genome be best explained by introducing a conceptual modeling perspective. Our intention is to show how the conceptual discussion followed to understand a particular genomic concept facilitated a complete understanding of the high-level ideas that are behind the corresponding concept as well as the details required to manipulate its relevant information. The Conceptual Schema –whose construction process is reported in this chapter- is the essential component intended to provide the required holistic context that enables reaching the goal of achieving a complete understanding of the human genome. According to these ideas, in this chapter we describe the Conceptual Schema evolution from its starting point to the present model, including the different versions needed to describe the domain precisely. We believe that it is important to present these successive versions to help understand the complexity of finding an agreement between biologists and conceptual modelers when a decision is to be taken for defining a genomic concept with the required precision. Those concepts whose interpretation is more problematic are emphasized. In order to achieve our objectives, the chapter starts with an analysis of the most relevant related work; in Section 3 an initial conceptual schema of the human genome is introduced to launch the discussion of how to build such a conceptual schema. After that, in Section 4, a set of iterations is presented, intended to show and understand how
Model-Based Engineering Applied to the Interpretation
309
the evolution of the most relevant, representative conceptual challenges guide the corresponding schema evolution process. The result of this discussion is the introduction of a current version of the conceptual schema of the human genome, which is ready to be properly exploited in an applied bioinformatic domain. One important consequence of the proposed Model-Driven approach is that the database that exactly corresponds to the Conceptual Schema can be built. This database can then be loaded with relevant data for the set of selected genes, following a bottom-up approach where the genomic data chaos is overcome by manipulating only those data that are considered relevant for the selected genes. In this way, the database will really work as a kind or “structured cupboard” where every piece of information is going to be properly disposed. Concluding remarks and the list of used references close the chapter.
2
Related Work
It is interesting to realize that we have not found too many relevant references where the Human Genome –or any other Genome- is faced from a IS Conceptual Modeling perspective. Even if the idea of applying conceptual modeling techniques on molecular biology has been tackled by some informatics and biologist in the last years, the approach has a lot of space to explore and discuss. The more relevant contributions in this field are those made by Paton et al. [5]. This work is an important reference for the labor developed in this chapter, because it can be considered as a starting point schema. In this proposal a collection of data models for genomic data is presented. These models describe elements involved in transcriptional and translational processes as well as the variant effects generated by them. As we will see through the chapter, the work presented here extends these ideas, and proposes a complete Conceptual Schema intended to be seen as a central, conceptual repository of Genomic information, which was not the objective in the referenced work. Some other attempts to model more viable genomes have been reported. An interesting experience is provided by the e-Fungi initiative [14,15], where a systematic comparative analysis of fungal genomes is supported. The e-Fungi database integrates a variety of data for more that 30 fungal genomes and it provides fungal biologists with a powerful resource for comparative studies of a large range of fungal genomes. This work is developed in a different domain –the fungi genome instead of the more complex human genome- but it shows a clear path of results exploitation that could be perfectly projected to our Human Genome modeling effort. There are some other interesting examples of conceptual modeling techniques used in molecular biology, although with a more specific view on a particular part, as the work of Ram [6] to model the protein. This proposal, even if it is a little bit out of our scope, includes a part related to genome which can be very useful for the development of a conceptual schema of a complete genome. In any case, our attempt goes beyond the particular view of modeling proteins, because we want to provide a whole conceptual schema for the Human Genome.
310
O. Pastor et al.
Additionally, a relevant set of bioinformatic implementations are based in major or minor degree on conceptual modeling techniques, and they have been accepted favorably. One example of that is the work of Kevin Garwood et al. [7] which is a model-driven approach for the partial generation of user interfaces for searching and browsing bioinformatics data repositories. This work demonstrates that conceptual schemas can be used to produce many applications in the future. Again, when compared with our work the use of conceptual modeling techniques focuses on some very specific part of a software production process –the user interface design- in the bioinformatics domain, while we want to provide a whole, unified conceptual view in the tradition of the IS modeling experience. These works are a few of the existing examples about the use of conceptual modeling in bioinformatics applications. They can be used to prove that conceptual modeling is an effective approach to help improve biologic research. It is our belief that the work described in this chapter is an important contribution to the global understanding of the human genome, because only having a Conceptual Schema to characterize it, will storing the right contents, manage them efficiently, and understand the precise relationships existing between phenotype (external manifestation of human properties) and genotype (their corresponding genomic code).
3
An Initial Conceptual Schema for the Human Genome
A conceptual schema of the human genome was proposed by Normal W. Paton in February 2000 [5] (see Fig. 1). In this model, an initial conceptual view of the human genome is described as a set of chromosomes divided in fragments that belong to a certain gene, which is an intragenic fragmentation. Paton [5] proposes to classify a chromosomic fragment as either a Transcribed Region fragment or a Non-Transcribed Region fragment. In this model, a Transcribed Region fragment represents the transcribed sequence of the gene excluding elements that are also involved in the transcription process, like promoter or terminator. A Non-Transcribed Region fragment includes the regulatory sequences and the chromosomal elements that are part of the chromosome but are not involved in the transcription process. Additionally, in this model a set of primary transcripts can be generated from a Transcribed Region fragment. We elaborated a new conceptual schema for the Human Genome, derived from Paton’s model. This model, presented in [4], provides a basis to fix the main features that are considered relevant to characterize the human genome basic components. The new conceptual schema introduces changes in the description of a genome. The main difference is the classification of chromosome segments. These chromosome segments are classified as Genic or Non–Genic, where a genic segment is seen as the composition of one promoter, one transcribed sequence, one terminator and many enhancer sequences in contrast with Paton‘s model where only the transcribed region in the gene is considered a genic segment. These genic components share a functional relation derived from the protein synthesis process, as
Model-Based Engineering Applied to the Interpretation
Fig. 1. Paton’s Model
311
312
O. Pastor et al.
the result of transcriptional and translational processes. Any other chromosomal sequence is considered as a non-genic segment (see Fig. 2). This can be seen as an example of how the need of conceptualizing with precision really helps to crystalize and form ideas while helping to fix important concepts that are not really so clear when we just look at the current biological data banks. In this case, considering that chromosome segments must be characterized as either genic or not genic, the gene structure is defined more clearly, and the information associated with the proposed structure will be adequately stored in the corresponding Genome database.
4
Conceptual Schema Evolution
In this section the evolution “suffered” by the model will be presented. The most relevant changes and conceptual challenges will be discussed to understand the learning process from the initial model to the current stable conceptual schema of the Human Genome. This is an interesting issue when applying Conceptual Modeling in this domain, because we have experimented how complicated fixing precise definitions can be in a context where new knowledge is discovered day after day. When modeling conventional organizational systems, its main components, their relationships and the functional processes are mainly known, and this makes the conceptual schema construction process viable and feasible. When facing the problem of modeling the human genome, we have seen how concepts and their representation can vary as the bio-genomic knowledge increases. In this section we will comment and discuss some relevant conceptual challenges, meaning by that the process followed to decide how to conceptualize some widely-known and used concepts of the human genome domain. As a very basic notion in this context, we will start with the notion of gene. The precise definition of gene is a very interesting topic that biologists still debate. The term has evolved, since it was first proposed by W. Johansen in 1909 based on the concept developed by Gregor Mendel, to fit the different paradigms and theories that have occurred in the history of Genetics (For a complete review see [10]). The classical view of a gene -a linear piece of DNA inherited from the parents who coded for a single protein- has considerably changed reflecting the complexity of the term. In the updated definition proposed by Gerstein in 2007 [10], a gene is “a union of genomic sequences encoding a set of potentially overlapping functional products” and this evolution of the concept carries important implications inside. Trying to explain such a fuzzy term to “conceptual modellers” is a very challenging and difficult task. During the conceptual modeling of the human genome, our group had to deal with many misunderstandings produced by the dramatically different way of handling concepts of biologists and software engineers. The fact that a gene sequence varies in different databases, the idea that two genes may share the same locus or that post-transcriptional events generate multiple products from one genetic locus are some of the concepts that puzzled software engineers. This reflects the changes that the concept had during the evolution
Model-Based Engineering Applied to the Interpretation
Fig. 2. An initial Conceptual Schema for the Human Genome
313
314
O. Pastor et al.
of the model. In early versions the Gene class was associated to GenicSegment class, which was a big DNA segment composed by one promoter, one transcribed sequence and one terminator and regulated by many enhancer sequences. In the following versions GenicSegment class became smaller, generalizing any genic segment with some functionality (promoter, transcribed sequence, etc...). Then the Gene class was associated to TranscriptionUnit class, which was a really rigid structure that combined the genic segments involved in the transcription process. In later versions, the concept of TranscriptionUnit class became broader and multiple compositions appeared for a single transcription unit, in agreement with Gerstein’s updated definition of Gene. This way we can model a concept whose definition is still discussed and assume that some of its characteristics may be ambiguous, unknown or even wrong. The TranscriptionUnit class, as it appears in the third version of the model (see Fig. 3), evolved in such a way that allows for the inexistence of promoter and terminator sequences, represented by cardinality constraints (0,1) in Promoter and Terminator classes. This change is relevant, considering that these transcription unit elements exist but are frequently not indicated in the data repositories. However, at least a transcribed sequence must be present, represented by cardinality constraint (1..*) in TranscribedSequence class. Following with the analysis of the domain, we found a new important concept that implies a change in the conceptual schema. An Enhancer sequence can be bound to certain proteins to enhance the transcription levels of the corresponding genes. This idea implies that enhancers are included in a group of regulator sequences that regulate the DNA transcription process, therefore the Enhancer class is replaced by RegulatorSequence class. Furthermore, if we take into account that a regulator sequence must be related with one or many transcription units then the RegulatorSequence class must be associated with TranscriptionUnit class. Besides, a regulator sequence is regulated by one or many regulator sequences. At this point, the model only included the concept of gene and its structure from a transcriptional point of view. Once the Transcription and the Translation were studied in depth, a new conceptual schema was defined to include concepts and products related to them (see Fig. 4). For a more detailed description of these processes see [11]. The protein synthesis process starts with the transcription: a DNA sequence is transcribed to a complementary RNA sequence. This RNA molecule is known as primary transcript. In the model, this new concept is represented by PrimaryTranscript class, which is associated to a set of instances of a new class: ElementTranscript. The specialization of this class in Exon and Intron classes describes the integration of exons and introns in the molecular structure of the primary transcript and represents the different types of partition fragments. A reflexive association “Prior,Next” for ElementTranscript class is added to indicate the order of exons and introns in a primary transcript sequence. The splicing process consists in the combination of certain exons of the primary transcript resulting in a mature mRNA. The outcome of the splicing
Model-Based Engineering Applied to the Interpretation
Fig. 3. Second version of the conceptual schema
315
316
O. Pastor et al.
process application to a primary transcript is represented by the SplicedTranscript class that is specialized in the classes mRNA and other RNA types. The aggregation relation “splice” between SplicedTranscript class and Exon class allows to identify the exons that are the result of the splicing process. The mRNA class represents the messenger RNA, a molecule that contains the information needed to synthesize a protein. The RNA sequence determines the amino acid order in the protein. The other classes include RNAs that are not necessarily translated into amino acid sequences: snRNA (small nuclear RNA, that participates in important nuclear processes), tRNA (transfer RNA, an important molecule in the transcription process) and rRNA (ribosomal RNA, which is part of the ribosome). The next step in protein synthesis is the migration of mature mRNA from the nucleus to the cytoplasm. There it associates to the ribosome and the translation process starts. The translation is the production of proteins by decoding the mRNA produced in the transcription. In this process, the mRNA molecule acts as a template for the synthesis of the corresponding amino acid chain. The decodification rules are specified by the Genetic Code. Notably, the mRNA molecule is not translated completely; the ORF (Open Reading Frame) is the part of the mRNA sequence used in the translation process. The PrimaryPolypeptide class is created to describe the protein primary structure: the amino acid chain obtained after the translation of an ORF. This amino acid chains suffers some chemical transformations and the final result is a functional protein which is represented in our model as the Protein class. The association between Protein class and PrimaryPolipeptide class is included to conceptualize that a primary polypeptide originates a protein and a protein is synthesized as a primary polypeptide. Besides this new view, some constraints were defined in the TranscriptionUnit class. The Promoter class has an integrity constraint which describes the order of the elements included in the same transcription unit. This constraint specifies that the final position of the promoter sequence is always the initial position of the transcribed sequence and, at the same time, the final position of the transcribed sequence is always the initial position of the terminator sequence. All the added classes and relations between classes listed below provide the bases for a more complete genomic model that, due to the size of the schema should be organized in two different views: the Genome view that includes the concepts from genome to transcription unit and the Transcription and Translation view that enclose all the concepts from the primary transcript to the fully functional protein. As a result of the evolution of the discussions, some more important changes arose. In previous versions of the schema, a gene was modeled taking into account its structure, which was considered fix. But in the new model, depicted in Fig. 5, the conceptualization of the gene is replaced by the idea of modeling a generic gene. At this point, the Allele class was introduced. An allele is each one of the alternative forms a gene may have in nature. These alternative forms differ by their sequences and any of them may produce changes on gene function. One
Model-Based Engineering Applied to the Interpretation
317
of these forms is considered the wild type and is distinguished from all other variants allele for being the most abundant in nature. The generic genic structure is conserved independently of the alternative forms of a gene. A genic segment will be associated with generic genic segments. This idea allows conceptualizing of the relation between the alternative forms of a gene and the gene structure. To explain the variation between alleles, we introduced a classification of the mutations occurred in any of these allele variants. The classification is made following two different criteria: the precision of the variation description and the phenotype that it produces. The first classification of the variations is divided in two categories. The precise variations are those that are described accurately in the conventional data bases. The imprecise variations are those that are described by text in the data sources, therefore it is not possible to automatically detect the exact position of the variation. The second classification of the allelic variants is divided in four types: 1) Genic Mutation, which names the variation that produces a pathologic effect. 2) Chromosomic Mutations, which describes the variation that affects more than one gene. 3) Natural Polymorphism, that characterizes a neutral variation, and 4) Unknown Consequence Changes, referring to those that report a variation with undiscovered consequences. To represent in the model all the newly acquired concepts, a set of new classes were introduced. The first of them is the concept of reference sample. A Reference class is created and the aggregation relation “belongs” is included to describe that a genome has a reference.The idea of modeling a generic gene removes the specialization of GenicSegment class in Promoter, TranscribedSequence, Terminator and RegulatorSequence classes. This change is made because a genic segment is not generic. The GenericGenicSegment class is added to the model to represent a generic genic segment. A genic segment can be associated to a generic genic segment, what is represented by the association relation “corresponds” between GenicSegment class and GenericGenicSegment class. This relation describes the concept that a genic segment always corresponds to a generic genic segment and a generic genic segment can be associated (or not) to one or many genic segments. Once again, we want to emphasize how the application of a CM-based perspective allows for dealing with the evolving knowledge that is so characteristic of this domain. With the discussion being focused on the conceptual model level, it is possible to include the novel interpretations in the conceptual model in a evolutionary way, always with a sound control on the semantics of the model. For instance, the distinction that was introduced between Genic Segment and Generic Genic Segment was very significant. Before introducing the Generic Genic Segment, the conceptual schema was not distinguishing between particular genome instances and a generic pattern of reference. While it was understood that individual samples should provide individual genomes, it became clear that the instance level had to be distinguished from the generic reference level.
318
O. Pastor et al.
Fig. 4. Third version of conceptual schema
Model-Based Engineering Applied to the Interpretation
Fig. 5. Fourth version of conceptual schema
319
320
O. Pastor et al.
This generic level denotes a common template valid for any genome structure, while the instance level represents individual occurrences of this generic, common structure. The introduction of the Generic Genic Segment notion related to but different from the former Genic Segment provides the required connection between the two perspectives, leading to a Conceptual Schema that properly captures this important information. Again, we see here a concrete proof of the benefits obtained by using a CM-based approach to fix understand the genomic domain. This allows also to exploit it correctly through the associated database, where the relevant information is accordingly structured. Additionally, the Allele class is added to the model to store the alternative forms of a gene found on the data repositories. Due to this, the gene is not associated anymore to the transcription unit but to alleles and the association relation between TrascriptionUnit class and Gene class is removed. The aggregation relation “has” between Gene class and Allele class is added to denote that a gene will always have one or many alleles associated. The aggregation relation “has” between TranscriptionUnit class and Allele class is included to specify that an allele is always associated to one or many transcription units. In this context, another aggregation relation “has” between Allele class and GenericGenicSegment class is included in this model to describe that an allele has one or many associated generic genic segments. Allele class is specialized in WildType class and AllelicVariant class. The WildType class represents the natural allele sequences and the AllelicVariant class represents the variant sequences that are not the wild type. The allelic variant groups are specified in the model by the specialization of AllelicVariant class in two groups of classes. The first group is specialized in Imprecise class and Precise class. The second group is specialized in GenicMutant, ChromosomicMutant, NeutralPolimorphism, and UnknownConsequenceChange.
5
The Current Version of Conceptual Schema
After the introduction of the main aspects related to the schema evolution, the current version of the Conceptual Schema of the Human Genome is presented in this section. It can be seen as the intermediate stable state that contains the current relevant information. At the same time the schema is ready for both evolving with the incorporation (or modification) of new conceptual issues, and being properly exploited through loading the adequate biogenomic contents using its corresponding database. The role of the database is vital in terms of driving an efficient and effective storage processing for genomic data. According to the well-known Information Systems principles related to database design, the database that corresponds to the conceptual schema has been built. Having clear the definition of the data structure, the next problem is naturally to load it with the adequate contents. Analyzing the current, diverse and often unstructured biological data repositories, the relevant information for particular genes can be detected, isolated and moved to the database. This gene-by-gene strategy opens the door to having a Genomic database containing just the selected,
Model-Based Engineering Applied to the Interpretation
321
correct and required information, instead of the tons of any type of information that we currently find in biological data sources that are most used. A set of experiments are being designed and implemented in the domain of the genomic clinical diagnosis, where the elaboration of a genomic report can be done much more efficiently and accurately using this Model-Driven Engineering approach. For the sake of understandability, the current version of the Conceptual Schema is divided in three main views (Gene-Mutation, Transcription and Genome). Fig. 6 is the general view of the model and is intended to show the interconnections between each one of the different views that will be described in detail in the following subsections. 5.1
Gene-Mutation View
In the Gene-Mutation view presented in Fig. 7, all the knowledge about genes, their structure and their allelic variants based on the different public databases is modeled. The principal entities in this view are the Gene class and Allele class. The Gene class models the concept of generic gene independently of the samples registered in the databases. In this class, we find attributes as Id_Hugo, a numeric code that represents the universal code for the gene according to HGNC (Human Genome Nomenclature Committee) [12]. The name attribute stores the common name of the gene. Chromosome attribute represents the number of the chromosome where the gene is located and the locus attribute represents a gene location into the chromosome according to NCBI information [13]. Finally, a summary of the gene is extracted from NCBI database, and it is stored in the summary attribute. Another main schema class is Allele class, which represents the instances of a generic gene and is the most important class of the model since all the information depends on it. This class contains all the relevant information about alleles such as allele databases, references, variants, generic genes and mRNA and DNA sequences. In the case of AllelicVariant class, all the information about the variation is represented as well as some identification and descriptive attributes. The ord_num attribute is the internal identification number of the allele in our database. Data_bank_source attribute is the external code of the database source. Another important class is AllelicRegion; this class represents the chromosomal region where the allele is, and contains an important attribute: sequence, which will store the complete DNA sequence of the allele. The attributes start_position and end_position will describe the beginning and the end of the allele in reference to the chromosome. The relation between Gene class and Allele class helps to identify any allele of a gene in the information system. The cardinality of this relation (1..1:0..N) allows to represent a gene with no allelic information. To represent external references for gene and allele information, we created certain classes in the conceptual schema. The GeneExternalIdentification class represents the identification of a gene in different public databases. The OthersAlleleDataBank class, is the same representation but for alleles.
322
O. Pastor et al.
As far as we know, an allele might be considered reference or variant. For this reason, the AllelicVariant class and ReferenceType class are specialized classes from Allele class. The ReferenceType class represents the alleles used as references in the existing databases, and the AllelicVariant class represents allelic variations of a reference allele in a database. There is a related association between ReferenceType class and AllelicVariant class, this association allows us to represent a relation between a reference allele and its variations. It is important to note that the WildType class has been replaced by the ReferenceType class, a more suitable term for bioinformatic purposes since it does not have genetical meaning. Once we propose an allelic variant, we determine a specialization hierarchy from the AllelicVariant class. This lead to the classification of allelic variants in two specializations. In G1, four different situations are considered: the allelic variation is specialized in GenicMutant when this variation affects a gene and it is specialized in ChromosomicMutant when the variation affects parts of the chromosome. The ChromosomicMutation class describes the chromosomic variation that affects one or many genes in the same chromosome. NeutralPolimorphisme class is a variation that does not affect the phenotype. And finally the UnknownConsequence class is used to represent the case when the variation consequences are not known yet. In G2, we classified the variation in two types: imprecise and precise. When we do not know details about the variation, we classify it as imprecise. There is a description attribute in the Imprecise class that stores information about the variation in text format. Opposite to that, when a variation is precise we can represent the position where the variation occurred. In this view we also modeled the allele segmentation and structure. We considered that an allele segment has a significant and indivisible sequence of DNA. For this reason we implemented the Segment class, with attributes such as ord_num, (that identifies a segment between all the allele segments), start_position (initial position of the segment in a chromosome) or end_position (the end of the segment). The sequence attribute stores the sequence between start_position and end_position. This class has four specialized classes classified by their function in the transcription process. The first of these specialized classes is Promoter, which represents the region of the DNA sequence that facilitates the transcription of the transcription unit; TranscribedSequence is the sequence of DNA transcribed by the ARN polymerase; Terminator is a fragment of DNA that marks the end of the transcription process. Finally, the RegulatorSequence class is a segment with regulatory functions that regulates one or many transcription units. After the definition of these specialized classes, a new class was created to model the concept of transcription unit. This class – TranscriptionUnit – has one attribute ord_num, which is used for internal identification between all the transcription units of the same allele. The relations between this class and the specialized classes have different means. The relation between TranscriptionUnit class and Promoter class means that a Transcription Unit has a unique promoter; since this one may be unknown, this relation has cardinality 1..1:0..1.
Model-Based Engineering Applied to the Interpretation
Fig. 6. Gene-Mutation View
323
324
O. Pastor et al.
The relation between TranscriptionUnit class and TranscribedSequence class has cardinality 1..1:1..*. This means that many transcript sequences may exist in the same transcription unit starting all at the same position. The relation between TranscriptionUnit class and Terminator class means that a transcription unit may have more than one terminator segment that can be unknown. The relation between TranscriptionUnit class and RegulatorSequence class means that a transcription unit may have many regulator segments, shared by different transcription units belonging to several genes in the most general case. Regarding the precision of the DNA sequences in the schema, it is interesting to remark that initially the concept of allelic region was introduced to alleviate the inaccuracy of the DNA sequences that represent copies of the same gene in different databases. On the other hand, when the same databases were browsed to look for data regarding the fragmentation in different components of a gene (promoter, transcribed sequence and terminator) those data were precise and detailed. The AlellicRegion class indicates the approximate region where a gene is contained, whereas the sequence attribute of the Segment class gives clear and precise data information of the beginning and the end of the transcribed region. 5.2
Transcription View
Another important view of our model is the Transcription view that is showed in Fig. 8. In this view the basic components related to the protein synthesis process are modeled. This is a very important part of the schema, as it enables linking the genotype-oriented features modeled above with the phenotype-oriented manifestations that can be perceived externally. Consequently, it requires to explain the relationships between genes and alleles with their corresponding external expression in terms of behavior. An interesting issue to be discussed in this context is how this transcription view relates to the functional and behavioral perspective of the conceptual model that we are building. The view that we are considering through the chapter is basically structural and data-oriented, meaning by it that what we are fixing is the class arquitecture of the model together with the relationships between classes. This is of course a needed first step, and it is strictly required to set the basis of the conceptual modeling approach defended in this work. While it is completely true that the transcription and translation processes have a complex and relevant behavioral perspective that should be properly described, it is also true that in the context of the description followed in this chapter, we are firstly interested in the consequences of those processes on the relevant classes and relationships that conform the basic data architecture of the system. This is why we leave behavioral and interaction modeling aspects for future versions of the model, once this structural part will be properly fully-specified. Following in that direction, the first class found in this view is PrimaryTranscript class, which represents the RNA copy from the DNA of the transcribed sequence (TranscribedSequence class). This class has an attribute sequence, which is a derived attribute from the Segment class. We implemented the PrimaryTranscriptPath class, in order to model the different splicing factor-driven partitions
Model-Based Engineering Applied to the Interpretation
Fig. 7. Transcription View
325
326
O. Pastor et al.
of the PrimaryTranscript. This class has an attribute ord_num used to identify a partition from all the partitions of a primary transcript. The ElementTranscript class has an ord_num attribute which helps us to identify a specific fragment from all of the partition fragments. The specialized Exon and Intron classes represent the types of the partition fragments. The Produces class represents different exon combinations from a primary transcript that produces a spliced transcript. The result of these combinations is the specialized classes mRNA and othersRNA types. The SplicedTranscript class has the resultant sequence of the exon combinations in the sequence attribute and ord_num attribute for its identification between all the allele spliced transcripts. The mRNA contains nucleotide sequences that could potentially encode a protein; this is what we know as ORF (Open Reading Frame). In our model, the ORF class has an id attribute as internal identification and a sequence attribute that stores codifying sequence. Then we have the PrimaryPolypeptide class, which describes the protein primary structure: the amino acid chain obtained after the translation of an ORF. This amino acid chain suffers some chemical transformations and the final result is a functional protein, which is represented in our model as Protein class. A protein could be formed by one or more primary polypeptides. In the Protein class we find a name attribute which represents the name of the resulting protein and its amino acid sequence in the sequence attribute. We want to close this subsection and focus on the next step to be completed: the phenotype connection. Having characterized where proteins come from, the conceptual schema is ready to include phenotypical information. Relating external, perceived features to the protein or set of proteins involved in their expression, a challenging research for linking genotype and phenotype is really to be analyzed, what is going to be for sure a very relevant direction of research in the next years. Following what we have called in this chapter a “model-driven engineering” perspective, this is going to be possible. As conceptual models become the key software artifact for dealing with the Human Genome, model transformations could be defined both at the upper direction –connecting requirements with the conceptual schema that include how these requirements are expressedand in the lower direction –linking the kwowledge represented in the Conceptual Schema with individual samples to infer potential problem or to identify particular human being characteristiscs. 5.3
Genome View
In Fig. 8, a conceptual schema of a complete individual genome is presented. This view is especially interesting for future applications, since massive parallel sequencing technologies will allow the complete sequencing of individual genomes at a very low price in the near future [8,9]. When this information becomes available we will be able to store it in our database, and it will be stored according to the precise conceptual structure fixed by the Conceptual Schema. First we have ResearchCentre class which represents the labs or research centres where a specific human genome was sequenced. We have name and description attributes
Model-Based Engineering Applied to the Interpretation
Fig. 8. Genome View
327
328
O. Pastor et al.
to record the genome source. Then Genome and Chromosome classes represent a complete genome sequenced by a specific research centre. A genome is a set of chromosomes, this is represented by cardinality 1..* relation between Genome class and Chromosomes class. The number attribute will determine a specific chromosome on a specific genome. The “couple” relation on the Chromosome class represents the concept of homologue pair, which means that every human cell will carry two equivalent chromosomes (one from the father and one from the mother) with different alleles for each gene. The ChromosomeSegment class represents all the segments that form a complete chromosome. This class has as sequence attribute to store the corresponding DNA sequence delimited by start_position and end_position attributes. Another important concept that we have represented in our model is the structure of a chromosome. We identified that a chromosome has two main types of segments: coding-related segments (represented by the GenicSegment class) and not coding-related segments (represented by the NonGenicSegment class); these two classes are specialized from the NonGenicSegment class: IntergenicRegion class, which represents the space between genes and ChromosomalElement class. The last one has three specialized classes that describe other elements of the chromosomes (Centromere class, Telomere class, and ORI class) whose function is to keep the chromosome functional and have little to do with protein production. It is interesting to remark that, at the beginning of the modeling process we expected to have a unique, complete, reference description for each gene. This description should include DNA and RNA sequences, alleles, segmentations, mutations and every detail that could be potentially included in the schema. This is the main reason of the GenericGenicSegment class addition, which tries to manifest the uniqueness of the gene description. Later, conscious of the fact that different databases gave different definitions of the same gene, we chose to drop the idea of a “generic gene” to substitute it by the possibility to include different descriptions of the same gene offered by different databanks. As a result of this OthersAlelleDatabank class was included, as well as the databank attribute in the Allele class. This is another clear example of concept evolution driven by the learning process or what we previously called a Fuzzy Conceptual Modeling Strategy.
6
Conclusions
The way in which the knowledge of the human genome domain is captured and acquired by the IS experts is one of the richest parts of this chapter. When biologists and software engineers interact, the underlying vocabularies and ontologies are too easily too far. To make them become as closer as possible is being a continuous challenge for the group of people working on this conceptual modeling of the human genome effort. In this chapter, we have tried to show how this evolution occurred, and how so basic notions as those of gene, genic segment, alleles, mutations or transcription, were refined step by step. Many times a definition accepted by everybody was changed after some time just because some vague
Model-Based Engineering Applied to the Interpretation
329
detail was missed out in the first rounds. As at the end, a concrete proposal of a conceptual schema is presented as a result of these discussions and interactions, we argue to provide with all the reported experience very valid material for any other interdisciplinary group interested in facing the problem of understanding the human genome. Another contribution of this chapter is directly related to the historical value recognized in conceptual modeling. The modeling benefits that biological systems research should get from IS theory and practice include at least: – working at a higher abstraction level to specify systems easily and more precisely, – enable reasoning to reason about a biological system prior to its construction to foresee consequences in advance, and to facilitate simulation and validation, – the real possibility of automating the subsequent systems development. It is our belief that real practice in the genomic domain is also strongly requiring the IS experts participation. It makes sense from our IS perspective to talk about the chaos of genome data, meaning that there are currently tons of data from the genome publicly available, each one with its corresponding database that is defined with an specific schema, data format, identifications, etc.. and where the integration of the different sources is a very difficult task that sometimes is just not possible. The proper management of this information ecosystem is an urgent need, because the available scientific literature and experimental data, the summaries of knowledge of gene products, the information about diseases and compounds, the informal scientific discourse and commentaries in a variety of forums, and many more genetic-related data is dramatically increasing day after day. To understand the relevant knowledge that is behind that huge amount of information, conceptual models should become the key software artifact. With our proposal of a Conceptual Schema for the Human Genome, including the way in which the model construction has evolved to reach the current situation, we believe to have demonstrated that such a task is feasible and fully useful. This is why we have chosen Model Driven-based Engineering applied to the Understanding of the Human Genome as the chapter title: if models become the key artifact used to interpret and exploit the knowledge that is around the human genome domain, its understanding and subsequent number of successful practical applications will clearly increase. Related with usefulness, we are now in the process of providing the adequate contents to the database that corresponds to the specified conceptual schema. We want to achieve a very simple but ambitious objective: if with Conceptual Models targeted at digital elements we can improve Information Systems Development, with Conceptual Models targeted at life we can directly improve our living. The Conceptual Schema for the Human Genome is our modest contribution in that direction.
330
O. Pastor et al.
References 1. Olivé, A.: Conceptual Modelling of Information Systems. Springer, Heidelberg (2007) 2. Falkenberg, E., Hesse, W., Lindgreken, W., Nilsson, E., Han, J., Rolland, C., Stamper, R., Van Assche, F., Verrijn-Stuart, A., Voss, K.: A Framework Of Information System Concepts. In: IFIP (1998) 3. Pastor, O., Molina, J.C.: Model-Driven Architecture in Practice. Springer, Heidelberg (2007) 4. Pastor, Ó.: Conceptual Modeling Meets the Human Genome. In: Li, Q., Spaccapietra, S., Yu, E., Olivé, A. (eds.) ER 2008. LNCS, vol. 5231, pp. 1–11. Springer, Heidelberg (2008) 5. Paton, W.N., Khan, S., Hayes, A., Moussouni, F., Brass, A., Eilbeck, K., Globe, C., Hubbard, S., Oliver, S.: Conceptual modeling of genomic information. Bioinformatics 16(6), 548–557 (2000) 6. Ram, S.: Toward Semantic Interoperability of Heterogeneous Biological Data Sources. In: Pastor, Ó., Falcão e Cunha, J. (eds.) CAiSE 2005. LNCS, vol. 3520, pp. 32–32. Springer, Heidelberg (2005) 7. Garwood, K., Garwood, C., Hedeler, C., Griffiths, T., Swainston, N., Oliver, S., Paton, W.: Model-driven user interface for bioinformatics data resources: regenerating the wheel as an alternative to reinventing it. Bioinformatics 7(532), 1–14 (2006) 8. Bornberg-Bauer, E., Paton, N.: Conceptual data modelling for bioinformatics. Briefings in Bioinformatics 3(2), 166–180 (2002) 9. Wheeler, D.A., Srinivasan, M., Egholm, M., Shen, Y., Chen, L., McGuire, A., He, W., Chen, Y., Makhijani, V., Roth, G.T., Gomes, X., Tartaro, K., Niazi, F., Turcotte, C.L., Irzyk, G.P., Lupski, J.R., Chinault, C., Song, X., Liu, Y., Yuan, Y., Nazareth, L., Qin, X., Muzny, D.M., Margulies, M., Weinstock, G.M., Gibbs, R.A., Rothberg, J.M.: The complete genome of an individual by massively parallel DNA sequencing. Nature 452, 872–877 (2008) 10. Gerstein, M.B., Bruce, C., Rozowsky, J., Zheng, D., Du, J., Korbel, J., Emanuelsson, O., Zhang, Z., Weissman, S., Snyder, M.: What is a gene, post-ENCODE? History and updated definition. Genome Res. 17, 669–681 (2007) 11. Alberts, B., Johnson, A., Lewis, J., Raff, M., Roberts, K., Walter, P.: Molecular Biology of the Cell. Garland Science, New York (2002), http://www.ncbi.nlm.nih.gov/books/bv.fcgi?rid=mboc4 12. Gene Nomenclature Committee, http://www.genenames.org 13. National Center for Biotechnology Information, http://www.ncbi.nlm.nih.gov 14. Hedeler, C., Wong, H.M., Cornell, M.J., Alam, I., Soanes, D., Rattray, M., Hubbrad, S.J., Talbot, N.J., Oliver, S.G., Paton, N.: e-Fungi: a data resource for comparative analysis of fungal genomes. BMC Genomics 8(426), 1–15 (2007) 15. e-fungi Project, http://www.cs.man.ac.uk/cornell/eFungi/index.html
Author Index
Bianchini, Devis
Neumayr, Bernd
161
Casamayor, Juan Carlos Celma, Matilde 306 Courter, Andrew 205 De Antonellis, Valeria
306
161
Embley, David W. 137 Eraso, Luis E. 306
Jannach, Dietmar
225
Kappel, Gerti 254 Kop, Christian 21
Pastor, Oscar 306 Porto, Fabio 285 Retschitzegger, Werner
254
Schrefl, Michael 68, 180 Schwinger, Wieland 254 Shekhovtsov, Vladimir A. 117 Shrestha, Rajiv 205 Spaccapietra, Stefano 285 Stumptner, Markus 180
Gao, Le 205 Grossmann, Georg 180 Gut, Alexander 225 Henderson-Sellers, Brian
68
93 Tao, Cui 137 Thalheim, Bernhard Thiagarajan, Rajesh
42, 68 180
Urban, Susan D.
205
Levin, Ana M. 306 Liddle, Stephen W. 137
Virrueta, Aremy
306
Mayr, Heinrich C. Melchiori, Michele
Wieringa, Roel 1 Wimmer, Manuel 254
21 161