Lecture Notes in Business Information Processing Series Editors Wil van der Aalst Eindhoven Technical University, The Netherlands John Mylopoulos University of Trento, Italy Norman M. Sadeh Carnegie Mellon University, Pittsburgh, PA, USA Michael J. Shaw University of Illinois, Urbana-Champaign, IL, USA Clemens Szyperski Microsoft Research, Redmond, WA, USA
8
Joaquim Filipe José Cordeiro (Eds.)
Web Information Systems and Technologies Third International Conference, WEBIST 2007 Barcelona, Spain, March 3-6, 2007 Revised Selected Papers
13
Volume Editors Joaquim Filipe José Cordeiro Polytechnic Institute of Setúbal – INSTICC 2910-761 Setúbal, Portugal E-mail: {j.filipe,jcordeiro}@est.ips.pt
Library of Congress Control Number: 2008930563 ACM Computing Classification (1998): H.3.5, H.4, J.1, K.4.4 ISSN ISBN-10 ISBN-13
1865-1348 3-540-68257-0 Springer Berlin Heidelberg New York 978-3-540-68257-8 Springer Berlin Heidelberg New York
This work is subject to copyright. All rights are reserved, whether the whole or part of the material is concerned, specifically the rights of translation, reprinting, re-use of illustrations, recitation, broadcasting, reproduction on microfilms or in any other way, and storage in data banks. Duplication of this publication or parts thereof is permitted only under the provisions of the German Copyright Law of September 9, 1965, in its current version, and permission for use must always be obtained from Springer. Violations are liable to prosecution under the German Copyright Law. Springer is a part of Springer Science+Business Media springer.com © Springer-Verlag Berlin Heidelberg 2008 Printed in Germany Typesetting: Camera-ready by author, data conversion by Scientific Publishing Services, Chennai, India Printed on acid-free paper SPIN: 12272464 06/3180 543210
Preface
This book contains the best papers from the International Conference on Web Information Systems and Technologies (WEBIST 2007), organized by the Institute for Systems and Technologies of Information, Control and Communication (INSTICC), endorsed by IW3C2, and held in Barcelona, Spain. The purpose of WEBIST is to bring together researchers, engineers, and practitioners interested in the technological advances and business applications of web-based information systems. It has four main topic areas, covering different aspects of web information systems, namely, internet technology; web interfaces and applications; society, e-business and e-government; and e-learning. WEBIST 2007 received 367 submissions from more than 50 countries across all continents. After a double-blind review process, with the help of more than 200 experts from the international program committee, and also after presentation at the conference, 23 papers were finally selected. Their extended and revised versions are published in this book. This strict selection made the conference appealing to a global audience of engineers, scientists, business practitioners, and policy experts. The papers accepted and presented at the conference demonstrated a number of new and innovative solutions for e-business and web information systems in general, showing that the technical problems in this field are challenging and worth further R&D effort. The program of this conference also included three outstanding keynote lectures presented by internationally renowned distinguished researchers. Their keynote speeches reinforced the overall quality of the event. The organization of these conferences required the dedicated effort of many people. Firstly, we must thank the authors, whose research and development efforts are recorded here. Secondly, we thank the members of the program committee and the additional reviewers for their diligence and expert reviewing. Thirdly, we congratulate the INSTICC organizing team for their excellent work. Last but not least, we thank the invited speakers for their invaluable contribution and for taking the time to synthesize and prepare their talks. We hope that you will find this collection of papers interesting and a helpful source of inspiration and that you will look forward to the book containing the best papers of next year’s conference. March 2008
Joaquim Filipe José Cordeiro
Organization
Conference Co-chairs Joaquim Filipe, Polytechnic Institute of Setúbal/INSTICC, Portugal Julià Minguillón, Open University of Catalonia, Spain
Program Co-chairs José Cordeiro, Polytechnic Institute of Setúbal/INSTICC, Portugal Slimane Hammoudi, E.S.E.O., France
Organizing Committee Paulo Brito, INSTICC, Portugal Marina Carvalho, INSTICC, Portugal Helder Coelhas, INSTICC, Portugal Bruno Encarnação, INSTICC, Portugal Vítor Pedrosa, INSTICC, Portugal Mónica Saramago, INSTICC, Portugal
Program Committee Jacky Akoka, France Abdullah Alghamdi, Saudi Arabia A.Y. Al-Zoubi, Jordan Rachid Anane, UK Margherita Antona, Greece Azita Bahrami, USA Matteo Baldoni, Italy Ken Barker, Canada Cristina Baroglio, Italy Sean Bechhofer, UK Andreas Becks, Germany David Bell, UK Orlando Belo, Portugal Bharat Bhargava, USA Paolo Bouquet, Italy Christos Bouras, Greece Amy Bruckman, USA Jack Brzezinski, USA Maria Claudia Buzzi, Italy
Elena Calude, New Zealand Diego Calvanese, Italy John Carroll, USA Nunzio Casalino, Italy Maiga Chang, Taiwan Weiqin Chen, Norway Adrian David Cheok, Singapore Evangelos Christou, Greece Jae Chung, USA Christophe Claramunt, France Isabelle Comyn-Wattiau, France Michel Crampes, France Alexandra Cristea, UK Daniel Cunliffe, UK John Philip Cuthell, UK Alfredo Cuzzocrea, Italy Georgios Dafoulas, UK Vasilios Darlagiannis, Switzerland Alessandro D'Atri, Italy
VIII
Organization
Valeria de Antonellis, Italy Sergio de Cesare, UK Steven Demurjian, USA Darina Dicheva, USA Y. Ding, Austria Asuman Dogac, Turkey Josep Domingo-Ferrer, Spain Chyi-Ren Dow, Taiwan Schahram Dustdar, Austria Ali El Kateeb, USA A. El Saddik, Canada Atilla Elci, Turkey Luis Ferreira Pires, The Netherlands Filomena Ferrucci, Italy Stefan Fischer, Germany Akira Fukuda, Japan Giovanni Fulantelli, Italy Erich Gams, Austria Daniel Garcia, Spain Franca Garzotto, Italy Dragan Gasevic, Canada José A. Gil Salinas, Spain Karl Goeschka, Austria Nicolás González-Deleito, Belgium Anna Grabowska, Poland Begoña Gros, Spain Vic Grout, UK Francesco Guerra, Italy Aaron Gulliver, Canada Pål Halvorsen, Norway Ioannis Hatzilygeroudis, Greece Stylianos Hatzipanagos, UK Dominik Heckmann, Germany Nicola Henze, Germany Dominic Heutelbeck, Germany David Hicks, Denmark Pascal Hitzler, Germany Kathleen Hornsby, USA Wen Shyong Hsieh, Taiwan Brian Hudson, Sweden Tanko Ishaya, UK Alexander Ivannikov, Russia Kai Jakobs, Germany Anne James, UK Narayana Jayaram, UK Ivan Jelinek, Czech Republic Jingwen Jin, USA
Qun Jin, Japan Carlos Juiz, Spain Michail Kalogiannakis, France Jussi Kangasharju, Germany Vassilis Kapsalis, Greece George Karabatis, USA Frank Kargl, Germany Roland Kaschek, New Zealand Sokratis Katsikas, Greece David Kennedy, China Ralf Klamma, Germany Mamadou Tadiou Kone, Canada Tsvi Kuflik, Israel Axel Küpper, Germany Daniel Lemire, Canada Tayeb Lemlouma, France Weigang Li, Brazil Leszek Lilien, USA Claudia Linnhoff-Popien, Germany Hui Liu, USA Pascal Lorenz, France Heiko Ludwig, USA Vicente Luque-Centeno, Spain Anna Maddalena, Italy George Magoulas, UK Johannes Mayer, Germany Carmel McNaught, China Ingo Melzer, Germany Elisabeth Metais, France Alessandro Micarelli, Italy Debajyoti Mukhopadhyay, India Kia Ng, UK David Nichols, New Zealand Andreas Ninck, Switzerland Dusica Novakovic, UK Vladimir Oleshchuk, Norway Andrea Omicini, Italy Kok-Leong Ong, Australia Jose Onieva, Spain Vasile Palade, UK Jun Pang, Germany Kalpdrum Passi, Canada Viviana Patti, Italy Chang-Shyh Peng, USA Guenther Pernul, Germany Rajesh Pillania, India Carsten Pils, Ireland
Organization
Volkmar Pipek, Germany Pierluigi Plebani, Italy Bhanu Prasad, USA Joshua Pun, Hong Kong Mimi Recker, USA Frank Reichert, Norway Werner Retschitzegger, Austria Norman Revell, UK Yacine Rezgui, UK Thomas Risse, Germany Marco Roccetti, Italy Dumitru Roman, Austria Danguole Rutkauskiene, Lithuania Wasim Sadiq, Australia Maytham Safar, Kuwait Eduardo Sanchez, Spain Abdolhossein Sarrafzadeh, New Zealand Anthony Savidis, Greece Vittorio Scarano, Italy Alexander Schatten, Austria Alexander Schill, Germany Heiko Schuldt, Switzerland Magy Seif El-Nasr, USA Jochen Seitz, Germany Tony Shan, USA Jamshid Shanbehzadeh, Iran Quan Z. Sheng, Australia Charles A. Shoniregun, UK Keng Siau, USA Miguel-Angel Sicilia, Spain Marianna Sigala, Greece Eva Söderström, Sweden
Miguel Soriano, Spain J. Michael Spector, USA Martin Sperka, Slovakia Frank Stowell, UK Carlo Strapparava, Italy Eleni Stroulia, Canada Hussein Suleman, South Africa Aixin Sun, Singapore Junichi Suzuki, USA Ramayah T., Malaysia Taro Tezuka, Japan Dirk Thissen, Germany Ivan Tomek, Canada Bettina Törpel, Denmark Arun-Kumar Tripathi, Germany Andre Trudel, Canada Thrasyvoulos Tsiatsos, Greece Klaus Turowski, Germany Michail Vaitis, Greece Christelle Vangenot, Switzerland Athanasios Vasilakos, Greece Jari Veijalainen, Finland Juan D. Velasquez, Chile Maria Esther Vidal, Venezuela Maurizio Vincini, Italy Viacheslav Wolfengagen, Russia Martin Wolpers, Belgium Huahui Wu, USA Lu Yan, UK Sung-Ming Yen, Taiwan Janette Young, UK Weihua Zhuang, Canada
Auxiliary Reviewers Isaac Agudo, Spain Gunes Aluc, Turkey Simona Barresi, UK Solomon Berhe, USA Simone Braun, Germany Tobias Buerger, Austria Patrícia Dockhorn Costa, The Netherlands Volker Derballa, Germany Wolfgang Dobmeier, Germany Thanh Tran Duc, Germany Stefan Dürbeck, Germany
Seth Freeman, USA Vittorio Fuccella, Italy Carmine Gravino, Italy Fu-Hau Hsu, Taiwan Larissa Ismailova, Russia Yildiray Kabak, Turkey Jan Kolter, Germany Jacek Kopecky, Austria Sergey Kosikov, Russia Damien Magoni, France Zoubir Mammeri, France
IX
X
Organization
Jaime Pavlich Mariscal, USA Sergio di Martino, Italy Jing Mei, China Michele Melchiori, Italy Alexandre Miège, Canada Alper Okcan, Turkey Wladimir Palant, Norway Andreas Petlund, Norway Vassilis Poulopoulos, Greece Sylvie Ranwez, France
Cyril Ray, France Christos Riziotis, Greece Claudio Schifanella, Italy Christian Schläger, Germany Paolo Spagnoletti, Italy Agusti Solanas, Spain Heiko Stoermer, Italy Fulya Tuncer, Turkey Knut-Helge Vik, Norway Christian Werner, Germany
Invited Speakers Azzelarabe Taleb-Bendiab, Liverpool John Moores University, UK Schahram Dustdar, Information Systems Institute, Vienna University of Technology, Austria Ernesto Damiani, University of Milan, Italy
Table of Contents
Invited Papers Programming Support and Governance for Process-Oriented Software Autonomy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A. Taleb-Bendiab, P. Miseldine, M. Randles, and Thar Baker Representing and Validating Digital Business Processes . . . . . . . . . . . . . . . Ernesto Damiani, Paolo Ceravolo, Cristiano Fugazza, and Karl Reed
3
19
Part I: Internet Technology Semi-automated Content Zoning of Spam Emails . . . . . . . . . . . . . . . . . . . . . Claudine Brucks, Michael Hilker, Christoph Schommer, Cynthia Wagner, and Ralph Weires
35
Security and Business Risks from Early Design of Web-Based Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Rattikorn Hewett
45
Designing Decentralized Service Compositions: Challenges and Solutions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Ustun Yildiz and Claude Godart
60
Grid Infrastructure Architecture: A Modular Approach from CoreGRID . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Augusto Ciuffoletti, Antonio Congiusta, Gracjan Jankowski, Michal Jankowski, Ondrej Krajiˇcek, and Norbert Meyer The Privacy Advocate: Assertion of Privacy by Personalised Contracts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Michael Maaser, Steffen Ortmann, and Peter Langend¨ orfer
72
85
Efficient Queries on XML Data through Partitioning . . . . . . . . . . . . . . . . . Olli Luoma
98
Contract Based Behavior Model for Services Coordination . . . . . . . . . . . . . Alberto Portilla, Genoveva Vargas-Solar, Christine Collet, Jos´e-Luis Zechinelli-Martini, and Luciano Garc´ıa-Ba˜ nuelos
109
Transparent Admission Control and Scheduling of e-Commerce Web Services . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Dmytro Dyachuk and Ralph Deters
124
XII
Table of Contents
Part II: Web Interfaces and Applications Life Cases: A Kernel Element for Web Information Systems Engineering . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Klaus-Dieter Schewe and Bernhard Thalheim
139
Finding Virtual Neighbors in 3D Blog with 3D Viewpoint Similarity . . . . Rieko Kadobayashi
157
Model Driven Formal Development of Digital Libraries . . . . . . . . . . . . . . . Esther Guerra, Juan de Lara, and Alessio Malizia
169
Predicting the Influence of Emerging Information and Communication Technologies on Home Life . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Michele Cornacchia, Vittorio Baroncini, and Stefano Livi
184
RDF Collections . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Saket Kaushik and Duminda Wijesekera
201
Logging and Analyzing User’s Interactions in Web Portals . . . . . . . . . . . . Gennaro Costagliola, Filomena Ferrucci, Vittorio Fuccella, and Luigi Zurolo
213
A Semantics-Based Automatic Web Content Adaptation Framework for Mobile Devices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Chichang Jou
230
Part III: Society, e-Business and e-Government Summarizing Online Customer Reviews Automatically Based on Topical Structure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Jiaming Zhan, Han Tong Loh, and Ying Liu
245
The Structure of Web-Based Information Systems Satisfaction: An Application of Confirmatory Factor Analysis . . . . . . . . . . . . . . . . . . . . . . . . Christy M.K. Cheung and Matthew K.O. Lee
257
Part IV: e-Learning Introducing Communities to e-Learning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Gavin McArdle, Teresa Monahan, and Michela Bertolotto Evaluating an Online Module on Copyright Law and Intellectual Property . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Carmel McNaught, Paul Lam, Shirley Leung, and Kin-Fai Cheng Using Games-Based Learning to Teach Software Engineering . . . . . . . . . . Thomas M. Connolly, Mark Stansfield, and Tom Hainey
277
292 304
Table of Contents
XIII
Agility in Serious Games Development with Distributed Teams: A Case Study . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Manuel Oliveira and Heiko Duin
314
Improving the Flexibility of Learning Environments: Developing Applications for Wired and Wireless Use . . . . . . . . . . . . . . . . . . . . . . . . . . . . David M. Kennedy and Doug Vogel
327
Evaluating an e-Learning Experience Based on the Sakai Environment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . F´elix Buend´ıa Garc´ıa and Antonio Herv´ as Jorge
338
Author Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
347
Invited Papers
Programming Support and Governance for Process-Oriented Software Autonomy A. Taleb-Bendiab, P. Miseldine, M. Randles, and Thar Baker School of Computing and Mathematical Sciences Liverpool John Moores University, Byrom Street, Liverpool, L3 3AF, UK
[email protected],
[email protected] [email protected],
[email protected]
Abstract. Business Process models seek to orchestrate business functions through the development of automated task completion, which is becoming increasingly used for Service-Oriented Architectures. This had led to many advances in the methods and tools available for software and language support in process modelling and enactment. Recent development in Business Process Execution languages, such as WS-BPEL 2.0 has widened the scope of process modelling to encompass cross-enterprise and inter-enterprise processes with a wide spread of often heterogonous business processes together with a range of associated modules for enactment, governance and assurance, to name but a few, to address non-functional requirements. Hence, the task of provisioning and managing such systems far outstrips the capabilities of human operatives, with most adaptations to operational circumstances requiring the system to be taken offline reprogrammed, recompiled and redeployed. This work focuses on the application of recent developments in language support for software autonomy whilst guaranteeing autonomic software behaviour. The issues to be addressed are stated with a supporting framework and language, Neptune. This is illustrated through a representative example with a case study evaluation reported upon. Keywords: Self-governance, programming language, process models.
1 Introduction Business Process Management is presently a well-established method for designing, maintaining and evolving large enterprise Business Process Models. Business Process Modelling or Management (BPM) originally consisted of workflow type graphical notations based on flow charts. In recent years, however, the scope of BPM, in general has widened to support an enterprise integration technology complementing Service-Oriented Architecture (SOA), Enterprise Application Integration (EAI), and Enterprise Service Bus (ESB). The current understanding of a process is one that orchestrates complex system interactions, and is itself a service capable of communicating and interacting with other processes according to industry standard protocols. Many languages and software tools have been proposed to develop process-oriented J. Filipe and J. Cordeiro (Eds.): WEBIST 2007, LNBIP 8, pp. 3–18, 2008. © Springer-Verlag Berlin Heidelberg 2008
4
A. Taleb-Bendiab et al.
software. Currently there is much interest in web based service orchestration to describe business processes through SOA and the emerging Service Component Architecture model, for example. The primary language for Business process support in this field is the Business Process Execution Language (BPEL), most usually represented by BPEL4WS and its successor WS-BPEL 2.0 [1]; though, other alternatives exist to support process–oriented software for implementing business processes on top of web services. In view of the complexity engendered by the heterogeneity in processes and their functional and non-functional requirements to support enterprise agility and business ecosystems, this work focuses on the support for runtime self-management and its predictable governance [2]. The aim is to contribute to new models of autonomic process-oriented Service Component Architecture (SCA). This approach is seen as vital to support current service intensive systems providing the core infrastructure for commerce and government [3], with its need for constant availability, to be always on and its continuous evolution based on current knowledge and process. Such requirements are evident in a healthcare decision process model which is oftenchanging to reflect new understandings, and the software used to aid the decision processes workflow is required to remain up to date to produce best-practice care [4]. Delays in updating software mean patients are diagnosed using out-of-date medical knowledge and guidelines. Coupled with distribution, where different parts of the application may be executed and stored on different machines, the complexity of deployment and its operating environment drives the need for computational techniques to aid in the analysis and actuation of changes in requirement whilst the software remains operational. Research into autonomic software has yielded many potential solutions to adaptation with respect to complexity [5], often from the viewpoint of providing self-aware behaviour to predict and react to changes in requirements for governance [6]. As identified in [7] however, a systematic approach to the adaptation of software at runtime to new or emergent requirements is still an outstanding issue for research. In addition, providing adaptation of a business process based on assured, bounded governance requires knowledge of not only what should be adapted, but also its impact on other processes and the safety of producing such adaptation within its defined boundaries. In effect, knowledge of the construction and the meanings behind the formation of a business process system is vital for bounded autonomy through adaptation to be accomplished [8]. Issues arise when this knowledge for autonomy is to be inferred at runtime, and adaptation is required to be instigated. Process implementations are most often deployed in a compiled denotation unsuited for reasoning upon to perform coordinated adaptation. Similarly, the compilation process and language used removes much of the semantic direction of the code; code is commonly written as instructions to achieve a goal, without implicitly specifying why those specific instructions were chosen for the task. In modern, distributed systems, it is often the case that source code is not available for adaptation, with the use of software as a service model hiding the implementation from consumers. As a result, much research focuses on behaviour driven adaptation and tuning of an application, rather than focusing on the construction and adaptation of its deployed code base.
Programming Support and Governance for Process-Oriented Software Autonomy
5
It is the contention of this paper that the functionality and knowledge required for autonomic adaptation of process models can be achieved via semantic analysis of the core components and code base of the software itself. This analysis can in turn inform traditional techniques of critique used in Autonomic Software, such as emergence identification and signal-grounding [9]. The paper highlights the limitations in current research into computational runtime adaptation and moves to identify properties of a new language Neptune that attempts to provide the requisite features to meet the goal of runtime computational adaptation of a system in general and process models in particular. Several proof of concept scenarios are given to further illustrate the approach advocated by Neptune and to justify a semantics-based approach. The paper concludes with discussion of limitations introduced by the approach, with work relating to possible future solutions given.
2 Business Process Runtime Requirements for Self-governance The inherent complexity of Business Process Models enactment, where process interactions and cross enterprise applications are involved, contributes towards the increasing interest in emerging autonomic SOA provision. Where autonomic software Table 1. Language Support Issues
Issues 1. Deployment Platform Reliance
2. Development Platform Reliance
3. Expressiveness
4. Application Representation 5. Component Effects 6. Data Distribution
Details There must be no reliance on an extension to, or specialism of, a specific language, operating system, or other system component to function. In effect, it must be deploymentplatform agnostic. The method of code adaptation for autonomic management must be accessible such that it has the ability to be dispatched by, and respond via, standard methods of invocation and communication. In effect, it must be developmentplatform agnostic. The semantic labelling of the processes ultimate purpose must be expressive enough to represent the semantics of base language components – which themselves may be implemented in any programming language that is supported in issue 2 – such that the effect, requirement, and mechanism of actuation of the component can be ascertained at runtime for use by the autonomic manager. A service application must be expressed in terms of the processes it is to represent using an activity-based classification, using the same semantic forms defined in Issue 3. The intended purpose of a component as provided in Issue 3 must be defined by its side effects to the state of the process it is used within. There must be support for runtime distribution of all information expressed within the process model via computation methods and communication standards.
6
A. Taleb-Bendiab et al.
capabilities attempt to predict and react to changes in software requirements for selfpreservation. Using the naturally derived autonomous behaviour exhibited by living organisms, research into autonomic computing is often linked with runtime adaptation and analysis to provide solutions to imbue self-aware behaviour to systems, machines and processes [10]. A wide variety of solutions exist, often targeting different aspects of A wide variety of solutions exist, often targeting different aspects of behaviour in terms of implementation and deployment. As stated in the introduction, Autonomic Computing has traditionally viewed systems from a behavioural perspective via monitoring. There are techniques however, that use code base analysis. Toskana [11] and similar approaches use AOP as a basis for adaptable change. Further holistic paradigms for autonomic software have been made using agent theory [12] treating a complete system as a series of agents, each with their own behaviour models to collectively perform set tasks and behaviour. Such an approach can also be applied to view business processes as sets of agent tasks and roles. Other problems arise from the lack of availability of source code. Often in systems such as service-oriented architectures, only consumption of behaviour is available with the actual implementation and its source code hidden. In these cases, adaptive middleware components that marshal and re-route calls are popular solutions [13]; these, however, often work at an abstraction layer above the code base, via policy documentation, forming of metamodel of behaviour reliant on observation. In line with the highlighted concerns, associated with providing autonomic management of implemented business process applications, there are a number of issues that need to be addressed; these have been collated in Table 1. A more detailed description may be found in the associated technical report [14].
3 Implementation of Autonomic Management As a conceptual basis, the qualities specified previously have been formed to provide the foundation of a scripting language, Neptune, to solve many of the outstanding issues highlighted in current research into runtime analysis and adaptation. Neptune can enable full runtime adaptation for autonomic self-management by addressing the issues previously identified. Issues 1 and 2 both indicate that the solution should sit above and be separate from any single language used to produce language components. In effect, the components are hidden from Neptune by way of a semantic interface: Neptune need only be able to dispatch calls defined within the semantic interface, rather than needing to directly host and manage the component itself. In this way, a bridge between Neptune and a base language is required both to facilitate communication between the base language and Neptune so that the language can instruct Neptune and interpret results from it, but also so Neptune can dispatch a call that is required to be handled by the base language. It is also clear that Neptune requires a form of expression to be defined such that it is capable of representing the ontological model desired to fulfil the remaining qualities. In keeping with established, classical works [15], Neptune will present information using a context-free language NeptuneScript.
Programming Support and Governance for Process-Oriented Software Autonomy
7
Figure 1 shows a schematic view of the method in which Neptune communicates with base language components. The Base Language Connector’s are themselves written within a base language rather than with NeptuneScript, and encapsulate the structure and communication methods needed to interact fully with Neptune and to be able to interpret and write NeptuneScript. Neptune itself provides numerous interfaces using standardised protocols such as COM, SOAP, and RPC forms such as Java RPC or .NET Remoting to enable a base language connector to dispatch a call to Neptune, and receive responses from it, as governed by Issue 2.
Fig. 1. Neptune/Base Language Interface
These calls are marshalled either directly to the higher Neptune Framework for processing, or situated as Neptune Base Language Objects (NBLOs), a semantic form that describes the intentions and conditions needed by a base language component. In addition, the NBLOs contain the requisite information to determine how to interoperate with the appropriate base language (BL) connector to marshal a call to the base language component. This information can be used by the Neptune Framework, to invoke the behaviour described in the NBLO. A semantic linker module within the framework encapsulates the functionality required to match and relate the semantics defined within process models, to those contained within the NBLOs to provide solutions for activities or tasks given in the process. Once a match is found, the calls to the base language contained within the NBLO are actuated. In effect, the axiomatic and operational semantics of a task are related to the denotational semantics given as an actuation within an NBLO’s definition. As discussed in Issue 3, the process of semantic manipulation and inference are well understood in literature and similar techniques are used to inform the linking module. This process is illustrated in Figure 2.
8
A. Taleb-Bendiab et al.
Fig. 2. The Neptune Framework
3.1 A Simple Example In order to illustrate the design basis of Neptune, and show the form NeptuneScripts take, the following example is used: A simple task is portrayed as a process that formats a given string as a title, using different styles that are implemented in distinct languages; a method bold in C# and a method italic in Java as shown below (Figures 3 and 4).
Fig. 3. Code for format.cs
Fig. 4. Code for format.java
Both are described using operational and axiomatic semantics within an NBLO definition to show that the purpose is to provide a feature to a type defined in Neptune, NString to transcend the types used in the base languages. In effect, the NBLO is defined by its effect on the state of any application it is part of – in keeping with
Programming Support and Governance for Process-Oriented Software Autonomy
9
Issue 5 – such that the type NString represents the state of a string through the process being executed. This is shown for the C# bold method in Figure 5. Tasks that require particular features of the type, requirements that are axiomatically defined, can thus be related to the relevant NBLO that can operationally ensure conformance to the requirement.
Fig. 5. NBLO Definitions
Evaluations, if possible, of the operational semantics defined can be included to determine if the goal of the identified process has been successfully actuated. Details of the base language component are provided in the form of an actuation and actual call needed by the base language connector, to provide the denotational semantics that the semantic linker module should produce. Each base language connector defines the meaning of a set of base types provided by Neptune which are related to equivalent types in the base language, such that NString represents a string as understood by Neptune, which is related to System.String in C# and the String type in Java via their respective base language connectors, to maintain interoperability. With this behavioural basis, a task can be written that requires a string to have two features, bold and italic. These are named a and b respectively, so that the requirements can be referenced within the body of the task for adaptation purposes. Neptune can rewrite the task to provide an action element that can satisfy the requirements specified in the task by inferring suitable denotational semantics by linking them to the purposes defined in the NBLOs such that the requirements hold. In this regard, Neptune relates the semantic description given in a task to the denotational semantics provided in the NBLOs given as actual source code invocations, without human intervention. This provides the maximum level of semantic-based assurance to the solution of the process task: each operation to take place via the base language components can be traced and asserted to individual requirements. As the requirements expressed in the task are met, Neptune can then marshal the appropriate calls to the base language connectors to instigate the actual functionality. To highlight the adaptive qualities of Neptune, the task can be rewritten at runtime to only require the code shown in Figure 6:
10
A. Taleb-Bendiab et al.
Fig. 6. Adapted Task Definition
As the actions produced before no longer satisfy the single requirement, such that nblo.Java.Italic provides a feature not required by the task, this can be treated as a trigger for adaptation. This is important as the requirements defined in the task can thus act as a governance mechanism ensuring the behaviour required is actually actuated. In this way, a developer can specify actions, which are asserted against the requirements and refined accordingly. The process of semantic linking can repeat given this trigger, to refine the action element of the task. Furthermore, a requirement can be specified to only be applied if the NBLO evaluates appropriately using the logic outlined in the evaluate element of the NBLO definition. As such, the task can be written a shown in Figure 7.
Fig. 7. Adapted Task Definition with New Inferred Action
In this way, the feature Bold is defined to be evaluated to true if a string starts with “
” and ends with “”. If the string does not contain this feature – it equates to false – then the requirement is actuated. This control structure is repeated within the action element of the task to optimise the process; without the control structure, the action element would require adaptation upon each execution of the task with a given string. To show how a Neptune process can be adapted to reflect new behaviour, a new assembly can be added to the runtime environment of the application, and described via a new NBLO that is registered with the controller at runtime with a similar underline class in C#. The task FormatStringAsTitle can be rewritten at runtime due to its exposition, to include the requirement of the functionality provided for by the new behaviour which is semantically linked and adapted by the respective modules. Thus, the semantic definition of NString.Underline implies that of NString.Bold to be present, allowing the task to be adapted as shown in Figure 8.
Programming Support and Governance for Process-Oriented Software Autonomy
11
Fig. 8. Inferred Bolded Behaviour
4 A Case Study Neptune has been used to develop a number of autonomic process models to test and evaluate its design, performance and further analyse its approach. In particular, Neptune has been used to implement an autonomic service-oriented decision-support process, which enables via situation and role-awareness, the reconfiguration of medical decision agents to adapt to the individual preferences of expert clinicians and hospitals local clinical governance rules. Below is a survey of Neptune’s suitability to produce self-adaptive autonomic business process systems bounded within organisational guidelines via examples of architectural and behaviour adaptation required to be implemented at runtime. 4.1 Architectural Autonomy In partnership with several NHS dental trusts, the triage process of dental access services were assessed and implemented in Neptune, using an activity-based BPM. The model defined individual tasks that represented data input, such that the process required knowledge of a patient. The data could feasibly be read from a database, if it was available, or from the operator directly asking the patient. As such, two NBLOs were defined to describe these methods of data input for the patient’s data, given their NHS ID as shown in Figure 9. As such, both NBLOs have the same purpose, however define different base language methods of obtaining the data. In addition, the getPatientFromOperator purpose specifies that it should only be enacted if the data was not available in the database. A task was setup to obtain this data, however, how the data was to be found was left to Neptune to deliberate upon. In this way, the actual behaviour within the system of locating a patient was left to autonomous control, it is not specified in any source code other than via intentions and semantics of its desired effect. During the lifetime of the application, a new Electronic Patient Register was produced. The existing mechanism for retrieving data within the process now had to not use the local database, but rather the new EPR via a web service interface, written in Java. Accordingly, the existing process model still held, however any reference to nblo.getPatientFromDatabase was required to be adapted to point towards a new
12
A. Taleb-Bendiab et al.
Fig. 9. NBLO Definition for Patient Lookup
NBLO getPatientFromEPR, which semantically defined the new mechanism for retrieving data. For brevity, its source is not fully included, however due to its web service basis, it set a requirement that a SOAP endpoint was made available to it before it could execute, giving a different semantic to that of the database. The refactor types are pre-defined blocks of utility, executed by Neptune via its sources. In this way, when called, the NBLO will rewrite any reference to the existing NBLO in the tasks and processes to the new method required. The original NBLO is then removed from Neptune, giving a redirection of behaviour at runtime. These mechanisms are the same as employed by IDE’s such as Eclipse and Visual Studio, and as such follow classical research into adaptation. In Neptune however, such adaptation is machine-led and performed at runtime to the existing source base without a recompilation, as the code is simply re-interpreted rather than being at a pre-compile stage. However, the interpreter handles refactoring tasks such that when called, it modifies the source code directly to instigate the change. Thus, rather than applying the policy in a meta-form, either via additional policy or monitoring, the changes are applied at the source code level, leading to cleaner and more manageable code base. To conclude, the actions are re-interpreted after the refactor commands were interpreted. As the getPatientFromEPR NBLO required a SOAP service endpoint to be discovered, the actions, after re- interpretation, contain the new requirement set forth by the NBLO, to yield line 7 in Figure 10. Thus, any point in which the process required a feature from the database was automatically adapted at runtime to provide the same feature from a different approach that itself required modifications to the source based on its own requirements, to maintain integrity. As all inferences are semantic based, changes to the requirements and properties of the new system could easily be instigated, and via the checks made by Neptune, full adaptation within the
Programming Support and Governance for Process-Oriented Software Autonomy
13
task could be made. In addition Neptune can be used to describe components implemented in different languages; these could be combined to produce a single solution to a set process. This was achieved without requiring the execution sequence of components to be specified. It was, instead, inferred autonomically from the boundaries described semantically via BPMs.
Fig. 10. Task with New Re-factored Actions
4.2 Behavioural Autonomy Using Neptune, the process that occurred after pain levels were ascertained could be modified directly, producing a unified decision process model for each clinician. Axioms set forth in the base decision model were set, allowing behaviour, such as instructing a patient to go to hospital, to be ratified against institutional procedures. In this way, as long as these were met, the requirements of the model could be instigated. Thus, the task, afterPain could be rewritten at runtime to invoke the method p.ToHospital (say). Other tasks and NBLOs could be written to extend the definition of p.ToHospital such that this can only occur if there is space in the hospital, or waiting times are lower than a specified value. As each aspect of information is defined independently, the cause and effect of such policy does not need to be determined: Neptune relates the semantics to an end-goal, rather than requiring human intervention to achieve runtime autonomic refactoring of behaviour. Where a behaviour is deemed unsuitable, a boundary is formed, such that if no hospitals were available to send a patient, the preference could not be safely realised, and as such, control defaults back to the original model which is rigorously defined. In this way concerns can be autonomously introduced to the system with assured behaviour through maintaining the purposes and requirements of the system.
5 Evaluation Given the increasing requirements for future distributed computing systems to be endowed with runtime adaptation capabilities, there are many applications for the Neptune framework and language, in that, autonomic middleware will incorporate a logical reasoning mechanism to deduce responses, throughout runtime, without halting the system. Presently the rules for autonomic response are often portrayed to the
14
A. Taleb-Bendiab et al.
system as a static rule set consisting of norms or modalities, such as permissible or obligatory. In many distributed settings, as the system evolves and moves away from its starting configuration, these rules will quickly become outdated. It is at this point that the logical mechanisms within Neptune become paramount to dynamically evolve the rule set and code base in line with the systems evolution. There have, therefore been many case study evaluations for this described approach in autonomic middleware applications. The following is a further evaluation of Neptune performance against the Sun Microsystem’s Pet Store and Microsoft’s PetShop benchmark for enterprise applications, which is outlined below. Though, the full description of this evaluation is outside the scope of this paper and can be found in [16]. 5.1 Adaptation Scenario Pet Store [17] is an architectural blueprint from Sun Microsystems highlighting many of the advanced capabilities available with Java as well as detailing the design guidelines for the language. The application produced by the Pet Store blueprint builds an imaginary web-based solution for browsing and purchasing animals within a storefront interface. There are facilitates within the application architecture for administrators of the system to add, remove, and modify the animals available for sale, with their data stored on a database. Thus the Pet Store is publicly available, best-practice system design architecture for e-commerce and enterprise systems in general that Neptune may be evaluated against. Furthermore shortly after the release of Pet Store, Microsoft implemented its own interpretation of the system, PetShop [18], such that a comparison could be made against the effectiveness of a Java implemented enterprise system, and a Microsoft .NET implementation of the same behaviour. The design of the Pet Store model is one that advocates the use of 3-tier architecture. In this approach, a logic layer, provided by language support, sits between a data layer, consisting of database functions and a presentation layer displaying dynamic web pages to the user. To assess the benefits of Neptune it is the logic layer that will be primarily of interest here to evaluate the impact on the system of the adaptation of process. In the PetShop model, the use of thin classes, that represent directly the data types used within the software, act as isolators of dependency between layers. The logic layer consists of classes that each encapsulate an entity used within the shop model; a customer class represents all the features and data model required from a customer, whereas the order class represents the processes that pertain to order transactions within the system. Methods are used to encapsulate processes, such that Product.getProductFromID encapsulates the process needed to obtain a product from the database given an ID. As such, to adapt a process, its associated class and method contained within the logic layer needs to be determined. 5.2 Introducing Neptune to the PetShop/Pet Store Other implementations of the original Pet Store model exist; this model is particularly of interest, as Neptune is required to be introduced to an existing model. In this way, Neptune can be considered as a concern, and thus crosscut against the existing classes representing the processes requiring adaptation. The logic layer in PetShop performs
Programming Support and Governance for Process-Oriented Software Autonomy
15
several key processes: The process of ordering a product, where customer product functions are executed to produce an order, is an example of such a process. It can be seen that adapting the behaviour of the order process, such that a modified process is generated in reference to a new behaviour using a Process-Oriented Architecture (POA), may highlight many of the design qualities given in Neptune, and contrast this against current best-practice advocated by the original designs. In the comparison between PetShop and Pet Store, several criteria were considered to compare the designs. In effect, the benefits of the PetShop were determined by the number of lines of code produced with performance and scalability judged by response time per user, and CPU resources required by both approaches. The scenario outlines that a process should be chosen from the PetShop example to be modelled within Neptune, and analysis made using the criteria set by Microsoft and Sun upon the relative impact of introducing Neptune to an existing system. The model should then be adapted to introduce a new behaviour, with a comparison made between the method required in Neptune to achieve this adaptation, and the method used in the traditional system design, with benefits and limitations discussed accordingly. The process of performing an order within the PetShop is split between the logic layer and the presentation layer. Data is retrieved to inform the process via discrete web pages, such that OrderShipping.aspx, for example, contains the trigger code that moves the process towards another task, when the user has completed the page and clicked a relevant button. The execution of the process then passes to the logic layer to interpret the data, and store this within the database. Once stored, the logic moves the execution to produce another page for the user. As such, the classes within the logic layer can be said to change the state of the process by signifying that a task is complete, and that data is added to the data layer structures, as these are the side effects of its actuation. Similarly, pages within the presentation layer can be said to introduce data to the state of process, and signify that the task of producing data is complete. In this way, exposing the logic layer and presentation layer classes with NBLOs to Neptune will allow the operational semantics of the classes to be imparted. 5.3 Results Using the criteria set forth in the original release of PetShop, the following results were obtained for the execution of a complete order process. These results will, therefore, show the impact of using Neptune in terms of complexity and performance. For response times, Microsoft ACT was used to provide dummy information to the pages so that the process could complete via machine instruction. Averages of 10,000 process executions were made with 100 concurrent visitors. Whilst the original PetShop did not include any need for reflection, it can be considered vital, due to the relationship between runtime analysis and adaptation, that for a system to be available for adaptation, reflection is required. As such, the same test was performed using a modified PetShop implementation that called its methods via reflection. In this way, a benchmark for any form of adaptation could be ascertained.
16
A. Taleb-Bendiab et al.
Fig. 11. Performance Relationship
The results, shown in Figure 11 indicate that the overhead of using Neptune is significant upon the first execution of the process. This is due to the semantic linker having to process the available NBLO descriptions and refactor the description to perform the task. Upon further execution however, as the tasks have already been provided the NBLOs that satisfy their requirements, execution time increases dramatically, introducing a slight overhead in terms of computational performance over the adaptation benchmark.
Fig. 12. Neptune Scalability Testing
Programming Support and Governance for Process-Oriented Software Autonomy
17
In terms of CPU usage the processes of Neptune produced more in the way of computational complexity than the traditionally compiled application. This is expected due to the interpretation that occurs at runtime via use of Neptune. The result of using reflection produced a slight increase in CPU consumption. However, the compiled Neptune system – produced after the first semantic linking – performed relatively worse. It should be noted that the implementation of the Neptune framework is such that it was designed as a prototype, rather than being optimised for performance. As such, it could be expected theoretically, that any overhead can be reduced further with subsequent revisions of the implementation of the framework. The scalability of Neptune in terms of performance (blue line) is measured against a baseline execution rate (green line) in Figure 12. This shows that at present the initial overhead of Neptune performance is quite high. This, however, may be offset by benefits of continuous availability later in the systems evolution.
6 Conclusions This work has proposed applying autonomic runtime support for Business Process Models to provide methods to address self-governance in service-oriented architectures. Specifically this has involved adaptation of the actual code base of the system by employing a scripting language and framework, Neptune, to reason over the perceived goals of the deployed software. This means that interacting or complex business processes specified in any manner can benefit from runtime adaptation. Further work is required around the issues of type safety and scalability in the semantic linking process; as the volume of NBLOs and semantic concerns increases there will be a corresponding rise in the complexity associated with maintaining the optimal behaviour required. In this work the method is shown to have practical and theoretical value in providing autonomous management in SOAs. Further work is required to make this approach more scalable. The present position is that a high initial computational overhead is encountered. Optimisation of the approach will aid scalability; offsetting the initial costs against future saving in system downtime due to the added robustness gained from the runtime adaptation facilities.
References 1. Arkin, A., Askary, S., Bloch, B., Curbera, F., Goland, Y., Kartha, N., Liu, C.K., Thatte, S., Yendluri, P., Yiu, A. (eds.): Web Services Business Process Execution Language Version 2.0. WS-BPEL TC OASIS (December 2005), http://docs.oasis-open.org/wsbpel/ 2.0/wsbpel-v2.0.pdf 2. Luciano, B., Di Nitto, E., Ghezzi, C.: Toward Open-World Software: Issue and Challenges. Computer 39(10), 36–43 (2006) 3. Adeshara, P., Juric, R., Kuljis, J., Paul, R.: A Survey of Acceptance of E-government Services in the UK. In: Proceedings of the 26th International Conference on Information Technology Interfaces, pp. 415–420 (2004)
18
A. Taleb-Bendiab et al.
4. Rousseau, N., McColl, E., Newton, J., Grimshaw, J., Eccles, M.: Practice based, longitudinal, qualitative interview study of computerised evidence based guidelines in primary care. British Medical Journal 326(7384), 1–8 (2003) 5. Dharini, B., Morrison, R., Kirby, G., Mickan, K., Warboys, B., Robertson, I., Snowdon, B., Greenwood, R.M., Seet, W.: A software architecture approach for structuring autonomic systems. In: DEAS 2005: Proceedings of the 2005 workshop on Design and evolution of autonomic application software, pp. 1–7. ACM Press, New York (2005) 6. Kephart Jeffrey, O., Chess, D.M.: The vision of autonomic computing. Computer 36(1), 41–50 (2003) 7. Mazeiar, S., Tahvildari, L.: Autonomic computing: emerging trends and open problems. In: DEAS 2005: Proceedings of the 2005 workshop on Design and evolution of autonomic application software, pp. 1–7. ACM Press, New York (2005) 8. Sharon, A., Estrin, D.: On supporting autonomy and interdependence in distributed systems. In: Proceedings of the 3rd workshop on ACM SIGOPS European workshop, pp. 1–4. ACM Press, New York (1988) 9. Martin, R., Taleb-Bendiab, A., Miseldine, P.: Addressing the signal grounding problem for autonomic systems. In: The Proceedings of the International Conference on Autonomic and Autonomous Systems, 2006 (ICAS 2006), July 2006, pp. 21–27 (2006) 10. Manish, P., Hariri, S.: Autonomic computing: An overview. Springer, Heidelberg (2005) 11. Michael, E., Freisleben, B.: Supporting autonomic computing functionality via dynamic operating system kernel aspects. In: AOSD 2005: Proceedings of the 4th international conference on Aspect-oriented software development, pp. 51–62. ACM Press, New York (2005) 12. Cristiano, C.: Guarantees for autonomy in cognitive agent architecture. In: ECAI 1994: Proceedings of the workshop on agent theories, architectures, and languages on Intelligent agents, pp. 56–70. Springer, New York (1995) 13. Yves, C.: Self-adaptive middleware: Supporting business process priorities and service level agreements. Advanced Engineering Informatics 19(3), 199–211 (2005) 14. Philip, M., Taleb-Bendiab, A.: Neptune: Supporting Semantics-Based Runtime Software Refactoring to Achieve Assured System Autonomy, Technical Report, Liverpool John Moores University http://neptune.cms.livjm.ac.uk/dasel 15. Chomsky, N.: Three models for the description of language. Information Theory, IEEE Transactions 2(3), 113–124 (1956) 16. Miseldine, P.: Language Support for Process-Oriented Programming of Autonomic Software Systems. PhD. Thesis, Liverpool John Moores University (2007) 17. Sun Microsystems, Java Pet Store demo. Java Blueprint (Accessed November 2007), https://blueprints.dev.java.net/petstore/ 18. Microsoft, using .NET to Implement Sun Microsystems’ Java Pet Store J2EE Blueprint Application (Accessed November 2007), http://msdn2.microsoft.com/en-us/ library/ms954626.aspx
Representing and Validating Digital Business Processes Ernesto Damiani1 , Paolo Ceravolo1 , Cristiano Fugazza1 , and Karl Reed2 1
Department of Information Technology, University of Milan via Bramante, 65 - 26013, Crema (CR), Italy {damiani,ceravolo,fugazza}@dti.unimi.it 2 Department of Computer Science, LaTrobe University Bundoora, Melbourne, Australia
[email protected]
Abstract. Business Process Modeling is increasingly important for the digitalization of both IT and non-IT business processes, as well as for their deployment on service-oriented architectures. A number of methodologies, languages, and software tools have been proposed to support digital business process design; nonetheless, a lot remains to be done for assessing a business process model validity with respect to an existing organizational structure or external constraints like the ones imposed by security compliance regulations. In particular, web-based business coalitions and other inter-organizational transactions pose a number of research problems. The Model Driven Architecture (MDA) provides a framework for representing processes at different levels of abstraction. In this paper, a MDA-driven notion of business process model is introduced, composed of a static domain model including the domain entities and actors, plus a platformindependent workflow model providing a specification of process activities. The paper focuses on semantics-aware representation techniques, introducing logicsbased static domain models and their relationship with Description Logics and current Semantic Web metadata formats. Then, the paper discusses some issues emerging from the literature on business processes representation and presents some research directions on the evaluation of the compatibility of business and IT processes with existing organizational environments and practices. The problem of implicit knowledge and of its capture in a manner which allows it to be included in business process design is also discussed, presenting some open research issues.
1 Introduction Today, the term “extended enterprise” is used designate any collection of organizations sharing a common set of goal. In this broad sense, an enterprise can be a whole corporation, a government organization, or a network of geographically distributed entities. Extended enterprise applications support digitalization of traditional business processes, adding new processes enabled by e-business technology (e.g. large scale Customer Relationship Management). Often, they span company boundaries, supporting a web of relationships between a company and its employees, managers, partners, customers, suppliers, and markets. Digital modeling improves the enterprise ability to dynamically J. Filipe and J. Cordeiro (Eds.): WEBIST 2007, LNBIP 8, pp. 19–32, 2008. c Springer-Verlag Berlin Heidelberg 2008
20
E. Damiani et al.
set up new business processes or to adjust existing ones to changing market conditions, increasing their ability to compete. Companies are therefore developing a growing interest in concepts, technologies and systems that help them to flexibly align their businesses and engineering processes to meet changing needs and to optimize their interactions with customers and business partners. In such a scenario, Business Process Modeling (BPM) techniques are becoming increasingly important. Less than a decade ago, BPM was known as “workflow design” and was aimed at describing human-based processes within a corporate department. Today, BPM is used to design orchestration of complex system interactions, including communicating with the processes of other companies according to well-defined technical contracts. Also, it is used to check compatibility and consistence of the business processes of collaborating business entities. A number of methodologies, languages and software tools have been proposed to support digital BPM; nonetheless, a lot remains to be done for assessing a business process model validity with respect to an existing organizational structure or w.r.t. external constraints like the ones imposed by security compliance regulations. In particular, web-based business coalitions and other inter-organizational transactions pose a number of research problems. Recently, the BPM and Web services communities have proposed many languages and standardization approaches that aim at describing business processes, especially from the perspective of Web services orchestration. Well-known examples include BPEL4WS, BPML, XLANG and WSFL. Most languages focus on the representation of the process message exchanges (the so-called process choreography) and on the control and data flow among the involved Web services (the process orchestration). The OMG’s Model Driven Architecture (MDA) provides a framework for representing processes at different levels of abstraction. In this paper we rely on a MDA-driven notion of business process model, composed of a static domain model including the domain entities and actors, plus a platform-independent workflow model providing a specification of process activities. We shall focus on semantics-aware business process representation techniques, introducing logics-based static domain models and their relationship with Description Logics and current Semantic Web metadata formats. Then, the problem of implicit knowledge and of its capture in a manner which allows it to be included in business process design is also discussed, presenting some open research issues. The paper is structured as follows: in Section 2 we introduce rule based business modeling, while in Section 3 we discuss some semantic models for rules. In Section 4, the relation of these semantic models with the ones proposed for Semantic Web metadata is discussed; Section 5 discussed how these two approaches can be successfully hybridized to meet modeling requirements. Finally, Section 6 shows a worked-out hybrid model example, and Section 8 draws the conclusion and outlines our future work.
2 Rule-Based Business Modeling Business modeling languages are aimed at providing companies with the capability of describing their internal business procedures using a formal syntax. Rule-based formalization, being close to controlled natural language statements, will will ensure that businesses involved in processes fully understand descriptions of themselves and other
Representing and Validating Digital Business Processes
21
participants in their business; also, it will enable organizations to flexibly adjust to new internal and B2B business circumstances. More specifically, rule-based business modeling is aimed at producing machine-understandable business statements in a standard rext-based notation. Business modeling activity usually starts from a shared, domainwide business vocabulary and derives an object-oriented model from it. The Object Management Group’s (OMG) Model Driven Architecture (MDA) decomposes the modeling acvtivity’s outcome into two main work products: – Business Rules (BR): declarative statements describing business domains: Controlled english syntax, easy enough to be understood by humans – Business Process Models (BPM): stepwise specifications of individual business processes Machine-understandable and detailed enough to be translated into computersupported workflows Business Rules determine what states and state transitions are possible or permitted for a given business domain, and may be of alethic or deontic modality. Alethic rules are used to model necessities (e.g. implied by physical laws) which cannot be violated, even in principle. For example, an alethic rule may state that an employee must be born on at most one date. Deontic rules are used to model obligations (e.g., resulting from company policy) which ought to be obeyed, but may be violated in real world scenarios. For example, a deontic rule may state that it is forbidden that any person smokes inside any company building. It is important to remark that widespread domain modeling languages such as the Universal Modeling Language (UML) typically express alethic statements only. When drawing a UML class diagram, for instance, the modeler is stating that domain objects belonging to each UML class MUST have the attribute list reported in the class definition, implicitly taking an alethic approach to domain modeling. In business practice, however, many statements are deontic, and it is often important (e.g., for computing metrics) to know if and how often they are violated. The recognition of the importance of this distinction was a major driver of OMG’s SBVR (Semantics of Business Vocabulary and Rules) proposal, aimed at specifying a business semantics definition layer on top of its software-oriented layers. We will now focus on BR syntax, postponing a discussion on their semantics to Section 3. To fix our ideas, in this Section we shall mention a few simple examples based on the (fictitious) global car rental company EU-Rent used in the Semantics of Business Vocabulary and Business Rules (SBVR) draft standard1. The basic types of business rules are listed below, together with some illustrative examples: – Business Rule (deontic: obligation or prohibition) – Example: It is required that the drop-off date of a rental precedes the expiration date on the driverlicense of the customer reserving the rental – Definition (alethic: something true by definition) – Example: The sales tax rate for a rental is the sales tax rate at the pick-up branch of the rental on the drop-off date of the rental 1
The EU-Rent case study was developed by Model Systems, Ltd., along with several other organizations, and has been used in many papers. The body of descriptions and examples may be freely used, providing its source is clearly acknowledged.
22
E. Damiani et al.
– Authorization (permission) – Example: A customer reserving a rental may provide what credit card guarantees the rental if the name on the credit card is the name of the customer or the credit card is for a corporate account authorized for use by the customer. In business rules, a structured vocabulary is used whose symbols belong to three lexical types, namely: – Terms – Examples: account, assigned car, branch, credit card, driver license, drop-off branch, drop-off date, pick-up branch – Names – Examples: EU-Rent, MilanBranch – Verbs (Forms of Expression) – Examples: credit card guarantees rental, customer has driver license Of course, in business rules the relation between terms and concepts is not one-to-one; a single concept can be represented by many terms. For instance, the concept “car” can be represented by alternative synonyms like “automobile” or by translations like “coche” or “voiture”. The BR language must also include a way to express facts as well as rules, and also metadata (annotations) about documents, facts, and rules. However, the OMG SBVR specification does not specify neither how to parse (controlled) natural language, nor how to phrase business rules. This is a source of flexibility, as the same fact can be represented in many equivalent sentential forms. For example, the assertion “the driver’s license expires on date XX/XX/XX” can be alternatively expressed as “The expiration date xx/xx/xxxxis on driver’s license” or as “The driver’s license has expiration date xx/xx/xxxx ”.
3 Logical Models for BR Languages The semantics of BRs has been traditionally described by providing a mapping from the business rules syntax to some well-known logic formalism. Three (potentially conflicting) basic requirements for the logical models underlying BRs have been identified: 1. High expressive power A basic overall requirement for the underlying logics is a high expressive power in specifying rules, in order to provide most of the description capabilities of controlled English. Business modelers accustomed to using English syntax would not accept any too severe limitation to the modeling language’s expressive power. 2. Tractability The need for performing rule checking and for executability, however, implies a strong desire for computational tractability. In other words, the underlying logics expressive power has to be carefully balanced against tractability. 3. Non-falsifiability BR logical semantics should rely on monotonic reasoning, i.e. on inference whose conclusions cannot be contradicted by simply adding new knowledge.
Representing and Validating Digital Business Processes
23
These three requirements have been often emphasized by business analysts and researchers as guidelines toward finding a correct logical interpretation of business models, but are NOT satisfied by current SBVR logical modeling. In order to understand this paradoxical situation, let us briefly discuss the state or research in this area. Much research work has been done to provide a logics-based model for BR including modalities. Indeed, supporting modalities does not mean that it is mandatory to map the BTs to a modal logic. For instance, work by the BR OMG team and specifically by Terry Halpin (including his open source package NORMA, which supports deontic and alethic rules, see sourceforge.net/projects/orm). The approach behind NORMA, the Object-Role Modeling (ORM) starts from controlled natural language to express available information in terms of simple or elementary facts. The information is then converted into an abstract model in terms of natural concepts, like objects and roles. NORMA addresses logical formalization for SBVR by mapping BR’s deontic modalities Required, Prohibited into modal operators obligatory (O), permitted (P) (used when no modality is specified in the rule) and forbidden (F). Deontic modal operators have the following rules w.r.t. negation: ¬Op ≡ P ¬p
(1)
¬P p ≡ O¬p ≡ F p
(2)
Other modal operators used for mapping BR alethic rules are necessary (true in all possible states of the business domain, ), possible (i.e. true in some state of the business domain, and , impossible (). Alethic operators’ negation rules are as follows: ¬♦p ≡ ¬p
(3)
¬♦p ≡ ♦¬p
(4)
Halpin’s NORMA approach represents BRs as rules where the only modal operator is the main rule operator, thus avoiding the need for a modal logics model. Some allowed SBVR formulations that violate this restriction may be transformed into an equivalent NORMA expression by applying modal negation rules and the Barcan formulae and their converses, i.e. ∀pF p ≡ ∀pF p (5) ∃p♦F p ≡ ♦∃pF p
(6)
For instance, the BR For each RentedCar, it is necessary that RentedCar was rented in at most one Date is transformed into It is necessary that each RentedCar was hired in at most one Date2 However, BR rules emerging from business modeling cannot be always transformed into rules where the only modal operator is the main operator. To support such cases, in principle there is no alternative but to adopt a semantics based on a modal logic; but 2
Another transformation that could be used in this context is the one based on Barcan formulae’s deontic variations, i.e. ∀pOF p ≡ O∀F p. We shall not discuss here in detail the application of these transformations to normalizing modal logic formulas; the interested reader can is referred to [12].
24
E. Damiani et al.
the choice of the “right” modal logic is by no means a trivial exercise, due to tractability and expressive power problems [12]. Modal logics reasoning engines do exist; for instance, MOLOG has been developed by the Applied Logic Group at IRIT from 1985 on, initially supported by the ESPRIT project ALPES. MOLOG is a general inference machine for building a large class of meta-interpreters. It handles Prolog clauses (with conjunctions and implications) qualified by modal operators, The language used by MOLOG can be multi-modal (i.e., contain several modal operators at the same time. Classical resolution is extended with modal resolution rules defining the operations that can be performed on the modal operators. according to the chosen modal logics. However, instead of choosing a modal logics and applying MOLOG-style modal reasoning, most current approaches to BR semantics are based on Horn Logic, the well-studied sublanguage of First-Order Logic (FOL) which is the basis of Logic Programming and counts on thousands of robust implementations. Of course, not every software rule engine is (or should be) able to process the entire Horn Logics; full Horn Logic rules are well-known to be Turing complete, hence undecidable. Again, here we shall not attempt to describe the BR-to-Horn mapping in full. Below we give some intuitive correspondences : – Each, Some: Quantifiers. – and, or, not: Logical Connectives. – The, An : Quantification and variable binding. For example. let’s consider a EU-Rent rule expressed in natural language as follows: “For class C rentals, if pick-up branch does not coincide with the drop-off branch, then rental duration cannot be less than two days”. This rule can be parsed to obtain a semiformal (parsed) rule syntax as follows: For each rental X, if X.class = C and X.pickup < > X.dropoff, then days(X.duration) > 1. The corresponding FOL formula can be written as follows: ∀x ∈ Rental, IsEqual(Class(x), C) ∪ IsEqual(P ickU p(x), DropOf f (x)) → IsGreater(Duration(x), 1) In many practical applications, it is possible to carry out deductions over a knowledge base composed of rules like Eq. 7 by standard Prolog-style forward- and backwardchaining rule reasoning. With respect to the requirements we identified at the beginning of the section, however, we remark that the OMG proposed format for encoding BR as generic Horn rules is undecidable in its full expressivity, so we can expect only partial implementations of SBRL reasoners. Also, this reasoning is clearly not monotonic (see the next Section 4.).
4 BR vs. Semantic Web-Style Knowledge Representation Semantic Web (SW) knowledge representation formats aim to integrate heterogeneous information sources by enabling decidable reasoning on metadata. Since individual Web-based systems are not expected to store an exhaustive “mirror” of external data
Representing and Validating Digital Business Processes
25
sources, Semantic Web languages (unlike FOL-based business rules) rely on monotonic reasoning. Intuitively, deductions that are made on data structures that are not exhaustively known to the system must not be contradicted by mere extension of the knowledge base (KB)3 . For applications, the most important effect of taking the SW modeling approach (as opposed to the SBVR one) is that monotonic reasoning yields a stronger notion of negation (¬): If individual i is not known to be member of class C (i.e., the assertion C(i) cannot be found in the KB), then it cannot be assumed to be member of class ¬C. This requirement, generally referred to as open-world assumption (OWA), has often been indicated as the correct interpretation of digital business models (see Requirement 3 in Section 3) because of the frequent under-specification of local business models and the possibly partial knowledge of remote data sources. In Sec. 5, concept Competitor will be defined as an example of open-world entity that may be included in a business model to adjust its dynamic behavior by considering external data structures. Open-world, SW-oriented knowledge representation languages like OWL [17] are based on a different subset of FOL than business rules. Namely, they rely on Description Logics (DL) [1]. However, mature implementations of DL reasoners exist and are constantly developing new features. OWL reasoners are capable of processing data structures that cannot be expressed in rule-based systems, such as recursive definitions and individuals not explicitly defined in the KB). As an example, consider the following SBVR definition and the corresponding DL axiom4: a product is produced by a manufacturer and is distributed in a country Product ≡ ∃producedBy.Manufacturer
∃distributedIn.Country If provided with the assertion Product(PROD-01), a DL reasoner will derive the following assertions with regard to individual PROD-01: ∃producedBy.Manufacturer(PROD-01)
(7)
∃distributedIn.Country(PROD-01)
(8)
Conversely, if individuals ITALY and COMP-01 are known to be related with PROD-01 as in the following assertions: distributedIn(PROD-01,ITALY) producedBy(PROD-01,COMP-01) 3
4
It is interesting to remark that, besides being different from BR semantics, monotonicity is also not the way mass market data storage applications, such as relational and object-oriented databases, interpret information when queried, e.g. by means of a SQL-like interface. Here we concatenate terms to obtain valid tokens, such as distributedIn and distinguish concept names with a leading uppercase letter.
26
E. Damiani et al.
a DL reasoner will derive, as a consequence of the definition above, the following assertion: Product(PROD-01)
(9)
The FOL counterpart of these derivations requires the biunivocal relationship to be split in two rules, because Horn clauses require the separation of the conditions triggering a rule (the antecedent or body) from the corresponding derivation (the consequent or head). Moreover, variables are assumed to be universally quantified; therefore it is not possible to achieve the exact semantics of the DL axiom above. Product(x) ← producedBy(x, y) ∧ Manufacturer(y) ∧ distributedIn(x, z) ∧ Country(z) Product(x) → producedBy(x, y) ∧ Manufacturer(y) ∧ distributedIn(x, z) ∧ Country(z)
Here only the first rule can be implemented, essentially because in Horn engines all variables in the consequent must also be present in the antecedent5. Consequently, only the implication in (9) can be derived using rule languages. While DL is technically a smaller FOL fragment than Horn logics, the above discussion shows that traditional production rules used by Horn rule engines are able to render only part of the semantics of DL-based formalisms. More importantly, rule systems share a weaker notion of negation (∼) according to which lack of knowledge is considered false (negative) knowledge, thus leading to a non-monotonic reasoning generally referred to as closed-world assumption (CWA): If individual i is not known to be member of class C (i.e., the assertion C(i) cannot be found in the KB), then it is assumed to be member of class ∼ C 6 . When modeling business domains, it should be possible to declare concepts and roles as closed (i.e., characterized by complete knowledge). Although in principle it would be possible to apply rule reasoning in order to evaluate a DL-based knowledge base in a closed-world fashion (e.g., by asserting negative facts on closed concepts), a more straightforward approach is the one of the so-called epistemic operators K and A. The former allows for evaluating DL concepts according to facts that are explicitly known to hold (i.e., have not been inferred by reasoning); as an example, the following concept definition identifies individuals whose filler of role producedBy is an individual that is known to be a Manufacturer instance: K-QUERY ≡ KproducedBy.KManufacturer
5
6
Unlike what happens in DL reasoners, which are capable of handling “hypotetical” individuals as fillers of roles producedBy and distributedIn. Note that we distinguish between classical negation as implemented by DL reasoners (¬) and the weaker notion implemented by rule systems (∼).
Representing and Validating Digital Business Processes
27
The epistemic operator K has been implemented (albeit only in queries, not as concept constructor) in the Pellet reasoner [14]. In turn, the auto-epistemic operator A is related with the notion negation-as-failure. Intuitively, AC corresponds to ¬ ∼ C. The following concept definition identifies individuals having as filler of property producedBy an individual that is assumed to be (i.e., is not explicitly asserted as not being) a Manufacturer instance. A-QUERY ≡ KproducedBy.AManufacturer
However, to the best of our knowledge, the autoepistemic operator A has not been implemented so far in any DL reasoner. While DL reasoners extensions look promising for their applications to business modeling, Horn rule reasoning remains useful to manage application-dependent data structures that need not be shared with the structural (DL-based) component of the KB. Also, when translating business rules into the corresponding logic representation, we may need to model constructs not expressible with DL alone. In fact, let aside the restriction to monadic and binary relations, it very easy to derive from SBVR rules role concatenations that are not comprised in the tree-shaped models that can be expressed with DL. As an example, the following SBVR rule =configures a simple cyclic pattern that nonetheless requires a Horn rule for its definition: a direct product is a product that is produced by a manufacturer that is a reseller of the product DirectProduct ← Product(x) ∧ producedBy(x, y)
∧Manufacturer(y) ∧ resellerOf(y, x)
5 Combining DL and Horn Rules As we have seen in Section 4, Horn rule engines do not entirely render the semantics of DL-based formalisms. On the other hand, when writing business rules we may need to model constructs which are not expressible with DL alone7 . This suggests the idea of representing complex business systems using separate structural (i.e., DL-based) and relational (i.e., rule-based) components . However, it is difficult to obtain all the expected logic entailments by mere composition of structural and relational reasoning on the corresponding portions of the KB. To clarify this, consider a barebones structural component constituted by concept definitions Manufacturer and Distributor, together with the following assertion, expressing that individual K-MART is known to be either a Manufacturer or a Distributor (although the system cannot tell which one): (Manufacturer Distributor)(K-MART) 7
Actually, the OWL 1.1 proposal extends the relational expressivity of the language to encompass role composition and additional features. We do not address the OWL 1.1 proposal in this paper as no mature software tools based on it are likely to be available in the short term.
28
E. Damiani et al.
Now we define concept Reseller in the relational component of the KB, as follows: a reseller is a manufacturer or a distributor Reseller(x) ← Manufacturer(x) Reseller(x) ← Distributor(x)
It is clear that is not possible to derive the assertion Reseller(K-MART) by applying the rules to the structural component of the KB or the other way around. The above example shows that even simple implications are not necessarily preserved when dividing the KB into separate components. Furthermore, exploiting full Horn clauses expressive power easily leads to undecidability. In order to achieve sound and complete reasoning (see Requirement 2 of Section 3) on hybrid KBs, it is essential to monitor the flow of assertions between the two components by constraining the possible clauses that populate the relational component of the KB. This can be done by requiring variables referred to by clauses to be associated with at least one non-DL atom in the rule body. The approach proposed in CARIN [11] requires variables from role predicates to satisfy this constrain. Another formalism, DL+log [16], generalizes existing results (namely, AL-log and DL-safe rules [4]) and proposes a weaker safety condition by applying this requirement only to predicates in the rule consequent. Finally, MKNF [5] is a recent proposal integrating all the above formalisms and relies on CARIN’s stronger notion of safety.
6 An Example of Hybrid Reasoning Let us now consider a hybrid knowledge base (HKB) composed of a structural component S, that is a theory in a subset of FOL (e.g. DL), and a rule component R (e.g. Datalog¬∨). In this example we follow the approach proposed in[16], where the rule component constrained in its interaction with the structural component according to a safeness condition. A = AS ∪ AR , with AS ∩ AR = 0 p ∈ AS is a structural predicate r ∈ AR is a rule component A HKB H is a pair (S, R), where the following safeness condition holds. Each rule R has the following form: p1 (X1 ) ∨ ... ∨ pn (Xn ) ← r1 (Y1 ), ...rn (Yn ), sk (Zk ), ∼ u1 (W1 ), ... ∼ un (Wn )
Such that: – – – –
each pi is a predicate of A; each ri , ui is a predicate of AR ; each si is a predicate of AS ; each variable occurring in R must occur in one of the ri ’s.
Representing and Validating Digital Business Processes
29
6.1 Hybrid Reasoning Let H be a KB where S defines the following predicates: ∀x P RODUCT (x) → ∃y.P RODUCED BY (x, y) ∧ MAN UF ACT URER(y) [A1] ∀x P RODUCT (x) → ∃y.DIST RIBUT ED IN (x, y) ∧ COUN T RY (y) [A2] ∀x COMP ET IT OR(x) → ∃y.SELL IN (x, y) ∧ T ARGET COUN T RY (y) [A3] ∀x T ARGET COUN T RY (x) → COUN T RY (x) [A4] ∀x MAN UF ACT URER(x) → RESELLER(x) [A5] ∀x DIST RIBUT OR(x) → RESELLER(x) [A6] ∀x IRRELEV AN TC OUN T RY (x) → COUN T RY (x) [A7] ∀x IRRELEV AN TC OUN T RY (x) → ¬T ARGET COUN T RY (x)) [A8] T ARGET COUN T RY (IT ALY ) [F1] T ARGET COUN T RY (F RAN CE) [F2] T ARGET COUN T RY (SP AIN ) [F3] COUN T RY (HUN GARY ) [F4] MAN UF ACT URER(COMP 01) [F5] DIST RIBUT OR(COMP 02) [F6] DIST RIBUT OR(COMP 03) [F7] SELL IN (COMP 01, IT ALY ) [F8] SELL IN (COMP 01, F RAN CE) [F9] SELL IN (COMP 03, HUN GARY ) [F10] P RODUCED BY (P ROD01, COMP 01) [F11] P RODUCED BY (P ROD03, COMP 03) [F12]
Also, in H, R defines the following predicates: DirectP roduct(x) ← resellerOf (y, x), P RODUCT (x), MAN UF ACT URER(y), P RODUCED BY (x, y) [R1] DirectCompetitor(x) ← resellerOf (y, x), DirectP roduct(x), COMP ET IT OR(y) [R2] resellerOf (COMP 01, P ROD01) [F13] resellerOf (COMP 03, P ROD03) [F14]
It can be easily verified that H satisfies the following ground assertions: – COM P ET IT OR(COM P 01). Based on axiom A3, selling in a target country it is sufficient to be a Competitor. – DirectP roduct(P ROD01) Based on axiom A1 together with facts F5 and F11, PROD01 is a Product; so rule R1 in conjunction with F13 satisfy this assertion. – DirectP roduct(P ROD03) Based on A1 and F11,; PROD03 is a Product so rule R1 in conjunction with F13 satisfy this assertion. Note that it is not relevant that COMP03 is not a Manufacturer because no assertion contradicts this possibility and according to the open world semantics a fact cannot be negated only on the basis of the lack of an assertion stating it. – DirectCompetitor(COM P 01) Since A3, COMP01 is a Competitor, then R2 in conjunction with F13 and R1 satisfy this assertion.
30
E. Damiani et al.
On the contrary, the following assertion cannot be derived in H: – ¬COM P ET IT OR(COM P 03). Due to the open world semantics a company selling in irrelevan t countries (i.e., in countries that are out of our companies’ target ) cannot be inferred not to be a Competitor, because the partial knowledge assumption do not justify us to consider asserted facts as the only possible ones. Note that this semantics prevents deriving false information without justification of asserted facts, and can be important in a strategical context.
7 Related Work Both the business and the software engineering communities have produced much work on Business Process Management. Providing a complete overview of the business management approach to process representation is clearly outside the scope of this paper. The reader interested in a business vision of BPM, seen as a tool for innovative management of virtual organizations, is referred to [8]. Software engineering research has been trying since long to add business process models to the palette of the system designer. For instance, the UML Profile for Automated Business Processes allows BPEL4WS processes to be modeled using ordinary UML tools. The BPEL4WS representation can be used to automatically generate web services artifacts (BPEL, WSDL, XSD) from a UML model meeting the profile, as described for instance in [7]. More recent work by Christian Huemer and Birgit Hofreiter [9] represents the semantics of UN/CEFACTs Modeling Methodology (UMM) business processes using BPEL, showing that BPEL is appropriate to capture business collaborations. An important aspect of inter-organizational business processes is governing different evolution paces due to distributed administration. Work by Stefanie Rinderle, Andreas Wombacher and Manfred Reichert [15] addresses one important challenge which has not been adequately addressed by other approaches, i.e. independent evolution of business process choreographies. If modifications are applied to processes in an uncontrolled manner, inconsistencies or errors might ensue. In particular, modifications of part of a shared process may affect the implementation of the other part of the process by other partners. The authors sketch a framework that allows process engineers to detect how changes to process fragments affect the whole process, relying on change semantics. Other works from the software engineering research community, focus on the alignment between business modeling and software development or execution. As discussed in [10] the general idea is that requirements models are built from organizational goals. Some of these works, such as for instance [13] or [6], are strongly related to UML and do not clearly specify some organizational aspects such as technology that implements business processes or the relationships among the different organizational views. Our own Knowledge Management group at the Software Engineering and Software Architectures Research (SESAR) lab, at the University of Milan’s Department of Information technology (http://ra.crema.unimi.it), has been carrying out work in business process representation and monitoring in the framework of the TEKNE and SecureSCM projects, as in [3] or [2].
Representing and Validating Digital Business Processes
31
8 Conclusions Over the past 30 years, many techniques have grown up around the development, management, and execution of business rules. In parallel, DL-based formalisms have been developed as a part of the Semantic Web proposal. Both technologies seem to have their own strengths and weaknesses. In this paper we we discussed the state of the art of business modeling formalisms, outlining the potential role of hybrid representations in business domain descriptions. We provided preliminary but hopefully useful answers to the following questions: – Are there ways to combine each technology to make them more beneficial than they are being used alone? – Are there enhancements that can be done to further leverage the rules technologies? These questions are being addressed by a collective interdisciplinary effort of the computer science and business management communities. At SESAR lab, we are developing answers to these questions as a part of our current research work. Acknowledgements. This work was partly funded by the Italian Ministry of Research under FIRB contract RBNE05FKZ2_004 (project TEKNE), and by the European Commission under project SecureSCM. The authors wish to thank Claudio Pizzi for his valuable comments on modal logic reasoning. Also, many thanks are due to Schahram Dustdar and Azzelarabe Taleb-Bendiab, Ernesto Damiani’s fellow keynote speakers at WEBIST, and to the conference chair Joaquim Filipe for creating a unique atmosphere for exchanging and discussing research ideas.
References 1. Baader, F., Calvanese, D., McGuinness, D.L., Nardi, D., Patel-Schneider, P.F. (eds.): The Description Logic Handbook: Theory, Implementation, and Applications. Cambridge University Press, Cambridge (2003) 2. Ceravolo, P., Damiani, E., Viviani, M.: Bottom-up extraction and trust-based refinement of ontology metadata. IEEE Transactions on Knowledge and Data Engineering 19(2), 149–163 (2007) 3. Ceravolo, P., Fugazza, C., Leida, M.: Modeling semantics of business rules. In: Proceedings of the Inaugural IEEE International Conference On Digital Ecosystems and Technologies (IEEE-DEST) (February 2007) 4. Donini, F.M., Lenzerini, M., Nardi, D., Schaerf, A.: AL-log: Integrating Datalog and Description Logics. J. of Intelligent Information Systems 10(3), 227–252 (1998) 5. Donini, F.M., Nardi, D., Rosati, R.: Description logics of minimal knowledge and negation as failure. ACM Trans. Comput. Logic 3(2), 177–225 (2002) 6. Eriksson, H.-E., Penker, M.: Business Modeling With UML: Business Patterns at Work. John Wiley & Sons, Inc., New York (1998) 7. Gardner, T.: Uml modelling of automated business processes with a mapping to bpel4ws. In: Proceedings of the First European Workshop on Object Oriented Systems (2003) 8. Fingar, P., Smith, H.: Business process management. The third wave
32
E. Damiani et al.
9. Hofreiter, B., Huemer, C.: Transforming UMM business collaboration models to BPEL. In: Meersman, R., Tari, Z., Corsaro, A. (eds.) OTM-WS 2004. LNCS, vol. 3292, pp. 507–519. Springer, Heidelberg (2004) 10. Diaz, J.S., De la Vara Gonzalez, J.L.: Business process-driven requirements engineering: a goal-based approach. In: Proceedings of the 8th Workshop on Business Process Modeling, Development, and Support (2007) 11. Levy, A.Y., Rousset, M.-C.: CARIN: A representation language combining horn rules and description logics. In: European Conference on Artificial Intelligence, pp. 323–327 (1996) 12. Linehan, M.: Semantics in model-driven business design. In: Proceedings of 2nd International Semantic Web Policy Workshop (2006) 13. Marshall, C.: Enterprise modeling with UML: designing successful software through business analysis. Addison-Wesley Longman Ltd., Essex (2000) 14. Parsia, B., Sivrin, E., Grove, M., Alford, R.: Pellet OWL Reasoner (2003) 15. Rinderle, S., Wombacher, A., Reichert, M.: On the controlled evolution of process choreographies. In: ICDE 2006: Proceedings of the 22nd International Conference on Data Engineering (ICDE 2006), Washington, DC, USA, p. 124. IEEE Computer Society, Los Alamitos (2006) 16. Rosati, R.: DL+log: Tight Integration of Description Logics and Disjunctive Datalog. In: KR 2006, pp. 68–78 (2006) 17. W3C. OWL Web Ontology Language Overview, http://www.w3.org/TR/owl-features/
Part I
Internet Technology
Semi-automated Content Zoning of Spam Emails Claudine Brucks, Michael Hilker, Christoph Schommer, Cynthia Wagner, and Ralph Weires University of Luxembourg MINE Research Group, ILIAS Laboratory Dept. of Computer Science and Communication 6, Rue Richard Coudenhove-Kalergi, 1359 Luxembourg, Luxembourg Abstract. With the use of electronic mail as a communicative medium, the occurrence of spam emails has increased and kept stable; and although spam can be detected by spam filtering, an element of risk of a continuously network overload remains. To add insult to injury, spammers augment novel, clever and astute techniques, which currently leads to a neck-and-neck race between protection and injury. Constantly novel approaches are necessitated to enhance spam filters through a more efficient and effective prevention. In this respect, we argue for a zoning of emails to classify a text in regions, where each region bears a meaning. Here, we use this technique for spams to identify these zones and to evaluate if an email is a spam. We introduce content zoning and prove our approach by a prototypical implementation including first results. Keywords: Content Zoning, Spam Email Detection, Text Analysis and Statistics, Human-Computer Interaction.
1 Introduction Originally, the word spam had firstly been referred to a meat product and had been an abbreviation of Shoulder of Pork and hAM (SPAM), but later on, to Spiced Pork And Meat. By a sketch of Monty Python’s Comedy team, the meaning of the word spam has changed as consequence of spammed by a group of Vikings repeating a song containing . . . spam, lovely spam. . . Due to this excessive use, the word spam has become a term in the Internet, meaning any kind of unsolicited messages or junk emails or bulk emails. The distribution of spam emails is understood as spamming, and the sender is named a spammer. While speaking about spam emails, a multitude of different spam types exist. Unsolicited commercial mail are usually dubious emails that offer products to special price rates, like the advertisement of pharmaceuticals. Chain letters comprise texts with rumors/news instruct the recipient to forward. Malicious forms are worms and viruses, which cause computer damages, and phishing emails that may cause personal damages - mostly of financial type or by requesting sensitive data. Both normal and spam emails consist of several parts, for example the email header and the email content (see Figure 1): the header includes the technical information, protocols, or spam flags, the subject line, and the sender/recipient information. The email content includes the technical information (source code), a readable text (content), including images, and the signature. In the recent years, the importance of electronic messaging is growing and the question of how to control spam emails more and more arises. J. Filipe and J. Cordeiro (Eds.): WEBIST 2007, LNBIP 8, pp. 35–44, 2008. c Springer-Verlag Berlin Heidelberg 2008
36
C. Brucks et al.
Fig. 1. Source code of an electronic mail: Zone 1 represents the HEADER of the email including SUBJECT (marked in green). Zone 3 is the CONTENT, Zone 4 the SIGNATURE.
Many solutions already exist and improvements to avoid this kind of spam messaging are mostly of similar nature. In [6], honeypots accounts are created and positioned to receive spam. Spam filter engines often refer to a rule-based knowledge structure [1,7,10]. Intrusion Detection Systems (IDS) have been enabled [5] that simulate the natural body system control while generating different kinds of cells and cell communication as well. In this respect, each delivered email package, which is sent from one network computer to another, is controlled: if a certain (spam) pattern is present, then the packet is deleted from the network to minimize the risk of intrusion. Beside of that, spam filters exist that directly detect spams, for example by white lists and black lists that contain accepted and denied email addresses. Or, patterns are fixed that is foreseen as to be used by spammers. Another alternative is to analyse the header and the content of emails to classify in spam and non-spam. In fact, natural language filter systems remove emails that are written in a foreign language as well as user-defined rules. Indeed, the spammers continuously introduce novel techniques in order to fail spam filters and to hide spam emails. For example, spam emails can contain text hidden between complex HTML tags or replace characters, e.g. zeros by O’s - a challenge for common-used spam filters. Another example is the recent trend in spam emails of using scrambled words, which means that text - correctly understood by human beings and identified by machines
Semi-automated Content Zoning of Spam Emails
37
with access to natural language dictionaries (thesauri) - cannot be processed seriously by normal spam filters: Aoccdrnig to a rscheearch at Cmabrigde Uinervtisy, it deosn’t mttaer in waht oredr the ltteers in a wrod are, the olny iprmoetnt tihng is taht the frist and lsat ltteer be at the rghit pclae. . . 1 In this article, we follow the idea of content zoning as it is shown in Figure 1 to improve the quality of spam filtering. With content zoning, a text becomes composed of different regions (zones) where each region receives a meaning. For example, a zone SUBJECT may describe the subject of an email, OFFER to the advertised product itself. Therefore, we introduce our content zoning solution and demonstrate it with COZO, an interactive content zoning system [3], on two disjunctive domains: finance and pharmacy. We conclude the paper with an outlook.
2 Content Zoning In [4,9], Argumentative Zoning - that refers to annotated zones to describe and to explain scientific documents - is presented. Its aim is to provide an intelligent library towards the generation of summaries on scientific articles and the detection of intellectual ownership of documents, e.g. for a better document management or for plagiarism detection. 2.1 Annotated Regions in Texts Our approach bases on similar principles to analyse spam emails, as means to separate spam from non-spam mails. A part of this work can also be seen in [3], which deals with the manual analysis of up-to-date spam emails to check if they contain significant patterns in the subject line and the content section. We focus on discovering content structures to describe common patterns in spam emails. With this, we intend to gain information that can be used to perform automatic recognition of zones for further work. Automatically derived information from spam emails can then support a classification of spam emails while separating them from non-spam emails. The aim of the content zoning technique is to provide a useful function or add-on to normal spam filters for the detection of patterns in the content of spam emails and because of the questions if spam emails have characteristic content structures. The separation into zones is done according to the given structure of the documents (e.g. different paragraphs) and especially because of the semantic meaning of the document parts. In the end, the zones are supposed to represent the various semantic units of the document. Besides the plain division of a document in zones, additional analysis can also be performed to gain further information about the separate zones. Rather simple to extract information could be statistics about the size and layout of zones but a more sophisticated analysis of their text content is also possible. The latter can lead to an extraction of the semantic content and purpose of a zone. Such kind of information can be used for various purposes, 1
http://www.mrc-cbu.cam.ac.uk/∼mattd/Cmabrigde/ - November 2007.
38
C. Brucks et al.
such as comparing documents to each other (regarding their analaysed zone structure and content). In the following, we refer to such information about the zones as zone variables. 2.2 Identification of Spam Emails We focus on a test set of spam emails that has been applied to the process of zoning; the test set mainly refers to the different categories of pharmaceutical (i.e. drug offerings) and financial email texts (more precisely, stock spam emails). In the following, we give a more detailed description of how the zones for these two types of spam emails are defined and which zone variables are used. A zone can be defined as a region in a document that is allocated to a specific information in this document. For example, when having a text document with a price offer, this price offer can be annotated by a tag; a logical name for that zone could be price offer. Of course, when having a huge document, zones can reoccur multiple times. Figure 1 has illustrated an example of a pharmaceutical spam email; each zone is represented by a color and an annotated text (HEADER, SUBJECT, CONTENT, SIGNATURE). Financial emails typically introduce themselves as texts of large content, where specific categories like the price, the offerings or the expectation of profits have their place including a very detailed and accurate description. Pharmaceutical texts, on the other hand, refer to texts of shorter size; its content includes in-note-form-like texts with direct offerings and less descriptive text. Sometimes, pharmaceutical spam emails are composed of an image or the text is written completely in white color. Generally, while analyzing financial and pharmaceutical spam emails and characterizing their content, respectively, it has become obvious that pharmaceutical spam emails have a lower amount of zones than financial spam emails [3]. To get granular on electronic spam mails of pharmacy and finance, we have observed that the most redundant text rubrics have been considered as to be the default zones. Since irrelevant text is often included in spam emails, a content may remain unsupported by zoning. The annotation tags have logical names, so that no supplementary information is needed to understand what the zone is about: – INFORMATION: The introductive zone that informs the reader about the intention of the email sending. As an example, the text motivates any kind of lack or absence. – OFFER: This zone represents the offer itself, for example a product or a service. It is accompanied with specific advantages and auspicious expectations. – PRICE: In the PRICE zone, the effective price is put. In many cases, it is a sub-zone of the zone OFFER. – LINK: This zone triggers as connective link to external web-sources. When clicking, the user becomes automatically connected. However, a definite and overall definition of the zones is hard to give, because content zoning alludes more to a informational positions in the text; it does not analyze content via text mining functionalities. We therefore introduce zone variables to identify parts of the text. A zone variable is defined as a parameter that describes specific information of a zone. By describing the zones with zone variables, more structured information can be
Semi-automated Content Zoning of Spam Emails
39
extracted of the zone content. The zone variables are one of the most important factors in content zoning, because they contribute to the statistical evaluation and finally to the detection of similarities in spam emails. In this work, two kinds of variables have been defined. Zone independent variables are defined as parameters which can be applied to every zone. Examples for zone independent variables are for example the position of the zone in the text, the length of the zone, expressed in number of characters, the number of words contained in the zone, or the most occurring word (under consideration of stopword elimination). Zone dependent variables on the other hand are defined as parameters, which are specific for a certain type of zone and hence can only be applied to zones of that type. Mostly, this kind of variables represents semantic parameters, for example top-level domain for links (e.g. .com, .net, .lu), telephone numbers for the address zones, value for price zones, currency for price zones (e, £, $) and so on.
3 The COZO Explorer The COZO Explorer (see Figure 2) integrates a spam thesaurus content-wide and a graphical user interface towards zoning in texts. Furthermore, it supports an automatic image sorting. 3.1 A Spam Thesaurus The Spam-Thesaurus refers to the content of electronic mails and follows the idea that transformation of different writing styles to a consistent form of spam keywords may
Fig. 2. The COZO Explorer consists of two panels: the reader panel (left) and the options panel (right) with the predefined 18 zones. A zone can be marked and colorized, the result is shown in the reader panel.
40
C. Brucks et al.
improve the prematurely detection of spams and an advancement of spam-filtering. In COZO, the thesaurus is a list that is continuously expanded by reading spam email streams. It consists of the most occurring words of all extracted subject lines plus their standardized meaning. Currently, the thesaurus has a keyword-list of more than one thousand entries, excluding its flections, which vary of lot: for example, the word antibiotics has no flection as the word Viagra compounds has more than 120. 3.2 The Graphical User Interface Most common used email types (HTML, EML, etc.) are supported, irrelevant information, like email header or HTML tags, can automatically be removed on demand; only the content remains. The user can zone the content. The results can be exported and stored in XML. In Figure 2, the zones are selected for a given text. To facilitate the zoning, 18 zones have been predefined. However, if there is no convenient zone for a certain email region, the user has the possibility to create his specific zones or to leave that region unzoned. Each zone is visualised by a different colour; for each zone variables, statistical information like the position of the zone in the email, the density of the zone, and the amount of characters in the zone, are computed. The output - definition of zones, calculated variables, and a picture of the zoned email with colours - is also stored in files for further analysis, e.g. picture matching. At this point of time, the zoning must be done manually.
4 Selected Results In the following, we will present some results from a set of spam emails captured from a period of nine months. The data set contains emails from various domains (e.g. credit offers, movie downloads, porn emails), but only spam emails from pharmacy and finance have been tested. 4.1 Zoning Pharmaceutical Spam Emails The mechanism of zoning is firstly applied to spam emails pre-classified as pharmaceutical emails. The number of the test spam set is 100, we have principally taken the four general zones (see Section 2.2) but a bit more adapted to the mentioned domain. For pharmacy, the OFFER zone represents the part where the pharmaceutical product is promoted, LINK the position where the linked homepage is placed; the zone INFORMATION refers to pharmaceutical product descriptions, PRICE to the indication of the price: interestingly, this zone occurs rather seldom. While zoning the spam emails, diverse observations can be done, for example that 64% of the spam emails share a sequential pattern of SUBJECT - OFFER - LINK INFORMATION whereas 16% a sequential pattern SUBJECT - OFFER - LINK. 8% has not a pattern at all. For the tests, no supplementary zones have been generated, the initial 18 zones mostly cover each email content. The zones of the emails from the data set have been colorized as in Figure 3. After having computed the zoning, the statistics are calculated and stored in a XML-file with tags like
Semi-automated Content Zoning of Spam Emails
41
Fig. 3. Content Zoning of an spam email from finance: the zones are colorized and marked and annotated by the rectangles. The pattern ZON E3 → ZON E4 is continuously present.
399 5 80 ...
We have obtained different results, for example that pharmaceutical spam share quite often a similar zoning. Consequently, the order and the size of the zones are similar as well as the calculated variables of each zone. 4.2 Zoning Financial Spam Emails For financial spam emails, the number of the test spam set has been set to 100 as well. Here, we have established principally a list of the following 8 zones: – – – –
ZONE 1 (NAME): Complete name of the company whose shares are offered. ZONE 2 (DESCRIPTION): Description of the company’s offering. ZONE 3 (CURRENT): Current price of the stock. ZONE 4 (PROMISED): Promised price, i.e. the speculative price that the share will reach in a few days time. – ZONE 5 (PROMOTION): Promotion, i.e. arguments to motivate the reader to buy the shares. – ZONE 6 (FACTS): Facts about the company, i.e. a short historical review of the company and its products. – ZONE 7 (NEWS): News about the company and its activities.
42
C. Brucks et al.
– ZONE 8 (FORWARD): Forward looking statement, i.e. a clear statement that no responsibility is taken over. These different zones have been set up manually, and at the beginning 20% of the financial emails have been analyzed to discover any kind of sequential zoning pattern. As financial, especially stock market spam emails, are assumed to be similar in their structure, nine different zones could be determined quite easily and then matched to the rest of the stock mails. Surprisingly, diverse patterns of zone sequences have been discovered, for example that a ZON E3 is always followed by ZON E4 (see Figure 3)2 .
Fig. 4. Coincidental arrangement of zoned spam emails before the image sort has taken place; financial and pharmaceutical spam emails are disordered
4.3 Automated Image Sorting As another alternative for spam email classification, we have downloaded and deployed the freeware software product ImageSorter3 to inspect collections of zoned spam emails and to sort them according to their colorized zones. As we have observed, both financial and pharmaceutical zoned spam emails differ in the number, occurrence and size. We take advantage of this and comply with the idea of accomplishing classification as computing the similarity between images: and, the similarity between zoned spam emails is geared to the color gradient of images. The figures 4 and 5 show some zoned financial and zoned pharmaceutical spam emails before and after an image sort has been applied. Zoned pharmaceutical spam emails are placed to the top; they are separated from zoned financial spam emails that are placed to the bottom. 2
3
The presented rectangles are just for a better understanding; COZO colorizes the zones but does not place a rectangle around it. ImageSorter is a freeware software that is currently developed at the Center of HumanComputer Interaction at the FHTW Berlin.
Semi-automated Content Zoning of Spam Emails
43
Fig. 5. Sorted zoned spam emails where zoned financial spam emails are put to the bottom, pharmaceutical to the top
5 Conclusions There are many advanced possibilities to classify spam emails using content zoning, for example to more automate the content zoning process including the discovery of annotated labels or to define a stronger similarity metrics according to the calculated zone statistics (zone ordering, zone sizes, etc.). The expansion of the idea of image sorting in order to realize the comparison between electronic mails is another claim: since the actual comparison is of course not limited to basic (exact) picture matching but must include more robust image sorting techniques as well. With this, we are able to perform a comparison of a new email to our existing spam email types (financial and pharmaceutical here) to decide whether or not the email is spam. Acknowledgments. This work is funded by the University of Luxembourg within the project TRIAS. We thank all the colleagues and friends of the MINE research group within the Intelligent and Adaptive Systems Laboratory (ILIAS).
References 1. Androutsopoulos, I., Koutsias, J., Chandrinos, K., Paliouras, G., Spyropoulos, C.: An evaluation of naive bayesian anti-spam filtering (2000) 2. Bayerl, S., Bollinger, T., Schommer, C.: Applying Models with Scoring. In: Proceedings on 3rd International Conference on Data Mining Methods and Databases for Engineering, Finance and Other Fields, Bologna, Italy, September 25-27 (2002)
44
C. Brucks et al.
3. Brucks, C., Wagner, C.: Spam analysis for network protection. University of Luxembourg (2006) 4. Feltrim, V., Teufel, S., Gracas Nunes, G., Alusio, S.: Argumentative Zoning applied to Critiquing Novices Scientific Abstracts. In: Computing Attitude and Affect in Text: Theory and Applications, pp. 233–245. Springer, Dordrecht (2005) 5. Hilker, M., Schommer, C.: SANA security analysis in internet traffic through artificial immune systems. In: Proceedings of the Trustworthy Software Workshop Saarbruecken, Germany (2006) 6. Lambert, A.: Analysis of Spam. Master Thesis, University of Dublin (2003) 7. Rigoutsos, I., Chung-Kwei, T.H.: A pattern-discovery-based system for the automatic identification of unsolicited e-mail messages (spam). In: Proc. of the Conference on Email and Anti-Spam (CEAS) (2004) 8. Sun, Q., Schommer, C., Lang, A.: Integration of Manual and Automatic Text Classification A Text Categorization for Emails and Spams. In: Biundo, S., Fr¨uhwirth, T., Palm, G. (eds.) KI 2004. LNCS (LNAI), vol. 3238, pp. 156–167. Springer, Heidelberg (2004) 9. Teufel, S.: Argumentative zoning: Information extraction from scientific text, Phd Thesis, University of Edinburgh, England (1999) 10. Wittel, G., Wu, S.: On attacking statistical spam filters. In: Proc. of the Conference on Email and Anti-Spam (CEAS) (2004)
Security and Business Risks from Early Design of Web-Based Systems Rattikorn Hewett Department of Computer Science, Texas Tech University Abilene TX 79601, U.S.A
[email protected]
Abstract. This paper presents a systematic approach for the automated assessment of security and business risks of web-based systems at the early design stage. The approach combines risk concepts in reliability engineering with heuristics using characteristics of software and hardware deployment design to estimate security and business risks of the system to be developed. It provides a mechanism that can help locate high-risk software components. We discuss limitations of the approach and give an illustration in an industrial engineering and businessto-business domain using a case study of a web-based material requirements planning system for a manufacturing enterprise. Keywords: Software risk assessment, software security, business risks, software architecture, internet-based system.
1 Introduction Web application systems play important roles in providing functions and business services for many organizations. The scope and complexity of web applications have grown significantly from small-scale information dissemination to large-scale sophisticated systems that drive services, collaboration and business on a global scale. Unlike conventional software systems, web-based systems grow and change rapidly. They also tend to be living systems having long life cycles with no specific releases and thus, require continuous development and maintenance [7]. Designing web applications involves not only technical but also social and political factors of the system to be built. Poorly developed web applications that continue to expand could cause many serious consequences. Because they are ubiquitous and deal with a huge and diverse population of users, they must be tolerant to errors, adverse interactions and malicious attacks. Even if they perform perfectly, they can still put the business of the enterprise deploying them at risk if they do not support sound business logic. Effective web-based system design involves both technical and managerial decisions that require a balanced integration of economic flexibility, sound engineering judgment and organizational support. A thorough understanding of risk can help provide the connection between technical concepts, user values and values created by the design. The ability to quickly estimate security and business risks early in the system development life cycle can help software developers and managers make risk-acceptable, J. Filipe and J. Cordeiro (Eds.): WEBIST 2007, LNBIP 8, pp. 45–59, 2008. c Springer-Verlag Berlin Heidelberg 2008
46
R. Hewett
cost-effective and informed decisions. In this paper, the terms security risk and business risk refer to the possibilities of adverse consequences, due to software behavior, to resource ownerships, and to business objectives of the system in which the software is used, respectively. Various techniques and guidelines to security risk analysis are available [10,16] and activities to increase security assurance of software are integrated throughout the software development life cycle (e.g., security requirements, code analysis, security and penetration testing). However, most developers perform security risk assessments of software systems at a system level after the systems have been implemented. For business challenges, efforts on incorporating business processes into enterprise applications are mostly in a requirement stage in order to create a design [4,6,13,15]. However, analyses of business implications of the design products are seldom addressed structurally. Software vulnerabilities can occur in codes (e.g., buffer overflow, SQL injection) and designs (e.g., internal controls that attackers can compromise). They can even exist in components that are not involved with security procedures. As many software-intensive infrastructure systems increasingly utilize COTS (Commercial Off-The-Shelf) components for rapid application development, ability to analyze security risks at a software component level becomes essential. As web applications are implemented on standard platforms, they inherit similar security weaknesses that are published and available publicly [1]. These known vulnerability data-bases are used widely as parts of basic principles for guidelines, standards, and vulnerability scanning tools such as Nessus [2]. Currently, like other information systems, most web-based systems rely on standard checklists, firewall rules and scanners for system security risk testing. One drawback of these tools is their lack of ability to assess vulnerabilities specific to custom web applications [17]. Most traditional risk analysis techniques are difficult to apply directly to modern software design, as they do not necessarily address potential vulnerabilities and threats at a component or environment level [18]. Particularly, web-based information systems are designed on multi-tier architectures spanning over multiple boundaries of trust. A vulnerability of a component depends on both its platform (e.g., J2EE on Tomcat/Apache/Linux, C# on a Windows .NET server) and environment (e.g., exposed network such as LAN versus secure DMZ). To adapt risk methodology to web application design, security risk analysis should consider risk factors from different design levels including software components, hardware platforms and network environments. Risk analysis provides assessment of risk to serve as a basis for decision-making to prevent, mitigate, control and manage risks of loss. This paper presents a systematic approach for automated security and business risk assessment of a web application system from its component-based designs. By combining risk concepts in reliability engineering with heuristics using characteristics of software design and hardware platforms, the security and business risks of a proposed system can be estimated. In contrast with rigorous risk analysis during the operational phase, our approach provides a simple early estimate of risks that can help locate design components that are at risk. Thus, mitigating strategies (e.g., allocating resources for rigorous testing, or redesigning the high-risk component) can follow.
Security and Business Risks from Early Design of Web-Based Systems
47
Section 2 describes related work. Section 3 defines relevant security terminologies and risk concepts, and describes the proposed approach for security risk assessment. Section 4 illustrates our approach to a web application in industrial engineering. Section 5 describes how the proposed approach can be adapted for assessment of business risks. Conclusion and future work are discussed in Section 6.
2 Related Work Much research in web information systems is concerned with techniques and methodologies for engineering web-based systems effectively by adapting principles from software engineering to web application design and development [3,12,7]. Risk methodologies have been employed to assess safety and performance associated with software systems in various application domains including industrial engineering, business and space science [14,5,19]. Risk methodology including FMEA (Failure Mode and Effect Analysis) has been applied to gain understanding about usability of web information systems [20]. Unlike our approach, none of these work addresses risks in security contexts. Identifying likelihoods in risk methodology in these web information systems are mostly subjective rather than analytical and objective. Our work is most similar to a methodology and framework proposed by Yacoub et al. [19]. However, instead of assessing the reliability of software components, we assess the security and business risks associated with software. Thus, their risk models contain different attributes than ours. Reliability risks are estimated from the complexity of software components whereas security and business risks are based on characteristics of software design as well as hardware platforms to be deployed.
3 Security Risk Analysis In this section, we define basic risk analysis concepts and relevant security elements in Section 3.1. Section 3.2 describes the proposed methodology. 3.1 Preliminaries Risk is defined as a multiplication of the probability of occurrence of an undesired event and its consequences [9]. Risk analysis is a process that aims to assess, manage and communicate risk information to assist decision-making. Risk assessment involves identification of adverse situations (or hazards or threats) and quantifying their risks by determining all likelihoods of their causes and the severity of their impacts. The risk measures obtained can assist in cost benefit study for resource planning to prevent, eliminate or mitigate these undesired situations. These are activities of risk management. Many techniques for risk assessment exist (e.g., FMEA, HAZOP (Hazard and Operability), FTA (Fault Tree Analysis) [8]), but they have been mainly applied to hardware rather than software artifacts. Risks associated with software (or software risks) are risks that account for undesired consequences of software behavior, in the system in which the software is used.
48
R. Hewett
Security and safety are two related perspectives of value judgments of acceptable risks. Security is concerned with the protection of ownerships of assets and privacy, while safety is more concerned with the protection of human lives and environments, factors that cannot be evaluated easily in monetary terms. Examples also include some assets in web applications e.g., trust and a company’s reputation. Sources of threats include malicious (e.g., deliberate attack, code injection) as well as non-malicious actions (e.g., natural disaster, human error, software system failure) that may lead to a security breach. Security attacks, whether intentionally targeted or not, can originate from internal and external threats depending on the degree of exposure of assets and how vulnerable the system is to attack. A system’s vulnerability refers to its weakness (i.e., specific attribute/environment) that makes a system more prone to attack or makes an attack more likely to succeed. Exploited vulnerabilities can compromise security. In finance, risks are estimated in terms of expected loss per year, ALE (Annualized Loss Expectancy), which is a product of the annual rate of occurrence and single loss expectancy. The former reflects probability, and the latter for consequences over a period of time. In security risk analysis, risks are estimated from adapted ALE formulae. Basic probability factors are related to exposure degrees, vulnerabilities and threats, whereas basic consequence factors are measured by asset values. Thus, for a given threat/adverse action, its security risk can be roughly estimated by: Risk = threat × exposure × vulnerability × severity where threat actually measures the threat likelihood. The first three contributes to probability measures, whereas the last measures its consequence. In this paper we omit threat, which can be easily integrated into our analysis at a later stage of the design. While the ALE concept works well in general financial and security risks, it may not be applicable for cases like terrorist attacks, which are rare or may never occur. 3.2 Methodology Quantitative risk assessment methodologies, such as the ALE-based approach, provide useful mecha-nisms to facilitate risk ranking for decision-making in risk management. However, they are often criticised for producing results that are too general (statisticlike) to be useful for specific decision-making, and producing results that are not credible enough as they rely heavily on subjective value assignments. The authors’ proposed methodology given below has two objectives: (1) to estimate security risks of a web-based software system from its early design configuration, and (2) to estimate probability measures more objectively. The latter is achieved by using heuristics based on relevant design attributes (more details later). Because web applications are typically designed with multiple architectural layers and trust boundaries, our risk methodology must incorporate relevant attributes from multiple design levels (e.g., component, network and hardware platforms). Figure 1 shows basic steps in the methodology. The methodology requires two inputs: a software design and a hardware deployment design. We assume that the software design is expressed in terms of an extended activity
Security and Business Risks from Early Design of Web-Based Systems
49
Fig. 1. Proposed risk methodology
or workflow diagram, where the software components of the web applications and use case actors are represented as well as activities in use cases [3]. The software design includes all use cases, each of which may have one or more scenarios. The hardware deployment design provides information about hardware platforms including host or server, their attributes (e.g., services, operating environments), and network attributes (e.g. IP addresses for network configuration, and LAN, WAN or DMZ for connection). Security scanning tools use this information to identify vulnerabilities of each host in the system. The hardware deployment plan also indicates software components designated to use, or run on, corresponding hosts. Examples of a workflow diagram and a hardware deployment diagram are shown in Figures 3 and 4, respectively. As shown in Figure 1, the first two steps use software design characteristics as heuristic measures of probability factors of risk. Step 1) computes interaction rates among software components for each scenario based on the design workflow. For simplicity, we assume that each outgoing interaction from the same component occurs equally likely (when insights about interaction likelihoods are available, appropriate weights can be applied). Step 2) constructs a graph whose nodes represent software components. Transition rates among components are computed by taking scenario likelihoods into account for combining interaction rates from each scenario. In security risk contexts, scenario likelihoods are heuristic measures of probabilities of potential threats (e.g., when scenarios involve adverse actions), whereas interaction rates are heuristics that partially measure degrees of component exposure. Step 3) uses characteristics of hardware deployment design as a heuristic measure of an additional probability factor of security risk. We first determine vulnerability sources of each host. This can be done by a table lookup procedure on publicly available vulnerability databases for vulnerability sources based on relevant host and network attributes (e.g., IP addresses, services). Examples of vulnerability sources include Apache Chunked-Code software on web servers, operating environments Windows XP SP2 and
50
R. Hewett
JRE 1.4.2 for running JAVA programs on application servers, and Oracle and TNS Listener software for database servers. Such vulnerabilities and their sources can be easily identified by scanning tools when hardware platforms to be deployed are already in operation. Next step is to measure vulnerabilities of each software component. Each component may require one or more hosts. For example, a component that runs on one application server may also need to query data on another database server. From the deployment design, we can determine all hosts required by each component. Let Hi be a set of hosts required by software component i and vh be a number of vulnerabilities of host h. For component i, a number of vulnerabilities, vi and its normalization to a vulnerability ratio, Vi are estimated as: vh and Vi = vi / vi . vi = h∈Hi
i
Here vulnerability ratios of software components are heuristic measures of likelihood factors of security risks. The more vulnerable software component is, the more likely that the system security can be compromised. Step 4) combines all probability factors obtained in Steps 2) and 3) to estimate the likelihood of each component being attacked (or security breach). Here we interpret this likelihood as a chance of a component under investigation to be the first being attacked in the system. Thus, all components and links along the path to reach the component are secured (unexposed to attack and unexploited vulnerability). The likelihood of a node being secured is estimated by subtracting its exposure degree, estimated by its weaknesses (i.e., vulnerability ratio), from one.
Fig. 2. Likelihoods of security breach in components
We elaborate this step by a small example of a dependency graph shown in Figure 2, where nodes S, A, B, C and D represent components of a software design. A doublelined arrow indicates that S is an entry point to software system execution. Each component and transition is annotated with a corresponding vulnerability ratio (in bold) and a transition rate, respectively. As shown in Figure 2, the likelihood of S to breach security is determined by the vulnerability ratio of S itself. The chance of A being attacked (via S → A) is estimated by its chance to be reached without attacks (i.e., S being secured and S transits to A) and its exposure factor (weaknesses). The former is estimated by the likelihoods of S being secured and S transiting to A, whereas the latter is estimated by A’s vulnerability ratio. Thus, the chance of A being attacked is estimated
Security and Business Risks from Early Design of Web-Based Systems
51
by (1 − 0.3) × 0.2 × 0.1 = 0.014. Similarly, the likelihood estimate of security breach in B is 0.224. The chance of C being attacked is estimated by the sum of the chances of C being attacked via S → A → C (i.e., (1 − 0.3) × 0.2 × (1 − 0.1) × 1 × 0.1 = 0.0126) and via S → B → C (i.e., (1 − 0.3) × 0.8 × (1 − 0.4) × 0.1 × 0.1 = 0.00336). This gives 0.01596. In general, the estimated probability along an access path is analogous to propagation of probabilistic effects in Bayesian network by using Bayes Rule [11]. The transition rate plays a similar role to a conditional probability of a destination being attacked given a source being reached. For simplicity, we do not explicitly represent access privilege. We assume that any activity in the same component can be equally accessed. However, not every component deployed by the same host can be accessed equally. We also assume that network connections between hosts do not have vulnerabilities. Such vulnerabilities can be easily incorporated in our proposed approach. Note that since all probability factors (transition rate, vulnerability ratio) are no greater than one, the longer the path to reach a component is, the harder it is to attack (i.e., its chance of being attacked decreases). For a path that contains cycles, we consider the maximum likelihood that can be obtained. Thus, the heuristic function we use for estimate likelihoods conforms to intuition in security contexts. Step 5), shown in Figure 1, estimates consequences of each adverse action in each software component using approaches similar to HAZOP or FMEA. In this paper we focus on probability estimation. Thus, we apply a typical severity analysis technique, categorize and quantify component consequences according to security standards [9]. By using the maximum severity obtained in each component, we can compute an estimate of security risks of each component of the system being design. For a risk estimate of an overall system, we estimate the likelihood of a system by multiplying the likelihood of a system to safely terminate by a vulnerability ratio of overall system components, and the system severity. We estimate the system severity by taking a majority of severity categories of components in the overall system. Section 4 illustrates our methodology in a case study of web application design.
4 A Case Study This section presents an illustration in an industrial engineering and business-to-business (B2B) domain with a case study, adapted from [12], of a web-based material requirements planning system for a manufacturing enterprise. 4.1 Web Application Design A web-based material requirements planning (MRP) system takes customer’s product order through the B2B web front and performs material requirements planning. The goal of the system is to respond quickly and effectively to changing requirements to minimize delay in production and product delivery. The system automatically updates orders and predicts the daily planned orders, which may be overwritten by a manager. The daily planned orders are sent to a job shop that simulates real-time scheduling and production. At the same time, the system determines materials required for the daily
52
R. Hewett
planned orders and sends orders of raw materials to appropriate suppliers. Job shop simulator reports on finished parts/products to the MRP system, where the manager may adjust some initial parameters (e.g., lead time, lot size) for scheduling production plan to meet customer order requirements. A supplier can view material order status and acknowledge the receipt of the order. The web application design consists of seven software components: MRP, MPS (Master Production Schedule), BOM (Bill of Material), JOB (Job shop simulator), CI (Customer Interface), MI (Manager Interface), and SI (Supplier Interface). MRP validates logins, records product orders and determines customer’s billing. MPS schedules appropriate daily production plan from a given product demand (e.g., from customer order) and other input parameters. The details of MPS are beyond the scope of this paper. BOM determines parts required for each product to be produced and sends order for raw materials to suppliers. JOB is a job shop component that simulates real-time scheduling and production in a manufacturing plant. CI, MI and SI facilitate web interfaces for different users. Customers and suppliers are required to login via a web front, while a manager can do remote login to MRP.
Fig. 3. Workflow diagram of a customer use case
To focus our illustration on risk methodology, we simplify a B2B process by assuming that all payment between a customer and a manufacturer, or a manufacturer and a supplier are off-line transactions. We represent the design in an extended workflow diagram as described in Section 3.2. To make it easy to understand our analysis approach, we present the workflow in three use cases, each of which has one scenario. The three use cases are customer orders products, manager adjusts demands/requirements, and supplier acknowledges material order. We assume that the likelihood of occurrence of each scenario is 0.4, 0.2, and 0.4, respectively. Figure 3 shows a workflow diagram of a process to handle customer orders for products. We assume customers have established business relationships with the manufacturing enterprise and therefore MRP is restricted for use with these customers only. Thus, customers and suppliers are required to login via a web front. The workflow starts at
Security and Business Risks from Early Design of Web-Based Systems
53
a dark circle and ends at a white circle. It represents both sequential (by directed link) and concurrent (indicated by double bar) activities among different components. For example, after determining parts required for products production, BOM sends results to job shop to simulate production of product orders and concurrently it places order of raw materials to appropriate suppliers. The workflow diagram is extended to include software components responsible for functions/executions of these activities. The components are shown in oval shapes on the right of the figure. The arrows between components indicate the interactions among them.
Fig. 4. Hardware architecture and deployment plan
Second part of the design is a hardware deployment plan. Figure 4 shows the design of hardware platforms required. Like many web applications, the MRP system has a three-tier architecture. The presentation layer requires a web server, W . The business logic layer contains three application servers: A1 − A3. Finally, a data layer employs four database servers: D1 − D4, each facilitates data retrieval from different databases as summarized at the bottom of the figure. The servers/hosts are connected via LAN with a firewall between the web server, W and the LAN access to the three application servers. The deployment plan annotates each server with software components that rely on its services as appeared in a square box on top of each server. For example, as shown in Figure 4, a manager can login remotely to MRP via MI, both of which run on application server A1. MRP and MI allow him to review production status and adjust product demands. Since MRP is also responsible for login validation, it needs to retrieve password data. Thus, unlike other application servers, A1 needs to be connected with D1. Next section describes the analysis for assessing security risks associated with each of these software components from a given design. 4.2 Estimating Security Risks The first step for the proposed risk methodology is to calculate interaction rates from each scenario of the web application design. As shown in Figure 3, MRP has one interaction to MPS but has a total of four interactions from it (two from MRP to a terminal state, one to CI and one to MPS). Thus, an interaction rate from MRP to MPS is 1/4 as shown in table M1 of Figure 5. M1 represents interaction rates among components obtained from a “customer orders product” scenario, where T is a terminal state in the
54
R. Hewett
Fig. 5. Tables of interaction rates in each scenario
Table 1. Vulnerability Measures
workflow. Similarly, we can obtaintables M2 and M3 representing interaction rates in the manager and supplier scenario, respectively. Step 2) constructs a dependency graph of software components. For each pair of components, we compute a transition rate using the corresponding interaction rates obtained from each scenario (M1 , M2 and M3 ) and a given likelihood of each scenario, as described in Figure 1. In Step 3), determine vulnerabilities of hardware platforms from a given deployment plan shown in Figure 4 by applying a scanning tool (if the required host and network configurations of the system are already in operation), or (otherwise) a table lookup procedure to public vulnerability databases. Table 1(a) shows vulnerability sources and their counts on our application deployment plan. As described earlier, the table lookup for vulnerability sources are based on software and its environments required on each host to provide their services. Next, to find a vulnerability ratio, Vi of component i, we determine host services required by the component. For example, the customer interface, CI requires the web server W for login and ordering products, and also requires query retrievals of product information from the database D2 for browsing. Similar results can be obtained for other components as summarized in Table 1(b). The second to last column shows the number of vulnerabilities associated with each component. This is estimated by summing a total number of vulnerabilities in each of the hosts required in the component. The last column shows vulnerability ratios with a total ratio of one for overall system vulnerability. Figure 6 shows the resulting graph obtained from Step 2) with annotated transition rates and vulnerability ratios (in bold), heuristic estimates of vulnerability of the design components. Here we assume that at the starting point S (a solid circle), the system has no vulnerability, whereas the terminating point T (a white circle) has a vulnerability ratio of the whole system (i.e., one). Step 4 determines likelihoods of attacks to (or security breach in) each component using the approach as described in the example of
Security and Business Risks from Early Design of Web-Based Systems
55
Fig. 6. A dependency graph of software components
Table 2. Severity analysis of the web application design
Figure 2. For example, the chance of CI being attacked (via S → CI) is 1×0.4×0.09 = 0.036. Note that the path to CI from MRP (via S → CI → M RP → CI) is ignored since it produces a lower likelihood and thus, not the worst case as desired. The chance of MRP being attacked is computed as a sum of likelihoods of MRP being reached from CI, MI, SI, and JOB, i.e., 0.06 + 0.02 + 0.04 + 0.01 = 0.14. A summary of component security breach likelihoods is given in the first row of Table 4. In Step 5), we estimate consequences of adverse actions in each component by categorizing consequences into five categories: CT (catastrophic: 0.95), CR (critical: 0.75), MG (marginal: 0.5), and MN (minor: 0.25) [9]. The estimated severity for each component is shown in Table 2. By selecting the maximum severity obtained in each component to be the component severity, and applying standard quantification to it, we can obtain risk of each component as shown in Table 3. Here MRP is the most high-risk component with 34.31%
56
R. Hewett Table 3. Risk analysis results
risk, whereas MI is the lowest with 2.64% risk. The risk of overall system is computed by multiplying the likelihood of reaching the terminating point, T with its vulnerability and a severity of the overall system. Here we use, CR, which is a severity category of a majority of all components. The result in Table 3 shows that the security risk of overall system is about 50%, which can be used as a baseline to compare with alternatives. These risks give relative measures for assisting decision-makings on security options from different designs.
5 Business Risk Analysis Although both safety and security can impact the business success of the organization deploying the software, business risks have different additional characteristics that deserve to be considered separately. Despite many technology advances, business application designers are facing with organizational challenges in customizing the application to fit in with existing business processes and rules in order to ensure effective coordination of work across the enterprise. This section illustrates how the proposed risk estimation framework in Section 3 can be applied to analyzing the business implications from the design. The risk methodology for business risks can be simplified from the methodology shown in Figure 1 by omitting Step 3 to computer system vulnerabilities and eliminating vulnerability factor in Step 4. However, more detailed severity analysis is necessary. In particular, we are interested in analyzing consequences of a business failure. The analysis can help identify counter measures for design improvement. This is accomplished by distinguishing different types data flow terminating points of a given component-based software design. For example, consider the web application design described in Section 4 in our case study. Figure 7 illustrates three possible types of exits: • Quit before enter (q1 ): the customer either did not have login information at hand or had no time, or did not like the early login compulsion (especially for first time visitors). For business perspectives, one counter measure to prevent losing potential customers is by removing the early login requirement. • Quit after product order (q2 ): the customer did not have sufficient credit payment, or changed decisions (e.g., due to products or cost after adding shipment charge). One strategy to counter measure failure at this stage could be to give customers more delivery and shipment options including discounts on large orders. Alternatively, the inventory could be running out of items. In such a case, a counter measure can be marketing research and increasing inventory or supply of high demand.
Security and Business Risks from Early Design of Web-Based Systems
57
Fig. 7. Determining business implications from software data flow design
• Quit after payment or after material order (q3 ): this is a normal exit that meets the business objectives, i.e., to obtain transactions from customer and to allow a manager to order the supply materials required for the products. Consequently, the separation of different types of failures yields a slightly different dependency graph of software component from our previous security analysis. A new dependency graph is shown on the right of Figure 7. The likelihoods of each software component can be estimated in a straightforward manner. For severity analysis, we analyze consequences of failures. Table 4 gives an example of failure consequences of business contexts in our example where q1 is not considered as a failure state. Severity analysis is organization dependent and in practice, quantifying severity can be assessed from experiences (e.g., using expected loss per year). If such data is not available, a common practice is to categorize severity into four standard categories: catastrophic (CT), critical (CR), marginal (MG), and minor (MN), each of which is respectively quantified by 0.95, 0.75, 0.50 and 0.25. For example, as shown in Table 4, quitting after payment in q2 results in loss of one order transaction, which is critical but not as severe as loss of opportunity for numerous new customers as quitting before enter in q1 . The rest of the proposed steps can be applied similarly to obtain business risks fornot meeting business objectives as designed. These are reflected in quitting in q1 and q2 .
Table 4. Severity of different stages of business failures
58
R. Hewett
6 Concluding Remarks This paper attempts to provide a general framework for security risk analysis of web application software design. We present a systematic methodology for automated preliminary risk assessment. Our focus here is not to estimate precise risk measures but to provide a quick and early rough estimate of security risks by means of heuristics based on characteristics of software design and hardware platforms to help locate high-risk components for further rigorous testing. Our risk model is far from complete partly due to inherent limitations in the design phase. Although, we illustrate our approach to web applications, it is general to apply to other software-intensive systems and to extend the framework and the methodology to include additional security elements in the model. The approach has some limitations. Besides obvious requirements on knowledge about system usage scenarios and component-based design, the approach is limited by insufficient data available in the early phases of software life cycle to provide precise estimation of the system security. However, the risk analysis methodology suggested in this paper can be used to estimate security risks at an early stage, and in a systematic way. Finally, the approach does not consider failure (to satisfy security criteria) dependencies between components. Thus, risks of a component connected to attacked components are determined in the same way as those that are not attacked. We plan to extend this framework to address this issue by providing mechanisms that take user roles, access privileges and vulnerability exploits into account. Additional future work includes refinement of explicit severity analysis and incorporation of attack scenarios in the risk models. Acknowledgements. This research is partially supported by the Sheldon Foundation. Thanks to Phongphun Kidsanayothin and Meinhard Peters for their help in the previous version of this paper whose parts were presented at the 3rd Conference of Web Information Systems and Technology.
References 1. Bugtraq (October 2006), www.securityfocus.com/archive/1 2. Nessus Vulnerability Scanner (October 2006), www.nessus.org 3. Barna, P., Frasincar, F., Houben, G.-J.: A workflow-driven design of web information systems. In: ICWE 2006: Proceedings of the 6th international conference on Web engineering, pp. 321–328. ACM, New York (2006) 4. Bleistein, S.J., Cox, K., Verner, J.: Requirements engineering for e-business systems: Integrating jackson problem diagrams with goal modeling and bpm. In: 11th Asia Pacific Software Engineering Conference, Busan, Korea (2004) 5. Cortellessa, V., Appukkutty, K., Guedem, A.R., Elnaggar, R.: Model-based performance risk analysis. IEEE Trans. Softw. Eng. 31(1), 3–20 (2005); Senior Member-Katerina GosevaPopstojanova and Student Member-Ahmed Hassan and Student Member-Walid Abdelmoez and Member-Hany H. Ammar 6. Csertan, G., Pataricza, A., Harang, P., Doban, O., Biros, G., Dancsecz, A., Friedler, F.: BPM based robust E-Business application development (2002) 7. Ginige, A., Murugesan, S.: Web engineering: An introduction. Multimedia 8, 14–18 (2001)
Security and Business Risks from Early Design of Web-Based Systems
59
8. Haimes, Y.Y.: Risk Modeling, Assessment, and Management. Wiley-IEEE (2004) 9. ISO. Risk Management - Vocabulary - Guidelines for Use in Standards. ISO Copyright Office, Geneva (2002) 10. Landoll, D.J.: The Security Risk Assessment Handbook: A Complete Guide for Performing. CRC Press, Boca Raton (2006) 11. Pearl, J.: Graphical models for probabilistic and causal reasoning. In: Handbook of Defeasible Reasoning and Uncertainty Management Systems, vol. 1, pp. 367–389 (1998) 12. Qiang, L., Khong, T.C., San, W.Y., Jianguo, W., Choy, C.: A web-based material requirements planning integrated application. In: EDOC 2001: Proceedings of the 5th IEEE International Conference on Enterprise Distributed Object Computing, Washington, DC, USA, p. 14. IEEE Computer Society Press, Los Alamitos (2001) 13. Russell, N., van der Aalst, W.M.P., ter Hofstede, A.H.M., Wohed, P.: On the suitability of uml 2.0 activity diagrams for business process modelling, pp. 95–104. Australian Computer Society, Inc., Hobart (2006) 14. Shahrokhi, M., Bernard, A.: Risk assessment/prevention in industrial design processes. In: 2004 IEEE International Conference on Systems, Man and Cybernetics, vol. 3, pp. 2592– 2598 (2004) 15. Singh, I., Stearns, B., Johnson, M.: Designing enterprise applications with the J2EE platform, p. 417. Addison-Wesley Longman Publishing Co., Inc., Amsterdam (2002) 16. Stoneburner, G., Goguen, A., Feringa, A.: Risk management guide for information technology systems. NIST Special Publication, pp. 800–830 (2002) 17. van der Walt, C.: Assessing Internet Security Risk, Part 4: Custom Web Applications, securityfocus.com (October 2002) 18. Verdon, D., McGraw, G.: Risk analysis in software design. Security & Privacy Magazine 2, 79–84 (2004) 19. Yacoub, S.M., Cukic, B., Ammar, H.H.: Scenario-based reliability analysis of componentbased software. In: ISSRE 1999: Proceedings of the 10th International Symposium on Software Reliability Engineering, Washington, DC, USA, p. 22. IEEE Computer Society Press, Los Alamitos (1999) 20. Zhang, Y., Zhu, H., Greenwood, S., Huo, Q.: Quality modelling for web-based information systems. In: FTDCS 2001: Proceedings of the 8th IEEE Workshop on Future Trends of Distributed Computing Systems, Washington, DC, USA, p. 41. IEEE Computer Society, Los Alamitos (2001)
Designing Decentralized Service Compositions: Challenges and Solutions Ustun Yildiz and Claude Godart INRIA-LORIA BP 239 Campus Scientifique F-54506 Vandœuvre-l`es-Nancy, France {yildiz,godart}@loria.fr
Abstract. This paper reports a new approach to the decentralized execution of service compositions. The motivation of our work is to provide a systematic solution for the advanced configuration of P2P service interactions when they are involved in a composition. In order to do so, we consider a service composition as a centralized workflow specification and derive corresponding cooperating process in a such way that the dependencies of the centralized specification are implemented as P2P interactions between underlying services that execute the derived processes. More precisely, we present the most important issues of deriving operation and propose corresponding solutions that run counter to naive intuition.
1 Introduction Services oriented architectures are emerging as a new standardized way to design and implement business processes within and across enterprise boundaries. Basic XML-based service technologies (e.g. SOAP [1], WSDL [2]) provide simple but powerful means to model, discover and access software applications over the Web as analogous to wellknown concepts of traditional RPC-like and component-oriented computing. The term composition is used to describe the design and implementation issues of the combination of distinct services into a coherent whole in order to fulfill sophisticated tasks that cannot be fulfilled by a single service. For developing business processes based on service compositions, there are two essential initiatives around the banner of Orchestration and Choreography. Orchestration approach considers executable processes (e.g. WS-BPEL [3] specifications) that specify the dependencies of composed services through a single execution unit as analogous to centralized workflow management [4]. In contrast to orchestration, choreography approach (championed by WS-CDL [5]) is more decentralized in nature. It considers a global declarative composition that captures the interactions in which composed services can involve each other. Global choreographies are used to verify the consistency of composed services that work in a collaborative manner. As in application software modelling, different composition approaches are good for different things. Orchestration based compositions, for example, are quite useful for control and monitoring purposes as they govern compositions from a centralized site, but are less useful for data intensive compositions for scalability limitations. Choreography based compositions are useful in elucidating P2P interactions and parallelism, but are more J. Filipe and J. Cordeiro (Eds.): WEBIST 2007, LNBIP 8, pp. 60–71, 2008. c Springer-Verlag Berlin Heidelberg 2008
Designing Decentralized Service Compositions: Challenges and Solutions
61
difficult to establish and control. The focus of this paper is on the design and implementation of decentralized service compositions that bridge the gap between these two approaches. Intuitively speaking, establish peer-to-peer (P2P) interconnections between composed services is a major requirement in the management of business processes [6] [7]. However, the efficient configuration of service interactions is a difficult problem. With respect to two essential approaches, this difficulty has multiple reasons. Following the orchestration approach, composed services are coordinated from a single execution unit and they are isolated from each other. This prevents their P2P interactions. Following the choreography approach, services can establish P2P interactions. However, their interactions may not be adequate to those required by process designers or other contextual criteria. Decentralized workflow management has been always appealing in the conception and the implementation of sophisticated configurations between process participants [8] [9] [10]. The principal reason that prevents their wide spread use is their architectural assumptions that make them unreasonable in the web context. Following the approach of decentralized workflow management, we developed a new approach to the decentralized execution of service compositions. This approach is based on deriving cooperating processes from a workflow-like centralized specification of a composition. Derived processes are deployed on composed services to implement the dependencies of the centralizes specification as P2P interactions between underlying services. In this paper, we discuss the most important issues of deriving operation. First, we present the process specifications that the deriving operation is intuitive. Next, we consider sophisticated specifications that combine control, data and conversational aspects, most of which, there is no intuitive deriving operation. Furthermore, we present our approach for deriving processes from the latter. The remainder of this paper is organized as follows. Section 2 reviews the approach and the related work. Section 2 reviews the related work and presents the background of our approach. Section 3 introduces presents the formal concepts that we employed. Next, section 4 is the core core of contribution while the last section concludes.
2 Background Figure 1 illustrates the overview of our approach. The left-hand side describes a composition as a centralized process specification. In this kind of implementations, service are isolated from each other. A centralized workflow engine executes the composition specification. Consequently, the messages that correspond to the fulfillment of control and data dependencies of composition flow back and forth to the centralized workflow engine. The left-hand side of the same figure illustrates the execution of the same composition with the derived fragments. In contrast to existing solutions, this approach is reasonable and inexpensive in many ways. Intuitively, a composition specification that includes service interactions can be modeled by an organization and can be easily received and executed by another one even over a different execution engine. This is a secondary outcome of standardization efforts as they provide a common understanding for service compositions. However, the mobility of standard process models or process instances between different organizations and tools has not been handled by a decentralization point of view.
62
U. Yildiz and C. Godart
Service
Service
Service
Service
d6 a
d1
Centralized Workflow Engine
a
a
3
6
a
a
0
1
a
8
d0
9
a
a
10
11
a 2
d2
a 12
a 5
a
a
d4
17
Workflow Engine a
a
13
14
a
a
15
16
a
a
a
18
19
20
7
a
d5
4
d3
Service
Service Service
Service
Service Service
(a)
(b)
Fig. 1. Comparison of centralized and decentralized service compositions
The idea of decentralized execution setting is not new. Issues related to decentralized execution of workflows has been considered as very important in the relevant research literature a decade ago [11]. If we consider previous works that are not relevant to service oriented paradigms and technologies, much of them are conceived to prevent the overloading of a central coordinator component. Their basic proposition is to migrate process instances from an overloaded coordinator to a more available according to some metrics [12] or execute sub-partitions of a centralized specification on distinct coordinators [13]. Although efficient, these approaches do not consider P2P interactions of workflow elements. The implementation of decentralization is relatively new for service oriented applications. A simple but crucial observation is that some of the recent approaches deal with decentralization by using centralized services or additional software layers what can be a strong hypothesis in the Web context rather than reasoning decentralization with decentralized processes [9]. Some recent works provide several different process partitioning techniques similar to ours. For example, [14] focuses on the partitioning of variable assignment activities of a BPEL processes. But, they do not discuss how they deal with the elimination of dead paths (Dead Path Elimination [15]). The works presented in [16] [17] are similar, but limited to the decentralization of control flow. Our work is also relevant to traditional program partitioning for distributed computing. However there are important differences. First, the aim of a decentralized execution is not to parallelize the execution as the business logic of centralized composition must be preserved. From this point of view, decentralized service composition is much more relevant to P2P computing rather than program parallelization. The motivation of our approach highlights the problems of arriving at a definition of an accurate operation that derives cooperating processes to be executed by composed services (i.e. how a control or data dependency of the centralized specification can be reestablished and preserved in a decentralized composition?).
3 Formal Considerations This section describes the formal underpinnings of our approach that requires rigorous semantics on modeled processes. It is important to clarify what we do not intent to
Designing Decentralized Service Compositions: Challenges and Solutions
63
propose a new process language or extends an existing one. The presented concepts are applicable to a variety of process modeling languages and execution engines. A process can be represented as a message passing program modeled by a directed acyclic graph [15]. We assume that the centralized composition is modeled as a workflow. At this point, we assume that workflow models are structured processes [18]. A structured process considers control connectors that abstract the control flow of a process as properly nested split/join pairs that characterize control connectors (OR-split, OR-join, AND-split, AND-join). Definition 1 (Process). A process P is a tuple (A, C, Ec , Ed , D) where A is the set of activities, C is the set of control connectors, Ec ⊆ A × C × A is the set of control edges, Ed ⊆ (A × A) × D is the set of data edges and D is the set of data elements. A process P has a unique start activity, denoted with as , that has no predecessors, and has a unique final activity, denoted with af , with no successors. A control edge from an activity ai to another activity aj means that aj can not be executed until ai has reached a certain execution state (termination by default). A data edge exists between two activities if a data is defined by an activity and referenced by another without interleaving the definition of the same data. For example, a data received from a service si with the execution of an activity ai can be used as the input value of an activity aj that consists of an interaction with a service sj . In this case, there is a data edge between ai and aj . It should be noted that our process modeling does not consider control edges between control connectors. We associate source, target: Ec ∪ Ed → A functions return the target and source activities of control and data edges. With respect to control flow we can also define a partial order, denoted by <, over A, with < ⊆ A × A such that ai < aj means there is a control path from ai to aj . A service oriented process consists of synchronous/asynchronous interactions with services and locally executed activities such as variable assignments. We can define a function ser: A → S that returns a service interacted with an activity. As the dynamic service binding is supported, the return value of function ser can be a real service instance if there is already a bound service and if not it can be another value that identifies the service that is supposed to interacted with that activity. Thus, we can define a process activity as below. Definition 2 (Activity). An activity a is a tuple (in, out, s, type) where in, out ∈ D that are input and output values of a, s ∈ S is the interacting service with S is the set of services, and type is the type of activity with ΣA is the set of activity types. The set of activity types can be defined as follows type ∈ {read, write, write/read, local} where read activities receive data from services, write activities send data to services, write/read activities consists of synchronous interactions that sends a data and waits the reply and finally, local refers to activities that do not consist of service interactions. During the execution of sophisticated processes, some services can be used with conversations. This means that different operations of the same service can be invoked several times with distinct activities. These activities characterize the conversation-based interactions with the same service. The set of conversational activities belonging to a service si is denoted by Asi . For each activity, with respect to its incoming and outgoing control and data edges, we can identify two sets that include
64
U. Yildiz and C. Godart
corresponding source and target activities pointed by these edges. The preset of an activity a is •a = {ai ∈ A|∃d ∈ Ed ∨ ∃c ∈ Ec , (source(d) = ai ∧ target(d) = a) ∧ (source(c) = ai ∧ target(c) = a)}. The postset of an activity a is a• = {ai ∈ A|∃d ∈ Ed P ∨ ∃c ∈ Ec , (source(d) = ai ∧ target(d) = a) ∧ (source(c) = ai ∧ target(c)=a)}.
4 Deriving Cooperating Processes Cooperating processes of a centralized specification can be derived by employing various data structures and algorithms. Our method associates the activities of the centralized specification to the processes of the the services that they refer to. So, the elements of Asi are included in the process Psi that is executed by si . In order to preserve the semantics of the centralized specification with P2P interactions, the activities dispatched in different cooperating processes must be wired. Figure 2 depicts a simple wiring example of two control dependent activities across their corresponding cooperating processes. Activities a1 and a2 consist of the respective invocations of s1 and s2 . In a decentralized composition, a1 and a2 are included respectively in Ps1 and Ps2 that s1 and s2 execute. When Ps1 executes a1 , it must inform Ps2 about the termination of a1 to allow it to execute a2 . Figure 2 illustrates wiring activities that respectively send and receive a corresponding control message. The same example can be considered for a data dependency that interconnects two activities on the same control path. However, this time the exchanged message would contain a process data and not a control messages.
a1:s1
a2:s2
P
a1:s1 Ps1
w as1
r as1
s2(message)
s2(message)
a2:s2
Ps2
Fig. 2. Wiring two activities of a centralized specification between two cooperating processes
4.1 Dealing with the Consistency of Derived Processes Although based on simple principles, the deriving operation can lead to the situations that are not adequate to the consistency of cooperating processes. Figure 3 illustrates an example where a data dependency that interconnects two activities can cause a blocking situation between corresponding cooperating processes. The same figure illustrates also how this situation can be dealt by propagating the state of activities at the source of data edges to processes that must be informed about the state of the latter. Actually,
Designing Decentralized Service Compositions: Challenges and Solutions
65
Fig. 3. Wiring two activities dependent by a data edge where the source activity may not be executed
this technique is an extension of the current Dead Path Elimination (DPE) procedure implemented in current workflow management systems. We extended it towards the decentralized execution in order to prevent blocking situations[19]. Next subsection overviews this procedure. Dead Path Elimination and Insuring Global Soundness of Cooperating Processes. The extension of dead path elimination requires the extension of formal dependencies. For example, which activities must be taken by DPE after the evaluation of transition conditions to outgoing paths from an OR-split point? This requires a set of dependencies between an OR-split activity and activities that are involved in DPE. We call this as a extended postset denoted ai •. Similarly, for an activity that can be taken by several DPE, the source activities that trigger DPE must be considered. We gather them within the extended preset denoted •ai . Figure 4 illustrates the extended postset activities of activity a1 . The process that executes a1 informs directly the corresponding processes of activities that are on the paths taken by DPE. In the same time, the status of activities that are on the paths taken by DPE are propagated to processes that expect their outcome. If a2 is not an OR-split activity and its outgoing control edges evaluates to true, its process sends true control messages to the processes of the control dependent postset activities. As analogous to the elements of an extended postset, the extended preset of an activity includes the OR-split points that DPE that takes it or the source of its incoming data edges can be triggered. Figure 5 illustrates the extended preset activities of activity a19 . 4.2 Putting the Deriving Operation into Practice Our method associates the activities of the centralized specification to the derived processes of the services that they refer to. So, the elements of Asi are included in the process Psi that is executed by service si . In order to preserve the semantics of the centralized specification with P2P interactions,the activities dispatched in different processes must be wired. The above examples illustrated the dependencies that must be
66
U. Yildiz and C. Godart
d6 d1 a3
a8
a9
a10
a11
a17
a6
d0 a0
a1
d4 a2
d2
a13
a14
a15
a16
a12
a18
a5
a19
a20
a19
a20
a7 a4
d5 d3
Fig. 4. The elements of the extended postset of a1
d6 d1 a3
a8
a9
a10
a11
a17
a6
d0 a0
a1
d4 a2
d2
a13
a14
a15
a16
a12
a18
a5 a7 a4
d5 d3
Fig. 5. The extended preset activities of a19
considered while activities are wired. Now, we detail the deriving of cooperating processes from the centralized specification and the extended dependencies that presented in the previous section. The wiring operation is considered separately for preset and postset dependencies. 4.3 Wiring with Postset Activities Here we describe how a process behaves after the execution of one of its activities to wire it with its dependent activities. The structure a i • describes a process fragment that a process executes following the execution of its activity ai . ai can be an activity that precedes a OR-split point or not. If so, it is the starting point of DPE. In our approach to decentralized execution, DPE is achieved with P2P interactions. The process that executes ai informs directly the corresponding processes of activities that are on the paths taken
Designing Decentralized Service Compositions: Challenges and Solutions
67
Algorithm 1. Wiring with postset activities Require: ai , Centralized Process Ensure: ai • 1: for all aj ∈ai,c • do 2: for all an ∈ A − {ai } s.t. (an < ai ) ∧ (aj < an ) do 3: for all ak ∈ •an,c do OR/false || 4: a ← aw i→n (ak (f )) j 5: end for OR/false || ← aOR/false 6: a
j
7: 8: 9: 10: 11: 12: 13: 14: 15: 16: 17:
j
OR/false empty(a ) j end for for all an ∈ A s.t. ∃ak ∈ •an,ws , aj < ak ∧ ak < ai do for all ak ∈ •an,ws s.t. (aj < ak ) ∧ (ak < ai ) do OR/false || a ← aw (a (f ))
i→n
j
k
end for end for for all an ∈ A s.t. ∃ak ∈ •an,ws , cc ∈ {OR-split}, (aj < ak ) ∧ (ak < ai ) ∧ (cc < ak ) ∧ (ai < cc) ∧ (ak < cc) do OR/true || a ← aw (a (t))
i→n
j
k
end for OR/false OR/true ai • ← (aw ) ⊕ (aw ) i→j (ai (f )) + aj i→j (ai (t)) + aj ||
18: end for 19: for all aj ∈ ai,st • ∪ai,wt • do ||
ai • ← aw 20: i→j (d = aj .edge) 21: end for +
22: ai • ← F ai
by DPE. In the same time, the status of activities that are on the paths taken by DPE are propagated to the processes that expect their outcome. If ai is not an OR-split activity and its outgoing control edges evaluates to true, its process sends “true” control messages to the processes of the control dependent extended postset activities. Algorithm 1 defines the part of the structuring operation of a i •. In the notations, ⊕, ||, + denote respectively exclusive, concurrent and sequential append operations to a process fragment. The algorithm operates on each activity of the centralized specification. It should be noted that a i • is executed following the execution of ai . If ai is skipped, because it is on a path taken by DPE, the activities of a i • are skipped in the same process. Informally, the algorithm can be described as follows: For each control dependent activity aj that follows ai , two fragments that correspond to “true” and “false” transiOR/false (line 2-8, and line tion conditions of a are derived. These are characterized as a j
j
OR/true OR/false 9-13) and a (line 14-16). a include activities that send “false” messages to j j processes that execute activities that are on the paths taken by DPE (line 4). Moreover,
68
U. Yildiz and C. Godart
activities that send “false” messages to processes that have incoming data edges from OR/false (line 11). the activities of DPE are included in a j
OR/true a includes activities that send “true” messages to processes that have incoming j data edges from the activities that are not taken by DPE (line 15). Note that, we take nested OR-split/OR-join fragments into account. These two fragments are structured sequentially after the corresponding “true” and “false” control messages that can be sent to aj . The latter are exclusively structured (line 17). The last step of a i • is sending the outputs of ai to the processes that expect them (line 20). An unique activity F ai characterizes an empty activity added at the end of a i •. It is wired with the empty start activities of other activities a∈Asi that follow ai with respect to their partial order. This operation is detailed in the next subsection. We explain the algorithm with examples taken from our previous papers [19][20].
Example: Activities a0 and a11 depicted in figure 4 consist of the invocations of different s1 ’s operations. In Ps1 , after the execution of a0 , one of the paths that go to a1 and a2 can evaluate to “false”. In Ps1 , a0 is followed by two concurrent branches that are followed by two exclusive branches that correspond “false” and “true” con trol messages to be sent to the processes that include a1 and a2 . We can denote a 0• w w w = [(aw 0→1 (a0 (t)) + 1) ⊕ (a0→1 (a0 (f )) + 2)] || [( a0→2 (a0 (t)) + 3) ⊕ (a0→2 (a0 (f )) + F 4)] + aw 0→4 (d1 ) + a0 . The activities that send control messages to the processes that execute the elements of a0,c • are followed either by activities that fulfill DPE or inform processes about the status of activities that are not on the paths taken by DPE. Thus, w w w w 1 = ∅, 2 = [aw 0→4 (a1 (f ))||a0→5 (a4 (f ))|| a0→7 (a1 (f ))||a0→9 (a5 (f )) ||a0→9 (a8 (f ))] + w w w w (a (f ))], 3 = [a (a (t))], 4 = [a (a (f )) || a (a (f ))||a [aw 0→16 5 0→10 2 0→6 2 0→3 2 0→10 (a3 (f )) w w F || aw (a (f ))] + [a (a (f )) || a (a (f ))]. The activity a is connected to 0 0→10 6 0→10 2 0→16 3 S a11 . a11 is the last element in Ps1 . If there were other ai ∈ As1 that a0
Designing Decentralized Service Compositions: Challenges and Solutions
69
Algorithm 2. Wiring with preset activities Require: ai , Centralized Process Ensure: •ai +S
1: •ai ← ai 2: for all aj ∈•ai,c do 3: for all ak ∈ A s.t. ∃cc ∈ {OR-split}, c ∈ Ec , c = (ak , cc) ∧ ai < cc} do OR ⊕ 4: •a ← ar (aj (f ))
k→i
i
5: 6:
end for || OR •ai ← arj→i (aj (t)) ⊕ •a i
OR 7: empty(•a i ) 8: end for 9: for all aj ∈ •ai,ws do 10: for all am ∈ A s.t. ∃cc ∈ {OR-split}, c ∈ Ec , c = (am , cc) ∧ (cc < aj ) ∧ (aj < cc)} do ⊕ r aOR 11: j ← am→i (aj (f )) 12: end for 13: for all an ∈ A s.t. ∃cc, cc ∈ {OR-split}, c ∈ Ec , c = (an , cc) ∧ (cc < aj ) ∧ (aj < cc) ∧ (cc < cc ) ∧ (cc < aj )} do
14: 15: 16:
r r OR aOR j ← aj ⊕ (an→i (aj (t)) + aj→i (d = aj .edge)) end for || OR a+ i ← •aj OR empty(•a ) +
17: 18: end for
j
+ − 19: •ai ← (a+ i + ai ) ⊕ (ai ) 20: for all aj ∈ •ai,ss do +
||
•ai ← arj→i (d = aj .edge) 21: 22: end for
The algorithm can be explained as follows: If ai is on paths that can be taken by DPE, the processes that execute activities that precede OR-split points can send the messages that contain “false” message for all elements of •ai,c (line 2-8). If the immediate precedent activity is executed, the “true” message is received only from the process that executes it (line 6). The activities that collect status of control dependent preset activities are structured exclusively as there is a single process that can send the corresponding message (line 4 and line 6). According to the content of control mes− sages, ai can be executed or skipped. a+ i and ai characterize the beginnings of two exclusive branches that respectively execute or skip ai (line 19). a+ i is followed with + + the execution of fragment a . a includes activities that collect “false” or “true” mesi
i
sages that can come from the processes that can start DPE that the activities of •ai,ws are on (line 9-18). The activities that collect “true” messages are followed by activities i is the collection of data that are sent that read the sent data (line 14). The last step of •a by processes that execute the elements of •ai,ss (line 20-22). It should be noted that the latter do not require the execution of activities that test the status of source activities. a− i that corresponds to the start of the exclusive branch that is executed if ai is taken
70
U. Yildiz and C. Godart
by DPE, is interconnected to F ai . Consequently, this operation enables the following activities of the same process with respect to their partial order. Example: Suppose that a4 and a16 depicted in figure 5 consist of the invocations of different s2 ’s operations. In Ps2 , a4 is preceded by two OR-split points which means that its path can evaluate to “false” at these two different points. Consequently, the processes that execute a0 or a1 can send “false” control message for the control dependent preset activities of a4 . If the path of a4 is not taken by DPE, the process of a1 sends “true” control message to Ps2 . After the reception of control messages, a4 can be executed or skipped. If it is executed, its inputs must be collected from processes that execute source activities. Thus, a+ 4 is followed with the execution of an activity that receives the data sent by the process that executes a0 . As •a4,ws is empty, there is no need to test the source of incoming edges. We can denote • a4 − r = S a4 + [ar0→4 (a1 (f )) ⊕ ar1→4 (a1 (f )) ⊕ ar1→4 (a1 (t))] + [(a+ + a (d )) ⊕ a 1 04 4 4 ]. S The activity a− is interconnected to a . For a , there are four preset activities 16 16 4 (a3 , a5 , a10 , a11 ) that provide inputs to a16 . Two of them may not be executed (a3 , a5 ). Consequently, the “false” status of these activities can be sent to Ps2 by the processes that can start DPE. The “true” status message can be sent only by the process that executes the first OR-split point that precedes the source activity of data dependence. Activities a10 and a11 are strong source activities. Consequently, there is no necessity to test their status. The activities that collect d4 and d5 are executed beS r fore executing a16 . We can denote •a 16 as follows •a 16 = a16 + a15→16 (a15 (t)) + r r r r [(a0→16 (a3 (f )) ⊕ a2→16 (a3 (f )) ⊕ (a2→16 (a3 (t)) + a3→16 (d3 )))||(ar0→16 (a5 (f )) ⊕ ar1→16 (a5 (f )) ⊕ (ar1→16 (a5 (t)) + ar5→16 (d2 )))] + [ar10→16 (d5 )||ar11→16 (d4 )].
5 Conclusions This paper has taken on the challenge of designing a technique for translating a centralized composition specification to a set of dynamically deployable cooperating processes. The proposed approach aims to allow orchestrated services to have P2P interactions through the processes that they execute. When coupled with an efficient process transformation technique, we believe that the common process representation brought forward by standards can deal with architectural concerns of decentralized execution settings without making unreasonable assumptions about composed services. More precisely, we presented the most important issues of deriving operation and propose corresponding solutions that run counter to naive intuition. Our contribution stands between orchestration and choreography initiatives that govern the implementation of service oriented applications. We demonstrated how an orchestration description can be effective with a decentralized execution as analogous to choreography that requires much more sophisticated design and run-time reasoning.
References 1. Box, D., Ehnebuske, D., Kakivaya, G., Layman, A., Mendelsohn, N., Nielsen, H.: Simple Object Access Protocol (SOAP) 1.1 (2000), http://www.w3.org/TR/SOAP 2. Christensen, E., Curbera, F., Meredith, G., Weerawarana, S.: Web Services Description Language (WSDL) 1.1. W3C. 1.1 edn. (2001), http://www.w3c.org/TR/wsdl
Designing Decentralized Service Compositions: Challenges and Solutions
71
3. IBM, BEA Systems, Microsoft, SAP AG, Siebel Systems: Business Process Execution Language for Web Services, version 1.1 (updated 01 February 2005) (2005), http:// www-128.ibm.com/developerworks/library/specification/ws-bpel/ 4. Georgakopoulos, D., Hornick, M.F., Sheth, A.P.: An overview of workflow management: From process modeling to workflow automation infrastructure. Distributed and Parallel Databases 3, 119–153 (1995) 5. Kavantzas, N., et al.: Web Services Choreography Description Language Version 1.0 (2005) 6. Yildiz, U., Godart, C.: Enhancing secured service interoperability with decentralized orchestration. In: Proceedings of the 23rd International Conference on Data Engineering Workshops, ICDE Workshops, pp. 725–733. IEEE Computer Society Press, Los Alamitos (2007) 7. Yildiz, U., Godart, C.: Information flow control with decentralized service compositions. In: Proceedings of the 5th IEEE International Conference on Web Services, ICWS, pp. 7–19. IEEE Computer Society Press, Los Alamitos (2007) 8. Alonso, G., Kamath, M., Agrawal, D., Abbadi, A.E., Gunthor, R., Mohan, C.: Exotica/ fmqm: A persistent messagebased architecture for distributed workflow management. In: IFIP Working Conference on Information System Development for Decentralised Organisations, Trondheim (1995) 9. Schuler, C., Turker, C., Schek, H.J., Weber, R., Schuldt, H.: Peer-to-peer execution of (transactional) processes. International Journal of Cooperative Information Systems 14, 377–405 (2005) 10. Atluri, V., Chun, S.A., Mazzoleni, P.: A chinese wall security model for decentralized workflow systems. In: ACM Conference on Computer and Communications Security, pp. 48–57 (2001) 11. Sheth, A., Kochut, K.: Workflow applications to research agenda: Scalable and dynamic work coordination and collaboration systems. In: NATO Advanced Study Institute on Workflow Management Systems and Interoperability, Istanbul, T¨urkiye (1997) 12. Bauer, T., Reichert, M., Dadam, P.: Intra-subnet load balancing in distributed workflow management systems. Int. J. Cooperative Inf. Syst. 12, 295–324 (2003) 13. Wodtke, D., Weißenfels, J., Weikum, G., Dittrich, A.K., Muth, P.: The mentor workbench for enterprise-wide workflow management. In: SIGMOD Conference, pp. 576–579 (1997) 14. Chafle, G.B., Chandra, S., Mann, V., Nanda, M.G.: Decentralized orchestration of composite web services. In: WWW Alt. 2004, pp. 134–143 (2004) 15. Leymann, F., Roller, D.: Production Workflow - Concepts and Techniques. PTR Prentice Hall, Englewood Cliffs (2000) 16. Baresi, L., Maurino, A., Modafferi, S.: Workflow partitioning in mobile information systems. In: MOBIS, pp. 93–106 (2004) 17. Sadiq, W., Sadiq, S., Schulz, K.: Model driven distribution of collaborative business processes. In: IEEE International Conference on Services Computing, SCC (2006) 18. Kiepuszewski, B., ter Hofstede, A.H.M., Bussler, C.: On structured workflow modelling. In: Wangler, B., Bergman, L.D. (eds.) CAiSE 2000. LNCS, vol. 1789, pp. 431–445. Springer, Heidelberg (2000) 19. Yildiz, U.: On Dead Path Elimination in Decentralized Process Executions (Research Report RR-6131, INRIA (February 2007), http://hal.inria.fr/inria-00132928 20. Yildiz, U., Godart, C.: Centralized versus decentralized conversation-based orchestrations. In: CEC/EEE, pp. 289–296 (2007)
Grid Infrastructure Architecture: A Modular Approach from CoreGRID Augusto Ciuffoletti1 , Antonio Congiusta2 , Gracjan Jankowski3 , Michał Jankowski3 , Ondrej Krajiˇcek4 , and Norbert Meyer3 1
3
INFN/CNAF, Bologna, Italy [email protected] 2 DEIS, Universit`a della Calabria, Italy [email protected] Computer and Automation Research Institute, Poznan, Hungary {gracjan,jankowsk,norbert}@man.poznan.pl 4 Masaryk University, Czech Republic, Brno [email protected]
Abstract. The European Union CoreGRID project aims at encouraging collaboration among european research institutes. One target of such project is the design of an innovative Grid Infrastructure architecture, specifically addressing two challenging aspects of such entity: scalability and security. This paper outlines the results of such activity, ideally extending the content of the official deliverable document. Keywords: Grid Computing, Grid Information Service, Workflow Management, Checkpointing, Network Monitoring.
1 Introduction According to [8], a Grid is a complex architecture consisting of a collection of resources, which are made available at user level through a number of services. Such definition opens the way to a number of functional components, whose definition is of paramount importance for the design of a Grid: their semantics anticipate the capability of a Grid to make an efficient use of the resources it contains, to offer differentiated levels of quality of service, and, in essence, to meet user needs. Given the complexity and importance of such infrastructure, its design should address modularity as a primary feature: services provided by the Grid infrastructure should be precisely defined as for their interface and semantics, and form an integrated architecture which is a framework for their implementation. Modularity makes viable the independent evolution of each component, and allows the customization of the overall infrastructure. In order to guarantee interoperability among components, standard interfaces are not sufficient. In fact, the capabilities of a certain functional component should be well understood, and agreed in the community that develops other interoperating services: typical requirements address resource access, workflow management, and security. Such semantics should be compatible with the expected needs of the user, be it a human or a Grid-aware application. J. Filipe and J. Cordeiro (Eds.): WEBIST 2007, LNBIP 8, pp. 72–84, 2008. c Springer-Verlag Berlin Heidelberg 2008
Grid Infrastructure Architecture: A Modular Approach from CoreGRID Streams
Functional components
Descriptors
73
GIS
Workflow Analyzer
Checkpoint Manager
− Job descr. − Session descr. − − − − −
Ckpt image descr. Ckpt provider descr. Session descr. Workflow descr. Job descr.
− Ckpt image desc. − Ckpt provider descr.
− Job descr.
User/Account Management
− User descr. − Environment descr.
− Job descr.
Resource Monitoring
Grid Information Service
User Interface
− Workflow descr. − Session descr.
− Resource descr. − Session descr.
− Session descr.
Fig. 1. Integration between the functional components of our framework. Each component is a distributed entity that contributes to resource management exchanging descriptors with other components. Persistent information flows are encapsulated into streams, represented by session descriptors).
In addition, past experiences [12] prove that there is a tradeoff between portability and reuse of legacy tools: when functionalities that were not designed for integration are included into an existing project, the whole project tends to inherit all portability problems of the legacy parts. A plugin oriented approach does not solve the problem, but tends to complicate the design, and may even restrict portability. Taking into account such problems, we indicate a wrapper oriented approach: legacy tools are not directly included in the design, but accessible through interfaces that comply with portability requirements of the hosting environment. The agent that implements such functionality (the “wrapper”) is in charge of publishing portability issues that characterize the specific resource. One key issue in the design of a Grid environment is the technology used to support the Grid Information System (GIS). It is more and more evident that a unique technology (for instance, a relational database) cannot satisfy all needs, and may exhibit real scalability limits in case of take off of the Grid technology [3]. Here we propose a differentiated strategy for such vital component, splitting its functionality into a directory service, and a streaming support. The monitoring infrastructure provides input to the GIS: we describe such infrastructure decomposed into resource and middleware monitoring, workflow monitoring and network monitoring.
74
A. Ciuffoletti et al.
Another key aspect of a Grid infrastructure is job submission. According to the GGF guidelines in [13], we consider a unique component that performs batch submissions, scheduling and local queuing, workload monitoring and control. However, such component needs support for checkpointing and accounting, two activities that appear to require capabilities that need to be addressed specifically. We introduce two components that implement such functionalities. The resulting Grid infrastructure should address both the need of e-science applications, mostly oriented to storage and computation intensive applications with moderate scalability, and emerging industrial applications, where the demand is variegated and includes the management of a large number of small jobs: in this perspective, flexibility is mandatory to allow customization. Since we want to follow a clean design strategy, we address interoperation and integration issues since the early steps, using the GIS as a backbone. As a consequence, the adoption of a programming style and tools that support polymorphism is mandatory: the “wrapper oriented” approach indicated above helps on this way. In Section 2 we indentify the functional components, and in section 3 we consider a GIS which provides an integration backbone. In figure 1 we depict a schematic view of our proposal.
2 Functional Components of a Framework Architecture The focus of a Grid infrastructure is on resource management: the goal is to compose the operation of basic services into higher level tasks. To this purpose, the Grid infrastructure accepts and processes task descriptions that articulate a stepwise composition of computing activities. The use of appropriate basic services, whose availability is constantly monitored by a Resource Monitoring component, is scheduled after unfolding the dependencies between atomic computational tasks. Resource scheduling extends not only in the name space, to determine which resource is to be used, but also in time, describing when a certain resource will be busy on a certain task. The operation of assembling resources in order to perform a complex task is associated to the Workflow Analyzer component, whose role is to accept the operational description of a complex task, to manage, and to monitor its unfolding. The unfolding of a workflow must be sufficiently flexible, in order to cope with unanticipated events that may affect resources, either improving or degrading their performance. The appropriate way to cope with such events is the logistic re-organization of workflow execution, which usually entails the displacement of stateful computations, by re-instantiating services whose state corresponds to an intermediate computational step. Two basic functionalities are offered: the registration of a snapshot of an intermediate state of a service, and the re-instantiation of the same service with the given intermediate state. All resources in a workflow participate to such reorganization, and the resulting workflow execution must be consistent with the expected semantics. The Checkpoint Manager component is in charge of supporting the logistic re-organization of a workflow, preserving the relevant state information of a component services in preparation for the reconfiguration of the supporting low level services. Specific checkpointing indications are inserted in the operational description provided to the Workflow Analyzer.
Grid Infrastructure Architecture: A Modular Approach from CoreGRID
75
Since resources are similar to goods, their sharing must be controlled accordingly, taking into account property and commercial value. In that sense, the Grid infrastructure provides identities to Grid users, and defines service semantics according to the identity of the user, thus enforcing individual property. Using the same tools, the usage of a certain service is quantified, and a commercial value associated with it. The User and Account Management component is appointed with such aspects. The whole Grid infrastructure hinges upon the Grid Information System (GIS), which supports the integration between the parts of this distributed entity. From an abstract point of view, the content of the Grid Information System represents the state of the Grid, and is therefore dynamic. However, while some data remains constant for long periods of time, other are updated frequently, for instance when such information represents the residual (or preemptable) share of a resource. The activity of a component is pervasive, and many distinct agents contribute to its implementation: for instance, each site can provide a Workflow Analyzer agent in charge of accepting user requests. Such approach fits naturally with security requirements, which are based on mutual identification among agents. Here we give a summary of the functionalities each component offers, and we outline their internal structures: we use as a reference the work of the partners of the CoreGRID Institute on Grid Information, Resource and Workflow Monitoring. 2.1 Workflow Analyzer The Workflow Analyzer cares about workflows management under several aspects such as mapping, scheduling, and orchestration of workflow tasks against the available, dynamic Grid resources. To such purpose, it has close interaction with the Grid Information System in order to discover and allocate appropriate resources. But, at the same time, it is also a source of information coming from the monitoring of the workflows being executed: most of such information is reused by the Workflow Analyzer itself for adjusting the ongoing executions. A Grid workflow can be specified at different levels of abstraction: in [6] abstract workflows and concrete workflows are distinguished, the difference being whether resources are specified through logical files and logical component names or through specific executables and fully qualified resources or services. According to this approach the workflow definition is decoupled from the underlying Grid configuration. For this reason, the mapping phase (also referred to as matchmaking) is particularly important for selecting the most suitable resources that better satisfy the constraints and requirements specified in the abstract workflow, also with regard to quality of service and workload. The mapping process produces a concrete workflow that is suitable to be executed by a workflow engine, providing for scheduling and execution management capabilities. It is worth observing that, in case of dynamic scheduling, it is possible to re-invoke the mapping process at runtime, in order to modify the concrete workflow instance as a result of relevant events modifying the status of candidate resources. However, it is important to instrument job descriptions before actual execution, in order to ensure that the workflow execution is suitably checkpointed: succinct requirements about workflow recoverability are in the Workflow description provided by the user.
76
A. Ciuffoletti et al.
When the workflow enters the running state, the Workflow Analyzer monitors its advancement, and takes appropriate actions in response to relevant events. During the workflow execution, monitoring is essentially related to the observation of the workflow status. In particular, information about the execution of each single job included in the overall workflow is reported by the monitoring system. Typical information is constituted by services execution status, failures, progress, performance metrics, etc. In case of failure, the workflow execution service itself tries to recover the execution, for example by reassigning the work to a different host in case of host failure. To implement fault tolerance on a more refined extent, it is necessary whenever possible to trigger checkpoint recording, and drive the restart of one or more jobs from the last available checkpoint. The decision whether to checkpoint or restart a workflow is made on the basis of information from Resource monitoring. 2.2 Checkpointing The Checkpointing component is built around the idea of Grid Checkpointing Architecture [9,10], a novel concept that defines Grid embedded agents and associated design patters that allows the integration of a variety of existing and future low-level checkpointing packages. The emphasis has been put to make the GCA able to be integrated with other components and especially with the upper layer management services, for instance the Grid Broker or the Workflow Analizer. The main idea of the GCA boils down to provide a number of Checkpoint Translation Services (CTS) which are treated as drivers to the individual low-level checkpointing packages. The CTSes provide a uniform front-end to the upper layers of the GCA, and are customized to the underlying low-level checkpointers. When the CTS is deployed on a Computing Resource, the GCA has to be informed about its existence. To fulfill this requirement each CTS is registered at component named Grid Checkpointing Service (GCS), and the GCS further exports the information about the available CTSes and related properties to the GIS, so that the GIS becomes the mechanism that connects the GCA with the external Grid environment. Additionally, from the point of view of the Workflow Analyzer, the GCS is the gateway to which a checkpoint request has to be sent. When the GCS receives the request of taking the checkpoint of a given job, it forwards the request to the appropriate CTS. The GCS is able to find the adequate CTS using the information that the execute-job-wrapper registers when a checkpointable job is started. The execute-job wrapper is a special program provided together with an associated CTS. The component that is in charge of submitting the user’s job to the given Execution Manager replaces the actual job with the adequate execute-job wrapper and passes to the second the original job and the global identifier assigned to this job. Which execute-job wrapper is to be used depends on which CTS has been matched to the job’s requirements, according with GIS records. When the execute-job wrapper is started it appropriately configures the GCA environment and finally with help of exec() syscall replaces itself into the original job. When the Workflow Analyzer decides that a given job is to be recovered, then the job has to be resubmitted in an adequate way: one relevant issue is that the job can be
Grid Infrastructure Architecture: A Modular Approach from CoreGRID
77
resubmitted only to a Computing Element associated with a CTS of the same type that was used to checkpoint the job. When a proper Computing Element is found the job is resubmitted to it, but instead of resubmitting the original job itself the recovery-job-wrapper is resubmitted. The original job, as well as the identifier of the checkpoint that is to be used in the recovery process, are passed to the recovery-job-wrapper as the arguments. The recovery-job wrapper is the counterpart of the execute-job wrapper used for the recovery activity. The recovery-job wrapper starts fetching the image, and the subsequent actions are similar to those performed by the execute-job wrapper. As a last step, the recovery-job wrapper calls the appropriate low-level checkpointing package to recover the job using the image indicated by the calling Workflow Analyzer. The GCA shares the motivations of the Grid Checkpoint and Recovery working group of the GGF [15], which is to include the checkpointing technology into the Grid environment. However, the GCA focuses mainly on legacy checkpointing packages and, notably those that are not Grid-aware, while the GridCPR “is defining a user-level API and associated layer of services that will permit checkpointed jobs to be recovered and continued on the same or on remote Grid resources”. Therefore, while GridCPR works on a specification that future Grid applications will have to adhere to in order to make them checkpointable, our effort is towards the integration of existing products into a complex framework. 2.3 User and Account Management The User and Account Management component [7] offers a controlled, secure access to grid resources, complemented with the possibility of gathering data from resource providers in order to log user activity for accounting and auditing purposes. These objectives are realized introducing authorization, ensuring an appropriate level of job isolation and processing logging data. A virtual environment encapsulates jobs of a given user and grants a limited set of privileges. Job activity is bound to a user identity, which is qualified as a member of an organization. The User and Account Management component is a pluggable framework, that allows combining different authorization methods (e.g. gridmap file, banned user list, VO membership based authorization) and different implementations of environments (virtual accounts, virtual machines, and sandboxes). The configuration of the framework is quite flexible and depends on detailed requirements which may vary between the resources, so the administrators may tune local authorization policy to the real needs and abilities. The internal architecture of an agent consists of 3 modules: an authorization module, a virtual environment module and a virtual workspace database. The authorization module performs authentication first (based on existing software, such as Globus GSI). The authorization is done by querying a configurable set of authorization plugins. The virtual environment module is responsible for creation, deletion and communication with virtual environments modeled as Stateful Resources. The module is also pluggable, so it is possible to use different implementations of VE. The database records operations on the virtual environments (time of creation and destruction, users mapped to the environment, etc.). These records together with the standard system logs and accounting data, provides complete information on user actions and resource usage.
78
A. Ciuffoletti et al.
2.4 Resource Monitoring The information on resources and accompanying middleware is provided by the Resource Monitor. Resource Monitor component collects data from various monitoring tools available on the grid. We do not presume any particular monitoring approach, since the current state of the art provides quite wide range of monitoring toolkits and approaches. It is however a difficult task to integrate and process monitoring information from various monitoring tools. Moreover, we cannot assume any scale of the resulting infrastructure thus scalability of the proposed solution, both in terms of amount of monitored resources and required processing throughput for monitoring data, must be emphasized. To achieve the desired level of scalability, with security and flexibility in mind, we propose the design of a Resource Monitor based on the C-GMA [11] monitoring architecture. C-GMA is a direct extension of the GMA [16] specification supported by the Open Grid Forum. The key feature supplied by the C-GMA is the introduction of several metadata layers associated with services, resources and monitoring data. The metadata may specify the data definition language used by the services, the non-functional properties and requirements imposed by the services and resources (such as security and QoS-related requirements) and others. The metadata are used in the matchmaking process implemented by the C-GMA architecture, which is essentially a reasoning on provided metadata about the compatibility of the services and data described by them. When the examined parties are considered compatible, the “proposal” is sent to them to initiate a potential communication. In this way, and with the introduction of various translation components, the C-GMA architecture enables the exchange of monitoring data between various monitoring services. The Resource Monitor service leverages this functionality by connecting to the C-GMA monitoring architecture and using translation services for various monitoring toolkits it collects the monitoring data and supplies them to the Grid Information System. Special attention is paid to Network Monitoring, since scalability issues appear as challenging. We have identified one basic agent, the Network Monitoring Element, which is responsible of implementing the Network Monitoring Service [5]. Network Monitoring Elements (NMEs) cooperate in order to implement the Network Monitoring component, using mainly lightweight passive monitoring techniques. The basic semantic object is Network Monitoring Session, which consists in the measurement of certain traffic characteristics between the Domains whose NMEs participate in the session. To improve the scalability of the Network Monitoring Service, the NMEs apply an overlay Domain partition to the network, thus decoupling the intra-domain network infrastructure (under control of the peripheral administration), from the inter-domain infrastructure (meant to be out of control of the peripheral administration). According to the overlay domain partitioning network, monitoring sessions are associated to Resources denoted as Network Elements (NE), corresponding to inter-domain traffic classes. The overlay domain partition is maintained in an internal distributed database, which allows the coordination among Network Monitoring Elements. The management of network monitoring sessions includes the control of periodic sessions, as configured by
Grid Infrastructure Architecture: A Modular Approach from CoreGRID
79
network administrators, and of on-demand sessions dynamically configured by applications, and uses a scalable peer to peer mechanism to diffuse updates.
3 Integration between Functional Components The central idea of the proposed architecture is to convey all the data through the Grid Information Service in order to have a standard interface across the different administrative sites and services (see [1,2] for a similar approach). One relevant feature of a data repository, and of the Grid Information System, is the volatility of its content. At one end we find “write once” data, that are not subject to update operations and have a relatively low volatility. At the other hand we find data that are frequently updated. The functionality associated to the Grid Information System is a mix of both: while certain data, like a Workflow description, fall in the “write-once” category, other kind of data, like resource usage statistics, fall into the category of data that are frequently updated: a solution that devises a common treatment for both kinds of data suffers of a number of inefficiencies, first the lack of scalability. Therefore our first step is to recognize the need of distinct solutions for persistent and for volatile data. One criteria to distinguish the two kinds of data is the length of the time interval during which the information remains unchanged, under normal conditions. Here we assume that a significant threshold is given by the typical job execution time: we consider as persistent those pieces of information that, under normal conditions, remain valid during the execution of a job. We call such informations descriptors: starting from the specifications of the components that compose our framework given in previous sections, we now classify the descriptors that are exchanged among them, and that collectively represent the persistent content of the Grid Information System. Workflow Descriptor. It is acquired from a user interface by the Workflow Analizer component. It has the function of indicating the stepwise organization of a Grid computation. It contains high level indications about the processing requested at each step, as well as dependencies among individual steps. It should be designed in order to hide all unnecessary details, for instance package names or versions, and focus on the functionality (for instance, “fast fourier transform”, or “MPEG4 compression”). During workflow execution, such structure is used by the Workflow Analizer component in order to monitor workflow execution. Job Descriptor. It is produced by the Workflow Analizer component, and fed to various other components: it is used by the Checkpointing component in order to prepare the execution environment with checkpointing facilities, and by the User and Account Management component in order to associate an appropriate environment to its execution. The Job description is used by the Workflow Analizer component in order to instruct resources about their activity, and during workflow execution, to monitor workflow advancement. Checkpoint Image Descriptor. It is produced by the Checkpointing component (in case of the GCA, this descriptor is produced by the CTS) upon recording a new
80
A. Ciuffoletti et al.
checkpoint. The descriptor contains the bookkeeping data regarding the newly created image. The data can be used by the Workflow Analizer in order to find the identifier of the image that is to be used in order to perform recovery and migration. The GCA itself, basing on the descriptor, is able to fetch the image to the node on which the given job is to be recovered. Checkpoint Provider Descriptor. It is produced by the Checkpointing component. The descriptor advertises the location of service that provides access and unified interface to a particular low-level checkpointing package. The Workflow Analizer uses such descriptior to find the node that provides the desired checkpointing package, as specified in job descriptor. Upon recovery, the descriptor allows finding the nodes offering the same package used for checkpointing. Session Descriptor. It is produced by a generic component, and supports the exchange of volatile data, as described below. User Descriptor. It is produced and used by the User and Account Management component. It contains a description of a user, like its name, institution, reachability, role, as well as security related data, like public keys. The Workflow Analysis component uses such data to enforce access restrictions when scheduling a Workflow. Environment Descriptor. It is produced and used by the User and Account Management component. It contains references to he descriptions of the resources associated to a given processing environment, as well as the access modes for such resources. This may correspond, for instance, to what is needed to run a specific kind of job, and to the identities of the users that are allowed to operate within such environment. The Workflow Analysis component uses such data in order to process a workflow description. Resource Descriptor. It represents usual resource descriptions, including storage, processing, network and network monitoring elements. The identification of a resource includes its network monitoring domain. The Workload Analyzer uses such descriptions in order to schedule job execution, and allocate checkpoint storage. The management of descriptors relies on a directory-like structure. Such structure cannot be concentrated in replicated servers, but distributed in the whole system based on local needs. Functional components that need to have access to such data should address a proxy, which makes available the requested information, or add/delete a descriptor. An LDAP directory provides a first approximation of such entity: however, descriptors are not organized hierarchically. A better alternative is an adaptive caching of those descriptors that are considered locally relevant: for instance, the descriptor of a monitoring session might be cached in a GIS proxy near the monitored resource. Descriptors are diffused in the system using a low footprint, low performance broadcast protocol, and cached wherever needed. The volatile data is represented by data that change during the execution of a job: a typical example is the workload of a computing element. Such data are produced by one of the components described in the previous section, and made available to a restricted number of other components. The storage into a globally accessible facility, included a distributed relational database, seems inappropriate since the information is usually transferred from a source to a limited number of destinations. The concept that is usually applied to solve such kind of problems is the multicast.
Grid Infrastructure Architecture: A Modular Approach from CoreGRID
81
A multicast facility appropriate for diffusing volatile data of a Grid Information System has many points in common with a Voice over IP infrastructure: the container of the communication is similar to a Session (as defined in the SIP protocol). In contrast with a typical VoIP application, the data trasfer within a session in mainly uni-directional and requires a low bandwidth with moderate real time requirements: we call streams the information flows associated to the trasport of volatile data within a Grid Information System. All of the components outlined in section 2 are able to initiate or accept a session with another component: security issues are coped with using the descriptors associated with the agents. E.g., a Resource will accept a call only from a Workflow analyzer that submitted a job. Here we outline some of the relevant streams: Resource usage Stream. It is originated by a resource, like a Storage Element, and summarizes the performance of the resource, as well as the available share of it. Typical callers are the Workflow Analyzer, either during the resource selection or the execution phase. Workflow Advancement Stream. It is originated by a Workflow Analyzer component, and reports the caller about the workflow advancement. Typical callers are user oriented interfaces. One characteristic of a session, that makes it not interchangeable with a directory service, is that the establishment of a session has a relevant cost, which is amortized only if the session persists for a significant time interval. For this reason we include sessions in the number of entities that have a descriptor recorded in the Grid Information Service. Such descriptor advertizes the existence of a given session: it is a task of the callee to create and make available an appropriate Session descriptor, as outlined above. Sessions can be activated on demand, or be permanently available: such option depends on the balance between the workload needed to activate a new session on demand, and of keeping it warm for connection. E.g., Network Monitoring sessions will be mostly activated on demand, while Storage usage statistics can be maintained permanently active.
4 Comparison with Other Works The architecture we propose takes into account the goals and achievements of a number of scientific, as well as industrial projects that accepted the challenges proposed by the design of an effective grid infrastructure. One outstanding project which is being developed to meet the requirements the scientific community is gLite: it is developed within the European EGEE project, the successor of DATAGRID. Its purpose is to capitalize tools and experience matured in the course of DATAGRID, in order to assemble a Grid infrastructure usable for high performance computation, first the LHC experiment on schedule for the next year. We consider gLite [12] as a precious source of experience about a real scale Grid environment. We considered as relevants the inclusion of a number of features that are not considered, or considered at an embrional level, in gLite. Namely, we introduce a specific component that takes into account job checkpointing, we adopt a more powerful
82
A. Ciuffoletti et al.
workflow description language (but gLite is working towards a DRMAA [13] compliant interface), we take into account the task of workflow monitoring under scalability requirements, also considering networking resources, we differentiate the functionality of the GIS into a high latency directory service, and a multicast real-time streaming service. Overall, with respect to gLite, we considered the need for a wide portability: although such problem is not overly relevant for the environment for which gLite has been developed, we considered it relevant in a broader scope. To improve portability we suggest the realization of an integrated framework for the whole infrastructure, hosting legacy components encapsulated in specific wrappers. With respect to implementations based on the DRMAA proposed standard [13] we consider the interactions between Resource Management and Checkpointing, since we observe that the resource management is the component in charge of instructing the resource about activities relevant to recovery and relocation of running jobs. Therefore we describe an interface between a component in charge of managing a transparent management of checkpoints, and another in charge of interpreting user requests. The N1GE by Sun [4] is considered as a relevant representative of the industrial effort towards the implementation of a Grid infrastructure. Such project recognises the problems arising from the adoption of a monolythic relational database, and adheres to the DRMAA standards as for job descriptions. In order to overcome scalability limits imposed by a monolythic databases, it adopts a more flexible commercial database, Berkeley DB [3]. In our proposal we identify the kind of services of interest for our infrastructure, and indicate complementary solutions, that cannot be assimilated to a relational database. This should improve scalability and resource usage. The focus of the GPE [14] prototype by Intel is to bridge users from non-Grid environments, and to provide an interface that will remain sufficiently stable in the future, shielding the user from the changes of a still evolving middleware technology. Therefore the focus is on the provision of a powerful interface that adapts to several kinds of users. In order to take advantage of legacy tools, like UNICORE [17], security issues are delegated to a specific component, the Security Gateway, that enfoces a secure access to sensitive resources. In our view this is a source of problems, since the presence of a bottleneck limits the performance of a system. Instead, we indicate a pervasive attention to security issues, in order to implement appropriate security issues inside each agent. We pay special attention to a Grid resource that is often overlooked: the network infrastructure. Such resource is difficult to represent and to monitor since, unlike other resources, its complexity grows with the square of system size. Yet this resource has a vital role in a distributed system as a whole, since its availability determines its performance, and directly reflects on jobs performance.
5 Conclusions CoreGRID is an European project whose primary goal is to foster collaboration among european organizations towards the definition of an advanced Grid architecture. One of the tasks that contributes to this achievement is targeted at the description of an Integrated Framework for Resource and Workflow Monitoring. In order to enforce integration since the early steps, the research and development activities from several research groups are included in the same container, with frequent and planned meetings.
Grid Infrastructure Architecture: A Modular Approach from CoreGRID
83
This paper presents an early result on this way, after two years from the beginning of the project. We have tried to understand the problems left opened by other similar initiatives, specifically aiming at scalability and security issues, and identified the actors inside our framework. The research groups have produced relevant results for each of them that are only summarized in this paper; instead, we focus on the integration among such actors, based on descriptors advertised in the Grid Information Service. Acknowledgements. This research work is carried out under the FP6 Network of Excellence CoreGRID funded by the European Commission (Contract IST-2002-004265).”
References 1. Aiftimiei, C., Andreozzi, S., Cuscela, G., Bortoli, N.D., Donvito, G., Fantinel, S., Fattibene, E., Misurelli, G., Pierro, A., Rubini, G., Tortone, G.: GridICE: Requirements, architecture and experience of a monitoring tool for grid systems. In: Proceedings of the International Conference on Computing in High Energy and Nuclear Physics (CHEP 2006), Mumbai India (2006) 2. Andreozzi, S., De Bortoli, N., Fantinel, S., Ghiselli, A., Rubini, G., Tortone, G., Vistoli, C.: GridICE: a monitoring service for Grid systems. Future Generation Computer Systems Journal 21(4), 559–571 (2005) 3. BerkeleyDB. Diverse needs, database choices. Technical report, Sleepycat Software Inc. 4. Bulhes, P.T., Byun, C., Castrapel, R., Hassaine, O.: N1 grid engine 6 – features and capabilities. Technical report, SUPerG (2004) 5. Ciuffoletti, A., Polychronakis, M.: Architecture of a network monitoring element. Technical Report TR-0033, CoreGRID (2006) 6. Deelman, E., Blythe, J., Gil, Y., Kesselman, C., Mehta, G., Vahi, K., Blackburn, K., Lazzarini, A., Arbree, A., Cavanaugh, R., Koranda, S.: Mapping abstract complex workflows onto grid environments. Journal of Grid Computing 1(1), 25–39 (2003) 7. Denemark, J., Jankowski, M., Matyska, L., Meyer, N., Ruda, M., Wolniewicz, P.: Usermanagement for virtual organizations. Technical Report TR-0012, CoreGRID (2005) 8. Foster, I., Kesselman, C., Nick, J., Tuecke, S.: The physiology of the grid: An open grid services architecture for distributed systems integration (2002) 9. Jankowski, G., Januszewski, R., Mikolajczak, R., Kovacs, J.: Grid checkpointing architecture - a revised proposal. Technical Report TR0036, CoreGRID - Network of Excellence (2006) 10. Jankowski, G., Kovacs, J., Meyer, N., Januszewski, R., Mikolajczak, R.: Towards Checkpointing Grid Architecture. In: Wyrzykowski, R., Dongarra, J., Meyer, N., Wa´sniewski, J. (eds.) PPAM 2005. LNCS, vol. 3911. Springer, Heidelberg (2006) 11. Krajicek, O., Ceccanti, A., Krenek, A., Matyska, L., Ruda, M.: Designing a distributed mediator for the C-GMA monitoring architecture. In: Proc. of the DAPSYS 2006 Conference, Innsbruck (2006) 12. Laure, E., Fisher, S., Frohner, A., Grandi, C., Kunszt, P., Krenek, A., Mulmo, O., Pacini, F., Prelz, F., White, J., Barroso, M., Buncic, P., Hemmer, F., Meglio, A.D., Edlund, A.: Programming the grid with glite. Technical Report EGEE-TR-2006-001, EGEE (2006) 13. Rajic, H., Brobst, R., Chan, W., Ferstl, F., Gardiner, J., Haas, A., Nitzberg, B., Tollefsrud, J.: Distributed resource management application API specification. Technical report, Global Grid Forum (2004), http://www.ggf.org/documents/GWD-R/GFD-R.022.pdf
84
A. Ciuffoletti et al.
14. Ratering, R.: Grid programming environment (GPE) concepts. Technical report, Intel Corporation (2005) 15. Stone, N., Simmel, D., Kielmann, T.: An architecture for grid checkpoint and recovery (gridcpr) services and a gridcpr application programming interface. Technical report, Global Grid Forum draft (2005) 16. Tierney, B., Aydt, R., Gunter, D., Smith, W., Swany, M., Taylor, V., Wolski, R.: A grid monitoring architecture. Technical Report GFD 1.7, Global Grid Forum (2002) 17. UNICORE, UNICORE plus final report. Technical report, BMBF Project UNICORE Plus (2003)
The Privacy Advocate: Assertion of Privacy by Personalised Contracts Michael Maaser, Steffen Ortmann, and Peter Langendörfer IHP GmbH, Im Technologiepark 25, 15236 Frankfurt (Oder), Germany {maaser,ortmann,langendoerfer}@ihp-microelectronics.com
Abstract. Privacy has been a hot topic in research for several years. A lot of different approaches to protect privacy have been proposed recently. Among these there are several tools for negotiation of privacy contracts. In this paper we present our privacy negotiation framework called “The Privacy Advocate”. It consists of three main parts: the policy evaluation unit, the signature unit and the preferences. In addition, our framework supports an interface for negotiation strategies, so that they are independent of the framework. The preferences can be expressed with a combination of P3P and APPEL. The tests we executed using a state of the art PDA clearly indicate that our framework can be used on mobile devices. The completion of a successful negotiation usually takes about 2 sec. including message transfer via an 802.11b wireless link. This involves multiple evaluations of proposals. Each of them is done in less than 250 msec. Keywords: Privacy, negotiation, P3P, privacy contract, user preferences, personalisation, mobile devices.
1 Introduction A lot of internet users are concerned about their privacy [1]. These concerns have led to the development of P3P [2] and APPEL [3] by the W3C. P3P provides means to service providers to express which data is gathered by the service provider as well as for which purpose etc. APPEL provides means to service users to express their preferences. The P3P approach is a pure opt in or opt out model [4, 5]. Thus, if the user preferences and the service provider policy do not fit to each other the only option of the user is not to use this service. The drawbacks of such a static approach are on both sides. On one hand at least some service users have to give up more privacy than what they would prefer or are excluded from using interesting services. On the other hand service providers have to balance their requirements against the user demands for privacy. A recent study [6] shows that a significant percentage of users is willing to provide additional data if the service provider offers incentives like free trial version of software, reduced prices for the service under negotiation etc. Thus, privacy negotiation tools can help to accommodate the needs of both parties, i.e. more data for service providers, better prices or better privacy for certain groups of users. J. Filipe and J. Cordeiro (Eds.): WEBIST 2007, LNBIP 8, pp. 85–97, 2008. © Springer-Verlag Berlin Heidelberg 2008
86
M. Maaser, S. Ortmann, and P. Langendörfer
Privacy negotiation tools should provide service providers and service users with means to express their own requirements. Please note, by the term negotiation we really mean a kind of bargaining about the content of the resulting privacy contract. Unlike other tools or protocols the requirements do not specify a list of alternatives which can be chosen from exclusively. Our approach describes requirements as negotiation intervals by stating their boundaries but support use of discrete values as well. We made both negotiation parties capable to offer counterproposals based on those intervals and former offers. Such intervals are especially useful for measurable data such as charges and position information. In this paper we present our privacy negotiation framework, named Privacy Advocate (PrivAd). PrivAd provides means for service providers as well as users to negotiate individual privacy contracts. Thereby PrivAd is fully interoperable with already existing P3P based infrastructure. Existing tools such as Privacy Bird [7] can effectively communicate with servers using PrivAd as well as PrivAd clients can process static P3P policies from regular web servers. This paper is structured as follows. Section 2 provides a short state of the art. The extensions of P3P that are needed to enable negotiations are briefly discussed in section 3. In the following section we present details of PrivAd and section 5 outlines its implementation. The paper concludes with a summary and an outlook on further research.
2 Related Work There are several privacy related tools that are based on P3P and APPEL specifications. AT&T’s Privacy Bird is a free plug-in for Microsoft Internet Explorer. It allows users to specify privacy preferences regarding how a website collects and stores data about them. When the user visits a website, the Privacy Bird analyzes the provided policy and indicates whether or not the policy matches the user’s preferences. The Microsoft Internet Explorer 6 and Netscape 7 embed a similar behaviour. They allow users to set some options regarding cookies and are capable of displaying privacy policies in human readable format. PAWS also relies on P3P and APPEL but allows discovery of privacy policies within ubiquitous environments [8]. The policy evaluation is not done on the mobile device. All these tools are a valuable step into the right direction, but they still lack means to individualize privacy policies. There are approaches that allow users to choose from a given number of alternatives presented by the service provider [9, 10]. Such behaviour does not characterize a real negotiation to us. These approaches rely on the fact that a non empty intersection of the user requirements and the policies offered by the service provider is given a priori. Such approaches are widely used in protocols such as SSL and TLS. In contrast to this we consider both negotiation opponents as equal and they both are enabled to offer proposals and counterproposals. [11] presents an approach for negotiating privacy policies using negotiation trees. This is similar to our approach but the prototype is not capable of automatic negotiation.
The Privacy Advocate: Assertion of Privacy by Personalised Contracts
87
3 Negotiation Extensions To enable negotiation scenarios the following documents are needed: Service and user preferences that describe the respective requirements, proposals and counterproposals from preferences and finally a mutually agreed privacy contract as the outcome of a successful negotiation. Preferences are secret policies of each party that are never exposed to the opposite side. The secret preferences specify boundaries that span a certain room to negotiate. These boundaries do not only define certain values or enumerations but can specify also single- or double-bounded intervals of real numbers or any other value. Surely unique requirements of users and service provider have to be considered too. In order to enable individualization a user has to state permissions and prohibitions whereas the service providers state requirements and facultative requests. This is done in the secret preferences of each party by respective tags that are added to the statements. Proposals or counterproposals contain complete envisioned contracts or parts of those. Further they may contain requests for deletion of certain parts. Such documents are exchanged alternately during the negotiation process. A privacy contract enumerates all data, purposes, recipients etc. whose release has been agreed on by both opponents during the negotiation. Privacy contracts must not contain any interval. All values have to be stated explicitly. Our vision of a privacy contract is mostly resembled by a P3P policy. Therefore we decided to enhance the P3P standard with different language features to enable negotiation possibilities. Since P3P does not support a description of intervals as needed for the preferences, we added tags to describe those intervals first. This is done by specifying boundaries with tags like and . Boundaries can be set for measurable data i.e. accuracy of location. For remuneration of service usage we integrated a special element to represent the amount of money to be paid for a service. Besides the necessary interval it contains unit and currency tags to enable respective conversion. Due to space limitation we omitted detailed description of all extensions. For further information please refer to [12]. 3.1 Requirements and Facultative Requests For service providers, the secret preference contains the data they want to collect and the purposes, retentions and recipients they want to use the data for. Above all, the concept allows dividing different classes of data items to accentuate the need for them. Thus the data items have to be classified into requirements and facultative requests. Requirements state all data needed to fulfil a service whereas facultative requests specify optional data items, i.e. useful or attractive information that are not essential for service usage. Statements in the secret preference of service provider that are not tagged as facultative requests are classified as requirements automatically. Logically proposals are never acceptable for the service providers until all requirements are fulfilled.
88
M. Maaser, S. Ortmann, and P. Langendörfer
3.2 Permissions and Prohibitions While requirements and facultative request at service provider preferences are selfexplanatory, the permissions and prohibitions in user preferences contain additional implications. Users are able to define statements in their preferences, which are explicitly prohibited or allowed. In this way users are capable of prohibiting single data or groups of data for certain purposes, recipients or retentions. It is also possible to prohibit data or groups of data for all purposes or all recipients etc. That is, prohibitions specify domains that are explicitly prohibited. Preferences that are solely composed of prohibitions, implicitly allow all non-stated data. Similarly, permissions explicitly allow data for certain purposes or recipients. If user preferences contain permissions only, all non-stated data are implicitly prohibited. Obviously, explicitly allowed or prohibited content of statements is more significant than comparable implicit ones. 3.3 Privacy Contracts Successful negotiations finish with a mutually agreed and signed policy, called privacy contract. Digital certificates analogous to those used by SSL are the basis for the used signatures. Since the privacy contract is an individual document regarding each pair of user and service provider it must be digitally signed by both parties. The P3P standard does not provide an opportunity to integrate digital signatures. Thus, to ensure backward compatibility, we use XML comments to integrate the digital signatures. This way we avoid misinterpreted policies by standard P3P applications. Our negotiation protocol is not capable to choose from a list of signature algorithms, analogue to SSL, yet. Thus, both negotiation parties have to use a predefined signature algorithm. Currently this is fixed to SHA/DSA.
4 Privacy Advocate The Privacy Advocate (PrivAd) is a framework that is capable to negotiate on the basis of P3P privacy policies including the introduced extensions. It is designed for both negotiation parties and may be used by service users as well as the service provider side. Hence we implemented a PrivAd client and a PrivAd server. The framework consists of a Policy Evaluation Unit, a Signature Unit, a Repository for preferences and signed policies, mechanisms for http communications and an interface for exchangeable Negotiation Strategies, see Figure 1. The graphical user interface of the PrivAd client allows the user to monitor and control the negotiation process and was implemented for demonstration purposes only. Surely an embedded version of PrivAd does not need such a graphical user interface and works unseen in background. The Signature Unit verifies signatures of incoming documents and signs outgoing ones. The Policy Evaluation Unit is the core of PrivAd and the negotiation concept. Every incoming proposal or policy has to be evaluated and compared with the preferences by this unit. The last component, the Negotiation Strategy, is responsible for calculating counterproposals. It is not a part of the framework. We intentionally made this part exchangeable to provide means of using alternative
The Privacy Advocate: Assertion of Privacy by Personalised Contracts
89
strategies. Exemplary we implemented two strategies described in Section 5.5, but the framework could be completed with any Negotiation Strategy. To create a negotiation enabled antagonist we embedded PrivAd into a Tomcat Web Server. To exchange negotiation messages between two negotiation parties or to collect a policy of a static device, the transport protocol http is used. Our Privacy Negotiation Protocol (PNP) used by PrivAd is a stop-and-go protocol in which each incoming policy triggers its evaluation and sending of an answer. Figure 1 displays the interaction of a PrivAd client with a PrivAd server or a regular P3P enabled web server respectively.
Fig. 1. Structure of the Privacy Advocate
4.1 Negotiation Process This section describes a single turn of the bargaining procedure, when a PrivAd client receives a policy as a proposal or counterproposal from a negotiation enabled service provider using PrivAd server. The Signature Unit verifies the signature of any received policy first. Incorrect signatures lead to stop the negotiation immediately. Policies that contain correct signatures are forwarded to the Policy Evaluation Unit that compares the content of the policy with the client’s preferences. The Policy Evaluation Unit returns a Boolean value that states whether or not the received policy matches the client’s preferences. In case of a match PrivAd has 2 options: 1.
2.
PrivAd signs the policy and sends it back to the service. If the incoming policy was already single signed both sides have a mutually signed contract now and may continue with fulfilling the service. PrivAd orders the Negotiation Strategy to create a counterproposal although the policy was acceptable. The Negotiation Strategy may create a counterproposal that contains more favourable terms and conditions for the client’s side which is then send back.
In case of a mismatch with own preferences, PrivAd has 3 options to response: 1. 2.
PrivAd cancels the negotiation because the policy was unacceptable. PrivAd orders the Negotiation Strategy to create a counterproposal that matches own preferences and returns it to the sender.
90
M. Maaser, S. Ortmann, and P. Langendörfer
3.
PrivAd cancels the negotiation because an integrated assessment function indicates that the negotiation process does not seem to advance and become successful.
Note that the Policy Evaluation Unit and the Negotiation Strategies are described in separate subsections later. The PrivAd server handles incoming proposals analogous. To provide more flexibility, PrivAd uses an independent Negotiation Strategy to create counterproposals. As seen above, PrivAd does not need a strategy to evaluate incoming documents but for appropriate responses only. It is possible that a complete bargaining needs many negotiation rounds. Therefore PrivAd is capable to store and adjust former proposals if necessary. Thus, counterproposals do not need to contain the complete policy. Only changes need to be submitted. If a negotiation party agrees to the proposal with all changes applied, a new contract proposal containing all involved statements is generated. That contract proposal is signed and sent to the opposite side. Finally the case to «negotiate» with regular P3P enabled web servers has to be considered, see section 5.1 for further details. PrivAd sends no document to such server but receives their static P3P privacy policy. This policy is evaluated by the Policy Evaluation Unit as usual. PrivAd client notifies the user about the acceptance of the policy via a message on the screen. Due to the static server no counterproposal is calculated in case of a mismatch with own privacy requirements. So, PrivAd provides the same functionality as existing P3P tools. In the case of especially tenacious strategies it may happen that the negotiation will never find an agreement, despite such agreement theoretically exists. These cases cannot be prevented by design. Thus PrivAd needs to provide an assessment function that allows to monitor progress of the negotiation as well as to calculate, whether a negotiation should be cancelled. The assessment function is independent of the strategies. Our current assessment function accounts the percentage of accepted statements to check the convergence of the negotiation process. If this percentage of the current negotiation round is less than the minimum of last five negotiation rounds, PrivAd cancels the current negotiation. In principle each user may define an own assessment function or adjust the number of negotiation rounds of the currently used function. 4.2 Policy Evaluation This section describes the procedure of checking a proposal against negotiation preferences. It is essential that every incoming document can be checked automatically. Therefore an algorithm and a data-model were developed, which determine the acceptance of a document in relation to own preferences. Surely these documents have to conform to P3P with or without the introduced extensions. As stated before, the two possible results of this check are the acceptance or denial of a proposal. For further explanations the following naming conventions are constituted. Statements of arrived proposals are called paragraphs and statements of preferences are called directives. The policy evaluation passes three different stages starting with the evaluation of individual elements of a statement up to the complete proposal.
The Privacy Advocate: Assertion of Privacy by Personalised Contracts
91
4.2.1 Elements of Statements The paragraphs and directives are interpreted as vectors of sets. These sets are purposes, recipients and data_group. Thus, elements of a statement are represented as follows. statement = ({}purposes,{}recipients,{}data_group) Depending on the use of the statement the sets may also be Ω. Ω specifies all possible members of a set, i.e. all purposes that are imaginable. Ω may be used in the user preferences to prohibit or allow data without referencing a certain purpose or recipient. Hence, it can be assumed that each element of a statement, which works as a paragraph, is neither empty nor Ω. Obviously empty sets or Ω are allowed to be used in directives only. Considering paragraphs and directives as vectors of sets, just these sets have to be compared. That is, we compare purposes from paragraphs with purposes from directives just etc. We call sets from directives with permissions permitted sets and sets from directives with prohibitions prohibited sets. The result of this comparison is one of the following logical values. • `not affected` Æ The compared sets are disjoint. • `definitely permitted` Æ The set of the paragraph is a subset of the permitted set. Hence it is allowed. • `possibly prohibited` Æ The intersection of the paragraph’s set and the prohibited set is not empty but the paragraph’s set contains elements that are not elements of the prohibited one. So final decision whether or not the paragraph can be accepted requires additional calculations, see section 4.2.2. • `possibly permitted` Æ The intersection of the paragraph’s set and the permitted set is not empty but the paragraph’s set contains elements that are not elements of the permitted one. So further processing is required analogous to `possibly prohibited `. • `definitely prohibited` Æ The set of the paragraph is a subset of the prohibited set. Thus it must be prohibited. Table 1. Logical values of sets in paragraphs to the respective sets in directives Directive is
permit
prohibition / {}+
§ ∩ D ≠ Ø, § ∩ D ≠ §, §∩D≠D
POSSIBLY PERMITTED
DEFINITELY PROHIBITED
POSSIBLY PROHIBITED
§∩D=Ø
NOT AFFECTED
NOT AFFECTED
NOT AFFECTED
§=D
DEFINITELY PERMITTED
DEFINITELY PROHIBITED
DEFINITELY PROHIBITED
§⊂D
DEFINITELY PERMITTED
DEFINITELY PROHIBITED
POSSIBLY PROHIBITED
§⊃D
POSSIBLY PERMITTED
DEFINITELY PROHIBITED
DEFINITELY PROHIBITED
D=Ω
DEFINITELY PERMITTED
DEFINITELY PROHIBITED
DEFINITELY PERMITTED
D=Ø
POSSIBLY PERMITTED
DEFINITELY PERMITTED
DEFINITELY PROHIBITED
Ø≠D≠Ω
prohibition / {}&
92
M. Maaser, S. Ortmann, and P. Langendörfer
Each element gets one such value assigned and is further processed in the next steps to finally determine the acceptance of the proposal. For easier notation the examined set from the directive is D and the respective set from the paragraph is §. According to this notation a lookup table, see Table 1, is used to determine logical values. Just see the headline to define the kind of directive and the first column to assign the set of the paragraph to determine the result. For determination of logical values it must be taken into account whether a directive is a prohibition or a permit. Since prohibitions forbid the combination of certain data for any of the stated purposes or recipients, we have to distinguish OR-sets {}+ and ANDsets {}&. While OR-sets prohibit all possible combinations of the contained elements, an AND-set only prohibits the given combination of the set. Thus, the OR-set is the more restrictive one. In permits this classification of sets is not necessary because every subset of them is also permitted. 4.2.2 Grading of Paragraphs After determining the logical values of sets in paragraphs, the next step of evaluation has to be done. The logical values of sets in a paragraph have to be concatenated to determine the logical value of the complete paragraph in relation to the directive. We use two additional lookup tables to define this concatenation operation. One lookup table is responsible for prohibiting directives and the other one for permits. Example: A user prohibits the combination of her data A and B for the purpose P by a prohibiting directive. A service requests data A for the purposes P and Q. According to Table 1 the requested data is `possibly prohibited` and the purposes are `definitely prohibited`. These logical values are concatenated resulting in the value `possibly prohibited`. Hence, the paragraph is acceptable because it requests not all of the prohibited data. A special focus lies on paragraphs, which got the logical values `possibly permitted` or `possibly prohibited`. All paragraphs which have been evaluated to those logical values have to be analyzed again. Please notice that both values are not equivalent. In the following we provide an example for `possibly prohibited` paragraphs. Suppose a directive with prohibited data {name, address, birth date}. The combination of this data could be misused to falsify an eBay account. Additionally assume a proposal containing two paragraphs. One paragraph requests the name and address, the other one requests the birth date. Both paragraphs are evaluated to `possibly prohibited` (refer Table 1) because the requested data_groups are only subsets of the prohibition {name, address, birth date}. So these individual paragraphs would still be acceptable, despite their combination is violating the directive. To avoid such cases, all `possibly prohibited` paragraphs are merged as long as possible in order to determine effectively, whether they are `definitely prohibited` or not. Thus, these two paragraphs have to be merged into one paragraph if possible. Two paragraphs that differ in one set only, can be merged to a single paragraph where the differing sets are conjoint. In our example the merged paragraph contains data {name, address, birth date} after finishing the merge operation and is checked again. Obviously this turns out to be `definitely prohibited`. Consequentially the logical values of both original paragraphs are changed to `definitely prohibited`.
The Privacy Advocate: Assertion of Privacy by Personalised Contracts
93
4.2.3 Aggregation of Acceptance of Proposal After the logical values for all paragraphs of a proposal are determined the results have to undergo another evaluation step. It is checked whether a certain paragraph was evaluated to `definitely prohibited` or `possibly permitted`. If such a paragraph exists in the proposal, the Policy Evaluation Unit signals the non-acceptance of the proposal. Otherwise it could be agreed to. In [13], the used data-model and the policy evaluation are defined in more detail. 4.3 Protocol Issues To exchange negotiation documents we used http as transport protocol. This protocol is widespread and ensures backwards compatibility with P3P. PrivAd uses the httppost method to pick up and send proposals or policies to negotiation enabled servers. If PrivAd detects no authorization to send an http-post request towards the negotiation opponent, a regular P3P enabled web server is assumed. Thus PrivAd will get the static policy, if available, by using the http-get method. That enables PrivAd to verify every existing static P3P policy. Also our PrivAd server responds with a standard policy at receiving an http-get request. That enables the server to answer requests of clients that are unable to negotiate such as most existing P3P tools. If an http-post request is received, the PrivAd server supposes a negotiation enabled client on the other side. Therefore PrivAd client sends an initial request to negotiation enabled server by http-post to initiate the negotiation process. This post request can be empty or contain a proposal.
5 Implementation 5.1 Mobile PrivAd Client The main restriction while implementing PrivAd client was to ensure that mobile devices can use it. Therefore the PrivAd client was implemented compatible to JDK 1.1.8. We use the JVM CrEme [14] to run PrivAd on PDA’s. The PrivAd client PDA version contains a graphical user interface to visualize the negotiation process on the PDA for plausibility checks and demonstration. In this GUI every transferred document and the preferences of the client are shown. The user has to choose a file with preferences and a URL as a target for negotiation. Pushing a button to run the next negotiation round enables the user to control the progress of the negotiation process. All policy evaluation and proposal generation is done by the PrivAd client. Note the code size takes about 150 Kbytes only. The privacy negotiation process was tested on a Toshiba Pocket PC e740 WLAN with a 400 MHz XScale processor and 64 MB RAM. For this test we used simple policies that were compiled with less than 4 statements on the PrivAd server and the mobile PrivAd client. For the air link we used 802.11b with a nominal data rate of 11 Mbit/sec. We measured results for accepted as well as for rejected policies. Comparing the results for different transmission speeds of the network shows clearly that the communication time dominates the time needed to complete the negotiation process. Using a wired connection the complete negotiation cycles were finished in
94
M. Maaser, S. Ortmann, and P. Langendörfer
less than 300 msec whereas it took about 2000 msec using the wireless connection. The pure calculation of proposals and counterproposals takes less than 250 msec on the mobile PrivAd client. 5.2 PrivAd Server The server side is implemented as a Java Servlet that runs in a Servlet container. For our reference application we used Apache Tomcat [15]. Similar to its client the PrivAd server is backward compatible to existing P3P tools. Therefore the server is able to behave like a regular P3P enabled web server, if it is necessary. On a first request the PrivAd server will respond with its initial policy. If it is an http-post request, the PrivAd server initiates the negotiation process with this client. By receiving a simple http-get request the server only replies with a default policy without any negotiation interests which is fully compatible to P3P standard. Hence the PrivAd server works interoperable with already existing tools like Privacy Bird, Internet Explorer or Netscape. Using http-post for policy exchange may inhibit options for invaders. This carries limited risk, because incoming messages are parsed and validated against the PrivAd schema. In worst case there will be an error accounted by parsing the element. Since all policies are interpreted as text and not as executables, the risk, that malicious code can be executed in system, is kept minimal. 5.3 Digital Signature All contracts have to be signed digitally. This will block man-in-the-middle-attacks, give proof of identity and offers more integrity. Incorrect signatures lead to cancellation of the negotiation process. In addition the Signature Unit signs an acceptable policy to generate a single signed contract proposal. Accepted contract proposals from the opponent are signed to generate a mutually signed contract. For digital signing we implemented the SHA/DSA algorithm in the Signature Unit. We rely on existing PKI as SSL/TLS also does. For future versions we intend to integrate selection means for different signature algorithms. 5.4 Negotiation Strategies Creation of a counterproposal depends on the Negotiation Strategy in PrivAd and its secret preferences. Simple Negotiation Strategies, for both negotiation parties are developed and integrated. For demonstration we implemented two strategies where both sides try to find a mutual contract by accommodating as fast as possible. To start we point out operation methods of a simple service provider strategy for PrivAd server. Consequently, the service provider creates and proposes an initial policy, which contains all requirements and facultative requests. Additionally, most profitable values from the intervals in the preferences are chosen in this policy to gather a maximum of information and money. On receiving a counterproposal it is applied to
The Privacy Advocate: Assertion of Privacy by Personalised Contracts
95
previous proposals to get a combined proposal and the provider strategy has to react in a more complex way: • The strategy acquires the logical values of all statements that the Policy Evaluation Unit has determined before. Logically statements that are neither explicitly nor implicitly prohibited do not require a counterproposal. • All acceptable statements in the proposal are mapped to respective statements in the preferences. • If all requirements are satisfied already, the strategy creates a new contract proposal from the combined proposal. All additionally acceptable statements are surely added to gain as many data as possible. The PrivAd server signs it and sends that new contract proposal back to the user. • If not all requirements are satisfied, a new contract proposal or counterproposal has to be created and proposed. Therefore all requirements are requested. • Unacceptable proposed statements are updated by new values matching the intervals in the preferences. Here the strategy may reduce the proposed charge for the provided service by 10 percent or similar. • If proposals instruct to discard statements that are requirements, the provider strategy proposes exactly these statements again since otherwise the service cannot be provided. However, the service provider never signs a policy, where not all minimum requirements from its preferences are included. • The created counterproposal is passed to PrivAd server that sends it back to the opponent. While service providers normally try to retrieve the highest possible amount of data, the users naturally negotiate in the opposite way. • The user gets an initial policy by querying the service provider at first time. • The PrivAd client checks the signature. Proposals are applied to previous ones if those exist. Then the acceptance of the combined proposal is evaluated. Thereby the Policy Evaluation Unit marks all statements in the proposal with logical values to determine prohibited ones. • If the proposed policy does not contain any prohibited statement this policy is signed by the Signature Unit and is returned. If this policy was a contract proposal, it is also signed by the PrivAd client and becomes a mutually signed contract. In that case both sides have a mutually signed contract now and may start using the service. Surely also an acceptable proposal could be counterproposed here. Thereby the strategy may cut the prizes for the service or adapt the proposed values to improved ones. • If it contains prohibited statements the strategy component generates a counterproposal. Prohibited paragraphs are the base for calculating the counterproposal. Below we display an operation breakdown on a single prohibited paragraph. As mentioned before, statements in preferences are called directives whereas statements in proposals are called paragraphs. o First, the prohibiting directives for this paragraph are searched in the client’s preferences. o Special focus is set on merged paragraphs. Certainly all prohibited paragraphs that were merged into one single paragraph have to be
96
M. Maaser, S. Ortmann, and P. Langendörfer
rejected in their isolated form as well. Since all paragraphs are numbered a rejection can be accomplished by returning an empty paragraph with that particular number. o Finally, the prohibited paragraph is intersected with the prohibiting directive. Here intersection means that all values of the paragraph are adjusted to match the intervals in the directive. The strategy may try to approximate the values of the counterproposal to the preferred boundary of the interval. o If the strategy has still not found an appropriate intersection, the prohibited paragraph has to be rejected without a replacement. o This process is repeated for every prohibited paragraph of the unaccepted policy. • Last step is to create a counterproposal containing all changed and newly generated paragraphs. All paragraphs, that were not listed or explicitly rejected in counterproposal are supposed to be accepted. • The created counterproposal is passed to the PrivAd client that sends it to the opponent.
6 Summary and Outlook This article determined the situation in stating and contracting privacy in the Internet. Lacking negotiation capability has been identified as a shortcoming. A privacy negotiation framework that allows mobile users and service providers to negotiate about data and its potential recipients, purpose etc. was presented. This framework PrivAd is downwards compatible with P3P and enables negotiation scenarios between PrivAd clients and static servers as well as between known P3P tools and PrivAd servers. The process of evaluating a privacy policy and the calculation of counterproposals was explained. Tests, done with a state of the art mobile device, prove that our mobile PrivAd client is functional. Successful negotiations, finishing with a mutually signed privacy contract, take about 2 seconds via wireless link and about 0.3 seconds via a wired link, including all message transfer. In the future the two negotiation parties and their types of requirements shall be merged, in order to create tools which can perform both sides of negotiation at a time. This will enable negotiations between enterprises and in the peer-to-peer domain.
References 1. Cranor, L.F.: Beyond Concern: Under-standing Net Users’ Attitudes About Online Privacy. In: Vogelsang, I., Compaine, B.M. (eds.) The Internet Upheaval: Raising Questions, Seeking Answers in Communications Policy, pp. 47–70. The MIT Press, Cambridge (2000) 2. Cranor, L.F., Langheinrich, M., Marchiori, M.: The Platform for Privacy Preferences 1.1 (P3P1.1) Specification (2006), http://www.w3.org/TR/P3P11/ 3. Cranor, L.F., Langheinrich, M.: A P3P Preference Exchange Language 1.0 (2002), http://www.w3.org/TR/P3P-preferences/
The Privacy Advocate: Assertion of Privacy by Personalised Contracts
97
4. Thibadeau, R.: A Critique of P3P: Privacy on the Web. The eCommerce Institute, School of Computer Science, Carnegie Mellon University (2000) 5. Bennicke, M., Langendörfer, P.: Towards Automatic Negotiation of Privacy Contracts for Internet Services. In: Proceedings of 11th IEEE Conference on Computer Networks. IEEE Society Press, Los Alamitos (2003) 6. Druecke, R.: Attitudes to Privacy at using mobile phones (in german), Technical Report, Mobile Internet Business, Nr. 3 (2006) ISSN 1861-3926 7. Privacy Bird, AT&T Corporation (2006), http://privacybird.com 8. Langheinrich, M.: A Privacy Awareness System for Ubiquitous Computing Environments. In: Borriello, G., Holmquist, L.E. (eds.) UbiComp 2002. LNCS, vol. 2498. Springer, Heidelberg (2002) 9. Preibusch, S.: Implementing Privacy Negotiation Techniques in E-Commerce. In: 7th IEEE International Conference on E-Commerce Technology, IEEE CEC 2005, Technical University Munich, Germany (2005) 10. El-Khatib, K.: A Privacy Negotiation Protocol for Web Services. In: Workshop on Collaboration Agents; Autonomous Agents for Collaborative Environments Halifax, Nova Scotia, Canada (2003) 11. Yee, G., Korba, L.: Privacy Policies and their Negotiation in Distance Education. In: Instructional Technologies: Cognitive Aspects of Online Programs. Idea Group Inc. NRC 46555 (2004) 12. Maaser, M., Ortmann, S., Langendoerfer, P.: NEPP: Negotiation Enhancements for Privacy Policies. In: W3C Workshop on Languages for Privacy Policy Negotiation and Semantics-Driven Enforcement, Ispra, Italy (2006) 13. Maaser, M., Langendoerfer, P.: Automated Negotiation of Privacy Contracts. In: 29th Annual International Computer Software and Applications Conference (COMPSAC 2005), Edinburgh, Scotland, UK. IEEE Computer Society, Los Alamitos (2005) 14. CrEme, version 3.25, NSICOM (2004), http://www.nsicom.com 15. Tomcat, version 5.5, Apache Software Foundation (2005), http://www.tomcat. apache.org
Efficient Queries on XML Data through Partitioning Olli Luoma Nice Business Solutions Finland Oy Lemmink¨aisenkatu 32 B, P.O. Box 115, 20521 Turku, Finland [email protected]
Abstract. The query evaluation algorithms of practically all XML management systems are based on structural joins, i.e., operations which determine all occurrences of parent/child, ancestor/descendant, preceding/following etc. relationships between node sets. In this paper, we present a simple method for accelerating structural joins which is very easy to implement on different platforms. Our idea is to split the nodes into disjoint partitions and use this information to avoid unnecessary structural joins. Despite its simplicity, our proposal can considerably accelerate XPath evaluation on different XML management systems. To exemplify this, we describe two implementation options of our method - one built from the scratch and one based on a relational database - and present the results of our experiments. Keywords: XML, XPath, structural joins, spatial indexes.
1 Introduction In many modern applications areas, such as bioinformatics and Web services, there is a need to efficiently store and query large heterogeneous data sets marked up using XML [1], i.e., represented as a partially ordered, labeled XML tree. This imposes a great challenge on data management systems which have traditionally been designed to cope with structured rather than semistructured data. In recent years, a lot of work has thus been done to develop new methods for storing and querying XML data using XML query languages, such as XPath [2] and XQuery [3]. At the heart of the XPath query language, there are 12 axes, i.e., operators for tree traversal. In this paper, we present a method for accelerating the evaluation of the major axes, i.e., the ancestor, descendant, preceding, and following axes, through partitioning. We make use of the observation that starting from any node, the main axes partition the document into four disjoint partitions. Our idea is to partition the nodes more accurately, store the partition information, and use this information to filter the nodes that cannot be contained in the result before actually performing the axis. Since our method is based on simple geometric properties of the preorder and postorder numbers of the nodes, it is somewhat similar to the XML indexing approaches based on spatial structures, such as R-trees [4] or UB-trees [5]. The method described in this paper, however, is very easy to implement even using B-trees, and thus it can easily be used to accelerate XPath evaluation in XML management systems built on relational databases, such as XRel [6] and XPath accelerator [4]. Furthermore, our approach can J. Filipe and J. Cordeiro (Eds.): WEBIST 2007, LNBIP 8, pp. 98–108, 2008. c Springer-Verlag Berlin Heidelberg 2008
Efficient Queries on XML Data through Partitioning
99
also be tailored to be used with other node identification schemes, such as the order/size scheme. The rest of our paper is organized as follows. In Section 2, we take a short look at the related work and in Section 3, we discuss the basics of the XPath query language. Our partitioning method is presented in Section 4. In section 5, we discuss the implementation of our method and in Section 6, the results of our experimental evaluation. Section 7 concludes this article.
2 Related Work In the XML research literature, there are numerous examples of different XML management systems. These systems can roughly be categorized into the following three categories: 1. The flat streams approach handles the documents as byte streams [7,8]. Obviously, accessing the structure of the documents requires parsing and consumes a lot of time, but this option might still be viable in a setting where the documents to be stored and queried are small. 2. In the metamodeling approach, the XML documents are first represented as trees which are then stored into a database. With suitable indexes this approach provides fast access to subtrees, and thus the metamodeling approach is by far the most popular in the field of XML management research. Unlike in the flat streams approach, however, rebuilding large parts of original documents from a large number of individual nodes can be rather expensive. The partitioning method discussed in this paper also falls into this category. 3. The mixed approach aims at combining the previous two approaches. In some systems, the data is stored in two redundant repositories, one flat and one metamodeled, which allows the stored documents to be queried and the result documents to be built efficiently, but obviously creates some storage overhead. This problem could be tackled by compression to which XML data is often very amenable. There are also examples of a hybrid approach in which coarser structures of the XML documents are modeled as trees and finer structures as flat streams [9]. A lot of work has been carried out to accelerate structural joins [10], i.e., operations which determine all occurrences of parent/child, ancestor/descendant, preceding/following etc. relationships between node sets. Some approaches are based on indexes built on XML trees, whereas others aim at designing more efficient join algorithms. The former category includes the so called proxy index [11,12] which effectively partitions the nodes into overlapping partitions so that the ancestors of any given node are contained within the same partition. Thus, when the ancestors of a given node are retrieved it is sufficient to check the partitions to which the node is assigned. Conversely, descendants can also be found efficiently since there is no need to check the partitions to which the node is not assigned. However, this approach is suitable for accelerating only ancestor/descendant operations. The method discussed in this paper, on the contrary, can also accelerate preceding/following operations.
100
O. Luoma
The other group of methods include, for example, the staircase join [13]. The idea of the staircase join is to prune the set of context nodes, i.e., the initial nodes from which the axis is performed. For instance, for any two context nodes n and m such that m is a descendant of n, all descendants m are also descendants of n, and thus m can be pruned before evaluating the descendant axis. Another method based on preprocessing the nodes was proposed in [14]. In this approach the nodes were partitioned somewhat similarly to the method discussed in this paper. However, both of these methods require a considerable amount of programming effort since they work by preprocessing the data rather than by building indexes. In the context of relational databases, for example, this would mean either reprogramming the DBMS internals or programming a collection of external classes to implement the join algorithms. Our method, on the contrary, requires very little programming effort even when implemented using a relational database, which we regard as the main advantage of our approach. The nodes of an XML tree can also be partitioned according to some other criteria than mere structure. In almost all XML management systems, an index is built on the names of the nodes, which essentially partitions the nodes according to their tags and makes it possible to locate nodes with a given tag very efficiently. An index can also be built on the element paths of the nodes or some more selective structural criterion. The F&B index [15] and the schema graph [16], for example, partition the nodes into equivalence classes such that two nodes are considered similar if they share the same element path and their subtree structures are similar. One should notice that the aforementioned methods and the method discussed in this paper are not exclusive but rather complementary.
3 XPath Basics As mentioned earlier, XPath [2] is based on a tree representation of a well-formed XML document, i.e., a document that conforms to the syntactic rules of XML. A simple example of an XML tree corresponding to the XML document <e>kl <e>ez is presented in Fig. 1. The nodes are identified using their preorder and postorder numbers which encode a lot of structural information1. The tree traversals in XPath are based on 12 axes which are presented in Table 3. In simple terms, an XPath query can be thought of as a series of location steps of the form /axis::nodetest[predicate] which start from a context node - initially the root of the tree - and select a set of related nodes specified by the axis. A node test can be used to restrict the name or the type of the selected nodes. An additional predicate can be used to filter the resulting node set further. The location step n/child::c[child::*], for example, selects all children of the context node n which are named “c” and have one or more child nodes. As a shorthand for axes child, descendant-or-self, and attribute, one can use /, //, and /@, respectively. 1
In preorder traversal, a node is visited before its subtrees are recursively traversed from left to right and in postorder traversal, a node is visited after its subtrees have been recursively traversed from left to right.
Efficient Queries on XML Data through Partitioning
101
1, 10 b
2, 2 c
3, 1 d=”y”
4, 6 c
5, 3 d=”y”
8, 9 c
6, 5 e
9, 8 e
7, 4 ”kl”
10, 7 ”ez”
Fig. 1. An XML tree Table 1. XPath axes and their semantics Axis Semantics of n/Axis parent Parent of n. child Children of n, no attribute nodes. ancestor Transitive closure of parent. descendant Transitive closure of child, no attribute nodes. ancestor-or-self Like ancestor, plus n. descendant-or-self Like descendant, plus n, no attribute nodes. preceding Nodes preceding n, no ancestors or attribute nodes. following Nodes following n, no descendants or attribute nodes. preceding-sibling Preceding siblings of n, no attribute nodes. following-sibling Following siblings of n, no attribute nodes. attribute Attribute nodes of n. self Node n.
Using XPath, it is also possible to set conditions for the string values of the nodes. The XPath query //record=‘‘John Scofield Groove Elation Blue Note", for instance, selects all element nodes with label “record” for which the value of all text node descendants concatenated in document order matches “John Scofield Groove Elation Blue Note”. Notice also that the result of an XPath query is the concatenation of the parts of the document corresponding to the result nodes. In our example case, query //c//*, for example, would result in no less than <e>kl<e>ez<e>kl <e>ez.
4 Our Partitioning Method As mentioned earlier, we rely on pre-/postorder encoding [17], i.e., we assign both preorder and postorder numbers for the nodes. This encoding provides us with enough information to evaluate the four major axes [4] as follows: Proposition 1. Let pre(n) and post(n) denote the preorder and postorder numbers of node n, respectively. For any two nodes n and m, n ∈ m/ancestor::* iff pre(n) <
102
O. Luoma
10
Post
Post
10
Pre
Pre 10
10
Fig. 2. Examples on the major axes (left) and partitioning using value 4 for p (right)
pre(m) and post(n) > post(m), n ∈ m/descendant::* iff pre(n) > pre(m) and post(n) < post(m), n ∈ m/preceding::* iff pre(n) < pre(m) and post(n) < post(m), and n ∈ m/following::* iff pre(n) > pre(m) and post(n) > post(m). This observation is actually slightly inaccurate since the attribute nodes are not in the result of, for instance, the descendant axis. However, if all nodes are treated equally, the proposition holds. The observation is exemplified on the left side of Fig. 2 which shows the partitioning created by the major axes using node (6,5) as the context node. The ancestors of node (6,5) can be found in the upper-left partition, the descendants in the lower-right partition, the predecessors in the lower-left partition, and the followers in the upper-right partition. Based on the previous observation, the following proposition is obvious: Proposition 2. For any two nodes n and m and for any p > 0, pre(n)/p ≤ pre(m) /p and post(n)/p ≥ post(m)/p if n ∈ m/ancestor::*, pre(n)/p ≥ pre(m)/p and post(n)/p ≤ post(m)/p if n ∈ m/descendant::*, pre(n) /p ≤ pre(m)/p and post(n)/p ≤ post(m)/p if n ∈ m/preceding::*, and pre(n)/p ≥ pre(m)/p and post(n)/p ≥ post(m)/p if n ∈ m /following::*. Proposition 2 provides us with simple means for partitioning the nodes into disjoint subsets. The partitioning is exemplified in Fig. 2 (right) which shows the nodes of our example tree partitioned using value 4 for p. It should now be obvious that when searching, for instance, the ancestors of node (6,5), it is sufficient to check the nodes in the shaded partitions since other partitions cannot contain any ancestors. In what follows, values pre(n)/p and pre(n)/p where n is a node residing in partition P are simply referred to as the preorder and postorder number of P and denoted by pre(P ) and post(P ), respectively. Many minor axes can also benefit from partitioning. The ancestor-or-self and descendant-or-self axes, for example, behave similarly to ancestor and descendant axes, and thus they can be accelerated using the same partition information. The results of preceding-sibling and following-sibling, on the
Efficient Queries on XML Data through Partitioning
103
other hand, are subsets of preceding and following and the results of parent and child subsets of ancestor and descendant so they can also be accelerated using the partitioning. However, other minor axes than ancestor-or-self and descendant-or-self are generally very easy to evaluate, and thus using the partition information to evaluate them is often just unnecessary overhead especially if we have an index which can be used to locate the nodes with a given parent node efficiently [18]. One should also notice that in practice, the nodes are not distributed evenly and there are several empty partitions. Thus, the number of non-empty partitions is usually much smaller than p2 .
5 Implementation Options 5.1 Relational Implementation To test our idea, we designed a relational database according to the principles described in the previous section. As the basis of our implementation, we chose the XPath accelerator [4]. In order to store the partition information, we employed relation Part, and thus we ended up with the following relational schema; the underlined attributes serve as the primary keys: Node(Pre, Post, Par, Part, Type, Name, Value) Part(Part, Pre, Post)
As in the original proposal, the database attributes Pre, Post, Par, Type, Name, and Value of the Node relation correspond to the pre- and postorder numbers, the preorder number of the parent, the type, the name, and the string value of the node, respectively. The database attributes Type, Name, and Value are needed to support node tests and string value tests; the axes can be evaluated using database attributes Pre, Post, and Par. The partition information is contained in the Part table in which the database attributes Pre and Post correspond to the preorder and postorder number of the partition, respectively. The database attribute Part in the Node relation is a foreign key referencing to the primary key of the Part relation. For the sake of brevity, we do not describe the XPath-to-SQL query translation in detail; the details can be found in [4] and [6]. For our purposes, it is sufficient to say that in order to perform structural joins, these systems issue SQL queries which involve nonequijoins, i.e., joins using < or > as the join condition, which can easily lead to scalability problems [12,18,19]. In XPath accelerator, the XPath query n0 /descendant::*, for instance, is transformed into the following SQL query: SELECT DISTINCT n1 .* FROM Node n0 , Node n1 WHERE n1 .Pre>n0 .Pre AND n1 .Post
Notice that tuple variables n0 and n1 are joined completely using slow nonequijoins. However, we can use the partition information stored into table Part to replace some
104
O. Luoma
of the nonequijoins with much faster equijoins, i.e., joins using = as the join condition. With the partition information, the same query can be evaluated much more efficiently using the following piece of SQL: SELECT DISTINCT n1 .* FROM Node n0 , Node n1 , Part a1 , Part b1 WHERE a1 .Part=n0 .Part AND b1 .Pre>=a1 .Pre AND b1 .Post<=a1 .Post AND n1 .Part=b1 .Part AND n1 .Pre>n0 .Pre AND n1 .Post
In simple terms, we use tuple variable a1 for the partition in which node corresponding to tuple variable n0 resides. Variable b1 corresponds to the partitions which can contain descendants of n0 ; condition n1 .Part=b1 .Part restricts our search into the nodes residing in those partitions. Finally, we simply use condition n1 .Pre>n0 .Pre AND n1 .Post
Efficient Queries on XML Data through Partitioning
105
The structural joins within a partition, then, are carried out using algorithm joinPart: joinPart(n, P , axis) in: Node n, partition P , XPath axis axis out: Nodes in n/axis::* of p if axis = preceding then for each m ∈ P if pre(m) < pre(n) and post(m) < post(n) result ← result + m if axis = preceding-sibling then for each m ∈ P if pre(m) < pre(n) and par(m) = par(n) result ← result + m ... return result
Again, our algorithm is heavily simplified. However, it is easy to extend the algorithm to support node tests and string value tests by checking the type, name, and string value attributes of the node. Notice also that we could dramatically lower the number of structural joins by considering the fact that the all nodes in partition P such that pre(P ) < pre(n)/p and post(P ) < post(n)/p are guaranteed to be in n/preceding::*, and thus they could be added to the result without performing the structural joins. We also implemented this option but found out that is did not lead to considerable gains in evaluation times. However, the benefits obviously depend on the implementation and in some cases, this approach can further accelerate the joins.
6 Experimental Results 6.1 Relational Implementation We implemented the relational option using Windows XP and Microsoft SQL Server 2000 running on 2.00 GHz Pentium PC Equipped with 512 MB of RAM and standard IDE disks. The algorithms needed to build the databases, as well as the algorithms for query translation, were implemented using Java. We built indexes on Node(Post), Node(Name), Part(Pre), and Part(Post) and a clustered index on Node(Part). As our test data sets, we used the 1998 baseball statistics (52707 nodes) and the collection of Shakespeare’s plays (327129 nodes), both available at http://www.ibiblio.org/examples. The sizes of these documents were 640 kB and 7.47 MB, respectively; values 1,2,4, ..., 256 were used as a partitioning factor p. Fig. 3 presents the query performance for the major axes using a relatively large set of context nodes; in this figure, the word “Partitions” refers to the value of p or the number of partitions per (preorder or postorder) dimension. In the case of baseball.xml, nodes with name “PLAYER” (1226 nodes) and in plays.xml, nodes with name “SCENE” (750 nodes) served as the context nodes. One should notice that our method can lead to considerable performance gains in ancestor and descendant axes even with a very small amount of partitions. In the case baseball.xml and 16 partitions per dimension,
106
O. Luoma
for example, there were only 55 partitions in total. Thus, there were only 55 rows in the Part table, which is certainly acceptable considering that there are no less than 52707 nodes in the Node table. Even in the case of 256 partitions per dimension, there were only 772 rows in the Part table for “1998statistics.xml” and 859 rows for “plays.xml”. In the case of preceding and following axes, however, no real acceleration could be observed. These axes could be actually evaluated very efficiently even without the partitionin, which is due to the very sophisticated join algorithms implemented in SQL Server. Nevertheless, considering that the ancestor and descendant axes were very time-consuming to evaluate without the partition information, we are still convinced that the small amount of partition information can indeed pull its weight. 6.2 Native Implementation The evaluation of our native implementation was carried out using the same equipment and data sets which were used in the relational case. Furthermore, the same number of partitions per dimension were used but in these tests, we randomly selected the context nodes. Fig. 4 and Fig. 5 present the results obtained using the native implementation; all results are averages for 100 iterations. Overall, the ancestor and descendant axes behaved similarly to the relational case, i.e., they were considerably accelerated. However, the preceding and following axes were also accelerated by roughly a factor of two. This is actually intuitively clear since an average node has much more predecessors and followers than it has ancestors and descendants, and thus the preceding and following axes usually result in much larger node sets. In the case of ancestor axis, for example, the partitioning can help us to filter out a massive amount of nodes, i.e., the most of the descendants, predecessors, and followers, whereas in the case of preceding axis, a much smaller amount of nodes with respect to the size of the result can be filtered. One should also notice that ordering the nodes according to their preorder numbers favors the preceding axis over the following axis. After the preorder numbers have been checked in the case of preceding axis, we still have to filter out the ancestors using the postorder numbers of the nodes. In the case of following axis, on the contrary, the descendants have to be filtered, which is a harder task since a node in an 120000
30000 ancestor descendant preceding following
ancestor descendant preceding following
100000
20000
Time (ms)
Time (ms)
25000
15000 10000 5000
80000 60000 40000 20000
0 1
10 Partitions
100
0 1
10 Partitions
Fig. 3. Relational results 1998statistics.xml (left) and plays.xml (right)
100
Efficient Queries on XML Data through Partitioning 120000
700000 ancestor descendant preceding following
100000
ancestor descendant preceding following
600000 500000 Joins
80000 Joins
107
60000
400000 300000
40000
200000
20000
100000
0
0 1
10 Partitions
100
1
10 Partitions
100
Fig. 4. Number of joins for the major axes using random context nodes for 1998statistics.xml (left) and plays.xml (right); native implementation 300 ancestor descendant preceding following
200
Time (ms)
Time (ms)
250
150 100 50 0 1
10 Partitions
100
2000 1800 1600 1400 1200 1000 800 600 400 200 0
ancestor descendant preceding following
1
10 Partitions
100
Fig. 5. Times for the major axes using random context nodes for 1998statistics.xml (left) and plays.xml (right); native implementation
XML tree usually has more descendants than it has ancestors. Conversely, sorting the nodes according to their postorder numbers slightly favors the following axis.
7 Conclusions In this paper, we discussed a partitioning method which can considerably accelerate the evaluation of XPath axes in different XML management systems. Our method is based on simple properties of the preorder and postorder numbers of the nodes in an XML tree, which makes it very easy to implement. This was exemplified by presenting two different implementations of our idea which both indicated that alhtough our idea is simple and very easy to implement, it can lead to considerable performance gains. Acknowledgements. The author would like to thank Professor Jukka Teuhola of University of Turku (Finland) who provided his valuable comments during the preparation of this paper.
108
O. Luoma
References 1. W3C (World Wide Web Consortium): Extensible Markup Language (XML) 1.0, http://www.w3c.org/TR/REC-xml/ 2. W3C (World Wide Web Consortium): XML Path Language (XPath) 2.0, http://www.w3c.org/TR/xpath20/ 3. W3C (World Wide Web Consortium): XQuery 1.0: An XML Query Language, http://www.w3c.org/TR/xquery/ 4. Grust, T.: Accelerating XPath Location Steps. In: Proceedings of the 2002 ACM SIGMOD Conference on Management of Data, pp. 109–120 (2002) 5. Kr´atk´y, M., Pokorn´y, J., Sn´asˇ el, V.: Implementation of XPath Axes in the Multi-Dimensional Approach to Indexing XML Data. In: Proceedings of Current Trends in Database Technology, pp. 219–229 (2004) 6. Yoshikawa, M., Amagasa, T., Shimura, T., Uemura, S.: XRel: A Path-Based Approach to Storage and Retrieval of XML Documents Using Relational Databases. ACM Transactions on Internet Technology 1(1), 110–141 (2001) 7. Peng, F., Chawathe, S.S.: XPath Queries on Streaming Data. In: Proceedings of the 2003 ACM SIGMOD Conference on Management of Data, pp. 431–442 (2003) 8. Barton, C., Charles, P., Goyal, D., Raghavachari, M., Fontoura, M., Josifovski, V.: Streaming XPath Processing with Forward and Backward Axes. In: Proceedings of the 19th International Conference on Data Engineering, pp. 455–466 (2003) 9. Fiebig, T., Helmer, S., Kanne, C.-C., Moerkotte, G., Neumann, J., Schiele, R., Westmann, T.: Natix: A Technology Overview. In: Chaudhri, A.B., Jeckle, M., Rahm, E., Unland, R. (eds.) NODe-WS 2002. LNCS, vol. 2593, pp. 12–33. Springer, Heidelberg (2003) 10. Al-Khalifa, S., Jagadish, H.V., Patel, J.M., Wu, Y., Koudas, N., Srivastava, D.: Structural Joins: A Primitive for Efficient XML Query Pattern Matching. In: Proceedings of the 18th International Conference on Data Engineering, pp. 141–152 (2002) 11. Luoma, O.: Supporting XPath Axes with Relational Databases Using a Proxy Index. In: Proceedings of the 3rd International XML Database Symposium, pp. 99–113 (2005) 12. Luoma, O.: Xeek: An Efficient Method for Supporting XPath Evaluation with Relational Databases. In: Local Proceedings of the 10th East-European Conference on Advances in Databases and Information Systems, pp. 30–45 (2006) 13. Grust, T., van Keulen, M.: Tree Awareness for Relational RDBMS Kernels: Staircase Join. In: Intelligent Search on XML Data, Applications, Languages, Models, Implementations, and Benchmarks, pp. 231–245 (2003) 14. Tang, N., Yu, J.X., Wong, K.-F., L¨u, K., Li, J.: Accelerating XML Structural Join by Partitioning. In: Proceedings of the 16th International Conference on Database and Expert Systems Applications, pp. 280–289 (2005) 15. Kaushik, R., Bohannon, P., Naughton, J.F., Korth, H.F.: Covering Indexes for Branching Path Queries. In: Proceedings of the 2002 ACM SIGMOD Conference on Management of Data, pp. 133–144 (2002) 16. Luoma, O.: A Structure-Based Filtering Method for XML Management Systems. In: Proceedings of the 15th International Conference on Database and Expert Systems Applications, pp. 401–410 (2004) 17. Dietz, P.F.: Maintaining Order in a Linked List. In: Proceedings of the 14th Annual Symposium on Theory of Computing, pp. 122–127 (1982) 18. Luoma, O.: Modeling Nested Relationships in XML Documents using Relational Databases. In: Proceedings of the 31st Conference on Current Trends in Theory and Practice of Computer Science, pp. 259–268 (2005) 19. Luoma, O.: Efficient Methods for Storing and Querying XML Data with Relational Databases. PhD Thesis, University of Turku (2007)
Contract Based Behavior Model for Services Coordination Alberto Portilla1,2, , Genoveva Vargas-Solar1, Christine Collet1 Jos´e-Luis Zechinelli-Martini2, and Luciano Garc´ıa-Ba˜nuelos3 1
Laboratoire d’Informatique de Grenoble BP 72, 38402 Saint Martin d’H`eres, France 2 Research Center of Information and Automation Technologies Universidad de las Am´ericas, Puebla Sta Catarina Martir, 72820, San Andres Cholula, Mexico 3 Facultad de Ciencias B´asicas, Ingenier´ıa y Tecnolog´ıa Universidad Aut´onoma de Tlaxcala Calzada Apizaquito S/N Km. 1.5, 90300 Apizaco, Tlaxcala, Mexico
Abstract. A key step towards consistent services coordination to provide non functional properties. In this sense, transactional properties are particularly relevant because of the business nature of current applications. While services composition has been successfully addressed, transactional properties have been main- ly provided by ad-hoc and limited solutions at systems’ back end. This paper proposes a flexible transactional behavior model for services coordination. We assume that given a flow describing the application logic of a service based application, it is possible to associate to it a personalized transactional behavior in an orthogonal way. This behavior is defined by specifying contracts and associating a well defined behavior to the activities participating in the coordination. Such contracts ensure transactional properties at execution time in the presence of exceptions. Keywords: Transactional behavior, advanced transactional models, services oriented systems, services based applications, web services.
1 Introduction A key step towards consistent services coordination is to provide non functional properties. In this sense, transactional properties are particularly relevant because of the business nature of current applications. While services composition has been successfully addressed, transactional properties have been mainly provided by ad-hoc and limited solutions at systems’ back end. In service based applications, a service is a software component, available in a network, that offers some functions by exporting an application programming interface (API). Nowadays, services based applications capture complex processes for achieving a goal, they are built by coordinating interactions among several services.
Supported by the Mexican Education Council (PROMEP-SEP), the Autonomous University of Tlaxcala, M´exico and the Jenkins Excellence Fellowship Program at UDLA.
J. Filipe and J. Cordeiro (Eds.): WEBIST 2007, LNBIP 8, pp. 109–123, 2008. c Springer-Verlag Berlin Heidelberg 2008
110
A. Portilla et al.
Figure 1 shows an example of a service based application composed using services of several providers. In the Figure, boxes represent activities and each activity represents a service method call. Let us consider that this application is related to a trip reservation with the following logic: given reservation information it is necessary to get a bank authorization for the payment (a1 ), if the payment has been guaranteed, it is necessary to reserve the flight (a2 ), a hotel (a3 ) and a car (a4 ), otherwise the reservation is canceled (a5 ). In addition, there are non functional issues that must be considered, some are related to application semantics and others are related to the activity semantics. In this example, at the application level we can consider the following aspects: before making a reservation (flight, hotel or car) the payment must be authorized (a1 ); a reservation cannot be rejected (a5 ) and processed at the same time (a2 , a3 and a4 ). At the activity level we can consider: there are several alternatives for executing an activity. For example, flight reservation can be completed using Mexicana, Air France or British Airways.There are critical activities for the success of the application. For example, while flight reservation and hotel reservation are critical for the acceptance of the trip reservation, an unsuccessful car reservation can be tolerated. Some activities can be retried in case of failure, for example, obtaining the bank authorization or booking the flight.
Fig. 1. Service based application example
In particular, these aspects illustrate transactional behavior. Recall that a transaction is a unit of work composed of several operations that must be handled as an execution unit. Such execution unit can be successfully executed or not [22,5]. Even if such a behavior has been addressed for managing data (DBMS) and process oriented applications (transactional workflows), existing solutions are not well adapted for: – synchronizing highly autonomous services execution within transactions and, – for dealing with long duration resources blocking. Transactional behavior for services coordination is currently addressed in three ways: it is explicitly defined within the application logic by adding sequences of activities to its workflow. Such activities implement advanced transactional models (e.g. [30], [25], etc.); it is implemented by transactional protocols (e.g. BTP [12], WS-Tx [3], etc.); or it is specified by transactional policies expressed using transactional coordination languages (e.g., πt-calculus[21], [1], etc.). However, this leads to complex service based applications difficult to maintain and hardly adaptable to different application requirements [17,29,6]. This paper proposes a contract based behavior model for specifying transactional properties to services coordination. Our approach addresses atomic behavior based on
Contract Based Behavior Model for Services Coordination
111
atomicity contracts for associating atomicity properties and recovery strategies to sets of activities (i.e., spheres [4,17,20]). Our model separates the specification of coordination and transactional behavior as follows: – Application logic is captured by using a coordination approach based on workflows. – Transactional behavior is defined by contracts according to the behavior of services participating in a coordination and the semantic of the target service based application. Given a coordination describing the application logic of a service based application it is possible to associate it with a transactional behavior. Different from existing proposals our approach enables the definition of transactional behavior without implying the modification of the original application logic. Besides, our model is not based in any advanced transactional model, rather it enables such kind of behavior if necessary. Our approach proposes recovery strategies for ensuring transactional behavior of services coordination. Such strategies can be chosen according to specific application requirements. Thus, the same type of application can have different associated behaviors according to given application requirements. The remainder of the paper is organized as follows. Section 2 gives an overview of services coordination with transactional properties. Section 3 introduces the main concepts of the transactional behavior model that we propose. Section 4 illustrates the specification of transactional behavior for a target application through a use case. Section 5 discusses the extensibility of our model. Section 6 compares our work with existing approaches that provide transactional properties to services coordination. Finally, Section 7 concludes this paper and discusses future work.
2 Coordination Model with Transactional Behavior Properties A coordination model can be used for expressing an application logic defined by an orchestration. An orchestration specifies the workflow whereby activities are synchronized using ordering operators (see Figure 2).
Fig. 2. Coordination UML class diagram
112
A. Portilla et al.
Acoording to its role within the application logic, an activity has an associated behavior that specifies the way it should be handled under the presence of exceptional situations. 2.1 Service A service represents a set of functions provided by an autonomous software component through a port. It is formally defined as a quadruple S-name, F M , N M , P where: – S-name is the name of the service. – F M and N M are interfaces consisting of a finite set of methods that provide respectively functional and non functional aspects, for example XA/XOPEN [27]. A method is defined by a triple M =M -name, In, Out where: • M -name is the name of the method. • In and Out are respectively the sets of input and output parameters. • A parameter is defined by a tuple P -name, T ype, where: ∗ P -name is the name of the parameter. ∗ T ype represents a predefined type such as boolean, string, integer, etc. – P is the communication port used for interacting with the service. The trip reservation application previously introduced uses three services providers (see Figure 1): bank controls financial operations executed on accounts (Provider 1), travel agency rates and manages flight and hotel reservations (Provider 2), and Provider 3 manages car rental. 2.2 Orchestration An orchestration describes an application logic through control and data flows. In our work an orchestration is defined using Petri nets. A Petri net consists of places, transitions, tokens, and directed arcs. A place models the states among two transitions. A transition represents a task, hereinafter, refered as activity. A token represents an execution state. A directed arc is used to link places and transitions. Therefore, a Petri net is defined as a triple P L, A, F [28] where: – P L is a finite set of places. – A is a finite set of transitions, P L ∩ A = φ. – F ⊆ (P L x A) ∪ (A x P L) is a set of arcs. Using Petri nets the control flow of an orchestration is expressed by order operators (sequential routing, parallel routing, selective routing, and iteration) that relate activities. The data flow is embedded within the control flow. Figure 3 shows the orchestration of the trip reservation example by using Petri nets: places are represented as circles (p1 , p2 , p3 , p4 , p5 and p6 ), transitions are represented as boxes (a1 , a2 , a3 , a4 and a5 ), tokens are represented as black circles, and flow relationships are represented as arrowed lines.
Contract Based Behavior Model for Services Coordination
113
Fig. 3. Service based application with Petri nets
2.3 Activity Behavior Depending on its role within an application logic, an activity can have different behaviors at execution time in the presence of failures [14,9]. There are three possible scenarios: 1. An activity can be rolled back/undone, but possible side effects must be taken into account. In the trip reservation example (see Figure 1), undoing the flight reservation (a2 ) can imply an extra charge. 2. A committed activity can be compensated by a so called compensation activity that semantically undoes its actions but it does not necessarily gets back the application to the previous state before the activity was committed. For example, in trip reservation application, canceling a flight reservation can be undone with an extra charge which does not return the balance account to the previous state. However, there are some activities that cannot be undone, for example personalizing an item using laser engraving. 3. The execution of an activity can be retried several times within the same coordination. In our example, obtaining the hotel reservation can be executed several times assuming that hotels accommodation is not easy during hollidays. However, retrying an activity can be associated to other constraints, such as the number of times that an activity can be retried. Considering the above scenarios an activity can be1 (see Table 1): – Non vital. The activity does not need to be compensated if it has to be undone/rolled back after having committed. For example, let us assume an activity that uses the trip reservation information for sending commercial information. It can be defined as non vital because the reservation can be done even if the the customer does not receive the publicity. – Critical. There is no way to compensate or undo the effects of an activity once it has been committed. Recalling the trip reservation example, the activity bank 1
For the time being, this classification is non exhaustive.
114
A. Portilla et al.
authorization is critical because once a reservation has been authorized the bank cannot change its opinion, so the execution of the reservation cannot be stopped for this reason. – Undoable. A commited activity can be undone by a compensating activity without causing side-effects. Once it has been compensated the activity can be retried. In our example the activity hotel reservation can be defined as undoable activity, assuming that some hotel reservation can be canceled at most three days before the date of arrival. – Compensatable. A commited activity can be undone by a compensating activity with associated side-effects. Once it has been compensated the activity can be retried but other aspects must be considered such as application restrictions, monetary costs, temporal constraints, etc. In our trip reservation example the activity flight reservation can be undone by an activity that cancels the flight and that reimburses a percentage of the original amount to the costumer account. However, retrying flight reservation when canceled can imply costs that might not be acceptable for the customer. Table 1. Types of activities Activity Causes Can be Can be type side effects compensated retried Non vital No No Yes Critical No Undoable No Yes Yes Compensatable Yes Yes Maybe
Execution State. At execution time, an activity can have the following associated states (see Figure 4): – – – – – – –
Ready when the activity is ready to be executed. Blocked is used for intercepting activate call for some purposes. Active when the activity is being executed. Failed when the activity execution has unsuccessfully finished. Committed when the activity execution failed. Withdrawn when the activity has been withdrawn. Compensated when the activity execution has been semantically undone or rolled back.
We consider activities as atomic, this means that when executed they commit or fail.
3 Transactional Behavior Model Inspired in [4], activities within an orchestration can be grouped together into a behavior contract that can have specific associated transactional properties (e.g., atomicity) and execution strategies for ensuring such properties. For example, the activities used for
Contract Based Behavior Model for Services Coordination
115
Fig. 4. Activity execution transition state diagram
implementing the payment process can be executed atomically, so that non authorized purchases do not lead to the generation of an invoice. Furthermore, several behavior contracts can be specified and composed within an application logic so that its behavior can be controlled in a fined grained manner. 3.1 Contract A contract specifies a transactional behavior for a set of activities (e.g. atomicity, isolation, etc.). It specifies a recovery strategy that must be executed in case of failure according to the contract type. Grouping Activities. A contract groups together sequences of activities or contracts as shown in Figure 2. In the following let A be a set of activities. This set may be obtained from an orchestration defined as a Petri net P L, A, F . The set of contracts associated to A denoted C(A) is defined by following rules: 1. If ai ∈ A, then (ai ) ∈ C(A) is a simple contract and ai is called the component of the contract. For example c2 in Figure 5, where c2 = (a5 ). 2. If c1 , c2 , ...,cn ∈ C(A), (c1 ,c2 ,...,cn ) ∈ C(A) is a complex contract and c1 , c2 , ...,cn are called the components of the contract. For example c1 in Figure 5 where c2 = ((a2 ), (a3 ), (a4 )). Similarly, c3 is a complex contract where c3 = (c1 , c2 ). Behavior. The behavior of a contract c at execution time can be deduced from the behavior of its components according to the following rules: – c is non vital, if its components are non vital. For example, assuming that components (a2 ), (a3 ) and (a4 ) are non vital the contract c1 is non vital (see Figure 5). – c is critical, if it contains at least one critical component that has been committed. For example assuming that the component (a2 ) is critical and it has already been committed, then contract c1 is critical (see Figure 5). Note that a token in place p3 indicates that (a2 ) is committed for a given execution state.
116
A. Portilla et al.
– c is undoable, if all its components are undoable or non vital. For example assuming that (a2 ) is non vital and (a3 ) and (a4 ) are undoable, contract c1 is undoable (see Figure 5). – c is compensatable, if all its components are non vital, undoable or compensatable. For example assuming that (a2 ) is non vital, (a3 ) is undoable, and (a4 ) is compensatable, contract c1 is compensatable (see Figure 5). Note that the behavior of a contract is determined at run time since it depends on the behavior of its components and on their execution state. For example, consider contract c1 in Figure 5 with component (a3 ) defined as critical. Then c1 is compensatable before (a3 ) commits, and it becomes critical after (a3 ) has been committed.
Fig. 5. Contracts example
Execution State. The execution state of a contract c can be: – Ready when c is ready to be executed, this implies that all its components are ready to be executed. – Active when c is being executed, at least one of its components is being executed. – Failed when c finished unsuccessfully. At least one of its vital components fails. – Committed when c finished successfully. All vital contract components commit. – Compensated when c has been semantically undone. At least one of its vital components has been compensated. We also consider contracts as atomic: the initial state of any contract is ready, then it is active during its execution, but after its execution, a contract either fails or commits. Well Formed Contract. A contract is well formed if it is guaranteed that it will commit or fail. Thus a contract c is well formed if it contains at most one critical component, denoted as ccr , or all components within the contract are not critical. Note that when c contains a critical activity, once ccr commits c must commit because ccr cannot be compensated or undone. In such a case, for ensuring that c commits when ccr commits, it is necessary that the set of components to be executed after ccr must be retriable. For
Contract Based Behavior Model for Services Coordination
117
example, contract c1 in Figure 5 is well formed if (a2 ) is compensatable, (a3 ) is critical and (a4 ) is undoable. Safe Point. Given a simple contract c its safe point is either the most recent critical component committed within c or the activity/contract before the first component of c. The safe point represents the last consistent state of a coordination before the execution of c. If c fails, it is possible to cancel the components within c and restart the execution from the safe point. For example, a1 can be a safe point for contract c1 . Assuming that a1 is the safe point of c1 , it is also the safe point for contract c3 if contract c2 is well formed and contract c2 is undoable (see Figure 5). Recovery Strategies. There are three main recovery strategies that can be implemented in case of failure: forward execution, backward and forward recovery. Recovery strategies help to decide wether the execution of a coordination can continue after the occurrence of a failure. Given a well formed contract c with its associated safe point sp: – Forward execution. If a retriable or non vital component in c fails the execution of c can proceed. For example considering that all components of c1 are non vital (see Figure 5), its execution can proceed even if one of its components fails. – Backward recovery. If a component in c fails and forward recovery cannot be applied then previously committed components in c are undone by their corresponding compensating activities until a safe point is reached.For example, assume that the components in c1 are defined as follows: (a2 ) and (a3 ) are compensatable and (a4 ) is critical. If during the execution of c1 , (a3 ) fails, then (a2 ) and (a3 ) are succesively compensated until a1 is reached. – Forward recovery. It combines backward recovery and forward execution. If forward execution cannot be applied because a critical or compensatable component has failed then apply backward recovery until a safe point is reached and then apply forward execution with a different execution path. Using one of the above recovery strategies depends on the type of component that fails and the type of previously executed components. 3.2 Atomicity Contracts A classic example of a transactional behavior is the atomicity requirement. We used the notion of contract and recovery strategies for defining so called atomicity contracts. We show in the following the definition of a strict atomicity contract2 . Intuitively, an atomicity contract specifies that given a contrat c all its components must be successfully executed or no component at all. Details on the definition of atomicity contracts can be found in [23]. The strict atomicity contract is defined by the following rules. Given a well formed contract c: – Rule 1. If a critical component in c fails, the contract fails and backward recovery is applied. 2
In fact, we are relaxing the classic atomicity concept.
118
A. Portilla et al.
– Rule 2. If a component in c fails and the critical component of c has already been committed then forward execution is applied until c commits. – Rule 3. If a component in c fails and the critical component of c has not yet been committed, forward execution is applied until c commits. If the component has failed repeatedly c fails and backward recovery is applied.
4 Defining Transactional Behavior for an e-Commerce Application Consider the e-commerce application illustrated in Figure 6. A costumer first provides an order and her/his account information, then commercial information is sent. If the purchase is authorized by the bank, the purchase can proceed; otherwise the order is canceled. For processing authorized orders, the stock is checked and the purchase process starts. If products are available the purchase is applied to an account, an invoice is sent and the order is completed; at the same time the packet is wrapped and delivered. On the other hand, if products are not available, the payment and the shipment are canceled.
Fig. 6. E-commerce application with atomicity contracts
Thus the application logic (see Figure 6) of such an application specifies activities for defining and authorizing an order (Get Order(), Get Payment Information(), Bank Autho rization()); and four other execution paths for: – Sending commercial information: Send Commercial Info(); – Stop a rejected order: Cancel Order(); – Processing an authorized order: Process Order(), Check Stock(), Wrap() and Deliver() (if there is enough stock); or Cancel Shipment(), Cancel Payment() (if there is not enough stock). – Executing the purchase: Apply Charges(), Send Invoice(), Finish Order().
Contract Based Behavior Model for Services Coordination
119
4.1 Activities The company that uses this application has specific business rules that must be respected. For instance: – BR1 : the bank authorization can be only executed once during the process of an order. – BR2 : once a package has been delivered it cannot be returned to the company. – BR3 : The company has implemented packages recycling strategies, so if a package has been wrapped, the wrapping paper can be removed and used for other packages. – BR4 : Invoices can be canceled. – BR5 : If clients are not satisfied with the products they can return them to the company but extra charges are applied. Considering such business rules, we associated the following behaviors to activities (see Figure 6): – – – –
Bank authorization() and Deliver() are critical activities (BR1 ,BR2 ). Wrap() and Send invoice() are undoable activities (BR3 , BR4 ). Apply charges() is a compensatable activity (BR5 ).
The remaining activities are considered as non vital ones because no specific business rule is specified in the application.
It must be noted that activities behavior depends on the application semantics. If the application changes its business rules, activities behavior can change. An advantage of our approach is that this can be easily done because the application logic and the transactional behavior aspects are defined separately. 4.2 Contracts The business rules defined in the previous section concern activities behavior. Still, the following business rules concern more complex patterns that complete the application semantics. The authorization of the bank is requested once the costumer information has been obtained (BR6 ). An order is finished only when charges have been applied to an account and corresponding invoice has been sent (BR7 ). Once packages have been packed they must be shipped (BR8 ). The purchase is finished once the package is delivered and the order has been processed (BR9 ). Thus the following contracts and their associated behaviors must be defined: – c1 is a critical contract that groups components involved in the order authorization ( (Get Order()), (Get Payment Information()), (Bank Authorization()) ). It is critical because it contains the critical component (Bank authorization()). – c2 is a compensatable contract that groups activities used for implementing the order processing ( (Apply Charges()), (Send Invoice()), (Finish Order()) ). It is compensatable because (Apply Charges() ) is a compensatable component. – c3 is a critical contract that groups activities for delivering the package ( (Wrap() ), (Deliver()) ). It is critical because it contains a critical component (Deliver()).
120
A. Portilla et al.
– c4 is a critical contract that groups contracts c2 and c3 for ensuring that the purchase will finish only if the package is wrapped and delivered and the payment has been executed and the invoice has been sent. It is critical because c3 is critical. Note that all contracts are well formed because they contain at most one critical component. Once contracts have been defined, they must be associated to atomicity behaviors that specify strategies to be undertaken at execution time in the presence of failure in order to ensure specific properties. In the case of our example, we have identified contracts requiring strict atomicity properties in order to fulfil BR7 -BR9 . Coupled with the strict atomicity contract that we defined in Section 3.2, c1 ensures BR6 , c2 ensures BR7 , c3 ensures BR8 , and c4 ensures BR9 .
5 Model Extensibility We claim that our transactional behavior model is flexible and adaptable to application needs because our model is not based in an specific advanced transactional model or extends an existing coordination language. As a prof of this, let us analyze the strict atomicity contract introduced in section 3.2 with respect to activities within such an atomicity contract. Let c be a simple well formed contract, c={a1 , a2 , . . . , an }. Let us consider, N V as the set of non vital activities, CR as the set of critical activities, U N as the set of undoable activities, and CO as the set of compensatable activities. There are four possibilities for activities within c: 1) all activities within c are non vital, 2) all activities within c are non vital or undoable, 3) all activities within c are non vital undoable or compensatable, and 4) there is a critical activity within c: – Case 1. This is the trivial case because c always commits. – Case 2. It extends the saga model because undoable activities can be retried, if some one of them fails it must be retried. However note that c will always commits because its activities will commit in a finite time after some retries. – Case 3. It extends the saga model if we assume that activities can be retried in a limited way. When activities cannot be retried it implements saga model behavior. This behavior has been also provided by flexible transaction model. – Case 4. It implements a kind of pivotal model such as proposed in flexible transaction model. As long as a critical activity does not commit, forward execution or backward recovery can be applied. Yet, when a critical activity commits, only forward execution is possible. It must noted that although this kind of behavior was introduced by flexible transactional model, in this case it only corresponds to one alternative execution path. By defining one atomicity contract using our model we have implemented several transactional models according to activity properties. We also have implemented such transactional models without modifying the application logic.
6 Related Works Several research projects have addressed transactional behavior for information systems, first in the context of DBMS and after for process oriented systems.
Contract Based Behavior Model for Services Coordination
121
In DBMS, transactional behavior has been tackled successfully to data through the concept of ACID transactions [5,22,7,15] and advanced transactional models [13,10,31]. These approaches are well suited when data reside in one site and transactions lifetime is short. Besides they operate with homogeneous execution units and therefore they are not applicable directly to others environments such as Internet. In workflow systems there are several approaches that aim at ensuring consistency among computations using process as execution units. WAMO [8] introduces a complex transactional language for workflows. [17] introduces an approach to add atomicity and exception handling for IBM-FlowMark. [16] proposes a workflow definition language to implement the saga model. [6] addresses atomic behavior for workflows by means of contracts. [25] proposes a model based in flexible transaction model for handling concurrency and recovery in process execution. However these approaches address transactional behavior in an ad-hoc fashion. In Web services, transactions are used to ensure sound interactions among business process. Web services transactions (WS-Tx) [3] and business transaction protocol (BTP) [12] remain as the most accepted protocols for coordinating Web services. A coordination protocol is a set of well-defined messages that are exchanged between participants of a transaction scope. However, these approaches do not offer mechanisms to ensure correctness in the specification because there is no reference model and the developer implements transactional behavior. Other approaches provide transactional frameworks. [11] introduces a model for transactional services composition based on an advanced transactional model. [1] proposes an approach that consists of a set of algorithms and rules to assist designers to compose transactional services. In [30] the model introduced in [25] is extended to web services for addressing atomicity. These approaches support the definition of atomic behavior based in the states of termination of activities with execution control flow defined a priori, which makes them not adaptable. There are other approaches, transaction policies [26], πt-calculus [2] and transactional Web services orchestration [19], that address transactional behavior to web services based on existing protocols (BTP and WS-Tx) and therefore implement partially transactional behavior without a clear separation among application logic and transactional requirements. In contrast to Web services protocols and frameworks, we consider that transactional aspects can be separated from the coordination specification and tackled using a general point of view. In this way our model fulfills the current spirit of reusing practices existing in software engineering.
7 Conclusions and Future Work The main contribution of this paper is an approach that adds transactional behavior to services coordination. For modeling transactional behavior, we introduce a model based on behavior of activities and set of activities. The contract concept is used for addressing a transactional behavior to contracts. Our model can be added to existing coordination approaches with minor changes but must important respecting the application logic. Our behavior model can be scaled because contracts can be defined recursively.
122
A. Portilla et al.
We are currently formalizing our model to provide a general framework for specifying transactional properties for services coordination and for ensuring such properties at execution time. We are also conducting the implementation of a contract management engine that can be plugged in existing orchestration engines to enact transactional orchestrations (see [24,18] for details). Finally, we will explore how our behavior model can be extended to address other properties such as isolation, and durability. Acknowledgements. This research is partially related to DELFOS project of the Franco-Mexican Laboratory of Informatics (LAFMI) of the French and Mexican governments.
References 1. Bhiri, S., Godart, C., Perrin, O.: Reliable web services composition using a transactional approach. In: IEEE International, (ed.) O. e-Technology, e-Commerce and e-Service, March 2005, vol. 1, pp. 15–21 (2005) 2. Bocchi, L., Ciancarini, P., Rossi, D.: Transactional aspects in semantic based discovery of services. In: Jacquet, J.-M., Picco, G.P. (eds.) COORDINATION 2005. LNCS, vol. 3454, pp. 283–297. Springer, Heidelberg (2005) 3. Cox, W., Cabrera, F., Copeland, G., Freund, T., Klein, J., Storey, T., Thatte, S.: Web services transaction (ws-transaction). Technical specification, BEA Systems, International Business Machines Corporation, Microsoft Corporation, Inc. (November 2004) 4. Davies Jr., C.T.: Data processing spheres of control. IBM Systems Journal 17(2), 179–198 (1978) 5. Delobel, C., Adiba, M.: Bases de donn´ees et syst`emes relationnels. Dunod, Informatique (1982) 6. Derks, W., Dehnert, J., Grefen, P., Jonker, W.: Customized atomicity specification for transactional workflows. In: Proceedings of the International Symposium on Cooperative Database Systems and Applications, pp. 155–164. IEEE, Los Alamitos (2001) 7. Doucet, A., Jomier, G. (eds.): Bases de donn´ees et internet, Mod`eles, languages et syst`eme, 1st edn. Hermes, Informatique et Syst`emes d’information. Lavoisier (2001) 8. Eder, J., Liebhart, W.: The workflow activity model WAMO. In: Conference on Cooperative Information Systems, pp. 87–98 (1995) 9. Eder, J., Liebhart, W.: Workflow recovery. In: Proceedings of the International Conference on Cooperative Information Systems, CoopIS 1996 (June 1996) 10. Elmagarmid, A.K., Leu, Y., Litwin, W., Rusinkiewicz, M.: A multidatabase transaction model for interbase. In: Proceedings of the sixteenth international conference on Very large databases, pp. 507–518. Morgan Kaufmann, San Francisco (1990) 11. Fauvet, M.-C., Duarte, H., Dumas, M., Benatallah, B.: Handling transactional properties in web service composition. In: Ngu, A.H.H., Kitsuregawa, M., Neuhold, E.J., Chung, J.-Y., Sheng, Q.Z. (eds.) WISE 2005. LNCS, vol. 3806, pp. 273–289. Springer, Heidelberg (2005) 12. Furniss, P.: Business transaction protocol. Technical specification, OASIS (November 2004) 13. Garc´ıa-Molina, H., Salem, K.: Sagas. In: ACM (ed.) 9th Int. Conf. on Management of Data, San Francisco, California, USA, pp. 249–259 (1987) 14. Gray, J.: The transaction concept: Virtues and limitations (invited paper). In: Very Large Data Bases, 7th International Conference, Cannes, France, Proceedings, September 9-11, 1981, pp. 144–154. IEEE Computer Society, Los Alamitos (1981)
Contract Based Behavior Model for Services Coordination
123
15. Gray, J., Reuter, A.: Transaction processing: concepts and techniques. Morgan Kaufmann Publishers, San Francisco (1993) 16. Grefen, P., Vonk, J., Apers, P.: Global tansaction support for workflow management systems: from formal specification to practical implementation. Very Large Data Base Journal 10(4), 316–333 (2001) 17. Hagen, C., Alonso, G.: Exception handling in workflow management systems. IEEE Transactions on Software Engineering 26(10), 943–958 (2000) 18. Hern´andez-Baruch, V., Portilla, A., Zechinelli-Martini, J.-L.: Rose: A transactional services coordination engine. In: 8th Mexican International Conference on Computer Science (ENC 2007). ENC-SMCC. IEEE, Los Alamitos (2007) 19. Hrastnik, P., Winiwarter, W.: Twso transactional web service orchestrations. Journal of Digital Information Management 4(1) (2006) 20. Leymann, F., Roller, D.: Workflow-based applications. IBM Systems Journal 36(1) (1997) 21. Milner, R., Parrow, J., Walker, D.: A calculus of mobile processes, part i/ii. Journal of Information and Computation 100, 1–77 (1992) 22. Ozsu, M.T., Valduriez, P.: Principles of distributed database systems, 2nd edn. Prentice Hall, Englewood Cliffs (1999) 23. Portilla, A.: Services coordination with transactional properties. Master’s thesis, UDLAINPG, M´exico, Grenoble (June 2006) 24. Portilla, A., Vargas-Solar, G., Collet, C., Zechinelli-Martini, J.-L., Garc´ıa-Ba˜nuelos, L., Hern´andez-Baruch, V.: Rose: a transactional services coordination engine. In: Proceedings of the 23emes Journees Bases de Donness Avancees, BDA 2007 (October 2007) 25. Schuldt, H., Alonso, G., Beeri, C., Schek, H.-J.: Atomicity and Isolation for Transactional Processes. ACM Transactions on Database Systems (TODS) 27(1), 63–116 (2002) 26. Tai, S., Mikalsen, T., Wohlstadter, E., Desai, N., Rouvellou, I.: Transaction policies for service-oriented computing. Data Knowl. Eng. 51(1), 59–79 (2004) 27. The-Open-Group: Distributed Transaction Processing: The XA Specification, X/Open Company Ltd., U.K (1991) 28. van der Aalst, W., van Hee, K.: Workflow Management, Models, Methods, and Systems, 1st edn. The MIT Press, Cambridge (2004) 29. Vargas-Solar, G., Garc´ıa-Banuelos, L., Zechinelli-Martini, J.-L.: Toward aspect oriented services coordination for building modern information systems. In: Encuentro Internacional de Computacion 2003. ENC-SMCC. IEEE, Los Alamitos (2003) 30. Vidyasankar, K., Vossen, G.: A multi-level model for web service composition. In: ICWS, p. 462. IEEE Computer Society, Los Alamitos (2004) 31. Wachter, H., Reuter, A.: The contract model. In: Elmagarmid, A.K. (ed.) Database Transaction Models for Advanced Applications, ch.7, pp. 219–263. Morgan Kaufmann Publishers, San Francisco (1992)
Transparent Admission Control and Scheduling of e-Commerce Web Services Dmytro Dyachuk and Ralph Deters Department of Computer Science, University of Saskatchewan 110 Science Place, Saskatoon, Canada [email protected], [email protected]
Abstract. Web Services are applications that expose functionality to consumers via public interfaces. Since these interfaces are defined, described and consumed using XML-based standards, Web Services outperform other middleware approaches (e.g. CORBA, RPC) in terms of platform interoperability and ease of use. Web Services support the concept of loosely coupled components, which in turn enables the development of more agile and open systems. However, this flexibility comes at the price of reduced control over the usage of the services that are exposed via the interfaces. This paper focuses on the transparent scheduling of inbound requests by introducing a proxy that prevents clients from directly accessing the provider. By manipulating the order and volume of requests sent to the provider it becomes possible to improve throughput and mean response time and to ensure consistent performance in overload situation. Keywords: SOA, Web Services, Scheduling.
1 Introduction According to the Four Tenets of Service Orientation [7], services are characterized by having clear boundaries and autonomy. Service Orientation (SO) also replaces data types as a means to describe input/output of services by using contracts and schemas and defines service compatibility by use of policies. In a service-oriented system [fig. 1], services are offered by service providers that register them with registries (e.g. UDDI). Service consumers (aka clients) discover at runtime service providers by simply queering the registries. Upon discovering a service provider, the consumer obtains from the provider the meta-data of the service that is then used to establish a binding to the provider. Since services are high-level constructs that hide implementation details, consumers can easily bind to unknown services across platform and language barriers, resulting in a system with very dynamic functional dependencies between its components. Consequently SO supports very loose coupling between consumers and providers, allowing for agile and open systems. Compared to other middleware approaches such as RPC (e.g. ONC-RPC) and object-oriented middleware (e.g. CORBA) SO differs in its lack of access transparency, since there is a very clear notion between local and remote. Service-oriented middleware (e.g. Web Services[6]) enables developers to expose functionality in terms of services, which can be described in a declarative manner ensuring interoperable and platform independence. Using IDE tools (e.g. Visual Studio [5], J. Filipe and J. Cordeiro (Eds.): WEBIST 2007, LNBIP 8, pp. 124–136, 2008. c Springer-Verlag Berlin Heidelberg 2008
Transparent Admission Control and Scheduling of e-Commerce Web Services
125
Registry (UDDI Server in Web Services)
s er ov
Pu bli sh es
sc Di
Accesses
Service Provider
Service Consumer
Fig. 1. Service Oriented Architecture
Eclipse [2]) and frameworks (e.g. Axis [1]) it is fairly easy for programmers to expose application interfaces and/or consume existing services, resulting in an ever-increasing number of Web Service deployments. However, the ease with which components (e.g. legacy systems) can now be exposed and consequently combined, raises serious concerns in regards to the dependability of the resulting system. It is important to remember that providers of services expose local resources (e.g. legacy applications) to potentially new and unknown loads. This is particularly worrisome since Web Services platforms tend to implement PS (Processor Sharing) as the default scheduling policy [11] which can easily lead to server overload situations that in turn can cause ripple effects throughout a system (e.g. faultpropagation). This paper focuses on the use of scheduling as a means to ensure that the exposed services are protected from overload situations and that the throughput and mean response time are optimized. The remainder of the paper is structured as follows. Section two presents an overview on server behaviour and scheduling. Section three presents a benchmark and an experimental setup of a system used to evaluate the scheduling of service requests. This is followed by experimentation and evaluation sections that discuss the results of the experimentation. Section six discusses related work. The paper concludes with a summary and outlook.
2 Services and Load If we assume that service providers do not share resources with other service providers (e.g. no two providers expose the same data base), than every service provider can be modelled as a server. If these providers are capable of handling multiple requests simultaneously, incoming service requests must be assigned a separate thread. Since each server has a finite amount of resources and each consumer request will lead to a temporary reduction of server-resources, it is interesting to examine the server’s behaviour under various loads. Studies [12] show that servers respond in a common way to loads. If a server is gradually exposed to an ever-increasing number of service requests it is possible to observe
D. Dyachuk and R. Deters
Throughput
126
with thrashing without thrashing Number jobs of on server under-load
saturation (peak load)
over-load
Fig. 2. Behaviour of Server
three distinct stages, under-load, saturation and over-load. At the beginning the server experiences a load that is below its capacity (under-load) and consequently it is not fully utilized. As the number of requests is increased the throughput (number of completed jobs per time unit) increased. As the rate of incoming requests increases the server experiences its saturation point (peak load). This marks the point where the server is fully utilized and operating at its full capacity. The saturation point marks also the highest possible throughput. Further increases of the rate at which the request arrive will now lead to an overload or thrashing effect. The service capacity is exceeded and “an increase of the load results in the decrease of throughput” [12]. The main reasons for a server to experience thrashing are either resource contention (overload of the physical devices e.g. CPU, memory, hard drive, etc.) or data contention (locking). Since server over-load situations lead to a decline in throughput it is important to avoid them. Heiss and Wagner [12] proposed the use of adaptive load control as the means for preventing overloads. This is achieved by first determining the maximum number of parallel requests (e.g. maximum number of simultaneous consumers) and then buffering/balking the requests once the saturation point has been reached. The impact of this approach can be seen in figure 2. The darker curve shows the characteristic three phases a server can experience, under-load, saturation and over load. Using an admission control (grey curve), the trashing is avoided due to the buffering/balking of requests above peak load. Adding an admission control into an already existing Web Services system can be achieved by use of the proxy pattern. As shown in figure 3, adding a proxy that shields/hides the original provider enables a transparent admission control. The role of the proxy is to monitor the rate at which consumers issue requests. Once the request rate exceeds the capacity of the provider, a First In First Out (FIFO) queue is used to buffer the excess request. The transparent admission control is a very effective approach for handling requests bursts [12,10]. However a basic FIFO queue ignores that different requests impact the server in different way e.g. different resource utilization. Since service-oriented middleware (e.g. Web Services) tends to favor declarative communication styles (e.g. SOAP messages), it is fairly easy to analyze the server bound traffic, determine the request and estimate the impact each request will have
Transparent Admission Control and Scheduling of e-Commerce Web Services
Proxy( Admission Control)
SOAP requests
Consumers SOAP responses
127
Service
Fig. 3. Transparent Admission Control
Scheduling SOAP request
SOAP request
SOAP request
SOAP responses
SOAP request
SOAP requests
Consumers
Admission Control
Service
Waiting queue
Fig. 4. Transparent Scheduling
on the service provider. This in turn leads to the possibility of scheduling (re-ordering) of service requests [fig. 4]. Scheduling of requests opens a broad spectrum of possibilities, like maximizing service performance in terms of interactions per time unit, minimizing variance of service response times, etc. This paper focuses on SJF (Shortest Job First) scheduling as a means for optimizing average response time. SJF (Shortest Job First) is a scheduling policy which minimizes the response time of light requests, for the price of the heavier ones. All incoming service calls are put in a waiting queue and executed in the order of their size as shown in figure four. Smith [16] proved that SJF is the best scheduling policy for minimizing the average response time if in the information on the job sizes is known and all the jobs are available at time zero. The problem of minimizing average response time in the setups where the jobs are arriving over time is proven to be NP hard [13], nonetheless due to successful deployments [10,8] SJF presents a noteworthy heuristics. In this paper we limit the discussion of scheduling to SOAP encoded, synchronous RPC style, stateless Web Services. In addition rather than using a complex resource model, requests (jobs) will be characterized by the load they create on the provider (server). The SJF scheduler is highly dependent on a correlation value of predicted and observed job length - the better the correlation the better the optimality of the schedule. While a correlation close to one (perfect predictions) achieves the optimal schedule [9], a correlation close to minus one (always wrong predictions) results in the worst schedule (instead of minimizing the response time, it will be maximized). A correlation of zero (random guess, equal amounts of correct and wrong predictions) leads to a random scheduling [9]. The SJF scheduler is highly dependent on a correlation value of predicted and observed job length - the better the correlation the better the optimality of the schedule.
128
D. Dyachuk and R. Deters
3 TPC-APP To achieve a realistic and domain independent evaluation of the transparent scheduling, the TPC-APP benchmark of the Transaction Processing Performance Council TPC was chosen. According to TPC [4] the “TPC Benchmark App [3] is an application server and web services benchmark. The workload is performed in a managed environment that simulates the activities of a business-to-business transactional application server operating in a 24x7 environment..” [4]. Since there were no free test-suites available, a reimplementation of the benchmarks was implemented following the TPC specifications.
Fig. 5. Topology of TPC-APP
As shown in figure 5, the TPC-APP scenario consists of clients, a third-party service, a bookstore service and a database server. The bookstore services and the database server are hosted on different machines (application server & DB server). TPC-App also introduces a third-party service (e.g. credit card service) to simulate external parties. The application server (bookstore service) exposes eight different methods shown in table one with their distribution in the client requests (e.g. 50% of all client calls are Create Order requests). Two methods are Writes (Create Order, Change Item), one is Read/Write (New Customer, Change Payment) and three are Reads (Order Status, New Products, and Product Detail). TPC-APP dictates that each of these methods should be treated as a transaction with ACID properties. Table 1. Bookstore Service Operations Operation New Customer Change Payment Method Create Order Order Status New Products Product Detail Change Item
Percetage 1.00% 5.00% 50.00% 5.00% 7.00% 30.00% 2.00%
Transparent Admission Control and Scheduling of e-Commerce Web Services
129
The database contained all the information operated by service, like customers, orders, inventory, etc. The size of the database was 80 Mb. The inventory table was scaled to 100 000 records, and the table describing orders and clients was proportional to the number of virtual clients. In order to reduce proxy complexity component inserted between clients and application server, the underlying communication protocol was changed from the HTTPS to HTTP.
4 Experiments Our TPC-APP implementation uses JSE as the officially allowed by TPC platform. The application server is Jakarta Tomcat 5.0.28 running on JSE 1.4. Tomcat’s default configuration is set to use a load balancing package. Load balancing was disabled to ensure a consistent behavior for the experiments (it also leads to a 15 ms speedup in average on each service call). Axis 1.3 served as framework for implementing the Web Services. All Web Services were implemented in a synchronous way and use the default HTTP 1.0 as the transport protocol. MySQL 4.1.12a was selected as the database server and the Connector J3.1.12 JDBC driver was used to link the services to the database. The application server, database server and virtual clients resided on separate machines of following configuration: Pentium IV 2.8 GHz, 2Gb of RAM. The third part services used Pentium 3, 600Mhz with 512 of RAM. The machines were connected with a 100Mb LAN and used XP SP 2 as their OS. The XP performance counters were used to collect the performance data. As the main metrics we chose throughput and average response time. The average response time is the mean value of all the response times of service during the measurement interval. Throughput is here defined as the number of successful service interactions per time unit. The measurement interval is 30 minutes. To emulate the behavior of multiple clients, a workload generator is used. The settings of the client session lengths were distributed according a discrete Beta distribution with shape parameters 1.5 and 3.0. [3]. In order to keep the number of simultaneous clients constant, new clients arrive after old clients have finished their session. According to TPC-APP the client’s business logic is ignored (takes zero time). Every client starts a new service call immediately after obtaining the results from the previous ones. The load on the service provider is a result of the number of simultaneous clients. HTTP 1.0 is the used transport protocol (closing the connection after each interaction). The timeout for the connection is set to 90 seconds. The clients are simulated by the workload generator and execute a sequence of requests that is determined by invoking a random generator at runtime (the setting represent the distribution shown in table one). The process of determining calls sequences has no memory and consequently the probability of the next operation being invoked does not depend on the previous call. Two types of experiments were conducted. The first experiment is used determine if posteriori knowledge (runtime statistics) can predict with sufficient accuracy the behavior of a job. The second type of experiments was used to determine the impact of scheduling.
130
D. Dyachuk and R. Deters
5 Evaluation 5.1 Job-Size Estimation In order to evaluate the sizes of the jobs we divided all the SOAP requests into classes. SOAP requests within the same class create approximately same loads on the service. Using the average response time of the class it is possible to predict the behavior of a request.
Cumulative Percent
100.0%
Complexity 1
80.0%
2 3
60.0%
4 5
40.0%
6 7
20.0%
8 0.0%
9 6 29
0 14
93
47
15
Fig. 6. Execution time of service methods (16 clients)
Cumulative Percent
100.0%
Method
80.0%
Change Item
60.0%
Change Payment Method
40.0%
Create Order New Customer
20.0%
New Products
0.0% 46
7
9
3 34
40
6 26
0 14
3 20
63
0
ResponseTime
Order Status Product Detail
Fig. 7. Execution time of “Order status“ with different SOAP request complexity (16 clients)
Trace studies of the TPC-APP benchmark workload show that a classification according to the name of a service operation allows achieving only a (weak) 0.35 correlation between the estimated and the actual response times. As can be seen in figure 6, some operations (“Order Status”, “New Products”) have a high variation of execution time, while operations, like “Change Payment”, exhibit more stable behavior. However, if the data contained in the SOAP messages is also used for classification a (strong) the actual response times is achieved [fig. 7]. The reason for the strong correlation is an effect of using a database centric scenario in which the costs of a service call relate to the number of updates/writes performed on the database. 5.2 Admission Control and Scheduling To obtain the data of a SOAP message is necessary to perform a partial request parsing. In order to minimize scheduling overhead and avoid double parsing we located the
Transparent Admission Control and Scheduling of e-Commerce Web Services
131
proxy component into the Tomcat framework (proxy resides on the same machine as app-server). Figures 9 and 10 show the observed throughput and response time in relation to used policy PS, FIFO, SJF and number of simultaneous clients. The first observation is that even with admission control (FIFO) and SJF, a decline in throughput can be observed. The decline is a result of the thrashing due to overload of their parsing parts. The overload situation the admission control and scheduler experience and can be easily solved/eased by hosting these components on a more resource rich host. In the current setting, the proxy that performs the scheduling/admission control is residing on a host that is shared with the application server 8.
Fig. 8. Topology of the scheduled environment
The second observation is the significant performance boost FIFO and SJF achieve in overload situations. While PS degrades as expected (at ca. 41 clients) SJF and FIFO begin to degrade only at around 76 clients. Until the load reached its peak values (41 users) the buffer queue created by the admission control is mostly empty therefore all scheduling policies behave in the same manner. After the load exceeded its peak value the excess requests were buffered and the throughput was preserved. Further load growth created a situation in which there were more elements in the queue to be scheduled so SJF and FIFO began to behave different. At higher loads SJF started outperforming
Mean response time, ms
9000 8000 7000 6000 5000
SJF
4000
PS
3000
FIFO
2000 1000
1 6 11 16 21 26 31 36 41 46 51 56 61 66 71 76
0
Number of clients
Fig. 9. Throughput of the service governed by various scheduling policies
132
D. Dyachuk and R. Deters
Throughput,iteractions/s
80 70 60 50
SJF
40
PS
30
FIFO
20 10
1 6 11 16 21 26 31 36 41 46 51 56 61 66 71 76
0
Number of clients
Fig. 10. Response times of the service governed by various scheduling policies
FIFO by 20% in average. Table 2 shows that scheduling did not affect the consumption of the resources like, memory or network. Meanwhile the CPU utilization went down from 63% to 37-40% due a lesser amount of parallel jobs. In general, low values of CPU utilizations are the result of using HTTP 1.0 since establishing a new socket causes a significant delay (up to 1 sec) and does not require the processor resources. It can also be seen that the database server’s CPU load and amount of reads/writes had highest values in case of SJF [table 3]. Thus even these loads were not fully utilizing the database. The under-utilization of the database can be explained by the low frequency of database queries issued by application server. Consequently, the application server is the bottleneck. In the current setup scheduling of the service requests mostly affected the performance of the bottleneck element - the application server. Table 2. Application server loads with 36 clients Policy Original FCFS SJF CPU, % 63 40 37 RAM, MB 557 557 558 Network Bytes/sec 271220 261629 73750
Table 3. Database server loads with 36 clients Policy Original FCFS SJF CPU, % 3.44 13.8 14.21 Memory, Mb 1.45 1.4 1.39 Network, Kbytes/sec 55.97 19.39 20.90 Reading, Mb/sec 0.11 0.32 0.34 Writing,Mb/sec 0.65 0.13 0.13
As a result of scheduling, the application server was able to produce database queries faster. Thus the database server utilization (CPU, read/write operations) increased by 10% [table 3]. Applying scheduling caused an interesting effect of “resource load
Transparent Admission Control and Scheduling of e-Commerce Web Services
133
equalization” in which the usage of overloaded resources decreased, while the utilization of lesser used resources increased. Nevertheless, this is an observed effect that requires research. It is important to note, that the skew of the response time distribution was small [fig. 11] and that the difference between the slowest and the fastest request reached only the ratio of one to six. Therefore the possibility of optimizations was lower and the bigger jobs were penalized only lightly. Figures 12 describe an example of a light [fig 12.a] and a heavy request [fig. 12].b in a medium load case (11 clients). SJF to FCFS heavy jobs experienced the 2% increase of the response time, while the response time of smaller jobs shrunk up to 45%. In any case, each of these policies significantly outperformed default PS.
Fig. 11. The distribution of the service response times (without network overhead)
350 300 250 PS
200
FIFO
150
SJF
100 50 0 Change Item
(a)
1800 1600 1400 1200 1000 800 600 400 200 0
PS FIFO SJF
Create Order
(b)
Fig. 12. “Light” and “Heavy” Operations
6 Related Work Elnikety et al. [10], present a method for admission control and request scheduling for multi-tired e-commerce Web-sites with a dynamic content in. The authors use a transparent proxy, as the intermediate scheduling component between application server and database server. The method enabled overload protection and preferring scheduling SJF augmented with aging method.
134
D. Dyachuk and R. Deters
SJF scheduling has been successfully used for http requests by Cherkassova in [8]. This work proposes the use of Alpha-scheduling to improving the overall performance. Alpha-scheduling, is a modified SJF algorithm and prevents the starvation of big requests, in way similar to aging methods [10]. The job size in [10] is estimated by classifying all http requests according a name of the requested servlet. In the domain of web services admission control research has focussed on supporting QoS differentiation (providing different levels of QoS). Siddaharta et al. [15] introduce the concept of using a Smartware platform as managing infrastructure for Web Services. The Smartware platform consists of three core components namely interceptor, scheduler and dispatcher. The interceptor intercepts the incoming requests, parses them and classifies them according to user and client (device) type. Once classified the requests are subjected to a scheduling policy the scheduler determines the order of execution. Finally the dispatcher routes the request to the service endpoint. Smartware implements a static scheduling by assigning priorities to service calls according the classification of the interceptor. As a main scheduling means the Smartware authors chose a randomized probabilistic scheduling policy namely lottery scheduling [17]. Sharma et al. [14] achieve QoS differentiation using predefined business policies. The authors examined static scheduling, in which the request class is determined by the application type, device type and client (user) level. The overall priority is calculated on the base of component priorities, for example it can be a sum or a product. The problem of starvation was eliminated by applying probabilistic scheduling. However, static scheduling does not appear to be a capable for ensuring differentiated throughput in case of fluctuating loads [14]. The authors suggest augmenting the static scheduling by altering priorities at runtime using the observed throughput. The adjustments to the required throughput are done using a correcting factor that is calculated as a penalization function of a hyperbolic nature, which speeds up slow responses and slows down fast responses in compliance with each request class.
7 Conclusions This paper presents the idea of transparent scheduling of Web Services requests as a means for achieving better performance. Using the TPC-APP, a two-tier B2B application, as a benchmark we evaluated the performance gains of SJF (Shortest Job First) compared to FIFO admission control and standard PS. By simply adding a proxy between consumer and provider, it was possible to achieve a transparent scheduling and admission control that lead to significant performance improvements in overload cases. In addition, the experimental evaluation showed that even in the absence of a priori knowledge, a SJF scheduler that uses observed runtime behaviour can lead to schedules that outperform FIFO and PS, making it an attractive approach for boosting Web Services performance. The results of the experimentation indicate that transparent scheduling can be applied to Web Services as an effective and easy to implement approach for boosting performance and avoiding service provider trashing. And while SJF does penalize larger jobs, this doesn’t seem to exceed 10% which seems acceptable given the overall gains.
Transparent Admission Control and Scheduling of e-Commerce Web Services
135
8 Future Work While the results of applying scheduling are very promising it is important to note that the current work only focused on a very simplified SOA architecture. Future work in transparent scheduling of Web Services will overcome this by addressing the following issues. 8.1 Document-Style In the current work we focused on RPC-style Web Services, that exhibit the basic request/response MEP (Message Exchange Pattern). Document-style interaction supports more complex MEPs and raises new question in regards to scheduling. 8.2 Composite Services Composite Services orchestrate the functionality provided by the other services thus creating complex environment with significantly more complex behaviour. We hope that by applying scheduling to this environment it should be possible to increase services dependability, performance, etc. 8.3 Service-Level Agreements (SLA) SLAs are an increasingly important aspect of SOA. Scheduling can be used as a means for achieving this by minimizing penalties and supporting QoS contracts in critical situations. Acknowledgements. This research has been supported by NSERC grants and the equipment provided by the Canadian Foundation for Innovation (CFI).
References 1. Apache Axis, http://ws.apache.org/axis/ 2. Eclipse, http://www.eclipse.org/ 3. TPC-App - Application Server and Web Service Benchmark, http://www.tpc.org/tpc-app/ 4. Transaction Processing Performance Council (TPC), http://www.tpc.org/ 5. Visual Studio Home, http://msdn.microsoft.com/vstudio/ 6. Web Services @ W3C, http://www.w3.org/2002/ws/ 7. Box, D.: Four tenets of service orientation. Technical report, Microsoft (2003), http:// msdn.microsoft.com/msdnmag/issues/04/01/Indigo/default.aspx 8. Cherkasova, L.: Scheduling strategy to improve response time for web applications. In: Bubak, M., Hertzberger, B., Sloot, P.M.A. (eds.) HPCN-Europe 1998. LNCS, vol. 1401, pp. 305–314. Springer, Heidelberg (1998) 9. Conway, R.W.: Theory of scheduling. Addison-Wesley Pub. Co., Reading (1967) 10. Elnikety, S., Nahum, E., Tracey, J., Zwaenepoel, W.: A method for transparent admission control and request scheduling in e-commerce web sites. In: WWW 2004: Proceedings of the 13th international conference on World Wide Web, pp. 276–286. ACM Press, New York (2004)
136
D. Dyachuk and R. Deters
11. Graham, S., Davis, D., Simeonov, S., Boubez, T., Neyama, R., Nakamura, Y.: Building Web Services with Java. Sams Publishing, Indianapolis (2004) 12. Heiss, H.-U., Wagner, R.: Adaptive load control in transaction processing systems. In: VLDB 1991: Proceedings of the 17th International Conference on Very Large Data Bases, pp. 47– 54. Morgan Kaufmann, San Francisco (1991) 13. Lenstra, J., Rinnooy Kan, A., Brucker, P.: Complexity of machine scheduling problems. Ann. of Discrete Math. 1, 343–362 (1977) 14. Sharma, A., Adarkar, H., Sengupta, S.: Managing qos through prioritization in web services. WISEW 00, 140–148 (2003) 15. Siddhartha, P., Ganesan, R., Sengupta, S.: Smartware - a management infrastructure for web services. In: Proc. of the 1st Workshop on Web Services: Modeling, Architecture and Infrastructure (WSMAI 2003), Angers, France, April 2003, pp. 42–49. ICEIS Press (2003) 16. Smith, W.E.: Various optimizers for single-state production. Naval Research Logistics Quarterly (1956) 17. Waldspurger, C.A., Weihl, W.E.: Lottery scheduling: Flexible proportional-share resource management. In: Operating Systems Design and Implementation, pp. 1–11 (1994)
Part II
Web Interfaces and Applications
Life Cases: A Kernel Element for Web Information Systems Engineering Klaus-Dieter Schewe1 and Bernhard Thalheim2 1
2
Massey University, Information Science Research Centre Private Bag 11 222, Palmerston North, New Zealand [email protected] http://isrc.massey.ac.nz/ Christian Albrechts University Kiel, Department of Computer Science Olshausenstr. 40, D-24098 Kiel, Germany [email protected] http://www.is.informatik.uni-kiel.de/˜thalheim
Abstract. On a high level of abstraction a Web Information System (WIS) can be described by a storyboard, which in an abstract way specifies who will be using the system, in which way and for which goals. While syntax and semantics of storyboarding has been well explored, its pragmatics has not. This paper contributes the first step towards closing this gap. For this we present life cases, which capture observations of user behaviour in reality. We discuss the facets of life cases and present a semi-formal way for their documentation. Life cases can be used in a pragmatic way to specify a story space, which is an important component of a storyboard. Keywords: Storyboarding, pragmatics, life case, application domain description.
1 Introduction Web information systems (WIS) augment classical information systems by modern web technologies. They aim in supporting a wide variety of users with a large diversity of utilisation stories, within different environments and with desires for personal webenhanced work spaces. The development of WIS is therefore adding the user, story and interaction dimension to information systems development. So far information systems development could concentrate on the development of sophisticated structuring and functionality. Distributed work has already partially been supported. WIS have been oriented towards a support for work. Therefore, WIS require at the same time a careful development and support for the interaction or story spaces beside the classical support for the working space of users. These dimensions complicate the system development process. On a high level of abstraction a WIS can be described by a storyboard, which in an abstract way specifies who will be using the system, in which way and for which goals. In a nutshell, a storyboard consists of three parts: – a story space, which itself consists of a hierarchy of labelled directed graphs called scenarios, one of which is the main scenario, whereas the others define the details of J. Filipe and J. Cordeiro (Eds.): WEBIST 2007, LNBIP 8, pp. 139–156, 2008. c Springer-Verlag Berlin Heidelberg 2008
140
K.-D. Schewe and B. Thalheim
scenes, i.e. nodes in a higher scenario, and a plot that is specified by an assignmentfree process, in which the basic actions correspond to the labels of edges in the scenarios, – a set of actors, i.e. abstractions of user groups that are defined by roles, which determine obligations and rights, and user profiles, which determine user preferences, – and a set of tasks that are associated with goals the users may have. In addition, there are many constraints comprising static, dynamic and deontic constraints for pre- and postconditions, triggering and enabling events, rights and obligations of roles, preference rules for user types, and other dependencies on the plot. While syntax and semantics of storyboarding has been well explored, its pragmatics apart from the use of metaphors [21] has not. Pragmatics is part of semiotics, which is concerned with the relationship between signs, semantic concepts and things of reality. Main branches of semiotics are syntactics, which is concerned with the syntax, i.e. the construction of the language, semantics, which is concerned with the interpretation of the words of the language, and pragmatics, which is concerned with the current use of utterances by the user and context of words for the user. Pragmatics permits the use of a variety of semantics depending on the user, the application and the technical environment. This paper contributes the first step towards pragmatics of storyboarding. For this we present life cases, which capture observations of user behaviour in reality. The idea is that these observations give rise to both a concrete scenario for the usage of the WIS including roles and user profiles. We then derive the candidate scenarios by abstraction. Further integration and refinement of these scenarios gives rise to the story space. We deal with the problem of extracting user models in a separate article. The third component of the storyboard, the tasks, has already been subject of strategic modelling. The life cases permit the refinement and further decomposition of these tasks.
2 Related Work Most methods for web application engineering such as OOHDM [19], WebML [3], HERA [9], WSDM [6] and variants of UML [4,11] pay little or even no attention to high-level modelling. The rationale underlying our work is that this is far too little to capture strategic and usage issues of WISs. The integration of goals and soft goals into the information systems development process has been proposed by [13,7]. The detailed distinction of intentions as targets, objects, objectives and aims is based on linguistics. The integration of the temporal dimension is necessary because of the information systems context. The extension by the representational dimensions has been made in order to specify aspects of WISs. Life cases extend and formalise scenarios used in software engineering and user modelling [5]. Early work like already discovered that human computer interfaces can be developed in techniques of movie writing or play development [22]. Life cases have already been envisioned in [2]. Scenarios as defined in software engineering are examples of interaction sessions. This kind of scenario starts with an outline of the interaction, and during requirements elicitation, details are added to create a complete description of interaction. They are specified by a general description of the start and
Life Cases: A Kernel Element for Web Information Systems Engineering
141
termination states of the system, a description of the normal flow of events, and a description of concurrent events [20]. Later scenario research has been integrated into agile programming. Customer involvement is one of the requirement of agile methods. Customers should be integrated as early as only possible into the development process [1]. Both approaches are to much oriented towards systems and do not consider life cases as we do. Scenario development has been applied to development of websites in [16]. In [8] life cases were integrated into the entire software engineering process. Recently, software engineering research developed an engineering approach to simple life case. Business use cases [15] generalise the requirements shell of [14] that is based on verbal description of event/use cases, requirement types, the rationale, originator, fit criterion, customer (dis)satisfaction, supporting materials, and the history. Our life case specification is not as complex as these business use cases. They combine intention, users, actors, constraints/assumptions/scope or context, tasks, migration, risks, costs, documentation, testing, and requirements for functions, data, look and feel, usability, humanity, operating, maintenance, support, security, culture, politics, legal and performance. We concentrate on the requirements the user has as such.
3 Facets of Intention The description of intention is based on a clear understanding of aims and targets of the WIS including a specification of long-range and short-term targets, expected visitors, characteristics of this audience, tasks performed by the users, necessary and unnecessary content, and finally an understanding of restrictions of usage. Utilisation scenarios are developed on the basis of intentions. An intention specifies what one purposes to accomplish or do, i.e. one has in mind to do or bring about. It has four facets that are based on the general characteristics of WISs: Purpose Facet: The purpose of stakeholders specifies an anticipated outcome that is intended or that guides the planned actions. It may be based on two additional pieces of information: The aims specify what is intended to be attained and what is believed to be attainable. The objectives are more general than the aims. They specify something toward which an effort is directed, e.g. goals or ends of an action. The purpose is already specified at the strategic layer. It depends on the audience intended for the WIS, and is influenced by the mission that is determined by the WIS provider. Intent Facet: The intent suggests a clearer formulation or greater deliberateness. It distinguishes between the following two aspects: The targets of stakeholders specify the steps of a plan and its termination or satisfaction conditions. The object of stakeholders is related to wishes or needs of users, and specifies an effort or activity of a user in order to satisfy the need. The intent facet is related to tasks the user wants to accomplish. The intent may be ordered into major intents specifying the main uses of the system and minor intents that may be only partially of interest. Intents may be generalized to themes that represent classes of intents.
142
K.-D. Schewe and B. Thalheim
Time Facet: The time facet is used for specification of the general time restrictions such as the design, which implies a more carefully calculated plan, and the end, which stresses the intended effect of an action, often in distinction or contrast to the action or means as such. Time facets may be very general. In this case, we use occasions to represent an entire class of time frames. Representation Facet: Intentions are described or annotated through utterances, words or pictures. The word representation is related to word fields used in the strategic model. The icon representation is based on metaphors. Word fields may be specialised to concept fields discussed later in this chapter. Representation is deeply dependent on the cultural environment and the community that uses the WIS. Intentions can be supported by providing stimuli that rouse or incite activity. Intentions may be restricted by a scope. The scope allows to concentrate on the main aspects. Typical restrictions are the cultural environment, education or other profile properties of potential users, or specific time facets for utilisation of the WIS. The first two facets of intention have a general form (objective, object) and a more concrete form (aim, target). The purpose facet depends on the mission and the audience, whereas the intent facet depends on the tasks. Therefore, the first facet specifies the ‘what’, the second the ‘how’, and the third facet the ‘when’ of an intention. We may either concentrate on a more general specification or a more concrete one. The different facets may be considered separately or altogether in a condensed form. The detailed consideration is necessary, whenever a fastidious audience requires sophisticated content. High demands come into consideration due to the content provided, the community that must be satisfied, the sophisticated functionality required for the WIS, or the high attention the WIS gains. Example 1. The purpose facet is often considered to cover ‘soft intentions’ or by ‘business rules’. In an information service a user may be interested or becoming more interested in some content. The intent facet is different and mainly driven by tasks the user tries to solve. Similarly, we may derive a number of intentions a user has in mind whenever s/he visits an edutainment site: Aim: An aim of an edutainment user may be to obtain a certificate that proofs the success of learning. Aims of providers of edutainment sites might also be binding the learner to some products such as the software of a supporter of the website. Typical general aims within an edutainment site are to grant equal rights to everybody; nobody should have higher rights than other learners. Objectives: Typical objective of using an edutainment site are greater ease of learning, greater pleasure or satisfaction during leraning, and more fun. Another general objective is security and privacy. Edutainment applications also host confidential information, e.g. information on progress and errors. The learners need to be sure of the security and privacy of their data. They should be informed of the security precautions. The audience of an edutainment site may be rather unspecific or more specific such as students of a college or analysts of a bank.
Life Cases: A Kernel Element for Web Information Systems Engineering
143
The mission of an edutainment site is to support learning by providing easy-to-grasp knowledge. At the same time, the provider may be interested in increased visitorto-customer conversion rate, increased number of returning customers, and increased revenue. The intent is related to the more concrete tasks a user has in mind while visiting a WIS. Example 2. Intents come directly from analysing the needs and the demands of users. Some possible tasks a user may want to accomplish in an edutainment site are the following: – A user realises that certain knowledge or information is necessary for solving a task. For instance, an analyst of a bank wants to know what the changes are in the customer community that led to a rise of faulty credits. – A student in a college needs to know methods for analysing data. The student is interested in a data mining course that provides material on his/her educational level, permits learning the content interactively, and is comparable to the material the student is currently working with in the college. Possible intents in an edutainment site are the following: Objects: There are a number of objects such as faster task completion, successful completion of more tasks, or commission of fewer errors. Targets: In the bank analyst case, the analyst needs to know data mining methods, to capture the achievements and the disadvantages and pitfalls of such methods. The themes in the bank analyst case could be the interest in learning association methods, preprocessing of data, and prediction methods. The time facet represents general time restrictions such as the time or the interval for visits of the WIS, the time and the interval of repeated visits, or the temporality that may force a user to leave the site. The representation facet represents the flavour or atmosphere of a site, which is largely determined by the other three facets. The set of intentions we should consider may be rather large. Moreover, the faceted representation may become too difficult to manage and to satisfy. The order we may use depends on the audience and on the provider. The audience is oriented towards solving tasks. The provider has a ‘mission’. We can use these two restrictions to figure out which kind of website supports this profile, to harmonize our understanding with the corporate identity, to order the target realisation (short-term, long-term), and finally to develop an understanding how the site will look like in two years from now on. Example 3. The intention prepare for a visit is based on a number of intents such as addressing visitors beforehand for some purpose, use, or activity. It includes the aim to put the visitor of the WIS in a proper state of mind. A task associated with preparation may be to work out the details of a plan in advance. The task may be extended by putting pieces of information together into a compound form or preparing a report. The time facet may range from ‘now’ until ‘next opportunity’. Typical metaphors supporting this intention are baskets, descriptions, and cultural journals. The intention prepare for a cinema visit extends the intention of prepare for a visit by a certain content, a refinement of the time facet to the next possible visit, and a refinement of the purpose facet now targeting at entertainment.
144
K.-D. Schewe and B. Thalheim
We can combine the description of intentions in a semi-formal way as follows. The items in the list are optional except the first one. Intention space: Purpose: Aims: Objectives: Intents: Targets: Objects: Themes: Time: Design: End: Occasion: Presentation: Atmosphere: Metaphors: Based On:
intention name outcome description list of aims list of objectives outcome description list of weighted targets list of weighted objects class of intents outcome description general flow effects, termination conditions list of objectives general style guide general description of atmosphere list of metaphors tasks, audience, mission, goal
4 Life Cases Life cases allow to overcome the information overload and lost in the hyperspace syndromes typically observed for WISs. For completion of tasks users need the right kind of data at the right moment of time, in the right dose, of the right form, in the complete extent, and within the restrictions agreed upon in advance. Moreover, users are limited in their abilities for verbalisation and digestion of data, and by their habits, practices, and cultural environment. These limitations may cause intellectual overburdening of users. Most systems that require sophisticated learning courses for their exploration and utilization did not consider these limitations and did not cope with real life situations. The approach we use for avoiding overload is based on observation of real applications before developing the system. We may extract life cases from observations in reality, which are very useful source, whenever a WIS is going to be developed from scratch or is required to provide a ‘natural’ behaviour. In the latter case users are not required to learn the behaviour of the WIS. Instead, the user can continue using a ‘classical’ behavioural pattern. Example 4. As a motivating example let us consider the life case relocation of a person, which consists of – the change of basic relocation data including the possible removal of data on the old location, – the change of official documents such as the passport, – the optional change of relation enhancements such as the registration of pets, relocation of cars,
Life Cases: A Kernel Element for Web Information Systems Engineering
145
– the change of personal specific data such as family enhancements, or relationships to religious bodies, – the change of data for additional relocation announcements such as tax, insurance changes, and – specific additional tasks such as applications for housing allowances. The person acts in the role of an issuer. We observe that relocation is enhanced by the profile of the issuer, by the specific tasks related to the relocation of the issuer, by specific laws and regulations, and by advanced functionality required for associating the life case with other life cases. from
proof of moving out
to
proof of moving in
address citzen office town clerk`s office factory inspectorate
issuer
passport
public authorities
income tax card associated documents
civil servant
special names
contracters
ministery tax office statistics agenca police aliens department TV/radio
automated contracting directory companies schools religious organizations parties
documentation agency
agencies
data protection official
special support
official bodies
recipients
pseudonyms fratemity
basic changes passport
identification
birth certificate degrees
necessary documents
certificaters
certificate of authority
actors
companies
for authorizing others pension approval certificate
proofs
special documents
employment documents marriage certificate
citizenship support of organization
driving licence child identification card
relocation address tax housing benefit house owner benefit
partners
housing allowance housing programme
children social support
housing eligibility associated parties special parking permission
forwarding period forwarding address insurance agencies health insurance tv/radio
insurance bank collect charges
potential changes potential changes
move life case
forwarding mail
pets
pet tax pets registration
supply
energy,gas water,sewage phone
address school
employment employer employment office
associated life cases contracting
vehicle documents car ownership
parking card special
documents from handicaped
housing house number
private people parties, organisation
house registration
special contracts
application for registration
obligation of secrecy registration directory accepted restrictions overruled restrictions
disclosure provided reasons
foreign resident
restrictions special/exceptional
foreign temporary second home
additional taxation
rent level
Fig. 1. Facets of the relocation life case
The life case relocation in Figure 1 consists of steps such as change of address data, change of data for associated people, change of registration data for cars, pets, etc., change of specific data, e.g. data for public authority responsible for aliens, change of data for social aid, etc. These steps are bundled together due to their relationship to one person and to one life case. The associations may be represented by adhesion of different steps, e.g. representing the association of steps by a hypergraph. The life case in the example above has overall more than 150 variations in Germany. The general target is the same everywhere. The treatment is very different. We can group these variations in around 2 dozen solutions. About half of them are already supported by WIS. The integration or matching of these WIS is far from being trivial. Moreover, the life case is associated to more than a dozen other life cases. Some of those cases are still only managed through paper exchange. This life case is one of the simplest among the life cases for proliferation and syndication of sovereign rights.
146
K.-D. Schewe and B. Thalheim
4.1 The Concept of Life Case Life cases are characterized by: Observations: We are interested in the collection and assessment of behaviour relating to the specific application. This would typically involve an observation of behaviour of users in real environments, including a background check that relates usage to intentions, goals or tasks. Processes: This involves arranging all the actions observed in an application into a main logical and coherent pattern. In some case, deviations of the main pattern must be modelled through exceptions. In most cases, we can use parallel execution of independent actions. Assessment: This involves the reconstruction of the sequence of actions and specific behaviour of users. This will aid in understanding the role each individual has within the story. It assists in developing the user profile. Individual Profiles: A list of background including physical, and behavioural characteristics of individuals is conducted. This list can also be used for deriving the most appropriate interview technique we discuss below. Interpersonal Coherence: A variation in the activity will relate to variations of other life cases. Significance of time and place: The choices made also depend on mobility, surrounding, and schedules of events of interest. Characteristics of the Life Case: Individuals using a service may be grouped by characteristics. Based on this grouping a similar behaviour is observed. Experience and Skills: Individuals may have their own experience with services provided in real life and thus use different behavioural pattern of service employment. In general, life case studies are designed to produce a picture of service employment that is as accurate as possible. Determining why, how, where, when and why a service is called using what data provides a better picture for utilisation scenario. As life cases are used for quality control, we must carefully record our observations. We either use a natural language specification or a semi-formal one as described later. Example 5. Let us consider the support of hotel search within an information service. In this case we may observe the behaviour of individuals in travel agencies while seeking for hotels. We observe that in most cases search based on associations is preferred over search by main properties such as name, address or facilities. Associations may be of a large variety, e.g. convenience to reach a hotel, location in certain maps, places of interest, or events that have caused the search. Other search criteria may be bargain or bundled offers. At the same time we observe that hotel search is combined with other intentions of users such as visiting cultural institutions. It may depend on results for search of other individuals. Another example of a life case study is the booking of train tickets depending on individuals, offers of railway companies, circumstances of individuals are using trains, etc.
Life Cases: A Kernel Element for Web Information Systems Engineering
147
4.2 Life Case Development Life cases may be developed and interpreted step by step: 1. The first step during life case collection is the survey of possible application cases we might consider. The observations could have more than one meaning and may follow a variety of user-related goals. In this case we consider the most likely meaning. 2. The second step involves a deep assessment of the life cases. We extract the different execution orders, the relationship among actions, and the roles individuals play these actions. 3. The third step extracts those life case characteristics that are distinguishing features and are relevant for time and location. At the same time we search for similarities within these life cases. 4. The final step is concerned with the characterization of potential users, their behavioural patterns, and their roles in various stages of the life case. Collectively, this information will produce a picture of the life case we are intending to support by a WIS. This may produce further life cases, or may aid in reducing the amount of development. It may result in a prioritisation of life cases under consideration, assist in the linkage of potentially related life cases, assist in assessing the potential of the WIS development, provide the WIS developers with relevant leads and strategies, and keep the WIS development on track and undistracted. These life case are mapped to scenarios in the sequel. The integration of scenarios can also be based on life cases. Example 6. Let us consider life cases of an information service for a city or a region: – Attracting visitors: The information we are providing is based on some information need visitors may have. The life case follows those that we observe during marketing activities. – Inhabitants information: The life case is based on selected information chunks that may be of interest to individuals, an ordering of the corresponding available information, and a derivable personal newspaper. – Informing tourists: The life case is similar to those observed in city information centres. – Providing official city information to inhabitants: This life case follows the message board metaphor that is used for newspaper information. During the development of city information services such as the service www.cottbus.de we have made a life case analysis for city information services and detected around 30 different life cases. We can categorize these life cases by the content and information demand of individuals. We distinguish: – life cases related to tourist content for inhabitants permanently living in the city, for inhabitants temporarily living in the city, for short-term tourists and business people on short term trip, for vacationists planning a trip for more than 5 days, for teenagers with uncertain wishes, for seniors with particular interests, for week-end visitors, for visitors of highlights, festivals, etc.;
148
K.-D. Schewe and B. Thalheim
– life cases related to official content for inhabitants on search through directory of public authority, for inhabitants on search for emergency cases, for inhabitants orienting themselves before applying at public offices, for inhabitants interested in problems of city management, etc.; – life cases related to business content, e.g. for investors considering engagement in the area. For the collection and development of life cases it is normally a good idea to interview the personnel currently providing the service. Life case should be checked against sufficy. As they may have a variety of traces, we add two stages to life case analysis: Exploration: The actual life case is provided to WIS customers and incorporated into their processes. Life case exploration can be conducted in normal environments of the user such as home or work. Then users can show the actions while they are explaining their aims and targets. Apprehension: Finally, cross checking of life cases and utilisation scenario is used for correction, extension and affirmation of the developed utilisation scenarios. During life case elicitation and specification users are confronted with sketches of scenarios. We ask what users think about these conceptualisations. The feedback is used for the refinement of life cases. We may use affinity diagramming, in which case we arrange the individual conceptions we discover on a blackboard, associate them, and discuss their relationship to the general characteristics of WISs. Another technique is based on card sorting similar to the model-view-control cards used in software engineering. In this case cards represent simple life situations. Their associations are then used for organising the conceptions. These sketches can also be used for discussing deviations from the normal case and for search of exceptional life cases. Instead of directing users to specific life cases we can show them two or more alternatives for the problem solution. Life case analysis goes beyond task analysis. It is simple and easy to carry out, lightweight and straightforward to understand, and yet quite accurate in identifying the real demands users of a WIS might have. While observing real life cases and mapping them to life cases that must be supported by the WIS we carry out a user-centered development. Additionally, we may identify and resolve conflicting or competing intentions. These conflicts can be resolved on the basis of life cases by accommodating both intentions, i.e. fulfilling the stakeholder intentions and supporting the demand of users. Besides observation of real life situations life case detection can also been based on interview techniques. Research on artificial intelligence has also resulted in a number of interview techniques: – Unstructured interviews can be considered as an informal and explorative conversation on goals a user is following and on tasks a user has to complete. Unstructured interviews observe the rules that are applied to brainstorming. All interruptions should be avoided. The questions asked must be short and simple to answer and should be open-ended questions. Interview partners should share their thoughts and
Life Cases: A Kernel Element for Web Information Systems Engineering
–
–
–
–
149
experiences. There is no judgement, confrontation or condescension. Users should not be lead toward a certain scenario. Unstructured interviews should only be interrupted if clarification is required. They need consecutive feedback in order to give the interviewees the impression that the interviewer is listening. While the interviewees are talking, everything that seems to be important is written down and used for later questions. Structured interviews are based on query plans. These plans can be based on the general characteristics of WISs. We start with easy questions on data and main functions and continue with the life cases of their utilization. Context and intentions are more difficult to capture. Structured interviews are also based on features, which can be shown to users for a rating of their importance. Life cases can also be detected by observing and critically analysing products of competitors. Elicitation of life cases from existing solutions is based on specifications, results from interviews with business users, excerpts from documents and spreadsheets, analyzed messaging and transactions, knowledge on the solutions that are currently used, meta-data and context information on the utilization within the current framework, and third party information. As reasons for restrictions cannot be captured, the copying of already existing solutions can only be used in exceptional situations. Users may also use protocols of “loud thinking”, in which case they are provided with a real-life problem of a kind that they deal with during their working life, and asked to solve it. They imagine that they are solving the problem presented to them. As they do so, they are required to describe each step and the reasons for doing what they do. The transcript of their verbal account is the protocol. In this case, they work and explain their current work. These protocols can be the basis for capturing a scenario. This interview technique should be combined with other interview techniques. There are other interview techniques that might be useful for life case elicitation such as the laddering or grid techniques. In these cases, a hierarchical structure of the application domain is formed by asking questions designed to move up, down, and across the hierarchy.
4.3 Semi-formal Representation of Life Cases Although it is sufficient for life cases to be stated in natural language, we may use a semi-formal representation instead using the following template: Life case: Characterisation: Tasks: Problems: Background: Objectives: Milestones: Content consumed: Content produced: Themes:
life case name outcome description list of user tasks list of problems general characterisation list of objectives Life case flow: general graphical graph of milestones consumed content items produced content items class of intents
150
K.-D. Schewe and B. Thalheim
Figures: actors list Expected profile: general profile description Collaboration: general collaboration description Context: general context description Time: temporality limitations Place: assignment of places Legacy: names of documents WIS: general WIS context Representation: general behavior Approaches: general description of approaches This template captures the following components of life cases: Characterisation: Life cases are characterised on the basis of strategic issues, the problem statement, background and objectives for the life case, the methodology that is used for solving the life case, and by describing the basic modules that are used for solving the life case. The characterisation is harmonized with intentions, tasks and goals. Life case flow: The life case flow combines the observations we made, the processes involved, and the data which are consumed or produced. The life case flow is mapped to a scenario. Figures: We develop profiles, especially those of individuals, as part of the WIS utilization portfolio, and interpersonal coherence specifications. Context: Time and location is explicitly described for life cases. The applicability of life cases is restricted by regulations, laws, and orders. These restrictions are seldom given in an explicit form. In addition, the context of life cases is given by the provider, the intended audience, the utilization history, and the availability of data due to the technical environment. We also use this information for the context specification. Requirements: Life cases are restricted by habits, general approaches, good practices, and boundary conditions for their application. They presuppose experiences and skills of the users involved. Note that life case we intent to support by a WIS can be completely different in real life. Sometimes we need a complete reorganisation of the business activities. In this case we should not map the real life case to a suite of associated scenarios, but rather envision a better organisation of the tasks and goals and then map these to a new envisioned hypothetical life case. Let us now outline briefly how life cases can be used in a pragmatic way to specify a story space. More precisely, we just want to derive a prototypical scenario. Other scenarios may result from other life cases, and the scenarios collected this way can be integrated and then be subject to further refinement. Let us concentrate on the life case relocation again. Example 7. In the case of the relocation of a person the steps identified in Example 4 give immediately rise to scenes in a scenario, i.e. in addition to an entry scene, say start, we obtain scenes for change of address data, change of data for associated persons,
Life Cases: A Kernel Element for Web Information Systems Engineering
151
change of registration data, change of specific data, change of data for social aid. For a start we only have a single actor citizen. In principle, the visit of any of these scenes is optional, which gives rise to a classification of actors into citizens with children, citizens with pets, foreign residents, etc. The life case of a citizen with children gives rise to a scenario for the change of data for associated persons, while the life case of a citizen with pets gives rise to a scenario for the change of specific data, etc.
5 Transforming Life Cases to Requirements 5.1 Mapping Life Cases to Business Use Cases Life cases are the major element of the application domain model. They characterise something that occurs, happens or takes place in an application or actions or instances of occurring. They may apply to a happening without intent, volition, or plan. They may denote events, incidents, episodes, and circumstances within an application. They are associated with the context description, the characterisation of the kind of the system, and the user characterisation that consists of profiles of users and of description of their intention. The description framework depicted in Figure 2 supports an elegant and sophisticated elicitation of requirements and the derivation of the presentation system specification. Application Domain Description Profiles H Users
- Intentions
H H j H -
Life case
Context Scope of the system
XXX XXX XX X z 9 ? Business use Stories Portfolio cases
Fig. 2. Elements to be specified for the application domain description
The life case study combines cognitive techniques with contextual approaches, WIS utilisation analysis and traditional approaches. This combination allows to avoid the known disadvantages of the more specific elicitation approaches. Users are characterised [18] through their profile and intentions. The profile consists of the working profile, the knowledge and abilities profile and of the psychological profile. The context specification allows to characterise the entire environment of the life case. Additionally, the solution may be restricted to certain systems and platforms. The concept of business use cases [12,15] generalises the use case concept and reflects cases or case of basic tasks in the reality. Context can be specified using the solution discussed in [10]. The storyboard describes the story space that consists of all possible stories for the given application [17]. Portfolio have been formally described in [18] and consist of tasks and the supporting instruments that are necessary. In the
152
K.-D. Schewe and B. Thalheim
sequel we decompose the life cases into business use cases and extract the portfolio. The decomposition must be invertible by a composition operation. This composition operation is the kernel of basic stories that consist of the business use cases and their story flow. We may condense this application domain description by an abstraction of users into groups depending on their general profile and their main intentions. A group is called actor. Actors can be represented by persona, e.g. ‘Jack-of-all-trade’ as a representation of a business man. The persona is characterized by an expressive name, their kind of profession, their purposes and intents, their technical equipment, their behaviour, skills and profile, disabilities , and specific properties such as hobbies and habits. Context and scope my be condensed to environment. We thus might use the elements in Figure 3. Application Domain Description - Life cases Actor
Environment
XXXX XXX XX z 9 ? Business use Stories Portfolio cases
Fig. 3. The abstractions of users, profile, intention, context and scope to actors and environment
5.2 The Algebra for Life Cases Life cases in an application domain are typically interrelated with each other, partially depend on each other, partially exclude each other, and may be (hierarchically) ordered. For instance, matters for inhabitants in a city office are relocation, registration, notice of departure, registration card, residence certificate, primary place of residence, conscript, income tax card, tariff rate for services, register of residents, archive of residents, clearance certificate and other certificates such as qualification certificate, inspection certificate, and social benefit certificate. These life cases are associated with each other. At the same time time, we may consider some of them to be basic life cases and some of them constructed from basic life cases. Whether a life case is considered to be basic depends on the application. We assume that basic life cases cannot be decomposed into separate life cases. This construction mechanism should be supported by an algebra of construction operations. We generally may assume that each life case consists of a number of components. The construction results then in a combination, replacement or removal of some of these components. Components may be essential (not removable) or auxiliary (optional). Life cases may also be related with each other. We may consider a positive relationship and a negative relationship. For instance, relocation and notice of departure are not compatible. The registration card is only issued for those that are registrable. The income tax card is based on the primary place of residence. We thus use predicates that relate life cases with each other.
Life Cases: A Kernel Element for Web Information Systems Engineering
153
We prefer binary predicates or relationship among life cases. Since life cases consist of components we do not need n-ary relationship if we wish to compare life cases with each other. The life case world (A , O , P ) consists of – basic life cases A , – algebraic operations O for computing complex cases such as combination of cases, abstraction of cases by projections, quotient of cases, renaming ρ of cases, union of cases, intersection of cases, full negation ¬ of cases, and minimal negation of cases within a given context, and – predicates P stating associations among cases such as the sub-case relation , a ∃
statement whether cases can be potentially associated with each other, a state∃
ment whether cases cannot be potentially associated with each other, a statement ∃
∃
whether cases are potentially compatible with each other, and a statement whether cases are incompatible with each other. The combination of two life cases results in a life case that has all components of the two life cases. The abstraction is used for a reduction of components of a life case. The quotient allows to concentrate on those components of the first life that do not appear in the second life case. The union takes all components of the two life cases and does not combine common components into one component. The full negation allows to generate all those components that do not appear in the life case. The minimal negation restricts this negation to some given context. We require that the sub-case relation is not transitively reflexive. The compatibility and incompatability predicate are not contradicting. The potential association and its negation must not conflict. The predicates should not span all possible associations among the life cases but only those that are meaningful in a given application area. We may assume that two life cases are either potentially associated or cannot be associated with each other. The same restriction can be made for compatibility. This life case world is very general and allows to derive more advanced operations and predicates. If we assume completeness of compatibility and association predicates we may use expressions defined by the operations and derived predicates: ∃
∃
The predicate := ∧ is used for diverging cases. ∃
∃
The predicate := ∧ is used for isolated from each other cases. ∃
∃
∃
∃
The predicate := ∧ ∧ ∧ , and is used for homogenizable cases. ∃
The predicate ⊕:= ∧ is used for heterogeneous cases. Life cases are either basic or constructed cases. We use a number of representations for visualisation of the life cases within the application domain: Life case maps are graphs consisting of nodes representing life cases and edges representing the sub-case relation and the combination of cases to complex cases. Life case hierarchies are forests representing cases and their sub-case relation.
154
K.-D. Schewe and B. Thalheim
Full life case maps are graphs consisting of nodes representing life cases and edges representing the sub-case relation and the combination, abstraction, union, intersection, quotient, full negation and minimum negation of cases if these cases are used within the application. The operations introduced above form the basis for the mapping of life cases to business use cases, to stories and to portfolio. The extraction of business use case from life cases is supported by five operations that are easing the task of efficiently acquiring relevant requirement prescriptions and of documenting them: 1. Life case projection narrows or scopes the life case to those parts (entities or concepts, axioms or invariants relating entities, functions, events, and behaviours) that are of concern for the business use cases. 2. Life case instantiation lifts the general cases to those that are of interest within the solution and instantiates variables by values which are fixed for the given system. 3. Life case determination is used for selecting those traces or solutions to the problem under inspection that are the most perspective or best fitting for the system envisioned. The determination typically results in a small number of scenarios for the life cases that are going to be supported. 4. Life case extension is used for adding those facets that are not given by the life case but are given by the environment or by the platforms which might be chosen or that might be used for simplification or support of the life case (e.g., additional data, auxiliary functionality). 5. Life case are often associated, adjacent, interact or fit with each other. Life case join is used to combine life cases into more complex and combined cases that describe a complex solution. Business use cases prescribe the “operating” part of the life cases for actors and the supporting system. Typically they are representing only one task or problem. Therefore, we determine which part of the life cases is going to be left out, which parts are emulated, and which solution is going to be preferred. The stories combine these business use cases to flows of activities. The combination is determined though the application of the five operations. The application of these operations allows to extract which subcases, which functionality, which events and which behaviour is shared among the business use cases. These shared facilities provide cross-cutting concerns among all business use cases and the exchange activities or scenes in the stories. They also hint on possible architectures of information systems and on separation into candidate components. For instance, entity sharing describe which information flow and development can be observed in the application. Functionality may be either shared by several components or may be a part of the user interface. The profiles are specifying tasks and obligations or permissions of users of the system. These profiles can be extracted from the life cases by application of these operations. The interaction between the users and the system can directly be mapped to message or life sequence charts. Additionally, we can extract quality properties. This information is later enhanced by computer requirements such as performance, dependability, maintenance, platform, and documentation requirements.
Life Cases: A Kernel Element for Web Information Systems Engineering
155
5.3 Mapping Life Cases to Requirements We can now map the three results of application domain description to properties typically used for requirements prescription. Software engineering has divided properties into functional and non-functional properties, restrictions and pseudo-properties. This separation can be understood as a separation into essential properties and non-essential ones. Functional and non-functional properties require prescription of main data structures, of the main functionality, of control facilities, and of collaboration of components of a system envisioned. Figure 4 depicts the mappings of elements of the application domain description to elements used in requirements engineering. The storyboard can be understood as a step-wise or scene-wise prescription of how a particular actor interacts with the system. Stories
Business use cases
Portfolio
Non-functional requirements
Functional requirements
XX XX XXX XXX X X XXX XXX 9 9 z X z? X ? ?
Requirements Prescription
Storyboard
Fig. 4. The use of application domain information for requirements elicitation and analysis
6 Conclusions In this paper we introduced the concept of life cases as a contribution to supporting pragmatics of storyboarding, which closes a gap in our development methodology. The general idea is to start from real life observations and to characterize them in a semiformal way. This gives rise to prototypical scenarios, tasks, roles and user profiles by abstraction. Several of such prototypes can then be integrated and refined to obtain the desired storyboard. The concept of life cases is new and original. It has already successfully been applied in our web information system projects, e.g. for information services (e.g., www. cottbus.de), for edutainment systems (e.g., DaMiT), and for community services (e.g., SeSAM). Use cases in UML [4] and business use cases [15] share some similar intentions, but are far too simplistic to capture the same information as the novel life cases. However, life cases still contribute only a part of storyboard pragmatics. They have to be complemented by user models, which should give rise to a deeper understanding of actors, and contexts. Both topics will be addressed next and published in due time.
References 1. Ambler, S.W.: Agile Modeling: Effective Practices for eXtreme Programming and the Unified Process. John Wiley & Sons, Chichester (2002) 2. Carroll, J.M. (ed.): Designing Interaction: Psychology at the Human-Computer Interface. Cambridge University Press, Cambridge (1991)
156
K.-D. Schewe and B. Thalheim
3. Ceri, S., Fraternali, P., Bongio, A., Brambilla, M., Comai, S., Matera, M.: Designing DataIntensive Web Applications. Morgan Kaufmann, San Francisco (2003) 4. Conallen, J.: Building Web Applications with UML. Addison-Wesley, Boston (2003) 5. Courage, C., Baxter, K.: Understanding your users: a practical guide to user requriements methods, tools & techniques. Morgan Kaufman, Boston (2005) 6. De Troyer, O., Leune, C.: WSDM: A user-centered design method for web sites. In: Computer Networks and ISDN Systems – Proceedings of the 7th International WWW Conference, pp. 85–94. Elsevier, Amsterdam (1998) 7. Giorgini, P., Mylopoulos, J., Nicchiarelli, E., Sebastiani, R.: Reasoning with goal models. In: Spaccapietra, S., March, S.T., Kambayashi, Y. (eds.) ER 2002. LNCS, vol. 2503, pp. 167–181. Springer, Heidelberg (2002) 8. Harel, D., Marelly, R.: Come, Let’s play: Scenario-based programming using LSCs and the play-engine. Springer, Berlin (2003) 9. Houben, G.-J., Barna, P., Frasincar, F., Vdovjak, R.: HERA: Development of semantic web information systems. In: Cueva Lovelle, J.M., Rodr´ıguez, B.M.G., Gayo, J.E.L., del Pueto Paule Ruiz, M., Aguilar, L.J. (eds.) ICWE 2003. LNCS, vol. 2722, pp. 529–538. Springer, Heidelberg (2003) 10. Kaschek, R., Schewe, K.-D., Thalheim, B., Zhang, L.: Integrating context in conceptual modelling for web information systems, web services, e-business, and the semantic web. In: Bussler, C.J., Fensel, D., Orlowska, M.E., Yang, J. (eds.) WES 2003. LNCS, vol. 3095, pp. 77–88. Springer, Heidelberg (2004) 11. Lowe, D., Henderson-Sellers, B., Gu, A.: Web extensions to UML: Using the MVC triad. In Conceptual Modeling – ER 2002. In: Spaccapietra, S., March, S.T., Kambayashi, Y. (eds.) ER 2002. LNCS, vol. 2503, pp. 105–119. Springer, Heidelberg (2002) 12. Maciaszek, L.: Requirements analysis and design. Addison-Wesley, Harlow, Essex (2001) 13. Mylopoulos, J., Fuxman, A., Giorgini, P.: From entities and relationships to social actors and dependencies. In: Laender, A.H.F., Liddle, S.W., Storey, V.C. (eds.) ER 2000. LNCS, vol. 1920, pp. 27–36. Springer, Heidelberg (2000) 14. Robertson, J., Robertson, S.: Mastering the Requirements Process. Addison-Wesley, Reading (1999) 15. Robertson, J., Robertson, S.: Requirements-Led Project Process. Addison-Wesley, Reading (2006) 16. Rosenfeld, L., Morville, P.: Information Architecture. O’Reilly, Cambridge (1998) 17. Schewe, K.-D., Thalheim, B.: Conceptual modelling of web information systems. Data and Knowledge Engineering 54, 147–188 (2005) 18. Schewe, K.-D., Thalheim, B.: Pragmatics of storyboarding for web information systems: Usage analysis. Int. Journal Web and Grid Services 3(2), 128–169 (2007) 19. Schwabe, D., Rossi, G.: An object oriented approach to web-based application design. TAPOS 4(4), 207–225 (1998) 20. Sommerville, I.: Software Enginering, 7th edn. Addison Wesley, San Francisco (2004) 21. Thalheim, B., D¨usterh¨oft, A.: The use of metaphorical structures for internet sites. Data & Knowledge Engineering 35, 161–180 (2000) 22. Vale, E.: The technique of screen and television writing. Simon and Schuster, New York (1982)
Finding Virtual Neighbors in 3D Blog with 3D Viewpoint Similarity Rieko Kadobayashi National Institute of Information and Communications Technology 3-5 Hikaridai, Seika-cho, Kyoto 619-0289, Japan [email protected]
Abstract. We propose a new mechanism for viewing entries of the 3D Blog system that enables users to find “virtual neighbors” who may have similar interests. The 3D Blog system allows a user to interact with a 3D model, annotate any spot on the model, and publish the annotations in a blog so that the user can easily communicate with other people and share interests and idea about 3D models of interest to the user. Since each annotation is saved as a blog entry with 3D viewpoint information, i.e., the user’s viewpoint and view direction as well as 3D information on annotated points, the system can detect similar entries using similarity of the 3D viewpoint information. Users can easily and intuitively compare the similar entries using the “playback” function which renders the 3D scene in the similar entries. Keywords: 3D Blog, 3D viewpoint information, Community support, Blog search, TrackBack, Viewpoint similarity.
1 Introduction The number of 3D Computer Graphics (CG) models available on the Internet is rapidly increasing due both to strong demand for 3D models in such domains as online catalogues and cultural heritage sites and to the advancement of digitizing and CG technology. For example, in the cultural heritage sector, there is an increasing demand to be able to record and visualize cultural heritage items in 3D CG models because these models have many advantages over traditional recording and presentation media such as 2D drawings. In online catalogues, 3D models can provide consumers with more detailed information and interactivity than photographs can. This means that users can change points of view as freely as if they were looking at an object in a real space, and change the texture or color of a 3D model as many times as they want to test how its appearance changes, which may actually be more difficult in real space. The number of blog users (bloggers) is also rapidly increasing. Blogs are now a very important and widespread medium for people to express and exchange ideas. One of the key features of a blog is that it allows readers to comment on blog entries, meaning that the owner of a blog site and its readers can easily communicate with each other. This bidirectional communication is facilitated by “TrackBack” mechanism which some blog systems use. J. Filipe and J. Cordeiro (Eds.): WEBIST 2007, LNBIP 8, pp. 157–168, 2008. c Springer-Verlag Berlin Heidelberg 2008
158
R. Kadobayashi
However, the use of 3D models is still restricted. Many 3D model applications provide users with some interactivity but do not provide users with a communication function for using the models. Assume that you are browsing an online catalogue and you like the design of a particular object. You may need to write a long sentence that explains why you think the design is excellent or take a snapshot of your computer display of the 3D model of the object if you want to recommend it to your friend. If the online catalogue supports communication using 3D models, the task is much easier, and you may only need to send a link to a 3D view of the object in a particular orientation and a comment to your friend. Another example in the cultural heritage sector arises when people visit a museum. Visitors can usually share the experience with their family, friends, or schoolmates, watching how they interact with exhibits. This observational or social learning is thought to be very important [1]. Sharing knowledge, interests, impressions, and experiences can deepen each person’s understanding and appreciation of the exhibits. In addition, there are a lot of hands-on exhibits in real museums that enable visitors to interact, which facilitates their understanding. However, existing 3D applications support only limited sharing and interactivity. This is especially true of applications for the general public. Experts can interact with 3D models using proprietary or expensive commercial software, but the general public is allowed only to view them passively, e.g. to see still images or animation in most cases. The experts may convey their interpretation of cultural heritage via 3D visualization, but the general public does not have any such opportunity. Of course, there are 3D applications targeting the general public such as virtual 3D museums. For example virtual Leonardo provides on-line chat functions so that users can communicate with each other [2]. The main problem in virtual 3D museums, from the viewpoint of communication and collaboration, is that users need to be in a virtual museum at the same time. Practically, this may be an overly restrictive requirement, and other means of collaboration should be provided [3]. To overcome these problems, the 3D Blog system has been proposed. This system allows a user to interact with a 3D model, annotate any spot on the model, and publish the annotations in a blog so that the user can easily communicate with other people and share interests, ideas, and questions about 3D models of interest to the user [4]. One of the main features of a 3D blog is that each annotation is saved as a blog entry with 3D information on the user’s viewpoint. Readers of a 3D Blog can reconstruct the exact same 3D scene the blogger saw when reading the blog entry, perhaps helping the readers to understand it better. In this paper, we propose a new mechanism for viewing 3D Blog entries that enables users to find similar entries easily and that intuitively calls their attention to the similarities among entries by using 3D models. It also enables them to find people who may have similar interests and to promote emergence of communities of people who share interests. The rest of this paper is organized as follows. Section 2 introduces the 3D Blog system and describes 3D viewpoint information. In Section 3, the algorithm for finding similar entries and the mechanism that links them are described. In Section 4 the algorithm for playing back similar entries is explained. Section 5 discusses the features of
Finding Virtual Neighbors in 3D Blog with 3D Viewpoint Similarity
159
Entry Snapshot
List of 3D models
Fig. 1. Top page of a 3D Blog. Annotations to 3D models are listed as blog entries in reverse chronological order. Each entry has a snapshot of the 3D scene the user saw when annotating.
the 3D Blog system that make it a very useful communication tool. Section 6 concludes the paper.
2 The 3D Blog System 2.1 Features 3D Blog enables users to interact with a 3D model, annotate any spot on the 3D model, and publish the annotations as a blog. It does not require any expensive proprietary software to make annotations or read blog entries. It only requires a 3D model viewer that runs as a browser plug-in for interacting with 3D models. The current 3D Blog system uses lily [5] as a blogging tool and Cortona VRML Client [6] as a viewer for the 3D models. Lily is a Ruby implementation of blosxom [7] written in Perl. Users need to download and install the plug-in, i.e., Cortona VRML Client, to their web browser (we are currently using Internet Explorer only) before they start 3D blogging. The main feature of 3D Blog is that it uses the 3D viewpoint information of the 3D scene that the user (blogger) sees when adding an annotation. The 3D viewpoint information is composed of viewpoint and view direction, which are expressed as 3D coordinates, and a 3D vector. The 3D viewpoint information, as well as the position of an annotated spot, is automatically added to the annotation by the system and stored in the database. It is then used to reconstruct the same 3D scene the blogger saw when a
160
R. Kadobayashi
reader reads entries. This is thought to greatly help readers to understand the content of the entries. The viewpoint is the point from which a user looks at the point being annotated. The view vector is a vector from the viewpoint to the point being annotated. The view direction is derived by normalizing the view vector. In implementation, these items are altered by the parameters specific to a computer graphics software. For example, in VRML, which we used in our system, the viewpoint and the view direction are expressed as parameters “translation” and “rotation” of a camera. This system is based on the assumption that impressions may change if the user’s point of view changes, even if the user looks at the same thing. The resulting design may enable users to convey their feelings more accurately if they can show the object as it looked when they saw it. The 3D viewpoint information is also useful for searching [8,9] and in the 3D Blog system is used to find similar entries. When a new entry is posted, the system detects similar entries based on a threshold set by users and sends links from the new entry to similar entries. We call this mechanism “automatic TrackBack.” It helps users find other users with similar interests and create communities and facilitates communication among users. To summarize, by embedding 3D viewpoint information into blog entries, the system enables users to easily understand the idea of blog entries by playing back a viewpoint of the annotated point, finding similar entries, and communicating with similar users, which facilitates forming communities. 2.2 Writing 3D Blog Entries When a 3D Blog user logs in, the top page of the user’s blogsite will open and show recent entries in reverse chronological order, as in ordinary blogsites (Figure 1). By selecting a 3D model from the list on the left-hand side of the window and clicking on the model’s name, the 3D model viewer will appear at the top of the list of entries enabling the user to browse and interact with the model in the walk-through, fly-through, or study mode and to add notes about the model (Figure 2). When the user clicks on a spot that he/she wants to annotate, a small ball, i.e. a marker, appears and covers the spot in the 3D model viewer. The user types an appropriate title and a comment in the dialogue box below the 3D model viewer and then clicks the ‘post’ button. Information, such as the position of the annotated area, the viewpoint of the virtual camera which is treated as that of the user, the date and time, the snapshot of the 3D model with the marker, and the annotation (i.e., the title and comments) are converted into a blog entry and stored in the blog database. The annotation is then added to the top of the entries of the user’s blog site as the latest entry. Users can annotate a 3D model as often as they desire. All annotations are automatically added to the blog site as separate entries. The user can change 3D models by clicking on a model’s name and continuing to blog on a different model.
Finding Virtual Neighbors in 3D Blog with 3D Viewpoint Similarity
161
3D model viewer
Text input box Entries on this 3D model
Fig. 2. Typical page layout for interacting with and annotating 3D models. 3D model viewer is displayed on top of entry list.
2.3 Reading 3D Blog At first glance, the top page of a 3D blogsite looks similar to the top pages of an ordinary blog, but there is a big difference. Every entry in the 3D blog includes information about the 3D viewpoint, i.e., the viewpoint from which the 3D model was annotated and about the spot to which the annotation was added. The snapshot included in each entry is an anchor to the 3D scene and, once it is clicked by a reader, the 3D model viewer pops up to render the 3D scene using the 3D viewpoint information while at the same time enabling the reader to read the entry (Figure 3). When rendering the 3D scene, the system first shows the 3D model in a default orientation and then changes the orientation of the model so that the 3D scene is exactly the same as the blogger saw. This animation brings to the reader’s attention how the blogger changed viewpoints when annotating. This is a great help to readers in understanding the content of entries, because it enables them to visually confirm it. Readers can add comments to an entry by clicking the ‘comment’ button at the bottom of the entry to open a dialogue box. Other basic functions such as viewing old entries and searching entries are also available.
3 Finding Similar Entries The 3D Blog system uses the 3D viewpoint information to find similar entries and to render 3D scenes and calls user attention to these similar entries by automatically
162
R. Kadobayashi
3D model viewer
Anchor to the 3D scene
Fig. 3. 3D model viewer pops up when a snapshot of an entry is clicked since a javascript is embedded as an anchor to the 3D scene. The user can read the entry and confirm how the 3D model looks.
(1) Targets are similar.
(2) Viewpoints are similar.
(3) Directions are similar.
(4) Gaze is similar.
Fig. 4. Similarity measures
Finding Virtual Neighbors in 3D Blog with 3D Viewpoint Similarity
163
setting TrackBack links. There is a blog service that connects entries that share the same keywords [10], i.e., semantic similarities. In contrast, the 3D Blog system uses the physical perspective similarities. In the current implementation, four measures are used to detect similar entries: target, viewpoint, direction, and gaze. Target is the distance between annotated points while viewpoint is the distance between viewpoints. Similarity in direction is obtained by cosine similarity of view direction vectors. Since the value of cosine theta is not linear, it is not reasonable to expect users to set the threshold. As a result, the degree of the angle between two vectors is used as a measure and users set the threshold from 0 to 180 degrees. Gaze is a combined measure of target and viewpoint. If target is less than or equal to the threshold and viewpoint is less than or equal to the threshold, the entries are similar in terms of gaze. Figure 4 depicts examples that are similar for each measure. Usually the TrackBack link is used by a blogger who writes an entry referring to another blogger’s entry to explicitly notify the other blogger of the reference. In the 3D Blog system, the system also automatically sends TrackBack to entries in the database if it finds similar entries as new entries are written. The system checks the similarities between each new entry and the stored entries whenever a new entry is posted. Then it makes the TrackBack links from the new entry to the stored entries. This means the latest entry by any user of the 3D Blog system has no automatic TrackBack links. The 3D Blog system includes the information about the similarity measures, i.e., the similarity score, as well as such information as the title and URL of the blog, which is normally included in the TrackBack ping [11]. The blogger’s own entries are excluded from the similarity check since it is unremarkable that entries by the same blogger tend to be similar. This automatic TrackBack mechanism helps 3D Blog users find similar entries and others who may have similar interests or perspectives. Users can set the threshold according to their preference. For example, some people may want to find people who have similar interests and set a low threshold for the distance between annotated spots. Others may want to find people who have a similar (physical) perspective and set that distance threshold low. 3.1 Finding Similar Entries: A Scenario Assume that user A has written an entry about a 3D model and the entry is stored in the database (see (1) in Figure 5). After user B writes an entry about the same 3D model (see (2) in Figure 5), the system checks for similar entries in the database. In this case, the system finds that an entry written by user A is similar to the entry that user B has just posted because the distance between the annotated spots of the two entries is very small and is less than the threshold specified by user A. It then sends TrackBack to user A’s entry so that user A will notice user B’s entry (see (3) in Figure 5).
4 Automatic Playback of 3D Blog Entries The 3D Blog system enables a user to see similar entries in playback mode as well as in list mode. The list mode, which is shown in Figure 6, looks similar to many blog pages.
164
R. Kadobayashi
User A
(1)
(2)
User B
(3) TrackBack
Fig. 5. Automatic TrackBack is conducted by the system when similar entries are found. Users may easily find other users with similar interests.
Similarity measures Target Viewpoint Direction Gaze
Similar entries
Score target = 0.22 viewpoint = 9.1 direction = 71.92 gaze = 9.1
Fig. 6. List display of similar entries. Entries are sorted in ascending order of the score of the measure selected by the user. In this case entries are sorted using the score of “target” measure.
Finding Virtual Neighbors in 3D Blog with 3D Viewpoint Similarity
165
Markers
Selected user name
Fig. 7. All markers placed by a selected user in similar entries are shown when “show markers” button is clicked. Users can easily grasp the distribution of annotated points.
The user’s entry is shown on top of the blog page and similar entries written by other bloggers are shown below the user’s entry. Users can choose the sort key from the four measures. For example, if target is chosen, the entries are sorted in ascending order of the target score. In the playback mode, by contrast, the user can playback the 3D scene in each similar entry so that she or he will notice which spots on a 3D model are of interest to other users and which of those spots are annotated by other users. First, the user chooses “user name” from the pull down menu located just below the 3D model viewer. This menu shows the names of those who have made entries similar to the user’s. Second, the user chooses a function by checking either “show markers” or “play views.” The “show markers” function shows all the markers from similar entries written by the selected user, as shown in Figure 7. Playback of the 3D Blog entries in “show markers” mode thus helps users to visually grasp the distribution of other user’s interests. If the user selects the entries of another user from the pull down menu, the user can graphically see which spots she or he annotated. Alternatively, the “play views” function renders all the 3D scenes of similar entries one by one, smoothly changing the viewpoint as shown in Figure 8. This helps users see which other users saw and annotated the 3D model and how the viewpoint changed spatially. The algorithm for “play views” is as follows: First the system collects all of a particular user’s similar entries for a particular 3D model and then categorizes them by writer. Finally the system sorts the entries in reverse chronological order by writer.
166
R. Kadobayashi
Initial scene of a selected 3D model
3D scenes of similar entries
Marker
Playback button User list
Fig. 8. Playback of 3D Blog entries. Choose a user from pull down menu and then click the “play views” button. The system automatically shows the 3D scenes of the entries in sequence in the 3D model viewer.
User B's Blog
B1 User A's Blog
B2 A1 B3 A2
B4
A3
B5
A4
User C's Blog
C1 A5 C2 C3 C4 C5
Fig. 9. Relation of similar entries. In this case, user A can notice that there are links between A’s entries and B’s entries or between A’s entries and C’s entries.
Finding Virtual Neighbors in 3D Blog with 3D Viewpoint Similarity
167
Figure 9 shows an example of a relation among entries. User A wrote five entries about a particular 3D model, and three of them have automatic TrackBack links from users B and C’s entries. Although entry B2 has two links to user A’s blog, it is treated as one link. In this case, user A can choose user B or user C in addition to himself from the pull down menu, and playback user B’s entries B1, B2, B4 or user C’s entries C2, C3, C5 or his or her own entries A1, A2, A3, A3, A5 in the 3D model viewer.
5 Discussion The most essential feature of the 3D Blog system is that it uses 3D viewpoint information of a user. In other words, the system places importance on from where the user observes the 3D model as well as on to which the user has an interest. Considering a way of asking in our daily life, it is quite often that we ask a question like “what is that?” when we do not know the name of what we are looking at. Those who are talking with you can understand what is of interest to you since they can share your viewpoint and follow your glance. It is, therefore, 3D viewpoint-based search is very natural way to find things of interest. In contrast, information about what a user looking at in the 3D Blog system is similar to a keyword in keyword-based search. If a user uses keyword-based search system, the user needs to know keywords or choose keywords from a list to formulate appropriate queries. This requires the user to have knowledge a priori about what he/she will search while 3D viewpoint-based search does not require such assumption. The automatic playback function enables users to take advantage of the 3D viewpoint information easily and intuitively. As the system automatically changes the 3D scene according to the viewpoint of entries, users do not need to struggle with the 3D model to recreate the 3D scene and therefore they can concentrate on understanding similarities among entries written by other users. Although the 3D Blog system has many advantages, there are many topics that need further research and development to make it fully functional. Development of an excellent user interface for 3D models, especially for annotating and comparing 3D models, is crucial. It is difficult, intuitively and otherwise, to select an arbitrary part of a 3D model, but this is essential when adding annotation. In addition, easy-to-use interfaces that compare 3D models in a single viewer should be provided since comparing objects is fundamental to research and discussion. Functions for analysis are also important in terms of community support tools. The “show markers” function can provide an overview of a spatial distribution of annotated points. Users can know which areas are of interest to community members while they cannot know why the areas are of interest. That is functions to support semantic analysis as well as physical analysis are needed.
6 Conclusions We discussed the automatic playback function for 3D Blog entries. This function is useful especially when viewing similar entries that the 3D Blog system automatically finds
168
R. Kadobayashi
based on 3D viewpoint information since users can easily and intuitively understand the similarities among entries by viewing 3D scenes. The 3D Blog system uses 3D viewpoint information to display blog entries and to find similar entries and users. With this tool, people can easily discuss a particular topic using 3D models. The automatic playback function expands this advantage. Although further investigation is needed on additional functions of the system to support user communication and knowledge sharing via 3D blogging, we believe 3D blogging with the automatic playback function has great potential and can be applied to many domains.
References 1. Falk, J.H., Dierking, L.D.: Learning from Museums: Visitor Experiences and the Making of Meaning. AltaMira Press (2000) 2. Paolini, P., Barbieri, T., Loiudice, P., Alonzo, F., Gaia, G., Zanti, M.: Visiting a Museum together: How to share a visit to a virtual world. In: Proceedings of Museums and the Web 1999. Archives & Museum Informatics, pp. 27–35 (1999) 3. Barbieri, T., Paolini, P.: Cooperative Visits for Museum WWW Sites a Year Later: Evaluating the Effect. In: Proceedings of Museums and the Web 2000. Archives & Museum Informatics, pp. 173–178 (2000) 4. Kadobayashi, R.: Automatic 3D blogging to support the collaborative experience of 3D digital archives. In: Fox, E.A., Neuhold, E.J., Premsmit, P., Wuwongse, V. (eds.) ICADL 2005. LNCS, vol. 3815, pp. 109–118. Springer, Heidelberg (2005) 5. Lily, http://lily.sourceforge.jp/ 6. ParallelGraphics. Cortona VRML Client, http://www.parallelgraphics.com/products/cortona/ 7. Blosxom, The zen of blogging (2006), http://www.blosxom.com/ 8. Kadobayashi, R., Tanaka, K.: 3D viewpoint-based photo search and information browsing. In: Proceedings of the 28th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 621–622. ACM, New York (2005) 9. Fujita, H., Arikawa, M.: A ubiquitous photo mapping considering users’ lines of sight. In: Proceedings of International Workshop on Ubiquitous Data Management (UDM 2005), pp. 71–77. IEEE Press, Los Alamitos (2005) 10. hatena, Hatena diary keyword (2007), http://d.hatena.ne.jp/keyword/ 11. SixApart. Trackback technical specification, http://www.sixapart.com/pronet/docs/trackback spec
Model Driven Formal Development of Digital Libraries Esther Guerra1, Juan de Lara2 , and Alessio Malizia1 1 2
Dep. Computer Science, Universidad Carlos III, Madrid, Spain [email protected], [email protected] Dep. Computer Science, Universidad Aut´onoma, Madrid, Spain [email protected]
Abstract. This paper shows our model-driven approach for the formal construction and validation of Digital Libraries (DLs). We have defined a Domain Specific Visual Language (DSVL) called VisMODLE, which allows the description of a DL using five different viewpoints: services, behaviour, collections, structure and society. From a meta-model based description of the different viewpoints, we have generated a modelling environment for VisMODLE. We have provided the environment with a code generator that produces XUL code for the DL’s user interface and composes the application using predefined components that implement the different services. Moreover, we have also added validation and simulation capabilities to the environment. Using the behavioural models (statemachine based), we can visually animate the system. In addition, the combined behaviour of actors and services can be transformed into a Petri net for further analysis. Keywords: Digital Libraries, Model Driven Development, Formal Methods, Meta-modelling, Graph Transformation.
1 Introduction The concept of Digital Library (DL) seems hard to be completely understood and evades definitional consensus. Licklider [10] visualized a collection of digital versions of the worldwide corpus of published literature and its availability through interconnected computers. More recently, a Delphi study [9] of DLs coalesced a broad definition: organized collection of resources, mechanisms for browsing and searching, distributed networked environments, and sets of services objectified to meet users’ needs. The Presidents Information Technology Advisory Committee (PITAC) Panel on DLs treats them as the networked collections of digital texts, documents, images, sounds, scientific data and software, that make up the core of today’s Internet and tomorrow’s universally accessible digital repositories of human knowledge [14]. Underlying these definitions there is the consensus agreement that DLs are fundamentally complex due to its inherently interdisciplinary nature. They are usually built from scratch using specialized architectures that do not benefit from previous DLs and software design experiences. The lack of formal models leads to branching efforts and has made interoperability (in both metadata and software levels) a crucial problem in the DL field. From the beginning of computer science, software engineers have sought methods to increase the productivity and quality of applications. A means to achieve such goal J. Filipe and J. Cordeiro (Eds.): WEBIST 2007, LNBIP 8, pp. 169–183, 2008. c Springer-Verlag Berlin Heidelberg 2008
170
E. Guerra, J. de Lara, and A. Malizia
is to increase the level of abstraction of system descriptions. In Model-Driven Software Development (MDSD) [15], models are the primary asset, from which the code of the application is generated. The idea is to capitalize the knowledge in a certain application domain by providing developers with DSVLs describing the domain concepts. DSVLs are less error-prone than other general-purpose languages and easier to learn because the semantic gap between the user’s mental model and the real model is smaller. Thus, from high-level, possibly visual models, a number of platform artifacts are generated which sometimes account for the 100% of the final application. Since the code generation process is automated from the models, there is a strong need to validate and verify them. In this respect, techniques for simulating the models (i.e. performing an animation showing their operational semantics) as well as to transform them into formal semantic domains for further analysis are of great interest. Our work applies MDSD techniques to the DL domain. In [11] we presented a DSVL called VisMODLE oriented to the description of DLs, and built a modelling tool for it with an integrated code generator. VisMODLE is visual, which makes easier for people to learn and interpret; it is domain-specific, leaving less room for misunderstanding; and is formal, making possible the simulation and validation of the models. It allows the specification of the different aspects of a DL using four dimensions (or diagrams): services, collections, structure and society (i.e. interactions of services and actors). In this work, VisMODLE is extended with a new type of diagram to specify behaviour (based on state machines). The operational semantics (i.e. simulator) of the new diagram type is formalized by using graph transformation [5]. This simulator allows a visual animation of the models (i.e. seeing messages being produced/consumed by actors and services) in order to validate the DL design and understand its behaviour. We have also designed a transformation from VisMODLE to Petri nets [13], a formal notation useful to specify concurrent and distributed systems. Its analysis techniques allow investigating system properties such as deadlocks, state reachability and invariants. Paper Organization. Section 2 introduces VisMODLE. Section 3 shows our MDSD approach for generating DLs. Sections 4 and 5 describe the simulation and analysis techniques we propose, and its application to VisMODLE. Section 6 presents related work, and section 7 ends with the conclusions and future work. As a running example, we develop a simple DL for a university library.
2 VisMODLE, a DSVL for Digital Libraries VisMODLE is a DSVL for the Visual MOdelling of Digital Library Environments. A preliminary version of the language was presented at [11], but we have added a new diagram type for expressing behaviour. In this way, the specification of a DL in VisMODLE encompasses now five complementary dimensions: multimedia information supported by the DL (Collection Model); how that information is structured and organized (Structural Model); the services of the DL (Service Model); the societies of actors and services that interact to carry out the DL behaviour (Societal Model); and the individual behaviour of actors and services (Behavioural Model). Figure 1 shows the complete meta-model.
Model Driven Formal Development of Digital Libraries
states
StateMachine
*
State * +name:String
+name:String {keyword}
171
+isInitial:Boolean *
behaviour
transition
<< Enum >>
+event:String +action:String
ServiceStatus
<< Enum >> StructType
+wait:int +nowait:int
+metadata:int +layout:int +relation:int
<< Enum >> Operation
<< Enum >> ActorStatus
BehaviouralElement +name:String {keyword}
reply *
Actor
*
+status:ActorStatus +events:String[]
on_event +event:String +response:String
*
Service
*
+sync:ServiceStatus +send_messages:String[] +reply_messages:String[] +response_messages:String[] *
*
*
+message:String
+get:int +add:int +del:int
+none:int +active:int +inactive:int +sleeping:int
send +message:String
node *
*
0..1
Struct * +id:Integer +type:StructType +value:String
link_type 0..1
Collection
operation
+name:String {keyword} +documents:String[]
−op:Operation
Fig. 1. The VisMODLE Meta-model
Collections are sets of static (e.g. text) and dynamic (e.g. video) elements. Our running example includes a collection model (not shown for space constraints) with a collection (called Library) of two documents: long1.pdf and long2.pdf. The only relevant entity for this kind of diagram in the meta-model is class Collection. The Structural diagram specifies how parts of a whole are arranged or organized. Structures can represent hypertexts, taxonomies, system connections, user relationships and containment. The window to the right in Figure 2 shows the structural model for the running example. Thus, collection Library is made of documents structured with Publication, Author and Title metadata information. The meta-model specifies that metadata entities (class Struct) can be connected together with the node relation (organized as a tree) and linked to a collection by a link type relation. Services describe activities, tasks, and operations (the functionality). Human information needs, and the process of satisfying them in the context of DLs, are well suited
Fig. 2. Modelling Environment for VisMODLE
172
E. Guerra, J. de Lara, and A. Malizia
to description with services, including fact-finding, learning, gathering and exploring. In this diagram type, we describe the kind of messages services can produce and consume, and their synchronization. The only relevant entity from the meta-model for this diagram is class Service. In our example, we make two services available: FrontDesk and DoSearch. The former is responsible for managing communications between actors and is asynchronous (sync attribute is set to nowait), while the latter executes queries on the DL and is synchronous. A Society is a set of entities and their relations. The entities include actors as well as hardware and software components, which either use or support services. Our metamodel is CSCW (Computer Supported Cooperative Work) oriented and thus includes the explicit description of the communication between entities and services. In complex systems such as DLs it is important to consider also the actors involved in the usage and so we introduced the society concept. A society is the highest-level view of a DL, which exists to serve the information needs of its entities and to describe the context of its use. Society diagrams are similar to UML collaboration diagrams. Figure 3 shows the society model for our example involving actors Student and Librarian. The scenario represents a Student borrowing a paper from the Library; he interacts with the FrontDesk service requesting the paper and obtaining a response message about its availability. The FrontDesk service forwards the borrow request to the Librarian actor. Then it sends a doc request message to the DoSearch service, which queries the document collection (get operation) using metadata information provided by the borrow request, and waits the result to send back the response. The service returns an is available boolean message which is propagated as a response to the Librarian and eventually to the Student.
Fig. 3. A Societal Model
Behavioural diagrams are used to specify the individual high-level activity of both services and actors by means of state machines. The transitions of the state machines are fired whenever a certain event occurs, which corresponds to the arrival of a message to the actor or service. In the transition, an action can also be specified, which is the sending of another message. Figure 4 shows the behavioural diagram for the Librarian (initially in state active). If he receives an event borrow request, he sends a doc request message to service DoSearch and changes his state to borrow request.
Model Driven Formal Development of Digital Libraries
173
Then, if he receives an event is available, he becomes active again and sends the availability to service FrontDesk. For the current DL example, we have defined four different behavioural models, for actors Student and Librarian as well as for services FrontDesk and DoSearch.
Fig. 4. A Behavioural Model
3 Model Driven Approach to Digital Libraries The overall architecture of our MDSD approach for DLs is shown in Figure 5. The upper part depicts the process of designing the DSVL VisMODLE with the help of experts in the field of DLs, and its implementation in the meta-modelling tool AToM3 . The environment for VisMODLE was defined by using the meta-modelling capabilities of AToM3 . Figure 6 shows a step in its definition. The window at the background (labelled “1”) partially shows the complete VisMODLE meta-model. The tool allows splitting the meta-model into different diagram types (called viewpoints). The five VisMODLE viewpoints are shown in the window labelled “2” in the figure. Window “3” contains the portion of the meta-model belonging to the Structural viewpoint. A special viewpoint named repository DigitalLibrary contains the complete meta-model. In this way, in the generated environment, the user builds instances of the different viewpoints, and a repository model is created in the background with the gluing of the different diagrams the user has built. Consistency relations (shown as arrows in window “2”) specify how the different diagram elements are copied into the repository, how the different diagrams are kept consistent and how changes are propagated to the other diagrams if necessary. These arrows contain triple graph grammar (TGG) rules, which are automatically generated by AToM3 [8]. In addition, a semantic view has been defined that allows expressing the semantics of the repository by using a transformation into Petri nets (see section 5). This is also performed by using TGG rules. With the information described before, AToM3 generated a customized modelling environment for VisMODLE (shown at the background of Figure 2). DL designers can use this environment to build VisMODLE models (layer “DSVL use” in Figure 5). The environment also integrates a simulator that allows the visual animation of the models. The simulator helps to validate and understand the DL design by observing
174
E. Guerra, J. de Lara, and A. Malizia
Fig. 5. The MDSD Architecture for DLs
Fig. 6. Building the Environment for VisMODLE
the message flow between services and actors (see section 4). As stated before, some analysis capabilities have been also integrated based on model transformation into Petri nets (see section 5). Finally, we have provided the environment with a code generator (called “LibGen” in Figure 5) able to produce XUL code for the user interface (UI) of the DL. The generator also selects predefined service components to implement the required functionalities of
Model Driven Formal Development of Digital Libraries
175
the VisMODLE models [11]. Figure 7 shows the generated interface for the example (for the Librarian). The generated UI is built upon a set of XUL template files that are automatically specialized depending on the attributes and relationships designed in the modelling phase. The layout template for the UI is divided into two columns. The left part manages the collections of documents and its metadata information. The right part manages visualization and multimedia information obtained from documents. The basic features provided with the UI templates are document loading and visualization, metadata organization and management.
Fig. 7. Generated GUI for the Example DL
For the composition of the services specified in the VisMODLE models, we have used XDoclet [17]. This is an open-source code generation library which enables Attribute-Oriented Programming for Java via insertion of special tags. It includes a library of predefined tags, which simplify coding for various technologies such as web services.
4 Simulation of DL Models In order to visualize and better understand the behaviour of a DL, we have built a simulator for VisMODLE. The simulation is performed in the repository, since it needs all the dynamic information expressed in VisMODLE (i.e. the societal and behavioural models). The simulation consists of the animation of the state machines that describe the behaviour of actors and services. In the simulation, it is checked that the events and actions specified in the state machines are coherent with the message interchange specified in the society. Note how simulation is a useful tool for the early detection of design errors.
176
E. Guerra, J. de Lara, and A. Malizia
The repository has been enriched with special elements used only for simulation purposes, but not for system specification. These elements are not part of VisMODLE; therefore, they do not appear in any diagram type. Thus, behavioural elements (i.e. actors and services) can receive messages through a special element called input, which in addition has a pointer to the current state of their state machine. Messages are depicted as blue rectangles with its name inside, while input elements are shown as black small rectangles. In the simulator, for a transition to be executed, a message matching the event specified in the transition has to be found in the behavioural element’s input. The other possibility is that the transition does not require any event. On the other hand, when a transition that defines an action with form “target element.message” is executed, then such message is created in the target element’s input. Figure 8 shows the DL example being simulated. A student has asked for a book, so he is in state student borrow (i.e. the input for actor Student has a pointer to the student borrow state). That means that the transition going from state student active to the current state has been executed, so a message borrow has been sent to service FrontDesk, as the action of such transition indicates. Indeed, the state machine for service FrontDesk (upper left corner) has a message borrow as input. The next simulation step would execute the transition going from the service’s current state to state front desk borrow, since the transition requires an event borrow, available in the input.
Fig. 8. A Step in the Simulation of the Example DL
This simulator has been built by using the graph transformation capabilities of AToM3 . Graph transformation [5] is an abstract, declarative, visual, formal and high-level means to express computations on graphs. Roughly, graph grammars are composed of rules, with graphs in their left and right hand sides (LHS and RHS respectively). In order to apply a rule to a graph (called host graph), first a matching (an occurrence) of the LHS has to be found on it. Then, the rule is applied by substituting
Model Driven Formal Development of Digital Libraries
177
the match in the host graph by the rule’s RHS. In addition, rules can define application conditions that restrict their applicability. One of the most used are the so-called Negative Application Conditions (NACs). These are graphs that must not be present in the host graph for the rule to be applied. On the contrary, Positive Application Conditions (PACs) are graphs that must be present in the host graph in order to apply the rule. Finally, we can combine meta-modelling and graph transformation allowing abstract nodes to appear in rules [5]. In this way, nodes can be matched to instances of any subclass, greatly improving the expressive power of rules. Our simulator is made of five rules, three of them shown in Figures 9, 10 and 11. Rule in Figure 9 assigns an input element to each behavioural element, that is, to each actor and service in the repository (i.e. each possible concrete subclass of class BehaviouralElement). Initially, the input does not contain any message, and points to the initial state of the behavioural element’s state machine. If the behavioural element already has an associated input element (NAC), the rule is not applied. In this way, we are sure there is only one input for service and actor.
Fig. 9. Simulation Rule create input
The other four simulation rules perform a simulation step (i.e. a state transition if the required event was produced) for different cases. For example, the rule in Figure 10 considers transitions where no action is defined. In this case, if a behavioural element receives a message whose label matches some of its outgoing transitions (LHS), then the message is processed and the current state is changed (RHS). The attribute condition in the LHS checks that the message name in the input is equal to the event specified in the outgoing transition of the current state.
Fig. 10. Simulation Rule process event
178
E. Guerra, J. de Lara, and A. Malizia
Similarly, rule in Figure 11 considers transitions with an action that sends a message to another behavioural element (i.e. the action has the form “target element.message”). Here, in addition to the previous conditions for a behavioural element to change its state, it is necessary that the target element accepts messages of that kind. This is given by the PAC below, which checks that the corresponding link has been specified in the societal model. Indeed, one of these three positive conditions has to be fulfilled in order to be able to apply the rule: either the source element is an actor that sends a message to a service (first case), or it is a service that sends a message either to an actor (second case) or to another service (third case). The use of abstract nodes (labels 8 and 11) together with the three PACs allows using a single rule for the three cases. This rule deletes the message from the input, changes the current state and creates a message in the target element input. The action below the RHS assigns the message specified in the action’s transition as the name of the newly created message.
Fig. 11. Simulation Rule process event with action
Finally, two similar rules (not shown in the paper) consider transitions where either an action is performed without the necessity of an event, or a transition occurs with neither event nor action associated.
5 Analysis of DL Models We have provided the VisMODLE environment with some analysis mechanisms to validate the dynamics of a DL specification. More in detail, given the set of models that conform a DL design, we can:
Model Driven Formal Development of Digital Libraries
179
1. check if some behavioural element reaches a deadlock state (i.e. local deadlock). In the case of services, we normally want to keep them always available, so they should not define final states. Final states are shown highlighted in the model as result of the analysis, and they are presented textually in a dialog window as well. 2. check whether a given behavioural element reaches certain state. The sequence of state transitions leading to the requested state is shown highlighted as a result, and also textually. 3. ask if a given message is always sent in any possible execution flow. The answer (true or false) is shown in a dialog window. 4. check if the system execution always finishes (i.e. global deadlock). 5. detect if there is some state for a behavioural element that is never reached in the given society. This may be considered a design error, since we should not define states that are not possible. 6. check if some behavioural element can receive an unbounded number of messages at some point, which could lead to an overflow. In order to provide these analysis mechanisms, we have expressed the operational semantics of the models by means of Petri nets [13] (using a semantic view, see Figure 6). Thus, we have defined a triple graph transformation system (TGTS) [7] that transforms the repository into the equivalent Place/Transition Petri net. Once the net is obtained, we internally use analysis techniques based on the reachability/coverability graph, as well as model-checking [3]. Similarly to graph grammars, TGTSs are made of rules that contain, in this case, triple graphs (instead of simple ones). Triple graphs are made of three different graphs: source, target and correspondence. In our case, the source graph is the repository, the target graph is the Petri net resulting from the transformation, while the correspondence graph contains elements relating the elements in the other two graphs. Figures 12 and 13 show some rules of the TGTS that build the Place/Transition Petri net from the VisMODLE repository.
Fig. 12. Triple Rule SM state to PN place
The idea of the transformation is transforming the state machines of each behavioural element into what we call Petri net modules. With this purpose, states are translated into places (triple rule shown in Figure 12) and transitions between states are translated into
180
E. Guerra, J. de Lara, and A. Malizia
Fig. 13. Triple Rule SM transition to PN transition
transitions between places (triple rule in Figure 13). If a state is initial, then the associated place will contain one token. In other case, the place will be empty. In addition, an extra place is created for each possible message invocation (i.e. for each possible event and action specified in each state machine). These places are the interface of the Petri net module of the behavioural element. If a transition needs an event for being executed, then an arc is created from the place corresponding to the event message, to the Petri net transition corresponding to the state machine transition. In that case, the Petri net transition can be executed only if an event of that type is received (i.e. a token is in the right interface place), and in addition, the behavioural element is in the right state (i.e. a token exists in the place corresponding to the state source of the transition). Similarly, if a transition specifies an action, an arc is created from the Petri net transition to the right place of the behavioural element’s interface specified in the action. In this way, executing the transition implies creating a token in the right interface place, and results in the interconnection of the different modules. Figure 14 shows the resulting Petri net after applying the TGTS to the DL example. Each analysis provided to the VisMODLE modelling environment implies evaluating a CTL logical expression on the net’s coverability graph [3]. With this purpose, we use a function that calculates the coverability graph of the net, as well as a model checker
Fig. 14. Place/Transition Petri Net for the DL Example
Model Driven Formal Development of Digital Libraries
181
to evaluate the expression, both implemented in AToM3 . In order to use these analysis techniques, users of the VisMODLE environment do not have to know Petri nets at all. All the analysis process is hidden. CTL expressions are defined when the modelling environment is generated, and hidden to the final user. The results of analysing the Petri net are back-annotated and shown to the user in the VisMODLE notation, which is the notation that he knows. This is possible since we maintain in the correspondence graph the relations between the elements in the DL and the ones in the Petri net. AToM3 provides declarative back-annotation mechanisms that allow the specification of the elements to highlight in the source model as a result of the analysis.
Fig. 15. Execution of Analysis Method Call
For example, Figure 15 shows the repository interface of the generated VisMODLE environment. The repository interface includes one button for each one of the defined analysis method calls. In order to check a property, the end-user has to open the repository and click on one of these buttons. Internally, the tool performs the transformation to the semantic domain, evaluates the CTL formula that corresponds to the property on the Petri net’s coverability graph, and returns the result according to the defined back-annotation mechanism. In the figure, the analysis called reachability for state front desk borrow is shown. The window to the right allows navigating through the different solutions (i.e. the different path executions) that lead to such state. The VisMODLE elements that conform a single solution are shown in the navigator and highlighted in the model. In this way, the analysis result is shown in terms of the original notation to the user, who does not need to have knowledge about any formal method. For the example DL, the net showed neither global nor local deadlocks, all the states were reachable, and the tool signalled the possibility of unbounded number of borrow and borrow request messages.
182
E. Guerra, J. de Lara, and A. Malizia
6 Related Work Formal models for DLs are rarely found, likely due to the complexity of the field. The approach of [16] defines DLs as a combination of a special-purpose database and a hypermedia-based UI, and formalizes them by using the Z language. In [2] a multidimensional query language for DLs is used that rely on the notions of views and versions, metadata formats and specifications, and a first-order logic based language. [6] presents a formal foundation theory on DLs, called 5S, based on streams, data structures, spaces, scenarios and societies. We were inspired on it in order to design VisMODLE. However, these formal approaches do not formally specify how to derive the DL implementation from the DL model. Other declarative approaches, such as the Digital Library Definition Language [12], the METIS framework [1] and the FEDORAs structoid approach [4], are not supported by a strict underlying formal theory. To the best of our knowledge, none of these approaches provide a customized environment supporting code generation as well as validation (animation) and analysis techniques. By using MDSD techniques, we help DL experts to cope with the complexity of DL designs without dealing with coding. In addition, the use of a DSVL makes it easier for people to learn the domain concepts and interpret the models, and allows modelling interactions among DL systems and users (as proposed in the HCI field). Finally, our approach generalizes some metadata schemas such as DC, in the sense that DC data structures can be modelled by using our structure entities and their relationships.
7 Conclusions In this paper, we have presented VisMODLE, a DSVL for building DLs in a modeldriven way. The language is made of five diagram types (services, collections, structure, society and behaviour) to describe the aspects of a DL. We have generated a modelling environment for it with the AToM3 tool. The environment integrates a code generator in order to produce the DL. Although it is in its alpha version, we have already used it to build prototypes. In addition, we have built a visual simulator using graph transformation to graphically validate the DL behaviour at the model level. In order to perform further analysis, a transformation into Petri nets has been designed, which allows checking model properties, such as reachability or deadlocks. This analysis is made by internally performing model-checking of the coverability graph by using predefined temporal logic formulae. In the future, we intend to take into account scalability issues, as well as to generate code for some orchestration language to reflect the overall behaviour of the societal model. Acknowledgements. Work sponsored by projects MODUWEB (TIN2006-09678) and MOSAIC (TIC2005-08225-C07-06) of the Spanish Ministry of Science and Education.
Model Driven Formal Development of Digital Libraries
183
References 1. Anderson, K.M., Andersen, A., Wadhwani, N., Bartolo, L.M.: Metis: Lightweight, flexible, and web-based workflow services for digital libraries. In: JCDL, pp. 98–109 (2003) 2. Castelli, D., Meghini, C., Pagano, P.: Foundations of a multidimensional query language for digital libraries. In: Agosti, M., Thanos, C. (eds.) ECDL 2002. LNCS, vol. 2458, Springer, Heidelberg (2002) 3. de Lara, J., Guerra, E., Vangheluwe, H.: Metamodelling, graph transformation and model checking for the analysis of hybrid systems. In: Pfaltz, J.L., Nagl, M., B¨ohlen, B. (eds.) AGTIVE 2003. LNCS, vol. 3062, pp. 292–298. Springer, Heidelberg (2004) 4. Dushay, N.: Using structural metadata to localize experience of digital content. CoRR, cs/0112017 (2001) 5. Ehrig, H., Ehrig, K., Prange, U., Taentzer, G.: Fundamentals of Algebraic Graph Transformation. Springer, Heidelberg (2006) 6. Gonc¸alves, M.A., Fox, E.A., Watson, L.T., Kipp, N.A.: Streams, structures, spaces, scenarios, societies (5s): A formal model for digital libraries. ACM Trans. Inf. Syst. 22(2), 270–312 (2004) 7. Guerra, E., de Lara, J.: Attributed typed triple graph transformation with inheritance in the double pushout approach. In: Technical Report UC3M-TR-CS-06-01, Universidad Carlos III de Madrid (2006) 8. Guerra, E., de Lara, J.: Model view management with triple graph transformation systems. In: Corradini, A., Ehrig, H., Montanari, U., Ribeiro, L., Rozenberg, G. (eds.) ICGT 2006. LNCS, vol. 4178, pp. 351–366. Springer, Heidelberg (2006) 9. Kochtanek, T.R., Hein, K.K.: Delphi study of digital libraries. Inf. Proc. Manag. 35(3), 245– 254 (1999) 10. Licklider, J.C.R.: Libraries of the Future. MIT Press, Cambridge (1965) 11. Malizia, A., Guerra, E., de Lara, J.: Model-driven development of digital libraries: Generating the user interface. In: Proc. MDDAUI 2006 (2006) 12. Maly, K., Zubair, M., Anan, H., Tan, D., Zhang, Y.: Scalable Digital Librar