Grid Economics and Business Models: 6th International Workshop, GECON 2009, Delft, The Netherlands, August 24, 2009, Proceedings (Lecture Notes in ... Networks and Telecommunications)

Lecture Notes in Computer Science Commenced Publication in 1973 Founding and Former Series Editors: Gerhard Goos, Juris Hartmanis, and Jan van Leeuwen

Editorial Board David Hutchison Lancaster University, UK Takeo Kanade Carnegie Mellon University, Pittsburgh, PA, USA Josef Kittler University of Surrey, Guildford, UK Jon M. Kleinberg Cornell University, Ithaca, NY, USA Alfred Kobsa University of California, Irvine, CA, USA Friedemann Mattern ETH Zurich, Switzerland John C. Mitchell Stanford University, CA, USA Moni Naor Weizmann Institute of Science, Rehovot, Israel Oscar Nierstrasz University of Bern, Switzerland C. Pandu Rangan Indian Institute of Technology, Madras, India Bernhard Steffen University of Dortmund, Germany Madhu Sudan Microsoft Research, Cambridge, MA, USA Demetri Terzopoulos University of California, Los Angeles, CA, USA Doug Tygar University of California, Berkeley, CA, USA Gerhard Weikum Max-Planck Institute of Computer Science, Saarbruecken, Germany

5745

Jörn Altmann Rajkumar Buyya Omer F. Rana (Eds.)

Grid Economics and Business Models 6th International Workshop, GECON 2009 Delft, The Netherlands, August 24, 2009 Proceedings

13

Volume Editors Jörn Altmann Seoul National University TEMEP, School of Industrial and Management Engineering College of Engineering 599 Gwanak-Ro, Gwanak-Gu, 151-742 Seoul, South-Korea E-mail: [email protected] Rajkumar Buyya University of Melbourne Grid Computing and Distributed Systems (GRIDS) Laboratory Department of Computer Science and Software Engineering 111, Barry Street, Carlton, Melbourne, VIC 3053, Australia E-mail: [email protected] Omer F. Rana Cardiff University School of Computer Science Queen’s Buildings, Newport Road Cardiff CF24 3AA, United Kingdom E-mail: [email protected]

Library of Congress Control Number: 2009932574 CR Subject Classification (1998): J.1, K.4.4, H.3.5, H.3, C.2 LNCS Sublibrary: SL 5 – Computer Communication Networks and Telecommunications ISSN ISBN-10 ISBN-13

0302-9743 3-642-03863-8 Springer Berlin Heidelberg New York 978-3-642-03863-1 Springer Berlin Heidelberg New York

This work is subject to copyright. All rights are reserved, whether the whole or part of the material is concerned, specifically the rights of translation, reprinting, re-use of illustrations, recitation, broadcasting, reproduction on microfilms or in any other way, and storage in data banks. Duplication of this publication or parts thereof is permitted only under the provisions of the German Copyright Law of September 9, 1965, in its current version, and permission for use must always be obtained from Springer. Violations are liable to prosecution under the German Copyright Law. springer.com © Springer-Verlag Berlin Heidelberg 2009 Printed in Germany Typesetting: Camera-ready by author, data conversion by Scientific Publishing Services, Chennai, India Printed on acid-free paper SPIN: 12739295 06/3180 543210

Preface

GECON - Grid Economics and Business Models Cloud computing is seen by many people as the natural evolution of Grid computing concepts. Both, for instance, rely on the use of service-based approaches for provisioning compute and data resources. The importance of understanding business models and the economics of distributed computing systems and services has generally remained unchanged in the move to Cloud computing. This understanding is necessary in order to build sustainable e-infrastructure and businesses around this paradigm of sharing Cloud services. Currently, only a handful of companies have created successful businesses around Cloud services. Among these, Amazon and Salesforce (with their offerings of Elastic Compute Cloud and force.com among other offerings) are the most prominent. Both companies understand how to charge for their services and how to enable commercial transactions on them. However, whether a wide-spread adoption of Cloud services will occur has to seen. One key enabler remains the ability to support suitable business models and charging schemes that appeal to users outsourcing (part of) their internal business functions. The topics that have been addressed by the authors of accepted papers reflect the above-described situation and the need for a better understanding of Grid economics. The topics range from market mechanisms for trading computing resources, capacity planning, tools for modeling economic aspects of service-oriented systems, architectures for handling service level agreements, to models for economically efficient resource allocation. The 6th International Workshop on Grid Economics and Business Models, GECON 2009, presented innovative results on the above-mentioned research topics. The workshop attracted high-quality paper submissions. In total, we received 25 submissions from researchers worldwide. Of those submitted papers of great quality, eight papers were accepted, constituting an acceptance rate of 32%. Each paper was reviewed 3 times at least and on average 3.4 times. For the proceedings, the eight accepted papers of this workshop have been grouped into three categories: (1) Market Models and Mechanisms; (2) Business Support Tools; and (3) Business-Related Resource Allocation. The first category “Market Models and Mechanisms” comprises three papers. In the first paper “DEEP-SaM - Energy-Efficient Provisioning Policies for Computing Environments” by Christian Bodenstein, Tim Püschel, Markus Hedwig, and Dirk Neumann, a social welfare maximizing market mechanism for clearing a market for computing jobs has been introduced. The mechanism, which considers energy costs and customer satisfaction levels, is validated through simulation. The results suggest its suitability for use in modern data centers. The second contribution “Response Deadline Evaluation in Point-to-Point Negotiation on Grids,” submitted by Sébastien Noël, Pierre Manneback, and Gheorghe Cosmin Silaghi, analyzes a Grid market, in which each player (consumer or resource

VI

Preface

owner) in the market can broker incoming service requests. Using a simulator, the authors investigate the impact of different policies on the service negotiation process and the response times of the request-offer messages. The third paper in this category “A Framework for Analyzing the Economics of a Market for Grid Services” by Robin Mason, Costas Courcoubetis, and Natalia Miliou, introduces an economic model for analyzing a broad range of issues that are inherent to markets for Grid services. The numerical results show the effect of risk aversion and the durability of resources on the system behavior (i.e., the clearing price and volume of trade). The second category, “Business Support Tools,” contains three papers. The first paper “The GridEcon Platform: A Business Scenario Testbed for Commercial Cloud Services” by Marcel Risch, Jörn Altmann, Li Guo, Alan Fleming, and Costas Courcoubetis, describes a testbed to enable the evaluation of services in a commercial setting. The authors explain the different components and services of the testbed as well as how they can be composed to form a Cloud computing business scenario. The second contribution, which is titled “VieSLAF Framework: Enabling Adaptive and Versatile SLA-Management” by Ivona Brandic, Dejan Music, Philipp Leitner, and Schahram Dustdar, provides a novel framework for specifying service level agreements and for managing SLA mappings. The suggested solution can handle nonmatching SLA templates of consumers and providers. The last contribution of this category, “Cost Optimization Model for Business Applications in Virtualized Grid Environments” by Jörg Strebel, presents a cost model. Using mixed integer programming-based optimization, the model can be used to minimize the IT expenditure of an enterprise and to support enterprises in their decisions to outsource business software applications. Within the third category “Business-Related Resource Allocation,” there are the two remaining papers. The first one paper, “Increasing Capacity Exploitation in Food Supply Chains Using Grid Concepts,” by Eugen Volk, Marcus Müller, Ansger Jacob, Peter Racz and Martin Waldburger analyzes the supply chain of the food industry and suggests a market-based solution using virtual organization concepts. Market participants can execute product-specific negotiations for all parts of the supply chain. The second paper, “A QoS-based Selection Mechanism exploiting Business Relationships in Workflows,” by Dimosthenis Kyriazis, Konstantinos Tserpes, Ioannis Papagiannis, Kleopatra Konstanteli, and Theodora Varvarigou introduces an algorithm for mapping workflows (and their sub-jobs) to service instances. The algorithm considers Quality of Service (QoS) constraints of end-users and the business relationships between service providers. In addition to these papers, we received many paper submissions on research-inprogress. Out of these submissions, we selected six papers. These papers provide an overview about ongoing research projects on Grid and Cloud economics, addressing economic-related research in Cloud computing and software services. We grouped these papers into three categories as well: (1) Economic and Legal Models; (2) Business Models; and (3) Economic-Aware Architectures. The papers of the first category present an analysis of participation levels in Volunteer Grids and a discussion of the jurisdiction of service level agreements. The papers of the second category deal with business models in the application area of medicine and health. The third category of

Preface

VII

papers introduces new architectures for virtual organizations and for composing service level agreements. All of these papers provide an excellent basis for further discussions. To make this workshop a success, many people contributed to this event. We would like to express our gratitude to the organizers of the 2009 Euro-Par Conference, in particular, Hai Xiang Lin, for their support in hosting the GECON 2009 Workshop in Delft, The Netherlands. We would also like to thank Alfred Hofmann and Ursula Barth of Springer, who collaborated very effectively. Finally, our gratitude goes to Marcel Risch. He dedicated his time and effort to set up the website, to communicate with authors in preparing the manuscript of the workshop proceedings, and to coordinate the publication of the proceedings with Springer.

August 2009

Jörn Altmann Rajkumar Buyya Omer Rana

Table of Contents

Market Models and Mechanisms DEEP-SaM - Energy-Efficient Provisioning Policies for Computing Environments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Christian Bodenstein, Tim P¨ uschel, Markus Hedwig, and Dirk Neumann

1

Response Deadline Evaluation in Point-to-Point Negotiation on Grids . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . S´ebastien No¨el, Pierre Manneback, and Gheorghe Cosmin Silaghi

15

A Framework for Analyzing the Economics of a Market for Grid Services . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Robin Mason, Costas Courcoubetis, and Natalia Miliou

28

Business Support Tools The GridEcon Platform: A Business Scenario Testbed for Commercial Cloud Services . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Marcel Risch, J¨ orn Altmann, Li Guo, Alan Fleming, and Costas Courcoubetis

46

VieSLAF Framework: Enabling Adaptive and Versatile SLA-Management . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Ivona Brandic, Dejan Music, Philipp Leitner, and Schahram Dustdar

60

Cost Optimization Model for Business Applications in Virtualized Grid Environments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . J¨ org Strebel

74

Business-Related Resource Allocation Increasing Capacity Exploitation in Food Supply Chains Using Grid Concepts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Eugen Volk, Marcus M¨ uller, Ansger Jacob, Peter Racz, and Martin Waldburger A QoS-Based Selection Mechanism Exploiting Business Relationships in Workflows . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Dimosthenis Kyriazis, Konstantinos Tserpes, Ioannis Papagiannis, Kleopatra Konstanteli, and Theodora Varvarigou

88

102

X

Table of Contents

Work-in-Progress on Economic and Legal Models Determinants of Participation in Global Volunteer Grids: A Cross-Country Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Junseok Hwang, J¨ orn Altmann, and Ashraf Bany Mohammed

116

The Determination of Jurisdiction in Grid and Cloud Service Level Agreements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Davide Maria Parrilli

128

Work-in-Progress on Business Models Engineering of Services and Business Models for Grid Applications . . . . . J¨ urgen Falkner and Anette Weisbecker Visualization in Health Grid Environments: A Novel Service and Business Approach . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Frank Dickmann, Mathias Kaspar, Benjamin L¨ ohnhardt, Nick Kepper, Fred Viezens, Frank Hertel, Michael Lesnussa, Yassene Mohammed, Andreas Thiel, Thomas Steinke, Johannes Bernarding, Dagmar Krefting, Tobias A. Knoch, and Ulrich Sax

140

150

Work-in-Progress on Economic-Aware Architectures Message Protocols for Provisioning and Usage of Computing Services . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Nikolay Borissov, Simon Caton, Omer Rana, and Aharon Levine Business Collaborations in Grids: The BREIN Architectural Principals and VO Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Steve Taylor, Mike Surridge, Giuseppe Laria, Pierluigi Ritrovato, and Lutz Schubert Author Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

160

171

183

DEEP-SaM - Energy-Efficient Provisioning Policies for Computing Environments Christian Bodenstein, Tim P¨ uschel, Markus Hedwig, and Dirk Neumann Albert-Ludwigs-Universit¨ at Freiburg im Breisgau Chair of Information Systems Research 79085 Freiburg, Germany

Abstract. The cost of electricity for datacenters is a substantial operational cost that can and should be managed, not only for saving energy, but also due to the ecologic commitment inherent to power consumption. Often, pursuing this goal results in chronic underutilization of resources, a luxury most resource providers do not have in light of their corporate commitments. This work proposes, formalizes and numerically evaluates DEEP-Sam, for clearing provisioning markets, based on the maximization of welfare, subject to utility-level dependant energy costs and customer satisfaction levels. We focus specifically on linear power models, and the implications of the inherent fixed costs related to energy consumption of modern datacenters and cloud environments. We rigorously test the model by running multiple simulation scenarios and evaluate the results critically. We conclude with positive results and implications for long-term sustainable management of modern datacenters.

1

Introduction

Spawning a great controversy amongst researchers, Nicholas Carr, in ”IT Doesn’t Matter” [Ca1] argued that IT itself does not matter, but the way how we use it. Reserving any judgment to the controversy, this statement seems to become more and more relevant, where Green-IT usage policies step-up in day-to-day ITbusiness processes. Perhaps Carr foresaw, what would today become the driving trend in management of datacenters worldwide; not only greening the hardware, but also greening the usage of it. But sustainability is no new policy in business management. Since the industrialization during the 18th and 19th centuries the role of sustainable management movement steadily evolved. Where technologies such as the steam machine simplified the transportation opportunities, they inadvertently led us to the current state of threatening global warming. Accompanying the trend to globalization, the improved information and communication technologies (ICT), required by the fast growing IT-industry added to the toll. Alarmingly, most lower-level users in the industry are not even aware of their impact. Wiessner-Gross1 claims that a single Google-search can be traced to emit approximately 7 grams of CO2. Currently, global carbon emissions, attributable to the ICT, are around 2% to 2.5% of the world totals. 1

http://www.timesonline.co.uk/tol/news/environment/article5488934.ece

J. Altmann, R. Buyya, and O.F. Rana (Eds.): GECON 2009, LNCS 5745, pp. 1–14, 2009. c Springer-Verlag Berlin Heidelberg 2009 

2

C. Bodenstein et al.

Hence, the energy efficient management of IT infrastructures is a key issue. To achieve this, the decision which server executes which operation at any given time (also known as the winner-determination problem) becomes crucial. Basic allocation mechanisms, like round-robin could possibly result in high-cost operations which could have been prevented using more energy-aware provisioning policies. This provisioning problem becomes more problematic when looking at the immense fixed cost component inherent to computing. Numerous authors ([Br1];[Ra2];[Se1]) while not always coherent on the exact magnitude of energy lost to inefficient hardware, would at least agree that the loss is significantly beyond 50% of the total energy consumed. However, to limit provisioning policies on strict green motives could greatly diminish their usefullness and applicability. The benefit of both providers and customers are important factors that may not be neglected in such decision models. For providers the benefit can usually be assumed to be profit. To determine customer benefit or customer satisfaction a simple metric will be introduced later in this work. The research question posed by this work is thus: Can current market-based provisioning mechanisms be adapted to consider fixed energy costs and customer satisfaction in their optimization dynamically, without losing economic value while still being computationally tractable? In reply to this research question, this work proposes, formalizes and numerically evaluates DEEP-SaM (Discriminatory Energy-Efficient Provisioning including Satisfaction Model), for clearing provisioning markets, based on the greedy maximization of welfare and the minimization of rejected jobs. DEEPSaM is novel as it: – includes energy costs in provisioning, allowing market-based platforms to allocate scarce resources in a truly efficient manner, – considers the fixed energy costs inherent to current computational devices, – considers service level agreements in the form of customer satisfaction, – is applicable in computing Clouds, Grids (to some extent) and datacenters. This work is structured as follows. Section 2 presents related work in the field of provisioning both with respect to market goals, energy efficiency and customer satisfaction, followed by the formalization of the provisioning model, the mathematical representation of DEEP-SaM in section 3 and the description of the fast-clearing DEEP-SaM heuristic in section 4. Section 5 presents the numerical evaluation of the models, followed by a short comment on possible implications this mechanism may have on job provisioning in section 6, before concluding and pointing to further research planned in section 7.

2

Related Work

To date, a lot of research has also gone into hardware inefficiencies and thermal hotspots in computing clusters ([Se1] found that most energy flowing into

DEEP-SaM - Energy-Efficient Provisioning Policies

3

systems is given off as heat, resulting that only about 10% (percentage varies in hardware complexity) of energy is actually available for energy optimization). With ample means of reducing heat in IT-infrastructures ([SS1];[CR1];[Ku1]), authors looked towards reducing energy consumption by the equipment itself. ([CM1];[FL1]) proposed solutions for efficiency in server clusters, ranging from energy-saving power supplies, to the powering down of unused hard drives to using different power states to conserve energy. Combining all these techniques, and a part of the three cornerstone approaches used in this work, ([Ha1];[Ra1];[RS1]) propose energy-efficient mechanisms and designs for datacenters. The above work focused on the technical aspects of provisioning in power-scarce environments given technical restrictions, rather than economic incentives. Another view on greening the use of IT, is the direct reduction in operating costs. [LR1] reuse these techniques and provide for promising contributions to energy management for servers. [RM1] presented an algorithm maximizing the rewards under a limited energy budget, without exceeding the deadlines or available energy. These algorithms were especially used to schedule on portable devices, where battery operation is crucial. More cost/price insight can be found in [HD1] and [CC1] who combine these solutions for server cluster environments. This is not dissimilar to [SS1], where the goal of the allocation is to minimize the average percentage of energy consumed by the application to execute across the machines in the ad hoc Grid, while meeting an application execution time constraint. Consolidated attempts are present in the work of ([MB1];[MG1];[PB1]) ranging from automated power management, to forcing servers to power-nap in underutilized phases. The idea of including additional objectives in scheduling decisions has been covered in various related work inspecting different aspects of those selected in this work. [FN1] discuss the application of economic theories to resource management. [AG1] present architecture for autonomic self-optimization, based on business objectives. Customer satisfaction has been mentioned as business objective and benchmark by [GC1], however they did not discuss how to include in scheduling mechanisms. An architecture for admission control on e-commerce websites that prioritizes user sessions based on predictions about the user’s intentions to buy a product is proposed by [PM1]. In this work, we intend contribute to the energy-efficient management of computing infrastructures, making use of utilization methods similar to ([Br1];[Ha1]), keeping the preferable revenue characteristics of [SN2], while implementing the consolidated methods proposed by ([MB1];[PB1]). This allows provisioning models to include energy costs in provisioning, allowing market-based platforms to allocate scarce resources in a truly efficient manner. Further we intend to provide for the goal of maintaining customer satisfaction , allowing the optimization to dynamically weigh the goals, as the provisioner sees fit. In its performance as a market mechanism, DEEP-SaM operates independent of the pricing mechanism invoked, and remains applicable even when faced with centralized computing environments.

4

3

C. Bodenstein et al.

The Provisioning Model

This section elaborates DEEP-SaM, a market-based model which consists of agents submitting time-sensitive computing jobs to the computing infrastructures such as Grids, Clouds or datacenters, nodes supplying computing resources, specifying the times that their machines are available, and an automated operator deciding allocation. Additionally, the computing environment may be required to handle additional jobs from other clusters or datacenters. Where some operators excel in storage, others may focus on processing, making the intercenter trade of tasks, an integral part of the provisioning process.

Fig. 1. Market Mechanism

Figure 1 depicts the base of operation in common resource infrastructures, but instead of using two separate agents; one for coordinating the resources and the other for computing the provisioning policy, we propose the use of a market mechanism to fulfill the task of both provisioning and coordination of computing resources. These include provisioning policies and other priorities directly influenced by the objective function of the market model. This step does not hinder the expandability of the model, as external restrictions, such as differing power models, can be arbitrarily added to the market model. This is easily attainable using goal programming approaches, by adding further objectives as goals and weights within the overall objective. To construct such a model, we first introduce the setting followed by a formal mathematical representation of the allocation problem. Depending on the valuation, the priorities are determined on the market which employs a greedy priority to accept only the most profitable jobs. Prioritizing the machine offers and job requests on the other hand is more intricate, and since it forms the target of this model, is covered in detail in 3.1. Secondly, we will elaborate a heuristic allocation algorithm which achieves a good approximation of its NP-hard counterpart, before proceeding to simulation evaluation in section 4.

DEEP-SaM - Energy-Efficient Provisioning Policies

3.1

5

The Setting

Computing power is the central scarce resource to be traded in an offline provisioning environment. There are two participating parties in the market: requesters, who wish to obtain computing resources, thereby maximizing their private utility functions and providers who supply the market with resources, maximizing their profits. These providers can range from private users to corporate datacenters, auctioning their idle resources. We assume that customers only request resources if their use is beneficial to them. Hence, the more requests are accepted the better. Furthermore we assume that the valuations submitted with the requests relate to their benefit for the customer. Important jobs are given higher valuations to assure that they are accepted. This leads us to the following simple customer satisfaction metric, the relative net  satisfaction (RNS):

JT jt xjnt cj vj  , JT jt cj vj

where xjnt is a binary decision variable deter-

mining if job j is allocated or not; cj is the capacity needed by job j and vj is the valuation per capacity unit for job j. Important to note here is that jobs are either allocated in requested phases, or not at all, meaning no stretching or squashing of jobs is allowed. Likewise, jobs are either executed in phases as requested, or not at all, resulting in no change of customer satisfaction through waiting time. Although this may seem limiting, it is necessary to allow for an analysis of changes to schedules when faced with just these two simple, yet major goals of satisfaction in terms of overall acceptance rates, and energy costs and their importance to policymakers. The enumerator of the equation therefore delivers the sum of the customer evaluations of all accepted jobs. To be able to compare results for different settings this term ins normalized by using a denominator that calculates the maximum possible evaluation in case all jobs are accepted. The market acts as a bulletin board for participants, meaning it collects offers and requests for a short period of time, before allocating the resources and clearing the board. Once this has been completed, the mechanism proceeds to collecting further posts. These periods will be referred to as phases. The allocation is primarily handled by a sealed bid auction mechanism, where requesters and suppliers do not know the other users requests and offers ([PK1];[SN2]). For the resource offers, let N describe the set of heterogeneous resource offers (”node”) made by the providers. When submitting such a resource offer n ∈ N the providers post the bit (rn , efn , cn , mn , fn , ln ) to the market board. rn (efn , μ) ∈ IR+ 0 specifies the node reservation price per computing power per phase, depicted in arbitrary monetary units (MU) where efn depicts the energy costs when machine n runs idle. cn ∈ IN and mn ∈ IN are the computing units (CPU) and memory available per time slot, where fn ∈ IN is the first available time slot in which these resources are available and ln ∈ IN is the last time slot; the time where the resources can be accessed [SN2]. Each node can be seen as a perfectly divisible virtual machine capable of executing multiple jobs in parallel as proposed by [BD1]. Providers are assumed to have complete information about their availability. When provisioning, the mechanism can determine with perfect foresight an optimal allocation scheme.

6

C. Bodenstein et al.

On the demand side, let j ∈ J represent requests to execute a job. Requesters report their bid (vj , cj , mj , fj , lj ) to the system. vj ∈ IR+ 0 represents the buyers maximum willingness to pay for a computing resources per unit time, depicted in monetary units (MU). cj ∈ IN and mj ∈ IN are the computing units (CPU) and memory required by the jobs per phase, where fj ∈ IN is the first phase in which these resources are required and lj ∈ IN is the last phase in which the resources are required. Each job is submitted and can only be executed in its entirety, allowing only complete allocation to the resources available in the designated start and end time, or none at all. Jobs can be migrated from one node to another, but each job can only be executed on one node at a time. Current schedulers would now aim to maximize the resource allocation, or to balance the overall system load. In this model the provisioner will additionally be the instrument as to which the energy consumption of the datacenter is optimized. By being able to control both the utilization levels (implicitly through job-stacking), and the allocation of jobs to nodes, the largest potential in energy conservation lies within the provisioning mechanism without losing any revenue. 3.2

Mathematical Formalization

Allocation is determined by the binary decision variable xjnt , where xjnt = 1 if job j is allocated to node n in time slot t, and xjnt = 0 if not. The decision variable ynt is the fixed cost initialization where ynt = 1 if machine n is powered in t, and ynt = 0 if not. The time horizon T = {t ∈ IN|fj ≤ t ≤ lj } ∪ {t ∈ IN|fn ≤ t ≤ ln } defines the set of all time slots for the underlying allocation problem. This ensures that no jobs or nodes are allocated in undefined time slots [SN2]. Hence, if J be defined as the set of job requests (resource requests) and N be the set of defined resource nodes (node suppliers), the winner determination problem can be mathematically formalized as follows:  JNT max : α x

jnt

  J T   f xjnt cj (vj − rn ) − NT nt ynt en cn jt xjnt cj vj + (1 − α) NT  NT f J T nt rn cn − nt en cn jt cj vj

(1)

Subject to: fj ≤ t ≤ lj , fn ≤ t ≤ ln , vj ≥ rn , N 

xjnt ≤ 1, xjnt ∈ {0, 1}

∀ j ∈ J, n ∈ N

∀ j ∈ J, t ∈ T

(2) (3)

n J 

xjnt cj ≤ cn ,

∀ n ∈ N, t ∈ T

(4)

j J 

xjnt mj ≤ mn ,

∀ n ∈ N, t ∈ T

(5)

cj ≤ ynt , cn

∀ n ∈ N, t ∈ T

(6)

j J  j

xjnt

DEEP-SaM - Energy-Efficient Provisioning Policies lj N   u=fj

xjnu = (lj − fj + 1)

N 

n

xjnt

∀ j ∈ J, t ∈ T

7

(7)

n

The objective (1) of this integer program is to dually maximize the overall utility of suppliers and requesters, subject to the hard constraint of minimizing the total energy costs and customer satisfaction, weighting the individual objectives using a goal programming approach. Constraint (2) ensures feasibility with respect to resource availability (fn ≤ t ≤ ln ), job-time constraints (fj ≤ t ≤ lj ), profitability (vj ≥ rn ). Constraint (3) ensures singularity; a job can only be allocated to one node in one time slot. Constraint (4) and (5) ensure that the resource constraints are abided, (6) initializes fixed costs when machines run idle and (7) enforces atomicity, the property of a task being executed either in its entirety or not at all; job migration is allowed. The allocation problem above is an instance of the multi-dimensional Generalized Assignment Problem ([MT1];[SN2]), thus NP-hard and as such not solvable in polynomial time. While smaller models may be solvable in seconds, largerscale problems take exponentially longer. We must thus resort to heuristics, as heuristic algorithms have the desirable property of solving hard problems near optimality in short periods of time. In the following subsection we define this heuristic to solve DEEP-SaM.

4

Heuristic Allocation Scheme

The DEEP-SaM heuristic approximates the optimal solution of the NP hard problem in polynomial time. The heuristic has been optimized for the special characteristics of the problem setting. The latter comparison with the optimal solution shows that the results of the heuristic nearly achieve the same outcome with a significantly lower run-time complexity. In the following, the term valuation is defined as both the consumer or provider valuation for one CPU unit. Furthermore the heuristic includes the parameter α allowing more weight to the customer satisfaction by diminishing the economic goals. DEEP-SaM Heuristic: 1 2 3 4 5 6 7 8

: DEEP SAM HEURISTIC(Jobs j, Nodes n, Double α) : j <- j.Sort(SortKeys = CPU.DEM * VAL; ASCENDING ) : n <- j.Sort(SortKeys = α * (CPU.CAP * VAL + (1-α) * EN); DESCENDING) : FOR EACH j IN jobs : FOR EACH n IN nodes : IF (n.cpu.capacity >= j.cpu.dem) and (n.mem.cap >= j.mem.cap) : IF (j.val < (n.en / n.cpu.cap * n.val)) : n.assignJob(j)

8

C. Bodenstein et al.

The input parameter of the heuristic is the set of jobs j and nodes n as well as the satisfaction parameter α. Both lists are sorted according to the resource demand and valuation. Following, the jobs are iteratively assigned to the nodes, starting with the job with the highest evaluation. Each job is assigned to the most economic node, which provides the required resources. Jobs, which cannot be assigned are rejected.

5

Numerical Experiments

For the evaluation of the proposed mechanism, we ran a number of numerical simulations, following a quantitative approach. 5.1

Data Generation

In order to simulate this scenario, the resource requirements and characteristics were all drawn randomly using lognormal distributions, similar to the data generation recommended by [Fe1] for the creation of parameters to simulate the model. The parameter values are always positive, since technically, there is no such thing as a negative runtime and the distribution is assumed to be skewed, implying a tight variance around the means. To achieve this simulation in the 2 model a lognormal distribution log IR+ 0 (μ, σ ) is used. As in related work, since reported valuations of users are uncommon in current traces to the best of our knowledge, we chose to combine these workload traced parameters, with the creation of artificial bids and valuations. The initial settings, a specific combination of parameters, were drawn from distributions based on real workload trace which originated from the parallel workload archive. We chose to generate the data in order to perform multiple simulations with the same distributions, to allow the individual traces to be comparable. As suggested in [Fe1], the computing and memory requirements were subject to lognormal distribution functions. The valuations and fixed costs, were subject to a uniform distribution. The two performance parameters to evaluate our model will be the profit calculated by P rof its :=

J  N  j

xjn vj cj −

n

J  N  j

xjn rn cj +

n

N  n

yn efn cn

(8)

and RNS. JT RN S :=

jt xjnt cj vj JT jt cj vj

(9)

All provisioners, DEEP-SaM, the heuristic, and the benchmark LaFi will be subject to vigorous testing. To be certain that both models are subject to the

DEEP-SaM - Energy-Efficient Provisioning Policies

9

same conditions, and to avoid too many discrepancies, the provisioning phase-line for both models has been reduced to a single-phase one, where provisioning takes place in discrete unit size phases. In each trial, all three evaluated provisioners have been compared based on the same data set. Further we assume nodes are powered on demand, implying nodes which are not required remain shut down. Power-up costs are explicitly considered as a part of the fixed energy costs efn . For multi-phase models, this is not sufficient as the fixed energy costs no longer account for both operation and power-up costs, since power-up costs are only incurred in the first phase. 5.2

Data Analysis

In this subsection, we present the results of the experiments based on the simulations generated above. The simulations and optimizations for the exact solutions were run using GAMS/CPLEX. All results below are averages of 10 simulation runs. Further, no restrictions were imposed upon the CPLEX solver, allowing the solver to run until it reached the optimal solution. Figure 2 (left) shows the performance of the optimal algorithm in terms of their monetary worth, given a changing α. The allocation changed as expected. By increasing α, the importance of the energy efficient model increases. This also implies a decrease in job acceptance (right), even though higher profits may still be achieved. This can easily be explained, by the effect of the customer satisfaction term. By increasing the importance of near-full utilization, the acceptance threshold in terms of profitability is lowered. For the evaluation, a commonly employed Largest Job First (LaFi) mechanism is used as benchmark. This mechanism sorts jobs by jobsize and allocates as ordered; large jobs are scheduled before smaller ones. This way it tries to maximize resource utilization. Figure 4 illustrates how the heuristic performs in comparison to the benchmark. The profit achieved using the DEEP-SaM heuristic is always at least 38% higher than with LaFi. This is due to the fact that it takes valuation of the jobs as well as costs into account. Even against the optimal algorithm, the heuristic

Fig. 2. Optimal allocation evaluation for changing α

10

C. Bodenstein et al.

Fig. 3. DEEP-SaM evaluation for changing α

Fig. 4. Heuristic evaluation for changing α

performs equally well, with a near 94% efficiency on average, in terms of profits. The scenario with α = 1 represents a scenario where customer satisfaction is not taken into account and only job valuations and energy cost of the nodes are considered. In such a scenario the heuristic tends reject jobs that do not achieve their relative share of the fixed energy costs even if they would result in a small contribution to profit. This leads to a slight underutilization of the resources. This can be balanced by taking customer satisfaction into account. Maximum profit is achieved for α = 0.8. The relative net satisfaction is between 11% and 18% higher compared to the benchmark. Obviously it is higher the more the satisfaction term is considered. Node usage is comparable to the benchmark for low values of α, as unprofitable nodes are turned of for rising α it decreases.

DEEP-SaM - Energy-Efficient Provisioning Policies

6

11

Implications for Job Scheduling

The significant results achieved above suggest promising implications for employing DEEP-SaM. As workload traces [Fe1] suggest, real computing settings are largely governed by markets where the job sizes, phases, and order numbers are far greater than those presented in this work. Even though this may pose a problem for the real-time use of the exact mechanism, the surprisingly high efficiency ratio for larger order numbers illustrate the usability of the heuristics. Besides, if GAMS/CPLEX in their optimization resorts to heuristics to find proper solutions in a timely manner, this may suggest that the use of both the heuristic and then the optimal algorithm in succession may prove useful. To fully utilize the market potential, the decision of which pricing instrument to be used becomes an important task. In this work, we resorted to proportional pricing methods, to ensure some degree of truthfulness in reporting, and spent more time on the actual provisioning policy, than the pricing decision. In this regard, a major implication of this model is that when the pricing decision is determined pre-optimization, a practice often encountered in current data-center providers, no further complexity is added to the model. The provisioner will hence function even if pricing is determined in an iterative bidding process.

7

Conclusion and Outlook

Today, datacenters are faced with an increasing, yet still fluctuating ([Fe1] real workload traces exhibit a high volatility over time) demand for their resources. This results in idle resources during off-peak phases wasting energy and inadvertently polluting the environment. Grid computing offers feasible solutions to this problem, allowing jobs to be provisioned decentrally. Since Grids however do not directly maintain their own resources, the problem of energy-efficiency has rarely been discussed. The current mechanisms presented in section 2 are only partially able to cope with the setting given. In this work we presented DEEP-SaM, both an exact mechanism, and its representing heuristic to provide for a meaningful alternative for this problem. As a market mechanism, it can be used as a fast and efficient approach to the provisioning management in datacenters centrally, allowing the provisioner to directly take charge of the efficiency process. This however does not restrict it to be used only in such environments. Initial studies in current research-at-work show that DEEP-SaM can just as well be used in Datacenters, Cloud- and Utility computing environments, where the nodes can be seen as the hierarchal machine cluster addressed in provisioning or scheduling. To overcome the computational complexity of the exact mechanism, we resorted to the use of the DEEP-SaM heuristic. While we did encounter a loss in efficiency by resorting to heuristics, a general notion for the use of heuristics, we feel the decrease in complexity and ease of use fully justifies this forfeit. We set out, to improve current market-based provisioning mechanisms to consider the energy costs involved in computing, without ignoring the important task of customer satisfaction. We fine-tuned the model to consider energy cost

12

C. Bodenstein et al.

functions with fixed-cost components in their optimization and were able to do so without losing economic value while still being computationally tractable. We achieved this by extending current market-based provisioning models, and created both DEEP-SaM and DEEP-SaM heuristic, a model which includes energy costs in provisioning, allowing market-based platforms to allocate scarce resources in a truly efficient manner by directly considering energy costs in optimization. We specifically designed DEEP-SaM to be able to cope with different pricing strategies, be it pre- or post-optimization. To allow for a broad application radius, DEEP-SaM is specifically designed to be ignorant of the source of jobs, allowing it to applicable not only in datacenters, but also in distributed environments, such as computing clouds or Grids to some extent. Another important aspect when looking at customer satisfaction models is the phenomenon that different clients need to be handled differently, as some are satisfied differently to others known as client classification. While some clients may enjoy VIP status, and thus need to be prioritised, others need to be put on hold. This of course drastically changes the setup of the customer satisfaction model. In this work we deliberately kept the satisfaction model simple to allow for a sound foundation to build up from, permitting further research to include more interesting and realistic scenarios of client classification. Initial work has already been published in [PB2]. Since we consider moving away from the use of state-of-the-art heuristics, and given advancements in the field of quantum information processing and quantum computing, we intend to design and evaluate the use of next-generation quantum provisioning algorithms in pair with quantum bids to clear huge-scale provisioning environments. These will include multi-attributive combinatorial provisions, as no longer only the CPUs be traded, but the memory requirements paired to the job aswell. This also allows for entanglement between jobs, whereas one job can only be processed given certain conditions or states of other jobs in the queue and provide for a promising approach for next-generation job provisioning.

References [AG1] Aiber, S., Gilat, D., Landau, A., Razinkow, N., Sela, A., Wasserkrug, S.: Autonomic Self-Optimization According to Business Objectives. In: ICAC 2004: Proceedings of the First International Conference on Autonomic Computing (ICAC 2004), pp. 206–213 (2004) [BD1] Bapna, R., Das, S., Garfinkel, R., Stallaert, J.: A Market Design for Grid Computing. INFORMS Journal of Computing (2006) [Br1] Bradbury, D.: Real Innovation - Building Canada’s largest green data centre. Backbone Magazine (2008) [BR2] Burge, J., Ranganathan, P., Wiener, J.L. (CASH’EM) Cost-Aware Scheduling for Heterogeneous Enterprise Machines. HP-Labs: http://www.hpl.hp.com/techreports/2007/HPL-2007-63.pdf (retrieved March 23, 2007) [BA1] Buyya, R., Abramson, D., Venugopal, S.: The Grid Economy. In: Proceedings of the IEEE 93 Nr. 3 (2005) [Ca1] Carr, N.: IT doesn’t matter. Harvard Business Review 81(5), 41–49 (2003)

DEEP-SaM - Energy-Efficient Provisioning Policies

13

[CM1] Chen, G., Malkowski, K., Kandemir, M., Raghavan, P.: Reducing power with performance constraints for parallel sparse applications. In: Proceedings of the 19th IEEE International Parallel and Distributed Processing Symposium (2005) [CC1] Chun, B.N., Culler, D.E.: User-Centric Performance Analysis of Market-Based Cluster Batch Schedulers. In: Proceedings of the 2nd IEEE/ACM International Symposium on Cluster Computing and the Grid. IEEE Computer Society, Los Alamitos (2002) [CR1] Coskun, A.K., Rosing, T.S., Whisnant, K.: Temperature aware task scheduling in MPSoCs. In: Proceedings of the conference on Design, automation and test in Europe, EDA Consortium, Nice, France (2007) [GC1] Ganek, A.G., Corbi, T.A.: The dawning of the autonomic computing era. IBM SYSTEMS JOURNAL 42(1) (2003) [Ha1] Hamann, H.F.: A Measurement-Based Method for Improving Data Center Energy Efficiency. In: Proceedings of the 2008 IEEE International Conference on Sensor Networks, Ubiquitous, and Trustworthy Computing (sutc 2008), vol. 00. IEEE Computer Society, Los Alamitos (2008) [HD1] Heath, T., Diniz, B., Carrera, E.V., Wagner Jr., M., Bianchini, R.: Energy conservation in heterogeneous server clusters. In: Proceedings of the tenth ACM SIGPLAN symposium on Principles and practice of parallel programming. ACM, Chicago (2005) [Fe1] Feitelson, D.G.: Workload Modeling for Performance Evaluation. In: Calzarossa, M.C., Tucci, S. (eds.) Performance 2002. LNCS, vol. 2459, pp. 114–141. Springer, Heidelberg (2002) [FL1] Freeh, V.W., Lowenthal, D.K., Pan, F., Kappiah, N., Springer, R., Rountree, B.L.: Analyzing the Energy-Time Trade-Off in High-Performance Computing Applications. IEEE Trans. Parallel Distrib. Syst. 18(6), 835–848 (2007) [FN1] Ferguson, D.F., Nikolaou, C., Sairamesh, J., Yemini, Y.: Economic models for allocating resources in computer systems. In: Market-based control: a paradigm for distributed resource allocation, pp. 156–183 (1996) [Ku1] Kurp, P.: Green computing. Communications of the ACM 51(10), 11–13 (2008) [LR1] Lefurgy, C., Rajamani, K., Rawson, F., Felter, W., Kistler, M., Keller, T.W.: Energy Management for Commercial Servers Computer 36(12), 39–48 (2003) [MT1] Martello, S., Toth, P.: Knapsack problems. John Wiley, Chichester (1990) [MB1] Mastroleon, L., Bambos, N., Kozyrakis, C., Economou, D.: Autonomic Power Management Schemes for Internet Servers and Data Centers. In: IEEE Global Telecommunications Conference, GLOBECOMM (2005) [MG1] Meisner, D., Gold, B.T., Wenisch, T.F.: PowerNap: eliminating server idle power. In: Proceeding of the 14th international conference on Architectural support for programming languages and operating systems. ACM, Washington (2009) [MW1] Mutz, A., Wolski, R., Brevik, J.: Eliciting honest value information in a batchqueue environment. In: 2007 8th IEEE/ACM International Conference on Grid Computing, pp. 291–297 (2007); Nielsen, L.S., Niessen, C.: Low-power operation using self-timed circuits and adaptive scaling of the supply voltage. IEEE Trans. Very Large Scale Integr. Syst. 2(4), 391–397 (1994) [PK1] Parkes, D.C., Kalagnanam, J., Eso, M.: Achieving Budget-Balance with Vickrey-Based Payment Schemes in Combinatorial Exchanges. In: IBM Research Report RC 22218 W0110-065 (2001) [PB1] Pinheiro, E., Bianchini, R., Carrera, E.V., Heath, T.: Load Balancing and Unbalancing for Power and Performance in Cluster-Based Systems. In: Workshop on Compilers and Operating Systems for Low Power (COLP) (2001)

14

C. Bodenstein et al.

[PM1] Poggi, N., Moreno, T., Berral, J.L., Gavald` a, R., Torres, J.: Web customer modeling for automated session prioritization on high traffic sites. In: Conati, C., McCoy, K., Paliouras, G. (eds.) UM 2007. LNCS, vol. 4511, pp. 450–454. Springer, Heidelberg (2007) [PB2] P¨ uschel, T., Borissov, N., Mac´ıas, M., Neumann, D., Guitart, J., Torres, J.: Economically enhanced resource management for internet service utilities. In: Benatallah, B., Casati, F., Georgakopoulos, D., Bartolini, C., Sadiq, W., Godart, C. (eds.) WISE 2007. LNCS, vol. 4831, pp. 335–348. Springer, Heidelberg (2007) [Ra1] Raghavendra, P.: Optimal algorithms and inapproximability results for every CSP? In: Proceedings of the 40th annual ACM symposium on Theory of computing. ACM, Canada (2008) [Ra2] Rasmussen, N.: Implementing Energy Efficient Data Centers. In: APC White Paper # 114 (2006) [RS1] Rivoire, S., Shah, M.A., Ranganathan, P., Kozyrakis, C.: JouleSort: a balanced energy-efficiency benchmark. In: Proceedings of the 2007 ACM SIGMOD international conference on Management of data. ACM, Beijing (2007) [RM1] Rusu, C., Melhem, R., Mosse, D.: Maximizing Rewards for Real-Time Applications with Energy Constraints. ACM Transactions on Embedded Computer Systems 2, 537–559 (2003) [SN1] Schnizler, B., Neumann, D., Veit, D., Weinhardt, C.: Trading Grid Services a Multi Attribute Combinatorial approach. European Journal of Operational Research 187(3), 943–961 (2008) [Se1] See, S.: Is there a pathway to a Green Grid, http://www.ibergrid.eu/2008/presentations/Dia%2013/4.pdf (retrieved March 23, 2007) [SS1] Sharma, R.K., Shih, R., Bash, C., Patel, C., Varghese, P., Mekanapurath, M., Velayudhan, S., Manu Kumar, V.: On building next generation data centers: energy flow in the information technology stack. In: Proceedings of the 1st Bangalore annual Compute conference. ACM, Bangalore (2008) [SS2] Shivle, S., Siegel, H.J., Maciejewski, A.A., Sugavanam, P., Banka, T., Castain, R., Chindam, K., Dussinger, S., Pichumani, P., Satyasekara, P., Saylor, W., Sendek, D., Sousa, J., Sridharan, J., Velazco, J.: Static Allocation of Resources to Communicating Subtasks in a Heterogeneous ad-hoc Grid Environment. Journal of Parallel and Distributed Computing 66(4) (2006) [SN2] Stoesser, J., Neumann, D., Weinhardt, C.: GreedEx- A Scalable Clearing Mechanism for Utility Computing. Electronic Commerce Research 8(4) (2008) [WH1] Waldsburger, C., Hogg, T., Huberman, B.A., Kephart, J.O., Stornetta, W.S.: Spawn: A Distributed Computational Economy. IEEE Transactions on Software Engineering 18(2), 103–117 (1992) [WP1] Wolski, R., Plank, J.S., Brevik, J., Bryan, T.: Analyzing Market-Based Resource Allocation Strategies for the Computational Grid. International Journal of High Performance Computing Applications (2001)

Response Deadline Evaluation in Point-to-Point Negotiation on Grids S´ebastien No¨el1 , Pierre Manneback1 , and Gheorghe Cosmin Silaghi2 1

University of Mons, Faculty of Engineering, Belgium {Sebastien.Noel,Pierre.Manneback}@umons.ac.be 2 Babe¸s-Bolyai University of Cluj-Napoca, Romania [email protected]

Abstract. In this paper we focus on access negotiation to services in decentralized commercial Grids. Besides being a resource provider, each site can play a brokering role, thus generate profit on the incoming service requests by subcontracting. In our configuration, negotiation is a pointto-point process consisting of couples request-offer messages. We present here an event-based simulator allowing specification of various policies that can affect the service negotiation process and the visualization of the information flow in the network. Further, we exploit the simulator to manage the response deadlines in the request-offer messages, so that sites accomplish their profit-maximization objectives. Simulations help to estimate an appropriate configuration of the policies depending on different Grid parameters leading to the maximization of the local profit. Keywords: Service access, Negotiation, Delegation.

1

Introduction

Nowadays, the Grid moves from a centralized architecture with coordinating entities toward a scalable fully decentralized infrastructure [4]. In a commercial context where sites try to make profit from their participation in the Grid, centralized management roles should be avoided because they introduce partiality issues. Moreover, central roles such as broker and resource discovery services represent a lack of architectural flexibility being issues of scalability. Grid sites could play these roles by themselves and try to make profit [5]. In our study we adopt this decentralized approach of a Grid with sites playing simultaneously the consumer, producer and broker roles. We introduce a bilateral point-to-point dialog mechanism to enable the brokering role at each Grid site. Two sites communicate by exchanging couples of request-offer messages with time deadlines. A site that receives a request message can either deliver by itself the requested service or can further propagate the message in the network trying to subcontract and make a profit. While the bilateral negotiation can follow the discounting pattern described in [7], we study the configuration of negotiation and management policies of a given site trying to optimize its behavior. To the best of our knowledge, such commercial Grids where multiple independent sites J. Altmann, R. Buyya, and O.F. Rana (Eds.): GECON 2009, LNCS 5745, pp. 15–27, 2009. c Springer-Verlag Berlin Heidelberg 2009 

16

S. No¨el, P. Manneback, and G.C. Silaghi

interract in a point-to-point manner do not exist yet and offer new ways of doing business in a service oriented environment. In this context, many management decisions have to be made at the site during the negotiation process so that it fully exploits the local resources in one hand and the subcontractors network in the other hand. The site should be configured to attract consumers by proposing valuable offers and to maximize the local profit at the same time. Our key goal is to study a point-to-point negotiation environment and, based on the observations made, to propose adapted management policies. In this paper, our major contribution is to present an event-based simulator and to report the empirical observations we obtained. The simulator allows the specification of various policies that can affect the negotiation process; it help the Grid designer to visualize the information flow in the network and to estimate performance indicators for various network topologies and node profiles. As an example we study how policies for response deadlines in the communicated messages can influence the profits. For various global configurations including the network topology density and the management policies of the other sites in the environment, we deduce the best deadline values, depending on the context. These values can be used by administrators as a base for the configuration of sites. The paper is organized as follows. Section 2 describes the negotiation pattern and the internal working of the negotiation process. Section 3 shows how the simulator works and helps the designer to tune the response deadlines in order to maximize the Grid sites profit. Section 4 shortly briefs related work and section 5 concludes the paper.

2

The Negotiation Model

This section describes briefly the negotiation pattern and the negotiation process conducted into each Grid site. To get more details about the negotiation model, we refer the reader to [11]. 2.1

The Negotiation Pattern

The Grid consists of a collection of networked sites i, each offering a set of resources/services Ri . Each site i is acquainted with a localy defined list of other sites Ai considered as potential services providers. Therefore, besides being a producer, each site can also play a brokering role by intermediating the service requests and trying to subcontract them in its acquaintance network Ai . Users Ui of a site are the local consumers. The sets Ri , Ai and Ui might be empty, leading to the particular cases of single-role site: broker-only, consumer-only or provider-only. We envision that each node plays also the brokering role and tries to generate income by subcontracting. The general setup is similar with the contract net protocol [12]. The negotiation starts when a user of a site i needs some service

Response Deadline Evaluation in Point-to-Point Negotiation on Grids

17

Ser and the site forwards this request SR in its acquaintance network Ai . The service request is defined as in eq. (1): SR = [Ser, Rew, P en, tr ]

(1)

– Ser is the service request itself. It includes the quality of service parameters such as resources type and quantity (e.g. 10 processors AMD Opteron 252) – Rew is the reward that the consumer is willing to pay – P en is the penalty fee requested to the provider in case of failure in the service delivery – tr is the deadline up to which the message initiator waits a response for the request The two parameters Rew and P en are highly important and should be estimated precisely. Compared to the market mean price which can be estimated by any grid participant, a high reward Rew grants a higher interest from the provider. In this case, the consumer can therefore expect a higher response priority. A high penalty P en is generally proposed by the consumer when he needs high guarantees on the resources availability. A provider engaging in a high penalty contract generally expects the reward to be high as well. Even though these two parameters are fundamental in a real contract negotiation, they are not discussed deeply in this paper since we focus on the deadline parameters. A real message exchange can be performed using an adapted specification such as WS-Agreement [14] but we adopt here a more compact representation while exploiting WS-Agreement terminology. Moreover, we use an arbitrary money unit/time scale and we suppose that every sites are synchronized with a common universal clock. We suppose that some site j receives a service request SR. Depending on the acquaintance network Aj of the site j, the request can be forwarded to other sites in Aj after being possibly adapted (for instance, the reward Rew and the deadline tr may be decreased). Site j can forward the request to multiple sites, leading to concurrent negotiations as presented in [10]. When forwarding the requests, site j needs to carefully select the deadline parameter tr for the subsequent concurrent negotiations. Optimising the selection of the deadline parameter tr is the main scope of this paper. At any time but most probably before tr , the site j can send a service offer SO in response to the service request SR. Eq. (2) defines the service offer: SO = [Ser∼ , Rew∼ , P en∼ , tv ]

(2)

An offer has the same structure as a request: it contains the reward, the penalty and a deadline tv defining the validity of the offer. This value indicates that the request initiator i has to accept the offer before time tv . Offer parameters can be slightly different from the request when the site can not (or do not want to) provide exactly what the initiator asked for. The symbol ∼ means that the proposed value is somewhere near the requested value. The negotiation between a requester site i and a partner site j involves several rounds of request-offer messages; at each round a site makes some concession

18

S. No¨el, P. Manneback, and G.C. Silaghi

in the negotiated parameters. In our protocol, only the consumer side finally concludes the contract by sending an acceptance notification. As the consumer asks simultaneously several sites to provide offers, he may have many options to choose from. In the bilateral negotiation rounds, smaller the concessions in the negotiated issues are, the more the probability for the consumer to accept the offers decreases. The parameter tr which we focus on in this paper has a large influence on potential offers the initiator i will collect. Our previously described delegation protocol is close to the one described in [1] but with a fundamental difference: in our study, sites at each side of a delegation chain are never put in relation in the sense that they do not know each other. Each site considers the next site in the chain as the real provider of resources and has no information whether the offer is subcontracted or not. 2.2

Management Policies

We define the internal working of a Grid site with a set of management policies responsible for decision making with regard to one or more aspects of the negotiation. Each policy is defined by some parameters that influence the way the negotiation is conducted. In this study, we structured the management policies as presented on figure 1.

Fig. 1. Management policies structure

The left part of figure 1 corresponds to the dialog with the potential consumers and the right part with the potential providers. The upper part of the figure represents the processing of an incoming service request arriving at the considered site while the lower part represents the path followed by offers. Figure 1 should be read starting from its upper left arrow, which symbolizes an incoming request. The first management policy to be executed on this request is the Delegation Policy. The Delegation Policy is responsible for deciding whether the incoming request should be delegated or not (or at least, should the site try to delegate). The output of this policy is a boolean result indicating the next policy that will be applied. If the answer is true, the site runs the Delegate Choice Policy. In

Response Deadline Evaluation in Point-to-Point Negotiation on Grids

19

the other case, the site will not delegate but can possibly create an offer based on local resources by running the Offer Formulation Policy. The Delegate Choice Policy represents the algorithm to choose among the sites in the acquaintance network Ai which will be considered as potential subcontractors. This policy is useful when the acquaintance network is large and the site does not possess enough resources to carry on many simultaneous negotiations. The site i might decide to filter the list Ai based on different factor such as reputation, specialization, etc. Next, site i invokes the Request Formulation Policy for the list of potential delegates selected by the Delegate Choice Policy. The Request Formulation Policy is responsible for building subcontracting requests for the sites selected by the previous applied policy. This policy is very important because it will define how aggressive and opportunistic will be the negotiation behavior. Together with the Offer Consideration Policy, the Request Formulation Policy represents the heart of the negotiation process with the subcontractors. The output of this policy is a list of requests that are forwarded to the potential subcontractors. The Offer Consideration Policy is executed every time an offer is received from a potential subcontractor. This offer is received in response to a previously forwarded request and arrives generally right before the time tr . The Offer Consideration Policy is the algorithm employed to estimate the value of the offer and choose whether to iterate and send a new request or to accept the received offer and send an acceptance notification. The Offer Formulation Policy is responsible for constructing offers based either on the reserved local resources or on the received offers from subcontractors. Again, the negotiation behavior might differ with various left-side partners. Related works have been conducted to design the policy responsible for the creation of new offers (Offer Formulation Policy) and the analyze of the received offers (Offer Consideration Policy) (for instance, see [9]).

3

Simulations: Managing the Response Deadlines

This section describes the simulator and the performed simulations: the hypothesis, the results and their interpretation. 3.1

Simulator

We performed different simulations using a Java event-based simulator developed to test the negotiation model described in section 2. The objective of this tool is not to simulate the exchange of packets like in a real TCP/IP network but to visualize at a higher level the transmission and the reception of messages between nodes. The simulator also abstracts the services/resources: it envisages that a site is endowed with some resources, which can be reserved for a given time frame.

20

S. No¨el, P. Manneback, and G.C. Silaghi

Fig. 2. Graphical user interface of the Java negotiation simulator

The simulator allows the designer to: – evaluate different policy configurations and observe their influence on the negotiation process – measure indicators at a global scale in the whole simulated Grid environment (e.g. global user satisfaction, resource utilization) – analyze some local values at a given site (e.g. number of contracts, local profit) In comparison to other existing simulators such as GridSim [2] and SimGrid [3], our simulator has some advantages. It offers a visual representation of the network during the execution of the simulation. As presented in figure 2, it is possible to observe and follow the message exchanges such as requests, offers and acceptance notifications. While visualizing the Grid environment, it is possible to interrupt the simulation, go backward and forward to determine why any event occurred. Moreover, one can adapt the policy parameters dynamically at runtime and observe immediately the result on the visualization window and on reports. To allow the execution of simulations with different sets of parameters on a cluster or on a Grid, it is possible to disable the graphical user interface to run the simulator as a batch job. This simulator was used to test different negotiation policies and more precisely, the response deadlines that have to be set when delegating requests.

Response Deadline Evaluation in Point-to-Point Negotiation on Grids

3.2

21

Simulation Objective

For all simulations, we focus on a given site and we observe its profits at the end of the simulation time frame, in various configurations of the Grid environment. We discuss the response deadline value tr introduced when subcontracting requests to other sites considered as potential providers.

Fig. 3. Delegation of a request arriving at a considered site

Figure 3 presents an incoming service request from a consumer. This request can be forwarded to one (or more) provider site(s). We want to find the best value for tr in order to optimize the local profit of the considered site. This value has to be carefully selected because it influences the potential offers and the number of sites that may be contacted. The Request Formulation Policy is responsible for formulating requests SR that will be forwarded to the potential subcontractors in the acquaintance network Ai . This policy has to set a value for tr before forwarding the request: this value specifies the deadline for the provider to send an offer. In the policy olicy used for these simulations, we introduce a configuration parameter named tP r  P olicy which influences the way tr is set. tr is bound as follows: olicy 0 ≤ tP ≤1 r olicy Based on a defined value of tP , the tr is calculated as in eq. (3): r olicy tr = ta + δ + tP (tr − ta − δ) r

(3)

where ta is the moment of arrival of the request (supposed equal to the time at which the Request Formulation Policy is executed) and δ is the time needed to transmit a message in the network. olicy When tP ≈ 0, tr is close to the actual time ta , the considered site does r not give much time to providers for sending an offer. Thus, the request SR has no time to go “deep” into the Grid: it is received only by sites included in the acquaintance network Ai . In this case, the site adopts a strategy to keep a long time for negotiation with the direct subcontractors.

22

S. No¨el, P. Manneback, and G.C. Silaghi olicy Table 1. Subcontracting characteristics for various levels of tP r olicy tP tr propagation time for negotiation r ≈ 0 (tr )min ≈ ta sites ∈ A long ≈ 1 (tr )max ≈ tr far short

olicy When tP ≈ 1, tr is as late as possible, that is close to time tr . The considr ered site decides to let many time for the direct providers of Ai to subcontract and negotiate with their own providers. Only a short time is kept for negotiation with the providers in Ai .

3.3

Simulation Configuration

In this part of the paper, we describe experimentation configuration including the simulation environment and the internal parameters of various sites, defining how they behave during the negotiation process. First, we present the configuration of the global environment. Each measure is obtained out of one experiment. For the same configuration, we run each experiment 30 times with a different random seed to get an average value of each indicator. The duration of a simulation experiment is 1000 time units; we interrupt the generation of new requests at time 800 so that sites are stable at the end of the simulation time frame. The Grid is composed of 10 sites; they represent big centers or companies offering a set of services on Internet. Considering the french Grid5000 infrastructure which is composed of 9 different sites, we believe that 10 sites is a realistic value for our simulations. The network topology is randomly generated, each grid site being randomly connected with the rest of the sites in the network. Each site is endowed with 0 to 50 units of a resource. There are 5 request sources (that is, users groups) randomly distributed among sites, that initiate service requests. The global load factor (the total amount of requested services reported to the total resource quantity in the network) is a-priori established randomly. All randomly generated numbers are uniformly distributed. We assume that no actor will successfully acknowledge a request before time tr ; there is no interest for the providers to answer before tr . Providers try to get the best possible resource allocation, therefore they will postpone the decision until the deadline. When formulating an offer, the reward estimation is made using the average value between the rewards in the last request and in the last offer of the negotiation session. More sophisticated negotiation strategies could also be defined such as the one presented in [8]. In the offer, the generating site does not change the service levels Ser and each site has full information about the market-price of a given resource.

Response Deadline Evaluation in Point-to-Point Negotiation on Grids

23

Fig. 4. Evolution of the profit for different values of the acquaintance network density

The various policies presented in figure 1 are configured as follows: – The Delegation Policy only decides if there is sufficient time to send and receive at least one couple request/offer. If the answer is yes, the request is forwarded to other sites int the acquaintance network. – The Delegate Choice Policy selects all sites in the acquaintance network as potential subcontractors. – The Offer Consideration Policy only considers the price to evaluate the utility of the offer. Each site will select the best price among the received offers. 3.4

Simulation Results

The objective of the first simulation is to observe the evolution of the profit for different values of the density of the acquaintance network Ai . The maximum density is when each site is connected with all the remaining sites in the grid. We generate a different number of links between sites to change the density of Ai : the curve denoted i − j means that we generate randomly n links between sites where i ≤ n ≤ j. The figure 4 presents the set of results obtained for this experiment. Using this graphic and having an estimation of the density of the acquaintance olicy network, a site administrator can identify the best range of tP values leading r to the highest local profit. This graph reveals some interesting results. We can observe that for a low network density, the considered site obtains a higher profit using a high value olicy of tP . For a low density (1-2 or 2-3), the considered site is linked to a low r olicy number of other sites. Therefore, by increasing tP , it expects that the request r will propagate far into the Grid and reach many other sites. More potential offers are generated leading to a higher profit.

24

S. No¨el, P. Manneback, and G.C. Silaghi

olicy Fig. 5. Evolution of the profit for different values of the tP in sites ∈ Ai r

For a high network density, the highest profit is obtained with a low value olicy of tP . Actually, in this configuration, the considered site is linked to many r other sites. There is no interest to let the request go “far” into the Grid; time should be saved for direct negotiation with the providers in Ai . We observed a local minimum in the high network densities curves obtained olicy olicy with tP equal to 0.7. This particular value depends on the tP used by r r other sites in the simulated network. In this case, the other sites in the neighborhood of the considered site send requests with a lower tr leading them to win most of the contracts. In the second simulation, we observe the evolution of the profit when the surolicy rounding sites use different values of their tP policy parameter. We analyze r P olicy the best tr values leading to the highest local profit. The results are preolicy sented in figure 5. On the x-axis we took the tP value of the considered site, r olicy while each line is drawn for some average of the tP value for the remaining r sites in the grid. The graphic corroborates two intuitive ideas. First, when the sites in Ai use olicy a low tP value (for instance 0.3 or 0.4), the considered site should use a low r P olicy tr as well to get a higher profit. This can be explained easily. A high value of this parameter means the considered site attempts to contact more sites in the olicy Grid. If at the same time, direct subcontractors in Ai are using a low tP , they r block the forwarding of the request and only a small number of sites are finally reached. In that case, the considered site would have obtained higher profits by olicy using a low tP and prioritize negotiation instead of propagation. r olicy Second, in the opposite case, when the sites in Ai are using a high tP value r P olicy (for instance 0.8 or 0.9), the considered site should use a high tr as well. The number of sites potentially reached using this strategy has to be estimated by the site administrator, taking into account the observations made in the first simulation.

Response Deadline Evaluation in Point-to-Point Negotiation on Grids

25

olicy We also observe that when the sites in Ai are using a tP < 0.3, the r olicy considered site is not able to make any profit whatever its own tP value is. r

4

Related Work

Grid research did many attempts to build simulators for various sorts of experiments. SimGrid [3] and GridSim [2] represent the most important attempts. GridSim focuses on a commercial environment with priced services and allows the researcher to study the economics of the grid. On the other side, SimGrid is a more general message-exchange simulator allowing one to generate and simulate a broader sorts of grids and distributed environments. In our context, we needed the ability to visualize the Grid during the negotiation, go back and forth in the simulated time to observe and understand decisions taken by the sites. These features, which are now offered by our simulation tool, were not proposed by GridSim and SimGrid. Subcontracting is an old conceptual problem in distributed computing. The Contract Net Protocol [12] which defines a message-exchange protocol to distribute tasks on multiple execution nodes, represents the basis for many studies concerning delegation. Relating with the grid, we can cite the work of BustosJimenez et al. [1] that tries to study the delegation in a peer-to-peer grid. However, in comparison to our study, this work does not tackle the commercial aspects and the negotiation process. Other studies concern resource discovery in a peer-to-peer grid [5,6,13]. Opportunistic strategies regarding the bilateral negotiation in the grid are presented in [7,8] and are of high interest for defining the site behaviour in our study.

5

Conclusion

In this paper we focused on a negotiation model adapted for delegating services in a commercial Grid. This model is based on a point-to-point messages exchange between the different peers of the Grid and delivers more flexibility and business opportunities for the Grid participants. We developed a Java event-based simulator for our delegation model. This simulator is a useful tool for the site administrator to test different behavior policies before their implementation in the real environment. We used the simulator to evaluate how the response deadlines influence the local profits. The response deadlines of the request/offer messages is one important parameter that influence the overall negotiation. The simulations have been conducted in various Grid configurations: for difolicy ferent density levels of the network and different values of the tP in the setup r of the neighboring sites. We concluded that the considered site should use a high olicy tP value for low density network and inversely. Moreover, we showed that r olicy this value should be adapted according to the tP of other participants in the r Grid.

26

S. No¨el, P. Manneback, and G.C. Silaghi

Further we plan to use the simulator to highlight other important parameters in the negotiation, like the rewards and the penalties, resource aggregation and bundling, various algorithms for decision making at local sites and the interaction between these parameters. Moreover, we have to observe the evolution of negotiation policies when increasing the number of participants in the Grid. A olicy method and tools to estimate the network density and the tP of the other r sites in the Grid environment have to be defined and developped and will be the object of a future work. Moreover, the influence of the network density and the configuration of surrounding sites has been studied separately; as a future step, it would be interesting to consider the interaction of these various parameters on each other. Acknowledgements. G.C. Silaghi acknowledges support from the Romanian Authority for Scientific Research under project IDEI 2452.

References 1. Bustos-Jimenez, J., Varas, C., Piquer, J.: Sub-contracts: delegating contracts for resource discovery. In: Talia, D., Yahyapour, R., Ziegler, W. (eds.) Grid Middleware and Services. Springer, Heidelberg (2008) 2. Buyya, R., Murshed, M.: GridSim: A Toolkit for the Modeling and Simulation of Distributed Resource Management and Scheduling for Grid Computing. In: Concurrency and Computation: Practice and Experience, vol. 14(13-15), pp. 1175–1220. Wiley Press, USA (2002) 3. Casanova, H., Legrand, A., Quinson, M.: SimGrid: a Generic Framework for LargeScale Distributed Experimentations. In: The 10th IEEE Intl. Conf. on Computer Modelling and Simulation (UKSIM/EUROSIM 2008) (2008) 4. Foster, I., Iamnitchi, A.: On Death, Taxes, and the Convergence of Peer-to-Peer and Grid Computing. In: Kaashoek, M.F., Stoica, I. (eds.) IPTPS 2003. LNCS, vol. 2735, pp. 118–128. Springer, Heidelberg (2003) 5. Iamnitchi, A., Foster, I.: A peer-to-peer approach to resource location in Grid environments. In: The 11th IEEE Intl. Symposium on High Performance Distributed Computing (2002) 6. Iamnitchi, A., Foster, I., Nurmi, D.C.: A peer-to-peer approach to resource discovery in Grid environments. In: High Performance Distributed Computing (2002) 7. Li, J., Sim, K.M., Yahyapour, R.: Negotiation Strategies Considering Opportunity Functions for Grid Scheduling. In: Kermarrec, A.-M., Boug´e, L., Priol, T. (eds.) Euro-Par 2007. LNCS, vol. 4641, pp. 447–456. Springer, Heidelberg (2007) 8. Li, J., Yahyapour, R.: A strategic negotiation model for Grid scheduling. ITSSA 1(4), 411–420 (2006) 9. Lin, R., Kraus, S., Wilkenfeld, J., Barry, J.: Negotiating with bounded rational agents in environments with incomplete information using an automated agent. Artificial Intelligence 172(6-7), 823–851 (2008) 10. Nguyen, T.D., Jennings, N.R.: A heuristic model for concurrent bi-lateral negotiations in incomplete information settings. In: Intl. Joint Conf. on Artificial Intelligence, Mexico, August 9-15 (2003) 11. No¨el, S., Manneback, P., Silaghi, G.C., W¨ aldrich, O.: Delegation in service access negotiation. In: Proc. of the CoreGRID Integration Workshop, Greece (April 2008)

Response Deadline Evaluation in Point-to-Point Negotiation on Grids

27

12. Smith, R.G.: The Contract Net Protocol: High-Level Communication and Control in a Distributed Problem Solver. IEEE Trans. Comput. C-29, 1104–1113 (1980) 13. Trunfio, P., Talia, D., Papadakis, H., Fragopoulou, P., Mordacchini, M., Pennanen, M., Popov, K., Vlassov, V., Haridi, S.: Peer-to-Peer resource discovery in Grids: Models and systems. FGCS 23(7), 864–878 (2007) 14. Andrieux, A., Czajkowski, K., Dan, A., Keahey, K., Ludwig, H., Nakata, T., Pruyne, J., Rofrano, J., Tuecke, S., Xu, M.: Web Service Agreement Specification (WS-Agreement), March 14 (2007)

A Framework for Analyzing the Economics of a Market for Grid Services Robin Mason1 , Costas Courcoubetis2 , and Natalia Miliou2 1

University of Southampton, Highfield, Southampton, United Kingdom 2 Athens University of Economics and Business, Athens, Greece

Abstract. This paper provides a single broad model for the analysis of a range of issues underlying a market for Grid services. The demand and the supply sides of such a market are being treated separately and the relation between the two sides is being studied. We provide numerical results in order to derive conclusions about the viability of a market for Grid services. Underlying our model are parameters such as the cost technologies, the random processes driving demand and supply and the size of the market. We study the effect of the model’s parameters, such as risk aversion or the durability of resources, on the system’s behavior, eg. on the clearing price or volume of trade. Keywords: Grid economics, Grid services, Grid market, spot market, market failure, competitive equilibrium.

1

Introduction

The remarkable evolution of computational power, as well as the growth of internet, have changed the approach of research on computational resources. Namely, computational resources have become less expensive and storage capabilities have increased dramatically. Clustering geographically remote computational resources and using them as one unified resource has become not only feasible but also economically beneficial. In this work we treat the question of how computational resources should be traded. We demonstrate the benefits (to both sellers and buyers of Grid services) from trade. And we examine how these benefits arise in a particular trading mechanism. We analyse a competitive market model, with no market power on either the buyer or provider side. We opt for this approach, for two main reasons; it employs a mechanism that is simple yet effective with desirable economic properties, and it is an extension of the standard stock market mechanism for storable commodities. We view this as the limit of a double auction in which the number of traders becomes large.1 The properties of double auctions in static settings have been 

1

Work partially supported by the EU-funded research project GRIDECON, FP6 2005 - IST5 - 033634. This double auction model has been adopted in the market mechanism developed in the project GRIDECON.

J. Altmann, R. Buyya, and O.F. Rana (Eds.): GECON 2009, LNCS 5745, pp. 28–45, 2009. c Springer-Verlag Berlin Heidelberg 2009 

A Framework for Analyzing the Economics of a Market for Grid Services

29

investigated both theoretically and empirically. The theory of double auctions (see e.g. [2]) shows that every non-trivial equilibrium of a double auction converges to the competitive market outcome as the number of auction participants grows. Moreover, the rate of convergence is fast, namely of the order of 1/N 2−α for any α > 0, where N is the number of auction participants. Satterthwaite and Williams, [6], have shown that the worst-case speed of convergence of the double auction is as high as for any other type of auction. This work strengthens earlier results of e.g. [5]. The experimental evidence backs up these theoretical results. Even from early work, such as [1] and [10], experiments have found that double auctions yield outcomes that are close to fully efficient, even with a small number of human participants. In [3] it was shown that the same is true with ”zero intelligence” traders. As it is explained in [4], the rules of the double auction are such that only a small number of participants is needed to get (very close) to the competitive outcome. Work by Cliff, [7], has challenged this conclusion. For example, he shows that, under some circumstances, agents who are less than fully rational can fail to get close to equilibrium in a double auction. But note that this does not necessarily imply that the efficiency of a double auction is low (since a high degree of efficiency just requires that high value buyers are generally matched with low value sellers). Moreover, recent work (by e.g., Preist and van Tol in [8]) extending Cliff’s analysis suggests that the rate at which actual play approaches the competitive outcome is quite high. See Parsons et al. in [9] for a very good summary of the literature. Some complementary work is in [11]. The authors analyze there the use of real options traded at an additional contract market, to efficiently manage economical issues arising from the realization of a flexible resource reservation scheme. Necessary conditions so that even risk neutral agents have incentives to participate in such a market, as it increases their expected utility, are being derived. In this work, we develop a framework for analysing the economic properties of the competitive equilibrium of a market for Grid services. The objectives of the framework are mainly to provide a single broad model, instead of a set of separate models, within which a range of issues can be assessed and to look separately at the demand and supply sides of the market, while recognising the relation between the two sides. This framework also permits validation of the model, by using numerical and simulation methods to assess the broad features of the model and derives a set of conclusions about the viability of a market for Grid services (both spot and forward), based on the underlying parameters of the analysis: for example, the cost technologies, the random processes driving demand and supply, the size of the market (number of participants). In section 2 a discussion on the competitive market model and our basic assumptions is being presented. The definition of our model, as well as the parameters studied in this paper is being described in section 3. A general solution for the competitive equilibrium with a Grid spot market is being presented in section 4. The simulation environment in which our experiments are being

30

R. Mason, C. Courcoubetis, and N. Miliou

conducted is described in section 5. Finally, the analysis of the situation when the market fails to satisfy the demand in computational resources is being analyzed in section 6.

2

Our Approach

The starting point of economic analysis is usually a competitive market model, with no market power on either the buyer or provider side. In this framework, the price and quantity of Grid services are determined by competitive equilibrium in which demand and supply are equal. In order to analyse this equilibrium outcome, we must first model explicitly the demand and supply sides of the market for Grid services. We shall do so by considering the dynamics of resource allocation for both users and providers of Grid services. On the demand side, each user is subject to random arrival of jobs that need to be completed. These jobs might also be of different sizes or durations. Each user has to decide whether to use the Grid or buy its own resources. Economies of scale, through e.g., multiplexing, mean that the Grid may have lower costs; but uncertainty over total demand and supply means that the user cannot be certain of getting adequate resources at a given price from the Grid. The user’s decision optimally trades off these factors. The aggregate of different users’ decisions determines at any time the total demand for Grid resources. The supply side is modelled similarly. Each provider of Grid resources has its own capacity for computational tasks; and faces random usage patterns for that capacity. It releases capacity to the Grid whenever its own demand for resources is less than its own capacity. The Grid then offers a way in which a provider of computational resources can recover the costs of capacity investment, in the event that its own demand falls short of capacity. The aggregate of different providers’ decisions determines at any time the total supply of Grid resources. Hence, the demand and supply sides of the model are related in a two-stage procedure. In the first stage, the economic agents choose the amount of computational resources to buy for themselves. In the second stage, each agent observes its own idiosyncratic demand for computational resources. Those with too little capacity buy resources from the Grid, forming the demand side of the model. Those with too much capacity offer resources to the Grid: the supply side. Differences between the agents—primarily heterogeneity in the size and duration of the jobs—mean that some agents buy very little capacity and are systematically on the demand side of the market. Under this approach, it is assumed that there is a single homogeneous good that is to be sold through the market. Hence, no differentiation among resources is feasible. Such a commodity market assumes a single market price for a unit of the good sold. There are clear indications that current virtualization technologies justifies such a commoditization assumption, for instance Amazon EC2 cloud sells virtual machines of similar specifications. That is, it is assumed that all market actors, both consumers and producers, have a small fraction of the total market share. This way, they cannot affect the market price by means of

A Framework for Analyzing the Economics of a Market for Grid Services

31

a strategy, thus the market is perfectly competitive and all agents are pricetakers. It is clear that this assumption does not hold in the existing oligopolistic commercial Grid environment where few large providers dominate the market. The first benchmark to analyse is the sequence of competitive spot markets, where demand and supply are equalised in each period. The spot market is analysed under a number of different scenarios concerning the stochastic processes determining the demand and supply sides of the market. In order to analyse the use of forward and options markets, we shall measure the extent to which Grid market participants are willing to pay to avoid the risk attached to an uncertain Grid market price. This is one key source of uncertainty facing Grid participants. (The other is their own demand for computational services; we do not consider insurance against this sort of risk. In a sense, the Grid market performs this role, since it allows users to buy and sell resources, in the event that their demand exceeds or falls short of their own supply of computational resources.) Risk averse users, faced with uncertainty about demand and supply conditions, will want to buy insurance against that uncertainty. With a very large number of users (in the perfectly competitive limit, a continuum), there is no aggregate uncertainty. But for each user and provider, of course, there would be uncertainty in each time period. The most direct analysis is to calculate the risk premium that users are willing to pay to insure themselves entirely against market fluctuations. The aggregate risk premium gives a measure of the potential for forward and options markets for Grid services to develop. This risk premium can be calculated numerically under a number of different scenarios, using simulations of the competitive market outcome over time.

3

The Model

There are N agents, indexed by i. There are an infinite number of discrete time periods, indexed by t. Each time period t is divided in to (up to) three stages. During the first stage the amount of resources to be bought is being determined: agent i chooses to buy an amount xi (t) ∈ R+ of computational resource. The cost of buying an amount x of this resource is c(x). We make the following general assumptions about the cost function: buying nothing costs nothing c(0) = 0; variable cost of acquiring the resource is increasing and convex in the resource amount: c (·) > 0, c (·) > 0 for x > 0. Taken together, these features mean that (a) agents can rationally choose not to buy any resource; (b) demand for resources is bounded (due to the convexity of variable costs). c(·) is a key (functional) parameter of the model. There are a number of modelling options for how resources evolve over time. One extreme is that a resource is non-durable, so that each agent starts each period with a zero stock of the resource. The opposite extreme is that the resource is infinitely durable; each agent therefore accumulates the resource over time, with no loss or degradation of the resource. In this case, the resource available to agent i in period t is Xi (t) = Xi (t − 1) + xi (t). In between these two extremes, the resource may be durable but suffer from depreciation

32

R. Mason, C. Courcoubetis, and N. Miliou

(e.g., technical obsolescence). We model this by using the depreciation coefficient 1 − β, where 0 ≤ β ≤ 1. Hence, the resource available to agent i in period t becomes Xi (t) = βXi (t − 1) + xi (t). The second stage is initiated when the demand is being realized. Agent i’s demand parameter yi (t) ≥ 0 is a random variable at the time that it chooses its resource level. The value of the parameter is realized after the resource level is chosen, but before the Grid market operates. A key modelling question is the process that yi follows. One representation is yi (t) = αi yi (t − 1) + it where yi (0) is a parameter; and it is drawn from a distribution with zero mean (e.g., a normal), subject to yi ≥ 0. In this first-order autoregressive process, αi is the degree of persistence in the process for {yi (t)}. If αi = 0, then {yi (t)} is a random walk. The third stage is the stage of the actual operation of the Grid market. Agents participate in the Grid, according to their resource levels and demand realizations. Potentially, an agent’s maximization problem is very complex: choices in a given period can depend on choices made in the previous period; the current period’s environment (e.g., realisations of demands); and the evolution of the system in the future. It would be extremely difficult to solve this system fully. Instead, we make the simplifying assumption that agents are ‘myopic’, solving in each period a static maximization problem subject only to the constraints generated by previous choices. In particular, the effect of current choices on future maximization problems is ignored. In this case, agent i’s maximization problem in period t is   max yi (t)u(z) − q(t)(z − (βXi (t − 1) + xi (t))) . (1) z≥0

u(·) is a utility function that is common to all agents and z models the aggregate need in computational resources; all heterogeneity is therefore contained within the realizations of {yi }. u(·) is assumed to be (weakly) increasing: for all agents, a greater use of resources cannot decrease utility. q(t) is the unit resource price that prevails in the Grid market at time t. An important modelling question is the functional form of the utility function. Since we shall be interested in considering forward markets, which effectively provide insurance for agents, risk aversion will be a key component of the model. For this, we need concavity of the utility function.

4

Solving for Competitive Equilibrium with a Grid Spot Market

In a competitive Grid spot market, a market-clearing condition must hold:   zi∗ = Xi (t), (2) i

zi∗

i

where is agent i’s optimal choice of z. Note that this market-clearing condition will indeed hold with equality. There cannot be more (instantaneous) demand

A Framework for Analyzing the Economics of a Market for Grid Services

33

  than there is supply: that it is not possible for i zi∗ > i Xi (t). If there  is, were excess supply—if i zi∗ < i Xi (t)—then this would imply that the price of Grid resources, q(t), would be zero. Then there would be no cost to increasing use of the resource (see equation (1)); consequently, each agent would raise its zi until the market-clearing condition (2) holds with equality. Each agent i perceives that the Grid price q(t) is unaffected by its choice of resource xi (t) and resource consumption zi . (This follows from the competitive market environment that we consider.) In this case, the first-order condition characterizing the optimal consumption of resources is    u−1 yq(t) yi (t) > 0, ∗ i (t) zi (q(t); yi (t)) = (3) 0 yi (t) = 0. Note, therefore, that agent i’s optimal consumption of the resource is independent of its own resource level xi . q(t) satisfies   zi∗ (q(t); yi (t)) = Xi (t), (4) i

i

and hence is a function of the vector of demand realizations y(t) and the total resource level X(t) ≡ i Xi (t). Denote the market-clearing Grid price as q(y(t), X(t)) to emphasize this dependence. Supposedly agent i solves the following optimization problem, when choosing its resource level max Ey yi (t)u(zi∗ (q(y(t), X(t)) − q(y(t), X(t))(zi∗ (q(y(t), X(t))− x

(βXi (t − 1) + x)) − c(x) . (5)

Consistent with the competitive market setting, agent i does not consider the effect of its choice of xi on q since X(t) is much larger than x. Therefore, the optimization problem that is being solved in (5) is max Ey [q(y(t), X(t))]x − c(x). x

(6)

Essentially, xi maximizes the net benefit of the user from the sale to the Grid at price q. Hence the first-order condition for its resource choice is   x∗i = c−1 Ey [q(y(t), X(t))] (7) if x∗i > 0 is optimal. However, note that this is not the exact optimization problem that an agent faces when choosing its resource level. Namely, the agent has to take into consideration the fact that resources being bought at time slot t in general last for multiple time slots and their net present value by selling them to the Grid should be used in the optimization problem 5 (instead of the revenue at time t only). Namely, if r is the rate of return that could be earned on an investment with similar risk, the net present value (NPV) of x (depreciated by 1 − β in each

34

R. Mason, C. Courcoubetis, and N. Miliou

 βs period) is ∞ s=t xq(s) (1+r)s . This problem is in general difficult to solve since the future price values are not known. Instead, we assume for the purpose of an approximation of the NPV that the price remains constant and equal to the r+1 current price. In this case the NPV simplifies to r+1−β xq(t) and the subproblem 6 is modified to r+1 max Ey [q(y(t), X(t))]x − c(x). (8) x r+1−β As a consequence, problem (7) is modified to x∗i = c−1



 r+1 Ey [q(y(t), X(t))] . r+1−β

(9)

Equation (9) with equations (3) and (4) define the market-clearing price q(t; y, X(t − 1)). Hence the sequence of events to solve the model in any given period t is – Compute the optimal resource demands x∗ (t) (which depends on expectations of the realizations of the agents’ types y(t) and the existing stock of resources x∗ (t − 1)). – Take the instantaneous realization of the agents’ types y(t). – For that realization and the agents’ resources, compute the market-clearing price q(t). – Given the market-clearing price q(t) and the realization of agents’ types y(t), compute the optimal demand z∗ (t). – Compute metrics of the performance of the Grid market: the aggregate indirect utility of participants:   

U (t) ≡ Ey yi (t)u(zi∗ (t)) − q(t)(zi∗ (t) − (βXi (t − 1) + x∗i (t))) − c(x∗i (t)) i

and the realized volume of trade: V (t) ≡ 4.1

 i

(10) |zi∗ (t) − (βXi (t − 1) + x∗i (t))|.

The Autarkic Benchmark

The outcome with a Grid spot market can be contrasted to the situation in which there is no trading of resources subsequent to demand realizations. Each agent’s maximization problem for the demand stage is then max{yiu(z)} z

subject to z ≤ xi .

(11)

The solution to this is simply zi† = xi : each agent consumes its own entire resource. For the optimal choice of resource under autarky, the agent faces max{Ey [yi ]u(x) − c(x)} x

(12)

which has a solution x†i given by Ey [yi ]u (x†i ) = c (x†i ).

(13)

A Framework for Analyzing the Economics of a Market for Grid Services

5

35

A Simulation Environment

In this section, we illustrate the main properties of a competitive Grid spot market by means of presenting a simple example and then establishing a general simulation environment. 5.1

An Example

To illustrate the model, this subsection looks at a very simple example. For the functions in the model, take the following forms c(x) = γx2 , u(z) = ln(z).

γ > 0;

(14) (15)

Suppose that the resource is entirely non-durable, so that the agents start each period with a zero resource stock. Suppose that each agent’s type yi is drawn from the uniform distribution on [0, 1] independently each period; the types of any two agents are independent draws from the same distribution. Given the realization of its type, agent i’s optimal demand is given by yi . q

zi∗ = Hence total demand is 

 zi∗ =

i

Market clearing requires that



∗ i zi

=

i

yi

q  i

(16)

≡

Y . q

(17)

x∗i , so that

 Y = x∗i ≡ X q i

(18)

or q = Y /X. Agent i’s optimal resource level is given by x∗i =

1 1 N Ey [q] = Ey [Y ] = . 2γ 2γX 4γX

(19)

Since x∗i = X/N , this means that X N = N 4γX

(20)

so that N X= √ ; 2 γ

1 x= √ . 2 γ

(21)

36

R. Mason, C. Courcoubetis, and N. Miliou

Elementary calculations show that Y is a random variable with a mean of N/2 and density function of  1 N! (−1)k (y − k)N −1 sgn(x − k). 2(N − 1)! k!(N − k)! N

(22)

k=0

These facts can be used to compute the (expected) volume of trade  N yi 1

V =E | √ − √ 2 γY 2 γ i

(23)

and the expected utility of each agent q 1

U ∗ = Ey yi ln(yi ) − yi ln(q) − yi + √ − . 2 γ 4

(24)

Note that there are two sources of uncertainty for the agent: its own demand, determined by yi ; and the aggregate market conditions, summarised by the marketclearing price q. Because of risk aversion, both sources of risk decrease the agent’s utility; that is Ey yi ln(yi )] < Ey [yi ] ln(Ey [yi ]), and Ey yi ln(q)] < Ey [yi ] ln(Ey [q]). (25) Since we are interested in the role of forward contracts, we calculate the size of the ‘risk premium’ attached to the market price, which is   RP ≡ Ey [yi ] ln(Ey [q]) − Ey [ln(q)] . (26) The risk premium is the cash amount that an agent would be willing to pay in each period in order to avoid the uncertainty attached to the market-clearing price. It is, therefore, the fee that a broker could charge to write a forward contract that insures the agent against fluctuations in the Grid price. These variables can be contrasted with the outcome from autarky: 1 x†i = √ , 4γ

zi† = x†i ;

(27)

the expected utility from autarky is2 1 1 U † = − ln(4γ) − . 4 4 2

(28)

Note that the expected utility under autarky may well be negative. This is not a problem: it is simply a question of normalization. The utility functions used in this model are, technically, von Neumann-Morgenstern utility functions, and hence are unique up to affine transformations. Put more simply, any constant can be added to the agents’ utilities without changing the outcome of the model. Hence negative utilities can be made positive by adding a large enough positive number. We take the simpler course of using U † as the benchmark utility level, and comparing it to U ∗ , the utility level with a Grid market.

A Framework for Analyzing the Economics of a Market for Grid Services

5.2

37

A More General Simulation Environment

In this subsection a more general simulation environment is being presented. Before describing the environment we provide three important remarks. First, we introduce demand parameters {yi } of general distributions. In the simulation experiment, we suppose that an agent’s demand parameter in period t is drawn from a Gamma distribution, with a scale parameter of 1 and a shape parameter ki . An agent’s shape parameter is determined initially (before the market opens) as a random draw from a uniform distribution on [0,1]. Hence agents are now ex ante asymmetric: agent i has a mean demand E[yi ] = ki , so that a high ki corresponds to high average demand. Draws between periods, and across agents, are conditionally independent (i.e., conditional on the realizations of {ki }). But note that the realization of ki induces intertemporal dependence in the realizations of agent i’s demand levels {yi (t)}t≥0 . Agents do not know in any period the initial realizations of {ki }, nor the per-period realizations {yi (t)}. It is common knowledge how the {ki } and {yi (t)} are generated. This is an extremely flexible way of specifying demand. The motivation behind choosing Gamma distribution consists in properties such as the divisibility and the relationship of Gamma distribution to the Gaussian and exponential distributions. Secondly, we allow agents to accumulate the resource over time either completely or partially, as has been already mentioned. Thirdly, we suppose that the utility function u(·) is given by  1−ρ z ρ > 0, ρ = 1 u(z) = 1−ρ (29) ln(z) ρ = 1. In general, u (z) = z −ρ > 0; u (z) = −ρz −(1+ρ) < 0;

−zu (z) = ρ. u (z)

(30)

These three properties mean that: 1. Utility is increasing in the amount consumed. 2. Utility is strictly concave, so that agents are risk averse. 3. The Arrow-Pratt coefficient of relative risk aversion, defined as −zu /u , is constant. Hence agents’ attitudes towards risk are parameterized completely by the constant ρ. We assume that the cost of acquiring a resource level x is γx2 , where γ > 0. With this set-up, agent i’s optimal demand in period t is  1 yi (t) ρ ∗ zi (t) = . (31) q(t) Market-clearing requires that 1   yi (t) ρ  = X(t) = Xi (t), q(t) i i

(32)

38

R. Mason, C. Courcoubetis, and N. Miliou

which means that  q(t) =



1

ρ

ρ i yi (t)

X(t)ρ

.

(33)

Hence

E[q(t)] =

 ρ 

1 ρ E i yi (t) .

(34)

x∗i (t) =

1 r+1 E[q(t)], 2γ r + 1 − β

(35)

x∗ (t) ≡

N r+1 E[q(t)]. 2γ r + 1 − β

(36)

X(t)ρ

Agent i’s optimal choice of resource in period t is

so that

Since X(t) = βX(t − 1) + x∗ (t), this gives a non-linear equation in X(t):  ρ 

1 N r+1 (X(t) − βX(t − 1))X(t)ρ = E yi (t) ρ 2γ r + 1 − β i

(37)

which can be solved numerically. With the resulting expression for X(t), the rest of the system can be computed. The outcome with trading in a competitive Grid spot market can be contrasted with autarky: x†i

 =

ki 2γ

1 1+ρ

, zi† = x†i .

(38)

The simulation experiment proceeds as follows: 1. We fix the number of agents. 2. Each agent starts with zero resources: Xi (0) = 0 for all i. 3. A vector of demand parameters k is drawn, where each ki is an independent draw from the uniform distribution on [0,1]. 4. Given its ki parameter, and the consequent expectations about its demand yi (0) and the market-clearing price q(0), agent i chooses its resource level x∗i (0) (as described above). 5. Agent i’s demand level yi (0) is realised, as a draw from the Gamma distribution Γ (ki , 1). Given this demand, and the prevailing market-clearing price, agent i chooses its optimal demand zi∗ (0). 6. The market-clearing price ensures that aggregate demand is equal to aggregate supply.

A Framework for Analyzing the Economics of a Market for Grid Services

39

0.7

ρ=2,β =1 ρ=3,β=1 ρ=2,β=0.8 ρ=3,β=0.8

0.6

0.5

0.4

0.3

0.2

0.1

0

0

5

10

15

20

25

30

35

40

45

50

Fig. 1. The market clearing price as a function of time. We observe that as resources have a higher depreciation rate, equilibrium prices increase, since less resources are available in the Grid market for trading.

7. The experiment proceeds to the next period t = 1, and the steps are repeated. 8. We use multiple (independent) random draws of {yi (t)} in each period t to calculate numerical averages and expectations (i.e., we use a Monte Carlo method). The output of the experiment is then the average expected utility of the agents (where the average is taken over the demand parameters {ki }) under Grid trading and autarky, in each period; the average volume of trade and the risk premium, in each period; and the total level of resources X(t) over time. Such results are shown in the figures 1–4, in which there are 20 agents; the risk aversion parameter either ρ = 2 or ρ = 3; and the cost parameter γ = 1. Experiments showed that accounting for the net present value of the resource x instead of myopically considering the revenue at time t only, does not result to a substantially different behavior of the system. Therefore we conduct our experiments, for computational convenience, with a fixed rate of return 0.2. The figures show some common patterns. The activity levels of the Grid market increase over time, in terms of the average expected utility of the market participants and the average volume of trade. Of course, in any particular period, the volume of trade (figure 2) may decrease, as a result of low realizations of demand. But on average, over a number of periods of trading, trade volumes increase. As agents accumulate resources, the aggregate supply in the market increases: see figure 3. As a result, the market-clearing price is generally decreasing: figure 1. (Again, in any given period, the market-clearing price may increase, as a result of high realizations of demand.) As the resource stock increases over time, it acts as a buffer to smooth out fluctuations that arise due to random demand realizations. Hence the variability of the market-clearing price declines over time; consequently, the average risk premium also falls: see figure 4. Changing the

40

R. Mason, C. Courcoubetis, and N. Miliou

90 ρ=2,β=1 ρ=3,β=1 ρ=2,β=0.8 ρ=3,β=0.8

80

70

60

50

40

30

20

10

0

0

5

10

15

20

25

30

35

40

45

50

Fig. 2. The average volume of trade as a function of time. We observe that as resources have a higher depreciation rate, the volume of trade decreases. This may be due to the higher equilibrium prices, see Fig. 1.

5 ρ=2,β=1 ρ=3,β=1 4.5

4

3.5

3

2.5

2

1.5

1

0.5

0

5

10

15

20

25

30

35

40

45

50

Fig. 3. The average resource level per agent as a function of time

depreciation coefficient 1−β also introduces changes in the market-clearing price and the average volume of trade, as it can be seen in figures 1 and 2 respectively. More specifically, when the resources are less durable, ie. the parameter β decreases, the market-clearing price increases, since less resources are available in the Grid market, see figure 1 . The increase of the equilibrium prices results to a lower value for the volume of trade, as can be seen in figure 2.

A Framework for Analyzing the Economics of a Market for Grid Services

41

−3

4.5

x 10

ρ=2, β=1 ρ=3, β=1 4

3.5

3

2.5

2

1.5

1

0.5

0

0

5

10

15

20

25

30

35

40

45

50

Fig. 4. The average risk premium as a function of time

6

Market Failure

The analysis so far has demonstrated that the operation of a Grid spot market can improve the expected utility of market participants, relative to autarky (no Grid trading). Further, since there is a positive risk premium that risk-averse agents attach to variability in the market-clearing price, there is scope in the market for the use of forward contracts. The risk premium decreases with the size of the market, and as the aggregate supply in the market increases. Nevertheless, it is always positive. Furthermore, we have assumed that all agents have the same degree of risk aversion. In practice, agents will differ in their risk preferences. Forward contracts will be particularly attractive to more risk-averse agents; and will be written by less risk-averse agents. The fact that Grid trading increases agents’ utilities is not surprising, given the general results from economics about the efficiency of competitive markets. Note, however, that the solution that we have analysed does not achieve full efficiency, since we have assumed that agents behave myopically in each period, ignoring the consequences in future periods of current choices. The main set of assumptions underlying the analysis are those that make up a competitive market: 1. Each agent ignores the effects of its own choices on the market-clearing price (even when there is a very small number of agents in the market). 2. There is no asymmetric information in the market: for example, all agents are equally well-informed about the characteristics of Grid resources. 3. There are no external effects: an agent’s choices of resource level and demand have no direct effects on other agents (other than those that operate through the market-clearing price).

42

R. Mason, C. Courcoubetis, and N. Miliou

Of these three assumptions, assumption 2 is likely to be the most critical for the existence and operation of a Grid market. Market power—the ability of agents to influence the market price—will affect the volume of trade and utility levels associated with the market. But it is very unlikely to cause a complete collapse of the market. Similarly, externalities may limit the scale of the market, but are unlikely to cause collapse. But if agents are poorly informed about the quality of resources traded in the market, then it may be that the market fails to operate at all. In order to assess the scope for this market failure, we study the simplified case, when β = 0 and ρ = 1, where c(x) = γx2 ,

γ > 0,

u(z) = ln(z); the resource is entirely non-durable; and each agent’s type yi is drawn from the uniform distribution on [0, 1] independently each period. Now suppose that an agent knows for certain that its own resource is reliable and can be used with probability 1 to service its demand. If an agent has to buy resources in the Grid, then it perceives that there is some probability φ that the Grid resources fail to satisfy its demand. In the event that this happens, the agent has to pay the cost of buying the resource in the market, and bears a utility cost of λ. The agent’s expected utility is then  yi ln(z) − q(z − xi ) z ≤ xi , ui = (39) yi ((1 − φ) ln(z) + φλ) − q(z − xi ) z > xi . Agent i’s optimal demand in this case is ⎧y i yi ≤ qxi , ⎪ ⎨q ∗ zi = xi qxi < yi ≤ ⎪ ⎩ yi qxi (1 − φ) q 1−φ < yi .

qxi 1−φ ,

The expected market-clearing price is  √ (1 − φ)(1 − 1 − φ) E[q] = 2γ . φ

(40)

(41)

The dependence of the expected market-clearing price on the failure parameter φ is shown in figure 5. As the figure shows, the market-clearing price is higher when the failure probability is lower. In the limit, when any Grid resource fails with probability 1, the market-clearing price is 0, and the market collapses for sure. Consequently, each agent buys  √ (1 − φ)(1 − 1 − φ) ∗ xi = (42) 2γφ of the resource.

A Framework for Analyzing the Economics of a Market for Grid Services

43

1 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 0

0

0.2

0.4

0.6

0.8

1

Fig. 5. The market-clearing price against the failure parameter φ

−0.5 −0.55 U+ −0.6 −0.65 −0.7 U*

−0.75 −0.8 −0.85 −0.9 −0.95 −1

0

0.2

0.4

0.6

0.8

1

Fig. 6. Expected utilities against the failure parameter φ

These facts can then be used, along with the realizations of the demand parameters {yi }, to determine numerically the equilibrium outcome, as a function of the failure parameter φ. It can be seen in figure 6, that if φ is sufficiently low agents will trade in the Grid market. Once Grid resources become sufficiently unreliable, autarky is preferred.

44

R. Mason, C. Courcoubetis, and N. Miliou

Obviously, a key parameter determining the ‘crossing point’—the critical value of φ at which the Grid market collapse—is the utility cost λ. We have modelled this in a very simple way here: as a constant that is the same for all agents. If this is an important feature of an emerging Grid market, then this aspect can be modelled further. For example, the utility cost could be allowed to be different for different agents. It could also be related to agents’ realized demand types y.

7

Conclusions

In this paper we analyzed the economics of a competitive market for Grid services. Namely, we provided an analytical model, as well as simulation results regarding the critical parameters of such a model. Accounting for different levels of risk aversion, we studied the behavior of the market clearing price, the volume of trade, the resource level per agent and the average risk premium. In all those cases we assumed that the resources do not necessarily vanish after the end of each period, but last longer and each agent accumulates a stock of such resources. We observed that in general the market clearing price decreases and the aggregate supply in the market increases as agents accumulate resources. We also analyzed the situation where Grid resources are not reliable and the market fails to satisfy the demand and observed that, unless Grid resources are highly unreliable, the agents still prefer to trade in the Grid market than to choose autarky.

References 1. Chamberlin, E.H.: An Experimental Imperfect Market. Journal of Political Economy 56(2), 95–108 (1948) 2. Cripps, M.W., Swinkels, J.M.: Efficiency of Large Double Auctions. Econometrica 74(1), 47–92 (2006) 3. Gode, D.K., Sunder, S.: Allocative Efficiency of Markets with Zero Intelligence (ZI) Traders: Market as a Partial Substitute for Individual Rationality. Journal of Political Economy CI, 119–137 (1993) 4. Gode, D.K., Sunder, S.: What Makes Markets Allocationally Efficient? Quarterly Journal of Economics 112(2), 603–630 (1997) 5. Rustichini, A., Satterthwaite, M.A., Williams, S.R.: Convergence to Efficiency in a Simple Market with Incomplete Information. Econometrica 62(1), 1041–1063 (1994) 6. Satterthwaite, M.A., Williams, S.R.: The optimality of a simple market mechanism. Econometrica 70(5), 1841–1863 (2002) 7. Cliff, D.: Minimal-intelligence agents for bargaining behaviours in market-based environments, Technical Report HP-97-91, Hewlett-Packard Research Laboratories, Bristol, England (1997) 8. Preist, C., van Tol, M.: Adaptative agents in a persistent shout double auction. In: Proceedings of the 1st International Conference on the Internet, Computing and Economics, pp. 11–18. ACM Press, New York (1998)

A Framework for Analyzing the Economics of a Market for Grid Services

45

9. Parsons, S., Marcinkiewicz, M., Niu, J., Phelps, S.: Everything you wanted to know about double auctions but were afraid to (bid or) ask, Technical Report, Department of Computer & Information Science, Brooklyn College, City University of New York (2005) 10. Smith, V.L.: An Experimental Study of Competitive Market Behavior. Journal of Political Economy LXX, 111–137 (1962) 11. Meinl, T., Neumann, D.: A Real Options Model for Risk Hedging in Grid Computing Scenarios. In: Proceedings of the 42nd Hawaii International Conference on System Sciences, pp. 1–10 (2009)

The GridEcon Platform: A Business Scenario Testbed for Commercial Cloud Services Marcel Risch1, Jörn Altmann1, Li Guo2, Alan Fleming3, and Costas Courcoubetis4 1

TEMEP, School of Industrial and Management Engineering College of Engineering, Seoul National University 599 Gwanak-Ro, Gwanak-Gu, Seoul 151-742, South-Korea [email protected], [email protected] 2 Department of Computing, Imperial College London 180 Queen's Gate, London SW7 2BZ, UK [email protected] 3 Real Time Limited Lochside House, 3 Lochside Way, Edinburgh Park, Edinburgh EH12 9DT, UK [email protected] 4 Department of Computer Science, Athens University of Economics and Business 47A Evelpidon Str., Athens 11362, Greece [email protected]

Abstract. Within this paper, we present the GridEcon Platform, a testbed for designing and evaluating economics-aware services in a commercial Cloud computing setting. The Platform is based on the idea that the exact working of such services is difficult to predict in the context of a market and, therefore, an environment for evaluating its behavior in an emulated market is needed. To identify the components of the GridEcon Platform, a number of economicsaware services and their interactions have been envisioned. The two most important components of the platform are the Marketplace and the Workflow Engine. The Workflow Engine allows the simple composition of a market environment by describing the service interactions between economics-aware services. The Marketplace allows trading goods using different market mechanisms. The capabilities of these components of the GridEcon Platform in conjunction with the economics-aware services are described in this paper in detail. The validation of an implemented market mechanism and a capacity planning service using the GridEcon Platform also demonstrated the usefulness of the GridEcon Platform. Keywords: Cloud computing, Grid computing, marketplace for computing resources, market-based resource allocation, service-oriented architectures, utility computing, market emulation tool, market mechanism.

1 Introduction In the past, many Grid research projects implemented models of markets for computing resources. Some examples include the Popcorn Market [1], the Spawn system [2], the Grid Architecture for Computational Economy (GRACE) [3], and Tycoon [4]. All J. Altmann, R. Buyya, and O.F. Rana (Eds.): GECON 2009, LNCS 5745, pp. 46–59, 2009. © Springer-Verlag Berlin Heidelberg 2009

The GridEcon Platform: A Business Scenario Testbed for Commercial Cloud Services

47

of these projects focused very much on providing economically efficient resource allocations or on using various market mechanisms to sell computing resources. However, all these implementations remained sample prototypes. They could not be re-used by other projects because of its static implementation. For example, the market mechanism developed by Popcorn is not interoperable with the market mechanism developed by Tycoon. Furthermore, all these market implementations did not consider the market environment, i.e. the value-added services around the market mechanism. To model a market environment comprehensively, we have developed the GridEcon Platform within the GridEcon Project [5] [6]. In particular, the GridEcon Platform allows evaluating economics-aware services in a commercial Cloud computing setting. The Platform is based on the idea that the exact working of such a service is difficult to predict in the context of a market and, therefore, an environment for evaluating its behavior and testing its interfaces in an emulated market is needed. The most important two components of the GridEcon Platform are the Marketplace and the Workflow Engine. The Workflow Engine component allows the simple composition of a market environment from different economics-aware services by describing the interaction between these services. The Marketplace allows the execution of different kinds of market mechanisms for trading computing resources. We demonstrate the usefulness of the GridEcon Platform by implementing a market mechanism (continuous double auction) and a capacity planning service and integrating them into the GridEcon Platform. The easy-to-perform integration enabled us to conduct performance measurements on the market mechanism and the capacity planning component. The remainder of the paper is structured as follows: In section 2, we give an overview of the state-of-the-art in Grid market research and economics-aware services for Cloud computing markets. In section 3, we describe the GridEcon Platform, in particular the Workflow Engine and the Marketplace component. Besides those, we introduce some value-added Cloud services that may be of use in a commercial computing resource environment. The implementation of the GridEcon Platform and a validation of the implementation is given in section 4. We conclude the paper in section 5.

2 State of the Art of Grid Market Research This section gives an overview of previous Grid market projects and their approach to developing market environments. This demonstrates that previous Grid market implementations did not take into account the need for platforms, on which businesses can test and evaluate their services in a realistic market setting. The Popcorn Market [1] [7] has identified three actors: buyers, sellers, and the market, which implements the market mechanism. By looking at the list of entities, it is already obvious that no support services has been considered during the implementation phase. While the market is flexible enough to support three different market mechanisms, the problem of implementing business services on top of the market has not been addressed. Therefore, this market is of limited use, if it comes to testing different business models of Cloud computing services as the effort for integrating additional services is quite high.

48

M. Risch et al.

The Spawn system [2] presents the idea of value-added services but without considering the full economic potential and the requirements of these services. Since the implementation of these services and their integration into the Spawn environment will be challenging, this environment is of limited use if it comes to modeling the behavior of additional market services within Spawn. The Grid Architecture for Computational Economy (GRACE) [3] has identified one central component: “The resource broker acts as a mediator between the consumer and Grid resources using middleware services. It is responsible for resource discovery, resource selection, binding of software, data, and hardware resources, initiating computations, adapting to the changes in Grid resources and presenting the Grid to the consumer as a single, unified resource” (p. 4, [3]). It can be clearly seen from the task description, that the Grid Resource Broker is a component with many functionalities, which do not have to be provided by the same entity. This would be the idea starting point for looking into services, which can be provided by various businesses competing for customers. However, since the Grid Resource Broker represents a monopolistic service provider, the GRACE implementation cannot be used to determine how the existence of different providers with different business models will affect the market. Furthermore, extending this architecture to provide different services would require significant effort which makes GRACE an unattractive choice for simulating economics-aware Cloud services. The Tycoon market is a highly distributed market [4], in which agents take over a number of jobs for the consumer. These jobs include: (1) interpreting the consumer’s preferences, (2) examining the state of the system, (3) make bids, and (4) verify that the acquired resources are available. The first job is very important to consumers, who have little or no Grid expertise and need help to determine the correct resource purchase. It is also a very challenging job, since consumers may not know how to express their preferences due to their lack of IT expertise. In such a case, the agent has to perform tasks such as capacity planning, portfolio management, and workflow analysis. All these tasks are highly complex and, thus, mean that the agents have to be very powerful to be able to perform them. The main weakness of this model is that, while agents could be provided by different providers, the idea of an agent market, in which consumers can buy agent services, has not been developed. Beyond the Grid Markets described so far, some additional research comes from the gridbus [8] project. This has been the only project that has introduced a number of services for the user, namely a Workflow Broker [9], the Nimrod-G Broker [10], and the Gridbus Data Broker [11]. Furthermore, gridbus has developed the GridSim [12] and CloudSim [13] simulation environments. However, these environments are aimed at analyzing the effects of different resource allocation algorithms as well as the effects of resource failures. The effect of additional services cannot be easily modeled in these environments. Neumann et al described the technological and economical challenges, which must be overcome when developing a Grid market, in [19]. In conclusion, existing Grid Market research has not considered the idea of allowing an evaluation of new services added to an existing market or an evaluation of a new design of a market. In particular, a component that helps composing these services has also not been proposed.

The GridEcon Platform: A Business Scenario Testbed for Commercial Cloud Services

49

3 The GridEcon Platform The GridEcon Platform, the architecture of which is shown in Figure 1, consists of three tiers: the service provider tier, the marketplace tier, and the economics-aware services tier. Each of these tiers is built using a service-oriented architecture (SOA). The interaction between the tiers and between the different value-added services is definable through the Workflow Engine. In order to achieve that, the Workflow Engine provides common interfaces to services. The marketplace tier encompasses the main trading location of this environment. It is flexible enough to allow trading of computing resources to be based on any kind of market mechanism such as auctions, bargaining, or posted prices. The service provider tier contains all providers of services, which are traded in the marketplace. No distinction will be made of the type of services. This means that the GridEcon Platform could be used to sell access to data, Software (Software-asa-Service), and computing resources. However, for simplicity, we focus on computing resources for the remainder of the paper exclusively.

Fig. 1. The GridEcon Marketplace Environment

The economics-aware service tier is the place where different providers can offer their value-added services. Although the GridEcon project has considered a number of services, which could be of use in such an environment, in a production-level market, there may be demand for additional services. In order to illustrate the workings of the GridEcon Platform in detail, in the following subsections, we will focus, in particular, on the Workflow Engine, the Marketplace component, and the Capacity Planning Service as an example for a value-added service.

50

M. Risch et al.

3.1 Workflow Engine Component The Workflow Engine is a component designed to facilitate the coordination and communication between the different components and services of the GridEcon Platform. With respect to the GridEcon architecture, the Workflow Engine sits between all other GridEcon Platform components and value-added services. It hides the underlying system complexity and the interactions within the system from end-users and, thus, makes the GridEcon Platform easier to work with. The entire system architecture for the Workflow Engine is shown in Figure 2. The Workflow Engine is a specification-driven system, i.e. the composition of the components and value-added services is defined by feeding a specification (i.e. workflow) into the Workflow Engine. This idea was borrowed from the multi-agent domain. This sort of system design significantly reduces the effort required for developing a market environment by reducing the development life cycle. The development of a different market environment is made simple by feeding the new workflow into the Workflow Engine. Because of this design, the Workflow Engine can also manage different market mechanisms. In other words, the GridEcon Platform is independent of any specific market mechanism. The Workflow Engine consists of a group of agents, where each agent conceptually represents a GridEcon component or value-added Service (e.g. Marketplace component, Capacity Planning service). Each GridEcon components is made public through Web services. The agents of the Workflow Engine communicate with each other to handle the interaction between the components. The interaction between agents follows a predefined interaction protocol, namely the current workflow. The language that is used to describe a workflow is called Lightweight Coordination Calculus (LCC) [14]. In detail, the Workflow Engine offers interfaces for users to submit asks and bids. It also offers various callback interfaces, which can be used by the economic-aware service to communicate with each other. These callback features of the Workflow Engine ensure that all communication can be performed asynchronously.

Fig. 2. Architecture of the Workflow Engine

The GridEcon Platform: A Business Scenario Testbed for Commercial Cloud Services

51

3.2 Marketplace Component The Marketplace component is responsible for managing the trading of resources. Since computing resources can be traded based on different market mechanisms, the Marketplace can be implemented with a number of different market mechanisms. For validating the Marketplace implementation, a continuous double auction (CDA) has been implemented as the market mechanism [6]. This market mechanism has been chosen, since it is similar to market mechanisms for other non-storable goods, such as electricity [15]. With respect to the Marketplace Component itself, this market mechanism has been chosen arbitrarily. The underlying principle for this market mechanism is that of a standard futures market: All parties announce the maximum price they are willing to buy for (i.e. bid price, for the purchasers) and the minimum price they are willing to sell for (i.e. ask price, for the sellers) in a specified time interval. These prices are recorded and put in a directory. Bids are, in fact, standing (or limit) orders and will not be immediately fulfilled unless there is a previously posted compatible reciprocal ask. If the bid price of a buy order exceeds the ask price of a sell order, the orders are immediately executed. If no compatible reciprocal bids are available, the bids remain in the directory until they are matched in the future or expire. The market mechanism takes into account both vertical and horizontal atomicity of bids and asks. Vertical atomicity in bids means that resources have to be provided by a single provider, however, the bidder is willing to switch resources during the usage period. Horizontal atomicity means that the resources are available for the entire usage duration by one provider, thereby minimizing switching costs. However, not all resources have to be offered by the same provider. For asks, vertical atomicity requires that all resources have to be sold to one user at a time. However, the resources can be sold to different users at different times. The horizontal atomicity requirement for asks means that the resources have to be sold for the entire availability time. This differentiation of atomicity gives buyers and sellers of computing resources the opportunity to specify whether they require all resources in one place, the resources for the entire duration, or both. The architecture of the Marketplace Component is shown in Figure 3. Since the CDA mechanism was implemented in the GridEcon Platform for testing purposes, this market mechanism has been highlighted. However, other market mechanism, such as described in [20], could also be implemented.

Fig. 3. The Marketplace Component Architecture

52

M. Risch et al.

As Figure 3 indicates, the architecture of the Marketplace Component also allows trading of different types of computing resource types (e.g. Res Type 1, Res Type 2). However, the interface of the Marketplace component remains unchanged, independent of the market mechanism and the resource type. 3.3 Value-Added Services The GridEcon project has identified a number of services, which may be of interest to resource consumers. All of these services have been examined so that the GridEcon Platform could potentially integrate any implementation of those. The first service is the Capacity Planning Service, which helps users to plan their computing resource capacity. Capacity planning is a difficult task in a commercial Cloud computing setting. The service has to analyze the existing infrastructure, the deployed applications, and the resource requirements before suggesting a course of action. An action can be the purchase of Cloud resources, the purchase of inhouse resources, or a combination of both. Furthermore, the Capacity Planning Service can suggest that some resources can be sold on the Marketplace if they are idle [16]. More detailed information about the Capacity Planning Service can be found in [17] [18]. The Fixed-Price Quotation Broker allows selling fixed price contracts even if the underlying market price for the resource is variable. This allows consumers to plan the Cloud computing costs with certainty. At the same time, this price will usually be higher than the actual price, since the Fixed-Price Quotation Broker is expected to generate profits for the provider. At the time when the contract is due, the Fixed-Price Quotation Broker places bids on the marketplace to buy the resources in order to fulfill its contractual obligations. The Portfolio Broker addresses the fear of Cloud users that their Cloud resources might fail. Since it is very difficult for Cloud users to determine which resources have a high risk of failure, they can use the Portfolio Broker to determine the failure risk of each resource in their resource portfolio. Based on this analysis and the user’s risk tolerance level, the Portfolio Broker can then suggest a method for reaching the risk tolerance level of the user. This method may require the diversification of resources to different vendors or the purchase of spare resources. The Insurance Broker provides insurances against a number of events that consumers fear. The GridEcon project has identified the following events which would be harmful to the Cloud user: (1) Non-delivery of the resources, (2) delivery of incorrect resources or (3) interruption of the resource availability. Using the Insurance Broker, consumers can protect themselves against the financial effects of these events. In return, the Insurance Broker will charge a premium which is proportional to the cost of the adverse event and the probability that this event will occur. Since all these services can be used with a number of market mechanisms, we have analyzed which services are useful with which market mechanism. The result of this analysis is shown in the following table: a √√ shows that the component is very useful; a single √ shows that the component is moderately useful, and an × shows that the component is not useful with respect to the market mechanism.

The GridEcon Platform: A Business Scenario Testbed for Commercial Cloud Services

53

Table 1. Service Comparison with Respect to the Market Mechanism Service

Capacity Planning Service Fixed-Price Quotation Broker Portfolio Broker Insurance Broker

Market Mechanism

Posted Price

Bargaining

Dutch Auction

Double Auction

√√

√√

√√

√√

×

×

√√

√√

√√ √√

√√ √√

√ √

√ √

The Capacity Planning Service is very useful regardless of the market mechanism, since its task is to help the consumer to decide which resources should be purchased and where to purchase them. Since this problem has to be solved regardless of the market mechanism, this component is useful in any utility computing market. The Fixed-Price Quotation Broker requires demand-based pricing, which is given when Dutch and double auctions are used. While posted prices can also be adjusted to reflect the demand, this occurs less frequently and, therefore, the Fixed-Price Quotation Broker will be less useful in such a market environment. For the same reason, it is also less useful in a bargaining market. The Portfolio Broker is useful in a non-anonymous market mechanism, such as the posted-price and bargaining markets. This is due to the fact that in such a market it can calculate the resource reliability not only for each resource type but also for each resource type of each provider. In an anonymized market, it can only calculate the resource reliability for each resource type across all providers. As the Portfolio Broker, the Insurance Broker is also more useful if the market mechanism is not anonymous, i.e. if providers can be uniquely identified. In this case, the Insurance Broker can determine more closely which provider is trustworthy.

4 Implementation and Validation All GridEcon Platform components and value-added services described in the previous sections can be orchestrated by the user through the Workflow Engine. Therefore, a typical GridEcon-Platform-based market environment always relies heavily on this central component. Figure 4 shows a sequence diagram illustrating the interaction of the Marketplace, Capacity Planning Service, and the Workflow Engine. It can be clearly seen in Figure 4 that the user only interacts with the Workflow Engine, even if different services are to be used. This approach hides the complexity of using the Cloud by giving the user a single point of entry into the Cloud. The following section describes the implementation of the Workflow Engine, validating the workings of the GridEcon Platform. By describing the simple integration of the market mechanism and the capacity planning service into a GridEcon-Platformbased emulated market environment, we validate the usefulness of the GridEcon Platform.

54

M. Risch et al.

Fig. 4. Interaction of Services and Components Orchestrated by the Workflow Engine

4.1 Implementation of the Workflow Engine The Workflow Engine is deployed as a Java Web Service using JAX-WS and is made available to end-users through the GridEcon Web Interface. The workflow specifications are created in XML and, therefore, can be edited by any XML editor. The system was evaluated with respect to three parameters: system performance, system reliability, and accessibility. To test these criteria, we simultaneously submitted 5000 workflow requests for the same task 1000 times. The tests were carried out on a Linux server with two 3.2GHz CPUs and 2GB memory. We used an Apache Tomcat 6.0 server running Java JRE1.6.0_02 to deploy the Workflow Engine. The results of the performance tests are shown in Figure 5.

Fig. 5. System Performance of the Workflow Engine

The best response time that we obtained on the above-mentioned configuration was 360.0864ms; the worst response time obtained was 473.3914ms. The average time for the 1000 tests was 417.1616ms. Since the GridEcon Platform did not fail during these tests, we derived that the Workflow Engine and the entire system, the GridEcon Platform, is reliable and accessible.

The GridEcon Platform: A Business Scenario Testbed for Commercial Cloud Services

55

4.2 Implementation of the Marketplace The Marketplace was implemented using Java Web Services technologies and has been deployed in the GridEcon Platform implementation as an axis2 AAR file. A major goal in the design of the Marketplace component was to provide a flexible, pluggable architecture, allowing easy configuration and expedite future development. In order to provide this extensibility and adaptability, the Spring Framework was utilized. The Spring Framework allows the central matching mechanism to be configured via dependency injection and without requiring any further compilation or deployment steps. Dependency injection allows a single market mechanism to be re-deployed with differing configuration and allows different market mechanisms to be developed and configured into the existing Marketplace component. This far, the only market mechanism implemented and tested is the continuous double auction mechanism as described in section 3.2 [5]. As one core responsibility of the Marketplace component is to initiate the start and stop of the resulting allocations (i.e. matched offers), a scheduler mechanism (through Open Symphony Quartz) was implemented. When an allocation is produced through the matching process, a scheduled event is created in Quartz that will initiate the startup of the resource at the scheduled time. The tests of the Marketplace component took the matching accuracy of the implemented CDA into account, as well as the time taken to find a match for a given bid and ask. The matching accuracy reached 100%, which was the design goal, since the matching algorithm of the marketplace is vital to the functioning of the market. Once this goal had been achieved, the performance of the matching algorithm with respect to the matching time was tested. An excerpt of the results can be seen in Table 2. The table shows the time for handling a single bid submission and the average times for servicing 50 requests under nominal load and heavy load. Nominal load corresponds to the expected average system load. The load was generated by having 30 providers submitting 300 asks and 10 bidders submitting 100 bids over a time period of 1000 trading days. The system’s performance was measured on a laptop with AMD Turion TL50 processor with 1 GB DDR2 running Windows XP. The heavy load case represents a situation in which the system has to handle increased load. In this case, 75 providers submitted 750 asks and 75 bidders submitted 750 bids over a time period of 1000 trading days. Table 2. Marketplace Performance Test Results

Task Submit Bid - No Match Submit Bid – Match

Single Transaction 45ms 43ms

Nominal Load 46ms 45ms

Heavy Load 88ms 76ms

It can be clearly seen in Table 2 that the matching process is quite fast, even under high system load. Besides validating the overall workings of the GridEcon Platform, the results also show that the Marketplace Component and, therefore, the GridEcon Platform is useful for validating market mechanisms. A market mechanism can simply be plugged into the Marketplace Component. It is simply a module within the Marketplace component.

56

M. Risch et al.

4.3 Implementation of the Capacity Planning Service To test the viability of a Capacity Planning Service and to show the usefulness of the GridEcon Platform, we developed a market simulation with the help of the GridEcon Platform. This emulation was aimed at showing that even basic prediction algorithms are valuable for capacity planning in Cloud Computing Markets. The Capacity Planning Service is expected to function within the previously described continuous double auction (section 3.2). The matching algorithm of the market applied horizontal and vertical bid atomicity, meaning that the buyer of a Cloud resource obtained all resources from a single provider. At the end of each day, each trader determines its expected demand for the next day. If the expected demand is larger than the number of in-house resources, the trader will bid for resources for the next day on the Cloud market. Should the number of required resources be lower than the number of in-house resources, the trader would attempt to sell the excess resources for the next day on the Cloud market. The traded resources are made available in the morning of the following day. Therefore, each market participant can change his role between being a buyer and being a seller. The GridEcon-Platform-based simulation consisted of 10000 traders, which traded resources for 500 trading days. A detailed overview of the simulation parameters is given in Table 3. Table 3. Simulation Parameters Overview Parameters

Number of traders Number of in-house resources (per trader) Market mechanism Number of simulated trading days Offer expiration time Demand distribution Demand distribution mean Demand distribution variance

Value

10,000 20-40 CDA 500 1 day Normal 30 30

A trader’s resource supply is calculated as the sum of existing in-house resources and purchased Cloud resources. Three of the 10000 traders were given identical demand for resources. In order to work with a realistic demand pattern, we used the results of network traffic measurements to the Web servers of the International University in Germany during a period from October 2007 to December 2008. The demand over this time period is shown in Figure 6 below. Each of these three traders used a specific prediction algorithm. The first trader used a basic prediction algorithm: It used its current demand level to determine the resource amount to be purchased or sold for the next day. The second trader used the past thirty days as a basis for a linear regression to predict demand. The third trader performed a Fourier analysis. The outcomes of the simulations are illustrated in Figure 7. A negative value shows that the trader has fewer resources than required, while a positive value indicates that the trader has more resources than it requires.

The GridEcon Platform: A Business Scenario Testbed for Commercial Cloud Services

57

Resource Demand 70

ResourceQ uantity

60 50 40 30 20 10 0 1

27

53

79

105 131 157 183 209 235 261 287 313 339 365 391 417 443 469 Time

Fig. 6. Network Traffic Demand from Web Server Measurements

Fig. 7. Resource Demand Prediction Results

It can be clearly seen in Figure 7 that, initially, all traders had sufficient resources at their disposal. However, after about 300 days, all were unprepared for a sudden demand spike. This result is expected, since all prediction methods are based on past measurements. Another factor that contributed to this result is that not necessarily all resource purchases are matched. As in any market, if there is no matching supplier, then the resource demand cannot be met. This is caused by a difference in prices or the horizontal and vertical bid atomicity that we applied. If this requirement were to be removed, the market would have much greater liquidity and the traders would have more resources at their disposal. With the help of the Capacity Planning Service, we also demonstrated how a valueadded service can easily be in integrated into a GridEcon-Platform-based market environment. The GridEcon Platform allowed integrating a prototype of the capacity planning service and conducting performance measurements on the value-added Capacity Planning Service. This highlights the usefulness of the GridEcon Platform for validating new economics-aware, value-added services.

5 Conclusion and Future Work In this paper, we have analyzed the existing approaches to utility computing markets, mainly the markets of Popcorn, Spawn, GRACE, and Tycoon. However, due to their lack of flexibility, these prototypes cannot be used to evaluate new market designs or the performance of economics-aware services in a market environment. This limitation has been recognized by the GridEcon project, which created the GridEcon Platform, a testbed for evaluating new designs of computing resource markets and for evaluating value-added services in a commercial Cloud Computing setting.

58

M. Risch et al.

As part of the GridEcon platform, two central components have been created: the Workflow Engine, which enables the composition of a market environment in a simple way, and the Marketplace component, which allows the execution of different market mechanisms. The Workflow Engine can be used by users of the GridEcon Platform to orchestrate different market scenarios needed for their analysis. The Marketplace component allows testing and validating different market mechanisms. The GridEcon Platform, in particular, the Workflow Engine, has been tested. It performed its tasks as expected and proved to be vital for developing additional valueadded services. The usefulness of the GridEcon Platform has been demonstrated by integrating a market mechanism (continuous double auction) into the GridEcon Marketplace Component and by integrating a capacity planning service into a GridEconPlatform-based emulated market environment. The GridEcon-Platform-based market environment allowed validating the market mechanism and the capacity planning service by conducting performance measurements.

Acknowledgement This work has been funded by the European Commission within the framework of the GridEcon project. GridEcon (contract no. 033634) is a STREP project under the Framework Program FP6 in the area of IST. The authors would also like to thank the members of the GridEcon consortium for their contributions. Special acknowledgement is due to Kostas Giannakakis, Alon Lahav, and Sergios Soursos.

References 1. Regev, O., Nisan, N.: The POPCORN market - An Online Market for Computational Resources. In: Proceedings of the First international Conference on Information and Computation Economies. ICE 1998. ACM, New York (1998) 2. Waldspurger, C.A., Hogg, T., Huberman, B.A., Kephart, J.O., Stornetta, W.S.: Spawn: A Distributed Computational Economy. IEEE Transactions on Software Engineering 18(2), 103–117 (1992) 3. Buyya, R., Abramson, D., Giddy, J.: An Economy Grid Architecture for Service-Oriented Grid Computing. In: 10th IEEE International Heterogeneous Computing Workshop. HCW 2001. IEEE Computer Society Press, San Francisco (April 2001) 4. Lai, K., Rasmusson, L., Adar, E., Zhang, L., Huberman, B.A.: Tycoon: An Implementation of a Distributed, Market-Based Resource Allocation System. Multiagent and Grid Systems 1(3), 169–182 (2005) 5. Altmann, J., Courcoubetis, C., Stamoulis, G.D., Dramitinos, M., Rayna, T., Risch, M., Bannink, C.: GridEcon: A market place for computing resources. In: Altmann, J., Neumann, D., Fahringer, T. (eds.) GECON 2008. LNCS, vol. 5206, pp. 185–196. Springer, Heidelberg (2008) 6. Courcoubetis, C., Dramitinos, M., Rayna, T., Soursos, S., Stamoulis, G.D.: Market mechanisms for trading grid resources. In: Altmann, J., Neumann, D., Fahringer, T. (eds.) GECON 2008. LNCS, vol. 5206, pp. 58–72. Springer, Heidelberg (2008) 7. The Popcorn Project (2008), http://www.cs.huji.ac.il/~popcorn/ 8. The gridbus project (2008), http://www.gridbus.org/

The GridEcon Platform: A Business Scenario Testbed for Commercial Cloud Services

59

9. Rahman, M., Buyya, R.: An Autonomic Workflow Management System for Global Grids. In: Proceedings of the Eighth IEEE International Symposium on Cluster Computing and the Grid. CCGrid 2008, May 2008, pp. 578–583. IEEE Computer Society, Washington (2008) 10. Buyya, R., Abramson, D., Giddy, J.: Nimrod-G: An Architecture for a Resource Management and Scheduling System in a Global Computational Grid. In: Proceedings of the 4th International Conference on High Performance Computing in Asia-Pacific Region. HPC Asia 2000, May 2000. IEEE Computer Society Press, Beijing (2000) 11. Venugopal, S., Buyya, R., Winton, L.: A Grid Service Broker for Scheduling Distributed Data-Oriented Applications on Global Grids. In: Proceedings of the 2nd Workshop on Middleware For Grid Computing. MGC 2004, October 2004, vol. 76, pp. 75–80. ACM, Toronto (2004) 12. Buyya, R., Murshed, M.: GridSim: A Toolkit for the Modeling and Simulation of Distributed Resource Management and Scheduling for Grid Computing. Journal of Concurrency and Computation: Practice and Experience 14(13-15), 1175–1220 (2002) 13. Calheiros, R.N., Ranjan, R., De Rose, C.A.F., Buyya, R.: CloudSim: A Novel Framework for Modeling and Simulation of Cloud Computing Infrastructures and Services (to be published in ICPP 2009) 14. Roberston, D.: A Lightweight Method for Coordination of Agent Oriented Web Services. In: Proceedings of the AAAI Spring Symposium on Semantic Web Services (2004) 15. European Energy Exchange (2009), http://www.eex.com/en/ 16. Risch, M., Altmann, J.: Cost Analysis of Current Grids and Its Implications for Future Grid Markets. In: Altmann, J., Neumann, D., Fahringer, T. (eds.) GECON 2008. LNCS, vol. 5206, pp. 13–27. Springer, Heidelberg (2008) 17. Risch, M., Altmann, J., Makrypoulias, Y., Soursos, S.: Economics-Aware Capacity Planning for Commercial Grids. In: Collaborations and the Knowledge Economy, vol. 5, pp. 1197–1205. IOS Press, Amsterdam (2008) 18. Risch, M., Altmann, J.: Capacity Planning in Economic Grid Markets. In: Proceedings of the 4th International Conference on Grid and Pervasive Computing 2009. GPC 2009, Geneve, Switzerland. LNCS, pp. 1–12. Springer, Heidelberg (2009) 19. Neumann, D., Stößer, J., Weinhardt, C., Nimis, J.: A Framework for Commercial Grids Economic and Technical Challenges. Journal of Grid Computing 6(3), 325–347 (2008) 20. Schnizler, B., Neumann, D., Veit, D., Weinhardt, C.: Trading Grid Services - A MultiAttribute Combinatorial Approach. European Journal of Operational Research 187(3), 943–961 (2008)

VieSLAF Framework: Enabling Adaptive and Versatile SLA-Management Ivona Brandic, Dejan Music, Philipp Leitner, and Schahram Dustdar Distributed Systems Group, Institute of Information Systems Vienna University of Technology, Vienna, Austria {ivona,dejan,leitner,dustdar}@infosys.tuwien.ac.at

Abstract. Novel computing paradigms like Grid and Cloud computing demand guarantees on non-functional requirements such as application execution time or price. Such requirements are usually negotiated following a specific Quality of Service (QoS) model and are expressed using Service Level Agreements (SLAs). Currently available QoS models assume either that service provider and consumer have matching SLA templates and common understanding of the negotiated terms or provide public templates, which can be downloaded and utilized by the end users. On the one hand, matching SLA templates represent an unrealistic assumption in systems where service consumer and provider meet dynamically and on demand. On the other hand, handling of public templates seems to be a rather challenging issue, especially if the templates do not reflect users’ needs. In this paper we present VieSLAF, a novel framework for the specification and management of SLA mappings. Using VieSLAF users may specify, manage, and apply SLA mapping bridging the gap between non-matching SLA templates. Moreover, based on the predefined learning functions and considering accumulated SLA mappings, domain specific public SLA templates can be derived reflecting users’ needs.

1

Introduction

Nowadays, well established and traditional resource sharing models are shifted towards novel market-oriented models revolutionizing existing Grid and High Performance Computing (HPC) concepts [8]. In market-oriented resource sharing models users discover resources on demand and pay for the usage of the specific resources. In turn they expect that besides requested functional requirements, non-functional requirements of the application execution are also fulfilled [1,19,12]. Non-functional requirements comprise application execution time, reliability, availability, and similar issues. Non-functional requirements are termed as Quality of Service (QoS) and are expressed and negotiated by means of Service Level Agreements (SLAs). SLA templates represent empty SLA documents i.e., SLA documents, with all required elements like parties, SLA parameters, metrics, and objectives, but without QoS values [9]. A large body of work deals with SLA based QoS negotiation and integration of QoS concepts into Grid management tools [10,17]. However, most of the J. Altmann, R. Buyya, and O.F. Rana (Eds.): GECON 2009, LNCS 5745, pp. 60–73, 2009. c Springer-Verlag Berlin Heidelberg 2009 

VieSLAF Framework: Enabling Adaptive and Versatile SLA-Management

61

existing work relies either on inflexible QoS models assuming that the communication partners have matching SLA templates or provide public SLA templates, which can be downloaded and utilized on the users’ system. On the one hand, matching SLA templates limit QoS negotiation only between partners where QoS relationship is already established off-line, or to partners who belong to a particular Grid portal [1]. On the other hand, publicly available SLA templates usually do not reflect users’ needs. Thus, in order to increase QoS versatility, flexible QoS models are necessary where negotiation is possible even between services which do not have matching SLA templates. The problems with nonmatching templates can be exemplified on a very simple example with differing terms of contract on both sides. The term price may be defined as usage price or service price, etc., leading to inconsistencies during the negotiation process. Another example of not matching templates are SLA templates which slightly differ in their structure. In this paper we approach the gap between existing QoS methods and novel computing paradigms like Cloud Computing by proposing VieSLAF, a framework for the management of SLA mappings. Thereby, mappings are defined by XSLT1 documents where inconsistent parts of one document are mapped to another document e.g, from the consumer’s template to the provider’s template. Moreover, based on SLA mappings and deployed taxonomies we eliminate semantic inconsistencies between consumer’s and provider’s templates. The purpose of the submitted SLA mappings is twofold: (1) using VieSLAF users may discover services on demand, define mappings to available templates, if necessary and finally start the negotiation with selected services. Therefore, the negotiation is not only limited to services belonging to a special portal or where a relationship is already established off-line; (2) based on VieSLAF ’s predefined learning functions and accumulated SLA mappings we facilitate user driven definition of public SLA templates. Based on a case study the presented SLA mapping architecture has been successfully used to manage SLA mappings in context of a Grid workflow management tool [4] and adaptable Cloud services [6]. Additionally to [4,6] where we presented the general approach for SLA mappings, in this paper we present (1) the VieSLAF architecture in detail with modules for the measurement of SLA parameters, (2) implementation details of the VieSLAF framework; and (3) first experimental results. The main contributions of this paper are: (1) description of the scenarios for the definition of SLA mapping documents; (2) definition of the VieSLAF architecture used for the semi-automatic management of SLA mappings (3) demonstration of learning functions, which can be used to obtain realistic public templates and (4) evaluation of the VieSLAF architecture using an experimental testbed. The rest of this paper is organized as follows: Section 2 presents the related work. In Section 3 we present our SLA mapping approach. In particular we discuss the management of SLA mappings, SLA transformations, and an example 1

XSL Transformations (XSLT) Version 1.0, http://www.w3.org/TR/xslt.html

62

I. Brandic et al.

SLA mapping document. Section 4 presents the VieSLAF architecture including the used semantic model, methods for SLA mappings and transformations, used registries and features for the SLA monitoring and adaptation of SLA templates. In Section 5 we discuss our first experimental results. Section 6 concludes this paper and describes the future work.

2

Related Work

Currently, a large body of work exists in the area of Grid service negotiation and SLA-based QoS. Most of the related work can be classified into the following three categories: (1) adaptive SLA mechanisms based on OWL, DAML-S and other semantic technologies [17,10,23]; (2) SLA based QoS systems, which consider varying service requirements but do not consider non matching SLA templates [1,20]; and (3) systems relying on the principles of autonomic computing [3,14,15]. Work presented in [18] discusses the incorporation of SLA-based resource brokering into existing Grid systems. Oldham et al. describe a framework for semantic matching of SLAs based on WSDL-S and OWL [17]. Dobson at al. present a unified quality of service (QoS) ontology applicable to the main scenarios identified such as QoS-based Web services selection, QoS monitoring and QoS adaptation [10]. Zhou et al. survey the current research on QoS and service discovery, including ontologies such as OWL-S and DAML-S. Thereafter, an ontology is proposed, DAML-QoS, which provides detailed QoS information in a DAML format [23]. Hung et al. propose an independent declarative XML language called WS-Negotiation for Web services providers and requestors. WSNegotiation contains three parts: negotiation message, which describes the format for messages exchanged among negotiation parties, negotiation protocol, which describes the mechanism and rules that negotiation parties should follow, and negotiation decision making, which is an internal and private decision process based on a cost-benefit model or other strategies [13]. Work presented in [1] extends the service abstraction in the Open Grid Services Architecture (OGSA) for QoS properties focusing on the application layer. Thereby, a given service may indicate the QoS properties it can offer or it may search for other services based on specified QoS properties. Quan et al. discuss the process of mapping a light communication workflow within an SLA context with different kinds of sub-jobs and resources [16]. Dan et al. present a framework for providing customers of Web services differentiated levels of service through the use of automated management and SLAs [9]. Ardagana et al. present an autonomic grid architectures with mechanisms to dynamically re-configure service center infrastructures, which is basically exploited to fulfill varying QoS requirements [3]. Koller et al. discuss autonomous QoS management using a proxy-like approach. The implementation is based on WS-Agreement [21]. Thereby, SLAs can be exploited to define certain QoS parameters that a service has to maintain during its interaction with a specific customer [14]. K¨onig at al. investigate the trust issue in electronic negotiations,

VieSLAF Framework: Enabling Adaptive and Versatile SLA-Management

63

dealing with how to trust a potential transaction partner and how to choose such partners based on their past behavior [15]. However, to the best of our knowledge none of the discussed approaches deals with user-driven and semi-automatic definition of SLA mappings enabling negotiations between inconsistent SLA templates. Moreover, none of the presented approaches facilitates user driven definition of publicly available SLA templates.

3

The SLA Mapping Approach

In the presented approach each SLA template has to be published into a registry where negotiation partners i.e., provider and consumer, can find each other. The management of SLA mappings and published services is presented in Section 3.1. The transformations between remote and local SLA templates are discussed in Section 3.2. Finally, an example SLA mapping document is presented in Section 3.3. 3.1

Management of SLA Mappings

Figure 1(a) depicts the architecture for the management of SLA mappings and participating parties. The registry comprises different SLA templates whereby each of them represents a specific application domain e.g., SLA templates for the medical, telco or life science domain. Thus, each service provider may assign his/her service to a particular template (see step 1 in Figure 1(a)) and afterwards assign SLA mappings, if necessary (see step 2). Each template a may have n services assigned. Available templates can be browsed using an appropriate GUI. Service consumers may search for the services using meta-data and search terms (step 3). After finding appropriate services each service consumer may define mappings to the associated template (step 4). Thereafter, the negotiation between service consumer and service provider may start as described in the next section. SLA mappings should be defined in a dynamic way. Thus, SLA templates can be updated frequently to reflect the actual SLAs used by service provides and consumers based on predefined adaptation rules (step 5). The adaptability functionality facilitates the generation of user driven public SLA templates.

5. <