This content was uploaded by our users and we assume good faith they have the permission to share this book. If you own the copyright to this book and it is wrongfully on our website, we offer a simple DMCA procedure to remove your content from our site. Start by pressing the button below!
Takahiro Hara, Vladimir I. Zadorozhny, and Erik Buchmann (Eds.) Wireless Sensor Network Technologies for the Information Explosion Era
Studies in Computational Intelligence, Volume 278 Editor-in-Chief Prof. Janusz Kacprzyk Systems Research Institute Polish Academy of Sciences ul. Newelska 6 01-447 Warsaw Poland E-mail: [email protected] Further volumes of this series can be found on our homepage: springer.com Vol. 257. Oscar Castillo, Witold Pedrycz, and Janusz Kacprzyk (Eds.) Evolutionary Design of Intelligent Systems in Modeling, Simulation and Control, 2009 ISBN 978-3-642-04513-4 Vol. 258. Leonardo Franco, David A. Elizondo, and Jos´e M. Jerez (Eds.) Constructive Neural Networks, 2009 ISBN 978-3-642-04511-0 Vol. 259. Kasthurirangan Gopalakrishnan, Halil Ceylan, and Nii O. Attoh-Okine (Eds.) Intelligent and Soft Computing in Infrastructure Systems Engineering, 2009 ISBN 978-3-642-04585-1 Vol. 260. Edward Szczerbicki and Ngoc Thanh Nguyen (Eds.) Smart Information and Knowledge Management, 2009 ISBN 978-3-642-04583-7 Vol. 261. Nadia Nedjah, Leandro dos Santos Coelho, and Luiza de Macedo de Mourelle (Eds.) Multi-Objective Swarm Intelligent Systems, 2009 ISBN 978-3-642-05164-7 Vol. 262. Jacek Koronacki, Zbigniew W. Ras, Slawomir T. Wierzchon, and Janusz Kacprzyk (Eds.) Advances in Machine Learning I, 2009 ISBN 978-3-642-05176-0 Vol. 263. Jacek Koronacki, Zbigniew W. Ras, Slawomir T. Wierzchon, and Janusz Kacprzyk (Eds.) Advances in Machine Learning II, 2009 ISBN 978-3-642-05178-4 Vol. 264. Olivier Sigaud and Jan Peters (Eds.) From Motor Learning to Interaction Learning in Robots, 2009 ISBN 978-3-642-05180-7 Vol. 265. Zbigniew W. Ras and Li-Shiang Tsay (Eds.) Advances in Intelligent Information Systems, 2009 ISBN 978-3-642-05182-1 Vol. 266. Akitoshi Hanazawa, Tsutom Miki, and Keiichi Horio (Eds.) Brain-Inspired Information Technology, 2009 ISBN 978-3-642-04024-5 Vol. 267. Ivan Zelinka, Sergej Celikovsk´y, Hendrik Richter, and Guanrong Chen (Eds.) Evolutionary Algorithms and Chaotic Systems, 2009 ISBN 978-3-642-10706-1
Vol. 268. Johann M.Ph. Schumann and Yan Liu (Eds.) Applications of Neural Networks in High Assurance Systems, 2009 ISBN 978-3-642-10689-7 Vol. 269. Francisco Fern´andez de de Vega and Erick Cant´u-Paz (Eds.) Parallel and Distributed Computational Intelligence, 2009 ISBN 978-3-642-10674-3 Vol. 270. Zong Woo Geem Recent Advances In Harmony Search Algorithm, 2009 ISBN 978-3-642-04316-1 Vol. 271. Janusz Kacprzyk, Frederick E. Petry, and Adnan Yazici (Eds.) Uncertainty Approaches for Spatial Data Modeling and Processing, 2009 ISBN 978-3-642-10662-0 Vol. 272. Carlos A. Coello Coello, Clarisse Dhaenens, and Laetitia Jourdan (Eds.) Advances in Multi-Objective Nature Inspired Computing, 2009 ISBN 978-3-642-11217-1 Vol. 273. Fatos Xhafa, Santi Caballé, Ajith Abraham, Thanasis Daradoumis, and Angel Alejandro Juan Perez (Eds.) Computational Intelligence for Technology Enhanced Learning, 2010 ISBN 978-3-642-11223-2 Vol. 274. Zbigniew W. Ra´s and Alicja Wieczorkowska (Eds.) Advances in Music Information Retrieval, 2010 ISBN 978-3-642-11673-5 Vol. 275. Dilip Kumar Pratihar and Lakhmi C. Jain (Eds.) Intelligent Autonomous Systems, 2010 ISBN 978-3-642-11675-9 Vol. 276. Jacek Ma´ndziuk Knowledge-Free and Learning-Based Methods in Intelligent Game Playing, 2010 ISBN 978-3-642-11677-3 Vol. 277. Filippo Spagnolo and Benedetto Di Paola (Eds.) European and Chinese Cognitive Styles and their Impact on Teaching Mathematics, 2010 ISBN 978-3-642-11679-7 Vol. 278. Takahiro Hara, Vladimir I. Zadorozhny, and Erik Buchmann (Eds.) Wireless Sensor Network Technologies for the Information Explosion Era, 2010 ISBN 978-3-642-13964-2
Takahiro Hara, Vladimir I. Zadorozhny, and Erik Buchmann (Eds.)
Wireless Sensor Network Technologies for the Information Explosion Era
123
Prof. Takahiro Hara
Dr. Erik Buchmann
Department of Multimedia Engineering Graduate School of Information Science and Technology Osaka University 1-5 Yamadaoka Suita, Osaka 565-0871 Japan
Institute for Program Structures und Data Organization Karlsruhe Institute of Technology Geb. 50.34 Am Fasanengarten 5 D-76131 Karlsruhe Germany
Prof. Vladimir I. Zadorozhny Department of Information Science and Telecommunications School of Information Sciences University of Pittsburgh 135 North Bellefield Ave. Pittsburgh, PA 15260 USA E-mail: [email protected]
ISBN 978-3-642-13964-2
e-ISBN 978-3-642-13965-9
DOI 10.1007/978-3-642-13965-9 Studies in Computational Intelligence
ISSN 1860-949X
Library of Congress Control Number: 2010929472 c 2010 Springer-Verlag Berlin Heidelberg This work is subject to copyright. All rights are reserved, whether the whole or part of the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations, recitation, broadcasting, reproduction on microfilm or in any other way, and storage in data banks. Duplication of this publication or parts thereof is permitted only under the provisions of the German Copyright Law of September 9, 1965, in its current version, and permission for use must always be obtained from Springer. Violations are liable to prosecution under the German Copyright Law. The use of general descriptive names, registered names, trademarks, etc. in this publication does not imply, even in the absence of a specific statement, that such names are exempt from the relevant protective laws and regulations and therefore free for general use. Typeset & Cover Design: Scientific Publishing Services Pvt. Ltd., Chennai, India. Printed on acid-free paper 987654321 springer.com
About This Book
The amount and value of information available due to rapid spread of information technology is exploding. Typical large enterprises have approximately a petabyte of operational data stored in hundreds of data repositories supporting thousands of applications. Data storage volumes grow in excess of 50% annually. This growth is expected to continue due to vast proliferation of existing information systems and the introduction of new sources of data. Wireless Sensor Networks (WSNs) represent one of the most notable examples of such new data sources. In recent few years, various types of WSNs have been deployed, and the amount of information generated by wireless sensors increases rapidly. The information explosion requires establishing novel data processing and communication techniques for WSNs. This volume aims to cover both theoretical and practical aspects related to this challenge, and it explores directions for future research to enable efficient utilization of WSNs in the information-explosion era. Written for: Researchers, practitioners, and graduate students in computer science Keywords: • Computational Intelligence • Wireless Sensor Network • Networked Sensing System • Information Explosion Era
Preface
Wireless Sensor Network Technologies for Information Explosion Era The amount and value of information available due to rapid spread of information technology is exploding. Typically, large enterprises have approximately a petabyte of operational data stored in hundreds of data repositories supporting thousands of applications. Data storage volumes grow in excess of 50% annually. This growth is expected to continue due to vast proliferation of existing information systems and the introduction of new data sources. Wireless Sensor Networks (WSNs) represent one of the most notable examples of such new data sources. In recent few years, various types of WSNs have been deployed and the amount of information generated by wireless sensors increases rapidly. The information explosion requires establishing novel data processing and communication techniques for WSNs. This volume aims to cover both theoretical and practical aspects related to this challenge, and it explores directions for future research to enable efficient utilization of WSNs in the information-explosion era. The book is organized in three main parts that consider (1) technical issues of WSNs, (2) the integration of multiple WSNs, and (3) the development of WSNs systems and testbeds for conducting practical experiments. Each part consists of three chapters. The first part is entitled as “Scheduling and Data Management”. In the first chapter, the authors compare goal and data oriented routing mechanisms for WSNs that have been developed in recent years. The chapter is intended to provide a better understanding of current data routing protocols for WSNs, and it points out open issues that should be subject to further research. The second chapter discusses two significant issues for energy-efficiency in WSNs: task scheduling and data aggregation. The authors propose scheduling and aggregation techniques, and discuss their future plan to interoperate these techniques. In the third chapter, the authors motivate, review and discuss research topics for secure data aggregation in WSNs. In particular, the authors introduce a general secure data aggregation framework and survey existing secure data aggregation schemes. The second part of the book is named “Networked Sensing Systems” (NSSs). Its first chapter provides an overview of NSSs in terms of scalability, heterogeneity and complex query processing. The authors survey existing approaches for three classes of research topics: (i) data management in WSNs, (ii) centralized
VIII
Preface
data stream processing in NSSs, and (iii) hybrid and distributed approaches to integrate approaches from these two classes. The second chapter reviews existing solutions for Stream Processing System (SPS) middleware. In particular, the authors present their Data Stream Application Manager approach. The approach supports the mapping of global queries to distributed sensor nodes and the integration of SPSs. In the third chapter, the authors propose a framework to support efficient query processing and advanced functions in SPSs. Such functions include complex event processing, probabilistic reasoning or continuous media integration. The framework adopts query optimization and data archiving techniques for multiple queries over data streams. The third part of the book has the title “Implementation and Development Support”. In the first chapter of this part the authors introduce CLAD, a wearable sensor management device. CLAD supports power- and data management approaches that are tailored for context-aware sensors. For example, CLAD allows to reduce the energy consumption by determining the optimal combination of accelerometers that must be active at the same time to provide sensor data with a given accuracy. The second chapter discusses implementation issues and distributed approaches for managing ubiquitous sensors in NSSs. The chapter surveys existing techniques and presents Mill, a geographical location-based overlay network. Mill supports multi-scale region search and multi-attribute search. The last chapter describes a wireless sensor network testbed, called X-Sensor. X-Sensor manages multiple wireless sensor networks that might have been deployed at different sites, but can be seamlessly accessed by remote users. Thus, X-Sensor allows their users to conduct extensive experiments with real sensor data acquired by multiple sensor networks deployed in different environments. We highly appreciate the effort of all authors in preparing chapters for this book volume. We also thank Professor Janusz Kacprzyk, Editor-in-Chief of Studies in Computational Intelligence, for his help with this book volume.
February 2010
Takahiro Hara Osaka University, Japan Vladimir I. Zadorozhny University of Pittsburgh, USA Erik Buchmann Karlsruhe Institute of Technology, Germany
Contents
Part I: Scheduling and Data Management Survey on Data Routing in Wireless Sensor Networks . . . . . . . Jos´e Cec´ılio, Jo˜ ao Costa, Pedro Furtado Energy-Efficient Task Scheduling and Data Aggregation Techniques in Wireless Sensor Networks for Information Explosion Era . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Akimitsu Kanzaki, Naoki Wakamiya, Takahiro Hara Secure Data Aggregation in Wireless Sensor Networks . . . . . . . Vimal Kumar, Sanjay Madria
Survey on Data Routing in Wireless Sensor Networks José Cecílio, João Costa, and Pedro Furtado
*
Abstract. Routing in sensor networks is very challenging, due to several characteristics that distinguish them from contemporary communication and wireless ad-hoc networks. Many new goal and data-oriented algorithms have been proposed for the problem of routing data in sensor networks. Most routing protocols can be classified as data-centric, hierarchical, location-based or QoSaware. Data-centric protocols are query-based and depend on the naming of desired data. Hierarchical protocols aim at clustering the nodes so that cluster heads can do some aggregation and reduction of data in order to save energy. Location-based protocols utilize the position information to relay the data to the desired regions. The QoS-aware are based on general network-flow modeling for meeting some QoS requirements. In this chapter, we will explore goal and data-oriented routing mechanisms for sensor networks developed in recent years. Our aim is to help better understanding current routing protocols for wireless sensor networks and point out open issues that should be subject to further research.
1 Introduction Routing in sensor networks is very challenging due to several characteristics that distinguish them from contemporary communication and wireless ad-hoc networks. First of all, it is not possible to build global addressing and routing algorithms exactly as for classical IP-based protocols for the deployment of sheer numbers of energy and processing capacity constrained sensor nodes. Second, in contrary to typical communication networks, almost all applications of sensor networks require the flow of sensed data from multiple regions (sources) to a particular sink. Third, generated data traffic has significant redundancy in it, since José Cecílio · João Costa · Pedro Furtado University of Coimbra, DEI/CISUC e-mail: {jcecilio,jpcosta,pnf}@dei.uc.pt
multiple sensors may generate the same data within the vicinity of a phenomenon. Such redundancy needs to be exploited by the data and goal-oriented routing protocols to improve energy and bandwidth utilization. Fourth, sensor nodes are tightly constrained in terms of transmission power, on-board energy, processing capacity and storage and thus require careful resource management. Due to such differences, many new algorithms have been proposed for the problem of routing data in sensor networks. These routing mechanisms have considered the characteristics of sensor nodes along with the application and architecture requirements. Almost all of these routing protocols can be classified as data-centric, hierarchical or location-based, there being also some distinct ones based on network flow or QoS awareness. Data-centric protocols are query-based and depend on the naming of desired data, which helps in eliminating many redundant transmissions. Hierarchical protocols aim at clustering the nodes so that cluster heads can do some aggregation and reduction of data in order to save energy. Location-based protocols utilize the position information to relay the data to the desired regions rather than the whole network. The last category includes routing approaches that are based on general network-flow modeling and protocols that strive for meeting some QoS requirements along with the routing function. In this chapter, we will explore the routing mechanisms for sensor networks developed in recent years. Each routing protocol is discussed under the proper category. Our aim is to help better understanding of the current data and goaloriented routing protocols for wireless sensor networks and point out open issues that can be subject to further research. There are some previous works surveying the characteristics, applications, and communication protocols in WSNs [25][32][38][52]. While most existing surveys address several design issues and techniques for WSNs, describing the physical constraints on sensor nodes, applications and architectural attributes, this chapter is devoted to the study of data and goal-oriented routing in sensor networks, describing and categorizing the different approaches for data delivery.
2 Classes of Algorithms The review of current approaches is based on an extensive survey of the state-ofthe-art in goal and data-oriented routing approaches. Different protocols and algorithms proposed in recent years are reviewed. We begin with Data Centric Protocols, which are centered on the data itself. Secondly, we analyse Hierarchical Routing, which consists on establishing a hierarchical route towards the data collection points. Thirdly, we survey protocols that use position information to relay the data to the desired regions (Location-based protocols), and lastly we consider QoS-Aware protocols, which take into account energy consumption and data quality. The network may have to satisfy certain QoS metrics (delay, energy, bandwidth) when delivering data to the base station, metrics which are used in the QoS-Aware protocols.
Survey on Data Routing in Wireless Sensor Networks
5
Table 1 lists the routing algorithms and classes that we review in this paper.
Table 1 Routing Algorithms and Classes Routing
Data-centric
SPIN
SPIN-PP SPIN-EC SPIN-BC SPIN-RL
Direct Diffusion Energy-aware Reliable Energy Aware Routing (REAR) Rumor MCFA Link Quality Estimatin Based Gradient Based Information-driven Acquire Hierarchical
LEACH EWC PEGASIS TEEN/APTEEN Energy-aware cluster-based Self-organized Minimum energy communication network Small minimum energy communication network
Location-based
Geographic Adaptive Fidelity Energy Aware Greedy Routing (EAGR) Geographic and Energy Aware
QoS Aware
SPEED MMSPEED Sequential Assignment Real-Time Power-Aware DCEERP Energy Efficient with Delay Guaranties (EEDG)
6
J. Cecílio, J. Costa, and P. Furtado
In order to select the most suitable routing mechanism for a sensor application, we have to classify the routing protocols according to the network and operational characteristics, and an objective model that describes the routing goal. Table 2 categorizes the routing protocols from Table 1 taking into consideration, besides the main classification used in that table, also the number of base stations, mobility characteristics, whether transmission power is considered fixed or variable, whether the approach uses data aggregation or not and whether it is query-based or not. Finally, we list the main goal for each alternative algorithm. In most applications, the number of base stations can be one or more than one. The increased number of base stations provides more robust data gathering, and may also decrease the network delay. If only one base station is present, the destination node for all messages is the same, while in the case of multiple base stations; the routing algorithms can take this into consideration to achieve desired goals. However, many techniques do not try to optimize the system when more than one base station is available. Another important factor is the transmission power, which can be either dynamically adjustable or fixed. When the transmission power is fixed, each sensor node transmits each message using the same energy level. In the other case, every node can calculate what energy level should be used to transmit a message to a neighboring node. This energy level may be inversely proportional to the cost assigned to the neighboring node. The task of routing in many protocols is to deliver the queries coming from the base station to the sensor which have the requested data, and return the requested data to the base station. Some protocols are based on hierarchy. A hierarchy level is assigned to each node, and a node only forwards those messages that are originated from a lowerlevel node. Optionally, a node aggregates incoming data and forward this aggregated data to upper-layer nodes. The base station can be found on the top of the hierarchy. The hierarchy construction can be dynamic or static. In the dynamic case, the role of the aggregator is rotated, and all nodes that have selected an aggregator will forward all data to their aggregator. The aim of forming hierarchies is to prolong the network lifetime. Some sensor applications only require the successful delivery of messages between a source and a destination. However, there are applications that need more guaranties. These are the real-time requirements of the message delivery, maximization of network lifetime and fault tolerance. The main objective of the real-time protocols is to completely control the network delay. The average-case performance of these protocols can be evaluated by measuring the message delivery ratio with time constraints. The lifetime is another important goal. Protocols try to balance energy consumption equally among nodes, considering their residual energy levels. However, the metric used to determine network lifetime is also application dependent. Most protocols assume that all nodes are equally important and they use the time until the first node dies as a metric, but the average energy consumption of the nodes may also be used as a metric.
Survey on Data Routing in Wireless Sensor Networks
7
Table 2 Properties of Routing Algorithms Number of base Transmission Data Mobility stations Power Aggregation
Query based
Goal
Yes
Yes
Lifetime, exchange metadata to reduce number of messages
Fixed
Yes
Yes
Establish efficient n-way communication paths for fault tolerance
Limited
Adjustable
Yes
Yes
Lifetime
1 or more
Limited
Adjustable
Yes
Yes
Lifetime and data delivery
Fixed
Yes
Yes
Reduce queries in network
Fixed
No
No
Data delivery
Protocols
Classification
SPIN
Data-centric
1
Possible
Fixed
Direct Diffusion
Data-centric
1 or more
Limited
Energy-aware
Data-centric
1 or more
REAR
Data-centric
Rumor
Data-centric
1
Very limited
MCFA
Data-centric
1
No
LQEBR
Data-centric
1
No
Fixed
No
No
Data delivery with minimal retransmissions
Gradient
Data-centric
1
Limited
Fixed
Yes
Yes
Data delivery through minimal number of hops
Information-driven
Data-centric
1
Limited
Fixed
yes
Yes
Optimization of Direct Diffusion to save more energy
Acquire
Data-centric
1 or more
Limited
Fixed
Yes
Yes
Optimization of queries in network
LEACH
Hierarchical
1
Fixed BS
Fixed
Yes
No
Formation distributed cluster to extend Lifetime
EWC
Hierarchical
1
Fixed BS
Fixed
Yes
No
Forming distributed cluster to extend lifetime and data delivery guaranties
PEGASIS
Hierarchical
1
Fixed BS
Fixed
No
No
Lifetime and bandwidth optimization
TEEN/APTEEN
Hierarchical
1
Fixed BS
Fixed
Yes
No
Real-Time and Lifetime
Energy-aware cluster-based
Hierarchical
1
No
Adjustable
No
No
Real-Time and Lifetime
Self-organized
Hierarchical
1
Possible
Fixed
No
No
Fault tolerance
Minimum energy communication network
Hierarchical
1
No
Adjustable
No
No
Lifetime and selfreconfiguration
Geographic Adaptive Fidelity
Location
1 or more
Limited
Fixed
No
No
Increase the network lifetime with the number of node increase
EAGR
Location
1 or more
Limited
Fixed
No
No
Optimization of Geographic Adaptive Fidelity algorithm
Geographic and Energy Aware
Location
1
Limited
Fixed
No
No
Reduce interest msgs in the whole network
8
J. Cecílio, J. Costa, and P. Furtado Table 2 (continued)
SPEED
QoS
1 or more
No
Fixed
No
Yes
Real-Time and Lifetime
MMSPEED
QoS
1 or more
No
Fixed
No
Yes
Real-Time and Lifetime (improve SPEED)
Sequential Assignment
QoS
1
No
Fixed
Yes
Yes
Real-Time
Real-Time PowerAware
QoS
1 or more
No
Adjustable
Yes
No
Real-Time and Lifetime
DCEERP
QoS
1
No
Fixed
Yes
No
Real-Time and Lifetime
EEDG
QoS
1 or more
No
Fixed
Yes
No
Deadlines Guaranties and Lifetime
Generically, these goals or qualities are achieved by introducing a set of heuristics into the routing algorithms: typically, lifetime can be controlled in routing algorithms by taking into account the level of power still available in the batteries during routing decisions, routing through the least power-consuming routes, controlling the transmission ratio, or by managing the duty cycle carefully; Minimizing delays is controlled typically by delay-based scheduling of node transmission queues; Fault tolerance is achieved in the reviewed algorithms by keeping multiple alternative routing paths and using an available one on-demand. While a minimum number of hops may minimize qualities such as delays, when nodes fail the algorithms are also able to choose alternative routes; Optimum bandwidth and minimum number of messages are typically achieved by using an optimal transmission packet size. Table 3 categorizes the routing protocols from Table 1 taking into consideration the main goals that each algorithm goes after. We describe these goals after the table. Table 3 Routing Goals
Protocols SPIN
Lifetime, min energy
Min. delay or *deadlines
Fault tolerance, Reconfiguration
x
Direct Diffusion
Opt. Bandwidth, min. nr. of msgs x
x
Energy-aware
x
REAR
x
Rumor
x x
MCFA
x
LQEB
x
Gradient Information-driven
Min. nr. hops
x x
Survey on Data Routing in Wireless Sensor Networks
9
Table 3 (continued)
Acquire
x
LEACH
x
EWC
x
PEGASIS
x
TEEN/APTEEN
x
x
Energy-aware cluster-based
x
x
x
Self-organized
x
Minimum energy communication network
x
Geographic Adaptive Fidelity
x
EAGR
x
x
x
Geographic and Energy Aware
x
SPEED
x
x
MMSPEED
x
x
Sequential Assignment
x
x
Real-Time Power-Aware
x
x*
DCEERP
x
x
EEDG
x
x
Most of the algorithms in Table 3 take into consideration the battery lifetime as the main goal (e.g. SPIN, Energy-aware routing, LEACH, Minimum energy communication network, Geographic Adaptive Fidelity). A few of them consider delays (e.g. Sequential Assignment), fault tolerance (Direct Diffusion, Selforganized Protocol) and bandwidth optimization (Rumor, Acquire). Some protocols focus on more than one goal (e.g. TEEN/APTEEN, Energy-aware cluster-based, SPEED, Real-Time Power-Aware). The principal goal for these last protocols is to control the network delay, but simultaneously try to minimize the dissipated energy in transmission, in order to extend the network lifetime as much as possible under the delay constraints. Table 4 categorizes the routing protocols from Table 1 taking into consideration their application type. Some typical application areas, such as habitat monitoring, environment monitoring, health care and Industrial environment are shown and the routing protocols are categorized in the table by typical application type.
10
J. Cecílio, J. Costa, and P. Furtado Table 4 Routing typical Application
Protocols
Project
Application type
PODS Hawaii [46]
Environment Monitoring
Gradient
Vital Sign [22]
Health
Acquire
Flood Detection [45]
Environment Monitoring
LEACH
Artificial Retina [34]
Health
Aware Home [9]
Industrial
Self-organized
SOMoM [30]
Military
Geographic Adaptive Fidelity
Great Duck [3]
Military
Geographic and Energy Aware
Aware Home [9]
Habitat Monitoring
Vital Sign [22]
Health
Direct Diffusion
TEEN/APTEEN
Sequential Assignment
The content of each protocol is explained in the following sections under each category.
3 Data-Centric Protocols Since transmiting data from every sensor node within the deployment region might result in significant unnecessary redundancy in data and incur in unnecessary energy and traffic expenditure, routing protocols that are able to select a set of sensor nodes and utilize data aggregation during the relaying of data have been considered. In data-centric routing, the sink sends queries to certain regions and waits for data from the sensors located in the selected regions. Since data is being requested through queries, attribute-based naming is necessary to specify the properties of data. Sensor Protocols for Information via Negotiation (SPIN) [60] is the first data-centric protocol, which considers data negotiation between nodes in order to eliminate redundant data and save energy. Later, Directed Diffusion [13] has been developed and has become a breakthrough in data-centric routing. Many other protocols have also been proposed based on Directed Diffusion, such as Energyaware routing [15], Rumor routing [17], Minimum Cost Forwarding Algorithm [21] and Gradient-Based Routing [49].
3.1 SPIN: Sensor Protocol for Information via Negotiation The SPIN family of protocols rests upon two basic ideas. First, to operate efficiently and to conserve energy, sensor applications need to communicate with
Survey on Data Routing in Wireless Sensor Networks
11
each other about the data that they already have and the data they still need to obtain. Exchanging sensor data may be an expensive network operation, but exchanging data about sensor data need not be. Second, nodes in a network must monitor and adapt to changes in their own energy resources to extend the operating lifetime of the system. Sensors use meta-data to succinctly and completely describe the data that they collect. If x is the meta-data descriptor for sensor data X, then the size of x in bytes must be shorter than the size of X, for SPIN to be beneficial. If two pieces of actual data are distinguishable, then their corresponding meta-data should be distinguishable. Likewise, two pieces of indistinguishable data should share the same meta-data representation. SPIN does not specify a format for meta-data; this format is applicationspecific. Sensors that cover disjoint geographic regions may simply use their own unique IDs as meta-data. The meta-data x would then stand for “all the data gathered by sensor x”. There are three messages defined in SPIN to exchange data between nodes. These are: – – –
ADV message to allow a sensor to advertise a particular meta-data, REQ message to request the specific data, DATA message that carry the actual data.
Figure 1, summarizes the steps of the SPIN protocol.
Fig. 1 SPIN Protocol
Node A starts by advertising its data to node B (a). Node B responds by sending a request to node A (b). After receiving the requested data (c), node B then sends out advertisements to its neighbours (d), who in turn send requests back to B (e-f).
12
J. Cecílio, J. Costa, and P. Furtado
Though this protocol has been designed for lossless networks, it can easily be adapted to work in lossy or mobile networks. Here, nodes could compensate for lost ADV messages by re-advertising these messages periodically. Nodes can compensate for lost REQ and DATA messages by re-requesting data items that do not arrive within a fixed time period. For mobile networks, changes in the local topology can trigger updates to a node’s neighbour list. If a node notices that its neighbour list has changed, it can spontaneously re-advertise all of its data. This protocol’s strength is its simplicity [60]. Each node in the network performs little decision making when it receives new data, and therefore wastes little energy in computation. Furthermore, each node only needs to know about its single-hop network neighbours. The following protocols make up the SPIN family of protocols [60]: –
–
–
SPIN-PP: This protocol has been designed to perform optimally for pointto-point communication. In this sort of communication, two nodes can have exclusive communication with each other without any interference from the other nodes. In such a network, the cost of communication for one node to communicate with n nodes is n times more expensive than communicating with one node. This protocol is a simple 3-way handshake protocol in which energy is not considered to be a constraint. When a node has some new data, it advertises this new data using the ADV messages to its neighbours. When a neighbouring node receives this advertisement, it checks the meta-data to see whether it already has the data item or not. In case it does not, it sends an REQ message back requesting for the data item. Upon receiving the REQ message, the originating node sends DATA messages containing the missing data to the requesting node. One major advantage of using this protocol is its simplicity and that each node requires to know only about its single-hop neighbours and does not require any other topology information. SPIN-EC: In this protocol, the sensor nodes communicate using the same 3-way handshake protocol as in SPIN-PP but there is a energyconservation heuristic added to it. A node will participate actively in the protocol only if it is above a certain energy threshold and believes it can complete all the other stages of the protocol. If a node receives an advertisement, it will not send out an REQ message if it does not have enough energy to transmit an REQ message and receive the corresponding DATA message. SPIN-BC: This protocol was designed for broadcast networks in which the nodes use a single shared channel to communicate. When a node sends out a message, it is received by all the other nodes within a certain range of the sender. In this protocol, a node which has received an ADV message does not immediately respond with an REQ message. It has to wait for a certain time before sending out the REQ message. When a node other than the advertising node receives the REQ message, it cancels its own request so
Survey on Data Routing in Wireless Sensor Networks
–
13
that there are no redundant requests for the same message. When the advertising node receives an REQ message, it sends the data message only once because it is a broadcast network even though it might have got multiple requests for the same message. SPIN-RL: This protocol makes two changes to the above SPIN-BC protocol. Each node keeps track of all the advertisements it hears and the nodes it hears them from. If it does not receive any requested data within a certain period of time, it sends out the request again. Next, the nodes have a limit on the frequency with which they resend the data messages. After sending out a data message, a node will wait for a certain period of time before it responds to other requests for the same data message.
3.2 Directed Diffusion This is another data dissemination protocol in which the data generated by the nodes is diffusing through sensor nodes by using a naming scheme for the data. The main reason behind using such a scheme is to get rid of unnecessary operations of network layer routing in order to save energy. Direct Diffusion [7] suggests the use of attribute-value pairs for the data and queries the sensors in an on demand basis by using those pairs. In order to create a query, an interest is defined using a list of attribute-value pairs such as name of objects, interval, duration, geographical area, etc. The interest is broadcast by a sink through its neighbours. Each node receiving the interest can do caching for later use. The nodes also have the ability to do in-network data aggregation, which is modeled as a minimum Steiner tree problem. The interests in the caches are then used to compare the received data with the values in the interests. The interest entry also contains several gradient fields. A gradient is a reply link to a neighbour from which the interest was received. It is characterized by the data rate, duration and expiration time derived from the received interest’s fields. Hence, by utilizing interest and gradients, paths are established between sink and sources. Several paths can be established so that one of them is selected by reinforcement. The sink resends the original interest message through the selected path with a smaller interval hence reinforces the source node on that path to send data more frequently. Figure 2, summarizes the Directed Diffusion protocol.
Fig. 2 Directed diffusion protocol phases
14
J. Cecílio, J. Costa, and P. Furtado
An entry has several fields - timestamp field which contains the last received matching interest, the gradient fields contain the data rate specified by each neighbour, the duration field which contains the lifetime of the interest. When a node receives an interest, it checks its interest cache to check if it has entry. It creates one if there is no matching interest and a single gradient field is created towards the neighbour from which the interest is received. If the interest exists, the timestamp and the duration fields are updated in the entry. A gradient is removed from its interest entry when it expires. A gradient specifies both the data rate as well as the direction in which the events are to be sent. A node may send an interest it receives to some of its neighbours to whom it will appear as if this node itself is the originating node. Therefore, there is diffusion of interests throughout the network. A sensor node which detects an event searches its interest cache for a matching interest entry. If it finds one, it generates even samples at the highest data rate which it computes from the requested event rates of all its outgoing gradients. The event description is then sent to all its neighbouring nodes for which it has gradients. Therefore the sink starts receiving low data rate events, when an event is observed, possibly along multiple paths. The sink then reinforces one particular neighbour to get the better quality events. Path repairs are also possible in Directed Diffusion. When a path between a source and the sink fails, a new or alternative path should be identified. For this, Directed Diffusion basically reinitiates reinforcement by searching among other paths, which are sending data in lower rates. In [18] the author suggest that employing multiple paths in advance so that in case of a failure of a path, one of the alternative paths is chosen without any cost for searching for another one. There is of course extra overhead of keeping these alternative paths alive by using low data rate, which will definitely use extra energy but more energy can be saved when a path fails and a new path should be chosen.
3.3 Energy-Aware Routing The basic idea of energy-aware routing is that to increase the survivability of networks, it may be necessary to use sub-optimal paths occasionally [14]. This ensures that the optimal path does not get depleted and the network degrades gracefully as a whole rather than getting partitioned. To achieve this, multiple paths are found between source and destinations, and each path is assigned a probability of being chosen, depending on the energy metric. [49]. The approach argues that using the minimum energy path all the time will deplete the energy of nodes on that path. Instead, one of the multiple paths is used with a certain probability so that the whole network lifetime increases. The protocol assumes that each node is addressable through a class-based addressing which includes the location and types of the nodes. There are 3 phases in the protocol: –
Setup phase or interest propagation: Localized flooding occurs to find all the routes from source to destination and their energy costs. This is when routing (interest) tables are built up.
Survey on Data Routing in Wireless Sensor Networks
15
The destination node initiates the connection by flooding the network in the direction of the source node. It also sets the “Cost” field to zero before sending the request.
Cost (N D ) = 0
(1)
Every intermediate node forwards the request only to the neighbours that are closer to the source node than oneself and farther away from the destination node. Thus at a node Ni, the request is sent only to a neighbour Nj which satisfies:
( ) d (N i , N D ) ≤ d (N j , N D ) d (N i , N S ) ≥ d N j , N S
(
(2)
)
where d N i , N j is the distance between Ni and Nj. On receiving the request, the energy metric for the neighbour that sent the request is computed and is added to the total cost of the path. Thus, if the request is sent from node Ni to node Nj, Nj calculates the cost of the path as:
(
)
(
C N j , N i = Cost (N i ) + Metric N j , N i
)
(3)
Paths that have a very high cost are discarded and not added to the forwarding table. Only the neighbours Ni with paths of low cost are added to the forwarding table FTj of Nj. ⎧⎪ ⎛ min FT j = ⎨i C N j , N i ≤ α ⋅ ⎜⎜ C N j , N i ⎪⎩ ⎝ k
(
)
(
⎫
)⎞⎟⎟⎪⎬ ⎠⎪⎭
(4)
Node Nj assigns a probability to each of the neighbours Ni in the forwarding table FTj, with the probability inversely proportional to the cost. 1 C N j , Ni P N j , Ni = 1 C N j , Nk k∈FT
(
)
(
∑ ( j
)
)
(5)
Thus, each node Nj has a number of neighbours through which it can route packets to the destination. Nj then calculates the average cost of reaching the destination using the neighbours in the forwarding table.
( ) ∑ P(N j , N i )⋅ C (N j , N i )
Cost N j =
i∈FT j
(6)
This average cost for Nj is set in the cost field of the request and forwarded.
16
J. Cecílio, J. Costa, and P. Furtado
–
–
Data Communication phase or data propagation – Data is sent from source to destination, using the information from the earlier phase. This is when paths are chosen probabilistically according to the energy costs that were calculated earlier. The source node sends the data packet to any of the neighbours in the forwarding table, with the probability of the neighbour being chosen equal to the probability in the forwarding table. Each of the intermediate nodes forwards the data packet to a randomly chosen neighbour in its forwarding table, with the probability of the neighbour being chosen equal to the probability in the forwarding table. This is continued till the data packet reaches the destination node. Route maintenance – Route maintenance is minimal. Localized flooding is performed infrequently from destination to source to keep all the paths alive.
3.4 Reliable Energy Aware Routing (REAR) Hassanein et al. [23] have proposed a routing algorithm in which reliability of packet delivery is high and which is also energy aware. REAR uses three types of nodes: a network Sink, Intermediate Nodes (IN) and Target Source (TS). This algorithm works on networks structured as two layers: one layer over which it provides the energy aware path using energy-reservation mechanism; and one transport layer, which provides reliability. Parts of REAR are as defined next: –
–
–
–
Service Path Discovery (SPD): In SPD, the node known as sink sends the request for path discovery over the network. It uses flooding for this path request. On the way, the broadcasting speed is combined with available energy to select the energy efficient path. Once the node is selected, it contains two logical energy levels (one is available energy and the other is reserved energy occupied by the path). When this candidate path reaches the source, it generates a path reservation request in which it reserves the required energy of nodes for this path. Any other path cannot use this energy until it is released. Backup Path Discovery (BPD): It is also initiated by sink and has the same procedure as SPD. The only difference is that it will not contain the nodes already selected for service path. The backup path is used in case of failure of the service path. Reliable Transmission: For reliable transmission, each sending node stores the data until it gets acknowledgment from the receiver. Due to low memory, packets are not stored in case of SP link failure. The source node will need to transmit all the packets again. Reserved Energy Release: When the link fails, an error message is transmitted to all the intermediate nodes, which release the reserved energy for that path.
Survey on Data Routing in Wireless Sensor Networks
17
3.5 Rumor Routing The Rumor Routing protocol [17] looks at routing queries to the nodes which have observed a particular event. It looks at creating paths leading to each event so that a query which is generated can be routed randomly till it finds the event path instead of flooding it across the network. The rumor routing algorithm uses a set of long-lived agents which create paths that are directed towards the events they encounter. Whenever an agent crosses path with a path leading to an event that it has not encountered, it adapts its behavior thus creating a path state which leads to both the events. When the agents come across shorter paths, they optimize the paths in the network by updating the routing tables to reflect the more efficient path. Each node maintains a list of its neighbours and an events table. When it encounters an event it adds it to its events table and might generate an agent in a probabilistic fashion. The agent also contains an events table like that of the nodes which it synchronizes with every node that it encounters. The agent has a lifetime of a certain number of hops after which it dies. Any node generating a query will transmit the query if it has a route to the event else it will transmit it in a random direction. If the node gets to know that the query did not reach the destination then it will flood the network. The lesser the number of queries which flood, the lesser the energy consumed.
3.6 Minimum Cost Forwarding Algorithm The Minimum Cost Forwarding Algorithm (MCFA) [21] exploits the fact that the direction of routing is always known. Hence, a sensor node need not have a and Gradient-Based Routing unique ID nor maintain a routing table. Instead, each node maintains the least cost estimate from itself to the sink. Each message to be forwarded by the sensor node is broadcast to its neighbours. When a node receives the message, it checks if it is on the least cost path between the source sensor node and the sink. If this is the case, it rebroadcasts the message to its neighbours. This process repeats until the sink is reached. In MCFA, each node should know the least cost path estimate from itself to the sink. This is obtained as follows. The sink broadcasts a message with the cost set to zero, while every node initially sets its least cost to the sink to infinity (∞). Each node, upon receiving the broadcast message originated at the sink, checks to see if the estimate in the message plus the link on which it is received is less than the current estimate. If yes, the current estimate and the estimate in the broadcast message are updated. If the received broadcast message is updated, it is resent; otherwise, it is purged and nothing further is done. However, the previous procedure may result in some nodes having multiple updates, and those nodes far away from the sink will get more updates from those closer to the sink. To avoid this, MCFA was modified to run a backoff algorithm at the setup phase. The backoff algorithm dictates that a node will not send the updated message until a*lc time units have elapsed from the time at which the message is updated, where a is a constant and lc is the link cost at which the message was received.
18
J. Cecílio, J. Costa, and P. Furtado
3.7 Link Quality Estimation Based Routing The Link Quality Estimation Based Routing (LQER) is proposed by Chen et al.[27]. LQER takes decision about data forwarding on the basis of a dynamic window (m,k) that stores the history of transmission success over the link. This dynamic window is represented as (m,k) where m represents the total successful transmission bits and k is the length of window. Minimum hop-count is also considered to make this protocol reliable as well as energy efficient. In the bit sequence of the window, the left-most bit represents the oldest and right-most bit represents the newest data. When the transmission is unsuccessful, 0 is inserted while 1 represents successful transmission. LQER estimates the link quality by m/k. The largest value of m/k is considered the best one. This protocol also considers the minimum hop-count value for selection on next hop. Using flooding mechanisms in which the sink starts advertising the hop-count sets minimum hop count field. Initially, all the nodes have maximum hop-count set. When a node receive any hop-count lesser than it has already recorded, it replaces the stored hop count with the new one. By doing this, it gets the shortest path. LQER first selects the neighbors having minimum hop-count and from that set, it chooses the node having largest m/k value to forward the data. This way it selects the best path for data transmission. Authors have compared the results of LQER with MCFA (Minimize Cost Forwarding Algorithm) using WSNsim simulator (developed by the authors). Results indicate that LQER is more energy efficient than MCFA.
3.8 Gradient-Based Routing Schurgers [15] have to proposed a changed version of Directed Diffusion, called Gradient-based routing (GBR). The idea is to keep the number of hops when the interest is diffused through the network. Hence, each node can discover the minimum number of hops to the sink, which is called height of the node. The difference between a node’s height and that of its neighbour is considered the gradient on that link. A packet is forwarded on a link with the largest gradient. The authors aim at using some auxiliary techniques such as data aggregation and traffic spreading along with GBR in order to balance the traffic uniformly over the network. Nodes acting as a relay for multiple paths can create a data combining entity in order to aggregate data. On the other hand, three different data spreading techniques have been presented: – –
–
Stochastic Scheme: When there are two or more next hops with the same gradient, the node chooses one of them at random. Energy-based scheme: When a node’s energy drops below a certain threshold, it increases its height so that other sensors are discouraged from sending data to that node. Stream-based scheme: The idea is to divert new streams away from nodes that are currently part of the path of other streams.
Survey on Data Routing in Wireless Sensor Networks
19
The data spreading schemes strives to achieve an even distribution of the traffic throughout the whole network, which helps in balancing the load on sensor nodes and increases the network lifetime. The employed techniques for traffic load balancing and data fusion are also applicable to other routing protocols for enhanced performance. Through simulation GBR has been shown to outperform Directed Diffusion in terms of total communication energy.
3.9 Information-Driven Sensor Querying and Constrained Anisotropic Diffusion Routing Two routing techniques, Information-driven Sensor Querying and Constrained Anisotropic Diffusion Routing, were proposed in [37]. Constrained anisotropic diffusion routing aims to be a general form of directed diffusion. The key idea is to query sensors and route data in the network such that information gain is maximized while latency and bandwidth are minimized. constrained anisotropic diffusion routing diffuses queries by using a set of information criteria to select which sensors can get the data. This is achieved by activating only the sensors that are close to a particular event and dynamically adjusting data routes. The main difference from directed diffusion is the consideration of information gain in addition to communication cost. In constrained anisotropic diffusion routing, each node evaluates an information/cost objective and routes data based on the local information/cost gradient and end-user requirements. Estimation theory was used to model information utility. In Information-driven sensor querying, the querying node can determine which node can provide the most useful information with the additional advantage of balancing the energy cost. However, Information-driven sensor querying does not specifically define how the query and information are routed between sensors and the sink. Therefore, Information-driven sensor querying can be seen as a complementary optimization procedure. Simulation results showed that these approaches are more energyefficient than directed diffusion where queries are diffused in an isotropic fashion and reach nearest neighbours first.
3.10 ACQUIRE In [44] the authors have proposed a technique for querying sensor networks called Active Qwery Forwarding in Sensor Networks (ACQUIRE). Similar to ACQUIRE views the network as a distributed database where complex queries can be further divided into several subqueries. The operation of ACQUIRE can be described as follows. The sink node sends a query, which is then forwarded by each node receiving the query. During this, each node tries to respond to the query partially by using its precached information and then forwards it to another sensor node. If the precached information is not up-to-date, the nodes gather information from their neighbours within a lookahead of d hops. Once the query is resolved completely, it is sent back through either the reverse or shortest path to the sink. Hence, ACQUIRE can deal with complex queries by allowing many nodes to send
20
J. Cecílio, J. Costa, and P. Furtado
responses. Note that directed diffusion may not be used for complex queries due to energy considerations as directed diffusion also uses a flooding-based query mechanism for continuous and aggregate queries. On the other hand, ACQUIRE can provide efficient querying by adjusting the value of the lookahead parameter d. When d is equal to network diameter, ACQUIRE behaves similar to flooding. However, the query has to travel more hops if d is too small. To select the next node for forwarding the query, ACQUIRE either picks it randomly or the selection is based on maximum potential query satisfaction.
4 Hierarchical Routing Hierarchical or cluster-based routing methods, originally proposed in wire networks, are well-known techniques with special advantages related to scalability and efficient communication. As such, the concept of hierarchical routing is also utilized to perform energy-efficient routing in WSNs. In a hierarchical architecture, higherenergy nodes can be used to process and send the information, while low-energy nodes can be used to perform the sensing in the proximity of the target. The creation of clusters and assigning special tasks to cluster heads can greatly contribute to overall system scalability, lifetime, and energy efficiency. Hierarchical routing is an efficient way to lower energy consumption within a cluster, performing data aggregation and fusion in order to decrease the number of transmitted messages to the sink. Hierarchical routing is mainly two-layer routing where one layer is used to select cluster heads and the other for routing. However, most techniques in this category are not about routing, but rather “who and when to send or process/ aggregate” the information, channel allocation, and so on, which can be orthogonal to the multihop routing function. The main aim of hierarchical routing is to efficiently maintain the energy consumption of sensor nodes by involving them in multi-hop communication. Cluster formation is typically based on the energy reserve of sensors and sensor’s proximity to the cluster head. LEACH [18] is one of the first hierarchical routing approaches for sensors networks.
4.1 LEACH Heinzelman [59] introduced a hierarchical clustering algorithm for sensor networks, called Low Energy Adaptive Clustering Hierarchy (LEACH). LEACH is a cluster-based protocol, which includes distributed cluster formation. LEACH randomly selects a few sensor nodes as cluster heads and rotates this role to evenly distribute the energy load among the sensors in the network and extend the network lifetime. In LEACH, the cluster head nodes compress data arriving from nodes that belong to the respective cluster, and send an aggregated packet to the sink in order to reduce the amount of information that must be transmitted to the sink. LEACH uses a TDMA/code-division multiple access (CDMA) MAC to reduce intercluster and intracluster collisions. However, data collection is centralized and performed
Survey on Data Routing in Wireless Sensor Networks
21
periodically. Therefore, this protocol is most appropriate when there is a need for constant monitoring by the sensor network. A user may not need all the data immediately. Thus, periodic data transmissions, which may drain the limited energy of the sensor nodes, are unnecessary. The authors of LEACH introduced adaptive clustering, i.e., reclustering after a given interval with a randomized rotation of the energyconstrained cluster head so that energy dissipation in the sensor network is uniform. They also found, based on their simulation model, that only 5% of the nodes need to act as cluster heads. The operation of LEACH is broken up into rounds, where each round begins with a set-up phase (when the clusters are organized) followed by a steady-state phase (when data transfers to the sink occurs). The duration of the steady state phase is longer than the duration of the setup phase in order to minimize overhead. During the setup phase, each node decides whether or not to become a cluster head for the current round. This decision is based on the suggested percentage of cluster heads for the network (desired percentage to become a cluster head, p) and the number of times the node has been a cluster-head so far. A sensor node chooses a random number, r, between 0 and 1. If this random number is less than a threshold value, T(n), the node becomes a cluster head for the current round. The threshold is set as: p ⎧ ⎪⎪ ⎛ ⎛ 1 ⎞⎞ T (n ) = ⎨1 − P ⋅ ⎜⎜ r ⋅ mod⎜ ⎟ ⎟⎟ ⎝ P ⎠⎠ ⎝ ⎪ ⎪⎩ 0
if n ∈ G (7) otherwise
Where G is the set of nodes that have not been cluster-heads in the last 1/P rounds. Using this threshold, each node will be a cluster-head at some point within 1/P rounds. All elected cluster heads broadcast an advertisement message to the rest of the nodes in the network that they are the new cluster heads. All the non cluster head nodes, after receiving this advertisement, decide on the cluster to which they want to belong. This decision is based on the signal strength of the advertisement. The non cluster head nodes inform the appropriate cluster heads that they will be a member of the cluster. After receiving all the messages from the nodes that would like to be included in the cluster and based on the number of nodes in the cluster, the cluster head node creates a TDMA schedule and assigns each node a time slot when it can transmit. This schedule is broadcast to all the nodes in the cluster. During the steady state phase, the sensor nodes can begin sensing and transmitting data to the cluster heads. The cluster head node, after receiving all the data, aggregates them before sending them to the sink. After a certain time, which is determined a priori, the network goes back into the setup phase again and enters another round of selecting new cluster heads. Although LEACH is able to increase the network lifetime, there are still a number of issues about the assumptions used in this protocol. LEACH assumes that all nodes can transmit with enough power to reach the sink if needed and that
22
J. Cecílio, J. Costa, and P. Furtado
each node has computational power to support different MAC protocols. Therefore, it is not applicable to networks deployed in large regions. It also assumes that nodes always have data to send, and nodes located close to each other have correlated data. It is not obvious how the number of predetermined cluster heads (p) is going to be uniformly distributed through the network. Therefore, there is the possibility that the elected cluster heads will be concentrated in one part of the network; hence, some nodes will not have any cluster heads in their vicinity. Furthermore, the idea of dynamic clustering brings extra overhead (head changes, advertisements, etc.), which may diminish the gain in energy consumption. Finally, the protocol assumes that all nodes begin with the same amount of energy capacity in each election round, assuming that being a cluster head consumes approximately the same amount of energy for each node. The author proposed an extension that is LEACH with negotiation. The main theme of the proposed extension is that high-level negotiation using metadata descriptors (as in the SPIN protocol) precede data transfers. This ensures that only data that provide new information are transmitted to the cluster heads before being transmitted to the sink.
4.2 Energy Efficient Weight-Clustering Algorithm in Wireless Sensor Networks Cheng et al. [36] presents an energy efficient, weight-clustering algorithm (EWC) which aims to reduce energy consumption by perfecting cluster formation procedure in cluster-based protocols. During a cluster head selection stage, the authors take several parameters into consideration. They distribute different weight coefficients to parameters such as residual energy, location and node degree, and the nodes with minimal combined weight become cluster heads. The EWC algorithm allows adjusting the coefficients based on network requirements. The main idea for EWC is to combine several different weight metrics such as residual energy, distance and node degree, and to take those into consideration in cluster head selection process. The authors have considered different parameters with different weights, according to specific system requirements. The selection of cluster heads can greatly affect the performance of the whole network. To decide how well a node is suited to become a cluster head, several features are taken into consideration, such as residual power, distance between cluster heads, nodes and base station, and node degree. The node degree is decided by the number of neighbors the node has. The authors proposed the following steps to select a cluster header: -
Estimating distance DBS between sensor node and base station.
-
Discover the neighbours of each sensor node v . Sensor nodes broadcast neighbour discovery messages, which contain the node ID, energy level and distance to base station.
-
Compute the degree diversity
δ
Δv . For node v : Δv = d v − δ .
represents the ideal number of cluster members a cluster can handle
Survey on Data Routing in Wireless Sensor Networks
23
d v denotes the number of neighbors sensor node v has within its transmission range t xrange . and
d v = N (v ) = -
∑ {dist (v, w) < t
w∈V , w≠ v
}
Discover the average distance between cluster head and cluster members DCM .
DCM = -
x range
Calculate energy portion
∑d
2 to CH
w∈V , w ≠ v
dv
EP . EP =
Einit Ev
Ev represents the residual energy for node v for round n while Einit represents the initial energy for node v .
Where
-
Calculate the combined weight
Wv for each node v using the equation
Wv = w1 E p + w2 Δv + w3 (Dto CH + Dto BS )
w1 , w2 and w3 ( w1 + w2 + w3 = 1 ) are the coefficient of weight factors. -
Weight exchange. After each node v calculates its weight, it exchanges weight information with its neighbors. Neighbors who receive the message store the information into their neighbor tables. Nodes with minimum weight will be selected as cluster heads and other nodes decide to join appropriate clusters according to the strength of announcement message they receive from cluster heads.
After the cluster topology is formed, the cluster head creates a schedule for cluster members and starts steady-state as the LEACH Algorithm.
4.3 Power-Efficient Gathering in Sensor Information Systems In [51], an enhancement over the LEACH protocol was proposed. The protocol, called Power-Efficient Gathering in Sensor Information Systems (PEGASIS), is a near optimal chain-based protocol. The basic idea of the protocol is that in order to extend network lifetime, nodes need only communicate with their closest neighbours, and they take turns in communicating with the base station. When the round of all nodes communicating with the base station ends, a new round starts, and so on. This reduces the power required to transmit data per round as the power
24
J. Cecílio, J. Costa, and P. Furtado
draining is spread uniformly over all nodes. Hence, PEGASIS has two main objectives: – –
First, increase the lifetime of each node by using collaborative techniques. Second, allow only local coordination between nodes that are close together so that the bandwidth consumed in communication is reduced.
Unlike LEACH, PEGASIS avoids cluster formation and uses only one node in a chain to transmit to the base station instead of multiple nodes. To locate the closest neighbour node in PEGASIS, each node uses the signal strength to measure the distance to all neighbouring nodes and then adjusts the signal strength so that only one node can be heard. The chain in PEGASIS will consist of those nodes that are closest to each other and form a path to the base station. The aggregated form of the data will be sent to the base station by any node in the chain, and the nodes in the chain will take turns sending to the base station. The chain construction is performed in a greedy fashion. As shown in Figure 3: Chaining in PEGASIS node c0 passes its data to node c1. Node c1 aggregates node c0 data with its own and then transmits to the leader. After node c2 passes the token to node c4, node c4 transmits its data to node c3. Node c3 aggregates node c4 data with its own and then transmits to the leader. Node c2 waits to receive data from both neighbours and then aggregates its data with its neighbours data. Finally, node c2 transmits one message to the base station. C0
C1
C2
C3
C4
Base Station Fig. 3 Chaining in PEGASIS
The difference from LEACH is to use multi-hop routing by forming chains and selecting only one node to transmit to the base station instead of using multiple nodes. PEGASIS has been shown to outperform LEACH by about 100 to 300% for different network sizes and topologies. Such performance gain is achieved through the elimination of the overhead caused by dynamic cluster formation in LEACH and through decreasing the number of transmissions and reception by using data aggregation. However, PEGASIS introduces excessive delay for distant node on the chain. In addition the single leader can become a bottleneck. An extension to PEGASIS, called Hierarchical PEGASIS, was introduced in [5] with the objective of decreasing the delay incurred for packets during transmission to the base station. For this purpose, simultaneous transmissions of data are studied in order to avoid collisions through approaches that incorporate signal coding and spatial transmissions.
Survey on Data Routing in Wireless Sensor Networks
25
The PEGASIS approaches avoid the clustering overhead of LEACH, they still require dynamic topology adjustment since sensor’s energy is not tracked. For example, every sensor needs to be aware of the status of its neighbour so that it knows where to route that data. Such topology adjustment can introduce significant overhead especially for highly utilized networks.
4.4 Threshold-Sensitive Energy-Efficient Protocols Two hierarchical routing protocols called Threshold-Sensitive Energy Efficient Sensor Network Protocol (TEEN) and Adaptive Periodic TEEN (APTEEN) are proposed in [1][2]. These protocols were proposed for time-critical applications. In TEEN, sensor nodes sense the medium continuously, but data transmission is done less frequently. A cluster head sensor sends its members a hard threshold, which is the threshold value of the sensed attribute, and a soft threshold, which is a small change in the value of the sensed attribute that triggers the node to switch on its transmitter and transmit. Thus, the hard threshold tries to reduce the number of transmissions by allowing the nodes to transmit only when the sensed attribute is in the range of interest. The soft threshold further reduces the number of transmissions that might otherwise occur when there is little or no change in the sensed attribute. A smaller value of the soft threshold gives a more accurate picture of the network, at the expense of increased energy consumption. Thus, the user can control the trade-off between energy efficiency and data accuracy. When cluster heads are to change Figure 4, new values for the above parameters are broadcast.
Fig. 4 Time line for the operation of (a) TEEN and (b) APTEEN
26
J. Cecílio, J. Costa, and P. Furtado
The main drawback of this scheme is that if the thresholds are not received, the nodes will never communicate, and the user will not get any data from the network at all. The nodes sense their environment continuously. The first time a parameter from the attribute set reaches its hard threshold value, the node switches its transmitter on and sends the sensed data. The sensed value is stored in an internal variable called sensed value. The nodes will transmit data in the current cluster period only when the following conditions are true: – –
The current value of the sensed attribute is greater than the hard threshold. The current value of the sensed attribute differs from sensed value by an amount equal to or greater than the soft threshold.
Important features of TEEN include its suitability for time-critical sensing applications. Also, since message transmission consumes more energy than data sensing, the energy consumption in this scheme is less than in proactive networks. The soft threshold can be varied. At every cluster change time, fresh parameters are broadcast, so the user can change them as required. APTEEN, on the other hand, is a hybrid protocol that changes the periodicity or threshold values used in the TEEN protocol according to user needs and the application type. In APTEEN, the cluster heads broadcast the following parameters: – – – –
Attributes (A): a set of physical parameters about which the user is interested in obtaining information; Thresholds: consists of the hard threshold (HT) and soft threshold (ST); Schedule: a TDMA schedule, assigning a slot to each node; Count time (CT): the maximum time period between two successive reports sent by a node.
The node senses the environment continuously, and only those nodes that sense a data value at or beyond HT transmit. Once a node senses a value beyond HT, it transmits data only when the value of that attribute changes by an amount equal to or greater than ST. If a node does not send data for a time period equal to CT, it is forced to sense and retransmit the data. A TDMA schedule is used, and each node in the cluster is assigned a transmission slot. Hence, APTEEN uses a modified TDMA schedule to implement the hybrid network. The main features of the APTEEN scheme include the following. It combines both proactive and reactive policies. It offers a lot of flexibility by allowing the user to set the CT interval, and the threshold values for energy consumption can be controlled by changing the CT as well as the threshold values. The main drawback of the scheme is the additional complexity required to implement the threshold functions and CT. Simulation of TEEN and APTEEN has shown that these two protocols outperform LEACH. The experiments have demonstrated that APTEEN’s performance is somewhere between LEACH and TEEN in terms of energy dissipation and network lifetime.
Survey on Data Routing in Wireless Sensor Networks
27
TEEN gives the best performance since it decreases the number of transmissions. The main drawbacks of the two approaches are the overhead and complexity associated with forming clusters at multiple levels, the method of implementing threshold-based functions, and how to deal with attribute-based naming of queries.
4.5 Energy-Aware Routing for Cluster-Based Sensor Networks In [41] the author have proposed a different hierarchical routing algorithm based on a three-tier architecture. Sensors are grouped into clusters prior to network operation. The algorithm employs cluster heads, namely gateways, which are less energy constrained than sensors and assumed to know the location of sensor nodes. Gateways maintain the states of the sensors and sets up multi-hop routes for collecting sensor data. A TDMA based MAC is used for nodes to send data to the gateway. The gateway informs each node about slots in which it should listen to other node transmission and slots, which the node can use for its own transmission. The command node (sink) communicates only with the gateways. The sensor is assumed to be capable of operating in an active mode or a lowpower stand-by mode. The sensing and processing circuits can be powered on and off. In addition both the radio transmitter and receiver can be independently turned on and off and the transmission power can be programmed based on the required range. The sensor nodes in a cluster can be in one of four main states: sensing only, relaying only, sensing-relaying, and inactive. In the sensing state, the node probes the environment and generates data at a constant rate. In the relaying state, the node does not sense the target but its communications circuitry is on to relay the data from other active nodes. When a node is both sensing and relaying messages from other nodes, it is considered in the sensing-relaying state. Otherwise, the node is considered inactive and can turn off its sensing and communication circuitry.
Fig. 5 A typical cluster in a sensor network
28
J. Cecílio, J. Costa, and P. Furtado
In Figure 5 it is shows an example of the state of sensors and routes within a typical cluster for a target-tracking application. A cost function is defined between any two nodes in terms of energy consumption, delay optimization and other performance metrics. Using this cost function as the link cost, a least-cost path is found between sensor nodes and the gateway. The gateway will continuously monitor the available energy level at every sensor that is active in data processing, sensing, or in forwarding data packets, relaying. Rerouting is triggered by an application-related event requiring different set of sensors to probe the environment or the depletion of the battery of an active node. To model the sensor network within the cluster, the authors assumes that nodes, sensors and gateway, are connected by bidirectional wireless links with a cost associated with each direction. Each link may have a different cost for each direction due to different energy levels of the nodes at each end. The cost of a path between two nodes is defined as the sum of the costs of the links traversed. For each sensing-enabled node, the routing algorithm should find a least-cost path from this node to the gateway. The routing algorithm can find the shortest path from the gateway to the sensing-enabled nodes and then using the transpose property. To account for energy conservation, delay optimization and other performance metrics, we define the following cost function for a link between nodes i and j: 7
∑ CF
k
k =0
( )
( )
= c0 × d ij l + c1 × f E j +
c2 + c3 + c4 + c5 + c6 × d ij + c7 × overall load Tj
(8)
Where: d ij : Distance between the nodes i and j E j : Current energy of each node j CFk are cost factors defined as follows:
( )
–
CF0: Communication Cost = c0 × d ij l , where c0 is a weighting constant
–
and the parameter l depends on the environment, and typically equals to 2. This factor reflects the cost of the wireless transmission power, which is directly proportional to the distance raised to some power l. CF1: Energy stock = c1 × f E j , this cost factor favors nodes with more
–
( )
energy. The more energy the node contains, the better it is for routing. The function f’ is chosen to reflect the battery remaining lifetime. c CF2: Energy consumption rate = 2 , where c2 is a weighting constant and Tj
Tj is the expected time under the current consumption rate until the node j energy level hits the minimum acceptable threshold. CF2 makes the heavily used nodes less attractive, even if they have a lot of energy.
Survey on Data Routing in Wireless Sensor Networks
–
–
–
–
29
CF3: Relay enabling Cost = c3 , where c3 is a constant reflecting the overhead required to switch an inactive node to become a relay. This factor makes the algorithm favor the relay-enabled nodes for routing rather than inactive nodes. CF4: Sensing-state Cost = c4 , where c4 is a constant added when the node j is in a sensing-sate. This factor does not favor selecting sensing-enabled nodes to serve as relays, since they have committed some energy for data processing. CF5: Maximum connections per relay: once this threshold is reached, we add an extra cost c5 to avoid setting additional paths through it. This factor extends the life of overloaded relay nodes by making them less favorable. Since these relay nodes are already critical by being on more than one path, the reliability of paths through these nodes increases. CF6: Propagation delay = c6 × d ij , where c6 is the result of dividing a weighting constant by the speed of wireless transmission. This factor favors closer nodes.
–
CF7: Queuing Cost = c7 ×
λ , where λ = Σλs for each sensor node s whose μ
route passes through the node j, λs is data-sensing rate for node s and µ is the service rate (mainly store-and-forward). The expression λ / µ is the average queue length for the M/M/1 queuing model. This factor can be mathematically simplified to be the overall load to the relay node, where the overall load is the total number of sensing-enabled nodes whose data messages are sent via routes through this node. Assuming equal service rate µ for each relay as well as equal data-sensing rate λs for each sensingenabled node, the constant µ can be reduced inside c7, and λ can be reduced to the overall load times the constant data sensing rate for each sensor λs. Thus, CF7 = c7 × overall load . This factor does not favor relays with long queues to avoid dropping or delaying data packets. The authors considered as best approach for solving the one-to-some shortest path is Dijkstra’s algorithm. In addition, Dijkstra's algorithm is shown to suit centralized routing. Therefore, they uses Dijkstra's algorithm with the link cost dij CFk . for the link between the nodes i and j, redefined as d ij =
∑ k
A variant of this routing approach has been proposed in [40]. The algorithm constrains the minimum transmission range in order to limit the delay. Simulation results have demonstrated that such approach consistently performs well with respect to both energy-based metrics, e.g. network lifetime, as well as contemporary metrics, e.g. throughput and end-to-end delay. The results also have indicated that combining the routing approach with the time-based medium arbitration can further increase the life of the network by an order of magnitude.
30
J. Cecílio, J. Costa, and P. Furtado
4.6 Self-Organizing Protocol In [35] it is described a Self-organizing Protocol (SOP) and an application taxonomy that was used to build architecture to support heterogeneous sensors. Furthermore, these sensors can be mobile or stationary. Some sensors probe the environment and forward the data to a designated set of nodes that act as routers. Router nodes are stationary and form the backbone for communication. Collected data are forwarded through the routers to the more powerful base station nodes. Each sensing node should be able to reach a router in order to be part of the network. A routing architecture that requires addressing of each sensor node has been proposed. Sensing nodes are identifible through the address of the router node to which they are connected. The routing architecture is hierarchical where groups of nodes are formed and merge when needed. The Local Markov Loops (LML) algorithm, which performs a random walk on spanning trees of a graph, was used to support fault tolerance and as a means of broadcasting. The algorithm for self-organizing the router nodes and creating the routing tables consists of four phases: • Discovery phase: Each node independently discovers its set of neighbours in the network and fixes its maximum radius of data transmission. • Organizational phase: During this phase the network is organized and the following operations are performed: – Node aggregate themselves into groups and groups are aggregated to form larger groups. In this way, a hierarchy of groups is formed in the network. The algorithm ensures that the hierarchy is height balanced. – Each node is allocated an address based on its position in the hierarchy. – A routing table of O (log n) is computed for every node in the network. – A broadcast tree and a broadcast graph spanning all nodes in the graph is constructed. The broadcast graph is then converted into a directed acyclic graph based on the source node in the network. • Maintenance phase: In the maintenance phase the following operations are performed. – In active monitoring, every node keeps track of its stored energy and constantly sends Iamalive message to its neighbours once in30 sec. In passive monitoring, a sensor nodes sends an activate message to its neighbours only on demand. – Every node constantly updates its routing table about the next hop in the least power consuming path and the shortest delay path to the groups as dictated by the algorithm. – Nodes also inform their neighbours of their routing tables and their energy levels to their neighbouring nodes.
Survey on Data Routing in Wireless Sensor Networks
–
31
Fault tolerant broadcast trees and broadcast graphs are maintained using Local Markov Loops(LML).
• Self-Reorganization phase: In this phase, a node may detect group partitions or node failures and change its routing table based on the new topology. If all neighbours of a node fail, then the node repeats the discovery phase. If a group partition occurs due to link or node failures, the sub groups reorganize and join with new groups. Group re-organization ensures that the hierarchy is still balanced.
In this approach, sensor nodes can be addressed individually in the routing architecture; hence, it is suitable for applications where communication to a particular node is required. Furthermore, this algorithm incurs a small cost for maintaining routing tables and keeping a balanced routing hierarchy. This protocol is not an on-demand protocol, especially in the organization phase of the algorithm, and thus introduces extra overhead. Another issue is related to the formation of a hierarchy. It could happen that there are many cuts in the network, and hence the probability of applying reorganization phase increases, which is an expensive operation.
4.7 Minimum Energy Communication Network Rodoplu et al. [57] have proposed a protocol that computes an energy-efficient subnetwork, the Minimum Energy Communication Network, for a certain sensor network utilizing low-power GPS. Minimum energy communication network (MECN) identifies a relay region for every node. The relay region consists of nodes in a surrounding area where transmitting through those nodes is more energy-efficient than direct transmission. The enclosure of a node i is created by taking the union of all relay regions node i can reach. The main idea of MECN is to find a subnetwork that will have fewer nodes and require less power for transmission between any two particular nodes. The protocol has two phases: –
–
It takes the positions of a two-dimensional plane and constructs a sparse graph (enclosure graph), which consists of all the enclosures of each transmit node in the graph. This construction requires local computations in the nodes. The enclose graph contains globally optimal links in terms of energy consumption. Finds optimal links on the enclosure graph. It uses distributed BelmannFord shortest path algorithm with power consumption as the cost metric. In case of mobility the position coordinates are updated using GPS.
MECN is self-reconfiguring and thus can dynamically adapt to node failure or the deployment of new sensors. It is assumed that every node can transmit to every other node.
32
J. Cecílio, J. Costa, and P. Furtado
4.8 Small Minimum Energy Communication Network In [33], a protocol Small Minimum Energy Communication Network (SMECN) is proposed by Li. In SMECN possible obstacles between any pair of nodes are considered. However, the network is still assumed to be fully connected as in the case of MECN. The subnetwork constructed by SMECN for minimum energy relaying is provably smaller (in terms of number of edges) than the one constructed in MECN. Hence, the subnetwork (i.e., subgraph G′) constructed by SMECN is smaller than the one constructed by MECN if the broadcast region is circular around the broadcasting node for a given power setting. Subgraph G′ of graph G, which represents the sensor network, minimizes the energy usage satisfying the following conditions: – –
The number of edges in G′ is less than in G while containing all nodes in G. The energy required to transmit data from a node to all its neighbors in subgraph G’ is less than the energy required to transmit to all its neighbors in graph G. Assume that r = (u, u1,…, v) is a path between u and v that spans k-1 intermediate nodes u1, … uk-1. The total power consumption of one path like r is given by C (r ) =
k −1
∑ ( p(u , u i
i +1
) + c)
(9)
i =1
where u = u0 and v = uk, and the power required to transmit data under this
protocol is p(u, v ) = t ⋅ d (u, v )n , and t is the instant of the time, n is the path loss exponent of outdoor radio propagation models n ≥ 2, and d(u,v) is the distance between u and v.
5 Location-Based Protocols Most of the routing protocols for sensor networks require location information for sensor nodes. In most cases location information is needed in order to calculate the distance between two particular nodes so that energy consumption can be estimated. Relative coordinates of neighbouring nodes can be obtained by exchanging such information between neighbours [43]. Alternatively, the location of nodes may be available directly by communicating with a satellite using GPS if nodes are equipped with a small low-power GPS receiver. To save energy, some location-based schemes demand that nodes should go to sleep if there is no activity. More energy savings can be obtained by having as many sleeping nodes in the network as possible. The problem of designing sleep period schedules for each node in a localized manner was studied in [62]. In this
Survey on Data Routing in Wireless Sensor Networks
33
section, we review two geographic routing protocols, such as Geographic Adaptive Fidelity and Geographic and energy aware routing.
5.1 Geographic Adaptive Fidelity Geographic Adaptive Fidelity (GAF) [62] is an energy-aware location-based routing algorithm. GAF conserves energy by turning off unnecessary nodes in the network without affecting the level of routing fidelity. It forms a virtual grid for the covered area. Each node uses its GPS-indicated location to associate itself with a point in the virtual grid. Nodes associated with the same point on the grid are considered equivalent in terms of the cost of packet routing. So equivalence is exploited in keeping some nodes located in a grid area to sleeping state in order to save energy. GAF can substantially increase the network lifetime as the number of nodes increases. In Figure 6 it is show a sample situation. Node 1 can reach any of 2, 3, 4 and 5 and nodes 2, 3, 4 and 5 can reach 6. Therefore nodes 2, 3, 4 and 5 are equivalent and three of them can sleep.
2
1
3
r 4
5 r
r
6 r
Fig. 6 Example of virtual grid in GAF
There are three states defined in GAF: – – –
discovery, for determining the neighbours in the grid; active, reflecting participation in routing; sleep, when the radio is turned off.
In order to handle mobility, each node in the grid estimates its time of leaving the grid and sends this to its neighbors. The sleeping neighbors adjust their sleeping time accordingly in order to keep routing fidelity. Before the leaving time of the active node expires, sleeping nodes wake up and one of them becomes active. GAF is a location-based protocol, it may also be considered a hierarchical protocol, where the clusters are based on geographic location. For each particular grid area, a representative node acts as the leader to transmit the data to other
34
J. Cecílio, J. Costa, and P. Furtado
nodes. The leader node, however, does not do any aggregation or fusion as in the case of hierarchical protocols.
5.2 Energy Aware Greedy Routing (EAGR) Razia et al. [48] suggest a location-based protocol that works on geographical information of the node as well as the energy level available in the sensor node. In most of the greedy routing algorithms, only shortest-path is calculated, keeping aside the fact that in this case the node present in most of the shortest paths will loose its energy very quickly. Therefore, it creates a hole in that area and results in dropping of packets. EAGR combines the location information and energy level of nodes so well that the workload is evenly distributed among the nodes that are alive. In EAGR, all nodes have the same energy level and a threshold energy level is set. Nodes having less than that energy level are considered dead. Then, the algorithm finds out the location of each node. All the nodes having energy level greater than their threshold value get information about their neighbor and create a table of their locations. On the basis of this table, average distance to its neighbor is calculated. For forwarding data, the algorithm selects the node having distance equal to or less to this average distance value and having maximum energy level amongst the neighbors. By considering energy level in selection, whenever needed, a new node is selected, and no single node gets its energy depleted more quickly, resulting in longer life for the network. In EAGR, packets only get dropped when the destination is dead or there is no further neighbor alive to forward data. Authors have compared the results of EAGR with shortest path greedy algorithm using OMNET++ simulator. The results show the great difference between these two protocols. EAGR is much better than simple greedy algorithm.
5.3 Geographic and Energy Aware Routing The protocol, Geographic and Energy Aware Routing (GEAR) [61], uses energyaware and geographically informed neighbour selection heuristics to route a packet toward the destination region. The key idea is to restrict the number of interests in directed diffusion by only considering a certain region rather than sending the interests to the whole network. By doing this, GEAR can conserve more energy than directed diffusion. In GEAR, each node keeps an estimated cost and a learning cost of reaching the destination through its neighbors. The estimated cost is a combination of residual energy and distance to destination. The learned cost is a refinement of the estimated cost that accounts for routing around holes in the network. A hole occurs when a node does not have any closer neighbor to the target region than itself. If there are no holes, the estimated cost is equal to the learned cost. The learned cost is propagated one hop back every time a packet reaches the destination so that route setup for next packet will be adjusted. There are two phases in the algorithm:
Survey on Data Routing in Wireless Sensor Networks
35
• Forwarding packets towards the target region: Upon receiving a packet, a node checks its neighbours to see if there is one neighbour, which is closer to the target region than itself. – If there is more than one, the nearest neighbor to the target region is selected as the next hop. – If they are all further than the node itself, this means there is a hole. In this case, one of the neighbors is picked to forward the packet based on the learning cost function. This choice can then be updated according to the convergence of the learned cost during the delivery of packets. • Forwarding the packets within the region: If the packet has reached the region, it can be diffused in that region by either recursive geographic forwarding or restricted flooding. Restricted flooding is good when the sensors are not densely deployed. In high-density networks, recursive geographic flooding is more energy efficient than restricted flooding. In that case, the region is divided into four sub regions and four copies of the packet are created. This splitting and forwarding process continues until the regions with only one node are left.
6 QoS-Aware Protocols The network needs to ensure Quality of Service (QoS) besides ease of deployment, energy efficiency and low cost. One of the major design goals of WSNs is reliable data communication under minimum energy depletion to extend the lifetime of the network. This may be achieved via aggressive energy management techniques. Owing to their poor energy conservation, traditional routing protocols are not suitable for WSN applications. It is highly desirable to employ an energy-efficient route discovery and data relaying techniques to transfer data between the sensor nodes and the base station. Some of the routing challenges and design issues that affect the routing process in WSN are: node deployment, data reporting method, node/link heterogeneity, fault tolerance, scalability, transmission media, data aggregation, connectivity, coverage and QoS. In QoS-based routing protocols, the network has to balance between energy consumption and data quality. In particular, the network has to satisfy certain QoS metrics (delay, energy, bandwidth) when delivering data to the base station. In this context, we review three routing protocols, such as SPEED, Sequential Assignment Routing and Real-time Power-Aware Routing.
6.1 SPEED SPEED [55] is a geographic routing protocol designed for real-time communication in sensor networks. SPEED handles congestion and provides soft real-time communication by using feedback control and non-deterministic geographic forwarding. It also provides a different way to handle dead-ends similar to the way it handles congestion. Non-deterministic geographic forwarding is used to balance the load among multiple routes. A node computes a relay speed to each of its
36
J. Cecílio, J. Costa, and P. Furtado
neighbours by dividing the advance in distance to the destination by the estimated delay to forward the packet to that neighbour. The node then forwards the packet to a neighbour closer to the destination that has a speed higher than a certain threshold with a probability based on that neighbour speed compared to other neighbours. If no neighbour has a speed higher than the desired speed, a neighbourhood feedback loop determines whether to drop the packet or reroute it in order to reduce the congestion. Backpressure rerouting is used to avoid both congestion and dead-ends by sending a backpressure beacon to the upstream node. The upstream node will try to forward the packet on a different route or further backpressure will occur until a route is found. Dead-end recovery using this backpressure mechanism does not guarantee to find a path. SPEED considers also other functions such as geocast and anycast which can be activated after the packet enters the destination area.
6.2 MMSPEED MMSPEED [21] is an extension of SPEED which supports service differentiation and probabilistic QoS guarantees. For delivery timeliness, multiple network-wide packet delivery speed options are provided for different traffic types, according to their end-to-end deadlines. In supporting service reliability, probabilistic multipath forwarding is used to control the number of delivery paths based on the required end-to-end reach probability. These methods are implemented in a localized way with dynamic compensation for the inaccuracies of local decisions. Like SPEED, since all mechanisms in MMSPEED work locally without global network state information and end-to-end path setup, it is scalable and adaptive to network dynamics. However, both SPEED and MMSPEED have a common deficiency: the energy consumption metric has not been taken into account.
6.3 Sequential Assignment Routing The Sequential Assignment Routing protocol (SAR) presented in [31] is one of the first routing protocols for WSNs to introduce the notion of QoS into routing decisions. The SAR protocol creates trees routed from one-hop neighbour of the sink by taking QoS metric, energy resource on each path and priority level of each packet into consideration. By using created trees, multiple paths from sink to sensors are formed. Furthermore, one of the paths can be selected according to the energy resources and QoS on each path. So, a routing decision in SAR is dependent on three factors: energy resources, QoS on each path, and the priority level of each packet. To avoid single route failure, a multi-path approach is used and localized path restoration schemes are used. The objective of SAR algorithm is to minimize the average weighted QoS metric throughout the lifetime of the network. The SAR algorithm offers less power consumption than the minimum-energy metric algorithm, which focuses only the energy consumption of each packet without considering its priority.
Survey on Data Routing in Wireless Sensor Networks
37
6.4 Real-Time Power-Aware Routing Real-time Power-Aware Routing (RPAR) [12] protocol achieves applicationspecified communication delays at low energy cost by dynamically adapting transmission power and routing decisions. RPAR features a power-aware forwarding policy and an efficient neighbourhood manager that are optimized for resource-constrained wireless sensors. RPAR addresses important practical issues in wireless sensor networks, including lossy links, scalability, and severe memory and bandwidth constraints. RPAR has two salient features: – –
It improves the number of packets meeting their deadlines at low energy cost. It has an efficient neighbourhood manager that quickly discovers forwarding choices (pairs of a neighbour and a transmission power) that meet packet deadlines while introducing low communication and energy overhead.
RPAR assumes that each packet is assigned a soft deadline by the application, which specifies the desired bound on the end-to-end delay of a packet. The primary goal of RPAR is to increase the number of packets that meet their deadlines while minimizing the energy consumed for transmitting packets under their deadline constraints. RPAR focuses on minimizing the energy consumed in packet transmissions. RPAR has capacity of dynamically adapt its transmission power and routing decisions based on workload and packet deadlines, include losses links and extreme resource constraints in terms of memory, bandwidth and energy. RPAR is a localized protocol that makes decisions based solely on one-hop neighbourhood information. This property enables RPAR to scale effectively to large WSNs.
6.5 Delay-Constrained Energy Efficient Routing Protocol Pothuri et al. [47] designs a heuristic solution to find energy-efficient path for delay-constrained data in WSNs (DCEERP). They employ topology control and have a modeling of the contention delay caused by MAC layer. A set of paths between source and sink nodes are identified and indexed in increasing order of their energy consumption. End-to-end delay is estimated along each of the ordered paths and the one with the lowest index that satisfies the delay constraint is selected. However, their solution is based on the assumption that nodes are equipped with two radios: a low-power radio for short-range and a high-power radio for long-range communication such that each node can reach the sink directly using its long-range radio. This requirement is energy inefficient and may not be practical.
6.6 Energy Efficient Routing with Delay Guarantees Ergen et al. [20] presents an energy efficient routing method with delay guarantees for sensor networks. They first exclude the delay constraint and formulate the
38
J. Cecílio, J. Costa, and P. Furtado
lifetime maximization as a linear programming (LP) problem with the goal of determining optimal routing paths and maximizing the minimum lifetime of each node in the network. The LP solution is first implemented in a centralized way and then approximated by an iterative algorithm based on least cost path routing. Then, delay guarantee is included in the energy efficient routing by limiting the length of routing path from each node to the sink. The simulation shows that the maximum delay can be limited to a certain level. However, one may find that the result is not flexible to meet application specified delay bound generally.
7 Data-Centric Routing Protocols for Mobility in WSN Many aspects of data routing are harder to deal with when nodes are moving, since issues such as connectivity, energy-efficiency and latency become more difficult to optimize. Solutions for node mobility typically rely on either frequently updating local information that is necessary for routing, such as node neighbours or routing tables, or on searching for the information on-demand, when data must be routed. The corresponding routing protocols can be classified into two categories, table-driven or demand-driven. The most relevant issue with the first one is updating frequency, while demand-driven approaches may have larger latencies due to on-demand routing. Mobility-aware data routing applies approaches such as frequently updated gradients to route data in the most efficient manner, aggregating data to reduce the time or frequency of sends or to deliver data when possible, or dealing efficiently with mobile data collection using mechanisms such as mules. In this section we first review table and on-demand driven routing as basis for mobility, then we review some data-centric routing approaches in mobility and finally we refer to some interesting projects addressing data routing in mobility in specific application contexts. Table-driven routing protocols are based on periodically exchanging routing information between nodes. Each node builds its own routing table which it can use to find a path to a destination. However, keeping the information up-to-date may require a lot of bandwidth, which is sparse, and extra battery power, which is usually limited in mobile ad-hoc wireless sensor networks. Nevertheless, information may still be out-of-date. Perkins et al [11] proposed a Destination-Sequenced Distance-Vector Routing Protocol (DSDV), which is a table-driven algorithm based on the classical Bellman-Ford routing mechanism. The basic idea of the design is to operate each Mobile Host as a specialized router, which periodically advertises its view of the interconnection topology with other Mobile Hosts within the network. Each entry in the routing table contains a sequence number, which is generated by the destination, and nodes need to send out the next update with this number. Routing information is distributed between nodes, by sending full dumps infrequently and smaller incremental updates more frequently. Chiang et al. [8] proposed the Clustered Gateway Switch Routing (CGSR). CGSR is a clustered multihop mobile wireless network. Each sensor node must keep a cluster member table where it stores the destination cluster head for each mobile node in the network. These cluster member tables are broadcast by each
Survey on Data Routing in Wireless Sensor Networks
39
node periodically using the DSDV algorithm. Sensor nodes update their cluster member tables on reception of such a table from a neighbour. Clausen et al. [54] proposed the Optimized Link State Routing (OLSR) Protocol. The OLSR is a proactive link-state routing protocol which uses Hello and Topology Control messages to discover and then discriminate link state information throughout the mobile ad-hoc network. Individual nodes use this topology information to compute next-hop destinations for all nodes in the network using shortest hop forwarding paths. Transier et al. [39] proposed a Scalable Position Based Multicast for Mobile Ad-Hoc Networks (SPBM), which was designed to improve scalability. It uses the geographic position of nodes to provide a scalable group membership scheme and to forward data packets. SPBM is mainly focused on the task of managing multicast groups in a scalable way. In contrast to table-driven routing protocols, on-demand protocols routes are not maintained up-to-date at every node, instead the routes are created as and when required. When a route from a source to a destination is needed, a kind of global search procedure is started. This task does not request constant updates to be sent through the network, but it still causes delays, since the requested routes are not available and have to be found. In some cases, the desired routes are still in the route cache maintained by the sensor nodes. When this is the case, there is no additional delay, since routes do not have to be discovered. The route remains valid while the destination is reachable, or until the route is no longer needed. Perkins et al. [10] has proposed the Ad hoc On-demand Distance Vector Routing (AODV) as an improvement to DSDV. AODV minimizes the number of broadcasts by creating routes on-demand as opposed to DSDV that maintains the list of all the routes. To find a path to the destination, the source broadcasts a route request packet. The neighbors in turn broadcast the packet to their neighbors till it reaches an intermediate node that has recent route information about the destination or till it reaches the destination. A node discards a route request packet that it has already seen. The route request packet uses sequence numbers to ensure that the routes are loop free and to make sure that if the intermediate nodes reply to route requests, they reply with the latest information only. Another feature is the use of timestamps in routing tables to manage the expired entries. Lee et al. [53] has proposed the solicitation-based forwarding (SOFA), a highly-responsive hop-by-hop routing protocol that results in increased application fidelity. SOFA represents a cost-effective, on-demand scheme that makes use of simple solicitation-based handshakes between a sender and multiple potential receivers at each wireless hop to negotiate the best forwarding path to a target destination when events occur in the sensor field.
7.1 Data-Centric Routing Approaches Data-centric routing focuses on data that is produced in a set of nodes and must be consumed elsewhere, and how to optimizing metrics such as energy, latency, fault-tolerance, disconnection or throughput using the data and how it is processed. For instance, the data from two sources (A and B) is aggregated at node
40
J. Cecílio, J. Costa, and P. Furtado
C, and the combined data is sent from C to the base station. This combination of data saves energy due to a reduced number of transmissions, and guarantees that all information from both sources is sent; Another example is a moving node that aggregates data until the node passes through some sink and handles the data to that sink. Data-centric routing has proven to be a good scheme for managing data and to optimize resources. In the following points we review some data-centric routing approaches in mobility. Wu et al. [29] proposed reliable cost-based data-centric routing (RCDR). The approach has two kinds of gradients, global and local gradient. When a data sink wants to collect data from the network, it sends out a data query to setup global gradient in the whole network. While this query message propagates in the network, each sensor establishes its own cost value toward this sink. Afterwards, any data sent towards the sink follows through global gradients by multipath routing. The gathered data is first routed to a data aggregation point. After the aggregation point has already processed the data from the surrounding sensors, it sends the aggregated report to the sink via global gradients. Each node sends its data by broadcasting with a unique identifier (the sequence number of the data). Each node will only forward the first data it receives and ignore other copies. When the data travels in the network, it flows through the lower cost nodes. If the connection between two nodes breaks because a node moves, the data is still forwarded by other nodes. The authors used simulation results to show that their approach benefits from the global-local paradigm they use. In a static network it increased the network reliability by at least 25% and in a dynamic topology, the local recovery approaches of clearly outperformed the traditional protocol in reliability, while energy consumption was less than 50%. Huang et al. [24] presented Spiral, a data-centric routing algorithm that balances the cost of route discovery and route length, making it efficient for shortterm communication of tens or hundreds of messages. Spiral reduces the communication overhead and the energy consumption for both route discovery and subsequent communication. Like the route discovery algorithms for one-shot communication, Spiral uses a walk-based mechanism to reduce the overhead of searching for the desired data. This results in a spiral-like search path that is not only more likely to find a closer copy of the desired data than random walk, but is also able to compute a shorter route because the network around the source is more thoroughly explored. The authors evaluated the spiral through simulation to show that in a 500-node network with an average degree of 20 and two copies of every data object, for a short-term communication of 40 data objects, the total communication cost of Spiral is only 72% of that by flooding, 81% of Expanding Ring Search (ERS) [58], 74% of random walk, and 73% of Depth First Search (DFS) [26]. Shah et al, in [28], present a model called Mobile Ubiquitous LAN Extensions (MULE). The idea is to exploit the mobility properties of a particular environment. They propose a new element, the MULE, which is responsible to get data from sensors, buffer it and later deliver it to the sink. The MULE can be any element belonging to the environment that offers mobility and can exchange data
Survey on Data Routing in Wireless Sensor Networks
41
in the network. Examples of such elements can be a taxi cab when the environment is a populated city, or a wild animal when the environment is a savannah. This system requires that each sensor has a buffer to store each generated data. If the buffer is full then new data is dropped. The buffer is cleaned after all data was transferred to MULE when it is in a sensor’s range. The amount of data that can be transferred on a MULE is a random variable and depends on factor such as the amount of time the MULE is in the communication range of the sensor. Anastasi et al. [6] proposed the Adaptive Data Transfer (ADT) protocol that reduces significantly the time required by a sensor to transfer its messages to the mule, performing remarkably close to the optimal case. In ADT protocol the sensor tries to guess the time instant when the mule will be at the minimum distance from it. Then, it transmits its data around the expected minimum-distance point. The authors assume a random mule arrival times, and no control on the mule’s motion. The sensor works with low duty cycle to save energy and upon receiving a beacon from the mule it switches to the fully operational mode. The sensor estimates the time needed to transfer all the messages in the buffer. After this, data is transferred to the mule. The sensor node transmits back to back a number of messages equal to the window size (W_SIZE), and then it stops waiting for an ack from the mule. If the ack is not received within a pre-defined timeout, the sensor increases missed_acks counter and retransmits all messages. After ACK_MAX consecutive missed acks, the sensor assumes that the mule has exited the communication range and stops the data transfer. If the ack is received on time the missed_acks counter is reset. The authors used simulation results to show that ADT not only reduces significantly the average data-transfer time, but also provides quasi-optimal performance. In addition, it is able to react quickly to variations in the external conditions and adapt to new conditions in a limited time. There are several projects in mobility that address data-centric routing in application domains. We describe a few such projects next. Arbabi et al [42] proposed using vehicular ad-hoc networks (VANETs) to measure and report various traffic statistics. They used a few pieces of road side infrastructure to gather the information and report it to local traffic management centers. Each used piece is able to collect data at various points on roads with little or no extra cost for maintenance, replacement, calibration, or infrastructure that they called task organizers (TO). To collect data each TO broadcasts a message containing location, destination and information about required data. Each vehicle passing through TO receives the data. Once the vehicle passes by the destination, the message is delivered and the vehicle sends a message back to the TO containing the VOL tag and its current speed. Once several messages have been received by the TO, it can calculate several statistics about traffic volume. Through simulation, the authors show that with 100% penetration rate, their methods can collect precise traffic data and that even with low penetration rates or low density traffic, we can collect high-quality estimates of travel times, time mean speeds, and space mean speeds.
42
J. Cecílio, J. Costa, and P. Furtado
Others two projects are Scalable Medical Alert and Response Technology (SMART) [56] and CodeBlue [19]. The goal of SMART [56] is to monitor any patient off-line, and when the patient returns to the health care center, the gathered data is transferred from the sensor to the centers data repository. CodeBlue [19] is a project that addresses data collection with mobility, and was developed to monitor patient’s health and track their location. Some of their physiological data may be monitored and when there is a problem an alert may be triggered. The goal of this project was to monitor patient’s health in hospitals to guarantee an efficient allocation of hospital resources.
8 Conclusions and Open Issues In this chapter, we have summarized recent research results on data routing in sensor networks and classified the approaches into four main categories, namely data-centric, hierarchical, location-based and QoS aware and we have discussed the effect of node placement strategies on the operation and performance of WSNs. Protocols which name the data and query the nodes based on some attributes of the data are categorized as data-centric. Many of the researchers follow this paradigm in order to avoid the overhead of forming clusters, the use of specialized nodes. On the other hand we present the cluster-based routing protocols, which group sensor nodes to efficiently relay the sensed data to the sink. The cluster heads are sometimes chosen as specialized nodes that are less energy-constrained. A cluster-head performs aggregation of data and sends it to the sink on behalf of the nodes within its cluster. Performance-controlled planned networks, where placement and routing must be intertwined and everything from delays to throughput to energy requirements is well-defined and relevant, is an interesting subject of current and future research. Real-time, deadline guarantees and their relationship with routing, mac-layer, duty-cycles and other protocol stack issues are interesting issues that would benefit from further research. Another interesting issue for data and goal oriented routing protocols is the consideration of node mobility. Most of the current protocols assume that the sensor nodes and the sink are stationary. However, there might be situations such as medical environments where the sink and possibly the sensors need to be mobile. In such cases, the frequent update of the position of nodes and the propagation of that information through the network may excessively drain the energy of nodes. There already exist some works addressing the mobility, but new data and goal-oriented routing algorithms are needed in order to handle the overhead of mobility and topology changes in such energy constrained environment. Other possible research consists on the integration of sensor networks with wired networks in industry. Nowadays, most of sensor networks in industry are made with wired cable. To expand these existing sensor networks we can use wireless sensors but it must be integrated into the existing wired network seemlessly.
Survey on Data Routing in Wireless Sensor Networks
43
Acknowledgement The research leading to these results has received funding from the European Community's Seventh Framework Programme (FP7/2007-2013) under grant agreement n° 224282.
References [1] Manjeshwar, A., Agarwal, D.P.: APTEEN: A Hybrid Protocol for Efficient Routing and Comprehensive Information Retrieval in Wireless Sensor Networks. In: Proc. Int’l. Parallel and Distrib. Proc. Symp., pp. 195–202 [2] Manjeshwar, A., Agarwal, D.P.: TEEN: a Routing Protocol for Enhanced Efficiency in Wireless Sensor Networks. In: 1st Int’l. Wksp. on Parallel and Distrib. Comp. Issues in Wireless Networks and Mobile Comp. (April 2001) [3] Mainwaring, A., Polastre, J.: Wireless sensor networks for habitat Monitoring. In: Proc. International workshop on WSNs and applications, Atlanta, Ga, USA (September 2002) [4] Azad, A.P., Chockalingam, A.: Mobile Base Stations Placement and Energy Aware Routing in Wireless Sensor Networks. In: WCNC 2006 proceedings, vol. 1, pp. 264– 269 (2006) [5] Savvides, A., Han, C.-C., Srivastava, M.: Dynamic Fine-Grained Localization in AdHoc Networks of Sensors. In: Proc. 7th ACM MobiCom, July 2001, pp. 166–179 (2001) [6] Anastasi, G., Conti, M., Monaldi, E., Passarella, A.: An Adaptive Data-transfer Protocol for Sensor Networks with Data Mules. In: Proc. of IEEE International Symposium on a World of Wireless, Mobile and Multimedia Networks, pp. 1–8 (2007) [7] Krishnamachari, B., Estrin, D., Wicker, S.: Modeling Data Centric Routing in Wireless Sensor Networks. In: Proceedings of IEEE INFOCOM, New York, NY (June 2002) [8] Chiang, C.C.: Routing in Clustered Multihop, Mobile Wireless Networks with Fading Channel. In: Proc. of IEEE Singapore International Conference on Networks, April 1997, pp. 197–211 (1997) [9] Kidd, C.D., Orr, R.J., et al.: The aware home: A living Laboratory for ubiquitous computing research. In: Streitz, N.A., Hartkopf, V. (eds.) CoBuild 1999. LNCS, vol. 1670. Springer, Heidelberg (1999) [10] Perkins, C.E., Royer, E.M.: Ad hoc on-demand distance vector routing. In: Proc. of IEEE Workshop on Mobile Computing Systems and Applications (February 1999) [11] Perkins, C.E., Bhagwat, P.: Highly dynamic Destination-Sequenced Distance-Vector routing (DSDV) for mobile computers. In: Proceedings of the conference on Communications architectures, protocols and applications, pp. 234–244 (1994) [12] Chipara, O., He, Z., Xing, G., Chen, Q., Wang, X., Lu, C., Stankovic, J., Abdelzaher, T.: Real-time Power-Aware Routing in Sensor Networks. In: 14th IEEE International Workshop on Quality of Service, June 2006, pp. 83–92 (2006) [13] Intanagonwiwat, C., Govindan, R., Estrin, D.: Directed diffusion: A scalable and robust communication paradigm for sensor networks. In: The Proceedings of the 6th Annual ACM/IEEE International Conference on Mobile Computing and Networking (MobiCom 2000), Boston, MA (August 2000)
44
J. Cecílio, J. Costa, and P. Furtado
[14] Ma, C., Yang, Y.: Battery-aware routing in wireless ad hoc networks – part I: energy model. In: Proc. 19th ITC-19, pp. 293–302 (2005) [15] Schurgers, C., Srivastava, M.B.: Energy efficient routing in wireless sensor networks. In: The MILCOM Proceedings on Communications for Network-Centric Operations: Creating the Information Force, McLean, VA (2001) [16] Westphal, C.: Opportunistic Routing in Dynamic Ad Hoc Networks: the OPRAH Protocol. In: 2006 IEEE International Conference on Mobile Ad-hoc and Sensor Systems (MASS), October 2006, pp. 570–573 (2006) [17] Braginsky, D., Estrin, D.: Rumor Routing Algorithm for Sensor Networks. In: The Proceedings of the First Workshop on Sensor Networks and Applications (WSNA), Atlanta, GA (October 2002) [18] Ganesan, D., et al.: Highly Resilient, Energy Efficient Multipath Routing in wireless Sensor Networks. In: Mobile Computing and Communications Review (MC2R), vol. 1(2) (2002) [19] Malan, D., FulfordJones, T., Welsh, M., Moulton, S.: CodeBlue: An Ad Hoc Sensor Network Infrastructure for Emergency Medical Care. In: International Workshop on Wearable and Implantable Body Sensor Networks (2004) [20] Ergen, S.C., Varaiya, P.: Energy Efficient Routing with Delay Guarantee for Sensor Networks. ACM Wireless Networks 13, 679–690 (2007) [21] Ye, F., et al.: A Scalable Solution to Minimum Cost Forwarding in Large Sensor Networks. In: Proc. 10th Int’l. Conf. Comp. Commun. and Networks, pp. 304–309 (2001) [22] Baldus, H., Klabunde, K., Musch, G.: Reliable Set-Up of Medical Body-Sensor Network. In: Karl, H., Wolisz, A., Willig, A. (eds.) EWSN 2004. LNCS, vol. 2920, pp. 353–363. Springer, Heidelberg (2004) [23] Hassanein, H., Luo, J.: Reliable Energy Aware Routing in Wireless Sensor Networks. In: Second IEEE Workshop on Dependability and Security in Sensor Networks and Systems, pp. 54–64 (2006) [24] Huang, H., Hartman, J.H., Hurst, T.N.: Data-Centric Routing in Sensor Networks using Biased Walk. In: Annual IEEE Communications Society on Sensor and Ad Hoc Communications and Networks, pp. 1–9 (2006) [25] Akyildiz, I., Su, W., Sankarasubramaniam, Y., Cayirci, E.: A survey on sensor networks. IEEE Communications Magazine 40(8), 102–114 (2002) [26] Clarke, I., Sandberg, O., Wiley, B., Hong, T.W.: Freenet: A distributed anonymous information storage and retrieval system. LNCS (2001) [27] Chen, J., Lin, R., Li, Y., Sun, Y.: LQER: A Link Quality Based Routing for Wireless Sensor Networks. Sensors 8, 1025–1038 (2008) [28] Jain, S., Shah, R., Brunette, W., Roy, S.: Exploiting Mobility for Energy Efficient Data Collection in Sensor Networks. In: Modeling and Optimization in Mobile Ad Hoc and Wireless Networks, WiOpt (2004) [29] Wu, J., Havinga, P.: Reliable Cost-based Data-centric Routing Protocol for Wireless Sensor Networks. In: Seventh ACIS International Conference on Software Engineering, Artificial Intelligence, Networking, and Parallel/Distributed Computing, pp. 267–272 (2006) [30] Liu, J.: On a Self-Organizing Multipath Routing Protocol in Mobile Wireless Networks. Journal of Network and Systems Management, 103–126 (2006) [31] Sohrabi, K., Gao, J., Ailawadhi, V., Potie, G.J.: Protocols for self-organization of a wireless sensor network. In: IEEE Personal Communications, October 2000, pp. 16– 27 (2000)
Survey on Data Routing in Wireless Sensor Networks
45
[32] Akkaya, K., Younis, M.: A Survey of Routing Protocols in Wireless Sensor Networks. The Elsevier Ad Hoc Network Journal 3/3, 325–349 (2005) [33] Li, L., Halpern, J.Y.: Minimum-Energy Mobile Wireless Networks Revisited. In: IEEE ICC 2001, vol. 1, pp. 278–283 (2001) [34] Schwiebert, L., Gupta, S.K.S., Weinmann, J.: Reserach challenges in wirelesss networks of biomedical sensors. In: Proc. 7th ACM International Conference on Mobile Computing and Networking (MobiCom 2001), Rome, Italy, July 2001, pp. 151–165 (2001) [35] Subramanian, L., Katz, R.H.: An Architecture for Building Self Configurable Systems. In: Proc. IEEE/ACM Wksp. Mobile Ad Hoc Net. and Comp., Boston, MA (August 2000) [36] Cheng, L., Qian, D., Wu, W.: An Energy Efficient Weight-Clustering Algorithm in Wireless Sensor Networks. In: FCST 2008, pp. 30–35 (2008) [37] Chu, M., Haussecker, H., Zhao, F.: Scalable Information Driven Sensor Querying and Routing for Ad Hoc Heterogeneous Sensor Networks. Int’l. J. High Perf. Comp. Apps. 16(3) (August 2002) [38] Mauve, M., Widmer, J., Hartenstein, H.: A Survey on Position-Based Routing in Mobile Ad-Hoc Networks. IEEE Network 15 (2001) [39] Transier, M., Füßler, H., Widmer, J., Mauve, M., Effelsberg, W.: Scalable PositionBased Multicast for Mobile Ad-hoc Networks. In: First International Workshop on Broadband Wireless Multimedia: Algorithms, Architectures and Applications (BroadWim 2004), San Jose, CA, USA (2004) [40] Youssef, M., Younis, M., Arisha, K.: A constrained shortest-path energy-aware routing algorithm for wireless sensor networks. In: The Proceedings of the IEEE Wireless Communication and Networks Conference (WCNC 2002), Orlando, FL (March 2002) [41] Younis, M., Youssef, M., Arisha, K.: Energy-Aware Routing in Cluster-Based Sensor Networks. In: The Proceedings of the 10th IEEE/ACM International Symposium on Modeling, Analysis and Simulation of Computer and Telecommunication Systems (MASCOTS 2002), Fort Worth, TX (October 2002) [42] Arbab, M.H., Weigle, M.: Using vehicular networks to collect common traffic data. In: Proc. of the sixth ACM international workshop on VehiculAr InterNETworking, pp. 117–118 (2009) [43] Bulusu, N., Heidemann, J., Estrin, D.: GPS-less Low Cost Out Door Localization for Very Small Devices. Tech. rep. 00729, Comp. Sci. Dept., USC (April 2000) [44] Sadagopan, N., et al.: The ACQUIRE Mechanism for Efficient Querying in Sensor Networks. In: Proc. 1st Int’l. Wksp. Sensor Network Protocol and Apps, Anchorage, AK (May 2003) [45] Bonnet, P., Gehrke, J., Seshadri, P.: Querying the physical world. IEEE Pers. Commun. 7(5), 10–15 (2000) [46] PODS, A Remote Ecological Micro-sensor Network Project website, http://www.PODS.hawaii.edu [47] Pothuri, P.K., Sarangan, V., Thomas, J.P.: Delay-constrained energy efficient routing in wireless sensor networks through topology control. In: International Conference on Networking, Sensing and Control, ICNSC 2006, pp. 35–41 (2006) [48] Haider, R., Younas Javed, M.: EAGR: Energy Aware Greedy Routing in Sensor Networks. In: Future Generation Communication and Networking (FGCN 2007), vol. 2, pp. 344–349 (2007)
46
J. Cecílio, J. Costa, and P. Furtado
[49] Shah, R., Rabaey, J.: Energy Aware Routing for Low Energy Ad Hoc Sensor Networks. In: The Proceedings of the IEEE Wireless Communications and Networking Conference (WCNC), Orlando, FL (March 2002) [50] Cui, S., Madan, R., Goldsmith, A.J., Lall, S.: Joint Routing, MAC, and Link Layer Optimization in Sensor Networks with Energy Constraints. In: ICC 2005, Korea (May 2005) (to appear) [51] Lindsey, S., Raghavendra, C.: PEGASIS: Power-Efficient Gathering in Sensor Information Systems. In: IEEE Aerospace Conf. Proc., vol. 3(9-16), pp. 1125–1130 (2002) [52] Tilak, S., Abu-Ghazaleh, N., Heinzelman, W.: A Taxonomy of Wireless MicroSensor Network Models. In: ACM Mobile Computing and Communications Review (MC2R), vol. 6(2) (2002) [53] Lee, S.-B., Kwak, K.J., Campbell, A.T.: Solicitation-based Forwarding for Sensor Networks. In: Proc. of Sensor and Ad Hoc Communications and Networks, pp. 90–99 (2006) [54] Clausen, T., Jacquet, P., Laouiti, A., Muhlethaler, P., Qayyum, A., Viennot, L.: Optimized Link State Routing Protocol for Mobile Ad Hoc Networks. In: IEEE International Multitopic Conference, Pakistan (2001) [55] He, T., Stankovic, J., Lu, C., Abdelzaher, T.: SPEED: A Stateless Protocol for RealTime Communication in Sensor Networks. In: International Conference on Distributed Computing Systems (May 2003) [56] Stair, T., Guttag, J., Greenes, R., Ohno-Machado, L.: Scalable Medical Alert Response Technology SMART [57] Rodoplu, V., Meng, T.H.: Minimum Energy Mobile Wireless Networks. IEEE JSAC 17(8), 1333–1344 (1999) [58] Heinzelman, W.B., Cheng, Z.: Searching strategy for multi-target discovery in wireless networks. In: Proc. of workshop on Applications and Services in Wireless Networks (2004) [59] Heinzelman, W., Chandrakasan, A., Balakrishnan, H.: Energy-efficient communication protocol for wireless sensor networks. In: Proceeding of the Hawaii International Conference System Sciences, Hawaii (January 2000) [60] Heinzelman, W., Kulik, J., Balakrishnan, H.: Adaptive protocols for information dissemination in wireless sensor networks. In: Proceedings of the 5th Annual ACM/IEEE International Conference on Mobile Computing and Networking (MobiCom 1999), Seattle, WA (August 1999) [61] Yu, Y., Estrin, D., Govindan, R.: Geographical and Energy-Aware Routing: A Recursive Data Dissemination Protocol for Wireless Sensor Networks. CLA Computer Science Department Technical Report, UCLA-CSD TR-01-0023 (May 2001) [62] Xu, Y., Heidemann, J., Estrin, D.: Geography-informed energy conservation for ad hoc routing. In: The Proceedings of the 7th Annual ACM/IEEE International Conference on Mobile Computing and Networking (MobiCom 2001), Rome, Italy (July 2001)
Energy-Efficient Task Scheduling and Data Aggregation Techniques in Wireless Sensor Networks for Information Explosion Era Akimitsu Kanzaki, Naoki Wakamiya, and Takahiro Hara
Abstract. This chapter presents our approaches for energy-efficient techniques in wireless sensor networks. First, we present requirements on scheduling mechanism in a wireless sensor network and introduce our approaches on energy-efficient and adaptive control mechanisms. Second, we introduce our data aggregation method that exploits the characteristics of data and communication in a wireless sensor network. Finally, we discuss our future plan to interoperate our lower layer and higher layer techniques.
1 Introduction Wireless sensor networks have become a widely-used system in recent years. Currently, various kinds of wireless sensor networks are being developed all over the world. Also, the number of sensor nodes in each wireless sensor network is rapidly increasing. Thus, wireless sensor networks are facing the ‘explosion’ of information generated by a considerable number of sensor nodes and new techniques to handle such a tremendous amount of information in wireless sensor networks. Here, in wireless sensor networks, it is common that sensor nodes have limited capabilities. Power supply, in particular, is one of the most critical constraints because sensor nodes are usually battery powered. Therefore, techniques against the information explosion must be energy-efficient for prolonging the service lifetime of a wireless sensor network. Until now, there have been many studies on energy-efficient techniques for wireless sensor networks. Lower layer techniques such as low-power circuitry and devices are fundamental to organization of an energy-efficient wireless sensor network. As we focus on higher-layer techniques such as MAC (Medium Akimitsu Kanzaki · Naoki Wakamiya · Takahiro Hara Osaka University, 1-5 Yamadaoka, Suita, Osaka, Japan e-mail: [email protected], [email protected], [email protected]
T. Hara et al. (Eds.): WSN Technologies for the Information Explosion Era, SCI 278, pp. 47–75. c Springer-Verlag Berlin Heidelberg 2010 springerlink.com
48
A. Kanzaki, N. Wakamiya, and T. Hara
Access Control), coverage/connectivity control, task scheduling, routing, data gathering/diffusion, and data aggregation, they are more application-oriented and their effectiveness heavily depends on an application to be adopted. Therefore, we need to carefully combine multiple techniques belonging to different control layers to satisfy application’s requirements. In this chapter, among feasible applications of wireless sensor networks, we consider a monitoring application, such as surveillance and environmental monitoring, which is one of the most promising applications, and discuss two major techniques, i.e. task scheduling and data aggregation, which contribute much to energy efficiency. We also give a brief idea on combination of these techniques. In a monitoring application, it is assumed that an enormous number of sensor nodes are randomly or regularly deployed in a target region. An application requires a wireless sensor network to periodically provide information on the whole target region, e.g. the distribution of temperature, or condition of points of interest in the region, and provide the detailed information on the area where an event, such as monitored temperature exceeding a predetermined threshold, occurs. Taking into account limited computational, memory, and communication capabilities of wireless sensor network and poor power supply, it is inefficient and resourceconsuming to gather data from all sensor nodes deployed. In particular, when the number of sensor nodes is too large against the network bandwidth, an attempt to collect all data for finer grain monitoring often results in loss of data for congestion and as a consequent degrade of QoS (Quality of Service) that an application perceives. Some techniques such as congestion control, error correction, and QoS control are helpful as far as the amount of traffic is within the effective bandwidth of a wireless sensor network, we need additional application-oriented techniques to reduce the traffic without violating application’s requirements. Furthermore, considering the size of a wireless sensor network and the number of sensor nodes, a centralized control which optimizes parameters and behaviors based on the global information obtained from sensor nodes is not realistic and not robust against the instability of wireless communication and the unreliability of sensor nodes. For this purpose, we present two essential techniques, i.e. task scheduling and data aggregation in this chapter. Task scheduling is to control the number and location of sensor nodes which are responsible for monitoring and reporting. Although preceding work on coverage and connectivity control can also appropriately schedule tasks among nodes, they mainly consider the stable and homogeneous environment and as such they cannot adapt to dynamic changes in monitoring frequency caused by a local event. With our task scheduling technique, each sensor node dynamically and autonomously adjusts its sensing and communication frequency in accordance with the condition of surroundings. Our task scheduling technique adjusts the number of sensor nodes to obtain data, our data aggregation technique further reduces the amount of data to transmit by exploiting properties of data and wireless communication. It is widely known that data obtained by sensor nodes has spatial correlation. For an application to monitor the distribution of temperature of the region, some sensor nodes can suppress transmission if their data can be complemented by data of nearby nodes. With our data aggregation technique, each sensor
Energy-Efficient Task Scheduling and Data Aggregation Techniques
49
node which engages in monitoring dynamically and autonomously decides whether to transmit data or not in accordance with the spatial correlation of its data to those emitted by neighbor nodes. We show experimental results to verify the effectiveness of our techniques by using different scenarios which characterize each technique the most. Then we further show how to incorporate these two techniques in an integrated manner for energy-efficient monitoring in wireless sensor networks. The reminder of this chapter is organized as follows. In Section 2, we introduce conventional techniques to achieve energy-efficiency in wireless sensor networks. In Sections 3. and 4, we describe our task scheduling and data aggregation techniques in detail. Then in Section 5, integrated schemes are introduced. Finally, we summarize this chapter and outlines the future direction of our study in Section 6.
2 Related Work In this section, we introduce some typical existing energy-efficient scheduling and data aggregation techniques. Also, we make clear the differences between these conventional techniques and our approaches.
2.1 Scheduling Techniques Scheduling plays a fundamental role in (i) effective utilization of limited wireless network capacity leading to lower delay and higher throughput in message transmission[21, 25], (ii) efficient utilization of limited battery power to prolong the lifetime of a wireless sensor network while keeping the coverage and connectivity at the desired level[9, 15, 31], and (iii) fair share of sensing and transmission tasks among sensor nodes to balance the energy consumption[8, 20, 32]. If the interval of sensing and data gathering is fixed, general sleep scheduling to wake up nodes at the regular intervals is effective in saving energy consumption and reducing data gathering delay. However, in some classes of applications, a node needs to change its sensing frequency to monitor a region or a target more frequently when it detects unusual situations and phenomena. Furthermore, the number of nodes that monitor and report the phenomena should be regulated in accordance with its criticality and importance. For example, in a case of environment monitoring applications, such as surveillance of a landslide and volcanic activity, it is not necessary to obtain and report sensor data per minute in a usual situation. Instead, once soils and stones begin to move, the surface temperature increases, or volcanic gasses blow out, nodes detecting the event must monitor and report the up-to-date status with a higher frequency as illustrated in Fig. 1. Object detection and tracking is another typical example, where a node having a moving target in its sensing region sends out reports more frequently than other nodes. We should note here that even in an unusual situation, not all sensor nodes must be involved in frequent monitoring and reporting. The number and location of sensor nodes that perform a frequent operation should be determined by taking into account the importance of the event in terms of energy and bandwidth preservation. In addition, nodes on a
50
A. Kanzaki, N. Wakamiya, and T. Hara
usual situation • hourly monitoring • from all nodes
unusual situation • some or all nodes perform frequent monitoring and reporting • number and location of per-minute monitoring nodes are adapted
Fig. 1 Adaptive sensing in environmental monitoring
path from those nodes to a receiver must frequently transmit the data. Sensor data should be transmitted to the receiver without introducing a delay at relaying nodes to wait for their next-hop nodes to wake up. Although there have been many studies on general scheduling, only a few of them address scheduling algorithms and mechanisms adaptive to dynamically changing sensing requirements. In [1], the authors proposed an algorithm to adapt the sampling frequency of a sensor by estimating the maximum frequency in the power spectrum of the target signal to satisfy the Nyquist frequency. However, they only consider adaptation at a single node or a single sensor device and how a wireless sensor network should be scheduled as a whole is out of the scope of the paper. Furthermore, the algorithm is applicable only to monitoring of targets with a slow variation over time. In [3], the authors proposed a method to adapt the density of active nodes in accordance with the variation in sensor data. By keeping more sensor nodes awake in the region showing a high variation and letting more nodes sleep in the region with a low variation, energy consumption is reduced while minimizing the impact of sleeping nodes. However, the decision is made in a centralized manner at a so-called parent node, which collects sensor data from child nodes, estimates the variation, and decides whether each child node can sleep or not. It means that independently of the current situation, parent nodes need to operate with the fastest operational frequency. None of the above two approaches and others can satisfy application’s requirements on adaptive sensing. In Section 3, we will show our solution, self-adaptive and self-organizing scheduling mechanisms.
Energy-Efficient Task Scheduling and Data Aggregation Techniques
51
2.2 Data Aggregation Techniques In wireless sensor networks, wireless communication generally consumes larger energy than other operations such as sensing and data processing. Thus, traffic reduction is one of the most important issues for reducing energy consumption. Here, in wireless sensor networks, it is common that sensor data acquired by nodes (readings) have spatial and temporal correlations. Considering such correlations, many data aggregation methods have been proposed to reduce data traffic. For example, several methods such as Distributed regression[14] and TREG (Tree-based REGression)[4, 5] reduce data traffic by using the regression approximation. These methods approximate data distribution in a certain area by a regression techniques using the readings of nodes in the area. By employing regression approximation, these methods reduce the amount of data transferred to the sink node. However, due to the characteristics of approximation, these method cannot guarantee that the differences between the results of approximation and the actual readings are smaller than the predetermined value. In addition, Distributed regression[14] needs the exchange of readings between nodes in order for nodes to share information for approximation. This may result in extra energy consumption. There are other methods that use statistical models to estimate the readings of nodes, such as BBQ[12] and Ken[10]. These methods can drastically reduce data traffic after constructing a statical model. This is because a statical model basically requires few readings to obtain the data distribution in the target area. However, the cost for constructing a statical model becomes extremely large because a huge number of data exchanges are needed. In addition, similar to the regression-based methods[4, 5, 14], these methods cannot control the maximum error between the obtained results and the actual readings because the results using a statical model become probabilistic. As another kind of model-based approaches, in Snapshot queries[18], several representative nodes elected from the entire network estimate the readings of other nodes. This method can also reduce data traffic since only the representative nodes have to communicate with the sink node. However, similar to other model-based methods, this method needs extra traffic for constructing models. Some methods focus on a temporal correlation of readings of a node. For example, PAQ (Probabilistic Adaptable Query system)[28, 29] constructs a temporal model of readings for each node. By using this model, the sink node can estimate the reading of a node based on the past readings of the node. Thus, no packet transmission occurs between a node and the sink node unless the constructed model becomes invalid due to the deterioration in accuracy. However, similar to the other methods described in this section, the results obtained by these methods are probabilistic. Thus, it becomes difficult to control the maximum error between the obtained results and the actual readings.
3 Self-adaptive and Self-organizing Scheduling Mechanisms In this section, we first briefly summarize requirements and assumptions on scheduling in a wireless sensor network consisting of a vast number of sensor devices,
52
A. Kanzaki, N. Wakamiya, and T. Hara
i.e. a source of information explosion. Then, we introduce our approaches on energy-efficient and adaptive control where algorithms and mechanisms are fully distributed and self-organizing to achieve high scalability and robustness.
3.1 Requirements on Adaptive Task Allocation in Wireless Sensor Networks In a densely deployed wireless sensor network, we need to assign and schedule sensing and transmission tasks among nodes to prolong the lifetime of the network, shorten message transmission delays, and avoid collisions among message transmission. In general, for efficient data gathering in periodic monitoring, nodes are scheduled to wake up in accordance with sensing intervals. Nodes further from the sink node wake up earlier than those closer to the sink node. When a node wakes up, it first receives sensing data from neighbors further from the sink node, next obtains sensing data around itself, and finally transmits aggregated data to a neighbor closer to the sink node, which wakes up at this timing to receive the message, that is, wakeup periods are staggered[2]. We would see concentric traveling waves of message propagation from the edges to the sink node (see Fig. 2). When applied to periodic monitoring, such scheduling leads to longer lifetime of a wireless sensor network. A scheduling mechanism needs to rotate sensing tasks among nodes and let redundant nodes sleep, so that they equally consume their limited energy resource while keeping the target region monitored at a certain degree. At the same time, a wireless sensor network is often requested to monitor a target more precisely and frequently once an emergency or unusual event happens. In such a case, the number of nodes that actively monitor the target and their sensing interval must be adaptively controlled in accordance with the application-dependent requirements. Especially when nodes are equipped with multiple sensing devices, e.g. temperature, light, and gas concentration, and monitor conditions of corresponding sensing targets, such adaptive scheduling becomes nontrivial. To avoid introduction of points of failure and the waste of time and bandwidth to collect the global information and A Zzz B Zzz D
E
Zzz
C Zzz
F
Zzz
G
Zzz
Zzz
Zzz
transmission range
from further nodes A
A
A
B,C
B,C
B,C
wake-up period D, E, F, G
messages
time
D, E, F, G
time
D, E, F, G
reception transmission period period
Fig. 2 Traveling wave-based data gathering
time sleep period
Energy-Efficient Task Scheduling and Data Aggregation Techniques
53
disseminate the schedule, the decision on whether to perform frequent sensing must be made by a node itself based on locally available information, i.e. conditions of monitored targets and the status of neighbor nodes. Throughout this section, our focus is on fully-distributed and self-organizing control. In all of our proposed mechanisms, sensor nodes operate on locally available information, which is obtained through sensing or message exchanges with neighbor nodes. There should not be centralized or optimized control.
3.2 General Description of Adaptive Task Scheduling Mechanisms Biological systems are known to show a self-organizing behavior, where the globally organized pattern of control emerges from mutual interaction among small entities which follow a set of simple rules based on locally available information. Typical examples is swarm intelligence, i.e. collective intelligence found in a group of social insects such as ants, termites, and honey bees[7]. In the field of Mathematical Ecology, such a self-organizing behavior in biological systems is modeled by nonlinear evolution equations. By adopting bio-inspired mathematical models, we can establish fully distributed and self-organizing network control. To accomplish task scheduling in a self-adaptive and self-organizing manner, we adopt biologically-inspired nonlinear mathematical models, i.e. a pulse-coupled oscillator model and a response threshold model[27]. As will be explained in more details, our proposal is fully distributed and selforganizing. Each node relies only on locally available information obtained through observation of its vicinity and message exchanges with direct neighbors. Because of such autonomous and distributed control, our proposal does not suffer from the fragility introduced in other proposals adopting a centralized or master-slave based control. Our proposal can easily adapt to dynamic changes in network caused by halt and movement of nodes without involving nodes further from a changing point by, for example, sending notification of change to initiate rescheduling as in other proposals. More importantly, it can adapt to dynamically changing sensing requirement through direct observation of the changing condition. Neither reporting to a central server nor negotiation with other nodes is required for decision. In comparison to centralized and optimized control, the number of nodes involved in sensing and reporting task could be larger in our proposal. However, it can provide better adaptability, robustness, and energy-efficiency than other proposals and we believe such characteristics of control mechanism is the most important in information explosion era where a tremendous number of nodes is distributed and embedded in the environment and no centralized control is possible due to the size, heterogeneity, and dynamic changes in a network system. Node i has two sensing states, the normal state and the frequent state for each sensor k (1 ≤ k ≤ kmax ), and they are denoted as xi,k = 0 and xi,k = 1, respectively. kmax is the number of sensing devices a node equipped with. Basically, node i is in the normal state on all sensors. It periodically wakes up, receives messages from
54
A. Kanzaki, N. Wakamiya, and T. Hara
neighbors, monitors sensing targets, and transmits a message containing sensor data at regular intervals of T seconds. We assume that the sensing interval in usual situations is the same among all sensors of different types. So that a message emitted by a node can be received by all neighbors, a node uses broadcasting in message emission. All nodes which are awake and within a communication range receive a message simultaneously. Nodes operate on traveling wave-based periodic data gathering at regular sensing intervals. We organize traveling waves in a self-organizing fashion by a pulsecoupled oscillator model, which explains the biological synchronization observed in a group of flashing fireflies[13, 23]. When conditions of a sensing target, such as temperature and gas concentration, change at a certain point in the monitored region, surrounding nodes detecting the change decide whether to monitor the target more frequently or not. To regulate the number of nodes engaged in frequent monitoring of the target in a self-organizing way, we adopt a response threshold model[6], i.e. a mathematical model of division of labor and task allocation in a group of social insects. Once a node considers to monitor the target k more frequently, it decreases its sensing interval to Tk = T /2ck . It wakes up at regular intervals of Tk . Note that in this section, we assume that a target corresponds to an environmental factor that a particular type of sensor (e.g., temperature) can sense. Therefore, we use symbol k for representing both a type of sensor and a target. Here, ck (0 ≤ ck ≤ [log2 (T /2τmax )]) is an integer value predetermined for sensing target k. So that the obtained sensor data are forwarded to the sink node at a higher frequency, nodes on the path to the sink node also adopt the same interval Tk . As a result, one or more traveling waves emerge in the wireless sensor network.
3.3 Adaptive Sensing Using Response Threshold Model To decide the sensing state for sensing target k at node i in a fully distributed and self-organizing manner, we adopt the response threshold model of division of labor and task allocation in social insects[6], which has also been applied to wireless sensor network control[19, 20]. In the response threshold model, a task has the demand, which evolves over time. As the number of agents engaged in the task increases, the demand decreases. An agent defines the threshold, which corresponds to hesitation in doing a task. The threshold also evolves over time according to whether an agent performs the task or not. Now, we consider demand si,k of sensing target k at node i. Following the response threshold model, the demand si,k changes as, mi,k (t) dsi,k = δi,k (t) − α , dt ni,k (t)
(1)
where δi,k (t) corresponds to the rate of increase of demand and α determines the work efficiency. mi,k (t) and ni,k (t) represent the number of neighbor nodes performing frequent sensing on sensor k at time t and the total number of neighbor nodes
Energy-Efficient Task Scheduling and Data Aggregation Techniques
55
with sensor k, respectively. In the equation, when the increase of demand is equivalent to the work output defined by the second term, the demand does not change. If the number of workers is insufficient, the demand increases. The definition of rate δi,k depends on application requirements. Let us assume that the surface temperature changes slowly and once it increases it stays high for a long period of time. In such a case, nodes are required to monitor temperature more frequently while temperature is increasing or decreasing, whereas the sensing interval can be long under a stable condition. It means that δi,k for temperature monitoring can be determined in accordance with the rate of temperature change. On the other hand, gas suddenly appears and the existence of gas is hazardous. Therefore, the absolute value is more important for gas sensors and δi,k should be defined based on the concentration. The probabilities that node i begins to work, i.e. wake up and monitor the sensing target more frequently and that node i stops frequent sensing are given by the following equations, respectively. P(xi,k = 0 → xi,k = 1) =
s2i,k (t) 2 (t) s2i,k (t) + θi,k
P(xi,k = 1 → xi,k = 0) = pi,k (t),
(2) (3)
where pi,k (t) ∈ [0, 1] is a parameter. For example, by using an inverse of changing rate of temperature as pi,k (t), we can expect that nodes immediately resume the normal sensing interval once the temperature becomes stable. The readiness of node to get engaged in a frequent sensing task depends on the threshold θi,k . The threshold is adjusted depending on whether a node performs frequent sensing or not. d θi,k −ξ , if i performs frequent sensing = , (4) ϕ , otherwise dt where ξ and ϕ are parameters. The threshold adjustment makes specialists having a small threshold and keep sensing the target frequently. In addition, the threshold adjustment makes the response threshold model insensitive to parameter settings, such as the range of demand si,k and α .
3.4 Adaptive Data Gathering Using Pulse-Coupled Oscillator Model To autonomously generate and maintain traveling waves of message propagation, we adopt a pulse-coupled oscillator model, which explains biological synchronization observed in a group of flashing fireflies [13]. In a pulse-coupled oscillator model, oscillators having different intrinsic frequencies eventually reach the global synchronization or phase-lock condition through mutual interactions.
56
A. Kanzaki, N. Wakamiya, and T. Hara
A pulse-coupled oscillator model has been applied to a variety of wireless sensor network control such as clock and timer synchronization [24] and scheduling [30].
Table 1 Information maintained at node i Parameter
Content
kmax φi T li Δi (φi ) τi Fi Si Θi Xi Yi Di
number of sensor devices phase of timer (φi ∈ [0, T ], d φi /dt = 1) normal sensing interval level value, i.e. number of hops from the sink node PRC (phase response curve) offset (0 < τmin ≤ τi ≤ τmax < 0.5T ) relay flag vector = { fi,k |1 ≤ k ≤ kmax } demand vector = {si,k |1 ≤ k ≤ kmax } threshold vector = {θi,k |1 ≤ k ≤ kmax } sensing state vector = {xi,k |1 ≤ k ≤ kmax } sensing state table = {∀ jX j } sensor data = {Di,k |1 ≤ k ≤ kmax }
In our mechanism, node i maintains a timer whose phase φi moves from 0 to T (d φi /dt = 1). T is the normal sensing interval. Node i also maintains level value li , PRC (phase-response curve) function Δi (φi ), offset τi (0 < τmin ≤ τi ≤ τmax < 0.5T ), and relay flag vector Fi = { fi,k |1 ≤ k ≤ kmax } for traveling wave-based data gathering. In addition, it also maintains demand vector Si = {si,k |1 ≤ k ≤ kmax }, threshold vector Θi = {θi,k |1 ≤ k ≤ kmax }, sensing state vector Xi = {xi,k |1 ≤ k ≤ kmax }, and sensing state table Yi = {∀ jX j } containing sensing state vectors of all neighbor nodes for adaptive sensing. Node i has sensor data Di = {Di,k |1 ≤ k ≤ kmax }, which are obtained by its own sensors and from neighbors. They are summarized in Table 1. Entries of vectors are updated based on the control information embedded in a received message from a neighbor node, which is emitted at the interval of sensing. It implies that there is no additional message transmission for control. Level value li indicates the number of hops from the sink node. Initially, the level value is set at infinity. A mechanism to set the level value appropriately at each node will be explained later. The sink node uses a level value of zero. The offset τi defines the interval of message emission between a node of level l − 1 and that of level l. Offset τi is chosen randomly to avoid synchronous message emissions among nodes of the same level. Details of timing issues will be explained later. The maximum offset τmax is determined taking into account the density of nodes. The PRC function determines the amount of phase shift on receiving a message from a neighbor. In biological synchronization, a firefly is considered to have a biological timer. When it is alone, a firefly periodically flashes based on its intrinsic frequency. When they form a group, a non-flashing firefly is stimulated by a flash of another firefly and shifts the phase of its timer by a small amount. If the phase
Energy-Efficient Task Scheduling and Data Aggregation Techniques
57
reaches a certain threshold, the stimulated firefly also flashes. At this time, both of the stimulating firefly and the stimulated firefly are synchronized each other in flashing. Eventually, all fireflies in a group begin to flash simultaneously, which is called global synchronization, or flash alternately keeping a fixed phase difference, which is called phase-lock or traveling wave. Emergence of global synchronization or traveling wave depends on the initial phase of oscillators and the form of PRC function. To generate concentric traveling waves of message propagation regardless of the initial phase condition in a wireless sensor network, we use the following PRC function for all nodes[26].
Δi (φi ) = a sin
π φi + b(τi − φi ), τi
(5)
where a and b are parameters which determine the speed of convergence. Relay flag fi,k is used to notify downstream nodes, i.e. nodes closer to the sink node, of the existence of upstream nodes, i.e. nodes further from the sink node, in the frequent state. When sensor node i relays a message received from an upstream node operating in the frequent state, relay flag fi,k is set as one, otherwise, fi,k is set as zero. Sensor data Di,k contains all sensor data, together with originator’s ID, generated or received during the wake-up period for sensing target k. Node i behaves in accordance with its phase φi and relay flag vector Fi . Node i wakes up when φi∗ = φi mod mink Tk becomes mink Tk − τmax , where mink Tk = T / maxk ( fi,k 2ck , 1), i.e. the minimum of Tk . On waking up, node i initializes its relay flag vector Fi to all zero, clears sensor data Di , and waits for message reception. During the reception period from φi∗ = mink Tk − τmax to 0, upstream nodes setting the same relay flag are scheduled to broadcast a message so that awake node i can receive messages. When node i receives a message from upstream node j with level value l j = li + 1, node i deposits the received sensor data D j . The broadcast message also contains relay flag vector F j , and sensing state vector X j . If any of relay flag f j,k ∈ F j is set as one in the received message, node i sets the corresponding relay flag fi,k as one. In addition, node i replaces or adds the received sensing state vector X j in its sensing state table Yi . When node i receives a message from node j with l j = li , it checks whether the received sensor data D j covers sensor data Di it has. If there are sensor data contained in the received sensor data D j , they are removed from Di . If Di,k becomes empty, node i sets relay flag fi,k as zero. As a result, the amount of sensor data in a message can be reduced and the number of sensor nodes involved in frequent relaying could be further reduced. Then, when φi∗ reaches mink Tk − ε , node i determines whether it moves to the frequent state for each of sensing targets ∀k ∈ {k|φi mod Tk = Tk − ε } or not. ε (0 < ε < τmin ) corresponds to the sensing delay. First, node i calculates ni,k and mi,k from sensing state table Yi . Next, node i derives demand vector Si using Eq. (1). Then, node i determines its sensing state vector Xi using Eq. (2) and Eq. (3). If any of sensing state xi,k is set as one, node i monitors sensing target k, deposits sensor data, and sets its relay flag fi,k as one. If φi is T − ε , i.e. timing for regular sensing,
58
A. Kanzaki, N. Wakamiya, and T. Hara determine sensing state set sensing state vector set relay flag vector obtain sensor data
wake up initialize relay flag vector clear sensor data
min k Tk − τ max node i
τ max
min k Tk − ε reception period
transmission period
0 (= min k Tk ) messages from upstream nodes deposit received sensor data update relay flag update state vector
broadcast message message from node of same level update sensing state table
phase
φi* = φi mod min k Tk
message from downstream node update level value stimulated
Fig. 3 Node behavior on timeline
node i obtains sensor data of all targets. After that, node i adjusts its threshold vector Θi using Eq. (4). When φi mod mink Tk becomes zero in 0 < φi < T , or φi reaches T , node i broadcasts a message, which is received by any of awake nodes in the range of radio communication. A message that node i emits contains node ID i, level value li , sensing state vector Xi , relay flag vector Fi , synchronization flag zi , and sensor data Di . zi is set as one only if phase φi is T on broadcasting the message. After broadcasting, the phase reaching T goes back to zero and node i keeps awake for τmax to receive a message from any of downstream nodes. When node i receives a message having synchronization flag z j of one from node j with l j < li , it performs the PCO-based synchronization. First it sets its level value li at l j + 1 and shifts its phase by an amount Δi (φi ) in Eq. (5). The phase shift is done only once during the τmax awake period to avoid being stimulated multiple times. When node i receives a message from node j with l j = li during this period, it only updates sensing state table Yi . After τmax has passed since broadcasting, node i goes to sleep. Therefore, a node is awake for the duration of 2τmax in one data gathering interval mink Tk , i.e., the duty cycle becomes 2τmax / mink Tk . The above node behavior is summarized in Fig. 3.
3.5 Evaluation In this section, we show the results of experiments that verify the effectiveness of our mechanisms. In order to evaluate the robustness and adaptiveness of our mechanisms, we assume an environment where sudden changes in sensing targets occur. In the experiments, we assume a wireless sensor network of 200 nodes randomly distributed in a 100 m × 100 m region. The region is divided into four 50 m × 50 m areas and named as A, B, C, and D, from the top left area clockwise. The sink node is located in the center of the region. Both sensing and communication ranges are fixed at 20 m. We assume two sensing targets, i.e. temperature and gas. The normal data gathering interval T is set as 160 seconds. For unusual conditions, we set ctemp =2 and cgas =4, and therefore, Ttemp =40 seconds and Tgas =10 seconds, respectively. In
Energy-Efficient Task Scheduling and Data Aggregation Techniques
59
the evaluation, we consider a scenario where two different types of adaptive task scheduling is required. When temperature changes, a point of change is stable, a change happens gradually, and frequent sensing is required while temperature is changing. On the other hand, a point of gas leakage keeps moving, a change happens at once, and frequent sensing is required while gas is there. By using the scenario, how our proposal effectively adapts to different type of change can be shown. We also compare our proposal to a case without adaptive task scheduling in terms of duty cycle and energy consumption. We use δi,temp = β |dvi,temp /dt| and δi,gas = vi,gas in Eq. (1), and pi,temp = 1 − β |dvi,temp /dt| and pi,gas = 1 − vi,gas in Eq. (3) for all nodes. vi,temp ∈ [0, 1] and vi,gas ∈ [0, 1] are temperature and gas concentration normalized by their maximum value, respectively. We empirically use β = 1, 000. A node consumes 52.2 mW for transmission, 59.1 mW for reception, 60 μ W in the idle mode, and 3 μ W in the sleep mode[22]. τmax and τmin is set as 1 second and 0.5 second, respectively. We assume that control information in a message amounts to 24 bits and one sensor data amounts to 16 bits. We use a=0.01 and b=0.5 for PRC in Eq. (5) and the parameters of the response threshold model are set to α = 1, ξ =0.01, and ϕ =0.001 in Eq. (4). These values are typical and used in other literatures. We should note here that bio-inspired models are basically insensitive to parameter setting and it is another benefit of adopting such models. Until 500 seconds, nothing happens where temperature vtemp is 0.1 and no gas is observed. From 500 seconds, temperature begins to linearly increase at a randomly chosen location in area D. At 1,500 seconds, temperature at the location reaches 0.9 and stays high until the end of simulation. Fig. 4 shows a snapshot of the simulator where temperature increases in area B. During the temperature increase, gas of concentration vgas =0.8 leaks out from a randomly chosen location in area C at 1,000 seconds. The gas moves to area A at the speed of 0.08 m/s. At 2,000 seconds, the gas disappears. Fig. 5 shows the time instants when a node, whose ID is shown on the left y-axis, broadcasts a message. Nodes are numbered based on the location as shown on the right y-axis. At first, all nodes are in the normal state as indicated by vertical lines of broadcasting instants. At 500 seconds around the end of an awake period, some nodes in area D detect the increase in temperature and decide to monitor temperature more often. Since they set the relay flag fi,temp in broadcast messages, nodes on the path to the sink node also set their relay flag and begin to operate at the shorter intervals of Ttemp =40 seconds. This can be observed as dots between vertical lines of normal sensing in areas C and B from about 550 seconds. At 1,500 seconds, nodes find that the temperature becomes stable again, and they return to the normal state. In a similar manner, at 1,000 seconds, some nodes in area C begin to operate at the shorter intervals of Tgas = 10 seconds for detecting the gas leakage. As gas moves to area A, nodes in area C eventually go back to the normal state while nodes in area A move to the frequent state.
60
A. Kanzaki, N. Wakamiya, and T. Hara
Fig. 4 Snapshot of simulator Gas Temperature
200 Label of Sensor Node
Area D
150 Area C
100 Area B
50 Area A
0 0
500
1000
1500
2000
2500
Time [seconds]
Fig. 5 Timing of message broadcasting
During the simulation experiments, the total energy consumption is 57.6 mJ per node and duty cycle is 0.03 per node. On the other hand, when we adopt 10 seconds as the regular sensing interval to prepare for gas leakage, the per-node energy consumption becomes 632 mJ and duty cycle is 0.2. It is apparent that our adaptive scheduling contributes to a longer lifetime of a wireless sensor network while satisfying the dynamically changing sensing demand.
Energy-Efficient Task Scheduling and Data Aggregation Techniques
61
4 Overhearing-Based Data Transmission Reduction As described in Section 2.2, there have been many data aggregation methods to reduce data traffic in wireless sensor networks. However, most of them needs packet exchanges in order to share information for reducing data traffic. Such a control traffic in the conventional methods may result in much extra energy consumption. To solve this problem, we have proposed a novel data aggregation method named ODAS (Overhearing-based Data Aggregaion method using Spatial interpolation) [16, 17], that does not need any control packets. ODAS reduces both of data and control traffics by exploiting the characteristics of sensor data acquired by nodes (readings) and wireless communication. In this section, we briefly present the characteristics of wireless sensor networks that our ODAS exploits. Then, we explain ODAS in detail. Note that for simplicity of discussion, we focus only on a single type of sensor (e.g., temperature) in this section. We also focus only on awake sensor nodes and do not consider to control the awake/sleep strategy.
4.1 Characteristics of Wireless Sensor Networks In this section, we assume monitoring applications which is a typical kind of applications of wireless sensor networks. In monitoring applications, it is common that users obtain the distribution of sensor data such as temperature and humidity in the whole target area. Here, considering these kinds of data, the reading of a node is regarded as data corresponding to the discrete point (location of the node). In addition, as described in Section 2.2, the readings generally have spatial and temporal correlations. Thus, the data at a point where no node exists can be estimated by using readings of the adjacent nodes. On the other hand, in wireless communication, radio signals transmitted by a node propagate to the area of the nodes’ wireless communication range. Thus, when a node sends a packet, the neighboring (awake) nodes which exist in the communication range can receive (overhear) the packet regardless of whether they are the destination of the packet or not. ODAS exploits the above two characteristics, that is, 1) spatial correlation of readings and 2) overhearing of packets, in order to reduce data traffic in wireless sensor networks. In ODAS, each node autonomously determines the redundancy of its readings based on the overheard readings transmitted by its neighboring nodes. Here, ‘redundant’ means that the reading can be estimated from other readings by the sink node based on the spatial correlation. Then, when the node determines its reading as redundant, the node stops transmitting its reading. By doing so, only the readings which are necessary to obtain the data distribution are transferred to the sink node.
62
A. Kanzaki, N. Wakamiya, and T. Hara
4.2 Assumptions in ODAS ODAS assumes that all nodes have the same communication range r and know the locations of themselves and all their neighbors. All nodes periodically and simultaneously senses a physical phenomenon, and send their readings to the sink node. The interval of sensing is specified by the application in advance. We call this interval cycle. The application also specifies the acceptable error range E, which is the acceptable difference between the actual reading and the estimated reading for every node. An estimated reading is the reading estimated based on the spatial correlation. Thus, the application should satisfy the following condition for every node: |vi − vSi | ≤ E
(1 ≤ i ≤ n),
(6)
where vi and vSi respectively denote the actual and the estimated readings of node i, and n denotes the number of nodes in the network. As the MAC (media access control) layer protocol, ODAS employs a TDMA (time division multiple access) based protocol. TDMA divides the wireless channel into time units, called slots. Each node is assigned a slot, and gets opportunities to transmit packets only in its assigned slots. In addition, TDMA groups multiple slots into a frame, and repeats the frame. In other words, each node can transmit packets at the interval of frame. Fig. 6 shows an example of TDMA where a frame consists of eight slots.
4.3 Overview of ODAS ODAS defines the following two phases in a cycle (see Fig. 7): • Redundancy determination phase: In this phase, each node determines the redundancy of its reading based on the overheard readings. Only when the node determines that its reading is not redundant, the node transmits its reading.
Redundancy Determination Phase Data Gathering Phase Start frame
Intermediate frames (F times)
Fig. 7 Overview of ODAS
End frame
Energy-Efficient Task Scheduling and Data Aggregation Techniques
63
• Data gathering phase: In this phase, the readings transmitted in the preceding redundancy determination phase are transfered to the sink node. Here, ODAS does not define the way to gather the readings in this phase. Thus, any kind of routing protocol can be applied. In the redundancy determination phase, there are three kinds of frames, the start frame, intermediate frames (F frames), and the end frame. In a frame, each node basically turns its radio transceiver off. In the slots assigned to each of its neighbors, the node turns its transceiver on and overhears packets. In its assigned slot, the node transmits a packet according to the result of the redundancy determination described in Section 4.4. Here, a packet contains information on node identifier and the reading of the transmitter. In what follows, we present the detailed behavior of nodes in each frame. Start frame: In the start frame, all the nodes that did not transmit packet in the start frame of the previous cycle determine to transmit packets without redundancy determination. We call these nodes trigger nodes. In addition, all nodes overhear packets transmitted by their neighbors. Here, when a trigger node overhears at least one packet, it cancels to transmit its reading. By doing so, packets are transmitted evenly according to the communication range r. These packets become information for nodes to determine the redundancy of their readings. We call these packets trigger packets. Intermediate frames: In an intermediate frame, each node operates according to the result of redundancy determination described in Section 4.4. End frame: In the end frame, the following nodes transmit packets: • The nodes that did not overhear any packet until the beginning of this frame. • The nodes that determines their readings as necessary (not redundant) at the beginning of this frame.
4.4 Procedures on Overhearing Every time a node overhears a packet, it determines the redundancy of its reading according to the following procedures: 1. Spatial interpolation: ODAS employs three-dimensional spatial interpolation to calculate the estimated reading. In what follows, we show the specific algorithm of the spatial interpolation. Here, we assume a three-dimensional space where x and y coordinates represent the location of the sensor node, and z coordinate denotes the reading at the corresponding location. • When a node has overheard one reading in the cycle, the node regards the overheard reading as its estimated reading as shown in Fig. 8(a). • When a node has overheard two readings in the cycle, the node derives a line segment whose end points are the overheard readings and its perpendicular which is parallel to the x − y face. After that, the node derives a flat surface that includes
64
A. Kanzaki, N. Wakamiya, and T. Hara z
z
Estimated reading Overheared reading y
Itself
y
Neighbor
x
x
(a) using one reading
(b) using two readings z
y
x
(c) using three readings Fig. 8 Spatial interpolations
both lines, and regards the value on the derived surface whose x and y coordinates are the location of the node, as the estimated reading as shown in Fig. 8(b). • When a node has overheard more than three readings in the cycle, the node firstly chooses three nodes for interpolation. Specifically, the node chooses ones whose locations construct a triangle containing the location of itself. When there are multiple candidates, the node chooses three nodes in which the total distance between itself and them is the smallest. On the other hand, when there is no candidate, the node chooses three nodes in which the total distance between itself and them is the smallest. After that, the node derives a flat surface that contains the overheard readings from the chosen three nodes, and regards the value on the derived surface whose x and y coordinates are the location of the node, as the estimated reading as shown in Fig. 8(c). Here, when no triangle can be constructed (i.e., the chosen three nodes exist on a line), the node derives a flat surface that includes the line and its perpendicular which is parallel to the x − y face, and regards the value on the derived surface whose x and y coordinates are the location of the node, as the estimated reading. 2. Redundancy determination: Using the calculated estimated reading, the node determines the redundancy of its reading by evaluating the following condition: |vi − vˆi | ≤ E,
(7)
Energy-Efficient Task Scheduling and Data Aggregation Techniques
65
where vi , vˆi , and E respectively denotes the actual reading of node i, the estimated reading of node i, and the acceptable error range specified by the application. Here, the left-hand side of the above condition is the difference between the actual and the estimated readings. We call this value interpolation error. When the calculated estimated reading vˆi satisfies the above condition, the node determines its reading as redundant, and stops transmitting the packet. Otherwise, the node transmits the packet in the succeeding frame determined in the next procedure. 3. Transmission frame determination: When a node determines its reading as necessary, the node determines to transmit the packet. Here, it is desirable to transmit the reading with a larger interpolation error earlier than others. This is because such a reading tends to be more informative than others for obtaining the data distribution. Thus, ODAS controls the order of packet transmissions by using the existence of multiple intermediate frames. Specifically, each node determines the number of frames to wait before transmitting the packet according to the interpolation error. To determine the number of frames to wait, ODAS uses the transmission frame determination table. This table defines the relation between the number of frames to wait and the interpolation error. Specifically, this table has g rows in which the l-th row indicates the following condition: ⎧ M M ⎨ E + g−1 l ≥ di > E + g−1 (l − 1), 1 ≤ l < g, (8) ⎩ di > E + M, l = g, where di and M respectively denote the interpolation error of node i and the predetermined threshold. When the condition of the l-th row is satisfied, the node waits (g − l) frames before transmitting the packet. Table 2 shows an example of this table. This table indicates that, the more the interpolation error is, the earlier the node transmits the packet. Here, when g = 1, M is set as 0. Thus, this table has only one rule; if di ≥ E, transmit packet in the next assigned slot. In addition, when the determined frame becomes later than the end frame, the node transmits the packet in the end frame. Table 2 Transmission frame determination table (E = 0.5, g = 4, M = 0.3) Condition 0.5 + 0.1 ≥ di > 0.5 0.5 + 0.2 ≥ di > 0.5 + 0.1 0.5 + 0.3 ≥ di > 0.5 + 0.2 di > 0.5 + 0.3
# of frames to wait 3 2 1 0
66
A. Kanzaki, N. Wakamiya, and T. Hara
4.5 Restoring Missing Readings According to the above procedures, the nodes that determine their readings as redundant do not send their readings. Thus, the sink node will not obtain the actual readings of those nodes. However, such ‘missing’ readings can be restored at the sink node by reenacting the procedure of the nodes. Specifically, for a node which has not sent its reading, the sink node firstly refers to the actual readings sent from neighbors of the node. After that, the sink chooses the readings that the node uses for spatial interpolation, and calculates the estimated reading of the node. By doing so, the sink node can estimate the reading of the node whose error always becomes smaller than the acceptable error range E.
4.6 Evaluation In this section, we show the results of simulation experiments that evaluate the effectiveness of our method. In the experiments, we assume a monitoring application which measures the temperature in a certain area. In the experiments, we randomly deploy 600 nodes and one sink node in a 100 m × 100 m flatland. The sink node is located at the corner of the area. The communication range of all nodes is set as 10 m. In addition, we assume that TDMA slot assignment has completed in advance so as to minimize the number of slots in a frame. As data distributions, we apply four distributions as shown in Fig. 9. The energy consumption model of a node is set as shown in Table 3 according to the specification of MICAz Mote[22]. The duration of a slot and the size of a packet are set as 20 ms and 64 bytes. Other parameters are set as shown in Table 4. In the experiments, we change one or two of the parameters and set others as default values in order to verify the effect of each parameter. Here, ‘ideal’ in the number of intermediate frames in Table 4 means the environment where an intermediate frame can be repeated until no node sends packet any more, i.e., all nodes have sent their packets or determined to stop sending their packets. The communication route from each node to the sink node is set in advance so as to minimize the number of hops (intermediate nodes) on the route. In the above environment, we evaluate (a) the total number of packets transmitted in a redundancy determination phase and (b) the total energy consumed in a cycle. Fig. 10 shows the simulation results changing the acceptable error range E. Fig. 10(a) is the total number of packets transmitted in a redundancy determination phase. As shown in this figure, the number of transmitted packets decreases as E gets larger regardless of the data distribution. This is because many readings satisfy the condition (7) when E is large. In the flat slope and compound distributions, the numbers of packets do not become less than a certain value (about 60) even when E becomes large. We analyzed the details of transmitted packets in the environments where the number of packets becomes this value, and confirmed that all of the transmitted packets are trigger packets (i.e., packets transmitted in the start frame). This indicates that all nodes determine their readings as redundant from the
Energy-Efficient Task Scheduling and Data Aggregation Techniques
67
z
z 19
19
18
18
17
17
16
16 㻣㻤
㻣㻤 15 0
㻟㻥 25
50
x [m]
75
15
y [m]
0
㻜
㻟㻥 25
50
y [m]
㻜
75
x [m]
(b) Flat slope
(a) 2D-Gaussian
z
z
19
19
18
18
17 17 16 16 15
㻣㻤 15 0
㻟㻥 25
x [m]
50
75
㻜
y [m]
0 㻞㻡
㻜
25 㻡㻜
x [m]
50 㻣㻡
y [m]
75
(d) Compound
(c) Waterfall
Fig. 9 Data distributions Table 3 Energy consumption model Operation Transmit Receive Listen (Overhear) Sleep (Turn transceiver off)
Current [mA] 22 22 2 0.15
trigger packets. On the other hand, in the 2D-gaussian and waterfall distributions, the numbers of transmitted packets are larger than those in other data distributions. This is because the change of data is too drastic, that makes difficult for ODAS to estimate the data distribution from fewer readings. Fig. 10(b) shows the total energy consumed in a cycle. As shown in this figure, the energy consumption shows the similar tendency with the number of transmitted packets. From this result, we can see that the energy consumption is affected by the number of transmitted packets.
68
A. Kanzaki, N. Wakamiya, and T. Hara Table 4 Parameter settings in the experiments Parameter
Value (Default) 0.1–1.0 (0.2) 1–10 (3) 0.1–1.0 (0.3) 1–8 (ideal)
140
0.45
120
0.4
Energy comsumption [mJ]
# of transmitted packets
E g M F
100 80 60 40 2D-Gaussian
20
Waterfall
Flat slope Compound
2D-Gaussian Flat slope Waterfall Compound
0.35 0.3 0.25 0.2 0.15 0.1 0.05 0
0 0
0.2
0.4
0.6
0.8
1
0
0.2
0.4
0.6
0.8
1
E
E
(a) The number of transmitted packets
(b) Energy consumption
Fig. 10 Effects of E
Fig. 11 shows the simulation results changing the parameters in the transmission frame determination table. In this figure, we only show the results in the 2DGaussian distribution. Note that the results in other data distributions show similar characteristics with those in Fig. 11. Fig. 11(a) is the total number of packets transmitted in a redundancy determination phase. As shown in this figure, the number of transmitted packets shows little changes even when the parameters in the transmission frame determination table change. This is because almost all nodes determine the redundancies of their readings only from the trigger packets. In other words, there are very few nodes that change their determinations by overhearing packets in intermediate frames. Fig. 11(b) shows the total energy consumed in a cycle. As shown in this figure, the energy consumption increases as g gets larger. This is because the number of frames to wait corresponding to an interpolation error becomes large when g increases. When the number of frames in a redundancy determination phase increases, the energy consumption becomes large due to the increase of energy consumed in the listen mode, that is, each node has to turn its transceiver on in the slots assigned to each of its neighbors. Also, the energy consumption increases as M gets larger. This is because the number of frames to wait becomes large when M increases.
140
0.9
120
0.8
Energy consumption [mJ]
# of transmitted packets
Energy-Efficient Task Scheduling and Data Aggregation Techniques
100 80 60 40 20
M=0.3
M=0.5
M=0.8
M=1.0
69
M=0.3 M=0.5 M=0.8 M=1.0
0.7 0.6 0.5 0.4 0.3 0.2 0.1 0
0 1
2
3
4
5
6
7
8
9
10
g
(a) The number of transmitted packets
1
2
3
4
5
6
7
8
9
10
g
(b) Energy consumption
Fig. 11 Effects of g and M (Data distribution: 2D-Gaussian)
Fig. 12 shows the simulation results changing the number of intermediate frames F. Fig. 12(a) is the total number of packets transmitted in a redundancy determination phase. As shown in this figure, the number of transmitted packets shows little changes even when the number of intermediate frames changes. Similar to the result in Fig. 11(a), this is because almost all nodes determine the redundancies of their readings only from the trigger packets. Comparing the data distribution, the number of transmitted packets in the 2D-Gaussian and waterfall distributions become larger than those in other distributions. As described in the result in Fig. 10(a), this is because the change of data in these distributions are more drastic than those in other distributions. Fig. 12(b) shows the total energy consumed in a cycle. As shown in this figure, the energy consumption increases as the number of intermediate frames gets larger regardless of the data distribution. As described in the result in Fig. 11(b), this is due to the increase of energy consumed in the listen mode. Thus, we can see that, in ODAS, it is effective to limit the number of intermediate frames for reducing energy consumption. In addition to evaluate the characteristics of ODAS, we compare the total energy consumption of ODAS with those of the following methods: • All gathering: This method is a baseline method in which all the readings are sent to the sink node. Specifically, this method consists only of the data gathering phase. After the sensing, all nodes send their readings to the sink node according to the predetermined communication route. • Clustering aggregation: This method is a data aggregation method that does not exploit overhearing of wireless communications. Specifically, this method divides the network into several clusters each of which consists one cluster head and several member nodes. In a redundancy determination phase, member nodes in each cluster directly transmit packets to the cluster head without overhearing packets. The cluster head in each cluster suppresses the redundant readings and sends the necessary readings to the sink node according to the predetermined
70
A. Kanzaki, N. Wakamiya, and T. Hara 140
0.25
2D-Gaussian Flat slope Waterfall Compound
Energy consumption [mJ]
# of transmitted packets
120 100 80 60 40 20
2D-Gaussian
Flat slope
Waterfall
Compound
0.2 0.15 0.1 0.05 0
0 0
1
2
3
4
5
6
7
0
8
1
2
3
4
5
6
7
8
F
F
(a) The number of transmitted packets
(b) Energy consumption
Fig. 12 Effects of F
Energy consumption [mJ]
16
1.6 Data gathering phase
1.4
Redundancy determination phase
14 12 10 8 6 4
Energy consumption [mJ]
18
1.2
Transmit & Receive Listen Sleep
1 0.8 0.6 0.4 0.2
2 0
0
ODAS
ODAS
Clustering aggregation
All gathering
Fig. 13 Total energy consumption
Clustering aggregation
Fig. 14 Energy consumed in a redundancy determination phase
communication route in the data gathering phase. Here, we assume that the readings sent to the sink node are same as those in ODAS. In both methods, the communication route from each node to the sink node is same as that in ODAS. In addition, considering the results in Fig. 12, we set the number of intermediate frames as 0. Here, there are several other data aggregation methods as described in Section 1.2.2. However, the assumptions of these conventional methods are different from that in our method. Thus, we have evaluated the total energy consumptions of ODAS and the above two methods. Figs. 13 and 14 show the results of the experiment. First, Fig. 13 shows the total energy consumption in each method. As shown in this figure, we can see that the all gathering method consumes much
Energy-Efficient Task Scheduling and Data Aggregation Techniques
71
more energy than other methods. On the other hand, ODAS and clustering aggregation reduce the energy consumed in a data gathering phase. In addition, in a redundancy determination phase, ODAS much reduces the energy consumption compared with clustering aggregation. Fig. 14 shows the details of energy consumed in a redundancy determination phase. From this figure, we can see that ODAS drastically reduces the energy consumption for packet transmission/reception. From these results, we can conclude that ODAS achieves the energy-efficient data gathering by reducing packet transmissions.
5 Energy-Efficient Monitoring Incorporating Task Scheduling and Data Aggregation In this section, we discuss brief ideas on integrating our task scheduling and data aggregation techniques in order to further reduce the energy consumption in monitoring applications. First, we discuss the introduction of our data aggregation method into our task scheduling mechanisms. In our task scheduling mechanisms, each node receives sensing data from neighbors further from the sink node (i.e., upstream nodes), performs sensing, and transmits the received and acquired data to a neighbor closer to the sink node (i.e., downstream node), in accordance with its sensing interval. Here, when the node determines the redundancy of its sensing data according to the redundancy determination in our data aggregation method, the data traffic can be further reduced. For instance in Fig. 15, node i performs sensing after receiving packets from upstream nodes k and l. In this case, node i can determine the redundancy of its sensing data using data acquired by nodes k and l as shown in the right hand side of Fig. 15. When node i determines its sensing data as redundant, node i discards its sensing data and transmit only the received data to the downstream node h. By doing so, the data traffic in the entire network can be suppressed. As a result, further reduction in the energy consumption can be achieved. Here, in our task scheduling
sink node
h j
n
Zzz
i
i
m
l
k
l
k
Fig. 15 Redundancy determination in our task scheduling mechanisms
72
A. Kanzaki, N. Wakamiya, and T. Hara
unusual event
Cycle: 5 [min.]
Cycle: 1 [min.]
sink node
Fig. 16 Task scheduling in our data aggregation method
mechanisms, since sensing data are transmitted according to level values of nodes, each node can rarely receive packets from neighbors closer to the sink node. Thus, some nodes may fail to determine its sensing data as redundant due to the lack of overheard sensing data. In order to avoid such a problem, it is effective that a node determines the redundancies of its upstream nodes using sensing data acquired by itself. For instance in Fig. 15, node h can determine the redundancy of sensed data acquired by its upstream node i using data acquired by nodes k, l (recorded in the received packet from node i) and itself. When the sensed data is determined as redundant, node h can discard the data from the packet. Second, we discuss the introduction of our task scheduling mechanisms into our data aggregation method. Our data aggregation method reduces the data traffic assuming that the interval of sensing and data gathering (i.e., cycle) is specified by the application in advance. By introducing the notion of task scheduling, the frequent sensing and data gathering can be achieved while keeping low data traffic. For instance, when an unusual event happens in a certain area, nodes in the area and on the communication routes from the area shorten the lengths of their cycles according to our task scheduling mechanisms, as shown in Fig. 16. By doing so, the system can provide the detailed information on the event while keeping low data traffic. Here, in our data aggregation method, it is necessary that the phases in a cycle (i.e., sensing, redundancy determination, and data gathering) are synchronized
Energy-Efficient Task Scheduling and Data Aggregation Techniques
73
among all nodes. Thus, in order to achieve frequent sensing in a certain area, the lengths of cycles must change simultaneously among all nodes in the area and on the routes from the area. Such simultaneous changes in the lengths of cycles can be realized by appending a notification to the packets. Specifically, when a node detects an unusual event, it appends information on the updated cycle to the packet. The information can be propagated to all its neighbors (in the redundancy determination phase) and all nodes on the route from the node (in the data gathering phase). Thus, these nodes can simultaneously update the lengths of their cycles according to the received information at the starting time of the next cycle. In addition, considering the requirement of monitoring applications, it is effective to adjust parameters in one of our techniques according to those in another technique. For instance, when the interval of sensing becomes small in a certain area according to our task scheduling mechanisms, it is expected that the application needs the detailed information on the area. In such a case, the acceptable error range E in our data aggregation method should be smaller. As described above, the integration of our techniques has much potential to reduce the energy consumption. In addition, several extensions can be considered to further improve the energy-efficiency in monitoring applications. In future, we will design and implement the details of the integrated scheme based on the above discussions.
6 Summary In this chapter, we introduce our proposed energy-efficient techniques in wireless sensor networks. First, we introduce energy-efficient and adaptive control of a wireless sensor network. Our approaches are fully distributed and self-organizing to achieve high scalability and robustness. Second, we introduce a data transmission reduction method, named ODAS (Overhearing-based Data Aggregation method using Spatial interpolation). This method exploits the characteristics of a wireless sensor network, spatial correlation of readings and overhearing of packets, and reduces data traffic for data gathering. In addition, we present a brief idea on combination of our techniques. In this chapter, the effectiveness of each technique is evaluated assuming different environment and application in order to make clear the characteristics of each technique. In future, we plan to conduct comprehensive experiments to evaluate the overall performance of our techniques. We also plan to design and implement the integrated scheme as described in Section 5. Acknowledgements. This research was partially supported by Grant-in-Aid for Scientific Research on Priority Areas (18049050) and for Scientific Research (S) (21220002), and “Global COE (Centers of Excellence) Program” of the Ministry of Education, Culture, Sports, Science and Technology, Japan.
74
A. Kanzaki, N. Wakamiya, and T. Hara
References 1. Alippi, C., Anastasi, G., Galperti, C., Mancini, F., Roveri, M.: Adaptive sampling for energy conservation in wireless sensor networks for snow monitoring applications. In: Proc. IEEE Int’l Conf. on Mobile Ad-hoc and Sensor Systems (MASS 2007), pp. 1–6 (2007) 2. Anastasi, G., Conti, M., Francesco, M.D., Passarella, A.: An adaptive and low-latency power management protocol for wireless sensor networks. In: Proc. ACM Int’l Workshop on Mobility Management and Wireless Access, MobiWac 2006 (2006) 3. Arici, T., Altunbasak, Y.: Adaptive sensing for environment monitoring using wireless sensor networks. In: Proc. IEEE Wireless Communications and Networking Conf. (IWNC 2004), vol. 4, pp. 2347–2352 (2004) 4. Banerjee, T., Choudhury, K., Agrawal, D.P.: Tree based data aggregation in sensor networks using polynomial regression. In: Proc. Int’l Conf. on Information Fusion (FUSION 2005), vol. 2, pp. 25–29 (2005) 5. Banerjee, T., Choudhury, K., Agrawal, D.P.: Distributed data aggregation in sensor networks by regression based compression. In: Proc. Int’l. Conf. on Mobile Ad-hoc and Sensor Systems (MASS 2005), pp. 290–293 (2005) 6. Bonabeau, E., Sobkowski, A., Theraulaz, G., Deneubourg, J.L.: Adaptive task allocation inspired by a model of division of labor in social insects. In: Proc. Biocomputing and Emergent Computation, pp. 36–45 (1997) 7. Bonabeau, E., Dorigo, M., Theraulaz, G.: Swarm intelligence: from natural to artificial systems. Oxford University Press, Oxford (1999) 8. Cerpa, A., Estrin, D.: ASCENT: Adaptive self-configuring sensor networks topologies. IEEE Trans. Mobile Computing 3(3), 272–285 (2004) 9. Chen, B., Jamieson, K., Balakrishnan, H., Morris, R.: Span: An energy-efficient coordination algorithm for topology maintenance in ad hoc wireless networks. ACM Wireless Network Journal 8(5), 481–494 (2002) 10. Chu, D., Deshpande, A., Hellerstein, J.M., Hong, W.: Approximate data collection in sensor networks using probabilistic models. In: Proc. Int’l Conf. on data engineering (ICDE 2006), pp. 48–60 (2006) 11. Dai, H., Han, R.: TSync: a lightweight bidirectional time synchronization service for wireless sensor networks. SIGMOBILE Mobile Computing and Communications Review 8(1), 125–139 (2004) 12. Deshpande, A., Guestrin, C., Madden, S.R., Hellerstein, J.M., Hong, W.: Model-driven data acquisition in sensor networks. In: Proc. Int’l Conf. on Very Large Data Bases (VLDB 2004), pp. 588–599 (2004) 13. Goel, P., Ermentrout, B.: Synchrony, stability, and firing patterns in pulse-coupled oscillators. Physica D 163, 191–216 (2002) 14. Guestin, C., Bodi, P., Thibau, R., Paski, M., Madden, S.: Distributed regression: an efficient framework for modeling sensor network data. In: Proc. Int’l Symposium on Information Processing in Sensor Networks (IPSN 2004), pp. 1–10 (2004) 15. Huang, C.F., Tseng, Y.C., Wu, H.L.: Distributed protocols for ensuring both coverage and connectivity of a wireless sensor network. ACM Trans. Sensor Networks 3(1) (2007) 16. Iima, Y., Kanzaki, A., Hara, T., Nishio, S.: Overhearing-based data transmission reduction for periodical data gathering in wireless sensor networks. In: Proc. Int’l Workshop on Data Management for Information Explosion in Wireless Networks (DMIEW 2009), pp. 1048–1053 (2009)
Energy-Efficient Task Scheduling and Data Aggregation Techniques
75
17. Iima, Y., Kanzaki, A., Hara, T., Nishio, S.: An evaluation of overhearing-based data transmission reduction in wireless sensor networks. In: Proc. Int’l Workshop on Sensor Network Technologies for Information Explosion Era (SeNTIE 2009), pp. 519–524 (2009) 18. Kotidis, Y.: Snapshot queries: towards data-centric sensor networks. In: Proc. Int’l Conf. on Data Engineering (ICDE 2005), pp. 131–142 (2005) 19. Labella, T.H., Dressler, F.: A bio-inspired architecture for division of labour in SANETs. In: Proc. Int’l Conf. on Bio-inspired Models of Network, Information and Computing Systems (Bionetics 2006), pp. 211–230 (2006) 20. Low, K.H., Leow, W.K., Marcelo, H., Ang, J.: Autonomic mobile sensor network with self-coordinated task allocation and execution. IEEE Trans. Systems, Man, and Cybernetics 36(3), 315–327 (2006) 21. Lu, G., Krishnamachari, B., Raghavendra, C.S.: An adapive energy-efficient and lowlatency MAC for tree-based data gathering in sensor networks. Wireless Communications & Mobile Computing 7(7), 863–875 (2007) 22. MICAz Datasheet.pdf, http://www.xbow.com/Products/Product_pdf_files/Wireless_ pdf/MICAz_Datasheet.pdf. (Cited June 8, 2009) 23. Mirollo, R.E., Strogatz, S.H.: Synchronization of pulse-coupled biological oscillators. Society for Industrial and Applied Mathematics. Journal on Applied Mathematics 50(6), 1645–1662 (1990) 24. Simeone, O., Spagnolini, U.: Distributed time synchronization in wireless sensor networks with coupled discrete-time oscillators. EURASIP Journal on Wireless Communications and Networking (2007), doi:10.1155/2007/57054 25. Sun, Y., Gurewitz, O., Johnson, D.B.: RI-MAC: A receiver initiated asynchronous duty cycle MAC protocol for dynamic traffic load. In: Proc. ACM Conf. on Embedded Networked Sensor Systems, SenSys 2008 (2008) 26. Taniguchi, Y., Wakamiya, N., Murata, M.: A traveling wave based communication mechanism for wireless sensor networks. Academy Publisher Journal of Networks 2(5), 24–32 (2007) 27. Taniguchi, Y., Wakamiya, N., Murata, M., Fukushima, T.: An autonomous data gathering scheme adaptive to sensing requirements for industrial environment monitoring. In: Proc. IFIP Int’l Conf. on New Technologies, Mobility and Security (NTMS 2008), pp. 52–56 (2008) 28. Tulone, D., Madden, S.: PAQ: time series forecasting for approximate query answering in sensor networks. In: R¨omer, K., Karl, H., Mattern, F. (eds.) EWSN 2006. LNCS, vol. 3868, pp. 21–37. Springer, Heidelberg (2006) 29. Tulone, D., Madden, S.: An energy-efficient querying framework in sensor networks for detecting node similarities. In: Proc. Int’l Symposium on Modeling, Analysis and Simulatoin of Wireless and Mobile Systems (MSWiM 2006), pp. 291–300 (2006) 30. Wakamiya, N., Murata, M.: Synchronization-based data gathering scheme for sensor networks. IEICE Trans. Communicatios (Special Issue on Ubiquitous Networks) E88B(3), 873–881 (2005) 31. Xing, G., Wang, X., Zhang, Y., Lu, C., Pless, R., Gill, C.: Integrated coverage and connectivity configuration for energy conservation in sensor networks. ACM Trans. Sensor Networks 1(1), 36–72 (2005) 32. Younis, O., Fahmy, S.: Distributed clustering in ad-hoc sensor networks: a hybrid, energy-efficient approach. In: Proc. IEEE INFOCOM 2004 (2004)
Secure Data Aggregation in Wireless Sensor Networks Vimal Kumar and Sanjay Madria
*
Abstract. The inherent resource constraints associated with wireless sensors make it important to minimize data transmissions within the network so that the sensor lifetime is improved and the network works for a longer period of time. Data aggregation is one such technique where data originating at different sensors is combined and transmitted together. This in-network processing of data helps in reducing the wireless communication but it also makes the task of providing security to the data much more challenging. In this chapter, we motivate, review and discuss research issues in secure data aggregation. In particular, we provide a discussion on general secure data aggregation framework and survey a few existing secure data aggregation schemes.
1 Introduction A wireless sensor network (WSN) consists of a number of low cost sensors which co-operate to monitor some physical or environmental variables. WSNs have emerged as a popular solution to many applications in both controlled and uncontrolled environments in fields as varied as weather sensing, wildlife monitoring, building safety, traffic monitoring, law enforcement and military among others. With the recent advancement in technology such as the size of the sensors becomes smaller and the capabilities increase, sensor networks are envisioned to cover an even broader range of applications. A wireless sensor network is an event based system consists of one or more sinks and a number of nodes act as sources. The base station acts as a sink; it can either be a laptop or a soldier carrying a powerful PDA or a flying UAV (Unmanned Aerial Vehicle). The sink disseminates a query into the network, for example, a query could be “Alert me when the temperature crosses 75 degrees”. The sensors act as the sources and keep sensing the environment until the required event happens. When the required event happens they publish this information for the sink to receive. Vimal Kumar · Sanjay Madria Department of Computer Science Missouri University of Science and Technology, Rolla, MO, USA e-mail: [email protected], [email protected]
Wireless sensors run on small batteries. Once these sensors are deployed in the field, it is not feasible to change the batteries when they run out. We want these sensor networks to work as long as possible. This calls for judicious use of the limited amount of energy the sensors have. Therefore, we need algorithms which minimize the use of energy while still producing the required results. Researchers investigating wireless sensor networks have found that computation on WSN typically less expensive than communication in terms of energy. On a power constrained sensor node, sending one bit on a wireless channel consumes energy roughly equivalent to executing 50 to 150 CPU instructions [19, 20]. Now that we know wireless communication drains the batteries very quickly, techniques which reduce the communication among sensors are needed. In-network data processing is one such technique which distributes some of the base station’s processing tasks in the network. The sensors sense the environment, generate the data and pre process it in the network before it reaches the base station. Data aggregation and data fusion are examples of pre processing which can be done within the network. In data aggregation where data is gathered from various sensors and specially selected sensor nodes apply aggregation functions on this data. This results in one single data point which is a representative of the collected data. However, as evident, it has a direct impact on the security aspect as now capturing a single aggregator node can compromise the aggregated value. We discuss data aggregation in detail in Section 2. In Section 3, we discuss the security issues in data aggregation. Section 4 presents the background in security. In Section 5, various network models used in secure data aggregation algorithms are discussed. Section 6 introduces a general framework for secure data aggregation algorithms. In Section 7 we review some of the algorithms based on the framework and finally we conclude the chapter in Section 8.
2 Data Aggregation The power of sensors lies not just in their capability to gather data but in their ability to process this data. The data collected from sensors needs to be processed in order to extract meaningful information from it. Many times this processing is done by means of aggregation functions like SUM, AVERAGE, MEAN, MEDIAN etc. Data aggregation in sensor networks pertains to the processing of sensor data, within the network, in a distributed manner. Sensor applications require aggregated data instead of the individual sensor data. For example, an environment sensing application will be interested in knowing the average temperature of the area instead of individual temperatures sensed by wireless sensors. In the scheme shown in Fig. 1 , the sensing nodes send their data to a node which is a designated aggregator. This node combines all the data it receives and sends this combined data up the hierarchy either to its parent or to the base station, depending upon the network model. The aggregators themselves may or may not generate data of their own. The data travels all the way to the base station in this manner. We observe that instead of sending each sensor’s individual data over a multi-hop path the aggregators only send one packet consisting of all the data. As can be seen, it reduces the number of messages being passed in the network, which in
Secure Data Aggregation in Wireless Sensor Networks
79
turn helps minimizing the energy consumption and also improves bandwidth utilization. In Fig. 1, if each sensor has to send its reading back to the base station, without aggregation, each of the leaf node sensors generates a reading and sends it to its parent. Since there are 9 sensors, we have 9 messages being generated here. At the next level, the sensors pass each message they receive from their children up the hierarchy and also generate their readings and pass them on. At this level in the hierarchy, a total of 9 + 3 = 12 messages are transmitted. Similarly at the next level, 12+1 = 13 messages are transmitted. So, in the network, a total of 9+12+13= 34 messages are being sent. If data aggregation is used then each of the 9 sensors pass their readings to their parents as above but here these readings and the parent’s own reading are aggregated and merged into a single reading so only 3 messages are communicated by the parents. At the next level, these three are aggregated again into 1 combined reading which is sent to the base station. Thus, we see that only 13 messages are being passed in the network compared to the 34 earlier when data aggregation was not used. In wireless sensor deployments where we talk of deploying thousands of sensors, this difference in the number of transmissions becomes significantly larger and cannot be overlooked.
sensors
base station aggregators
Fig. 1 Data aggregation in a wireless sensor network
In-network data aggregation also enables the network to directly provide services rather than raw data [11]. When we enable the nodes to do computation and processing of data rather than just sensing and forwarding messages, sensors possess processed information instead of just raw data. This can be directly sent to the subscribers. Last, but not the least, data aggregation also helps to eliminate the redundancy in the raw data. In wireless sensor networks deployed for reporting the occurrence of certain events, all of the sensors which listen to the event try to forward the same message to the base station. An intermediate sensor which acts as an aggregator can help in eliminating this redundancy by sending only one message up the hierarchy rather than all of them.
80
V. Kumar and S. Madria
3 Secure Data Aggregation Data aggregation is an efficient way to minimize energy consumption on sensors, but it also creates new security challenges. In a multihop sensor network, data is forwarded to and by a number of intermediate forwarding nodes, before it reaches the sink. The data is thus potentially exposed at each such forwarding node. Detecting a malicious node which manipulates data coming from its children in the routing tree is not trivial in such networks. Techniques such as digital signatures and commit and attest are used in such a scenario. However, this problem becomes even more difficult when we introduce data aggregation. Now, along with leaf nodes and forwarders nodes we also have aggregator nodes. Once we aggregate the data originating from different nodes, the inherent identification information contained in the data is lost. If the aggregated data turns out to be an outlier, it is hard to identify the malicious node which caused it. The problem here is to find out whether one of the sources generated bad data, is it the forwarder who manipulated the data or is it the aggregator who corrupted the data it received from the sensors? Moreover, wireless sensor networks are usually deployed in hostile, unattended environments where they are susceptible to attack by an adversary. Let’s take the example of a battlefield surveillance application [21] where thousands of sensors are deployed randomly over the area to be monitored (e.g. dropped aerially). The goal of the system is to track and monitor the movements of military targets such as tanks and vehicles. The battlefield is an example of a high risk, hostile and unattended environment. Sensors deployed in such an environment run the risk of being detected and made unusable or even worse, made to steal data and work against the network. We cannot rely on data coming from such a network without proper security mechanisms in place. Add to this, the fact that in order to save costs and make them inexpensive, sensors are usually not made tamperresistant and we have an interesting and challenging problem of secure data aggregation on wireless sensor networks. When a sensor is compromised, the attacker gains access to all the cryptographic material stored inside it. Access to this data may allow the adversary to generate malicious code, listen to the network activity, delay system response or even inject false data into the system. The security risks and the requirements for prevention of these risks can be classified under three heads data confidentiality, data integrity and data availability.
3.1 Data Confidentiality Confidentiality is a fundamental issue in every aspect of security. It can be defined as “ensuring that information is accessible only to those authorized to have access”. A network should have the ability to protect the data, the secret keys as well as its identity from a passive attacker. There are two points of attack in a wireless sensor network, the nodes and the communication channel. A secure network providing data confidentiality will prevent any information leakage from the nodes
Secure Data Aggregation in Wireless Sensor Networks
81
and will provide a secure channel for these nodes to communicate. This is vital for wireless sensor networks where communication through the wireless channel is susceptible to eavesdropping. In our battlefield surveillance example, ensuring data confidentiality will mean not letting the adversary get hold of the data from the network. Encryption is the fundamental technique used for achieving confidentiality. Most algorithms provide secure communication by using an encryption mechanism and periodically changing the encryption key to keep it secure on the node.
3.2 Data Integrity Let’s assume that some of the nodes in our example are compromised. Data confidentiality protects the data from reaching un-intended parties, but it does not guarantee unaltered, uncorrupted data. Data may be corrupted by an adversary who has compromised a node or even by an unreliable channel, without the presence of an intruder [13]. In wireless sensor networks, where the communication is often multi hop, there is a possibility that a compromised node in the path between the sender and the receiver, alters the message in between. Data integrity deals with ensuring that the received data has not been tampered with, in the path between the receiver and the sender. In our example it will mean blocking off any attempts by the adversary to inject false data into the network. Message authentication codes and digital signatures are often used for integrity checking purposes.
3.3 Data Availability An often overlooked aspect of security is data availability which means that the data is available to the end user in a timely fashion. Ensuring data availability means making sure that the systems responsible for collecting, processing and delivering data function correctly and provide timely information. The system should be resilient in the face of hardware failures as well as attacks such as the denial of service attack. Availability is an important security primitive and a network which does not provide data at the time its needed is at least as bad as not providing it at all. Even worse is, trusting an uncompromised network only to find out the data is not available when needed. Going back to our battlefield surveillance example, damaged or faulty sensors may affect the availability of information. In such a scenario information not available on time may prove to be catastrophic.
4 Background in Security While setting up a system to provide data confidentiality, data integrity and data availability in a WSN, we have to keep in mind the resource constraints on sensors. Memory limitations put a constraint on the amount of data we can have in the memory at a given time. Although computation is less expensive than communication, it still is not free. Limited available energy forces us to avoid complex
82
V. Kumar and S. Madria
communications and thus renders many attractive options unfeasible. As a result of these constraints, algorithms like public key cryptography, which are widely used and preferred in traditional networks, do not remain as viable and preferred. Although we have optimized algorithms for these problems when working in a non-sensor, resource abundant environment, we need to look for new solutions for a resource constrained environment. In this section we briefly look at some of the security primitives, which are widely used in secure data aggregation in wireless sensor networks.
4.1 Elliptic Curve Cryptography Public key cryptography, also known as asymmetric cryptography is widely used now days in distributed environments. From key distribution to secure communication and message signing, public key cryptography is everywhere. However, public key algorithms generally are power hungry and clearly not suitable for wireless sensors. Over the last few years Elliptic curve cryptography (ECC) has emerged as an attractive and viable public key system for constrained environments. ECC offers considerably greater security for a given key size, compared to the first generation public key systems. The smaller key size also makes possible more compact implementations for a given level of security, which means faster cryptographic operations, running on smaller chips. This leads to less heat production and less power consumption. Effects of heat are necessary to consider here. There are two issues that heat production can create at a wireless sensor. First, heat eventually translates to energy, which is being wasted and second and the more important issue is that excessive heat can interfere with a sensor’s immediate environment, causing faulty readings. Going back to our discussion of the key size, we can see that ECC offers the equivalent security of a public key system for much lesser key size and also solves the problem of key distribution over insecure channel as in symmetric key cryptography. A small key size is not the only advantage of ECC compared to the first generation public key systems; it is computationally less expensive too. This can be seen from the table below. The NSA has decided to move to elliptic curve based public key cryptography wherever appropriate in their implementation effort. Table 1 NIST recommended key sizes. (From www.nsa.gov)
Secure Data Aggregation in Wireless Sensor Networks
83
Table 2 Relative Computation cost of Diffie Hellman and Elliptic Curves. (From www.nsa.gov)
Security Level (bits)
Ratio of DH Cost : EC Cost
80 112 128 192 256
3:1 6:1 10:1 32:1 64:1
Table 3 Energy cost of digital signature and key exchange computation [mJ]
Algorithm
RSA -1024 ECDSA -160 RSA -2048 ECDSA -224
Signature
Sign 304 22.82 2302.7 61.54
Verify 11.9 45.09 53.7 121.98
Key Exchange
Client 15.4 22.3 57.2 60.4
Server 304 22.3 2302.7 60.4
In [35], the authors compared the energy cost of RSA and ECC on an 8-bit microcontroller platform, and found ECC to have significant advantage over RSA in terms of energy required to complete the operations. The authors compared RSA signature scheme with ECDSA and RSA key exchange with ECDH key exchange scheme. Their results are compactly presented in Table 3. It is evident that ECC operations require much less energy overall than corresponding RSA operations. The comparatively high cost of verification is not an issue as mostly the verification is done at the base station. Elliptic curve cryptography is a discrete log (DL) system where the security is based on the intractability of the elliptic curve discrete log problem in a finite field. An elliptic curve E can be defined as an equation of the form:
where 4a3 + 27 b2 ≠ 0(mod p), p is a prime number greater than 3. We choose a base point G, and make it public for use. A private key k is selected as a random integer and then the public key P = k*G is calculated. The Elliptic Curve Discrete Log Problem (ECDLP) implies that it is hard to determine k given the public key P. This problem is harder than the original discrete log problem and elliptic curve cryptosystems are built around it.
84
V. Kumar and S. Madria
4.2 Homomorphic Encryption An encryption algorithm is said to be homomorphic if it allows for the following property to hold.
⊕
⊕
The above equation implies that in a homomorphic encryption scheme an operation performed on the encrypted data produces the same result as when the encryption is done after the operation has been performed on the plaintext first. There are two most common types of homomorphic encryption schemes, additive homomorphic schemes where we have enc ( a + b) = enc(a) + enc (b) and multiplicative homomorphic schemes where we have, enc ( a * b) = enc(a) * enc (b). One of the open problems in cryptography is to determine whether a fully homomorphic cryptosystem exists which can implement both of the above equations. Examples of homomorphic encryption schemes are RSA cryptosystem [30], which is a multiplicative homomorphism and ElGamal cryptosystem [31] which is an additive homomorphism. The advantage of using these cryptosystems is that we can carry out operations on ciphertext without needing the decryption key. This helps in protecting the decryption key by not exposing it at hostile places and requires less computation as no decryption is needed. Homomorphic encryption schemes like ElGamal have also been combined with elliptic curve cryptography which makes it very efficient to use on sensors. An example of homomorphic encryption is shown in Fig. 2. Data is generated by two nodes which is encrypted homomorphically and added together. This added ciphertext when decrypted is equal to the sum of the unencrypted data.
Node
Enc 10101
111010 Dec Addition
101110
111111
Enc 01010
110111 101010 + 010101 = 111111
Node
Fig. 2 Homomorphic Encryption
4.3 Digital Signatures Digital signatures [30] are fundamental to data integrity techniques. They can be thought of as the digital equivalent of handwritten signatures in real life but they
Secure Data Aggregation in Wireless Sensor Networks
85
are much more than that. Digital signatures provide non repudiation like hand written signatures and additionally provide data integrity. Asymmetric or public key cryptography is instrumental to digital signatures. The basic idea is to associate something unique with the signer. For this purpose a public-private key pair is generated. The private key is given to the signer and the public key to the verifier. If the private key is not compromised any data encrypted using it can be attributed to the signer. A block diagram of a digital signature scheme is produced below. The first step is to use a hashing function and find the hash of the data. The hash can be understood as a digital fingerprint of the data. A hash function has the property that given any bit string of arbitrary length, it produces a fixed length bit string called the hash. A secure hash function should also possess the following two properties.
Data
Hash
Signature
Data
Message
Private Key
Fig. 3 Digital Signature Generation
Signature
Data
Hash1 Hash2
Message Public key Yes = Data/Hash un-tampered No = Data/Hash tampered
Fig. 4 Digital Signature Verification
=?
86
V. Kumar and S. Madria
1. Pre image resistance: Given a hash function H if we already know the output bit string y, it is computationally infeasible to find a bit string x such that H(x) = y. 2. Collision resistance: It is computationally infeasible to find two distinct x1 and x2 such that H(x1) = H(x2). To create a digital signature the signer just encrypts the hash using his private key and sends this along with the data to the receiver. Verification of the signature by the receiver takes place as shown in the block diagram above. The verifier uses the same hash function as the signer to find the hash of the data. Next, he uses the public key of the signer to decrypt the signature he received. This decryption gives the verifier the hash of the data generated by the signer. If the two hashes match, we can be sure that data hasn’t been tampered with. If they don’t, it means either the hash or the data has been changed.
4.4 Aggregated Digital Signature Given n signatures on n distinct messages by n distinct users, it is possible to aggregate all these signatures into a single signature. This single signature will convince the verifier that the n users signed the n original messages [10]. In a wireless sensor network, this single signature can replace the individual signatures of the sensors and thus help us save the overhead of sending all the signatures along with the aggregated data. Each sensor in the network generates a digital signature on the data it produces and forwards both the data and the signature to its parent. The parent aggregates the data as well as the digital signatures and passes it to its parent. This process continues till the base station obtains the final aggregated data and the aggregate signature. Aggregated digital signature techniques [10] are of two kinds, one where each sensor shares its signing key with the base station so that the base station is able to verify the digital signature. The other is where along with the data and the signature the keys are also aggregated, so that at the end we get an aggregate key which can be used to verify the aggregate data using the aggregate signature.
4.5 Merkle Hash Trees A Merkle hash tree [4] is a tree of hashes which is used for integrity verification in wireless sensor networks. In a wireless sensor network where sensors are deployed in the form of a hierarchy, each sensor generates a reading and a hash on that reading. The reading and the hash is then passed on to the parent. The parent generates the hash of the hashes of its children and passes it further up. This continues until the root node of the tree obtains the top hash (Fig. 5). The top hash with other intermediate hashes in the tree can be used to verify the correctness of any particular reading. Generally, for checking the integrity of data sent by a particular sensor,
Secure Data Aggregation in Wireless Sensor Networks
87
the base station queries the sensors for their hash values. It then calculates the hash of the value to be checked and regenerates the top hash itself using the received hashes and plugs in the hash for the value to be checked. This regenerated top hash is compared against the one received earlier to verify the data. A number of schemes like [5, 6, and 7] have used Merkle hash trees to verify the integrity of the aggregate data in WSNs.
Top Level Hash
Hash 1
Hash 0
Hash 00
Hash 01
Hash 10
Hash 11
Data 00
Data 01
Data 10
Hash 11
Fig. 5 Merkle Hash Tree
5 Network Models for Data Aggregation Once the wireless sensors are dropped on to the deployment field, they need to be organized before they can start working. The organization of wireless sensor networks for data aggregation can be done in two different ways; structured or unstructured. A structured wireless sensor network can either be tree based or cluster based.
5.1 Tree Based Approach A simplified tree based network model is shown in Fig. 6. A tree based approach is the easiest hierarchical organization for data aggregation. The leaf nodes are the sources, which collect the data from the environment and send it one level up. Some intermediate nodes are designated as aggregators, which receive the data
88
V. Kumar and S. Madria base station
aggregators
sensors
Fig. 6 Hierarchical tree based network
sent from other nodes and aggregate them. The aggregators may or may not generate their own data. If they do, it is combined with the data received by them. Which sensors are designated aggregators depends upon various factors like, amount of resources left on the sensor, position of the sensor within the network, processing costs etc. Data flows from the leaf nodes upwards and is aggregated by the designated aggregators at each level until it reaches the root. Calculation of optimized paths for routing data through the tree is a complex process. This is often done at the resource rich sink node, and the sensor nodes are arranged accordingly. An evident drawback of this scheme is if a packet from a particular node is lost (due to channel issues, or due to node failure) the data of the whole subtree under that node is lost. This problem can be severe if the node is at a high level in the hierarchy. SDAP (Secure hop by hop Data Aggregation Protocol for sensor networks) [5] and PEGASIS (Power Efficient GAthering in Sensor Information Systems) [16] are examples of routing approaches based on aggregation trees.
5.2 Cluster Based Approach Another hierarchical model for data aggregation is the cluster based approach as shown in Fig. 7. This approach is very similar to the tree based approach. The sensors are divided into clusters, and one sensor in each cluster is elected as a cluster head. The election of the cluster head is based on the same factors as those discussed for aggregators in the above scheme. The sensors in each cluster send their data to the cluster head, which aggregates this data and communicates it to the sink. Cluster heads are always active and must keep their radios on at all times to receive data causing quick battery drain. Examples of cluster based routing approach are COUGAR [17] and LEACH [18]. LEACH also takes the battery drain factor into account and periodically chooses a new cluster head for each cluster.
Secure Data Aggregation in Wireless Sensor Networks
89 cluster
cluster head
sensors
base station Fig. 7 Cluster based network
5.3 Unstructured or Structure Free Approach A fixed structured approach is suitable for data gathering applications where the traffic pattern remains unchanged for a large period of time. For more dynamic scenarios such as an event based system, the overhead for constructing and maintaining a structure may be much more than the savings due to aggregation. Moreover, structured approaches also introduce a delay. A node at a high level in a structure may be forced to wait until all of the nodes below it have finished sending their data. Accurate knowledge of this delay is required for the structure to work properly which may not be possible in a dynamic environment. This has led to research on protocols which do not rely on a structure for efficient data aggregation. In [15], the authors have worked on one such structure free approach. They have designed algorithms for spatial convergence (packets should meet at the same node) and temporal convergence (packets should meet at the same time) of data, and combining them, they show that their structure free approach works better than structured approaches for event based systems.
6 General Framework for Secure Data Aggregation Any secure data aggregation protocol can be roughly divided into three main phases. The bootstrapping phase, the data aggregation phase and the integrity verification phase. Each phase carries out specific tasks and may consist of more than one algorithm. The phases may also overlap. In this section, we will go over each of these phases in detail.
6.1 Bootstrapping Phase The bootstrapping phase in any system is one where the infrastructure is initialized to carry out the intended task. With respect to wireless sensor networks, bootstrapping
90
V. Kumar and S. Madria
deals with setting up the network and the keys to carry out secure communication. As we have seen, a WSN is typically composed of a large number of sensor nodes, which makes manual deployment rather unfeasible. The deployment in most large WSNs is random. In a randomly deployed network, topology information is not available beforehand. Unknown network topology is one of the issues we have to deal with in such networks. As discussed in the Network model section, the first thing to deal with, after the sensors have been deployed is the organization of the network. Once the network has been set up the other issue we tackle in the bootstrapping phase is of secure key distribution. These keys are required to perform secure data transmission and integrity checks in subsequent phases. As the security of the network depends on these keys, it is imperative to have means of securely distributing these keys to the sensors without letting the adversary get hold of them. In a large WSN it is not feasible to individually change the configuration of each sensor. Therefore, the sensors (i) use pre-distributed keying information (ii) exchange information with each other and/or (iii) exchange information with computationally robust nodes to establish a secure network [3]. Key distribution in wireless sensor networks can be done in three ways, pair-wise, group-wise and network-wise key distribution [3]. Let’s examine each of them one by one. 6.1.1 Pair Wise Key Distribution The easiest way of doing pair-wise key distribution is to have pre distributed keys on sensors before they are deployed. After deployment these sensors communicate with their neighbor using the common keys they share. We review some of the pair wise key distribution techniques here. Master Key Based Solutions One simple solution is to store a master key on all the sensors before deployment. After deployment the sensors communicate with their neighbors, and achieve a new pair wise key using the master key. The disadvantage here is that if one of the nodes is compromised, the whole network is compromised as it is easy to derive all the pair wise keys. Pair Wise Key Pre Distribution On the other hand, we can also have each sensor store distinct pair wise keys for all possible pair of sensors. If our network consists of N sensors then in this scheme, each sensor will have N-1 pair wise secret keys stored on it. The scheme has good resilience against the adversary, but it’s impractical to store all the N-1 keys on each sensor. Moreover the scheme is not scalable as once the network has been deployed, it’s not possible to add new nodes in the network such that the older nodes will be able to communicate securely with the new ones. Random Key Pre Distribution This scheme addresses the redundancy in the above scheme and yet provides good resilience. Instead of storing all N-1 keys on each sensor a subset of these keys are
Secure Data Aggregation in Wireless Sensor Networks
91
stored. Each sensor is pre loaded with Np keys, where N is the total number of sensors in the network and 0 < p ≤ 1, is the probability that two nodes will share a common key. In the key setup phase a pair wise key is generated for each ID pair, and Np keys are stored on each sensor along with the ID. In the shared-key discovery phase, the sensors broadcast their IDs to all their neighbors. Neighboring nodes can check whether they have the key for the node ID they received and then use the common key they share for communication. This scheme sacrifices key connectivity for scalability. Key connectivity can be varied as a function of the probability p. Location Based Pair Wise Key Distribution If location information is known beforehand then it can be exploited for key distribution. Liu et al’s scheme takes advantage of location information to improve key connectivity while requiring minimum storage. The idea is if the location of a node can be predicted, it can be pre loaded with the pair wise keys for its c closest neighbors. So each sensor SA is pre loaded with its own key KA and keys for its c neighbors KB1, KB2…. KBc from which the pair wise keys can be generated. If the deployment error is low, key connectivity will be high despite having stored only c keys. 6.1.2 Group Wise Key Distribution Group wise key distribution techniques are useful for hierarchical networks, like the cluster based approach, where the cluster head may need to multicast a message. Nodes in the cluster must share a group key to be able to listen to the group communication. Group key distribution can be of the following types. Group Key Distribution Using Pair Wise Keys In this scheme [32], we assume a prior pair wise key distribution such that a node SA shares pair wise keys KBi with its neighbors SB1,SB2,…,SBk. Node SA which wants to communicate with all its neighbors first generates a unique group key KAg and then sends this group key to its neighbors as ENCKA,Bi[ KAg]. The security of this scheme depends upon the security of the underlying pair wise mechanism. Polynomial Based Group Wise Key Distribution This scheme [33] uses a secret multivariate polynomial P(x1, x2,… xk,) whose shares are distributed across the k nodes in the group. Nodes in the group can generate a common group key by evaluating this polynomial. Asymmetric Group Wise Key Distribution In this scheme [34], each sensor is preloaded with ECC domain parameters. In the key setup phase the sensors use the domain parameters to calculate their public
92
V. Kumar and S. Madria
private key pairs, in the key distribution phase, the public key is distributed to all nodes within the group. 6.1.3 Network Wise Key Distribution In sensor networks, the base station needs a secure way to broadcast its messages. This can be done using a network wise key. Moreover a network wise key is also needed to maintain end to end privacy. Different approaches for a network wise key distribution are discussed below. Master Key Based Solution A naïve approach will be to pre distribute a single key to all the sensors in the network. Sensors can then communicate with the base station using this key. This is inherently insecure as compromise of even a single node will lead to the whole network being compromised. Asymmetric Network Wise Key Distribution The base station generates a public private key pair and distributes the public key within the network. Any node wishing to communicate with the base station can communicate using the public key. In the case of end to end privacy, homomorphic encryption is used where every node encrypts the message using the public key and the data gets aggregated on the intermediate node without requiring it to be decrypted. Elliptic curve cryptography can be used for efficient usage of public key cryptography on wireless sensors.
6.2 Secure Data Aggregation Phase The objective of this phase is to aggregate data securely which can be broken down into two sub-objectives. First, secure the communication between nodes using the keys obtained in the bootstrapping phase and second, aggregate data using this secure channel. Data aggregation schemes can be classified into two groups based on which type of encryption is used, hop by hop schemes and the end to end schemes. In hop by hop encryption schemes, the sensing nodes generate data, encrypt it and send it to the aggregator; this encrypted data is decrypted at the aggregator nodes, and encrypted back after aggregation. This process is repeated at every node on the way to the base station. Since encryption and decryption are done at each hop, it is termed hop by hop encryption. In the latter, the sensing nodes encrypt the data but only the base station has the ability to decrypt it, the data is not decrypted at the intermediate nodes. Since, encryption and decryption is done only at the ends, it is termed end to end encryption. The main issue we deal with in this phase is of data confidentiality. The way it is achieved differs depending on which scheme we are using hop by hop or end to end.
Secure Data Aggregation in Wireless Sensor Networks
93
6.2.1 Hop by Hop Encryption In the bootstrapping phase, keys are securely distributed to the sensors. These keys are brought into use in the data aggregation phase. In the hop by hop encryption schemes the sensors share keys with their parents. Before sending the data over to its parent, a sensor encrypts it with the pair wise key shared with its parent. The encryption can either be symmetric key or asymmetric key. The parent decrypts the data using the shared pair wise key and aggregates all such data received from its children. This aggregate is then encrypted by the pair wise key the parent shares with its parent and sent over. This process continues till the data reaches the base station which decrypts it in a similar fashion. 6.2.2 End to End Encryption In hop by hop encryption, the data becomes vulnerable while at the sensor nodes as this is where it is decrypted to perform the aggregation. If a node is compromised, the attacker can easily get hold of the insecure data. End to end encryption provides protection from this, as the data is not decrypted anywhere in the network, but the base station. Also, since the encryption and decryption operations are computationally expensive and time consuming, end to end encryption helps save resources. To achieve end to end security, we need a way to aggregate data without having to decrypt it during the time it is in the network. This can be done using homomorphic encryption algorithms as discussed earlier. In the key distribution phase the base station generates a public-private key pair and distributes the public key on to the sensor nodes. The sensors use this public key to encrypt the data and send it to their parent. The parent receives the encrypted data from all its children. Since, the encryption algorithm is homomorphic, aggregating the encrypted data results in aggregation of the plaintext data. This aggregated encrypted data is sent further up the hierarchy. Similar aggregation takes place at all the aggregators and the base station receives the entire aggregated data. Decrypting this data using the base station’s private key, gives us the aggregate of the sensors reading. To further strengthen the security, multiple layers of encryption can be used. Onen and Molva use the CTR encryption which is an additive homomorphic encryption to provide multiple layers of encryption in their scheme. At any point in the network, the data is behind multiple layers of encryption. Each sensor on the path to the base station removes some of these layers, adds data in a homomorphic fashion, then adds new layers of encryption. Only the base station is able to remove all of the layers and decrypt the data. In case there is a compromised node in the network, multiple encryptions ensures that this node will not be able to decrypt data not meant for it. In our example application, the sensors will use the keys generated in the bootstrapping phase to encrypt the data, before it is sent over the wireless channel. This will ensure data confidentiality by preventing the adversary from getting data while it is in transmission.
94
V. Kumar and S. Madria
6.3 Integrity Verification Phase At the end of the data aggregation phase, the base station receives encrypted data from the network. Encryption of data ensures that any unauthorized party, which does not have the encryption key, is not able to see the data. Even though confidentiality is maintained, some questions still remain unanswered. What if a malicious node inside the network injects false data? What if a compromised aggregator manipulates the data? The objective of the integrity verification phase is to answer these questions. Going back to our example we are looking to tackle the situations where a compromised node tries to pass incorrect information or an aggregator tries to suppress the information about spotting an enemy vehicle. Our purpose is to make sure that any tampering with the original data is detected. Most of the work for integrity verification is done in parallel to the data aggregation phase. Digital signatures are the most common method to keep a check on the integrity of data. After sensing the data in the data aggregation phase, in addition to the encrypted data, sensors also generate a signature on it and send it along with the data. In a hop by hop scenario, every time an aggregator aggregates the data it is also supposed to generate a new signature before sending the data over. This way the signature and the data travel all the way up to the base station. To verify the integrity of data the base station can use the signature, the aggregate data and can query the aggregators repeatedly for verification at each level. In an end to end scenario this scheme no longer works. Since the aggregators do not have access to the data they are not able to generate signatures on the aggregate. This problem can be solved using the aggregated digital signatures. Recalling from the background section an aggregate digital signature scheme is one which is able to aggregate n different signatures on n different messages by n different users such that the aggregate signature can be used to verify the aggregate of n messages. In an end to end scenario, the children nodes are made to sign their messages using an aggregate signatures scheme. When the data and the signatures reach the parent, it aggregates them both and sends them forward. The aggregate signature along with the aggregate data can be used for integrity verification. A popular technique for ensuring data integrity is using Merkle hash trees. Merkle hash trees can be used both in hop by hop and end to end schemes. The sensors in a cluster send data and a hash value for the data to the cluster head. This hash is used as a commitment for the data. The cluster head maintains a tree of all the received hashes. In the integrity verification phase the base station can query the cluster head for the original data and hashes to verify whether the data it received is consistent with the one committed to the cluster head.
7 Secure Data Aggregation Schemes In this section we review some secure data aggregation schemes and relate to the secure data aggregation elements discussed before to see how all of them fits together. We review each phase of the scheme, and the algorithms used therein.
Secure Data Aggregation in Wireless Sensor Networks
95
Let’s take the example of an area monitoring application. Wireless sensors are deployed over the area. These sensors sense the environment, generate the temperature readings and send them to the base station, where they can be used appropriately. The sensors can be organized in one of the following two ways after being deployed, either in the form of clusters or a tree hierarchy. Schemes 7.2, 7.6 and 7.8 use a clustering approach while 7.1, 7.3, 7.4, 7.5 and 7.7 organize the sensors in the form of a tree. Fig. 8 shows these two types of network models. In case of a fire, the darkened nodes which are the ones closest to the fire, detect the abnormally high temperature and send the reading to the base station. How this information reaches the base station, depends on the particular scheme being used which will be discussed below. The base station after receiving the information may take necessary steps to control the fire.
Base station
Base station
(a)
sensors
(b)
sensors
Fig. 8 (a) Clusters of sensors, (b) Tree of sensors. The arrows show the direction of data flow. Data may include the encrypted reading, node id, MAC, digtal signature and other information depending on the scheme being used
7.1 Secure Aggregation for Wireless Networks (SAWN) This paper [27] by Hu and Evans is one of the early papers on secure data aggregation. The scheme proposed is based on hop by hop encryption, as the data is decrypted before being aggregated and encrypted at every intermediate node. The authors focus entirely on data integrity, with just a mention of symmetric key cryptography for encryption of data. The paper exploits two main ideas, delayed aggregation and delayed authentication for resilience against node compromise.
96
V. Kumar and S. Madria
Bootstrapping Phase The authors assume a self organizing protocol which is used to form a tree hierarchy, where each node has an immediate parent. μ-TESLA is adopted for authentication of broadcast messages transmitted by the base station. The base station generates a key chain using a one way function F where Ki = F(Ki+1).Each sensor is preloaded with K0 before deployment where K0 = F n(K). For confidentiality it is assumed that the sensor nodes are initialized with a symmetric secret key KAS shared with the base station. This key can be used to derive encryption keys using a counter. The base station knows KAS and can synchronize its counter value. Data Aggregation Phase Each leaf node senses data and sends the reading to its parent along with its node id and a message authentication node MAC (KAi, Ri). The key used for generating MAC is known to the node and the base station, but not to other sensor nodes. The parent node cannot yet authenticate the data. It just retransmits all the readings and the MACs it received from its children to its parent, along with the MAC of the aggregate readings. The grandparent now does the aggregation of the readings. This process is repeated till the aggregate data reaches the base station. Integrity Verification Phase The base station verifies the aggregate data using the MAC it receives. To enable intermediate sensor nodes to authenticate the data they received, the base station broadcasts the temporary nodes keys using μ-TESLA. The nodes can now verify the data they received, and send out an alarm if a forged message is detected. The protocol is designed to ensure that a single compromised node is not able to make false claims about other node’s readings, but it can be done once a node and its parent collude. Moreover, retransmission of readings incurs additional overhead which increases if an intermediate node has a lot of children.
7.2 A Secure Data Aggregation and Verification Protocol for Sensor Networks (SecureDAV) SecureDAV [5] is a hop by hop encryption scheme proposed by Mahimkar and Rappaport. The paper is divided into two parts, first dealing with the cluster key establishment (CKE) and the second deals with the verification of encrypted data. The network is cluster based where each cluster has a designated cluster head, which communicates with the base station at one end and with the sensors of the clusters at the other end. Bootstrapping Phase Elliptic curve cryptography is used to securely bootstrap the keys. Each sensor node is assumed to be preloaded with the ECC domain parameters and the EC
Secure Data Aggregation in Wireless Sensor Networks
97
public key of the base station. The ECC domain parameters are used to derive the public-private key pair and the public key is broadcasted in the cluster. This key pair is used for secure communication within the cluster. The paper presents the CKE protocol which distributes a secret share to each of the sensor within a cluster. This share is used to generate partial signatures on the data. The shares are such that all subsets of t + 1 shares provided by honest players define the same unique secret key, where t = n/2 and n is the number of sensors in the cluster. Data Aggregation Phase The sensors transmit their encrypted readings and their hashes to the cluster-head. In each cluster, the cluster head computes the average of the aggregated data and broadcasts it within the network. The sensors compare this average with their reading, and if the difference is within a threshold, a partial signature using the shared secret is generated and sent to the cluster head. The cluster head combines all of the partial signatures into a full signature and sends this to the base station along with the data. An attacker cannot regenerate the cluster key, unless it compromises more than t nodes. Integrity Verification Phase The threshold signature provides authenticity of the data. To provide data integrity a Merkle hash tree is used. When the sensors send encrypted data to the cluster head, they also send the hash of this data. The cluster head uses this hash to generate the hash tree. To verify any particular reading the base station queries the cluster head for the original reading and the other hashes of the tree. Using these hashes and the reading the base station generates the top hash and compares it with the one it received.
7.3 A Secure Hop-by-Hop Data Aggregation Protocol for Sensor Networks (SDAP) SDAP [6] is a hop by hop data aggregation scheme. It uses a novel probabilistic technique to divide the tree into smaller groups so as not to place more trust on higher level nodes in the tree. It then again uses a probabilistic technique for integrity verification. Bootstrapping Phase The network model in the paper is assumed to be of a tree, rooted at the base station. SDAP does not rely on any specific tree construction algorithm as long as the network works as a tree. It is assumed that the base station cannot be compromised, and it has a secure mechanism like μ-TESLA to authenticate its broadcasts. Every node shares a secret key with the base station and each pair of neighboring nodes share a pair wise key.
98
V. Kumar and S. Madria
Data Aggregation Phase SDAP recognizes that in a hierarchical approach, the nodes placed near the base station carry the aggregates of nodes under them. If such a node is compromised, false values from it will have more impact than other nodes in the tree. To avoid placing more trust on higher level nodes, the authors use a probabilistic technique to divide the tree into multiple logical groups of equal size. A group leader is elected in each group which receives data from all the group members. The nodes in the group send their id, the data encrypted by the pair wise key and a MAC generated on the data to the group leader. The group leader aggregates this data and sends it to the base station over multiple hops. Integrity Verification Phase This phase is termed the verification and attestation phase in SDAP. The base station first identifies the suspicious groups, by finding outliers using the Grubb’s test. A request for attestation is then sent to the group leader. A probabilistic path is followed down the group and the request is forwarded to each node on the path. Nodes receiving the request send back their data and id to the base station. In addition the parent nodes also send the MAC they calculated. The base station uses all this information to verify the data it received earlier.
7.4 Secure Hierarchical In-Network Aggregation in Sensor Networks (SHDA) The paper [14] discusses a hop by hop secure data aggregation scheme which is resilient against multiple malicious nodes. Symmetric key encryption is used in the data aggregation phase and a distributed scheme is used for integrity verification which aids in reducing congestion near the base station. Bootstrapping Phase As the name suggests, the scheme uses a hierarchical tree based structure. The tree may or may not be optimized and can be constructed using any tree construction algorithm like TAG. The network also contains a base station and a querier. Each sensor is initialized with a unique identifier s and shares a secret key with the querier. The sensors are also able to perform symmetric key encryption and decryption. Additionally, a secure broadcast mechanism like μ-TESLA is assumed to exist. Data Aggregation Phase After receiving the query for data from the base station, sensors generate data, encrypt it and send it over to the base station. The aggregator then aggregates the data and generates a hash for it. The hash and the data are passed on up the
Secure Data Aggregation in Wireless Sensor Networks
99
hierarchy. The aggregator one level up, commits it by computing a hash on all of the hashes it received. This process goes on until the top hash and the aggregated data reaches the base station and hence the querier. The commit process at every node, creates a hash tree in the network. Integrity Verification Phase In this phase, known in the paper as the result checking phase, the committed hashes are used to check the data for integrity. Unlike most of the other schemes, this paper discusses a distributed integrity verification technique which is as follows. Upon receiving the final commitment value, the base station disseminates it into the network. At the same time, aggregators also disseminate information which will allow other sensors to verify whether their data was combined into the aggregate or not. Sensors, after verifying this send an authentication code up the aggregation tree. These authentication codes are aggregated along the way. The querier after receiving the aggregated authentication code can verify whether all the nodes have confirmed their data in the aggregate or not.
7.5 Secure Hierarchical Aggregation in Sensor Networks (SHA) This secure data aggregation scheme [1] proposed by Julia Albath and Sanjay Madria is an end to end encryption scheme. The paper uses homomorphic encryption for confidentiality and aggregate digital signatures for data integrity purposes. Bootstrapping Phase The authors assume a tree based hierarchical network model. The paper uses elliptic curve cryptography for both confidentiality and data integrity. Elliptic curve domain parameters are assumed to be preloaded on each sensor node along with the base station’s public key and a network wide integer. This integer is used to generate a new random integer k at set intervals. Data Aggregation Phase The paper uses a variant of Elliptic curve ElGamal (ECEG) algorithm for homomorphic encryption. ECEG is additively homomorphic. The data generated by the leaf nodes is encrypted using this algorithm and sent to the parent. The parent collects the encrypted data from all its children, performs a homomorphic aggregation and sends it to its own parent. Data travels up the hierarchy likewise until it reaches the base station. Integrity Verification Phase The paper uses ECDSA to generate digital signatures which can be aggregated on the parent. The network wide integer is used to generate a new public private key pair for each round of processing. A new encryption key each time is necessary because, it is easy for an adversary to discover the key if more than one message is
100
V. Kumar and S. Madria
signed using the same key. The sensors sign the message using the private key and send the public key and the signature along with the message in the data aggregation phase. The parent not only aggregates the data, but also aggregates the signatures and the public keys of all its children. The base station finally receives all aggregate data, aggregate signatures and aggregate public keys of all the nodes in the network. Verification can be done by using the aggregate public key to decrypt the signature and comparing this to the hash of the aggregate data.
7.6 An Efficient and Verifiable Concealed Data Aggregation Scheme in Wireless Sensor Networks (CDA) This end to end encryption scheme [8] proposed by Sun et al, combines Mykletun et al’s concealed data aggregation with Boneh and Gentry’s aggregated signature scheme. Mykletun et al’s concealed data aggregation (CDA) has the property of additive homomorphic encryption, and is based upon elliptic curve ElGamal ECEG cryptosystem. The aggregated signature scheme is based on bilinear maps, with the property that if there are n users and the size of each signature is m, the combined signature is of size m rather than n*m. Bootstrapping Phase Each sensor node is pre loaded with its private key and the base station’s public key. The sensor nodes are also able to calculate a secure hash function H. The sensors after deployment divide themselves into clusters and elect a cluster head. Data Aggregation Phase Before a sensor sends its data to the cluster head it encodes the data in a special format. The encoding is in such a manner that, individual data can easily be extracted from the aggregation. This has a direct impact on the size of the data. A large number of sensors in the cluster would mean a very lengthy message. Each sensor also signs its message and sends it to the cluster head. After, the cluster head has received data from all other sensors in the group. It aggregates them using the point addition on the elliptic curve field. It also aggregates the signature it receives. Integrity Verification Phase The base station decrypts the data it receives and then decodes it to reveal the individual readings. These individual readings then are used for verification along with the aggregate signatures.
7.7 Efficient Aggregation of Encrypted Data in Wireless Sensor Networks (EDA) This paper [28] presents a secure data aggregation scheme based on homomorphic encryption. Homomorphic encryption allows the aggregator to execute aggregation
Secure Data Aggregation in Wireless Sensor Networks
101
functions on the encrypted data without decrypting it, which makes it an end to end scheme. The authors have only dealt with data confidentiality and have not touched upon the data integrity part. Bootstrapping Phase It is assumed that each sensor shares a distinct long term key Ki with the base station. This key is derived from a master secret, which is only known to the sink. The scheme uses a hierarchical network model, but no details have been specified. Data Aggregation Phase For confidentiality protection, the authors use an encryption scheme which is additively homomorphic. The encryption scheme is based on modular arithmetic as shown below. c = Enc (m,k,M) = m +k (mod M) m= Dec (c,k,M) = c – k (mod M) where, m is the message mє[0, M-1], c is the ciphertext and k is the key. This encryption scheme is additively homomorphic in nature, which allows the encrypted data to be added as it is forwarded towards the base station. Integrity Verification Phase This scheme does not tackle the issue of data integrity, however the authors do mention that authentication in WSN can be used to prevent unauthorized nodes from injecting false data, which can be efficiently performed in a hop by hop manner.
7.8 A Secure Data Aggregation Scheme for Wireless Sensor Networks (SDA) The scheme [7] proposed by Ren et al is an end to end encryption scheme. The end to end part is achieved using the Domingo-Ferrer privacy homomorphism (DFPH) which is resilient against known cleartext and ciphertext only attacks. The scheme uses groups inside clusters to have resilience in the event of some nodes being compromised. Bootstrapping Phase Prior to deployment, each sensor node is loaded with a list of key and key index pairs. A collection of encryption keys is selected from a key pool to form a key set to be used. The sensors form clusters after deployment. Each node builds an
102
V. Kumar and S. Madria
authentication key with the cluster head using ECDH and sends its key index list to it. The cluster head after receiving the key lists from all of the sensors assigns a particular key index to each sensor. Sensors sharing that key index are thus, logically partitioned into subgroups. Data Aggregation Phase Sensors have the secret key corresponding to the key index assigned by the cluster head. The data is encrypted using this key. The cluster head collects this data and aggregates it group wise, which is then sent to the base station which can easily decrypt it. Integrity Verification Phase Along with the data, the sensors also send the MAC of the data, encrypted by the pair wise key shared with the cluster head. The cluster head uses this MAC for checking the integrity of the data.
8 Comparison of Schemes In the previous section, we have discussed various secure aggregation schemes. In this section, we provide a comparison of the above discussed schemes. Let’s do a relative comparison of the schemes we have discussed in this section. The first secure data aggregation scheme we discussed was SAWN [27] which achieves resilience by delaying the aggregation and authentication at the upper levels. This implies that buffer space is needed to hold the data until it is aggregated. The scheme only offers data integrity as the paper does not talk about achieving data confidentiality. It also does not take any action once it detects a compromised node. Moreover, it is possible to alter data if a parent and child collude. SecureDAV [5] improves the SAWN scheme by introducing digital signatures and a hash tree for integrity verification. Network is divided into clusters and sensors in a cluster are provided with a unique share of a secret key using which they generate partial signatures on the data. Data integrity is maintained until the attacker is able to compromise a certain number of (threshold) sensors. Like SAWN, SecureDAV too is a hop by hop technique but makes use of elliptic curve cryptography. The drawbacks are that it has a high communication cost of data validation and can only support the AVERAGE function. EDA [28] approach proposed by Castelluccia et al. unlike the above two schemes focuses on the data confidentiality part. The scheme is based on homomorphic encryption so that there is no need of decryption of the ciphertext at intermediate nodes for aggregation purposes which makes the scheme end to end and saves data from being exposed at the nodes. The scheme performs well in a non noisy environment, but generates significant overhead in a noisy unreliable channel, as the identities of the nodes not participating in aggregation has to be sent along with the data. Next in line is the SHDA [14]
Secure Data Aggregation in Wireless Sensor Networks
103
algorithm proposed by Chan et al. in 2006. It is a hop by hop secure data aggregation scheme which is resilient against multiple malicious nodes. Symmetric encryption is used for encryption and a distributed aggregate-commit-prove framework is used to reduce network traffic near the base station. The scheme handles both data integrity and confidentiality. However, too much transmission and computation as a consequence of distributed integrity check may be counted as its drawbacks. Like SecureDAV [5], SDAP [6] algorithm is capable of handling more than one compromised nodes. SDAP is based on two main ideas, divide and conquer and commit and attest. First it divides the aggregation tree into smaller logical subtrees so as to reduce the trust on high level nodes, and then uses commit and attest for data integrity preservation. The survey in [29] shows that data transmission in SDAP is relatively higher than other schemes. When the number of children of each node is large, the number of transmissions is equivalent to a non aggregation scenario. Next scheme SDA [7] is an end to end scheme which uses homomorphic encryption. Like SecureDAV and SDAP, this scheme shows resilience even if some of the nodes are compromised by dividing the sensors in a cluster into groups. Sensors within a group share a key and the data is aggregated group wise. Like SDA, CDA [8] too, is an end to end scheme which uses homomorphic encryption. It also uses aggregate signatures to provide data integrity. The scheme uses Domingo Ferrer privacy homomorphism (DFPH). However, it is proven that PH is unsecure against chosen plaintext attacks. CDA as pointed out in [29] is computationally expensive and increases the power consumption of the sending node. Moreover, the scheme is not scalable. Number of nodes in the network has a direct impact on the size of the message which is communicated. The last scheme we discussed is SHA [1] which like the previous two schemes is end to end and preserves data integrity by using additive digital signatures. The drawback though is that the signature scheme can only verify if the data has been tampered with or not, it cannot identify the malicious node. In the tables given below we provide a brief summary of the various protocols in terms of different parameters. For the definition of the abbreviations refer to Section 7. Table 4 A comparison of various secure data aggregation schemes Scheme
HBH
SAWN SecureDAV SDAP SHDA SHA CDA EDA SDA
X X X X
ETE
Symmetric key
Public Homomorphic Integrity key Encryption Verification
X X X X X X X X
X X X
X X X X
X X X X X X X
104
V. Kumar and S. Madria
Table 5 Detailed comparison of secure data aggregation schemes showing the algorithms used in various phases. EC-PKC refers to Elliptic curve public key cryptography, EC EG is Elliptic curve ElGamal, DFPH is Domingo-Ferrer privacy homomorphism and EC DSA is Elliptic curve Digital Signature Algorithm Scheme
Network Model
SAWN SecureDAV
Hierarchical Clustered
SDAP
Hierarchical
SHDA SHA CDA
Hierarchical Hierarchical Clustered
EDA
Hierarchical
SDA
Clustered
Encryption Scheme
Integrity Veri- Key Distribution fication Scheme Scheme
Symmetric key MAC Not specified EC-PKC Merkle Hash Asymmetric Tree (MHT) group wise key distribution Symmetric key MHT with a Not specified probabilistic scheme Symmetric key Hash tree Not specified EC EG EC DSA Not specified Mykletun’s Boneh and Not specified CDA Gentry’s Mod stream ci- Not specified Not specified pher DFPH MHT Random Key Predistribution
9 Conclusions and Future Directions In this chapter, we discussed some of the security primitives like elliptic curve cryptography, homomorphic encryption, Merkle hash trees and aggregated digital signatures which are important in designing secure data aggregation schemes. Along with these, we introduced a general framework for secure data aggregation. At the end, after having gone through the security basics and the framework, we reviewed some secure data aggregation schemes to put everything in perspective and see how the bits and pieces all work together. Most data aggregation techniques use static network topology, i.e. the clustered or tree based hierarchical approaches. These approaches work well in environment monitoring applications, but may not be suitable for a more dynamic event based system. Approaches like structureless data aggregation discussed in [15] are better suited to dynamic environments. However this is a relatively new direction and requires further research. Further, we also note that the end to end encryption schemes until now have been restricted around the additive and multiplicative homomorphic encryption. A fully homomorphic encryption scheme like the one proposed by Gentry [10] can release this restriction. Such an encryption scheme will enable us to design more efficient and robust systems. In-network data aggregation in wireless sensor networks provides many benefits in exchange for a small additional computational cost. Reduced energy
Secure Data Aggregation in Wireless Sensor Networks
105
consumption is one such benefit, which is paramount for battery operated sensors. This and other benefits like reduced data redundancy and the ability to provide services directly, make in-network data aggregation on sensor networks a very attractive proposition. However, we need to figure out the security issues in a cost effective manner before we can harness the potential of data aggregation. Techniques like elliptic curve cryptography, homomorphic encryption and aggregated signatures are positive steps in this direction, which allow us to implement robust and yet energy and cost wise economic solutions.
References [1] Albath, J., Madria, S.: Secure Hierarchical Aggregation in Sensor Networks. In: Proceedings of IEEE Wireless Communications and Networking Conference (2009) [2] Perrig, A., Stankovic, J., Wagner, D.: Security in wireless sensor networks. Communications of the ACM 47, 53–57 (2004) [3] Camtepe, S.A., Yener, B.: Key Distribution Mechanisms for Wireless Sensor Networks: a Survey. Rensselaer Polytechnic Institute. technical report TR-05-07 (March 2005) [4] Merkle, R.: A certified digital signature. In: Brassard, G. (ed.) CRYPTO 1989. LNCS, vol. 435, pp. 218–238. Springer, Heidelberg (1989) [5] Mahimkar, A., Rappaport, T.S.: SecureDAV: A secure data aggregation and verification protocol for sensor networks. In: Proceedings of the IEEE Global Telecommunications Conference (2004) [6] Yang, Y., Wang, X., Zhu, S., Cao, G.: SDAP: a secure hop-by-hop data aggregation protocol for sensor networks. In: MobiHoc 2006: Proceedings of the seventh ACM international symposium on Mobile ad hoc networking and computing (2006) [7] Ren, S.Q., Kim, D.S., Park, J.S.: A Secure Data Aggregation Scheme for Wireless sensor Networks. In: ISPA Workshops 2007, pp. 32–40 (2007) [8] Sun, H.-M., Hsiao, Y.-C., Lin, Y.-H., Chen, C.-M.: An Efficient and Verifiable concealed Data Aggregation Scheme in Wireless Sensor Networks. In: Proceedings of the 2008 International Conference on Embedded Software and Systems, pp. 19–26 (2008) [9] Acharya, M., Girao, J., Westhoff, D.: Secure Comparison of Encrypted Data in Wireless sensor Networks. In: Proceedings of the Third International Symposium on Modeling and Optimization in Mobile, Ad Hoc, and Wireless Networks, pp. 47–53 (2005) [10] Boneh, D., Gentry, C., Lynn, B., Shacham, H.: A Survey of Two Signature Aggregation Techniques. In: Cryptobytes 2003 (2003) [11] Sorniotti, A., Gomez, L., Wrona, K., Odorico, L.: Secure and Trusted in-network Data Processing in Wireless Sensor Networks: a Survey. Journal of Information Assurance and Security 2, 189–199 (2007) [12] Krishnamachari, B., Estrin, D., Wicker, S.B.: The Impact of Data Aggregation in Wireless Sensor Networks. In: 22nd International Conference on Distributed Computing Systems Workshops, ICDCSW (2002) [13] Ozdemir, S., Xiao, Y.: Secure data aggregation in wireless sensor networks: A comprehensive overview. In: Computer Networks (2009), doi:10.1016/j.comnet.2009.02.023
106
V. Kumar and S. Madria
[14] Chan, H., Perrig, A., Song, D.: A Secure Hierarchical In-network Aggregation in Sensor Networks. In: CCS 2006 (2006) [15] Fan, K.-W., Liu, S., Sinha, P.: Structure-Free Data Aggregation in Sensor Networks. IEEE Transactions on Mobile Computing 6, 929–942 (2007) [16] Lindsey, S., Raghavendra, C., Sivalingam, K.M.: Data Gathering Algorithms in Sensor Networks using Energy Metrics. IEEE Transactions on Parallel Distributed Systems 13, 924–935 (2002) [17] Yao, Y., Gehrke, J.: Query Processing for Sensor Networks. In: ACM CIDR (2003) [18] Heinzelman, W.B., Chandrakasan, A.P., Balakrishnan, H.: An Application Specific Protocol Architecture for Wireless Microsensor Networks. IEEE Transactions on Wireless Coomunication 1, 660–670 (2002) [19] Piotrowski, K., Langendoerfer, P., Peter, S.: How public key cryptography influences wireless sensor node lifetime. In: Proceedings of the 4th ACM Workshop on Security of ad hoc and Sensor Networks, SASN 2006 (2006) [20] Peter, S., Piotrowski, K., Langendoerfer, P.: On concealed data aggregation for wireless sensor networks. In: Proceedings of the IEEE Consumer Communications and Networking Conference 2007 (2007) [21] Bokareva, T., Hu, W., Kanhere, S., Ristic, B., Gordon, N., Bessell, T., Rutten, M., Jha, S.: Wireless Sensor Networks for Battlefield Surveillance. In: Proceedings of the Land Warfare Conference, LWC (2006) [22] Jonker, W., Petkovic, M.: Using secret sharing for searching in encrypted data. In: Jonker, W., Petković, M. (eds.) SDM 2004. LNCS, vol. 3178, pp. 18–27. Springer, Heidelberg (2004) [23] Alzaid, H., Foo, E., Nieto, J.G.: RSDA: Reputation based Secure Data Aggregation in Wireless Sensor Networks. In: Ninth International Conference on Parallel and Distributed Computing (2008) [24] Zhang, W., Das, S.K., Liu, Y.: A trust based framework for secure data aggregation in wireless sensor networks. In: IEEE SECON 2006 proceedings (2006) [25] Shamir, A.: How to share a secret. Communications of the ACM 22(11), 612–613 (1979) [26] Liu, F., Cheng, X., Ma, L., Xing, K.: SBK: A Self-Configuring Framework for Bootstrapping Keys in Sensor Networks. IEEE Transactions on Mobile Computing (July 2008) [27] Hu, L., Evans, D.: Secure Aggregation for Wireless Networks. In: Workshop on Security and Assurance in Ad hoc Networks (2003) [28] Castelluccia, C., Mykletun, E., Tsudik, G.: Efficient Aggregation of Encrypted Data in Wireless Sensor Networks. In: MobiQuitous 2005 (2005) [29] Alzaid, H., Foo, E., Nieto, J.G.: Secure Data Aggregation in Wireless Sensor Network: a survey. In: Proceedings of the sixth Australasian conference on Information security (2008) [30] Rivest, R.L., Shamir, A., Adleman, L.M.: A Method for Obtaining Digital Signatures and Public Key Cryptosystems. Communications of the ACM 21 (1978) [31] El Gamal, T.: A public key cryptosystem and a signature scheme based on discrete logarithms. In: Blakely, G.R., Chaum, D. (eds.) CRYPTO 1984. LNCS, vol. 196, pp. 10–18. Springer, Heidelberg (1985) [32] Dutertre, B., Cheung, S., Levy, J.: Lightweight key management in wireless sensornetworks by leveraging initial trust. T.R SRI-SDL-04-02, System Design Laboratory (2004)
Secure Data Aggregation in Wireless Sensor Networks
107
[33] Blundo, C., Santis, A., Herzberg, A., Kutten, S., Vaccaro, U., Yung, M.: Perfectlysecure key distribution for dynamic conferences. In: Brickell, E.F. (ed.) CRYPTO 1992. LNCS, vol. 740, pp. 471–486. Springer, Heidelberg (1993) [34] Steiner, M., Tsudik, G., Waidner, M.: Key Agreement in Dynamic Peer Groups. IEEE Transactions on Parallel and Distributed Systems (2000) [35] Wander, A.S., Gura, N., Eberle, H., Gupta, V., Shantz, S.C.: Energy analysis of public-key cryptography for wireless sensor networks. In: Pervasive Computing and Communications, PerCom 2005 (2005)
Data Management Solutions in Networked Sensing Systems Levent G¨urgen, Claudia Roncancio, Cyril Labb´e, and Shinichi Honiden
Abstract. Since the emergence of sensor oriented applications, sensor data management has become a very active research domain. We can distinguish three classes of solutions in this area, which we will explore in this chapter. The first is sensor networks which provide distributed processing of sensor data. The second is data stream management systems which are centralized systems that deal with stream data flowing from sensors. And finally, some hybrid approaches have also recently appeared which aim to integrate these two different types of solutions. This chapter provides an overview of these three groups of proposals and discusses their approach to deal with issues such as scalability of sensing systems, heterogeneity of sensors and processing complex operations on sensor data stream.
1 Sensor Data Processing With the advances in technology, in particular in miniaturization and wireless communication, computing devices have become increasingly autonomous, small, cheap and powerful. Sensing devices are becoming a notable part of our daily lives that facilitates monitoring physical events occurring around us. There are many kinds of sensing devices: temperature, pressure, chemical sensors, GPS devices, audiovisual devices, RFID readers, etc. They can be wired or wireless, autonomous or assisted. They are already being used in various domains: industrial applications Levent G¨urgen CEA-Leti Minatec, Grenoble, France e-mail: [email protected] Claudia Roncancio · Cyril Labb´e Grenoble University, Grenoble, France e-mail: {claudia.roncancio,cyril.labbe}@imag.fr Shinichi Honiden National Institute of Informatics, Tokyo, Japan e-mail: [email protected] T. Hara et al. (Eds.): WSN Technologies for the Information Explosion Era, SCI 278, pp. 111–137. c Springer-Verlag Berlin Heidelberg 2010 springerlink.com
112
L. G¨urgen et al.
[20] such as monitoring of production chains or container tracking; medical applications [38, 54]; environmental monitoring [59, 19, 70]; urban [26] or domotic [35] applications1. Processing sensor measured data has brought new problems in computer science. Among them, energy efficient information gathering, declarative querying of data streams, heterogeneity and scalability of sensing systems [36, 7, 40, 31]. This chapter considers important challenges and solutions for the data management in networked sensing systems. We distinguish three classes of solutions for sensor data management: • wireless sensor networks (WSN) [7]. In the rest of this chapter we will consider WSN as an ad hoc network composed of sensors having a certain autonomy of energy, as well as a capacity of storing, processing and wirelessly transmitting their measures (e.g., temperature, pressure or location data). They exploit computing capabilities of sensors to distribute data processing. Section 2 presents the basic concepts in WSN domain. • data stream management systems (DSMS) [40] are systems evaluating declarative queries over data streams. A data stream is an unbounded sequence of data ordered by their timestamp. They are generated continuously by sources such as web transaction logs, financial applications and network traffic information. DSMS can also be used for processing data streams containing measures provided by the sensors. This approach is described in Section 3. • hybrid systems integrate the two first approaches in order to benefit from their respective strengths. The objective is to better deal with the aspects related to scalability, heterogeneity, at the same time allowing semantically rich queries on sensor data. Hybrid approaches are presented in section 4. Finally section 5 concludes this chapter with a discussion on the place of these sensor data management solutions in broader systems such as ubiquitous information systems.
2 Wireless Sensor Networks (WSN) This section identifies and presents the main characteristics of WSNs in sections 2.1 and 2.2 respectively. It then deals with sensor data processing issues. In section 2.3, it introduces the role of the network level in data processing, then section 2.4 presents solutions involving higher layers adopting a ”database” approach for sensor networks. A discussion in section 2.5 concludes this section.
2.1 Introduction A wireless sensor network is an ad hoc and auto-configurable network without a fix infrastructure, composed of sensor nodes having some capacities of computing, storage, and wireless communication. Sensors dispersed in the environment measure 1
For more application examples readers can refer to [25] and [62].
Data Management Solutions in Networked Sensing Systems
113
physical characteristics or detect events. Data are sent to one or more base stations which provides the interface between sensors and applications (see figure 1).
Fig. 1 Wireless sensor networks: diagram of data being sent to a base station
Unlike traditional networks used for other various objectives, building a sensor network is motivated by a unique objective: ”collecting sensor data without needing a fixed infrastructure”. Routing and data gathering issues are tightly related, and the term ”wireless sensor network” is mostly used to refer to a platform for sensor data gathering and processing. Sensors’ embedded operating systems also have an important role in data processing. One of their main objectives is to abstract the collection of sensor measurement and to facilitate the transmission of processed data. They also play the role of basic query processors as they still do not exist at a different layer. Discussion on operating systems for sensors is out of the scope of this chapter. Interested readers can refer to [51], [48] and [18]. Collecting and processing raw sensor data and transforming them into ”information” is one of the main goals. Moreover, specific characteristics of wireless sensor networks (see next section) introduce problems that involve several computer science domains such as networks, operating systems, databases and distributed systems.
2.2 Characteristics of WSN WSNs share some characteristics of ad hoc wireless networks but also have some unique characteristics, in particular concerning their resources, communication mode and addressing capabilities. The main characteristics of WSNs can be listed as follows: Limited resources: the size of sensors may vary from some few millimeters to centimeters cubes. Consequently, this limits their computing, storage and communication capacity, despite remarkable progress in miniaturization. Energy being the most important resource. Typically a sensor is equipped with a battery (e.g., AAA type) which can last several hours or days in an active mode and several months or years in sleep mode. The lifetime of an entire sensing system depends on the individual
114
L. G¨urgen et al.
sensors. Since sensors participate to routing, data processing and auto-configuration, the unavailability of some nodes may affect the whole system’s operation. Large number of sensors: the decreasing production costs now allow deploying numerous sensors for various applications. The increasing number of nodes improves the fault tolerance and accuracy of data by the increase of redundancy. Nevertheless, it increases the complexity of auto-organization as well as the quantity of data to process. The routing protocols and data processing techniques should therefore take into account the ”high scale” factor of WSNs [36]. Data-centric routing: Unlike traditional networks, nodes in WSNs may not possess a unique address or identifier. Sensor-oriented applications are mostly data-centric [36]. In other words, applications are focused on the data collected by the sensors and not by the sensors themselves. For instance a query such as ”What is the temperature measured by the sensor with ID 3424” is an unlikely query for an application, while ”At which area sensors measure more than 40◦C” has more relevance. Nodes are identified by the properties of their data (in the example, the location and the temperature), whence the term data-centric.
2.3 Routing in WSNs Routing protocols for WSNs must handle the constraints introduced by their special characteristics mentioned above. In particular, economy of energy plays an important role in their design. The issue is to construct routes from sensors to base station(s) and to define routing protocols maximizing the lifetime of the network. Routing protocols may be classified in three categories according to the structure of the network: Flat, hierarchical or geographical [8, 6]. In flat routing, all nodes have the same responsibilities, while hierarchical solutions form sensor clusters and attribute more important responsibilities to cluster heads (for instance sensors having better energy, communication or storage capacity). Finally, geographic routing consists of addressing packets to a particular region rather than a particular node. The Zigbee alliance [10], issued from the industrial world, constitutes an effort for standardization of an energy-efficient routing protocol for resource constrained devices. It defines a specification for the network layer supporting different topologies: star, mesh or tree topologies. There already exists Zigbee certified products and it is foreseen that the standard will be largely adopted by manufacturers. More details on network related aspects can be found in [8]. In the next section we present how the routing protocols may have a major role in sensor data processing, in particular for in-network data aggregation.
2.4 Sensor Databases First sensor data processing techniques were used for specific applications: each application had its own sensors with a specific data gathering language, processing techniques and communication protocols. However the multitude of sensor-oriented
Data Management Solutions in Networked Sensing Systems
115
applications raised needs for a common way of sensor data querying by more generic solutions, reusable for different applications. ”Sensor databases” [21, 56] respond to this requirement by providing a declarative approach for sensor data gathering and processing. This evolution is actually similar to the one of database management systems (DBMS) that occurred more than three decades ago: uncoupling storage and processing of data. We consider a sensor database as a combination of persistent data containing meta-information of sensors (their ID, name, location, etc.) and instantaneous, nonpersistent and transitory data concerning measures collected on the state of the monitored physical environment. This approach considers each sensor as a data source. Thus, a sensor network is seen as a distributed database. Queries are injected to the sensor network from a base station and they are propagated via the routing protocol. Query evaluation is distributed among numerous sensors. This strategy reduces the size and quantity of data being transmitted in the network because each sensor sends its data if and only if the data conformed to the conditions specified by the query (e.g., if temperature is greater than 40◦C). It is worth noting that most existing query evaluation techniques, as they are being used by classical DBMS, do not apply to sensor databases. The reason is that they do not consider the limited resources of sensors, the uncertain and noisy sensor data and the continuous, real-time nature of queries. Declarative querying Significant research work on data management in WSNs has been done during the last few years. TinyDB [58] and COUGAR [75] were the first to propose a declarative method to express sensor data acquisition. The most common approach is to adopt a relational model for data representation and to use SQL-like languages for query formulation [58]. Sensor data are exported as tuples conforming to a given schema. Schemas are mostly sensor specific. For example, tuples produced by a temperature sensor can be of the form [sensorId, location, temperature, timestamp]. Tuples provided by a set of sensors are considered as a virtual table that is partitioned horizontally on the sensors. This virtual table is ”conceptually” updated continuously by new measurements produced by the sensors. Supporting continuous queries that take into account the most recent values in quasi real-time becomes very important. Continuous queries are evaluated continuously on data produced by the sensors in order to reflect the most recent state of the monitored environment. Continuous queries behave like processes reporting about physical events detected by the sensors in real-time. This is particularly important for monitoring applications. For instance, in TinyDB it is possible to define the periodicity of sending query results, as well as the lifetime of the query. For example, the query shown in figure 2 gives the identifier of temperature sensors that measure greater than 40◦ C in Room C every 30 seconds during 6 minutes.
116
L. G¨urgen et al.
Fig. 2 TinyDB query
[58] also provides ways for reducing data size and rate by delegating more computing to sensors. For instance, event-based queries can be defined: query is evaluated only if a defined event is detected. Data windows are another way to reduce network traffic and energy consumption. They contain most recent data measured during a given time interval. Instead of all the data in this window, only an aggregation (e.g., average, minimum maximum, histogram) is sent. One of the most efficient ways to preserve sensor’s energy consumption is to aggregate data in the network before transmitting it to the base station. This approach is developed in recent research [65]. It takes advantage of the multi-hop nature of routing protocols. Each sensor aggregates its measure with the one of its predecessor before transmitting the answer to the following sensor. This aggregation may either use a classical aggregation operator (sum, count, average, etc.) or just add its measurement to the same packet it received. The number of sent messages is minimized and energy consumption decreases compared to a solution where every sensor directly communicates with the base station [50]. Indeed, the execution of instructions on a sensor requires less energy compared to those consumed by wireless transmission of data. It is shown in [56] that the necessary energy for sending one bit of data is equivalent to about 800 instructions. Nonetheless, it is important to note that it is not possible to calculate in the network, aggregation functions that need the entire data (histogram, median, quantile, etc.) for the result.
2.5 Discussion This section presented a brief overview of basic WSN concepts, in particular on data management aspects. Database point of view is now largely adopted for declarative querying of sensor data. Those proposals try to adapt well-known techniques of database management systems to sensor networks. The main objective is to separate application needs and querying aspects to facilitate sharing of the same sensors by different applications. Research on WSN continues to explore new challenges, such as quality of service [23], mobility management [30], auto-configuration [52], heterogeneity [14], reliability [72] and security [60]. Solutions from all domains should consider the energy constraint of sensors, a limiting factor with no anticipated improvement in the near future. Initially used by experimental academic applications, sensors have now been used by many application domains (industrial, medical, military, etc) requiring better quality of service, reliability, integrity, security and privacy. These issues are thus gaining importance. Scalability is another emerging issue to consider since
Data Management Solutions in Networked Sensing Systems
117
the decreasing cost of electronics allows the deployment of large farms of sensors. The latter also raises the question of scalable management of sensing systems [42]. Moreover, management operations may consist of modifications on sensors (reconfiguration, installation of new software, parameter changes) that can have an impact on the consistency of the system. Therefore they should be performed in a way that preserves the integrity of the system [45].
3 Data Stream Management Systems In section 2, we have presented data processing approaches that are specifically conceived for wireless sensor networks. Here, we describe data stream management system (DSMS) solutions that support management of, not only sensor data, but any data that is continuously generated in stream [40]. Research work on this topic converge to the following definition for data streams. A data stream is an unbounded sequence of data ordered by their timestamp. Management of data streams by traditional DBMS is problematic due to their special characteristics. In section 3.1 we give these characteristics by discussing the main differences between DBMS and DSMS, which lead us to precise the term ”data stream”. The section 3.2 then presents how data streams are processed continuously by DSMS. Moreover, in these systems, queries should be evaluated in quasi realtime, thus resistance to the overload due to frequent data items generated by many data sources is a very important point which is discussed in section 3.3. Finally, the section 3.4 proposes a synthesis of this approach.
3.1 DSMS vs. DBMS and Data Streams The main objective of both DSMS and DBMS is to provide generic data management for applications. However there are differences on the nature of data to manage and queries to evaluate. Classical DBMS offers ways of querying stored persistent data. Queries are transitory in the sense that they are evaluated on the existing data at the time of the reception of the query and are discarded after that. On the other hand, data processed by the DSMS are transitory streams and they are temporarily stored in a waiting queue. Since the quantity of storable data is limited, data are simply removed once they are processed. Queries should be persistent and evaluated continuously in order to take into account newly arriving stream data (see figure 3). Among the most effective applications, we can mention control of network traffic [29], analysis of transaction logs (e.g., financial or telecommunication transactions) [28], monitoring of actions on dynamic web pages [24], and management of sensor data [11]. These applications motivated many proposals of DSMS in the research community, among them, STREAM [11], Aurora [1], and TelegraphCQ [22]. Besides, there are some recent commercial offers issued from research [27, 67, 9]. Their goal is to allow fast decision taking and rapid reaction for business applications, in which data processing time is critical.
DSMS adopt a data stream model inspired by the sequential model [63] which is an extension to the well-known relational model. Tuples are ordered by the value of an attribute (in general the timestamp). They conform to a data schema that depends on the target application. For example, a simple schema for temperature sensor data: sensors(id, location,temperature,timestamp) {< i, l,t,ts > ∈ sensors if the sensor of identifier i and of location l measures the temperature value t at instant ts. } The unbounded nature of data streams introduces problems for blocking operators [15]. Indeed, these operators need the entire input to produce an output. Creating finite ”data windows” over infinite data streams is one way to perform these operators. Existing proposals differs on query evaluation techniques and window representations that are presented in the following section.
3.2 Continuous Queries on Data Streams Queries on stream data should be evaluated continuously in order to take into account the new values appending to streams. As in sensor networks, queries in DSMS are continuous. However, one has to note the little nuance on periodicity of query execution. In fact, in sensor networks, this periodicity is defined by sensor’s data sampling and sending rate, while DSMS do not control that data sampling and sending rate. Consequently, the periodicity is not to be defined by the queries, except for ones that are evaluated on periodically sliding windows (explained below). Queries consist of a set of operators which consume data as input, execute operations, and produce results in a continuous manner. A result stream can be used by another operator. Extended variants of SQL are usually used in this context (e.g., STREAM [11] and TelegraphCQ [22]). The syntax of queries on data streams is close to the syntax of queries addressed to traditional DBMS, but their evaluation is quite different as mentioned in section 3.1. Query languages for data streams are not all
Data Management Solutions in Networked Sensing Systems
119
formalized. The most serious effort comes from STREAM with the Continuous Query Language, CQL [12]. Alternatively to the extended SQL approach, Aurora [1] proposes a graphical interface to express queries. Users place boxes representing operators and arrows representing data streams and interconnect them to create queries. A semi-structured approach comes from the Niagara project [24], where XML document streams are queried by a language similar to XML-QL [34]. Windows Processing potentially infinite data streams is not really problematic when the query concerns simple operations such as selection according to a filter or projection of data on some attributes. However, if a query contains blocking operators [15], infinite data stream must be reduced to a finite set. It is the case for join and certain aggregation operators such as histogram and median. DSMS use two types of windows to evaluate blocking operators: • time-based windows: their size is defined by a time unit. For instance a window of size of 10 minutes is used for the following periodically executing continuous query "Send every minute, the average of the temperature measured since 10 minutes" • count-based windows: their size is defined by the number of tuples contained in the window. They are used for queries such as "Send every minute, the average of the last 10 temperature measurements" Windows bounds usually shift to integrate the new values continuously arriving (see figure 4). They are called ”sliding windows”.
Fig. 4 Example of a window of size 6 seconds and sliding every 4 seconds
Windows may have different behaviors for the shifting of their bounds (e.g., periodically sliding windows, tumbling2, landmark3, etc.). Methods of defining these behaviors vary. STREAM uses two expressions for defining two types of windows: 2 3
Intersection of two consecutive window is empty. Only one of the bounds of the window shifts, other stays fix.
120
L. G¨urgen et al.
[ROWS n], to specify a count-based window of n elements; and [RANGE t seconds], to specify a temporal window of t seconds. For instance, a query formulated in accordance with the schema sensors given in section 3.1, which requests the average of measures since 30 seconds, is shown in figure 5.
Fig. 5 Windowed query with STREAM
The answer stream contains averages of measures collected during the last 30 seconds since the last iteration of the query evaluation. The evaluation is continuous and stops when there is no more input data. TelegraphCQ uses for loops in order to define more generic temporal windows (e.g., landmark, fix, sliding windows) by specifying the shifting policy and the number of units to shift on each loop iteration [22]. The syntax of window definition is shown in the figure 6.
Fig. 6 Window definition with TelegraphCQ
Thus with TelegraphCQ the preceding query can be written as shown in figure 7.
Fig. 7 Windowed query with TelegraphCQ
Let us note that CQL is limited to sliding windows and TelegraphCQ does not define temporal windows. Moreover, none of them allows defining the periodicity for sliding time of windows. This period is implicit: each time that there is a new entry in the window. [43] proposes a generic definition of windows allowing to create windows of different types and behaviors. Evaluation of continuous and complex queries makes optimization particularly critical. Next section briefly presents the issues related to processing of highly frequent and dynamic data processing.
Data Management Solutions in Networked Sensing Systems
121
3.3 Resistance to Overload Data arrival rate is not controlled by the DSMS and may be unpredictable. Systems should dynamically adapt their behavior to tolerate high arrival rates. If the system is not fast enough to process all tuples of a data stream, several solutions may be considered. For instance, it is possible to use waiting queues and in case of overflow of these queues, certain tuples can be ignored or they can be aggregated (e.g., an average or a histogram). For example STREAM [73] sacrifices the exactitude of query results by proposing approximative query results in order to better manage the system resources. Besides, dynamic re-optimization of queries is introduced by the adaptive query evaluation engine StreaMon [16]. For the same reason, TelegraphCQ uses eddies [13]. An eddy routes tuples to operators in accordance with a policy that should make pass the tuples by all operators of a given query. Eddy can dynamically change the operator execution order, and therefore can provide a continuous reoptimization of queries. On the other hand, Aurora uses a load shedding mechanism [68] that, in case of system overload, drops some tuples of data stream. Shedding may be done randomly or semantically according to a given ”quality of data” policy specific to applications. Finally, let us note that Aurora, TelegraphCQ and STREAM also propose mechanisms for query sharing between data streams and for data sharing between queries. This is particularly important for efficient (re-)usage of system resources.
3.4 Discussion In general, in current DSMS, data are sent to a central server where query evaluation takes place, hence the overload problem even if servers are usually powerful machines. In fact, an important challenge in these systems is to resist to increasing number and rate of stream data, as well as the number of queries to evaluate. Distributed DSMS have been proposed recently [3, 53, 74] to face these issues. In such a context, optimization problem becomes crucial [57]. Query optimization in traditional DBMS is based on the knowledge on the physical structure and the distribution of data. Data access is direct and query results are precise. This differs from data stream processing, in which access to data is sequential, data rate is not controlled and data may sometimes be redundant and noisy. Classical optimization techniques are therefore not all applicable for queries on data streams. Optimization should also be continuous and dynamic in order to adapt to variable conditions of these highly dynamic systems [64]. Data stream processing techniques are certainly very useful for sensor data management. In sensors context, query evaluation overload issues may be improved by using computing capabilities of sensors and/or the software proxies managing sensors. This idea is exploited by hybrid systems presented in the next section.
122
L. G¨urgen et al.
4 Integration of WSN and DSMS The preceding sections presented two different sensor data processing approaches: wireless sensor networks and data stream management systems. This section introduces a third family of solutions: hybrid systems. The common aim of these systems is to integrate the advantages of the first two approaches, therefore to handle data of a great number of heterogeneous sensors with a maximum number of complex queries that filter, aggregate and integrate data streams issued by sensors. Some systems target specific sensors for a given application domain, while others target more generic sensors varying from micro-sensors to bigger ones such as weather stations. This section is based on the study of several hybrid systems. A short description of seven of them is given in section 4.1 (see table 1 for a summary). Section 4.2 then compares these systems with WSN and DSMS solutions in the light of three important aspects: system scalability, sensor heterogeneity support and the complexity of the supported queries.
Table 1 Original contributions and target applications of studied hybrid systems
4.1 Hybrid Systems This section introduces the main characteristics of seven hybrid systems, namely, Fjords [55], Borealis [3], IrisNet [39], Hourglass [66], GSN [4], HiFi [37] and SStreaMWare [44]. 4.1.1
Fjords
Fjords (Framework in Java for Operators on Remote Data Streams) is a proposal of a distributed infrastructure on which we can build sensor data querying systems [55]. It provides a mechanism for creating query execution plans that can integrate dynamic data from sensors with static data from traditional databases. It adopts a distributed three-level hierarchical architecture: sensor networks, proxies and a central query evaluator (see figure 8). Complex queries that can not be evaluated on sensors can therefore be evaluated on the proxies or on the central query evaluator. Proxies hide the specifities of sensors and also manage the heterogeneity of resources of sensor networks (energy consumption, bandwidth, sampling rate, etc.).
Data Management Solutions in Networked Sensing Systems
123
Fig. 8 Three-level architecture of Fjords (extracted from [55])
Fjords has been used for an application that monitors traffic conditions on highways. Application carries out the special feature of Fjords, which is to be able to join data streams flowing from sensors detecting vehicles with the static data on the segments of the highway. 4.1.2
Borealis
Borealis is a distributed DSMS [3]. It extends the centralized DSMS Aurora [1] for distributed execution of continuous queries. Evaluation of continuous queries are shared among several sites which are represented by Borealis nodes. A Borealis node may interface with one or more sensor networks via proxies [2] (see figure 9). Every sensor network provides an adapter that exports an API by which the state of the network (used bandwidth, remaining energy, etc.) can be provided to the system. This information is used for query optimization. Adapters are used to manage the heterogeneity of sensor networks. Borealis also deals with the issues caused by distribution such as distributed optimization, remote communication, fault tolerance, node localization, etc. Borealis has been the first step towards distributed DSMS leading to processing of sensor data streams at larger scales. It focuses on evaluation of complex queries by providing a support of QoS, dynamic optimization and high availability. 4.1.3
IrisNet
IrisNet (Internet-scale Resource-Intensive Sensor Network Services) considers a sensor network at internet scale [39]. By adopting the idea of internet, the goal is to allow users to search, to bind and to use sensing services. It provides software components to facilitate the development and deployment of sensor services, of different types, highly distributed in the network. The architecture is based on Sensor Agents (SA) and Organizing Agents (OA). SA provide a generic API for access to sensor data. They are hosted by powerful units (such as PC) that are directly connected to sensors, typically microphones, webcams, temperature sensors, etc. They can execute relatively complex algorithms such as image processing. Data are processed (e.g. filtered) and sent to the nearest
124
L. G¨urgen et al.
(a) Overview of the architecture
(b) Proxies and adapters for heterogeneity management
Fig. 9 Integration of DSMS Borealis with WSN (extracted from [2])
OA, which will update the database concerned by these new data. Data are partitioned hierarchically among OA where they are stored. The hierarchy is defined by the geographical location (one OA by region, department, city, etc.). This hierarchy is also reflected in the data descriptions which are expressed by XML documents. Queries are formulated by using the XPath query language [71]. See figure 10 for a query example. IrisNet has implemented and experimented a parking space finder application. Webcams and toy cars are used for demonstration [33].
Fig. 10 Example of a query for free lots in Oakland (extracted from [39])
Data Management Solutions in Networked Sensing Systems
4.1.4
125
Hourglass
Hourglass, as IrisNet, proposes an internet based infrastructure to interconnect sensors and applications via services [66]. A service-oriented approach is used to facilitate the integration of different types of sensor services. Queries are based on the principle of publish-subscribe. Query evaluation operators, such as filters and aggregations, are implemented as services. Combinations of these services enable constructing complex queries that are evaluated in a distributed manner on the nodes implementing these services. As shown in figure 11, queries are expressed as ”circuits” that interconnect data producers and consumers via intermediate data processing operators.
Fig. 11 Example of a query plan expressed as a ”circuit” in Hourglass (extracted from [66])
The target application domain of Hourglass is the medicine: allocation of ambulances to nearest hospitals, monitoring health status of patients via sensors, communication of medical data from patients to doctors, etc. 4.1.5
GSN
GSN (Global Sensor Networks) is conceived for fast and flexible deployment of applications that need integration of data coming from heterogeneous sensors [4]. GSN relies on a peer-to-peer architecture. Each GSN peer contains a set of virtual sensors and several software components for data management (query processor, notification mechanism, storage manager, etc.). A virtual sensor gathers data stream from one or more sensors, evaluates queries and produces results in stream. GSN uses a virtual sensor deployment descriptor, written in XML, that integrates data of various local or remote sensors (see figure 12 for an example). SQL queries are embedded in these descriptors and they are executed on data that are temporally stored in form of relations in a traditional database management system. Data streams and data windows are represented in form of ”views” over these relations. In consequence, as shown in [4], the performance of the DBMS used for storing data and
126
L. G¨urgen et al.
evaluating queries is a determining factor for the performance of GSN. In order to decouple stream characteristics from the used standard SQL language, parameters concerning stream processing (e.g., sampling rate, data window size, etc.) are expressed separately from the SQL queries.
Fig. 12 Virtual sensor descriptor of GSN (extracted from [5])
GSN is demonstrated by using adapters implemented for different types of sensors (webcams, RFID readers, temperature sensors) [5]. 4.1.6
HiFi
HiFi adopts a hierarchical organization based on geographic locations [37]. Sensors constitute the leaf nodes of the hierarchy. Intermediary nodes are powerful computing units that can process complex operations on data (e.g. filtering, cleaning, aggregation, join, etc.). Indeed, the ”location-centric” nature of sensor applications and the hierarchical structure of organizations that deploy applications fit well to this hierarchical view of the architecture. As shown in figure 13, queries are evaluated on several levels, each one performing a specific task: clean, smooth, arbitrate, validate, analyze (CSAVA). Aggregations or filters can be evaluated near to the sources. This reduces data transfers, which is certainly convenient for scalability of the system. Query evaluation relies on the DSMS TelegraphCQ [22], which supports complex SQL-like continuous queries. For heterogeneity management, [49] proposes to use Virtual Devices (VICEs) that transform raw data of heterogeneous sensors to more clean, generic and understandable data for the HiFi querying system. HiFi particularly focuses on real-time processing of complex sensor events that need to be integrated at higher levels in the hierarchy. [61] illustrates HiFi with object and human tracking applications using RFID technology.
Data Management Solutions in Networked Sensing Systems
127
Fig. 13 CSAVA process of HiFi (extracted from [37])
4.1.7
SStreaMWare
SStreaMWare [44] is a middleware for processing heterogeneous sensor data streams. It is based on a distributed and service-oriented architecture [42]. Main targets of SStreaMWare are large scale applications that need to integrate and aggregate data coming from heterogenous sensors belonging to different manufacturers, providers or organizations. SStreaMWare can easily be integrated to existing infrastructures thanks to its service-oriented architecture. The architecture of SStreaMWare follows the natural hierarchy of geographical locations (see figure 14). Data processing tasks are distributed among several levels in the hierarchy, nearest possible to sensors in order to better handle the increasing number of data and queries. Data processing facilities are provided in terms of ”generic” services that hide the heterogeneity of sensors by exporting common querying methods. Services implement the data and query model SStreaM [43], which provides evaluation of queries on multiple data streams and window operators. SStreaMWare also deals with issues related to the ”management of sensor farms” [45], a subject little explored in this domain. It concerns not only read-only query operations on sensors, but also management operations (configuration, software install, performance monitoring) which can modify the overall configuration of the system and possibly invalidate the results of the ongoing continuous queries. It proposes a concurrency control mechanism to manage the conflicts that can occur by concurrent execution of these operations [45, 46]. SStreaMWare has been used to monitor electric power materials4 [17]. The demonstration [47] illustrates its usage in different other contexts.
4
French government funded project, PISE.
128
L. G¨urgen et al.
Fig. 14 Hierarchical architecture of SStreaMWare (extracted from [44])
4.2 Analysis This section analyses, from the data management point of view, the scalability of the sensor systems, their capability to manage sensor heterogeneity and the support they provide to complex queries. 4.2.1
Scalability
Sensor system’s scalability concerns several aspects, among them the number of sensors, volume of the data stream and number of queries. This section focuses on the architectural choice and their impact on scalability. WSN and DSMS Solutions based on WSN consider data sources (i.e., sensors) as autonomous units having some computing and storage capabilities. Data can therefore be processed in the network on the sensors. This leads energy and bandwidth saving that allow to build more scalable solutions. On the other hand, in DSMS, the server and network can be more easily overloaded because of their centralized approach. In sensor networks, data processing and transmission tasks are distributed, overload does not appear and scalability is facilitated. In fact, even though some DSMS propose techniques to offer a given quality of service (e.g., dynamic adaptability, query and data stream sharing, approximate results), bottlenecks are unavoidable with the high cost of data stream processing on a large scale. Hybrid Systems Hybrid solutions adopt a distributed architecture in order to share out data processing tasks among various levels in the system. This facilitates scalabilty management
Data Management Solutions in Networked Sensing Systems
129
in terms of data volume and number of queries. The studied systems have either hierarchical or peer-to-peer architectures. Peer-to-peer architecture: Hourglass proposes an internet-based infrastructure that relies on an overlay network in which each node can be a producer or consumer of data. GSN also has a peer-to-peer approach in order to address large scale systems and to better manage dynamic addition/suppression of nodes from the system. However, the limited control on the nodes and the difficulty to maintain a directory to discover and to make nodes collaborate together, are the main drawbacks of these systems. Hierarchical architecture: In general, sensor applications are location-oriented, since sensors reflect the state of the environment in which they are physically placed. This implies that a geographical hierarchical organization is a good choice for sensing systems. This is particularly the case for target applications of HiFi and SStreaMWare which adopt a hierarchical architecture: the lowest levels collect data and, going up in the hierarchy, data are aggregated and integrated in order to produce information concerning larger regions covered by many sensors. Similarly, IrisNet collects data by its Sensor Agents and sends them to geographically distributed Organizing Agents which are responsible for storing the data. Fjords proposes a three level hierarchical architecture, in which low-level sensor networks provide data to a central query evaluator, through an intermediate proxy level. A proxy is placed geographically near to its corresponding sensor network. Similarly, Borealis uses intermediary nodes that interface with one or more sensor networks that can execute query operators. A Borealis query has a tree form, thus for each query, a structure of node hierarchy is created. Borealis also proposes load shedding mechanism to reduce query evaluation load [69]. Very few large scale evaluations have been performed by these systems, due to the lack of necessary infrastructures that can provide realistic numerical results. At large scale, integration of heterogeneous sensors is also important, since sensing systems will typically consist of different types of sensors from various providers. Table 2 summarizes the architectural choices of the hybrid systems.
Table 2 Architectural choices of the hybrid systems
4.2.2
Heterogeneity
This section deals with the sensor heterogenity support of studied systems.
130
L. G¨urgen et al.
WSN and DSMS The heterogeneity problem is generally not tackled by WSN solutions. They assume homogeneous sensors (same communication protocol, same query processors, same data schema, etc.), which is restrictive, in particular in large scale systems. DSMS do not handle heterogeneity of stream data sources either. Sources have to send data to a server in accordance with a given data schema. Among DSMS, only TelegraphCQ proposes placing adapters near sources to translate source specific data formats to the system’s common format. Hybrid systems Hybrid systems consider strong heterogeneity of sensors (different size, capacity, provider, software, etc.) and propose adequate solutions. The use of adapters (or wrappers) is the most popular solution. The main idea is to add an adaptation layer in order to provide a homogenous view over the system. This layer provides means to control the sensors and to achieve transformation of queries and their answers to formats manageable by sensors and the system respectively. The main idea is the same for all the systems, although each system names it differently: proxies for Fjords and Hourglass, adapters for HiFi and SStreaMWare, wrappers for GSN and Borealis, and Sensing Agents for IrisNet. In Hourglass, IrisNet, GSN and SStreaMWare, this abstraction is provided as services. A service-oriented approach facilitates the management of the heterogeneous and dynamic aspects of sensing systems. However, it introduces issues such as naming, discovery and orchestration of services. In Borealis and Fjords, proxies are used to manage resource heterogeneity of sensor networks: energy, bandwidth, sampling rate, etc. In all these systems, adapters are important and active elements of query evaluation. Table 3 lists taxonomies used by different hybrid systems to refer adapters. Table 3 Software entities for heterogeneity management in hybrid solutions
4.2.3
Query Complexity
This section discusses the complex data processing approaches of the studied systems. WSN and DSMS WSN solutions use relatively complex algorithms to make sensors collaborate for query evaluation. Data and query routing, as well as synchronization messages bring a considerable overhead to sensor nodes, which have limited resources. Besides, operators such as joins between data of different sensors and aggregation operators that need completeness of data (histogram, median, quantile, etc.) can not be calculated ”in” sensor networks.
Data Management Solutions in Networked Sensing Systems
131
On the other hand, centralized data processing in DSMS does not need such complex algorithms for data collection. DSMS servers are powerful units (contrary to sensors) which have control over all data streams and may evaluate complex queries on them. Hybrid systems Hybrid solutions use relatively powerful units in order to execute operators that may need important computing power and storage. These units can also partially centralize data streams to perform, for instance, join operations between data streams or aggregation operations. Nevertheless, some systems can still delegate computing to sensors when they have enough capacity. Efficiency of declarative queries for sensor data processing is now well admitted. SQL-like queries are the most popular ones. HiFi uses a declarative language for its CSAVA (Clean, Smooth, Arbitrate, Validate, Analyze) steps. GSN also uses SQL queries embedded in its virtual sensor deployment descriptors written in XML. IrisNet prefers to use XPath queries over its data descriptions structured in XML. Hourglass expresses query plans (namely, ”circuits”) in XML, which contain given predicates to find the desired sensor service. Borealis and SStreaMWare propose a graphical tool (based on boxes and arrows) to create query execution plans. And finally, Fjords does not use any particular language, and argues that it offers an architectural construction suited to all languages. All the studied solutions propose a distributed processing of sensor data. In IrisNet, Borealis, Fjords, HiFi and SStreaMWare, adapters (and possibly sensors and/or sensor networks) contribute to the data processing. Fjords, Borealis and SStreaMWare decomposes queries into sub-queries, each one being evaluated on different levels of the system. HiFi executes specific types of queries for each level of the system. In IrisNet, geographically distributed Organizing Agents are responsible for storing sensor data and evaluate queries on them. IrisNet integrates a semantic cache mechanism to optimize the evaluation of similar queries [32]. In GSN, queries are evaluated by virtual sensors. It is noteworthy that GSN (respectively IrisNet) evaluates SQL style (respectively XPath) queries on persistent data, while the others evaluate queries on transitory data streams. Finally, Hourglass uses a publishsubscribe mechanism for distributed query evaluation. Table 4 summarizes query related aspects of hybrid solutions. Table 4 Summary of query related aspects of hybrid solutions
132
L. G¨urgen et al.
5 Conclusion The use of sensors is increasing rapidly in numerous application domains as they provide opportunities to improve monitoring and information management. High level sensor data management involves several key issues. This chapter presented the three most important categories of sensor data management systems available today. The first one is wireless sensor networks which provide ”in-network” data processing. The second one, coming mainly from the database community, is data stream management systems. They propose on-line server based management of ”infinite” data streams. The last category builds on the two preceding approaches to provide hybrid solutions. These categories were presented and analyzed considering three important aspects related to the increasing number and diversification of sensors and application domains. We discussed sensor system’s scalability, their ability to tackle sensor heterogeneity and to support complex queries. We pointed out the special ability of hybrid solutions to tackle heterogeneity of large scale networked sensing systems. Numerous heterogeneous devices (personal computers, smart phones, sensors, etc.) contribute to make information omnipresent. Among them, sensors become important actors for ubiquitous information systems. The synthesis presented in this chapter makes an assessment of the existing sensor data management systems and evaluates their relative maturity. We particularly note important research results in the area of continuous query support for sensor data processing. This is essential for applications requiring ”real-time” sensor data monitoring. Nevertheless, other important services have to be provided to improve sensor data management in environments that have become more and more dynamic and heterogeneous. For instance, besides data querying operations, management operations on sensors (software updates, configurations, performance monitoring and diagnostic operations) must be supported by sensing systems [41]. Privacy and security are also extremely important. They should be present in all the stages of the sensor data management systems in order to be widely accepted both in industrial contexts and every day’s life. More generally, significant research is still necessary to come out with complete sensor data management systems providing more than querying facilities. The dynamic integration of such systems (e.g., with plug&play mechanisms) in more classic information systems will allow to construct real ubiquitous information systems. Acknowledgments. The authors would like to thank the editors for their valuable comments and Vincent Olive for his contribution to previous work on sensor data management.
References 1. Abadi, D., Carney, D., C ¸ etintemel, U., Cherniack, M., Convey, C., Lee, S., Stonebraker, M., Tatbul, N., Zdonik, S.: Aurora: a new model and architecture for data stream management. VLDB Journal 12(2), 120–139 (2003)
Data Management Solutions in Networked Sensing Systems
133
2. Abadi, D., Lindner, W., Madden, S., Schuler, J.: An integration framework for sensor networks and data stream management systems. In: VLDB, pp. 1361–1364 (2004) 3. Abadi, D.J., Ahmad, Y., Balazinska, M., Cetintemel, U., Cherniack, M., Hwang, J.H., Lindner, W., Maskey, A.S., Rasin, A., Ryvkina, E., Tatbul, N., Xing, Y., Zdonik, S.: The Design of the Borealis Stream Processing Engine. In: Second Biennial Conference on Innovative Data Systems Research (CIDR 2005), Asilomar, CA (2005) 4. Aberer, K., Hauswirth, M., Salehi, A.: The global sensor networks middleware for efficient and flexible deployment and interconnection of sensor networks. Tech. Rep. LSIRREPORT-2006-006, Ecole Polytechnique F´ed´erale de Lausanne, EPFL (2006) 5. Aberer, K., Hauswirth, M., Salehi, A.: A middleware for fast and flexible sensor network deployment. In: VLDB 2006: Proceedings of the 32nd international conference on very large data bases, VLDB Endowment, pp. 1199–1202 (2006) 6. Akkaya, K., Younis, M.F.: A survey on routing protocols for wireless sensor networks. Ad Hoc Networks 3(3), 325–349 (2005) 7. Akyildiz, I.F., Su, W., Sankarasubramaniam, Y., Cayirci, E.: Wireless sensor networks: a survey. Computer Networks 38(4), 393–422 (2002) 8. Al-Karaki, J.N., Kamal, A.E.: Routing techniques in wireless sensor networks: a survey. IEEE Wireless Communications 11(6), 6–28 (2004) 9. Aleri: Aleri, inc. (2008), http://www.aleri.com/ 10. Alliance, Z.: Zigbee specifications, version 1.0 (April 2005) 11. Arasu, A., Babcock, B., Babu, S., Datar, M., Ito, K., Motwani, R., Nishizawa, I., Srivastava, U., Thomas, D., Varma, R., Widom, J.: STREAM: The stanford stream data manager. IEEE Data Eng. Bull. 26(1), 19–26 (2003) 12. Arasu, A., Babu, S., Widom, J.: The CQL continuous query language: Semantic foundations and query execution. Tech. Rep. 2003-67, Stanford University (2003) 13. Avnur, R., Hellerstein, J.M.: Eddies: Continuously Adaptive Query Processing. In: Chen, W., Naughton, J.F., Bernstein, P.A. (eds.) Proceedings of the 2000, ACM SIGMOD International Conference on Management of Data, USA, May 16-18, pp. 261–272. ACM, New York (2000) 14. Awan, A., Jagannathan, S., Grama, A.: Macroprogramming heterogeneous sensor networks using cosmos. SIGOPS Oper. Syst. Rev. 41(3), 159–172 (2007), http://doi.acm.org/10.1145/1272998.1273014 15. Babcock, B., Babu, S., Datar, M., Motwani, R., Widom, J.: Models and issues in data stream systems. In: PODS 2002: Proceedings of the twenty-first ACM SIGMODSIGACT-SIGART symposium on Principles of database systems, pp. 1–16. ACM Press, NY (2002), http://doi.acm.org/10.1145/543613.543615 16. Babu, S., Widom, J.: StreaMon: an adaptive engine for stream query processing. In: SIGMOD 2004: Proceedings of the, ACM SIGMOD international conference on Management of data, pp. 931–932. ACM Press, New York (2004), http://doi.acm.org/10.1145/1007568.1007702 17. Baude, F., Bottaro, A., Brun, J.M., Chazalet, A., Constancin, A., Donsez, D., G¨urgen, L., Lalanda, P., Legrand, V., Lestideau, V., Mari´e, S., Marin, C., Moreau, A., Olive, V.: Extension de passerelles OSGi pour les domaines de la distribution e´ lectrique: Mod`eles et outils. In: OSGI Workshop in conjonction with UBIMOB 2006, 3rd French-speaking conference on Mobility and Ubiquity computing (2006) 18. Bhatti, S., Carlson, J., Dai, H., Deng, J., Rose, J., Sheth, A., Shucker, B., Gruenwald, C., Torgerson, A., Han, R.: MANTIS OS: an embedded multithreaded operating system for wireless micro sensor platforms. Mobile Networks and Applications 10(4), 563–579 (2005), http://doi.acm.org/10.1145/1160162.1160178
134
L. G¨urgen et al.
19. Biagioni, E., Bridges, K.: The application of remote sensor technology to assist the recovery of rare and endangered species. International Journal of High Performance Computing Applications 16(3) (2002) 20. Bonivento, A., Carloni, L.P., Sangiovanni-Vincentelli, A.: Platform-based design of wireless sensor networks for industrial applications. In: DATE 2006: Proceedings of the conference on Design, automation and test in Europe, Belgium, pp. 1103–1107, European Design and Automation Association (2006) 21. Bonnet, P., Gehrke, J., Seshadri, P.: Towards sensor database systems. In: Tan, K.-L., Franklin, M.J., Lui, J.C.-S. (eds.) MDM 2001. LNCS, vol. 1987, pp. 3–14. Springer, Heidelberg (2000) 22. Chandrasekaran, S., Cooper, O., Deshpande, A., Franklin, M., Hellerstein, J., Hong, W., Krishnamurthy, S., Madden, S., Raman, V., Reiss, F., Shah, M.: TelegraphCQ: Continuous dataflow processing for an uncertain world. In: CIDR (2003) 23. Chen, D., Varshney, P.K.: QoS support in wireless sensor networks: A survey. In: International Conference on Wireless Networks, Las Vegas, Nevada, USA, pp. 227–233 (2004) 24. Chen, J., DeWitt, D.J., Tian, F., Wang, Y.: NiagaraCQ: a scalable continuous query system for internet databases. In: SIGMOD 2000: Proceedings of the 2000 ACM SIGMOD international conference on Management of data, pp. 379–390. ACM Press, New York (2000), http://doi.acm.org/10.1145/342009.335432 25. Chong, C.Y., Kumar, S.P.: Sensor networks: evolution, opportunities, and challenges. Proceedings of the IEEE 91(8), 1247–1256 (2003) 26. Coleri, S., Cheung, S., Varaiya, P.: Sensor networks for monitoring traffic. In: 42nd Allerton Conference on Communication, Control and Computing (2004) 27. Coral8 (2008), http://www.coral8.com/ 28. Cortes, C., Fisher, K., Pregibon, D., Rogers, A.: Hancock: a language for extracting signatures from data streams. In: KDD 2000: Proceedings of the sixth ACM SIGKDD international conference on Knowledge discovery and data mining, pp. 9–17. ACM Press, NY (2000), http://doi.acm.org/10.1145/347090.347094 29. Cranor, C., Gao, Y., Johnson, T., Shkapenyuk, V., Spatscheck, O.: Gigascope: high performance network monitoring with an sql interface. In: SIGMOD 2002: Proceedings of the 2002 ACM SIGMOD international conference on Management of data, p. 623. ACM Press, New York (2002), http://doi.acm.org/10.1145/564691.564777 30. Dantu, K., Rahimi, M., Shah, H., Babel, S., Dhariwal, A., Sukhatme, G.S.: Robomote: enabling mobility in sensor networks. In: IPSN 2005: Proceedings of the 4th international symposium on Information processing in sensor networks, p. 55. IEEE Press, Piscataway (2005) 31. Dejene, E., Scuturici, V., Brunie, L.: Hybrid approach to collaborative context-aware service platform for pervasive computing. Journal of Computers (JCP) 3(1), 40–50 (2008) 32. Deshpande, A., Nath, S., Gibbons, P.B., Seshan, S.: Cache-and-query for wide area sensor databases. In: SIGMOD 2003: Proceedings of the 2003 ACM SIGMOD international conference on Management of data, pp. 503–514. ACM Press, New York (2003), http://doi.acm.org/10.1145/872757.872818 33. Deshpande, A., Nath, S., Gibbons, P.B., Seshan, S.: Irisnet: Internet-scale resourceintensive sensor services. In: SIGMOD 2003: Proceedings of the 2003 ACM SIGMOD international conference on Management of data, p. 667. ACM Press, New York (2003), http://doi.acm.org/10.1145/872757.872856 34. Deutsch, A., Fernandez, M.F., Florescu, D., Levy, A.Y., Suciu, D.: XML-QL: A Query Language for XML (1998)
Data Management Solutions in Networked Sensing Systems
135
35. Estrin, D., Culler, D., Pister, K., Sukhatme, G.: Connecting the physical world with pervasive networks. IEEE Pervasive Computing 1(1), 59–69 (2002), http://dx.doi.org/10.1109/MPRV.2002.993145 36. Estrin, D., Govindan, R., Heidemann, J., Kumar, S.: Next century challenges: scalable coordination in sensor networks. In: Proceedings of the fifth annual ACM/IEEE international conference on Mobile computing and networking, Seattle, WA USA, pp. 263–270 (1999) 37. Franklin, M.J., Jeffery, S.R., Krishnamurthy, S., Reiss, F., Rizvi, S., Wu, E., Cooper, O., Edakkunni, A., Hong, W.: Design Considerations for High Fan-In Systems: The HiFi Approach. In: CIDR, pp. 290–304 (2005) 38. Gao, T., Greenspan, D., Welsh, M., Juang, R.R., Alm, A.: Vital signs monitoring and patient tracking over a wireless network. In: 27th Annual International Conference of the IEEE EMBS, Shanghai, pp. 102–105 (2005), http://ieeexplore.ieee.org/xpl/freeabs_all.jsp? arnumber=1616352 39. Gibbons, P.B., Karp, B., Ke, Y., Nath, S., Seshan, S.: IrisNet: An architecture for a worldwide sensor web. IEEE Pervasive Computing 02(4), 22–33 (2003), http://doi.ieeecomputersociety.org/10.1109/MPRV.2003. 1251166 40. Golab, L., Ozsu, M.T.: Issues in data stream management. SIGMOD Rec. 32(2) (2003), http://doi.acm.org/10.1145/776985.776986 41. G¨urgen, L., Honiden, S.: Management of networked sensing devices. In: Proceedings of the 2nd. International Workshop on Sensor Network Technologies for Information Explosion Era (2009) 42. G¨urgen, L., Labb´e, C., Olive, V., Roncancio, C.: A scalable architecture for heterogeneous sensor management. In: DEXA Workshops, pp. 1108–1112 (2005) 43. G¨urgen, L., Labb´e, C., Roncancio, C., Olive, V.: SStreaM: A model for representing sensor data and sensor queries. In: International Conference on Intelligent Systems and Computing: Theory and Applications, ISYC (2006) 44. G¨urgen, L., Roncancio, C., Labb´e, C., Bottaro, A., Olive, V.: SStreaMWare: A service oriented middleware for heterogeneous sensor data management. In: International Conference on Pervasive Services, ICPS (2008) 45. G¨urgen, L., Roncancio, C., Labb´e, C., Olive, V.: Transactional issues in sensor data management. In: 3rd International Workshop On Data Management for Sensor Networks (DMSN 2006) in conjunction with VLDB 2006, pp. 27–32 (2006) 46. G¨urgen, L., Roncancio, C., Labb´e, C., Olive, V.: Update tolerant execution of continuous queries on sensor data. In: Fifth IEEE International Conference on Networked Sensing Systems, INSS (2008) 47. G¨urgen, L., Roncancio, C., Labb´e, C., Olive, V., Donsez, D.: Scalable management of heterogeneous senor data in dynamic environments (demonstration). In: Fifth IEEE International Conference on Networked Sensing Systems, INSS (2008) 48. Han, C.C., Kumar, R., Shea, R., Kohler, E., Srivastava, M.: A dynamic operating system for sensor nodes. In: MobiSys 2005: Proceedings of the 3rd international conference on Mobile systems, applications, and services, pp. 163–176. ACM Press, New York (2005), http://doi.acm.org/10.1145/1067170.1067188 49. Jeffery, S.R., Alonso, G., Franklin, M.J., Hong, W., Widom, J.: Virtual Devices: An extensible architecture for bridging the physical-digital divide. Tech. Rep. UCB/CSD-051375, EECS Department, University of California, Berkeley (2005), http://www.eecs.berkeley.edu/Pubs/TechRpts/2005/5575.html
136
L. G¨urgen et al.
50. Krishnamachari, B., Estrin, D., Wicker, S.B.: The impact of data aggregation in wireless sensor networks. In: ICDCSW 2002: Proceedings of the 22nd International Conference on Distributed Computing Systems, pp. 575–578. IEEE, Washington (2002) 51. Levis, P., Madden, S., Gay, D., Polastre, J., Szewczyk, R., Woo, A., Brewer, E.A., Culler, D.E.: The emergence of networking abstractions and techniques in TinyOS. In: NSDI, pp. 1–14 (2004) 52. Liu, T., Martonosi, M.: Impala: a middleware system for managing autonomic, parallel sensor systems. In: PPoPP 2003: Proceedings of the ninth ACM SIGPLAN symposium on Principles and practice of parallel programming, pp. 107–118. ACM Press, New York (2003), http://doi.acm.org/10.1145/781498.781516 53. Logothetis, D., Yocum, K.: Wide-scale data stream management. In: ATC 2008: USENIX 2008 Annual Technical Conference on Annual Technical Conference, pp. 405– 418. USENIX Association, Berkeley (2008) 54. Lorincz, K., Malan, D.J., Fulford-Jones, T.R.F., Nawoj, A., Clavel, A., Shnayder, V., Mainland, G., Welsh, M., Moulton, S.: Sensor networks for emergency response: Challenges and opportunities. IEEE Pervasive Computing 3(4), 16–23 (2004), http://dx.doi.org/10.1109/MPRV.2004.18 55. Madden, S., Franklin, M.J.: Fjording the stream: An architecture for queries over streaming sensor data. In: ICDE 2002: Proceedings of the 18th International Conference on Data Engineering, p. 555. IEEE Computer Society Press, Washington (2002) 56. Madden, S., Franklin, M.J., Hellerstein, J.M., Hong, W.: TAG: A tiny aggregation service for ad-hoc sensor networks. In: OSDI (2002) 57. Madden, S., Shah, M., Hellerstein, J.M., Raman, V.: Continuously adaptive continuous queries over streams. In: SIGMOD 2002: Proceedings of the 2002 ACM SIGMOD international conference on Management of data, pp. 49–60. ACM, New York (2002), http://doi.acm.org/10.1145/564691.564698 58. Madden, S.R., Franklin, M.J., Hellerstein, J.M., Hong, W.: TinyDB: an acquisitional query processing system for sensor networks. ACM Trans. Database Syst. 30(1), 122– 173 (2005), http://doi.acm.org/10.1145/1061318.1061322 59. Mainwaring, A., Culler, D., Polastre, J., Szewczyk, R., Anderson, J.: Wireless sensor networks for habitat monitoring. In: WSNA 2002: Proceedings of the 1st ACM international workshop on Wireless sensor networks and applications, pp. 88–97. ACM Press, NY (2002), http://doi.acm.org/10.1145/570738.570751 60. Perrig, A., Stankovic, J., Wagner, D.: Security in wireless sensor networks. Communications of the ACM 47(6), 53–57 (2004), http://doi.acm.org/10.1145/990680.990707 61. Rizvi, S., Jeffery, S.R., Krishnamurthy, S., Franklin, M.J., Burkhart, N., Edakkunni, A., Liang, L.: Events on the edge. In: SIGMOD 2005: Proceedings of the, ACM SIGMOD international conference on Management of data, pp. 885–887. ACM Press, NY (2005), http://doi.acm.org/10.1145/1066157.1066275 62. R¨omer, K., Mattern, F.: The design space of wireless sensor networks. IEEE Wireless Communications 11(6), 54–61 (2004), doi:10.1109/MWC.2004.1368897 63. Seshadri, P., Livny, M., Ramakrishnan, R.: The design and implementation of a sequence database system. In: Proceedings of VLDB 1996, San Francisco, USA, pp. 99–110 (1996) 64. Seshadri, S., Kumar, V., Cooper, B.F.: Optimizing multiple queries in distributed data stream systems. In: ICDEW 2006: Proceedings of the 22nd International Conference on Data Engineering Workshops, p. 25. IEEE Computer Society, Washington (2006), http://dx.doi.org/10.1109/ICDEW.2006.107
Data Management Solutions in Networked Sensing Systems
137
65. Sharaf, M.A., Beaver, J., Labrinidis, A., Chrysanthis, P.K.: Balancing energy efficiency and quality of aggregate data in sensor networks. The VLDB Journal 13(4), 384–403 (2004) 66. Shneidman, J., Pietzuch, P., Ledlie, J., Roussopoulos, M., Seltzer, M., Welsh, M.: Hourglass: An infrastructure for connecting sensor networks and applications. Tech. Rep. TR-21-04, Harvard University (2004) 67. StreamBase: Streambase systems, inc. (2008), http://www.streambase.com/ 68. Tatbul, N., C ¸ etintemel, U., Zdonik, S.B., Cherniack, M., Stonebraker, M.: Load shedding in a data stream manager. In: Proceedings of the 29th VLDB Conference, pp. 309–320 (2003) 69. Tatbul, N., Zdonik, S.: Dealing with overload in distributed stream processing systems. In: ICDEW 2006: Proceedings of the 22nd International Conference on Data Engineering Workshops (ICDEW’06), p. 24. IEEE Computer Society, Washington (2006), http://dx.doi.org/10.1109/ICDEW.2006.45 70. Ulmer, C., Alkalai, L., Yalamanchili, S.: Wireless distributed sensor networks for in-situ exploration of mars. Tech. Rep. 2003-67, NASA (2003) 71. W3C: Xml path language(xpath) (2008), http://www.w3.org/tr/xpath 72. Wan, C.Y., Campbell, A.T., Krishnamurthy, L.: Psfq: a reliable transport protocol for wireless sensor networks. In: WSNA 2002: Proceedings of the 1st ACM international workshop on Wireless sensor networks and applications, pp. 1–11. ACM, New York (2002), http://doi.acm.org/10.1145/570738.570740 73. Widom, J., Motwani, R.: Query processing, resource management, and approximation in a data stream management system. In: Proceedings of The First Biennial Conference on Innovative Data Systems Research (CIDR), pp. 245–256 (2003), http://citeseer.ist.psu.edu/widom03query.html 74. Xiong, X., Elmongui, H.G., Chai, X., Aref, W.G.: Place: A distributed spatio-temporal data stream management system for moving objects. In: 8th International Conference on Mobile Data Management, pp. 44–51 (2007) 75. Yao, Y., Gehrke, J.: The cougar approach to in-network query processing in sensor networks. SIGMOD Rec. 31(3), 9–18 (2002), http://doi.acm.org/10.1145/601858.601861
Integration of Heterogeneous Sensor Nodes by Data Stream Management Michael Daum, Frank Lauterwald, Martin Fischer, Mario Kiefer, and Klaus Meyer-Wegener
Abstract. Wireless Sensor Networks (WSNs) will be an important streaming data source for many fields of surveillance in the near future, as the price of WSN technologies is diminishing rapidly, while processing power, sensing capability, and communication efficiency are growing steadily. Data-stream analyses should be distributed over the entire network in a way that the processing power is well utilized, the sensing is done in a semantically reasonable way, and communication is reduced to a minimum as it consumes much energy in general. Surveillance experts of different domains need technical experts in order to deploy those distributed data stream analyses. Data-stream queries often realize data-stream analyses. Especially surveillance scenarios that base on Sensor Data Fusion (SDF) will need the integration of heterogeneous data sources produced by potentially heterogeneous sensor nodes. This chapter overviews existing WSN middleware solutions, Stream Processing Systems (SPSs), and their integration. An approach that maps a global data-stream query to distributed and heterogeneous sensor nodes and SPSs opens a path to solve the problems mentioned above. Integration is achieved in two ways: semantic integration is done implicitly by the partitioning and mapping using rules that retain the semantics of the global query through the entire distribution and deployment process; technical integration is achieved during mapping and deployment with the help of the knowledge about platforms and connections.
1 Objectives Wireless Sensor Networks (WSNs) consist of nodes that are widely distributed. These sensors are the data source for data stream processing in many scenarios. Data stream processing is an option for distributed processing that is more efficient Michael Daum · Frank Lauterwald Chair for Computer Science 6 (Data Management), University of Erlangen-N¨urnberg, Germany e-mail: [email protected],[email protected] T. Hara et al. (Eds.): WSN Technologies for the Information Explosion Era, SCI 278, pp. 139–172. c Springer-Verlag Berlin Heidelberg 2010 springerlink.com
140
M. Daum et al.
than sending all data to a central processing unit. As data volume grows rapidly, the central processing approach becomes more and more impracticable. Data processing has to be distributed in a way that operators like filtering and aggregation can help to reduce communication already in the vicinity of the sensors. We expect complex scenarios with many distributed data sources as well as complex event processing and complex sensor data fusion in the near future. These scenarios will add new operators to the well-known set of operators. Yet programming these scenarios should be simplified in the sense that users may write declarative queries instead of procedural code in some programming language. These queries combine expressions of operators in a very compact notation. The given scenarios may add new operators. From our point of view, sensor network nodes are Stream Processing Systems (SPSs) with limited capacities. As the SPSs are heterogeneous, their query languages are heterogeneous, too. There are approaches that configure WSN nodes by using SQL-based query languages; others use graph-based query languages that correspond to the data flow. At least, most WSN nodes can be individually programmed in programming languages that are similar to C. We subsume all kinds of systems that process streams under the term SPS and call each instance of an SPS a node. Besides sensor network nodes, Data Stream Management Systems (DSMSs) are also SPSs. They are often coupled with WSNs to process the data streams produced by sensors. This requires an integration of stationary SPSs and WSNs. The query languages of DSMSs are more powerful and they are not limited with respect to power supply and communications. The heterogeneity of query languages is caused by the heterogeneity of different configurable data sources. A directed graph of distributed nodes forms the processing network that can process a query. The deployment of a query has to consider topology, performance, and the nodes’ heterogeneity. This abundance of crucial aspects will surely overwhelm domain experts who just want to observe a scenario by sensors. We envision the domain experts to define abstract queries without considering platform-specific constraints of the nodes, topology, etc. Only the query and the sensor data matter. For this purpose, this chapter follows the Model Driven Architecture (MDA) approach with our middleware prototype Data Stream Application Manager (DSAM). This is facilitated by a central repository that stores the metadata of the observed scenario. It manages all data sources and offers a semantic description of sensor data.
1.1 Introductory Example As a fictitious introductory example (Fig. 1), we assume that biologists want to use WSNs for the surveillance of the behavior of animals. The biologists want to find out in which area the observed animals are under stress and where they go to for recreation. A query determines the area of recreation as the animal’s position at 10 min after the last stress event. It uses sensor nodes that can communicate with each other and may have different sets of sensors, different locations, and different sets of installed modules. An example is depicted in Figure 1. Node1 measures skin
Integration of Heterogeneous Sensor Nodes by Data Stream Management
141
conductivity level (SCL) with sensor S1 and body temperature (TEMP) with sensor S2. Node2 can communicate with the base station and has higher energy capacity than Node1. S3 connected to Node2 delivers position data.
Applicaton „POR_Node1“ POR Node1“ Applicaton „POR_Node2“
S3
WIN_1
Join WIN_2
Gatew way
S1
Query Processor Module Builder Libraries
Fig. 1 Example query for animal surveillance
The locations of input and output streams, sensors, schema information, and topology are part of the catalog. In our example, we have three sensors that should be merged if: • The animal’s TEMP is higher than 38 ◦ C and its SCL is greater than 8 Mho. • The last position of a stress situation and the position of recreation (10 min later) are of interest. • The observation at the points of recreation lasts at least 2 min. The sample query has three input streams and one output stream. Further, the query has two subqueries in the input stream list. The first subquery has three abstract operators: one filter operator that selects all interesting SCL values from the first stream, another filter operator that does the same for TEMP values from the second stream, and a ternary merge operator that merges all three streams after filtering the first and the second. The merge operator waits for all input events and creates one element including all inputs. The second subquery simply adds the current time to the sensor value. In our main query, the last items of the two subqueries are joined if the temporal condition is fulfilled. The last items are realized by sliding windows. The main difference between a merge and a join operator is that merge operators use input values only once in the resulting events. We will explain the query language in Sect. 3.4.
142
M. Daum et al.
Users like behavioral scientists want to describe their needs using an abstract query language without considering the sensor network’s topology in order to describe a query in a formal way. Further, there are different ways of defining partial queries and configuring WSN nodes. This leads to a top-down approach that uses an abstract query definition and does query partitioning, mapping, and deployment automatically. We will explain the partitioning of queries in Sect. 4. Handling distributed heterogeneous nodes will be described in Sect. 5.2.
1.2 The MDA Approach The Model Driven Architecture (MDA)1 is a top-down approach for developing software systems. Its basic idea is mapping a Platform Independent Model (PIM) that is described in a Domain Specific Language (DSL) to code that can be executed. [21] gives a good overview of the ideas of the MDA approach. The PIM may e.g. be described in UML as it is in many business software systems. Alternatively, other OMG language standards and non-OMG languages can be the description language for the PIM, too. A PIM is mapped to one or more Platform Specific Models (PSMs) (Fig. 2). The information about the bridges between the PSMs is generated by the “first transformation”. The “second transformation” is the generation of code that can be run on the platforms. The “code bridges” can be derived directly from the knowledge about the “PSM bridges”. Some of the main goals of MDA are the lowering of development costs by using software generators and a better manageability by using higher levels of abstraction. The same goals will be relevant for deploying queries and applications to a large number of heterogeneous WSN nodes. For domain experts, it will be relevant what the sensor network should do. They are neither interested in platform-specific restrictions of nodes nor in the programming or integration of nodes. In the introductory example (Fig. 1), the query describes the problem of the domain experts that is deployed to two nodes. This chapter draws the parallels between the MDA and this approach. An abstract (generic, SPS independent) query and the semantic description of sensor data correspond to the Platform-Independent Model (PIM). The abstract query is partitioned and deployed to platform-specific nodes. To the best of our knowledge, considering platform-specific restrictions, topology, and cost estimations in the partitioning process together is a new approach in the field of WSNs and stream processing. The resulting set of partial queries and configurations is platform-specific and corresponds to the Platform-Specific Model (PSM) of the MDA approach. In the last step of generating platform-specific code, this approach supports the integration of heterogeneous platforms as the platform characteristics of the nodes are known. 1
www.omg.org/mda
Integration of Heterogeneous Sensor Nodes by Data Stream Management
143
PIM first transformation
PSM
first transformation
PSM Bridge
second transformation
Code
PSM second transformation
Code Bridge
Code
Fig. 2 Mapping between Models in the MDA approach [21]
1.3 Data Stream Application Manager This chapter describes our efforts to address the research challenges that arose while building Data Stream Application Manager (DSAM). DSAM is a prototype that provides a central manager of data stream applications. Its main goal is the integration of heterogeneous SPSs. As it is not practicable to send all data to a central site, DSAM supports distributed query processing (also called in-network query processing in WSNs). DSAM achieves this integration by using the MDA approach. This facilitates deploying code to heterogeneous nodes as well as creating adapters between the nodes. From a stream processing point of view, a WSN in total is a configurable data source that behaves like an SPS (e.g. TinyDB [26]) and single WSN nodes can be seen as SPSs with limited capacity and computing power [14]. The introductory example shows the deployment of a query on two nodes. The base station symbolizes DSAM in a simplified way; DSAM consists of a central management component and distributed deployer components that can interact with WSNs. The presentation of DSAM’s architecture is not in the focus of this chapter as we rather focus on its concepts. Integration of heterogeneous data sources has been analyzed in other fields as well, e.g. in the form of wrappers and mediators for distributed databases [34]. The usage of wrappers for syntactic integration is similar for databases and data streams. Mediators are application-dependent components that represent domain-knowledge and as such are not part of DSAM. It is however possible to use predefined queries as mediators within DSAM.
1.4 Existing Approaches for the Integration of Sensor Networks There are several projects that support in-network query processing in WSNs and their integration with SPSs. This section presents a survey of the most related
144
M. Daum et al.
projects compared to DSAM. The following sections will offer more detailed comparisons of DSAM and those projects in the specific contexts. The Global Sensor Networks (GSN) project [5] offers a middleware that integrates heterogeneous sensor networks at network level. Each kind of sensor network is abstracted by a virtual sensor. A virtual sensor produces exactly one data stream. There is a wrapper for each supported sensor platform that transforms data from the different formats to the internal schema of GSN. SStreaMWare [20] integrates heterogeneous sensors by offering Open Services Gateway initiative (OSGi) services. It supports SQL-like queries. A sensor query service receives queries and
Table 1 Comparison of existing approaches and DSAM Borealis/REED
TinyDB
DSAM
Architecture
GSN
(distributed) distributed autonomous GSN gateways and a containers central manager
SStreamWare
distributed Borealis nodes
Query Definition Capabilities of QL Query Propagation to WSN
distributed middleware nodes and a central manager graph-based
full
full
full
limited
full
manual
hidden by Proxy Query Service (PQS)
manual partitioning, automatic “neighborhood optimization” pushing of selections and joins to REED/TinyDB
automatic
automatic
distributed processing of Borealis nodes having operators that can receive data from a WSN
distributed in-network stream processing
distributed in-network stream processing
distributed adapters data stream schemas
not required
distributed adapters data stream schemas and mapping
configuring of WSN and definition of virtual sensors
Data Processing
Technical Integration Semantic Integration
Metadata Management
PQS receives partial query; propagation unknown, but theoretically possible processing of hierarchical flow temporary from the PQSs to relations derived the Gateway from Query heterogeneous Service (GQS) data sources within the gateway and from all gateways to a central component distributed implementation of adapters PQSs temporary automatic service relations lookup and binding service create global data schema definitions of hierarchical virtual sensors lookup services
Stream Processing Engine
GSN
SStreaMWare gateways and control site
Support of Potential Users
virtual tables for automatic service SQL-queries discovery
code generation/ query mapping
virtual table “sensors”
distributed catalog distributed tables synchronized to the root node Borealis TinyOS applications and gateway box- and arrow-diagrams
virtual table “sensors” and TinySQL
global view on static and volatile metadata third party (Borealis, STREAM at the moment) holistic top-down approach
Integration of Heterogeneous Sensor Nodes by Data Stream Management
145
distributes subqueries as tree-based query plans to different gateways that offer a gateway query service. Each gateway query service organizes a further decomposition of subqueries and sends the smallest units of subqueries to the proxies of the diverse sensors. The heterogeneous sensors are integrated by a proxy query service that offers a generic interface. TinyDB [26] and Cougar [17, 35] are middlewares that facilitate SQL-like queries in a network of WSNs. Borealis [1] is an SPS that supports distributed stream processing by grouping operators on distributed Borealis nodes. Queries are defined by box-and-arrow diagrams. Grouping and distribution of operators has to be done manually. REED [4] realizes the integration of the WSN-application TinyDB [26] and Borealis. In [25], operators are pushed from Borealis to TinyDB manually. The results motivate us to make more efforts in Cross Border Optimization (CBO), i.e. distributing queries to both WSNs and SPSs. Tab. 1 compares the different approaches of using WSNs as configurable data sources for stream processing with DSAM. The compared systems have similarities in their architectures. There are distributed nodes (GSN containers, SStreamWare’s gateways, Borealis, etc.) with full capabilities of operators that integrate WSN nodes having lower capabilities. TinyDB is an exception: it manages only one WSN and has less capabilities compared to the other solutions that use full-fledged stream processing systems at higher level. The biggest differences of the approaches are the ways of integrating WSNs and propagating partial queries; Sect. 5.1 offers a detailed comparison. DSAM supports a direct propagation of partial queries to WSN nodes by using a top-down approach. The top-level query language of DSAM for global queries is graph-based. DSAM can integrate systems that use SQL-like query languages by query mapping. The main focus of DSAM is integrating existing WSNs and higher-order stream processing systems. The challenges are query partitioning, query mapping, and integration. The main advantage of DSAM is its ability to integrate already existing systems and configurable data sources.
2 Stream Model and Metadata Data-stream technologies can be a powerful tool for the integration of sensor networks as sensor data is streaming by nature. Their main promise is to ease the definition of complex queries by providing standardized tools and languages. The heterogeneity of sources however poses some integration problems. This section addresses two of these problems: The common definition of what comprises a stream and the required metadata that allows the global definition of queries. The integration of heterogeneous streaming data sources needs a common definition of a global stream model that encloses most of the existing and relevant stream models. The global stream model is the basis for the global query language as the different stream models are for the different query languages. In complex distributed scenarios, both heterogeneity and topology of sensor nodes have to be considered. Automatic operator placement furthermore needs performance characteristics of nodes, communication paths, etc. There are different solutions in the
146
M. Daum et al.
literature that deal with metadata in distributed stream processing systems and sensor networks.
2.1 Stream Model Deploying global stream queries to a heterogeneous network of nodes requires a common understanding of what comprises a stream. This section compares existing stream models, which are then used as a basis for a global stream model. There are some characteristics common to most stream models: Streams are “append-only relations with transient tuples” [7]. Transient tuples are either discarded after processing or explicitly stored. In data stream systems, tuples must have a well-defined schema; otherwise they cannot be processed. The SPS cannot guarantee any further properties for the arriving data stream tuples, because they are created externally to the system. An SPS may use different stream models internally and externally. The external stream model describes the properties a stream needs to have so that an SPS can process it, while the internal stream model is used inside the SPS. The external stream models determine which and how SPSs may be integrated with each other, while the internal stream models influence the operator semantics (Sect. 3.1). In the examination of existing stream models, we regard the following criteria as relevant for a classification: • Timestamps and validity Most SPSs use timestamps to denote when an event happened. Timestamps may be generated by the data sources (external timestamps) or by the SPS upon arrival of an item (internal timestamps). A second timestamp may be added to denote how long an item is valid. Without this information, it may be necessary to denote the validity of items in the query definition. Both variants may be combined (e.g. even if an item has an expiry date, the system may discard items at some other time earlier or later which is given in the query definition). Timestamps can be part of either the user data or the metadata. In the first case, queries may access (and possibly even alter) timestamps. In the second case, timestamps can only be used by the SPS internally. • Uniqueness of items Stream models differ in their guarantees for uniqueness of items. It is usually not possible to guarantee uniqueness of user data (a sensor node may return the same temperature value several times). If uniqueness of items is required, this has to be done via timestamps. In the absence of timestamps or if timestamps do not guarantee uniqueness (e.g. because of insufficient granularity), a simple sequence of numbers may be used. Uniqueness is usually more of a concern for the mathematically precise definition of semantics than it is for actual query processing. In addition to stream models, existing SPSs also differ in their delivery semantics. The following criteria may be used to distinguish systems.
Integration of Heterogeneous Sensor Nodes by Data Stream Management
147
• Lost or duplicated items Systems may react differently to lost or duplicated items. Unless there is a notion of a “next” item (e.g. by means of sequence numbers), it is not possible to detect if tuples have been lost. • Ordering Another interesting difference between SPSs is the issue of ordering. For a data source with either timestamps or sequence numbers, it is possible to wait for outof-order items and reorder them. This can be realized by different SPS-specific techniques like sorting with slack parameters [2], heartbeats [6], punctuations [24], or k-constraints [8]. If more than one data source is connected to an SPS, reordering depends on synchronized clocks. 2.1.1
Comparison of Stream Models
This section compares the models of several SPSs in order to identify their similarities and differences. STREAM STREAM [6] has a relational stream model. Each stream S is a bag (i.e. multiset) of items < s, τ >. τ ∈ T is an ordered set of discrete timestamps. The timestamp is not part of the stream’s schema. There can be more than one item with the same timestamp. As a stream consists of a multiset of items, uniqueness is not guaranteed. STREAM defines a relation as a multiset, too. Timestamps are external and the items are supposed to arrive in the correct order. PIPES PIPES [23] distinguishes between raw, logical, and physical streams. A raw-stream item (e,t) is an event that comes from an external data source and occurred at time t. System timestamps can be added to an event if no timestamp comes from the data source; when tuples arrive at the system, the input streams are implicitly ordered by system time. Items with external timestamps are expected to arrive in correct order. Otherwise an extra component reorders the items. Logical-stream items (e,t, n) have multiplicity n of elements e at a definite point in time t. The logical stream definition is used for defining the logical algebra. Physical streams are an outstanding concept of PIPES. Physical-stream items (e, [tS ,tE )) have two discrete timestamps. tS is the first point in time that the event e is valid at and is identical to the timestamp of the raw stream. tE is the first point in time at that the event e is not valid anymore; it is added by the window operator depending on the window size (windows are explained in Sect. 3.2). Physical streams are important for invalidation of tuples in sliding windows. From our point of view, only the raw stream definition is relevant to us, as it is the stream definition at the entry point of PIPES.
148
M. Daum et al.
Aurora Aurora [2] has a stream model (T S, A1 , . . . , An ) that is similar to the stream models of STREAM and PIPES. The timestamp is hidden from queries. Borealis [1] is the successor of Aurora. In Borealis the tuple of a data stream has the following schema2 : Key Attribute Tuple-ID Tuple Type tg tl A1 A2 . . . An Queries can only use the attributes A1 , . . . , An . tg is the global timestamp that is added by the system upon arrival of an item. tl is only used internally for measuring Quality of Service (QoS) parameters. Uniqueness is guaranteed by the tuple-ID. In this model, the timestamp tg is not the only relevant attribute for the correct order of tuples. Ordering can be required for each user data attribute. Operators in Aurora and Borealis can deal with disordered tuples by using so-called slack parameters for ordering. As Aurora and Borealis support load shedding, they can deal with tuple losses. Cayuga Cayuga [13] has a stream model that is founded on expressive pub/sub systems [12]. Events < a;t0 ,t1 > have a starting point in time t0 and an endpoint in time t1 . a is just an abbreviation for a relational tuple. Instantaneous events have identical timestamps t0 and t1 . The temporal order is defined by t1 , i.e. event e1 is processed before e2 if e1 .t1 ≤ e2 .t1 . GSN GSN [5] uses timestamped tuples, too. The management of the timestamp attribute (TIMEID) is implicit, i.e. all arriving tuples are timestamped automatically by a local clock. Timestamps can be used like normal attributes. More timestamps can be used by the query definition, e.g. both the implicit timestamp of arrival and timestamps emitted by the data sources can be used in the same query. 2.1.2
Towards a Common Stream Model
This section describes our efforts to develop a common stream model that needs to be more general than the described models. This common stream model should be mappable to different models without semantic loss. The disadvantage of a common stream model is the loss of some interesting concepts of special-purpose stream models. We will explain the decisions we made regarding the dimensions of classification mentioned above and discuss some preliminary ideas on how mapping between this stream model and the ones used by different SPSs is possible. 2
Code analysis of Borealis Version: Summer 2008.
Integration of Heterogeneous Sensor Nodes by Data Stream Management
149
Timestamps We assume that each item has at least one timestamp. This timestamp can be added if the data source does not provide it. Thus, there is no semantic guarantee about the timestamp. It may be the creation time of an item or the arrival time at the first system under our control. We prefer the former. Further concepts like duration in Cayuga can only be supported by using normal user data attributes. As queries often have to work with timestamps, we are convinced that timestamps should behave like normal attributes and be accessible to the query. Uniqueness As most systems do not require items to be unique and uniqueness may be difficult to enforce, our common stream model does not make any guarantees regarding uniqueness of items. Lost and Duplicated Tuples It may be impossible to determine if tuples are lost within a sensor network. Thus, our stream model does not guarantee completeness. Currently, the user has to understand the implications of using lossy transmission protocols. We assume that no duplicated tuples exist3 . Ordering In our stream model, items that arrive at a node have to be in correct order. This assumption is made by most of the analyzed systems. When timestamps are created by the system, this is always the case. Otherwise, existing solutions for sorting are used.
2.2 Metadata Metadata is a crucial aspect in the integration of heterogeneous nodes. It supports both domain experts and the partitioning process. We distinguish static and dynamic metadata. Static metadata usually remains constant (e.g. capabilities of nodes), while dynamic metadata may change while a query is running (e.g. data rates). Different kinds of metadata are used for query optimization, description of data sources, etc. Many of them are similar in most of the related systems. However, these systems usually differ in the storage of metadata and in the monitoring of dynamic metadata. Metadata in DSAM is described in a level of detail as far as it is relevant in the further sections. 3
This refers only to duplicates created by network protocols e.g. re-sending of tuples that were not acknowledged. Several tuples with identical values are still allowed.
150
2.2.1
M. Daum et al.
Metadata of WSN Middleware and SPSs in Existing Solutions
TinyDB TinyDB is a middleware for TinyOS nodes that supports queries similar to SQL [26]. There is a virtual table sensors that can be queried by TinySQL. Sensors consists of attributes like nodeid, light, temperature, etc. Each node in TinyDB has a local catalog containing its local attributes, events, and user-defined functions. Each attribute has the following metadata: • • • • •
Power: Cost to sample this attribute (in J) Sample time: Time to sample this attribute (in s) Constant?: Is this attribute constant-valued (e.g. id)? Rate of change: How fast the attribute changes (units/s) Range: Dynamic range of attribute values (pair of units)
The local catalog is copied to the root node for query optimization. Global Sensor Networks The virtual sensor definition is an XML-file containing the stream’s name, address, output structure, input streams, and SQL queries that led to the output stream [5]. The GSN middleware enables the access to other data sources by using different wrappers, e.g. TinyOS, cameras, and RFID readers. The GSN middleware manages all virtual sensor definitions centrally. For integration, the output structure of a virtual sensor’s stream and its name are relevant. The technical integration is realized by the available wrappers that are provided by the GSN middleware. Borealis Borealis has a local catalog within its query processor and a global catalog [1]. The local catalog holds all metadata that belongs to a local query fragment that is processed by the query processor. As Borealis is a distributed SPS, the global catalog stores all information about distributed query fragments. All dynamic metadata is gathered by monitoring components. PIPES PIPES uses static and dynamic metadata and propagates dynamic metadata through the query graph [9]. This is necessary for adaptive query processing. SStreaMWare SStreaMWare [20] suggests a management system that offers all relevant metadata [19]. This includes topology, energy, CPU, memory, etc. The management system centralizes all dynamic metadata.
Integration of Heterogeneous Sensor Nodes by Data Stream Management
2.2.2
151
Metadata in DSAM
This section discusses the metadata catalog used by DSAM. As requirements for metadata may change, its main focus is on extensibility. We discuss only the elements that are necessary for integration of different SPSs and partitioning of global queries. There are extensions for a more detailed description of data sources and sensor data that are necessary for both semantic descriptions and for cost estimation; their description is beyond the scope of this chapter.
spsdatatypes
datatypes
spstypes t
operators t
spsoperators t
fields
parameters t
nodes queryusesnodes
queryusesoperators
hosts connection
streams
queries
queryusesqueries
schemas h
queryusesstreams
Fig. 3 Metadata of DSAM
DSAM uses the same conceptual schema for both static and dynamic metadata, i.e. all data can be accessed in the same way. Fig. 3 gives an overview of the catalog; the arrowheads denote the references. The whole scenario contains a number of nodes that are deployed to hosts that are connected to each other. Here, only direct connections are stored; transitive connections are considered by the query processing. Nodes furthermore have an SPS type. An SPS type contains all information that is needed for the technical integration during the code generation process. The list of available operators is necessary for the partitioning. The streams table stores all streams, i.e. input streams, output streams, and streams between nodes. Internal streams of partial queries that run on a node need not be stored in the catalog. Both global queries and partial queries are stored in the queries table while their relationship is stored in queryusesqueries. From a conceptual point of view, queries at PIM- and queries at PSM level are stored in the same way. The generation of partial queries creates internal streams among the partial queries. These internal streams are the relevant information for the integration of heterogeneous nodes and correspond to the PSM bridges (Fig. 2).
152
M. Daum et al.
While most solutions offer a central metadata catalog, some also make extensive use of local metadata. DSAM only relies on the central catalog. As we want to integrate different third-party SPSs, it is not feasible to use local metadata. The main feature of DSAM’s catalog is the management of heterogeneous SPS types that allows to integrate new SPS types and operators.
3 Definition of Queries Up to now there are no standards for query languages that can be used as a global query language. In order to deploy global queries to a heterogeneous network of nodes, a global query language is required. At the beginning, this section explains concepts of existing data stream queries that are relevant for queries in WSNs. Afterwards this section explores different types of proposed stream query languages and then introduces Abstract Query Language (AQL), the query language used by DSAM.
3.1 Operators A data-stream operator receives items from one or more input streams, creates items, and delivers them to one or more output streams. In general, operators are application-independent and their output can be used as an input for the next operator in a directed graph of operators. The literature distinguishes between blocking (join, aggregate, sort, etc.) and non-blocking operators (selection, projection, simple functions, etc.) [7]. Non-blocking operators can directly create output items when an item is received. Blocking operators have to wait and collect a sequence of items from one or more input data streams in order to create an output item. Another classification of operators distinguishes between stateful and stateless operators [1]. In most cases, blocking operators are also stateful operators. Most SPSs support stateless relational operators like selection, projection, and renaming with no relevant semantic differences. These operators can easily be implemented for WSN nodes with low capacities, too. Stateful operators can only be deployed on WSN nodes if the size of the state does not exceed the available capacity. In a distributed scenario, it is helpful to deploy selection and aggregation operators to WSN nodes as this reduces the amount of communication and saves energy. For the definition of the relevant sequence of items, most SPSs use window definitions. Window definitions are used for e.g. sort, join, and aggregate. The window definition describes the operator’s evaluation. If extra-regular evaluation is needed, most SPSs can use a slide parameter additionally to a window definition. Slide parameters bring the aspect of continuousness into queries as they define the cycle of evaluation. There are relevant differences in definition and implementation of windows as the next section will show.
Integration of Heterogeneous Sensor Nodes by Data Stream Management
153
3.2 Window Definition and Time Semantics Stateful operators differ in their window definitions as there are operator windows and source windows. Operator windows are windows that may be defined for each operator [1, 2]. In contrast, source windows are defined for input streams. They are mostly used by SPSs that define their semantics on the relational data model [6, 23]. In heterogeneous networks, it might be relevant to map both window definitions to each other. GSN [5] does not have this problem as window definitions are part of the virtual sensor definition; window definitions behave like source windows. The definition of windows is directly connected to a stream (virtual sensor definition). By the window definition, each input stream is evaluated and stored in a temporary relation. GSN uses the input-triggered approach, i.e. all temporary relations are updated when a new item arrives, and all consumers of this virtual sensor are notified. This is possible as the joining of data sources is done in the GSN containers. In most systems, windows are time-based, i.e. an item is valid for a period of time. Other possibilities are count-based, i.e. an item invalidates when a certain number of items has arrived. Value-based is more general than both time-based and countbased. Value-based means that any attribute can be used for the decision of expiry of an item. Queries having operator windows can be easily mapped to SQL queries having source windows by using subqueries. Mapping source windows is trickier as the global definitions of source windows cannot be mapped to local operator-window definitions. E.g. this problem occurs for a source-window definition that includes the last 10 tuples. If the first operator is a filter operator and the second operator should calculate an aggregate function, an operator window of the aggregate operator with size 10 would not accord with the source window. The size of the operator window would depend on the selectivity of the filter operator in order to achieve the same semantics as the source operator. A workaround is using a value-based window and an ID or an auxiliary attribute that enumerates the stream elements. Expiring elements can be calculated by the enumeration. There are differences in the semantics of window definitions that rely on implementation details. Due to the characteristics of blocking operators, the arrival of an item does not necessarily produce a result. And due to query descriptions, it is not the arrival that is relevant but a sequence of arrivals or a period of time. There are different possible points in time for creating results. • Input triggered: The operator is triggered by the arrival of an item • Local-clock triggered: The operator is triggered by a clock • Pull-based: The operator is activated by its successor Input triggered can lead to imprecise evaluation of slide parameters as results are only created when items arrive; systems like PIPES rely on incessantly sending data-stream sources for precise semantics. Local-clock triggered means an operator sleeps, but it can wake up by itself. Pull-based is very precise but loses the advantages of SPSs as they act like Database Management Systems (DBMSs) and may have high latencies. A solution for improving the quality of input-triggered
154
M. Daum et al.
evaluation is negative tuples. A special operator at the entry of the SPS sends a negative tuple if the according tuple expires [18]. An efficient solution for source windows in distributed environments like WSNs could be the approach of the physical stream in PIPES. Here, a sensor node would add an expiration timestamp to each tuple. This approach would however need a homogeneous implementation of operators using windows in different SPS types. As these implementations are heterogeneous, we suggest using either source windows for each node or operator windows.
3.3 Types of Query Languages Recent years have seen the introduction of several query languages. Almost every SPS defines its own query language. We distinguish three families of query languages: programming languages, SQL-like languages and graph-based languages. 3.3.1
Programming Languages
The user writes code that produces the desired result. Examples for this family are e.g. nesC [15] (TinyOS), and Java (SunSpot). Programming languages are unmatched in extensibility and expressive power. They are usually easily mappable to systems that are freely programmable like sensor nodes - only a suitable compiler is needed. However, they cannot be mapped to SPSs that provide their own high-level languages. Programming languages are difficult to use for domain experts who are not programmers. Furthermore, it is difficult to automatically split a single program in order to deploy its parts to different systems. While this may be mitigated by special language constructs, the use of such constructs places an additional burden on the application developer. For these reasons programming languages are of limited value for the definition of global queries. 3.3.2
SQL-Like Languages
Some SPSs, namely STREAM, Nile [18], TelegraphCQ [11], and PIPES use SQLlike queries. As they are historically based on database technologies and have descriptive query languages similar to SQL, these systems are often called Data Stream Management Systems (DSMSs) in analogy to DBMSs. The DSMS translates the query into a query graph. The whole query is in focus of interest and there is no explicit definition of operators. As the relational approach is recycled, the query language maps streams to temporary relations. The user adds window definitions to the streaming data source in order to define a set of consecutive tuples. As a query does not have any further window definitions, we call these windows “source windows”. Negative tuples realize source windows in Nile and STREAM. PIPES uses its physical schema. The set of supported operators is limited to the relational operators due to the query language definition. User-Defined Aggregates (UDAs) can extend the set of available aggregate functions. One weakness of SQL-like languages might be the
Integration of Heterogeneous Sensor Nodes by Data Stream Management
155
set-oriented data model. The temporal sequence of data stream items matters both in stream processing and in query definition. Especially sensor-data processing and Sensor Data Fusion (SDF) need a set of special stream operators that should be part of the set of core operators. SQL-like languages allow the user to describe the relevant result of a query without thinking about how it is obtained. This allows the system to apply various optimization techniques as it is not restricted in how it computes the results. SQL-like languages play a major role in relational database systems. Before being executed, SQL-like queries are translated to a query graph (more specifically: a tree) where the nodes consist of operators and the edges represent the data flow among operators. This makes partitioning easy: any subset of operators may be placed on a system. The fixed structure and small set of available operators makes it simple to map SQLlike languages to different target languages. This fixed structure is also their greatest weakness: queries must be representable as trees and made up of a relatively small set of simple operators. While some SQL-like languages may be extended by new aggregate functions, it is not possible to add new operators. This would be difficult as each type of operator has its fixed position in an SQL-like statement and new operators do not have a “natural” position where they belong (though usually the only reasonable place for new operators would be in the FROM-clause). An example for an operator that might be relevant for SDF is RESAMPLE in Aurora [2]; this operator has no analogy in SQL-like query languages. Perhaps for this lack of extensibility and expressiveness, available commercial products (e.g. Streambase4 , System S [16]) use graph-based languages instead of SQL-like ones. 3.3.3
Graph-Based Query Languages
Graph-based query languages reflect the logical flow of data items through a network of connected operators. They define a query as a graph where the nodes represent operators and the edges represent data flow among operators. Existing graph-based languages differ in a couple of ways. The languages are usually closely tied to a certain system; thus some of the differences (especially syntactic ones) are actually properties of the implementing system and not of the languages themselves. Boxes and arrows are used by process-flow and work-flow systems. [10] describes the idea of box-and-arrow query graphs. The query graph represents the data flow. It is directed and acyclic in the case of Aurora. Borealis and Streambase are founded on Aurora. System S [16] proposes a descriptive graph-based query language. Basically, graph-based query models need directed graphs that do not have to be acyclic; e.g. Borealis can handle cycles in queries. Each node is individually configurable in a query graph and represents an operator. An operator may change the data stream’s schema. The operators’ configuration determines the inner schemas. Inner schemas are either derived automatically or have to be declared by the user. 4
www.streambase.com
156
M. Daum et al.
The set of supported operators is conceptually unbounded. Most graph-based query languages support relational operators and an additional set of special stream operators. Those special operators are enormously important in real data-stream scenarios with sensor-data fusion. Defining a core set of relevant stream operators is still an open problem. Graph-based query languages are extensible: All that is needed is a new operator implementation and an extension of the language grammar that allows this operator to be used. Such languages may be partitioned and mapped just like SQL-like languages. Their expressive power depends on which graphs may be represented in a given language. Cycles, multiple output streams or multiple inner streams are examples of constructs that cannot be represented in SQL-like languages but possibly in graph-based ones. Graph-based query languages differ in their usability, which can be partly attributed to their different focus - who is expected to write (and read) queries and how often. Some systems provide graphical editors that make it possible to visually create queries - which is usually the easiest way for domain experts.
3.4 AQL – The Query Language of DSAM DSAM processes global queries and configures the nodes, i.e. it partitions global queries, maps partial queries to either operator assemblies or special destination languages, and deploys them on the adequate nodes. This supports the integration of heterogeneous stream-emitting and stream-processing nodes, in order to deploy large queries. Some sensors and data sources emit data that is processed by different types of SPSs (Fig. 4). We assume that users want to describe their needs in form of a query by using a uniform query language without considering the topology of sensor nodes. AQL is SPS-independent and used as global query language. It is completely descriptive by describing just data sources, data sinks and abstract operators. There are three classes of abstract operators: monadic operators, combining operators, and separating operators. The membership of an abstract operator in one of these classes is determined by its number of input and output streams. Monadic operators have one input and one output stream like the filter, map, or aggregate operators. Operators of the combining-operator class are characterized by one output stream
Fig. 4 Scenario of heterogeneous stream components
Integration of Heterogeneous Sensor Nodes by Data Stream Management
157
and a minimum of two input streams e.g. the union or join operators. A member of the separating-operator class has one input stream and multiple output streams, e.g. the split operator. Figs. 5 and 6 show an excerpt of AQL’s syntax and how the three operator classes are placed in a query.
Queries can use nested subqueries as source streams. Each subquery can unite different streams and separate the streams once. This pattern of subqueries allows arbitrarily complex directed graphs. The following excerpt of an AQL query shows two streams S1 and S2 that are combined and split into two streams. Sources and sinks have distinct addresses that are stored in the catalog (Sect. 2.2.2). There is a subquery that adds time information to stream S2. The subquery only consists of the combining operator Merge. The top-level query has three monadic operators, the combining operator Union and the separating operator Split. The decorator $2 assigns the Filter to the second input stream, i.e. the resulting streams of the subquery.
Fig. 6 Combining, monadic, and separating operators in AQL
AQL supports both source windows and operator windows. Mapping between source windows and operator windows is tricky but possible by insertion of additional fields (map operator). When we designed AQL, we wanted a language that is both extensible and SPSindependent, so we chose a graph-based language. The set of operators can be easily extended and queries can be designed by an intuitive GUI tool. In order to remain SPS-independent, AQL defines a set of abstract operators. An abstract operator can be mapped to different operator implementations on different SPSs as long as they adhere to the specification of the abstract operator. The set of allowed mappings is stored in the catalog. Some SPSs use advanced operator semantics for standard operators; e.g. Aurora can add a sorting to some operators that need tuples in a certain
158
M. Daum et al.
order. As the specification of an abstract operator is the lowest common denominator of the operator implementations that it is mapped to, these special cases are not supported up to now. It is possible to define an abstract operator that behaves exactly like an advanced operator of an SPS. However, this reduces SPS-independence as such an operator cannot be mapped to other SPS types.
4 Partitioning Partitioning refers to splitting a global query into parts that may be individually deployed to distributed nodes. Deployment consists of two tasks. First, it has to be decided what is to be deployed where. Second, a technical infrastructure is required that can automatically implement this deployment. We refer to the first task as “deployment decision” and to the second one as “deployment implementation” or simply “deployment”. Especially in heterogeneous environments, partitioning and deployment decision are difficult to separate because constraints on possible deployments affect possible partitionings. Thus, we discuss both problems in this section and subsume them under the term “partitioning”. Related literature calls this problem “operator placement” decision [22, 27]. We will discuss the technical realization of the deployment in Sect. 5. Partitioning is a crucial step as it greatly influences the quality and the performance of a distributed query. As quality and performance requirements are query-dependent, a highly flexible partitioning process is required. Furthermore, partitioning has to consider many constraints, e.g. regarding availability of input streams, capacities of nodes, etc. Metadata is essential for partitioning as it contains all information about data sources, topology of nodes, and even the set of available operators. During cost estimation, performance characteristics and knowledge about streams is used to choose a “best” plan according to some objectives. There are conflicting objectives like minimizing CPU load and energy consumption on a node, maximizing the data quality (reducing load-shedding), minimizing latency, etc. In most database systems, the number of input and output operations is a reasonable metric for costs. In distributed data stream systems, metric and objective strongly depend on the concrete scenario. A solution is weighting estimated costs for all relevant objectives [27]. In the remainder of this section, we discuss how other projects deal with partitioning before we explain the solution we have chosen for DSAM.
4.1 Operator Placement Decisions in Existing Work TinyDB [26] has the same query on each node, the GSN middleware [5] focuses on the integration of different sensor data streams. In both approaches, no operator placement decision is necessary.
Integration of Heterogeneous Sensor Nodes by Data Stream Management
159
Stream Based Overlay Network (SBON) [27] provides an approach for operator placement decisions in a network of homogeneous nodes of distributed SPSs. The cost model uses a blended metric, i.e. it tries to optimize two or more metrics (e.g. latency, load, etc.) at once. The approach maps the metrics to a virtual cost space. This virtual space is a continuous mathematical and multi-dimensional metric space. Euclidean distances correspond to the optimized cost metric. All nodes have positions in this virtual cost space. Nodes can be producers, consumers, and processing nodes that can be used for placing an operator. The virtual cost space is modeled after physical rules. The model equates costs with spring pressure. Relaxation models the overlay network having massless bodies (operators) connected by springs (operator links). The optimization goal is spring relaxation that places the operator on a node in a way that spring pressure is minimized. [30] assumes sensor network nodes with low capabilities acquiring the data and a hierarchy of nodes having increasing network bandwidth and computational power. They provide an algorithm that offers a globally optimal solution for the placement of expensive conjunctive filters and joins. [36] considers the problem of placing both operators and intermediate data objects. Intermediate data objects are objects containing data that are neither sensor data nor final data. A set of queries is optimized so that the total cost of computation, memory consumption, and data transmission is minimized. One of the most relevant differences to other approaches is that they require only information exchange between neighbors.
4.2 Operator Distribution in DSAM In DSAM, the task of the operator distribution process is both to split a global query into several so-called partial queries and to decide which partial query should be deployed on which node. Thus, the partial queries are the unit of distribution. The deployment process is facilitated by an extensive metadata catalog. The catalog holds a list of available nodes that can run an abstract operator. We assume that some operators are not available on each node. For example, some special operators for data fusion or User-Defined Operators (UDOs) might be installed on a few nodes only. This leads to structural constraints on a query’s deployment: • • • •
Input streams are available at a certain place/node Some nodes can execute some abstract operators that others can not Nodes have individual capacity and performance behavior Nodes’ connections have reachability constraints and performance behavior
The partitioning process can be realized in an arbitrarily complex way, and partial queries may be of different granularity. For the sake of simplicity, we first assume individual operators as unit of distribution. Naively, each operator may be placed on an arbitrary node. For n operators and m nodes there are nm possible distributions, one of which has to be chosen. This number is too large for an exhaustive search. On the other hand, there are some additional restrictions. Some operators may not
160
M. Daum et al.
be available on all nodes. Nodes and communication paths have limited capacity. Furthermore, only a few distributions are good enough to be considered at all. In mathematics, this problem is known as Generalized Assignment Problem (GAP). In a GAP, tasks have to be assigned to agents in a fashion that minimizes some global cost criterion. The costs for a task may vary among agents. Each agent has a capacity that must not be exceeded. We model operators as tasks and nodes as agents. Unfortunately, GAP is an NP-complete problem, thus it can only be solved approximately in acceptable time by using heuristics. For the discussion of possible solutions to the GAP, the following definitions are necessary: T
= { t | t is an SPS type }
(1)
Nt N
= { nt | nt is an SPS node and t ∈ T } = t∈T Nt
(2) (3)
Ot = { ot | ot is an operator type and t ∈ T } O = t∈T Ot C
= { c | c is a cost model }
(4) (5) (6)
Further, we have the global query graph that is a directed graph G = (V, E) containing vertices V and edges E. Vertices represent abstract operator instances. M ⊆ N defines the set of SPS nodes that are available for a query. We have different cost models for different SPSs. The costs for edges comtranscost (sending items between two operators) consist of communication and transformation costs. We assume no costs if two adjacent operators are assigned to the same node. In short, the costs of executing an operator instance v on an SPS node m are: (7) cmv = load(m, v) + comtranscostv load(m, v) calculates the costs for running an operator on a node. A further explanation of load(m, v) is beyond the scope of this chapter. With these definitions, we can adapt the GAP to the needs of DSAM’s partitioning process: xmv ∈ {0, 1},
xmv denotes whether an operator instance v is running on node m. bm is the capacity of a node, and amv is the load that an operator v causes if it is run on m. Equation 8 postulates that an operator instance is either deployed on a node (1) or not (0), while equation 9 simply states that each operator instance is deployed on exactly one node. Equation 10 defines the primary optimization goal: to minimize the total costs incurred by each operator instance running on its respective node. Equation 11
Integration of Heterogeneous Sensor Nodes by Data Stream Management
161
places further constraints on acceptable solutions: No node may be burdened with a total load that exceeds its capacity. Each connected group of operators that run on the same SPS is combined to a partial query. We save all partial queries in the catalog of DSAM. Each partial query uses a set of input streams and has a set of output streams. In Fig. 7, the relevant excerpt of the catalog shows the dependencies among partial queries, global queries, and “inner” schemas. The schemas of output streams are defined by the partial queries (fk pub query).
queryusesqueries query parent
fk_query
index
fk_parent
queries id name query graph
queryusesstreams
fk_query
query
fk_pub_query
stream
streams
fk_stream id name
schemas
pub_query schema
id
ext_source
name
fk_schema
Fig. 7 Partial Queries in Metadata
The discussed idea relies on the ability to estimate costs for executing a partial query on a node as well as for transmitting results to the next node in the query graph. These estimations are derived from existing metadata. While topology, node types, and available operators are known, good estimations for data rates, selectivities, etc. are difficult to obtain. In contrast to other solutions, DSAM’s partitioning is more general as new systems or operators do not require any changes to the algorithm; it suffices to configure the cost estimators. Our current implementation allows balancing quality of operator placement and the time spent on finding a solution. This may be beneficial for large networks where an exhaustive search for a best solution is not feasible.
5 Integration of WSNs The partitioning step of the previous section leads to partial queries. In top-down approaches such as DSAM, these partial queries have to be deployed to the distributed nodes. In other approaches, applications for WSNs and the query of the SPS are
162
M. Daum et al.
independently developed and deployed. In this point, most existing approaches and DSAM’s approach differ. All related approaches use WSNs as data sources, e.g. for surveillance. The processing within the WSN nodes is kept simple due to restricted capabilities. The common assumption is that the result of a WSN or rather a WSN query is a data stream that can be processed by SPSs.
5.1 Integration of WSNs in Existing Approaches SStreaMWare StreaMWare [20] has a three-level query processing. The control site is the central manager of SStreaMWare and provides a global Sensor Query Service (SQS). It communicates with gateways that might be widely distributed on different hosts. Each gateway has a set of Proxy Query Services (PQSs). Each sensor network needs an implementation of a PQS in order to provide a uniform interface that is used by the GQS. The implementation of a PQS can be either an adapter or a proxy. A PQS adapter communicates with a proprietary proxy of a WSN and can only use the functionality of this proxy. A PQS proxy communicates directly with the sensors. This enables data processing both within the sensor network and within the PQS. Having a uniform interface, all PQSs can conceptually process the same kind of partial queries. As a minimum, they can process the relational selection and projection [20] within the PQS. PQSs provide a homogeneous view over heterogeneous data sources. PQSs differ in the provided attributes. The management of provided sensors and attributes is an outstanding feature of SStreaMWare. An instance of a Lookup Service (LS) runs on the central manager and on each gateway. These LSs provide information about available sensors that can be used for queries dynamically without having a central registry. GSN GSN [5] uses the concept of virtual sensors in order to get a homogeneous view on heterogeneous sensors. Each virtual sensor definition describes a temporary relation. From a technical point of view, there are two groups of adapters: remote adapters and local adapters. Remote adapters refer to virtual sensors that are installed on remote GSN containers. Local adapters support different kinds of data sources. These adapters offer a streaming data source. A query can only use the given temporary relations. In contrast to SStreaMWare and DSAM, top-down propagation of partial queries is not in the focus of GSN as the approach is bottom-up.
Integration of Heterogeneous Sensor Nodes by Data Stream Management
163
TinyDB TinyDB [26] is a TinyOS application. TinyOS provides the technical integration, communication, etc. As TinyDB is installed on all nodes, the view to single nodes is quite homogeneous. Integration is done by the base station. It parses a query and propagates it to the sensor network. Query propagation forms a routing tree having the base station at the root. All sensor data is sent back in reverse direction of the query propagation. Borealis/REED Some approaches use TinyDB as a configurable data source. REED [4] integrates TinyDB and Borealis. Its integration framework [3, 29] uses wrappers that provide a standardized API for the sensor networks. Each query processor of a Borealis node has a proxy that gathers statistics about the sensor networks from the wrappers. These statistics include constraints that are necessary in order to reject impossible operator movements. Further, the proxy organizes the optimization, i.e. pushing operators to the sensor network.
5.2 Code Generation, Deployment, and Integration in DSAM DSAM maps partial queries to the corresponding platform-specific query languages. Fig. 8 shows the whole deployment process of a global query on different kinds of nodes. The results of the partitioning process are partial queries that can be deployed on the according node, i.e. the node must provide all necessary operators. We sketch the mapping process of partial queries to target query languages, the generation of source code for SPSs that are just programmable and do not support query languages, and the deployment of partial queries. A short example supports the presentation of our concept. 5.2.1
Mapping
The query mapping is a two-phase process and supports different target languages. Mapping corresponds to the second transformation of the MDA approach (Figs. 2, 8). It takes an abstract operator graph representing a partial query as input and generates queries in the specified query language as output. As preconditions for the mapping process, DSAM has to provide for each target language: • Operator transformation rules Each rule transforms the syntax of an abstract operator into the syntax of the target language and adds metadata. • Target language template A template defines the structure of a target language and can also make use of information from the catalog.
In the first phase we map an abstract operator graph to intermediate data structures. The second phase uses the query language template and constructs the target language queries from the intermediate data structures. Each node is of a specific SPS-type and therefore supports a specific query language. A query-mapper component transforms the partial query, which will be deployed on a specific node. Fig. 9 gives an example for mapping a partial query to an SQL-based query language.
PartialGraph Node
S3
S2
Filter
Filter
Split
Filter Union
S1
S4
CREATE STREAM S3 AS SELECT * FROM S1 WHERE expr1 AND expr3 AND expr4 UNION ALL SELECT * FROM S2 WHERE expr2 AND
Mapped Queries expr3 AND expr4;
CREATE STREAM S4 AS SELECT * FROM S1 WHERE expr1 1 AND expr3 3 AND expr5 5 UNION ALL SELECT * FROM S2 WHERE expr2 AND expr3 AND expr5;
Fig. 9 Query mapping example for SQL-based target languages
Integration of Heterogeneous Sensor Nodes by Data Stream Management
Fig. 10 Generated sample application code for a BTnode
5.2.2
Code Generation
Some stream processing components do not support query languages. Especially data sources like WSNs have limited capabilities and can only be configured by individual software deployment. In [14], AQL is used for global queries that are deployed on different BTnodes. A BTnode is a typical sensor node developed at the ETH Zurich, which is based on an Atmel ATmega128 micro controller. DSAM supports the invocation of code-generation modules. The result is code that uses a set of operators and system components. Let us revisit the example from Sect. 1.1 which tracked animals’ stress levels. Fig. 10 shows an excerpt of generated BTnode code for this scenario. The partial query has an external stream and a local sensor as input streams. It uses a join operator with operator windows and a merge operator. The catalog of DSAM knows which operators are available on a node. The resulting code can be linked and deployed in the usual way. 5.2.3
Deployment and Integration
Technical integration is achieved during mapping and deployment with the help of the knowledge about platforms and connections (Fig. 8). The addresses of inner streams between SPS and WSN nodes can be directly derived from the knowledge
166
M. Daum et al.
about partitioning. This knowledge helps configuring adapters or generating the source code (Fig. 10) of WSN applications. For the deployment we assume direct contact between DSAM and the node or multi-hop deployment5. The deployment on BTnodes is described in [14]. The main differences between DSAM and other approaches are: DSAM exploits the MDA analogy in order to deploy global queries automatically to heterogeneous nodes. This makes it very flexible, as code can be deployed on target systems in a generic way. The target systems are not required to run any special software (as opposed to e.g. TinyDB). In the absence of such special software, a central manager is required. In this respect, DSAM is similar to SStreamWare or TinyDB as opposed to systems like GSN or Borealis that do not require a central manager.
6 Examples and Evaluation To demonstrate the benefits of operator propagation to sensor networks, we present further examples and give the results of some measurements that support our approach.
6.1 Mapping to SQL-Like Query Languages for WSNs The fictitious example scenario for the usage of DSAM is a modern hospital that monitors vital signs of patients with the help of wireless sensors attached to a patient. Each of these wireless sensors emits a data stream with the patient’s id and the vital signs. For tracking, all patients are equipped with a Radio Frequency IDentification (RFID) chip that emits the patient’s id to a reader. RFID readers are distributed over the hospital and form the global stream POSITION that contains PAT ID, Antenna ID, and Time. An SPS combines the POSITION stream and the streams emitted from the wireless sensors of each patient. Physicians can configure which of a patient’s vital signs they want to monitor. 6.1.1
Example Query
For an example query, we assume a filter criterion for the heartbeat rate. The minimum threshold for the heartbeat is 60 beats and the maximum threshold is 160 beats. We are only interested in the last position and the current vital signs of the patient. Therefore we define a count-based window of size 1 on the POSITION stream and a time-based sliding window of 1 second on the PAT SENSOR 1 stream. The additional partitioning ensures that the last position and the current vital signs of all patients are going to be recognized. These requirements result in the AQL query in Fig. 11. 5
At the moment we are using Deployer components for the SPSs and direct flushing for the WSN nodes.
Integration of Heterogeneous Sensor Nodes by Data Stream Management
The abstract operator graph for the above AQL query is shown in Fig. 12. From this abstract operator graph, the distributor component creates two partial queries. The distribution algorithm pushes the filter operator to the wireless sensor of the patient.
WSN
Join
Pat_Sensor_1
SPS Filter Pat_Stat
Position Fig. 12 Abstract operator graph with partitioning
Partial query one is mapped to the target language of the wireless sensor and partial query two to the target language of the SPS used in the scenario. We assume SQL-based query languages for both nodes and obtain the following two queries as a result:
CREATE STREAM QUERY_STREAM_1 AS SELECT * FROM SENSORS WHERE Heartbeat <= 60 OR Heartbeat >= 160 CREATE STREAM PAT_STAT AS SELECT * FROM QUERY_STREAM_1[PARTITION BY Pat_ID RANGE 1s], POSITION[PARTITION BY Pat_ID ROWS 1] WHERE QUERY_STREAM_1.Pat_ID = POSITION.Pat_ID
Fig. 13 Query mapped to SQL-like language
Note: Internally all wireless sensors have only a stream called SENSORS. So we map stream PAT SENSOR 1 to SENSORS. This mapping has to be configured in the metadata catalog.
168
M. Daum et al.
6.2 TinyOS Application Fig. 14 shows an excerpt of a runnable TinyOS application. This is a case study that is realized in TinyOS2 and is run on Intel Mote 2. All operators are realized as configurable components. The predicates have to be implemented in a way that can be interpreted by the operator instances. In the given example, all configured operators are wired. The generator for TinyOS code is still work in progress and its use still requires some manual work. Yet, the results obtained so far absolutely meet our expectations. We expect DSAM to completely generate these wiring applications in the near future.
#include "../../QueryProcessor.h" #define COMPOSITION_TEST_1 Composition_Test_1 configuration TestAppC {} implementation { components LedsC; components MainC; components TestC as App; components SensorC; ... /* stream components */ components new StreamC() as SensorStream; components new StreamC() as FilterOutputStream1; components new StreamC() as FilterOutputStream2; ... /* operation components */ // temp < 100 components new FilterOpC("02:04:100", 1); // output field = room , temperature. group by room. order by room. window size = ← 2. sliding window = 2. aggregate function = avg(temperature) components new AggregateOpC("05,02", 2, FIELD_NAME_ROOM, FIELD_NAME_ROOM, 2, 2, ← AGG_FUNCTION_AVG, FIELD_NAME_TEMP); components new MapOpC("?:02+2:?:?:05+1", 5); // tid = tid, temp = temp + 2, light = ← light, humidity = humidity, room = room + 1 // predicate: left.room == right.room. output fields = left.room, left.temperature, ← right.light. left buffer size = 2, right buffer size = 2 components new JoinOpC("05:01:05", 1, 2, 2, 1, 1, "L05;L02:R03", 3); /* data sender components */ components new DataSenderC(1, ACTION_SEND_TO_CLIENT) as DataSender; ... /* wiring */ /* SensorC */ SensorC.StreamWriter -> SensorStream; /* FilterOpC */ FilterOpC.InputStreamReader -> SensorStream; FilterOpC.OutputStreamWriter_1 -> FilterOutputStream1; FilterOpC.OutputStreamWriter_2 -> FilterOutputStream2; /* AggregateOpC */ AggregateOpC.InputStreamReader -> FilterOutputStream1; AggregateOpC.OutputStreamWriter -> AggregateOutputStream; ... /* DataSender */ DataSender.StreamReader -> JoinOutputStream; DataSender.AMSend -> AM.AMSend[AM_QUERY_RESULT]; DataSender.Packet -> AM; }
Fig. 14 TinyOS application
Integration of Heterogeneous Sensor Nodes by Data Stream Management
169
6.3 Measurements [25] summarizes the results of [32]. In this earlier work, neighborhood-optimization - a concept of Borealis [1] - is applied to TinyDB as a neighbor of a Borealis node. Figure 15 shows the development of the estimated lifetime. The gap is caused by relocating the aggregate into the sensor. After the relocation the estimated lifetime increased enormously. Though this is not a result of DSAM, we expect similar improvements with DSAM. 140000
120000
lifetime
100000
80000
60000
40000
20000
0 1
5
10
15
20
25
30
35
Fig. 15 Improvement of the lifetime score
7 Future Work – Maintenance and Adaptation of Query Plans in Heterogeneous WSNs Queries in WSNs may run for a significant amount of time during which the environment may change drastically. This may happen due to failing nodes, changing data rates, or changing topologies if WSN nodes are mobile or connections fail. In this case, it may be beneficial or even necessary to redistribute a global query. If it is possible to just stop the old query and distribute a new one, this problem is trivial with the help of an infrastructure like DSAM. However, taking a query offline during redeployment is often not feasible - continuous queries are expected to return results continuously. Especially in the presence of stateful operators (e.g. aggregates), intermediate results may be lost by redeployment. An interesting solution to this problem is hot redeployment, i.e. the deployment of the new version of a query while the old one is still running as well as a seamless switch to the new one. A technical solution for storing the state in BTnodes during redeployment can be found in [14]. Automatic redeployment decisions pose some interesting additional problems: First, monitoring has to gather information about nodes’ load, battery state, etc. Second, it has to be determined when a possible redeployment should be computed. A periodic redeployment strategy computes a new partitioning in certain time intervals. It may be a good choice if the partitioning is relatively cheap to compute and a relatively high gain by redeployment is expected. Its major advantage is that it does not require additional monitoring data.
170
M. Daum et al.
An on-demand-strategy triggers a redeployment computation in the case of certain events, e.g. the addition or removal of nodes or if some nodes are overloaded. This has the advantage of consuming CPU power for computation only when actually necessary. However, in some cases, it may be too late (e.g. when overloaded nodes are used as a trigger). It relies on the existence of external events that may be used as triggers. A heuristic strategy is quite similar to the on-demand-strategy, but more general. While the latter relies on trigger events that require a redistribution, the former utilizes “rules of thumb” to determine when to compute a redistribution. To our best knowledge, no heuristics exist for solving this problem.
8 Conclusion This chapter presented an approach for efficient query definition and deployment in heterogeneous WSNs and their integration with SPSs. Its main idea is the definition of global and abstract queries on streaming data by users that are not used to the programming of WSN nodes. For this purpose, we defined a data model for streaming data and considered an appropriate query language. All descriptive information of data sources are stored in a catalog. The definitions of queries include the choice of the relevant data sources, the definition of a graph-based query, and the destinations of the query. Distribution of queries and their deployment are not left to the user but done automatically by the middleware. The prototype of DSAM can do the partitioning of data stream queries. It maps partial queries to traditional query definitions of existing SPS products and supports code generation for programmable WSN nodes. In our earlier work [25], cross-border optimization between TinyDB and Borealis is presented. The measurements that are related to this paper but could not be presented due to space limitations showed that pushing operators to the WSN can extend the lifetime many times over. These results motivate us to integrate crossborder optimization in the query partitioning process. Up to now most query languages do not consider time and quality constraints. Most SPSs are best effort DSMSs [28]. First approaches deal with quality constraints [33] by defining deadlines in SPSs. Especially traditional sensor technology has strict quality requirements for the processing of sensor data. Another question is how loss-tolerant an application is; e.g. load shedding is a relevant aspect in stream processing [31]. Beyond the definition of data stream queries, these quality requirements will be relevant for real scenarios. Partitioning and mapping will have to consider these requirements.
References 1. Abadi, D., Ahmad, Y., Balazinska, M., Cetintemel, U., Cherniack, M., Hwang, J.H., Lindner, W., Maskey, A.S., Rasin, A., Ryvkina, E., Tatbul, N., Xing, Y., Zdonik, S.: The Design of the Borealis Stream Processing Engine. In: 2nd Biennial Conference on Innovative data Systems Research, CIDR (2005)
Integration of Heterogeneous Sensor Nodes by Data Stream Management
171
2. Abadi, D., Carney, D., Cetintemel, U., Cherniack, M., Convey, C., Lee, S., Stonebraker, M., Tatbul, N., Zdonik, S.: Aurora: A New Model and Architecture for Data Stream Management. VLDB Journal 12, 120–139 (2003) 3. Abadi, D.J., Lindner, W., Madden, S., Schuler, J.: An Integration Framework for Sensor Networks and Data Stream Management Systems. In: 13th international conference on very large data bases, VLDB (2004) 4. Abadi, D.J., Madden, S., Lindner, W.: REED: robust, efficient filtering and event detection in sensor networks. In: 31st Conference on Very Large Data Bases, VLDB (2005) 5. Aberer, K., Hauswirth, M., Salehi, A.: Infrastructure for Data Processing in Large-Scale Interconnected Sensor Networks. In: International Conference on Mobile Data Management, MDM (2007) 6. Arasu, A., Babu, S., Widom, J.: The CQL continuous query language: semantic foundations and query execution. VLDB Journal 15, 121–142 (2006) 7. Babcock, B., Babu, S., Datar, M., Motwani, R., Widom, J.: Models and Issues in Data Stream Systems. In: Proceedings of 21st ACM Symposium on Principles of Database Systems, PODS 2002 (2002) 8. Babu, S., Srivastava, U., Widom, J.: Exploiting k-Constraints to Reduce Memory Overhead in Continuous Queries Over Data Streams. ACM Transactions on Database Systems (TODS) 29, 545–580 (2004) 9. Cammert, M., Kr¨amer, J., Seeger, B.: Dynamic metadata management for scalable stream processing systems. In: Proc. of First International Workshop on Scalable Stream Processing Systems (2007) 10. Carney, D., Cetintemel, U., Cherniack, M., Convey, C., Lee, S., Seidman, G., Stonebraker, M., Tatbul, N., Zdonik, S.: Monitoring streams - a new class of data management applications. In: Proceedings of the 28th international conference on Very Large Data Bases, VLDB Endowment, vol. 28, pp. 215–226 (2002) 11. Chandrasekaran, S., Cooper, O., Deshpande, A., Franklin, M.J., Hellerstein, J.M., Hong, W., Krishnamurthy, S., Madden, S., Raman, V., Reiss, F., Shah, M.: TelegraphCQ: Continuous Dataflow Processing for an Uncertain World. In: Proceedings of the 2003 CIDR Conference (2003) 12. Demers, A., Gehrke, J., Hong, M., Riedewald, M., White, W.: Towards Expressive Publish/Subscribe Systems. In: Ioannidis, Y., Scholl, M.H., Schmidt, J.W., Matthes, F., Hatzopoulos, M., B¨ohm, K., Kemper, A., Grust, T., B¨ohm, C. (eds.) EDBT 2006. LNCS, vol. 3896, pp. 627–644. Springer, Heidelberg (2006) 13. Demers, A., Gehrke, J., Panda, B.: Cayuga: A General Purpose Event Monitoring System. In: 3rd Biennial Conference on Innovative Data Systems Research (CIDR 2007), pp. 412–422 (2007) 14. Dressler, F., Kapitza, R., Daum, M., Str¨ube, M., Preikschat, W.S., German, R., MeyerWegener, K.: Query Processing and System-Level Support for Runtime-Adaptive Sensor Networks. In: Kommunikation in Verteilten Systemen, KIVS (2009) 15. Gay, D., Levis, P., von Behren, R., Welsh, M., Brewer, E., Culler, D.: The nesC language: A holistic approach to networked embedded systems. ACM SIGPLAN Notices 38(5), 1– 11 (2003) 16. Gedik, B., Andrade, H., Wu, K.L., Yu, P.S., Doo, M.: SPADE: The System S Declarative Stream Processing Engine. In: ACM SIGMOD Conference, SIGMOD (2008) 17. Gehrke, J., Madden, S.: Query Processing in Sensor Networks. IEEE Pervasive Computing 3(1), 46–55 (2004) 18. Ghanem, T., Hammad, M., Mokbel, M., Aref, W., Elmagarmid, A.: Query Processing using Negative Tuples in Stream Query Engines. Tech. Rep. TR 04-030, Purdue University (2004)
172
M. Daum et al.
19. G¨urgen, L., Honiden, S.: Management of Networked Sensing Devices. In: International Conference on Mobile Data Management, MDM (2009) 20. G¨urgen, L., Roncancio, C., Labb´e, C., Bottaro, A., Olive, V.: SStreaMWare: a Service Oriented Middleware for Heterogeneous Sensor Data Management. In: 5th International Conference on Pervasive Services (ICPS), pp. 121–130 (2008) 21. Kleppe, A., Warmer, J., Bast, W.: MDA Explained: The Model Driven Architecture: Practice and Promise. Addison-Wesley, Reading (2003) 22. Kossmann, D.: The State of the Art in Distributed Query Processing. ACM Computing Surveys (CSUR) 32(4), 422–469 (2004) 23. Kr¨amer, J.: Continuous Queries Over Data Streams-Semantics And Implementation. Ph.D. thesis, Philipps-Universit¨at Marburg (2007) 24. Li, J., Maier, D., Tufte, K., Papadimos, V., Tucker, P.A.: Semantics and Evaluation Techniques for Window Aggregates in Data Streams. In: Proceedings of the 2005 ACM SIGMOD international conference (2005) 25. Lindner, W., Velke, H., Meyer-Wegener, K.: Data Stream Query Optimization Across System Boundaries of Server and Sensor Network. In: 7th International Conference on Mobile Data Management, MDM (2006) 26. Madden, S.R., Franklin, M.J., Hellerstein, J.M., Hong, W.: TinyDB: An Acquisitional Query Processing System for Sensor Networks. ACM Trans. Database Syst. 30, 122– 173 (2005) 27. Pietzuch, P., Ledlie, J., Shneidman, J., Roussopoulos, M., Welsh, M., Seltzer, M.: Network-Aware Operator Placement for Stream-Processing Systems. In: 22nd International Conference on Data Engineering, ICDE 2006 (2006) 28. Schmidt, S.: Quality-of-service-aware data stream processing. Ph.D. thesis, Technische Universit¨at Dresden (2007) 29. Schuler, J.: Query Optimization in Data Stream Architectures. Master’s thesis, University of Erlangen-N¨urnberg (2004) 30. Srivastava, U., Munagala, K., Widom, J.: Operator Placement for In-Network Stream Query Processing. In: 24th ACM SIGMOD-SIGACT-SIGART Symposium on Principles of Database Systems (PODS 2005), pp. 250–258. ACM Press, New York (2005) 31. Tatbul, E.N.: Load Shedding Techniques for Data Stream Management Systems. Ph.D. thesis, Brown University (2007) 32. Velke, H.: Query Optimization between Data Stream Management Systems and Sensor Network Query Systems. Master’s thesis, University of Erlangen-N¨urnberg (2005) 33. Wei, Y., Prasad, V., Son, S.: QoS Management of Real-Time Data Stream Queries in Distributed Environments. In: 10th IEEE International Symposium on Object and Component-Oriented Real-Time Distributed Computing (ISORC), pp. 241–248 (2007) 34. Wiederhold, G.: Mediators in the Architecture of Future Information Systems. IEEE Computer 25(3), 38–49 (1992) 35. Yao, Y., Gehrke, J.: The Cougar Approach to In-Network Query Processing in Sensor Networks. SIGMOD Rec. 31(3), 9–18 (2002) 36. Ying, L., Liu, Z., Towsley, D., Xia, C.: Distributed Operator Placement and Data Caching in Large-Scale Sensor Networks. In: 27th Conference on Computer Communications IEEE (INFOCOM 2008), pp. 977–985 (2008)
Stream-Based Real World Information Integration Framework Hiroyuki Kitagawa, Yousuke Watanabe, Hideyuki Kawashima, and Toshiyuki Amagasa
Abstract. For real world oriented applications to easily use sensor data obtained from multiple wireless sensor networks, a data management infrastructure is mandatory. The infrastructure design should be based on the philosophy of a novel framework beyond the relational data management for two reasons: First is the freshness of data. To keep sensor data fresh, an infrastructure should process data efficiently; this means conventional time consuming transaction processing methodology is inappropriate. Second is the diversity of functions. The primary purpose of sensor data applications is to detect events; unfortunately, relational operators contribute little toward this purpose. This chapter presents a framework that directly supports efficient processing and a variety of advanced functions. Stream processing is the key concept of the framework. Regarding the efficiency requirement, we present a multiple query optimization technique for query processing over data streams; we also present an efficient data archiving technique. To meet the functions requirement, we present several techniques on event detection, which include complex event processing, probabilistic reasoning, and continuous media integration.
1 Introduction Wireless sensor networks (WSNs) have recently gained much attention as a way of monitoring the real world in a variety of contexts. A WSN comprises multiple Hiroyuki Kitagawa · Hideyuki Kawashima · Toshiyuki Amagasa Graduate School of Systems and Information Engineering, and Center for Computational Sciences, University of Tsukuba 1-1-1 Tennodai, Tsukuba, Ibaraki, 305-8573, Japan e-mail: [email protected], [email protected], [email protected] Yousuke Watanabe Global Scientific Information and Computing Center, Tokyo Institute of Technology 2-12-1 Ookayama, Meguro-ku, Tokyo, 152-8550, Japan e-mail: [email protected]
T. Hara et al. (Eds.): WSN Technologies for the Information Explosion Era, SCI 278, pp. 173–204. c Springer-Verlag Berlin Heidelberg 2010 springerlink.com
174
H. Kitagawa et al.
sensor nodes. Each sensor node is made up of a wireless communication module, a computation module and a sensing module. Each sensor node continually generates sensor data. A variety of sensing devices can be attached to a sensor node, which include temperature, humidity, acceleration, light, and so forth. A number of WSN projects are under way. X-Sensor [18] provides a multiple WSN testbed. It deploys more than 100 wireless sensor nodes at multiple Japanese Universities located in diverse geographical regions. X-Sensor not only monitors sensor data continually, but also archives it. The primary purpose of X-Sensor is to provide a fertile research environment for WSN researchers. ExScal [6] deploys over 1000 sensor nodes spread over a 1.3km by 300m remote area in Florida, USA. ExScal provides specialized XSM and XSS sensor nodes. S-Room [23] is an experimental room with many sensors to monitor and detect daily life events. Each daily object in the room is attached to a micro sensor node. By analyzing sensor data obtained from the devices, S-Room provides several interesting services, such as an automatic blog generator of daily objects and real-time physical event notification. GSN [2] is a middleware infrastructure to support the management of sensor data. It adopts the virtual sensor concept so that virtual sensors abstract implementation details from physical sensors. Users can pose SQL-based queries to virtual sensors to retrieve sensor data. Since a WSN generates sensor data continuously through multiple sensor nodes, the amount of generated sensor data usually grows huge. Therefore, for applications to use the data easily, a data management mechanism is needed. Much work has been done on in-network data processing, which processes data in a single WSN. TinyDB [21] provides relational query processing to support the reduction of communication and power consumption on sensor nodes. D-JENGA [17] provides efficient probabilistic reasoning techniques in a WSN. The coverage of these in-network data processing techniques is limited to a single WSN, and each work is dedicated to a specific application. On the other hand, when considering applications that focus on multiple WSNs, research issues change from dedicated mechanisms to a general infrastructure. Sensor data obtained from multiple WSNs should be accessed equivalently; therefore a kind of abstraction should be incorporated into the infrastructure. Obviously, an abstraction process follows a framework, and a framework is decided for a type of application depending on the purpose. Simply stated, the primary purpose of every sensor data application is real-time event detection from massive and noisy data. Therefore, the framework should satisfy two requirements: (R-I) efficient processing requirement to realize real-time processing of massive data, and (R-II) advanced functions requirement to realize event detection from noisy data. This feature is required by sensor data applications, but it is not required by general-purpose relational stream processing applications such as network traffic monitoring [19]. Readers may think that a relational database management system (RDBMS) is appropriate for the framework because it has been widely used for many kinds of real applications over several decades. However, an RDBMS has the following critical problems. (P-I) performance problem: an RDBMS once stores each arrival data object on a persistent device (usually a disk); the data object is then read into
Stream-Based Real World Information Integration Framework
175
memory for query processing. This methodology forces at least one disk access for each data arrival, which degrades performance in both response time and throughput. (R-I) is difficult to satisfy. (P-II) function problem: an RDBMS provides simple relational operators such as selection, projection, join, aggregation, union, etc. The operators effectively process static, clean, symbol-based data. However, sensor data is inherently noisy, and the operators do not directly support event detection. Therefore, (R-II) is difficult to satisfy. Over the RDBMS framework, this chapter presents a stream-based real world information integration framework. The core concept of the framework is stream processing. The stream processing concept involves two features. The first is to process arrival data in memory to achieve high performance. Second is to provide a variety of advanced functions beyond simple relational operators. By modeling sensor data as stream data, the framework fulfills the requirements. This chapter describes the proposed framework in detail based on our work [36, 37, 38, 30, 28, 35, 25]. The framework contains two approaches. (A-I) efficient data processing related to (R-I), and (A-II) real world oriented advanced functions related to (R-II). (A-I) includes a query optimization technique and an efficient data archiving technique. Because integrated sensor data streams from multiple WSNs become massive, the query optimization technique is effective in processing massive sensor data streams. Some applications require that data streams be archived to integrate them with static relational data; the archiving technique is useful for these applications. (A-II) includes real world oriented advanced functions on stream processing. This part presents some of them, including complex event processing, probabilistic reasoning, and continuous media integration. The rest of this chapter is organized as follows: Section 2 describes related work. Sections 3 and 4 describe two approaches related to (A-I). Section 3 describes an event based stream processing model and a multiple query optimization technique. Section 4 describes an efficient technique to provide persistence to data streams using a DBMS. Section 5 then describes advanced functions related to (A-II). It briefly describes advanced techniques on stream processing. Section 6 concludes this chapter and indicates future directions.
2 Related Work 2.1 Basics on Stream Processing The core concept of our framework is stream processing [24], which models sensor data as a type of stream data. Stream data are processed by an infrastructure called the stream processing engine. This subsection describes the engine. A stream processing engine receives an infinite length of tuples, processes them in memory for efficiency, and outputs the result. Processing is based on relational operators such as selection, projection, join, aggregation, union, difference, etc. Some operators, including join and aggregation, cannot deal with an infinite length of tuples. Therefore the window operator [5] converts them to a finite length of
176
H. Kitagawa et al.
tuples. These operators then maintain processing states to satisfy the semantics of the window operator. Stream processing is well researched. The work focuses mainly on continuous query processing in main memory. Tapestry [33] supports continuous queries on append-only relational databases. Streams can be regarded as a kind of append-only relation. Continuous queries in Tapestry are timer-based queries that include temporal conditions in WHERE clauses. OpenCQ [20] is a system that integrates distributed heterogeneous information sources and supports continuous queries. Continuous queries in OpenCQ consist of three parts: query, trigger condition, and terminal condition. When the trigger condition is satisfied, the query is executed continuously until the terminal condition is satisfied. OpenCQ supports both arrival-based and timer-based queries. Sophisticated multiple query optimization is not addressed in tapestry and OpenCQ. NiagaraCQ [12] proposes a multiple query optimization method for its continuous queries. It addresses arrival-based and timer-based queries, and the method includes the incremental append and deletion of queries, where new query execution plans are constructed based on existing plans. However, continuous queries in NiagaraCQ are simple and do not use window operators to specify time intervals of interest as in the window join. Nor do they address the execution time specification. Thanks to its simplicity, the traditional multiple query optimization method to extract common subexpressions can work in the context of NiagaraCQ. CACQ [22] is another system designed to process a large number of continuous queries. Based on eddy [7], it realizes adaptive processing, dynamically reordering operators to cope with changes in arriving data properties and selectivities. CACQ focuses only on arrival-based queries and does not address timer-based queries. CACQ evaluates queries aggressively; it identifies operators as soon as they become executable and evaluates them immediately. In cases where queries are associated with window-based time intervals and/or the execution time specifications, this query evaluation method may generate more query results than are needed. Other work is also related to data stream processing. PSoup [10] is another system employing eddy [7]. By regarding incremental query insertions as a stream of queries, it treats data and queries symmetrically. Aurora [9] and Borealis [1] achieve QoS-based data processing. When data arrives too frequently, some inputs are dropped to shed load based on QoS specifications. PeerCQ [16] is a decentralized system for information monitoring over peer-to-peer networks. It balances load using the P2P framework. ACQP [21] treats queries over wireless sensor data streams. It can manage properties of sensor devices, such as sampling rates. Additionally, it looks closely at query optimization to lower power consumption. Multiple query optimization schemes were originally proposed in the context of relational query processing [29]. Basically, they concentrate on extracting common subexpressions from among multiple queries to share intermediate query results. To reduce the cost of looking for common subexpressions from a multitude of queries, [26] proposes a search algorithm using heuristics.
Stream-Based Real World Information Integration Framework
177
2.2 Advanced Functions on Stream Processing In essence, the purpose of sensor data applications is to recognize the real world. Relational data processing can support that purpose, but effectiveness is limited because of its simplicity. To appropriately achieve the purpose, a variety of functions are needed, such as reasoning, event recognition, and media oriented processing. This subsection briefly describes such functions in conventional work. Specifically, we describe complex event processing, probabilistic reasoning and continuous media stream management. With complex event processing, Cayuga [14] and SASE [3] are close to our work. Cayuga [14] is a prototype event stream processing system, which enables highspeed processing of large sets of queries expressed in the Cayuga algebra. SASE [3] describes events in a formalism related to regular expressions and uses variants of an NFA model. One recent paper of SASE [3] presents a rich declarative event language referred to as SASE+. SASE+ can be used to define a wide variety of Kleene closure patterns to extract a finite yet unbounded number of events with a particular property from the input stream. Such patterns always appear in event streams on wireless sensor network applications. With probabilistic reasoning in a WSN, D-JENGA [17] exists. D-JENGA uses Bayesian networks and processes data streams similar to our research. Although D-JENGA calculates probability values of raw data values obtained from sensor devices, we calculate the probability values for tuple streams that show event occurrences after analyzing tuples obtained from sensor devices. With continuous media data streams, FISQL [39] is a query language for schema conversion. It allows a user to write variables in the FROM clause, which means information sources referred to by a query are not fixed. However, the variable scheme in FISQL is not for dynamic source selection but for schema conversion. Further, FISQL does not focus on stream data.
3 A Multiple Continuous Query Optimization Method Based on Query Execution Pattern Analysis [36, 37] This section covers the first part of (A-I) described in Section 1 and presents an efficient query processing technique that can be applied to integrated massive sensor data streams. First, Section 3.1 defines the continuous query and presents examples. We explain execution time specification to designate when the query should be evaluated and window specification to select tuples within the specified time intervals. Second, Section 3.2 shows an architecture of the data stream processing system. We then present a multiple query optimization method for event-based continuous queries in Section 3.3. Multiple query optimization techniques are usually used to achieve efficiency in query processing. The basic idea is to extract operators that commonly appear in multiple queries and to derive query plans in which intermediate results of these common operators are shared among the queries. However, queries having the same operators may share many intermediate results when they
178
H. Kitagawa et al.
are executed at close instants, but may involve only disjointed data when executed at completely different instants. Therefore, just as with common subexpressions, query execution timing is a key to deciding an efficient query execution plan. Our optimization method takes this point into account.
3.1 Event-Driven Continuous Query 3.1.1
Basic Model
We model data streams as unbound relations. A delivery unit from the data stream is a tuple belonging to a relation. It has an embedded attribute T S to hold its arrival timestamp. Following is the definition of continuous query as it is used in this section. A continuous query CQ(M , E, W ) consists of a set of master information sources M (shortly master), query expression E, and windows specification W . A master information source M (∈ M ) is a data stream. It provides trigger events to evaluate CQ. Users can specify streams as masters even if they are not referred to in expression E. When new tuple m has arrived from master M, query CQ is triggered and E is applied to tuples s arriving from a stream S within the interval [m.T S −WS , m.T S]. WS (∈ W ) is a size of window on the stream S referred in E. Differential results from previous evaluations are then returned, usually generated by new tuples. resultCQ (t) = E(t) −
E(t p )
t p ∈prev exec(t)
where E(t) is the results of evaluating E at time t, and prev exec(t) is a set of previous execution times before t. 3.1.2
Syntax
Following is the syntax of the continuous query used in this section. A query consists of MASTER–SELECT–FROM–WHERE clauses (Figure 1). MASTER clause gives a set of master information sources M . Data arrivals from the master information sources trigger activation of the query. SELECT–FROM–WHERE clauses specify query expression E. They are almost the same as with SQL except for window specifications. We specify windows by “[. . .]” after each source description in a FROM clause. Users can specify different windowsizes for each data stream. If sizes are not given explicitly, they imply infinite windows. Window on stream I is logically the same as applying the selection σI.T S∈[t−windowsize, t] (I) with master arrival time t. Let us now look at an example of continuous query. Figure 2 indicates the requirement “When stock quote data about company ‘A’ comes from data stream Quote, deliver it immediately.” Here, because we must execute the query when Quote data arrives, Quote is specified as a master.
Stream-Based Real World Information Integration Framework MASTER master source,. . . SELECT attr 1 ,. . . FROM source 1 { [windowsize 1] } , . . . WHERE conditions Fig. 1 Syntax of continuous query
179
MASTER Quote SELECT * FROM Quote WHERE Quote.name = ’A’ Fig. 2 Example continuous query
MASTER Quote SELECT * FROM Quote[6hour], News[6hour] WHERE Quote.name=News.name
MASTER Clock 0 SELECT * FROM Quote[6hour], News[6hour] WHERE Quote.name=News.name
Fig. 3 Query including window join
Fig. 4 Query referencing clock stream
Figure 3 is another example including the join operator over two streams. This requirement says that “When new data from Quote arrives, deliver tuples integrating it with News data that contains news of the same company and has arrived within the last six hours.” In this example, the size of the window equals 6 hours. 3.1.3
Clock Stream
In our framework, we focus on not only events caused by data arrival, but also timer events. We also address timer-based queries by introducing the notion of a clock stream. A clock stream is a data stream provided by the query processing system, and it periodically delivers alarms. Specifying clock streams as masters, we can express timer-based queries that are executed at the same intervals. Figure 4 shows an example using a clock stream, “Every midnight, deliver result integrating Quote and News data within last six hours,” where Clock 0 expresses a clock stream that delivers an alarm every night at midnight.
3.2 Data Stream Processing System for Event-Based Continuous Queries 3.2.1
System Architecture
Figure 5 shows the architecture of the system assumed in this section. The architecture follows the mediator/wrapper model. A wrapper is responsible for a corresponding information source. It detects arrivals of new data units and transforms them into the internal data model. As mentioned in Section 3.1.1, we model data streams as unbound relations. A wrapper notifies Mediator of arrival events. When Mediator receives an event, it evaluates continuous queries corresponding to the event. Queries are evaluated according to query plans. Optimizer applies the multiple query optimization method described in Section 3.3 to a set of queries, and derives query plans. Mediator reports execution status to Optimizer. Optimizer records
180
H. Kitagawa et al.
Continuous queries
Users
Query Analyzer, Optimizer Execution status
Event-driven query processing
Query plans Query result
Mediator Request query Retrieve results Wrapper
Wrapper
Execute SQL Retrieve results RDBMS
Clock Streams
Notify events Wrapper
New data unit
Data Stream
New data unit
Data Stream
Fig. 5 Architecture of data stream processing system
execution history for each query. Execution history is important information for multiple query optimization. 3.2.2
Execution Mechanism in Mediator
Event-driven Operator Evaluation Figure 6 shows the internal execution model in Mediator. In this system, a continuous query given by the user is analyzed and transformed into a query plan by Query Analyzer. A query plan is represented as a directed acyclic graph consisting of operators. Each operator has parameters needed by its evaluation such as master information sources, windows, predicates and so on. An operator has a queue for each input stream, and it holds input tuples as long as they are included within the current window of the stream. When Mediator receives an event notification, Scheduler searches query plans corresponding to the event. Operators are activated from the bottom of the query plan. An activated operator generates output tuples, and it appends them to the input queues of the upper operators. When tuples reach the top of the query plan, they are regarded as query results and delivered to users. Queue Management A queue is associated with an operator and holds tuples that have arrived from information sources or were generated by other operators. A queue consists of two data areas: a delta part and a window part. A delta part holds new tuples that are not yet processed, and a window part holds tuples that are processed once and still included in the query’s window range. When an operator is activated, it extracts tuples in the
Stream-Based Real World Information Integration Framework
181
Fig. 6 Execution mechanism in Mediator
Fig. 7 Window join
delta part of its input queues, and appends results to the queues of upper operators. Tuples in the delta part are then moved to the window part. We need to maintain whole tuples included within the query’s window to produce the results of join. If no masters arrive for a long time, tuples in the queue drop out from the window range. Tuples becoming obsolete are removed from both the delta part and window part before the operator is evaluated. Computation of Window Join Window join is a join operation for multiple streams with window specifications. It is a popular operation in stream data processing. The system we assume here also
Fig. 8 Example 1 Stream_1 MASTER Clock 13 SELECT * FROM Stream 1[8hour], Stream 2[8hour], Stream 4[8hour]
Stream_2
Stream_3
Fig. 9 Example 2 Stream_4 MASTER Clock 0 SELECT * FROM Stream 1[8hour], Stream 2[8hour], Stream 5[8hour]
Fig. 10 Example 3
Stream_5 0
12
13
0
Fig. 11 Reference ranges of queries
supports window join, and tuples are held by a window while they are included in a window range. Obsolete tuples in a window are removed before the join operator is activated. We can get differential results from previous evaluations by computing the following expression. window join(R, S) = ΔR 1 ΔS ∪ ΔR 1 WS ∪WR 1 ΔS Where ΔR , ΔS , WR and WS express the delta parts and the window parts of queues corresponding to R and S (Figure 7). We assume that generated tuples inherit all timestamps of original tuples.
3.3 Multiple Query Optimization Method Based on Query Execution Pattern Analysis 3.3.1
Basic Idea
We now explain our multiple query optimization method for event-based continuous queries. Consider a scenario optimizing three queries: Figures 8, 9, and 10. For example, Figure 8 means “Each day at noon, deliver results integrating tuples of Stream 1, Stream 2 and Stream 3 having arrived within the last 8 hours.” According to their master clauses, Example 1 is executed each day at noon, Example 2 is each day at 13:00, and Example 3 is every night at midnight. These queries contain the common operation integrating Stream 1 and Stream 2, and we expect sharing results of these operations to reduce processing cost.
Stream-Based Real World Information Integration Framework
183
Ranges referred to by window join of each query are depicted in Figure 11. The execution time of Example 1 seems close to the execution time of Example 2, and the range referred to by Example 1 overlaps the range of Example 2. Example 3, however, is separated from other queries. Therefore, the intermediate results of window join, Stream 1 1 Stream 2, generated by Example 1 contain almost the same results as the one generated by Example 2, but not by Example 3. The effectiveness of multiple query optimization is affected by whether or not generated results can be reused. In this case, we do not gain any benefits if we force Example 3 to share results with other queries. In fact, performance may degrade because sharing results in extra scans to get unnecessary intermediate results. In this scenario, we should cause Example 1 and Example 2 to share results, and execute Example 3 separately. To optimize multiple continuous queries with different execution times and window intervals, we form clusters containing queries that have close execution times and large overlaps of window intervals. Our method consists of the following two steps. 1. It analyzes data arrival logs and extracts query execution patterns. It then forms clusters of continuous queries such that queries in the same cluster are likely to be executed at close instants. Section 3.3.2 mentions clustering of queries based on similarities of execution patterns. 2. It extracts common subexpressions from among queries in each cluster and decides the execution plan. Section 3.3.3 explains the generation of query plans that share the common operators. 3.3.2
Clustering Continuous Queries
We define the similarity of continuous queries based on the overlap of ranges referred to by queries. If arrival patterns of all master sources were as predictable as clock streams, we could determine whether or not overlaps of ranges exist using the method explained in Section 3.3.1. In most external data streams, however, we cannot anticipate query execution patterns exactly; thus, it is difficult to check overlaps of queries. In our scheme, Optimizer analyzes execution patterns by simulating query invocations using an arrival data log of underlying streams. First, collecting the data log arriving within interval T, we prepare sets of arrival times on each stream. We denote the set of arrival times of source Ii (1 ≤ i ≤ n) as Si . Next, for each query Q j (1 ≤ j ≤ m), we compute a subset of Si (named USESiQ j ), whose elements are referred to by Q j . USESiQ j is obtained as the union of arrival times of data units referred to by each execution of Q j as follows: USESiQ j =
s∈MS j
{t|t ∈ Si ∧ t ∈ [s − windowsizeSi Q j , s]}
184
H. Kitagawa et al.
where MS j , the set of execution times of Q j , equals the union of arrival times of data units from all masters for Q j . And windowsizeSi Q j is the size of window on Si specified in Q j . We then compute similarity(Qa , Qb ), based on the USESiQa and USESiQb . We define the similarity as follows: similarity(Qa , Qb ) =
|USESiQa ∩USESiQb | Ii ∈(Re f (Qa )∩Re f (Qb )) |USESi Qa ∪USESi Qb | min
The right side of the equation tells how much overlap exists between USESiQa and USESiQb , where Re f (Q) expresses the streams specified in the FROM clause. If Re f (Qa ) ∩ Re f (Qb ) equals an empty set, we regard their similarity as 0. The similarity between queries having the same masters and windows becomes 1. If we have m queries, we finally get m × m symmetric matrix A (called similarity matrix). For example, we have a sample data set shown in Figure 12 for queries Example 1, Example 2 and Example 3. The similarity between Example 1 and Example 2 is computed as follows: |USEStream1 ,Example 1 ∩USEStream1 ,Example 2 | |{t3,t9}| = = 0.67 |USEStream1 ,Example 1 ∪USEStream1 ,Example 2 | |{t1,t3,t9}| |USEStream2 ,Example 1 ∩USEStream2 ,Example 2 | |{t2,t10}| = = 1.0 |USEStream2 ,Example 1 ∪USEStream2 ,Example 2 | |{t2,t10}| similarity(Qa , Qb ) = min({0.67, 1.0}) = 0.67 We get a similarity matrix like that in Figure 13. The element at (k, l) in the matrix corresponds to the value of similarity(Example k, Example l). Optimizer computes similarities for all pairs of given queries. It then forms clusters having queries with high similarities. To generate clusters, we use a normal algorithm of hierarchical clustering [27], where queries included in the same cluster must have high similarity greater than or equal to threshold θ . By changing the parameter θ , we can manage sizes of generated clusters. θ is the criterion for deciding to share operators based on how high the rate of overlaps is guaranteed. When θ is set to 1, operators will be shared only with queries referring to exactly the same range. On the other hand, setting θ to 0 means that we have only one cluster including all the queries. This is the case of multiple query optimization in which execution timing is not taken into account. Figure 14 shows the result of generating clusters from the similarity matrix in Figure 13. As mentioned above, changes in θ cause changes in the number of generated clusters. It is unusual for the initial query set to be fixed while the system is running. In the real world, however, ad hoc insertion and deletion of queries may occur. If we want to maintain the best similarity, we need to reconstruct clusters and to retry finding common subexpressions at every insertion or deletion. Doing so, however, is too expensive. In our scheme, we recompute only when the number of changes goes above a predetermined constant value. At insertion, a new query is appended to the
Stream-Based Real World Information Integration Framework 0hour Stream_1
12hour 13hour t1
Stream_2 Stream_3 Stream_4
t3 t2
0hour t7
12hour 13hour t9
t6
0hour t14
t10
t4
t12
t5
t11
Stream_5
185
t13
t8
t15
Example_1 Example_2 Example_3
USE Stream_1,Example_1 = { t1, t3, t9 } USE Stream_2,Example_1 = { t2, t10 } USE Stream_3,Example_1 = { t4, t12 } USE Stream_1,Example_2 = { t3, t9 } USE Stream_4,Example_2 = { t5, t11 }
USE Stream_2,Example_2 = { t2, t10 }
USE Stream_1,Example_3 = { t7, t14 } USE Stream_5,Example_3 = { t8, t15 }
USE Stream_2,Example_3 = { t6, t13 }
Fig. 12 Log about arrival times(sampling range=2days)
Example_1
⎡
⎤ 1.00 0.67 0.00 ⎣ 0.67 1.00 0.00 ⎦ 0.00 0.00 1.00 Fig. 13 Similarity matrix in the example
Example_2
0.67
0.00
0.00
theta=1.0 theta=0.6
Example_3
theta=0.0
Fig. 14 Cluster generation in the example
cluster having the query with highest similarity. At deletion, unneeded queries are removed from the cluster, but no reconstruction occurs. In our scheme, we assume streams have fixed or stable data arrival patterns. We may, however, see changes in stream properties over a long time. When we have identified significant changes in query execution patterns, we need to reconstruct clusters using recent logs. 3.3.3
Finding Common Subexpressions and Sharing
As described in Section 3.3.2, each cluster consists of queries having high similarities. This guarantees that reusable intermediate results can be generated by common
186
H. Kitagawa et al.
operators. Therefore, the remaining steps are to find common subexpressions and derive optimal query plans for each cluster. We of course use a traditional multiple query optimization method [26]. First, it finds common operators to be merged from each cluster. Common operators mean that they have the same parameters: predicates in selection or join operators, attribute list in the projection operator. Operators having the same parameters with different masters or windows sizes are regarded as common operators. When common operators are found, their query plans are merged. Merged operators have the union set of masters and the maximum size of window for each operator. Finally, the optimal plan is derived for each query to minimize the total processing cost. For example, we consider a scenario for the cluster including Example 1 (Figure 8) and Example 2 (Figure 9). Both queries contain the common operation Stream 1 1 Stream 2. In the first step, we find possible query plans for each query. Example 1 has several paths to obtain a complete result, such as (Stream 1 1 Stream 2) 1 Stream 3 and Stream 1 1 (Stream 2 1 Stream 3). Figure 15 shows all possible plans for Example 1 and Example 2. Figure 15 uses AND–OR DAG representation [26] to express query plans. AND–OR DAG is a graph notation consisting of AND nodes and OR nodes. An AND node corresponds to operators, and an OR node represents results of an AND node. In Figure 15, circles indicate AND nodes, and boxes are OR nodes. For simplicity, we do not consider physical features such as join algorithms here. In the next step, we merge OR nodes with multiple occurrences into one node. Figure 16 expresses the merged result and the node Stream 1 1 Stream 2 is merged. Information on masters is also merged on AND nodes that are descendant of the merged OR node. Finally, we choose the optimal query plan from merged DAGs. As mentioned earlier, estimating the exact cost on data streams is very difficult; therefore we choose a query plan that maximizes the number of shared operators. In Figure 16, the plan (Stream 1 1 Stream 2) 1 Stream 3 maximizes the number of shared operators. Optimal query plans are shown as bold lines in Figure 16. Based on the derived query plan, actual processing is performed (Figure 17). In Figure 17, when an alarm of Clock 12 arrives at noon, Stream 1 1 Stream 2 is executed, then the results are stored into two output queues. Although the window join Stream 1 1 Stream 2 is executed on the arrival of both Clock 12 and Clock 13, it does not generate redundant results, because tuples in delta parts are consumed at every execution and moved to window parts while they are covered by windows. 3.3.4
Experiments
This section shows the results of our experiments. Environments and data sets Figure 19 shows the environment for experiments. We use synthetic streams in our experiments. Two sources are normal data streams that deliver data at 5-second
Stream-Based Real World Information Integration Framework Stream_1,Stream_2,Stream_3
clock_12
clock_12 Stream_1, Stream_2
Stream_2, Stream_3
clock_12
clock_12 Stream_1
Stream_1,Stream_2,Stream_4
clock_13
clock_12 clock_13 Stream_1, Stream_3
Stream_1, Stream_2
Stream_3
clock_13
Stream_2, Stream_4
Stream_1, Stream_4
clock_13
clock_12 clock_13
Stream_2
187
Stream_1
Example_1
clock_13
Stream_2
Stream_4
Example_2
Fig. 15 DAG showing possible execution plans Example_1 Stream_1,Stream_2,Stream_3
Fig. 16 DAG after merging all common sub-expressions
Example_1
master: clock_12
master: clock_13
Example_2
MASTER Clock X SELECT *
Stream_3
Stream_4 master: clock_12, clock_13
Stream_1
Stream_2
Fig. 17 Execution of optimized query plan
FROM Stream 1 [W], Stream 2 [W] WHERE
Stream 1.attr
=
Stream 2.attr
Fig. 18 Query template
intervals. There are ten clock streams that deliver alarms at 5-minute intervals, but their arrival times differ. We use queries generated using the template in Figure 18. As a master for each query, we randomly specify a clock stream from the ten clock
188
H. Kitagawa et al. 㫄㫌㫃㫋㫀㫇㫃㪼㩷㫈㫌㪼㫉㫐㩷㫆㫇㫋㫀㫄㫀㫑㪸㫋㫀㫆㫅 㫊㫀㫅㪾㫃㪼㩷㪼㫏㪼㪺㫌㫋㫀㫆㫅
㫋㫀㫄㪼㩿㫊㪼㪺㪀 㪏㪇 㪎㪇
CPU UltraSPARC-II 296MHz × 2 RAM 1GB OS Solaris 2.6 Prog. Lang. Java (J2SE 1.4.0, J2EE 1.3.1) RDBMS PostgreSQL 7.3.1
Fig. 19 Experiment environment
㪍㪇 㪌㪇 㪋㪇 㪊㪇 㪉㪇 㪈㪇 㪇 㪇
㪉㪇㪇
㪋㪇㪇 㪍㪇㪇 㫅㫌㫄㩷㫆㪽㩷㫈㫌㪼㫉㫀㪼㫊
㪏㪇㪇
㪈㪇㪇㪇
Fig. 20 Optimization vs. no optimization
streams. We also randomly select the windowsize from a range between 30 and 300 seconds. Optimization vs. No Optimization First, we compare execution with multiple query optimization and execution of all queries separately. The parameter of clustering θ is fixed at 0.5. We measure total processing times after having received stream data for 15 minutes. The results are shown in Figure 20. The horizontal line expresses the number of queries, and the vertical line expresses total processing times. According to Figure 20, execution with multiple query optimization is considerably faster than separate single query executions. Varying Clustering Parameters In this experiment, we investigated the effect of parameter θ . Although the main objective here is to minimize total processing time, we are also interested in reducing response time. Therefore, we examine both total processing time and response time. Queries with random windowsizes We prepare 200 and 300 queries that have random windowsizes. As Figure 21 shows, the number of generated clusters is followed by changing θ . The processing times in each case are shown in Figure 23. The case in which θ is set to 0.5 results in the best performance with 200 queries. It is 18 seconds faster than the case in which θ is set to 0, which indicates multiple query optimization without taking into account execution timing. With 300 queries, 0.6 is the best and 33 seconds faster than θ = 0. On the other hand, when θ is set too high, performance degrades. The degradation occurs because an attempt to maintain clusters with high similarities causes a decrease in the number of queries contained in one cluster; therefore, it affects the efficiency of multiple query optimization. We next focus on the relationship between parameter θ and average response time. Here, the response time is the interval between the time of master data arrival
Stream-Based Real World Information Integration Framework
㫅㫌㫄㩷㫆㪽 㪺㫃㫌㫊㫋㪼㫉㫊 㪉㪌㪇
㪉㪇㪇㩷㫈㫌㪼㫉㫀㪼㫊
189
㫋㫀㫄㪼㩿㫄㫊㪼㪺㪀
㪊㪇㪇㩷㫈㫌㪼㫉㫀㪼㫊
㪉㪇㪇㩷㫈㫌㪼㫉㫀㪼㫊
㪊㪇㪇㩷㫈㫌㪼㫉㫀㪼㫊
㪇㪅㪋
㪇㪅㪏
㪉㪌㪇㪇 㪉㪇㪇㪇
㪉㪇㪇 㪈㪌㪇㪇
㪈㪌㪇 㪈㪇㪇
㪈㪇㪇㪇
㪌㪇
㪌㪇㪇
㪇
㪇
㪇
㪇㪅㪈
㪇㪅㪉
㪇㪅㪊
㪇㪅㪋
㪇㪅㪌
㪇㪅㪍
㫇㪸㫉㪸㫄㪼㫋㪼㫉㩷㱔
㪇㪅㪎
㪇㪅㪏
㪇㪅㪐
㪈
㪇
Fig. 21 Number of clusters (random windowsizes)
㪉㪇㪇㩷㫈㫌㪼㫉㫀㪼㫊
㫋㫀㫄㪼㩿㫊㪼㪺㪀
㪇㪅㪉
㪇㪅㪍
㫇㪸㫉㪸㫄㪼㫋㪼㫉㩷㱔
㪈
Fig. 22 Average response time (random windowsizes)
㪊㪇㪇㩷㫈㫌㪼㫉㫀㪼㫊
㫅㫌㫄㩷㫆㪽 㪺㫃㫌㫊㫋㪼㫉㫊
㪈㪇㪇 㪐㪇 㪏㪇 㪎㪇 㪍㪇 㪌㪇 㪋㪇 㪊㪇 㪉㪇 㪈㪇 㪇
㪉㪇㪇㩷㫈㫌㪼㫉㫀㪼㫊
㪊㪇㪇㩷㫈㫌㪼㫉㫀㪼㫊
㪉㪌㪇 㪉㪇㪇 㪈㪌㪇 㪈㪇㪇
㪌㪇 㪇
㪇
㪇㪅㪉
㪇㪅㪋
㫇㪸㫉㪸㫄㪼㫋㪼㫉㩷㱔
㪇㪅㪍
㪇㪅㪏
㪇
㪈
㪇㪅㪈
㪇㪅㪉
㪇㪅㪊
㪇㪅㪋
㪇㪅㪌
㫇㪸㫉㪸㫄㪼㫋㪼㫉㩷㱔
㪇㪅㪍
㪇㪅㪎
㪇㪅㪏
㪇㪅㪐
㪈
Fig. 23 Total processing time (random win- Fig. 24 Number of clusters (windowsizes dowsizes) following Gaussian)
㫋㫀㫄㪼㩷㩿㫄㫊㪼㪺㪀
㪉㪇㪇㩷㫈㫌㪼㫉㫀㪼㫊
㪊㪇㪇㩷㫈㫌㪼㫉㫀㪼㫊
㪉㪇㪇㪇 㪈㪏㪇㪇 㪈㪍㪇㪇 㪈㪋㪇㪇 㪈㪉㪇㪇
㫋㫀㫄㪼㩿㫊㪼㪺㪀
㪉㪇㪇㩷㫈㫌㪼㫉㫀㪼㫊
㪊㪇㪇㩷㫈㫌㪼㫉㫀㪼㫊
㪐㪇 㪏㪇 㪎㪇 㪍㪇 㪌㪇 㪋㪇
㪈㪇㪇㪇 㪏㪇㪇 㪍㪇㪇 㪋㪇㪇 㪉㪇㪇 㪇
㪊㪇 㪉㪇 㪈㪇 㪇
㪇㪅㪉
㪇㪅㪋
㪇㪅㪍
㫇㪸㫉㪸㫄㪼㫋㪼㫉㩷㱔
㪇㪅㪏
Fig. 25 Average response time (windowsizes following Gaussian)
㪈
㪇
㪇㪅㪉
㪇㪅㪋
㪇㪅㪍
㪇㪅㪏
㪈
㫇㪸㫉㪸㫄㪼㫋㪼㫉㩷㱔
Fig. 26 Total processing time (windowsizes following Gaussian)
and the time a result is generated for a query. We then investigate average response times over the set of queries (Figure 22). In Figure 22, average response time in the case of θ = 0 is especially long. This occurs because operators are shared with queries having significantly different sized windows. Although processing time with a large window is longer than with a small window, shared operators should be processed with adjustments to the largest window. When this is done, queries with small windows are forced to wait, and average response time is increased.
190
H. Kitagawa et al.
Queries with windows following Gaussian distributions In the previous experiment, we assumed that distribution of windowsizes is uniformly random, but in the real world, it may follow unbalanced distributions. To cope with this imbalance, we attempt to make the sizes of windows follow mixed Gaussian distribution, which has three peaks at 90, 150 and 210 seconds. We generate and execute 200 and 300 queries whose windowsizes follow the distribution. The results are shown in Figure 24, Figure 26, and Figure 25. With 200 queries, the total processing time seems good at 0.4, 11 seconds faster than the case in which θ = 0. 0.5 is the best, and 22 seconds faster than θ = 0 with 300 queries. The response time with θ = 0 is the worst.
3.4 Summary As described in Section 3.3.4, our approach has a great advantage over single execution. In these experiments, we show the relationship between θ and total processing times or response times. With appropriate θ , processing is faster than with optimization that does not take into account execution patterns (θ = 0). How to decide the appropriate value of θ is an important problem. The optimal value of θ depends on many factors, such as types of operators, lengths of windows, arrival rates, sizes of data items, and so on. When processing stationary data streams, we try simulations with several θ values, then we determine the optimal value. This is one of most dependable approaches. From the above discussion, we conclude that our multiple query optimization technique contributes to solving the (P-I) performance problem from the standpoint of query processing.
4 Integrating a Stream Processing Engine and Databases for Persistent Streaming Data Management [38] This section covers the second part of (A-I) described in Section 1 and it presents an integration of stream with relational data. Some sensor data applications need a system that supports not only continuous query processing but also sophisticated disk management. From the standpoint of disk management, stream processing engines are not better than traditional DBMS. Therefore, combining a stream processing engine and DBMS is a reasonable approach to implement a data stream management system. This section presents Harmonica, a data stream management system based on an architecture combining a stream processing engine [32] and databases. It provides a SQL-like query language that supports continuous queries, one-shot queries, persistence requirements and integration requirements for streams and databases. Our language is designed as an extension to the previous continuous query languages and SQL. Harmonica correctly estimates an optimal tuple writing ratio so that the archiving process can be executed efficiently.
Stream-Based Real World Information Integration Framework
191
4.1 Example Scenario We assume an application system for an industrial plant managing gas turbines (Figure 27). Behavior of individual turbines is monitored by many types of sensors, temperature (ttxd11), revolutions per minute (rpm), and amount of fuel (fsr). Each sensor periodically produces data each second. In this example, we need both real-time monitoring over stream data (e.g., online failure detection) and archiving stream data into persistent storage (e.g., trouble analysis).
4.2 System Architecture Here is the architecture of Harmonica (Figure 28). The system consists of the following modules: • Query parser: This module analyzes queries given by users and constructs query plans. Our query language is explained in Subsection 4.3. • Feasibility validator: This module decides whether or not a given query plan is feasible. The validation scheme is presented in Subsection 4.4. A query that has passed the validation process is sent to the stream processing engine. It collects the information needed to validate plans from the stream processing engine and stream archiver. • Stream processing engine: StreamSpinner [32] is a stream processing engine we implemented earlier. A characteristic of our engine is event-driven operator evaluation. When the engine receives an event-notification from a wrapper, it evaluates operators associated with the event. If a query plan contains operations to access DBMS, the stream processing engine makes requests of the stream archiver. The engine delivers the query result to the user.
Fig. 27 Industrial plant monitoring
192
H. Kitagawa et al.
Fig. 28 System architecture
• Wrapper: A wrapper takes charge of a stream. When it receives data from the stream, it transforms the data into a tuple. It notifies the stream processing engine of an event to trigger operator evaluation. • Stream archiver: The stream archiver module manages databases. It maintains schema information on all databases and transfers access requests to corresponding databases. • Database connector: A database connector holds a database connection. Connectors to MySQL, PostgreSQL, and SQL Server are available.
4.3 Harmonica Query Language 4.3.1
Syntax
The following introduces the query language used in the system. We employ a relational model as the common data model in the system. Data streams are modeled as unbounded relations. Each delivery unit from a stream is regarded as a tuple included in the relation. A relation consists of multiple normal attributes and one special attribute to hold an arrival timestamp for each tuple. The syntax of our query language is presented in Figure 29. A MASTER clause specifies events to trigger query processing. When new data arrives from the streams written in the MASTER clause, the query is evaluated by the stream processing engine. If a MASTER clause is omitted, the query is regarded as a traditional oneshot query. SELECT–FROM–WHERE–GROUP BY clauses are the same as with SQL except for the time-based window specification in the FROM clause. A window specification is used to specify sliding-window. It consists of an interval and an origin of the moving window. For example, “Turbine[1min, now]” in a FROM clause indicates tuples delivered from the Turbine stream within the most recent minute. A
Stream-Based Real World Information Integration Framework
window slides as time progresses, and tuples within the range are evaluated at each event occurrence. Although INSERT INTO, CREATE TABLE and DROP TABLE are omitted in Figure 29, users can give requirements using these clauses. 4.3.2
Examples
Figure 30 is continuous query filtering stream data. It means “Deliver data when its temperature exceeds 100 degrees.” The MASTER clause indicates that the query is triggered by an arrival of Turbine data. Width of the temporal window is 1 millisecond, and the origin is the arrival time of the Turbine data. ttxd11 is an attribute including a value from a temperature sensor. Figure 31 is a persistence requirement. It says “When new data arrives from the Turbine, store it into the Turbine log table if the temperature value exceeds 100.” Turbine log is the name of a table in DBMS. rpm is an attribute with a value of revolutions per minute. When a MASTER clause is not given in a query, the requirement is for a oneshot query. Figure 32 is an example of a one-shot query. “Retrieve an average of temperature data that arrived on June 16 from Turbine log table.” Turbine log is a table stored in DBMS. “1day” gives the interval of the query’s window, and “June 16, 2007 00:00 000” gives the origin of the window. “avg” is an aggregate function to calculate the average. Following is an example of a query integrating stream and DBMS. Figure 33 means “Report similarity between recent temperature values from Turbine stream and historical temperature values in turbine log.” Clock 1minute is a timer stream provided by the system. “array” is a function to convert a set of attribute values into an array. “sim” is a function to compute similarity between two arrays.
194
H. Kitagawa et al. MASTER Clock 1minute SELECT sim(S1.V, S2.V) FROM ( SELECT array(ttxd11) AS V FROM Turbine [1min, now] ) AS S1, ( SELECT array(ttxd11) AS V FROM Turbine log [1min, “June 16, 2007 00:00 000”] ) AS S2
Fig. 33 Integration of stream data and historical data
Fig. 34 An example of query plan
4.3.3
Query Plan
Query parser constructs a query plan from a given query. A query plan is a tree of operators. Figure 34 shows the query plan constructed from the query in Figure 33. When a data arrival from Clock 1minute is notified to the stream processing engine, the query plan is designated for evaluation. The operators included in the plan are evaluated from bottom to top of the tree. The plan contains seven operators, including projection, cartesian-product, grouping and function evaluation.
4.4 Feasibility Validation This subsection describes a feasibility validation scheme for a query plan. Our scheme is an extension of Ayad’s method [8]. However, their original method does not consider persistence requirements or integration queries for streams and databases. Here, we assume that the average input rate for each stream is almost stable. Although bursty data arrivals may happen in data streams, our objective is to detect (in advance) queries that cause stationary overflow. We think that a system with sufficient queues can treat bursty arrivals.
Stream-Based Real World Information Integration Framework
195
Fig. 35 Validation example
4.4.1
Definition
Except for one-shot queries, a query is triggered by events specified by the MASTER clause in the query. To keep pace with data arrivals from streams, the system must finish evaluations of all operators corresponding to an event before the next event occurs. If evaluation tasks are continuously stacked in the system, the system will eventually crash. We use “unit time” to specify the average time interval between one event and the next. As described in Subsection 4.2, our system consists of a stream processing engine and DBMS. We can divide query plan q into two parts: stream processing part SPq , which is mainly processed by the stream processing engine, and writing part W Pq , which is processed mainly by DBMS. SPq is a subtree of the query plan constructed by removing the store operator from q. W Pq is a store operator when q is a persistence requirement. Otherwise, W Pq is empty. Before defining feasibility of a whole query plan, we define feasibility of SPq and W Pq . Definition 1. Let ok (1 ≤ k ≤ n) be an operator included in SPq and τk be the average time needed by operator ok to evaluate input tuples per unit time. A stream processing part SPq is feasible if the following condition is satisfied. n
∑ τk < unit time
(1)
k=1
Formula 1 means that the sum of evaluation time of all operators should not exceed unit time. Definition 2. Let λSP be an output rate produced by the stream processing engine with a query plan q, and let λDB be the maximum writing rate to DBMS. A writing part W Pq is feasible if the following condition is satisfied.
λSP < λDB
(2)
196
H. Kitagawa et al.
Formula 2 means the number of output tuples produced by the stream processing engine should not exceed the number of maximum writing rate to DBMS. Definition 3. Query plan q is feasible if and only if both the stream processing part SPq and the writing part W Pq are feasible. 4.4.2
Validation Algorithm
According to the above definitions, our algorithm consists of three steps. 1. A query plan q is constructed from a query, and it is separated into the stream processing part SPq and the writing part W Pq . 2. We obtain τk , which is the average time needed by each operator ok in SPq . How to estimate τk is explained in Subsection 4.4.3. After that, Formula 1 is validated. If SPq is feasible and W Pq is not empty, we go to the next step. 3. Formula 2 is validated. An output rate of stream processing engine λSP is obtained by the previous step. The system estimates λDB using the method described in Subsection 4.4.4. 4.4.3
Estimating Operator Costs
Validator traverses a query plan from bottom to top. It estimates the cost of each operator based on Ayad’s cost model [8]. For each operator ok , we derive an output rate λk , a time to process tuples arriving in unit time τk , and the number of output tuples included in window Wk . Wk is required to compute costs for the window-join and cartesian-product. The stream processing engine provides information needed by this estimation: input rates of streams λi , a time to process one tuple in an operator Co . We first explain how to estimate costs of unary operators such as selection, projection, grouping, and function evaluation. We then present an estimation for binary window-join and cartesian-product. Unary Operators Let λi be an input rate to the operator ok , Wi be the number of input tuples included in the window and fk be ok ’s selectivity (=#outputs/#inputs). Output rate λk is derived by the input rate: λk = fk λi . And, τk , the time to process tuples arriving in unit time, is also obtained from the input rate λi : τk = Ck λi . Finally, we can obtain Wk , the number of output tuples included in the window, from the number of input tuples in the window: Wk = fkWi . In selecting and grouping operators, getting accurate selectivity values is sometimes difficult. In such cases, we use fk = 1 pessimistically. For projection and function evaluation operators, we always use fk = 1. Binary Operators Suppose a binary-join operator is connected to two input streams L and R. Let λL and λR be input rates of L and R respectively, and let WL , WR be numbers of tuples included in the windows of L and R respectively. A new tuple from L is compared
Stream-Based Real World Information Integration Framework
197
to tuples held in the R’s window; therefore the number of output tuples produced by input tuples from L equals fk λLWR . In the same way, the number of output tuples produced by input tuples from R is fk λRWL . Therefore, the output rate of operator ok is obtained by the following formula.
λk = fk (λLWR + λRWL ) The time to process tuples arriving in unit time is calculated as follows:
τk = τL + τR = Ck (λL + λR) Finally, the number of output tuples included in the window becomes as follows: Wk = fkWLWR In the estimation for cartesian-product operator, we use fk = 1. 4.4.4
Estimating the Maximum Writing Rate to DBMS
To validate Formula 2, we need λSP , which is the output rate produced by stream processing, and λDB , which is the maximum writing rate to DBMS. λSP is obtained by the estimation in Subsection 4.4.3, because λSP equals the output rate of the top operator in SPq. The maximum writing rate to DBMS depends on environment, such as machine power and implementations of DBMS. In addition, the rate may change according to the data size of one tuple. Generally, the writing rate for a large tuple is lower than that for a small tuple. It is impractical, however, to measure actual writing rates for all sizes of data. Our approach measures rates only for different N sample sizes. With N samples, we can estimate a writing rate for other data sizes by applying linear approximation. λDB for query plan q is estimated by the following formula.
λDB rate estimate(tuple size(Sq ))
(3)
Sq is a schema corresponding to the output produced by query plan q. tuple size is a function to compute average data size based on the schema, and rate estimate is a function to estimate writing rate by applying linear approximation. 4.4.5
Validation Example
The following illustrates the validation process for the query in Figure 31. At the first step, the system constructs a query plan (Figure 35). The plan consists of selection operator o1, projection operator o2, and store operators o3. Next, the system validates whether or not SP is feasible. In this example, we suppose λTurbine and λDB are 10 and 5 (tuples/second). We also suppose f1 and f2 equal 0.3 and 1.0, C1 and C2 equal 0.02 and 0.01. Based on the estimation method in Subsection 4.4.3, we can obtain λ1 = f1 λTurbine = 0.3 ∗ 10 = 3 (tuples/second).
198
H. Kitagawa et al. input rate decision result : : : 33 tuples/s feasible succeed (hit) 34 tuples/s feasible overflow (miss) 35 tuples/s not feasible overflow (hit) : : :
Fig. 36 Experiment results for the query in Fig. 31 qid operators estimated border 1 store 34 tuples/s 2 selection, store 68 tuples/s 3 projection, store 35 tuples/s 4 selection, projection, store 70 tuples/s 5 cartesian product, (left) 3 tuples/s projection, store (right) 5 tuples/s
And, τ1 = C1 λTurbine = 0.02∗10 = 0.2 (second). For the projection operator o2, λ2 = f2 ∗ λ1 = 1.0 ∗ 3 = 3 tuples/second. τ2 = C2 λ1 = 0.01 ∗ 3 = 0.03 second. Formula 1 becomes τ1 + τ2 = 0.2 + 0.03 = 0.23 < 1; therefore SP is feasible. Finally, feasibility of W P is checked. Because o2 is located at the top of SP, λSP equals λ2 . λSP is smaller than λDB ; therefore W P is feasible. Our result confirms that the query plan in Figure 35 is feasible. 4.4.6
Experiment
We investigated the accuracy of validation results. Our environment consists of a Pentium D 3GHz CPU, 2GB memory, Windows Vista Business OS, MySQL 5.0 and JDK 6. Because we chose the default parameters for MySQL, it was not tuned. In this experiment, our system first validates the query in Figure 31 with several input rates. We then try to execute the query in the system. The result is presented in Figure 36. Although there is a miss near the border, overall accuracy seems good. If we want to improve accuracy, we need to estimate λSP and λDB more precisely. We show the summary of our experiments validating 5 queries (Figure 37). Figure 37 indicates that our method can work for several types of queries.
4.5 Summary This Section presented Harmonica, a data stream management system, which combines a stream processing engine [32] and databases. It provided SQL-like queries that support not only persistence requirements, but also integration requirements for streams and relations. Experiments showed that Harmonica was able to estimate the amount of writable tuples to a DBMS for a variety of operator trees. It indicates that Harmonica makes it possible to execute stream archiving in the optimal ratio. Therefore, Harmonica contributes to solving the (P-I) performance problem from the standpoint of archiving.
Stream-Based Real World Information Integration Framework
199
5 Real World Oriented Advanced Functions This section covers the full part of (A-II) described in section 1. It also includes real world oriented advanced functions on stream processing for applications to detect events. This section starts by briefly presenting pattern matching and reasoning with probabilities. The first two probabilistic event detection techniques are developed because of the uncertain nature of sensor data. Sensor data is inherently noisy; therefore it is often modeled with probabilities [34]. The section then briefly presents video oriented techniques. Continuous media integration is developed because media type oriented data processing is important in sensor data processing to improve the effectiveness of approaches. Using event detection information, the techniques make it possible to eliminate unnecessary network traffic.
5.1 Complex Event Processing [30] This research presents a working framework and a query language to support probabilistic queries for composite event detection over probabilistic event streams. The language allows users to express Kleene closure patterns for complex event detection in the physical world. Our processing method first detects sequence patterns over probabilistic data streams using active instance graph, a new data structure that handles active states with an NFA. Our method then computes the probability of each detected sequence pattern based on its lineage. Through the benefit of lineage, the probability of an output event can be directly calculated without taking into account the query plan. An optimized plan can be selected. Finally, we evaluate performance of our method and compare the results with the original and optimized query plan. The experiment clearly shows that our technique outperforms a naive query plan.
5.2 Probabilistic Reasoning [28] This research presents the integration of probabilistic data streams and relational database using Bayesian Networks, which is one of the most useful techniques for reasoning uncertain contexts. This research has three contributions. For the first contribution, we modeled the Bayesian Networks as an abstract data type in an object relational database. Bayesian Networks were stored as objects, and we defined new operators to integrate Bayesian networks and relational databases. A Bayesian Network is organized in a graphical model; therefore it does not directly fit into a relational database constituted of relations. Our new operators allow a part of the data to be extracted from Bayesian Networks as relations. For the second contribution, to allow continuous queries over data streams generated from a Bayesian Network, our method introduces lifetime, a new concept. Although the Bayesian Network is a well known reasoning method, it is not yet treated in data stream systems. The lifespan
200
H. Kitagawa et al.
allows a Bayesian Network to detect multiple events for each evaluation of a continuous query. For the third contribution, we present efficient methods for probability values propagations. The methods omit unnecessary update propagations for continuous queries. Experiments clearly show that our algorithm outperforms Pearl’s message passing algorithm.
5.3 Continuous Media Integration 5.3.1
Processing Video Camera Streams [35]
Integrating numeric and text streams is comparatively easy, but integrating video streams and other sources is difficult because of video stream properties: highlyfrequent, large volume and complex binary data. With this background, we developed a video stream management system. The research provides a SQL-like query interface for heterogeneous information sources including video streams. To integrate video streams, we employ an abstract data type to hold a subsequence of video frames and functions to extract meta data objects from video data objects. Beyond that, we also present a dynamic source selection scheme for some applications, such as moving objects tracking using video streams. The scheme is used when information sources to be accessed may change according to changes in user interest. Our system dynamically accesses necessary information sources and saves system resources by closing connections to unnecessary information sources. 5.3.2
Dynamic Resource Allocation [25]
Research on distributed stream processing is important. We focus on distributed stream processing, which makes it possible to dynamically select target information sources during query processing. When target information sources change, a data transfer route and network traffic from the target information sources to users also change. We therefore present a management system that keeps optimal network traffic in distributed stream processing with dynamic selection of target information sources. To keep traffic low, the system reconsiders operator allocations according to changes of target information sources. In this research, we evaluated the efficiency of our distributed management system by experiments.
5.4 Summary This section briefly presented advanced functions we have developed. They include complex event processing, probabilistic reasoning, and continuous media integration. These functions are useful for sensor data applications, but they are not supported in relational data processing. For that reason we believe this section contributes to (A-II).
Stream-Based Real World Information Integration Framework
201
6 Concluding Remarks and Future Work 6.1 Concluding Remarks This chapter described new techniques for processing data streams generated from multiple WSNs. When considering applications that focus on a single WSN, the reduction of power and network consumption for in-network processing are research issues. On the other hand, when considering applications that focus on multiple WSNs, the research issues change to general data management. Sensor data obtained from multiple WSNs should be accessed equivalently; therefore a kind of abstraction should be incorporated into an infrastructure that manages them. An abstraction process requires a framework; we presented the stream-based real world information integration framework in this chapter. The framework provides two features: (A-I) efficient data processing and (A-II) real world oriented functions. With (A-I), we described two techniques. First, we described a novel multiple query optimization technique (Section 3). We showed that our optimized approach has a great advantage over naive execution. Query optimization is important in processing massive sensor data streams in real-time. Second, we described an integration method of a stream processing engine and databases for persistence (Section 4). For integration, we described an optimized data writing technique and a feasibility validation scheme for persistence requirements. The integration method is important when archiving massive data streams efficiently. With (A-II), we described a variety of functions we developed. The functions include probabilistic reasoning (Section 5.2), complex event processing (Section 5.1), and continuous media integration (Section 5.3).
6.2 Future Work Unfortunately, the world does not yet have a definitive answer for managing sensor data. The path to the answer can be roughly split in two directions. Multiple directions exist because requirements for sensor data applications are spread over wide areas. The first direction is to enrich functions of infrastructures tailored to each application domain. This is based on the “one size does not fit all” philosophy [31]. This limits the application domain specifically, and incorporates fertile functions into an infrastructure over relational operators. Sensor data are inherently noisy and have a variety of applications; therefore the functions are based on each purpose. The functions may include probabilistic reasoning [34], noise filtering [15], sequence data processing [3], and uncertain data management [4]. Each infrastructure may be scalable and provide sufficient functions for a specific domain of applications, but they are not individually applicable to other domains. The second direction is to limit functions of an infrastructure, and to provide only simple functions. This is based on the KISS (Keep It Simple, Stupid) philosophy, which is adopted on web scale key-value storages such as BigTable [11] or PNUTS [13]. Key-value stores are popular because they are scalable on shared nothing PC
202
H. Kitagawa et al.
clusters, which are relatively less expensive than tightly-coupled specialized hardware. And typical web applications require neither relational operators nor ACID transaction. This direction has a questionless shortage, which urges applications to construct application specific functions by themselves. The amount of sensor data may sometimes rise above the amount of web contents; therefore this direction may also be popular when that occurs. The above two directions should be adopted according to the situation. If an application domain is clear, then the first approach should be applied whether the scale of streams and applications is huge or small. If an application domain is unclear and the size of streams and applications is huge, then the second approach should be applied. Otherwise, the relational data management technology can be applied. In summary, the approaches should be chosen according to the scale and application demand.
Acknowledgement This research has been supported in part by the Japan Science and Technology Agency (CREST), the Grant-in-Aid for Scientific Research from JSPS (#1820005, #21240005, #20700078) and MEXT (#19024006).
References 1. Abadi, D.J., Ahmad, Y., Balazinska, M., Cetintemel, U., Cherniack, M., Hwang, J.H., Lindner, W., Maskey, A., Rasin, A., Ryvkina, E., Tatbul, N., Xing, Y., Zdonik, S.B.: The design of the borealis stream processing engine. In: Proc. CIDR (2005) 2. Aberer, K., Hauswirth, M., Salehi, A.: A middleware for fast and flexible sensor network deployment. In: Proc. VLDB, pp. 1199–1202 (2006) 3. Agrawal, J., Diao, Y., Gyllstrom, D., Immerman, N.: Efficient pattern matching over event streams. In: Proc. ACM SIGMOD (2008) 4. Agrawal, P., Benjelloun, O., Sarma, A.D., Hayworth, C., Nabar, S., Sugihara, T., Widom, J.: Trio: A system for data, uncertainty, and lineage. In: Proc. of VLDB, pp. 1151–1154 (2006) 5. Arasu, A., Babu, S., Widom, J.: The cql continuous query language: Semantic foundations and query execution. VLDB Journal 15(2) (2006) 6. Arora, A., Ramnath, R., Ertin, E., Sinha, P., Bapat, S., Naik, V., Kulathumani, V., Zhang, H., Cao, H., Sridharan, M., Kumar, S., Seddon, N., Anderson, C., Herman, T., Trivedi, N., Zhang, C., Shah, R., Kulkarni, S., Aramugam, M., Wang, L.: Exscal: Elements of an extreme scale wireless sensor network. In: Proc. of IEEE RTCSA, pp. 102–108 (2005) 7. Avnur, R., Hellerstein, J.M.: Eddies: Continuously adaptive query processing. In: Proc. ACM SIGMOD, pp. 261–272 (2000) 8. Ayad, A.M., Naughton, J.F.: Static optimization of conjunctive queries with sliding windows over infinite streams. In: Proc. ACM SIGMOD, pp. 419–430 (2004) 9. Carney, D., C ¸ etintemel, U., Cherniack, M., Convey, C., Lee, S., Seidman, G., Stonebraker, M., Tatbul, N., Zdonik, S.: Monitoring streams – a new class of data management applictions. In: Proc. VLDB, pp. 215–226 (2002) 10. Chandrasekaran, S., Franklin, M.J.: Streaming queries over streaming data. In: Proc. VLDB, pp. 203–214 (2002)
Stream-Based Real World Information Integration Framework
203
11. Chang, F., Dean, J., Ghemawat, S., Hsieh, W.C., Wallach, D.A., Burrows, M., Chandra, T., Fikes, A., Gruber, R.E.: Bigtable: A distributed storage system for structured data. In: Proc. OSDI (2006) 12. Chen, J., DeWitt, D., Naughton, J.: Design and evaluation of alternative selection placement strategies in optimizing continuous queries. In: Proc. IEEE ICDE, pp. 345–356 (2002) 13. Cooper, B.F., Ramakrishnan, R., Srivastava, U., Silberstein, A., Bohannon, P., Jacobsen, H.A., Puz, N., Weaver, D., Yerneni, R.: Pnuts: Yahoo! ’s hosted data serving platform. In: Proc. VLDB (2008) 14. Demers, A., Gehrke, J., Hong, M., Riedewald, M., White, W.: Towards expressive publish/subscribe systems. In: Ioannidis, Y., Scholl, M.H., Schmidt, J.W., Matthes, F., Hatzopoulos, M., B¨ohm, K., Kemper, A., Grust, T., B¨ohm, C. (eds.) EDBT 2006. LNCS, vol. 3896, pp. 627–644. Springer, Heidelberg (2006) 15. Deshpande, A., Madden, S.: Mauvedb: Supporting model-based user views in database systems. In: Proc. ACM SIGMOD (2006) 16. Gedik, B., Liu, L.: Peercq: A decentralized and self-configuring peer-to-peer informaiton monitoring system. In: Proc. ICDCS, pp. 490–499 (2003) 17. Kadota, M., Aida, H., Nakazawa, J., Tokuda, H.: D-jenga: A parallel distributed bayesian inference mechanism on wireless sensor nodes. In: Proc. of International Conference on Networked Sensing Systems (2006) 18. Kanzaki, A., Hara, T., Ishi, Y., Wakamiya, N., Shimojo, S.: X-sensor: a sensor network testbed integrating multi-networks. In: Proc. of Int’l Workshop on Data Management for Information Explosion in Wireless Networks (DMIEW 2009), pp. 1082–1087 (2009) 19. Li, J., Tufte, K., Shkapenyuk, V., Papadimos, V., Johnson, T., Maier, D.: Out-of-order processing: A new architecture for high-performance stream systems, pp. 274–288 (2008) 20. Liu, L., Pu, C., Tang, W.: Continual queries for internet scale event-driven information delivery. IEEE TKDE 11(4), 610–628 (1999) 21. Madden, S., Franklin, M., Hellerstein, J., Hong, W.: The design of an acquisitional query processor for sensor networks. In: Proc. ACM SIGMOD, pp. 491–502 (2003) 22. Madden, S., Shah, M., Hellerstein, J., Raman, V.: Continuously adaptive continuous queries over streams. In: Proc. ACM SIGMOD, pp. 49–60 (2002) 23. Maekawa, T., Yanagisawa, Y., Sakurai, Y., Kishino, Y., Kamei, K., Okadome, T.: Web searching for daily living. In: Proc. ACM SIGIR, pp. 27–34 (2009) 24. Motwani, R., Widom, J., Arasu, A., Babcock, B., Babu, S., Datar, M., Manku, G.S., Olston, C., Rosenstein, J., Varma, R.: Query processing, resource management, and approximation in a data stream management system. In: Proc. CIDR (2003) 25. Ohki, K., Watanabe, Y., Kitagawa, H.: Evaluation of a framework for dynamic source selection in stream processing. In: Proc. International Workshop on Data Management for Information Explosion in Wireless Networks, DMIEW 2009 (2009) 26. Roy, P., Seshadri, S., Sudarshan, S., Bhobe, S.: Efficient and extensible algorithms for multi query optimization. In: Proc. ACM SIGMOD, pp. 249–260 (2000) 27. Salton, G.: Automatic Information Organization and Retrieval. McGraw-Hill Book Company, New York (1968) 28. Sato, R., Kawashima, H., Kitagawa, H.: The integration of data streams with probabilities and a relational database using bayesian networks. In: Proc. of IEEE International Workshop on Sensor Network Technologies for Information Explosion Era, SeNTIE (2008) 29. Sellis, T.: Multiple-query optimization. ACM TODS 13(1), 23–52 (1988) 30. Shen, Z., Kawashima, H., Kitagawa, H.: Efficient probabilistic event stream processing with lineage and kleene-plus. International Journal of Communication Networks and Distributed Systems 2(4), 355–374 (2009)
204
H. Kitagawa et al.
31. Stonebraker, M., C ¸ etintemel, U.: One size fits all: An idea whose time has come and gone. In: Proc. IEEE ICDE, pp. 2–11 (2005) 32. StreamSpinner Team: http://www.streamspinner.org 33. Terry, D., Goldberg, D., Nichols, D.: Continuous queries over append-only databases. In: Proc. ACM SIGMOD, pp. 321–330 (1992) 34. Tran, T., Sutton, C., Cocci, R., Nie, Y., Diao, Y., Shenoy, P.: Probabilistic inference over rfid streams in mobile environments. In: Proc. IEEE ICDE (2009) 35. Watanabe, Y., Akiyama, R., Ohki, K., Kitagawa, H.: A video stream management system for heterogeneous information integration environments. In: Proc. of 2nd International Conference on Ubiquitous Information Management and Communication, ICUIMC 2008 (2008) 36. Watanabe, Y., Kitagawa, H.: A multiple continuous query optimization method based on query execution pattern analysis. In: Lee, Y., Li, J., Whang, K.-Y., Lee, D. (eds.) DASFAA 2004. LNCS, vol. 2973, pp. 443–456. Springer, Heidelberg (2004) 37. Watanabe, Y., Kitagawa, H.: Query result caching for multiple event-driven continuous queries. Information Systems (2009) (to appear) 38. Watanabe, Y., Yamada, S., Kitagawa, H., Amagasa, T.: Integrating a stream processing engine and databases for persistent streaming data management. In: Wagner, R., Revell, N., Pernul, G. (eds.) DEXA 2007. LNCS, vol. 4653, pp. 414–423. Springer, Heidelberg (2007) 39. Wyss, C.M., Wyss, F.I.: Extending relational query optimization to dynamic schemas for information integration in multidatabases. In: Proc. ACM SIGMOD, pp. 473–484 (2007)
Toward Construction of Wearable Sensing Environments Kazuya Murao, Tsutomu Terada, and Shojiro Nishio
Abstract. The downsizing of computers has led to wearable computing devices that have attracted a great deal of attention. A wearable computer in wearable computing environments runs various applications with various sensors (wearable sensors). The beginning of this chapter describes the present situation with wearable sensing by giving examples of several applications using state-of-the-art technologies, and outlines underlying problems with wearable sensing. Even though low-power consumption is an important issue since the sensors are connected wirelessly and their batteries should be small due to restrictions with size and weight in a wearable sensing environments, conventional wearable systems did not flexibly control power supply and consumed excess power resources for unused sensors. Additionally, sensors frequently become unstable by several reasons such as breakdown of sensors. As a solution to these, we introduce a wearable sensor management device CLAD that has various functions for power management and sensed data management. The latter half of this chapter describes its application with taking power-reduction in context-aware systems as an example. Even though various systems using accelerometers have been proposed to recognize very minute motions and states in the area of context awareness, the energy consumption is not taken into consideration. In actual life, the granularity of contexts that the users require differs according to their situations. Therefore, the proposed system changes the granularity of cognitive contexts on their situations and supplies power on the basis of the optimal Kazuya Murao · Shojiro Nishio Department of Multimedia Engineering, Graduate School of Information Science and Technology, Osaka University, Japan e-mail: [email protected],[email protected] Tsutomu Terada Department of Electrical and Electronic Engineering, Graduate School of Engineering, Kobe University, Japan e-mail: [email protected] T. Hara et al. (Eds.): WSN Technologies for the Information Explosion Era, SCI 278, pp. 207–230. c Springer-Verlag Berlin Heidelberg 2010 springerlink.com
208
K. Murao, T. Terada, and S. Nishio
sensor combinations with higher degree of accuracy and fewer number of sensors. In addition, the number of sensors is reduced within the tolerances for accuracy in proportion to the rest of the power resources. Moreover, the accuracy is improved by taking context transitions into consideration. Even if the number of sensors changes, since the data for cut off sensors are complemented with our proposed algorithm, no extra classifiers and training data are required. Power consumption can be reduced without large losses in accuracy by using our system.
1 Introduction The downsizing of computers has led to wearable computing devices that have attracted a great deal of attention. Wearable computing is different from conventional computing in three ways [1]: 1. Hands-free operation: Information can be obtained without need for manual operation because the computer is equipped. 2. Power always on: The computer is always accessible because the power is always on. 3. Adaptation to personal needs: Complete services are provided according to detailed information on users obtained from sensors. Wearable sensors, which are those a user puts on his/her body, mainly enable various services to be provided such as navigation [2] and health care [3]. Although a user might use several sensors to receive multiple services, some sensors may not always be able to be used, e.g., health-care services when battery power is low or navigation services when the user is indoors. Since conventional systems using multiple sensors as introduced above cannot individually control the power supplied to these sensors, sensors that are unused also continue to consume power. Reducing power consumption is important because of the limited battery life in wearable computing environments. Moreover, sensors can become unstable due to malfunctions, power shortages, and overcurrent, and it is difficult to detect unstable operation only from sensed data. In addition, since each system works independently, one system hardly uses sensors installed on the other system. Along with the progress in wearable computing, many context-aware systems with various kinds of sensors have recently been introduced, such as systems with electromyographs [4], electrocardiograms [5], galvanic skin reflexes (GSRs) [3], and manually configured devices [6]. Context-aware systems are applied to many services i.e., health care [3], recognition of workers’ routine activities [7], and support during assembly and maintenance tasks [8]. A health-care system can recognize situations with daily habits in real time with heat sensors, GSR sensors, accelerometers, electric sphygmographs, GPS, geomagnetic sensors, and gyros. This system can recognize contexts, and provides advice to users to improve in their lives.
Toward Construction of Wearable Sensing Environments
209
In these examples, accelerometers are employed for context awareness. Although there are cameras, GPS, gyros and geomagnetic sensors as sensors for detecting location or action, these sensors have problems such as that the degree of wearability and accuracy are low, and that they cannot obtain both movement and direction while being static. However, accelerometers can obtain spatial movement and direction while being static by detecting earth gravity. Moreover, accelerometers have higher degree of resolution and are enough small to be equipped. For these reasons, accelerometers play an important role and are best for recognizing behavioral contexts between current sensors; however, number of the sensors used for existing systems is predetermined and fixed. Even though a lot of sensors are used to obtain higher degree of accuracy, there is a case where not all sensors are used. We think we would say current architectures are not optimal since there is a room we could conserve power. Even though the number of sensors in conventional systems are predetermined and fixed, if some sensors could flexibly be turned off, that would lead to reduced power consumption without greatly deteriorating accuracy. Thus far, connections from sensors to PC have been wired, which was not suit for wearable computing from point of view of wearability; positions of sensors are restricted or wires might be caught on a doorknob. Sensors have recently been connected wirelessly [9], which is expected to make context-aware services popular, while power reduction is a crucial issue because sensors have their own batteries. Different to main computers that are worn in the shade such as in bags or on hips, sensors are more wearable since they are worn on wrists or ankles. Therefore, batteries should be small. In addition, wireless sensors consume greater power for communication to PC than wired sensors, consequently controlling the power supplied to sensors is meaningful. Even though we have shown one of the three features of wearable computing: “power always on”, this means a wearable system on the whole is always on to support users or be ready for users’ demands. In these environments, each application has to strive to conserve power to support users all the time or as long as possible. This chapter describes a cross-linkage for assembled devices (CLAD) [10], which is a relay device between wearable sensors and a wearable computer. CLAD achieves flexible power supply and error control. In addition, this chapter also describes a context-aware system as an application of CLAD that changes the combination of accelerometers to reduce energy consumption [11]. Accuracies for all combinations of sensors are given, users or applications then indicate the tolerance for accuracy. A context-aware system finds the optimal combinations of sensors and notifies CLAD of these. Subsequently, CLAD manages the power supply on the basis of the optimal combinations of sensors and complements missing data for cut off sensors. The reminder of this chapter is organized as follows. Section 2 presents wearable sensing. Section 3 introduces sensor management devices. Section 4 describes power reduction in context-aware system. The performance of our system is discussed in Section 5. Finally, Section 6 concludes the chapter.
210
K. Murao, T. Terada, and S. Nishio
2 Wearable Sensing This section describes the current studies on wearable sensing and problems that they have. Subsequently after Section 3, we explain our proposal, which would be a solution to the problems. First, as mentioned in the introduction, wearable computing is a new style of computing that allows us to place sensors, actuators, displays, and PCs on our bodies. The modality that uses wearable sensors and provides situated services in the environment is called wearable sensing. Figure 1 shows a wearer with wearable sensors. The user is using sensors such as accelerometers, biomedical sensors, and a GPS. While the user is using the system, the sensors continue capturing data from the user. Consequently, some sensors with storage can store the data locally, while others send these to a wearable computer. As well as acquiring data, the wearable computer integrates and analyzes the sensing data, e.g., the user has been working for a long time, he/she is in the vicinity of a target building, or his/her pulse rate is too fast. After such processes, an application stores the data in local storage to prepare a diary, or some others provide situated services such as presenting maps or alerts to users through a head-mounted display (HMD) or a headset. When we use wearable sensing services, multiple sensors are generally combined and used for three purposes. • Multiple homogeneous sensors for improving accuracy What cannot be fully understood only with one sensor can be comprehended with multiple sensors. An example of this kind of usage is a context-aware system. A context-ware system recognizes user actions such as walking, running, and standing, and provides situated services. We call these human actions context in this field. One method of recognizing user actions is track them with markers to recognize minute actions accurately. However, it is difficult to keep capturing images with cameras and the ambient infrastructure is also costly; therefore, machine learning of sensor values is generally used in wearable computing environments. A lot of context-aware systems have been proposed thus far, and we introduce some of them here. Laerhoven et al. [12] recognized human actions by using 30 accelerometers and they measured their distinctivenesses from the number of sensors and actions. In addition, they implemented context-aware clothing that recognized events such as putting on or taking off a coat, by displaying on its front pocket the name and function of the user, his/her photo and whether or not he/she was currently granted a valid pass, as long as he/she wore the coat. Ho et al. [13] used two accelerometers on the thigh and ankle to recognize contexts to prevent mobile devices from interrupting humans while they were working. Their system found changes in contexts by recognizing basic sitting, standing, and walking contexts which indicated the user may not have been concentrating and it was time to notify him/her with messages.
Toward Construction of Wearable Sensing Environments
211
Wearable PC integrates and analyzes data, and stores them or activates output devices. E.g., Prepare a daily log for retrospective viewing and displaying documents through HMD or headset. Wearer
GPS Medical sensors
Wearer’s data are captured by sensors. E.g., Movement, location, and pulse rate.
Accelerometers
Fig. 1 Wearer with wearable sensors
• Heterogeneous sensors for improving accuracy Here, we introduce some research using two or more kinds of sensors to detect or recognize objects that have not been done accurately with only one kind of sensor. InterTrax2 [14] is a tracking sensor, which has three gyros, two accelerometers and three geomagnetic sensors. By using InterTrax2 with HMD, 3D graphics such as those in games change in accordance with human head tracking. In more detail, users of flight simulators can check instruments and gauges or the sky condition just by facing the objects. DRM [15] is a position detection system, which combines a geomagnetic sensor, an accelerometer, a gyro and a GPS. The geomagnetic sensor always detects direction, and the walking distance is calculated from the accelerometer. In a reinforced concrete building where geomagnetics malfunction, gyro assists the accelerometer instead of the geomagnetic sensor. Although this inertia navigation can detect position, errors accumulate during long use. Therefore, DRM compensates for position with GPS, and even with infrequent updates, the position can be accurately obtained. Stiefmeier et al. [8] also used position information to recognize actions associated with bicycle maintenance such as turning pedals and inserting bulbs. They detected hand positions with respect to the bicycle with two ultrasonic transmitters on both wrists and four ultrasonic listeners in a room. The gestures themselves were recognized with accelerometers, gyros, and geomagnetic sensors on the user’s hands, resulting in a high degree of accuracy. A system that recognizes nurses’ routine activities [7] supports their regular work. They have to memorize what they did during the day to communicate this to colleagues to avoid mistakes such as overdosing. However, this is time consuming and mistakes might still occur. This system recognizes their activities with accelerometers and detects their locations with IR-ID receivers. IR-ID receivers installed above both sides of doors capture an ID sent from IR-ID transmitter attached to a nurse’s cap. The system tracks room-level locations as well as the behaviors of nurses when they pass through doors from the strength of the received ID.
212
K. Murao, T. Terada, and S. Nishio
Apart from these, localization with radio waves [16] and broad detection of positions by reading RFIDs [17] have been proposed, and using these in combination has improved their accuracies. • Heterogeneous sensors for various kinds of services There are some applications utilizing various kinds of sensors to provide various services. They provide one high-quality service as a set of various functions in contrast to the above research that has only focused on one item of information (e.g., position or context). A health-care system [3] offers multiple heterogeneous sensors to supply various kinds of services. A watch-like device and a PDA load three-axis accelerometers, photoelectric pulse wave sensors, galvanic skin reflex (GSR) sensors and thermometers, and they play their own roles. In short, the accelerometers capture movements, the amount of activity, and the number of steps. The photoelectric pulse wave sensors capture the pulse rates and the condition of the autonomous nervous system. The GSR sensors and thermometers obtain the user’s mental and psychological conditions and his/her temperature. This system obtains user’s life habits from these sensors in real time and presents advice on his/her health. Thus far, we have introduced wearable sensing applications from the points of view of accuracy and the quality of services. There is, however, a trade-off between accuracy and energy consumption. The battery should be small due to restrictions with size and weight especially in a wearable computing environment; therefore, lowpower consumption is also an important issue. One of the main purposes of the Porcupine project [6] was to reduce power consumption. Porcupine involved a switch-ball device that replaced an accelerometer. One switch ball output binary data whether tilted or not, and it had nine switch balls heading all around. Its power consumption was very low because of its simplicity but its accuracy in daily activities was greatly inferior to that of an accelerometer. This is because an accelerometer has better resolution than other devices. As one effective usage, Laerhoven et al. [18] proposed a technique to detect sleep posture with Porcupine. Since body postures while sleeping are basic, they result in very different tilt switch patterns. Users have to equip multiple kinds of sensors to use several applications. There has been some research that has tackled on multiple sensor management, and we describe them here. Personal Mobile Hub [19] is a hub supporting various heterogeneous devices whose protocols are different. However, Personal Mobile Hub is not for wearable computing but for general input and output devices, and it does not control the power supply. Phidget [20] is a toolkit that can freely use sensors and actuators just by connecting them to the Phidget Interface. However, Phidget just transmits data and does not manage these or power. Even though a reduction in the number of hardware devices themselves has been proposed to solve problems with power management in wearable computing, no research using middleware devices has been proposed.
Toward Construction of Wearable Sensing Environments
213
Conventional systems using multiple sensors have several problems. For example, a navigation system using GPS, an accelerometer, and a gyro sensor to acquire position is extremely accurate but has heavy power consumption because all sensors are used simultaneously. However, GPS cannot be used indoors, and sensors frequently become unstable because of breakdowns, power shortages, and overcurrents. It is difficult to detect instability from only sensed data. We developed a new sensor management device called Cross-Linkage for Assembled Devices (CLAD) [10] to solve these problems, which has various functions for power and sensingdata management. CLAD is positioned between the wearable computer and its sensors, and it manages the sensors to flexibly control power supply to conserve energy, and flexibly control errors to achieve sufficiently reliable sensed-data. The details on CLAD are described in the next section. Subsequently, we introduce a contextaware system that takes into consideration energy consumption as an application of CLAD’s architecture.
3 Sensor Management Device We proposed CLAD [10] which is a relay device positioned between a wearable computer and wearable sensors. CLAD manages connected sensors to (1) flexibly control the power supply to conserve energy, and (2) flexibly control errors to achieve sufficient accuracy for sensed data. There is a photograph of the CLAD prototype in Figure 2. Its dimensions are W 76 × H13 × D70 mm and the sensor is W 45 × H12 × D12 mm. CLAD has its own power source and manages connected sensors. CLAD monitors the voltage and the current to detect power shortages and overcurrents. Each sensor has a microcomputer (CPU) to process commands from CLAD. Information about the sensor (type, accuracy, output range, start-up time, operating voltage, and operating current) is stored in the CPU. CLAD has five main characteristics. • Alternative device retrieval and changeover CLAD detects sensor anomalies from consecutive outlying data points and sensor data interruptions. When these occur, CLAD identifies an alternative device by referring to the sensor profile information, and if one exists, CLAD activates it. • Power supply control CLAD always monitors its internal power source. If CLAD detects a power shortage, it reduces power consumption by cutting off power supply to some of the sensors on the basis of a user-defined policy. • Overcurrent detection If overcurrent is detected, CLAD cuts off all power supply to ensure safety. • Error detection CLAD detects problems such as outlying data and a draining batteries in sensors. Since CLAD notifies the PC of these problems, applications can deal with them individually by displaying messages such as recommending battery changes.
Fig. 2 Photograph of CLAD prototype with enlarged inset
• Pseudo-data generation When a sensor is turned off and there is no alternative device, CLAD can generate pseudo-data from learned data and their correlation to other sensors. This function improves operational reliability. The most characteristic function of CLAD is pseudo-data generation. Generally speaking, not specified in this field, there are three answers to missing data. • Listwise deletion This method uses no sensed data when the data includes at least one piece of missing data. Since we assume data are complemented when the sensor breaks down, this method cannot be used because missing data arrives consecutively in this situation. • Pairwise deletion This method uses sensed data only after missing data have been removed. However, this method causes a change in sensed data dimensions, which imposes several restrictions on context-recognition algorithms. Therefore, this method is not appropriate as a generalized mechanism. • Imputation This method uses sensed data after some values have been complemented in the missing data. Since this method does not change the dimensions of sensed data, CLAD users do not need to take the data complementary data into consideration. Taking these characteristics into account, CLAD employs imputation to complement data. Figure 3 shows example pseudo-data generation for a context-aware
Toward Construction of Wearable Sensing Environments
215
Step 1 Cognitive vector
Step 0
sensor 1-x
Pair database
Step 2 k-NN
sensor 1-y
sensor 1-x
sensor 1-z
sensor 1-y
sensor 2-x
…
Complemented vector
Nearest pair vector calculated from sensors 1-4
sensor 1-x
sensor 1-x
sensor 5-x sensor 5-y
sensor 1-y
sensor 1-y
sensor 1-z
sensor 1-z
sensor 5-z
sensor 2-x
sensor 2-x
…
… sensor 4-z sensor 5-x
Step 3 Pseudo data
…
sensor 4-z
sensor 1-z sensor 2-x
sensor 4-z sensor 5-x sensor 5-y sensor 5-z
sensor 4-z sensor 5-x
sensor 5-y
sensor 5-y
sensor 5-z
sensor 5-z
Fig. 3 Pseudo-data generation
system with five accelerometers. We have assumed that sensor 5 has been shut off by a breakdown. The pseudo-data are generated in four steps. • Step 0. Construct pair database CLAD has already collected sensed data (pair vectors) for all contexts and constructed a database of pair vectors (pair database). • Step 1. Acquire cognitive vector When the input contains missing data for some reason, i.e., sensor breakdown, the missing data is removed, and we call the remaining part the cognitive vector. • Step 2. Extract pair vector from pair database The system finds the pair vector in the database that is nearest to the cognitive vector by using the k-nearest neighbor (k-NN) method. • Step 3. Extract pseudo-data from pair vector The data for sensor 5 (missing data) are replaced by those from the extracted pair vector. A complemented vector is then generated and will be the input for the context-aware system. The distance in pseudo-data generation is calculated among working sensors between cognitive vector X = (x1x , x1y , x1z , · · · , x j , · · · , x5x , x5y , x5z ) and pair vector Pi = (pi1x , pi1y , pi1z , · · · , pi j , · · · , pi5x , pi5y , pi5z ) (i = 1, · · · , N). The subscripts, 1x or 5y, mean the x-axis of sensor 1 or the y-axis of sensor 5, and all components such as x1x have a scalar value. The N is the number of samples in the pair database. If our mechanism employs the Euclidean distance for calculating the distance between the cognitive vector and pair vector, it is difficult to classify contexts where the data for working sensors for two contexts are nearly equal and only missing data differ. Therefore, we focused on the correlation between values of sensors being equipped, and the k-NN method complemented data more accurately by using the correlation. We use the Pearson product-moment correlation coefficient to find correlation(x, y):
216
K. Murao, T. Terada, and S. Nishio
N (x − x)(y − y) ∑i=1 i i , correlation(x, y) = ∑Ni=1 (xi − x)2 ∑Ni=1 (yi − y)2
(x = y).
(1)
Generally speaking, an absolute value from 0.0 to 0.2 for the correlation coefficient means there is scarcely any correlation, from 0.2 to 0.4 means some correlation, from 0.4 to 0.7 means a good correlation, and from 0.7 to 1.0 means a strong correlation. This approach is used to apply the k-NN method to all working sensors and uses the sum of the Euclidean distance multiplied by the correlation coefficient defined as correlated distance d. The correlation is calculated from the variance of the pair vector. When we complement the data for sensor m, this is complemented with the data for sensor m in a pair vector whose dm,i is minimum as derived by dm,i = (2) ∑ {x j − pi j }2 ∗ correlation(xm, x j ). j∈working
The Euclidean distances in this method among strongly correlated sensors carry much more weight and slightly correlated sensors carry little weight. We confirmed from our pilot study that a method with correlation improved accuracy by 5.36% on average for five subjects, nine contexts, and five accelerometers. Finally, we find the nearest pair vector, PI=argmini (d1 ,···,di ,···,dN ) , and the system outputs the complemented cognitive vector, C = (c1x , c1y , c1z , · · · , c j , · · · , c5x , c5y , c5z ). x j ( j ∈ working) (3) cj = pI j ( j ∈ mal f unctioning) Some might think that using multiple classifiers for each sensor combination is as practical as our approach. However, the main advantage of our approach is that the pseudo-data generation works independently of classifiers. Since complete data are always assumed in classifiers, and no configurations are required for them, if better classifiers are developed in the future, they will be easy to integrate with our algorithm.
4 Power-Reduction in Context-Aware Systems This section describes power reduction in context-aware systems, which is basic technology of wearable applications. Context-aware systems require multiple sensors to accurately recognize contexts, and power reduction is a critical problem. The purpose of pseudo-data generation is to manage hardware errors in sensors to maintain the accuracy of context recognition. However, by focusing on redundancy in sensors, low-power consuming context-aware systems can be constructed [11]. This research utilized power management and pseudo-data generation, which were discussed in the previous section, and focused on events when the cognitive contexts (contexts to be recognized) and required accuracy levels differed according to varied situations and applications. Context-aware systems consume much less battery
Toward Construction of Wearable Sensing Environments
217
by taking the situation into consideration. This section describes the details of our system and how power consumption is reduced on the basis of three types of power saving and the system itself.
4.1 Required-Accuracy-Based Power Saving The required accuracy differs according to the situation. For example, while the highest accuracy is always required in fine-grained services, some users prefer low power consumption (long battery life-time) in daily activities. We set the threshold for accuracy in serious situations (e.g., aero-space and battlefield) at 90%; then, the best sensor combination is the least number of sensors that meets the threshold. However, in normal situations, the threshold is set at lower values. Setting a threshold in this way can flexibly provide a trade-off between accuracy and power consumption compared to that with conventional systems that only work only at full capacity. However, merely turning off sensors simply not only leads to low power consumption but also low accuracy. Subsequently, we propose mechanisms that limit power consumption while retaining accuracy.
4.2 Context-Granularity-Based Power Saving Conventional context-aware systems require numerous sensors to recognize contexts with high degree of accuracy. However, in actual life, not all trained contexts can be chosen. While a health-care system needs to recognize many detailed contexts, an information-presentation system on a head-mounted display (HMD) [13] only has to determine whether a user is moving and it is possible to recognize such easy contexts with fewer sensors. This section defines a context group, which is a subset of trained contexts. In the given situations in Figure 4, Situation 1 is used by an application that needs to know whether the user is moving or not. When a user is advised not to exercise hard by his/her doctor, Situation 2 is used by an application to alert the user of high levels of activity. Situation 3 is used for a health care application that calculates the number of calories consumed by recognizing detailed contexts. This method works as follows. First, a user selects a situation according to his/her circumstances or active applications. Second, our system finds a sensor combination with the least number of active sensors while fulfilling the same threshold for accuracy as that described in section 4.1. If there is no situation that the user needs, he/she can define a new situation by regrouping contexts. For example, when a doctor wants to know the context as to whether the a patient has passed away, the doctor makes lying, kneeling, and sitting one group and makes the other contexts another group. Where the former context group lasts for a long time, the doctor may judge the patient is in a critical condition. Distinguishing life and death is one of the most important roles played by wearable systems. It can be done efficiently for military personnel and elderly citizens living alone. Since it is sufficient to recognize the contexts in
Situation 1 with only one sensor, we can achieve a low power-consuming contextaware system without any extra classifiers or training data by turning off redundant sensors and complementing the missing data for them.
4.3 Context-Transition-Based Power Saving People transiting from one action to the next basically continue the current context and this restricts the context that happens next. Table 1 lists a context transition obtained from the results of our preliminary evaluation of five people (three men and two women) with nine contexts (walking, running, descending steps, ascending steps, bicycling, lying, kneeling, sitting, and standing). The candidates for the context after bicycling are expected to “continue riding a bike” or “get off a bike”. Lying and kneeling seldom occur in actual life. To use these characteristics first, we list possible candidates for the context from all the contexts listed in Table 1. Second, a classifier is trained for all the previous contexts, i.e., when a user trains a classifier for recognizing bicycling, the training data only includes walking, bicycling, and standing (see Table 1). The other contexts are trained likewise. Finally, when a user is bicycling, the system recognizes contexts using the trained classifier for bicycling. Restricting the candidates of possible contexts in this way based on the current context greatly increases recognition accuracy. This means that our system requires fewer sensors, and power consumption can be reduced. This mechanism becomes more effective when cognitive contexts have increased. In addition, context-transition is automatically constructed from the records of daily activities obtained from fully operational sensors.
Toward Construction of Wearable Sensing Environments
4.4 Context-Aware System The construction of a context-aware system introduced in this chapter is outlined in Figure 5. A context-aware system processes input in response to user contexts and a context-recognition system positions a part that analyzes the meaning of sensed data and recognizes user actions. The context-recognition system obtains sensed data through CLAD, and incorrect data are complemented in a complementary unit if this exists. The recognition unit then analyzes the meaning of sensed data. Data do not need to be analyzed such as position data from GPS and temperature are directly sent to the rule engine at this stage. The event-driven rule engine [21] provides services according to rules described with a set of three commands: Event, Condition, and Action. Action has a description that controls input and output devices, and it controls these devices through plug-ins or CLAD. For example, it is possible to describe rules that provide alerts when pulse exceeds 180 bps or present maps of a user’s vicinity when he/she stops. There have been many kinds of algorithms used for recognition. Our context aware system employs the Support Vector Machine (SVM) [22] as a classifier. We also implemented several classifiers such as Memory Based Reasoning (MBR) and Self-Organizing Maps (SOMs) [23]. Since the evaluation results tended to be the same for all classifiers and SVM had the best total accuracy, we selected it for our explanation. SVM is a classification algorithm that often provides competitive or superior accuracy for a large variety of real-world classification tasks [22]. Consider the problem of separating a set of training data (x1 , y1 ), (x2 , y2 ), · · ·, (xJ , yJ ) into two classes, where xi ∈ RN is a feature vector and yi ∈ {−1, +1} is its class label. Assuming that the classes can be separated by the hyperplane, w ∗ xi + b, and no knowledge about the data distribution is given beforehand, the optimal hyperplane is the one with the maximum distance to the closest points in the training dataset. We can find the optimal values for w and b by solving 1 min ||w||2 2
subject to yi (w ∗ xi + b) ≥ 1, ∀i = 1, · · ·, n.
(4)
220
K. Murao, T. Terada, and S. Nishio Context-aware system
Output devices (HMD, LED, speaker, motor…) Command to devices
The factor of 1/2 is used for mathematical convenience. By using Lagrange multipliers, λi (i = 1, · · ·, n), this can be rewritten as: N
max
N
∑ λi − ∑
i=1
N
λi λ j yi y j xTi x j ,
subject to
i, j=1
∑ yi αi = 0, λi ≥ 0,
(5)
i=1
and the results in a classification function are f (x) = sign
n
∑ λ i yi x i ∗ x + b
.
(6)
i=1
Most of the λi take zero. Those xi with nonzero λi are so-called support vectors, all of which are on each hyperplane. Where classes are not separable, the Lagrange multipliers are modified to 0 ≤ λi ≤ C, i = 1, · · ·, n, where C is the penalty for misjudgment. This arrangement is called a soft margin and a reason SVM performs so well. The original optimal hyperplane algorithm proposed by Vapnik was a linear classifier. The data from the input space RN is mapped to a high dimensional feature space by using x → Φ (x) to obtain a non-linear classifier. However, nonlinear classifiers were created by applying the kernel trick to maximum-margin hyperplanes. Assuming there exists a kernel function, K(x, x ) = Φ (x) ∗ Φ (x ), a non-linear SVM can be constructed by replacing the inner product, x ∗ x , by the kernel function, K(x, x ), in Eq. (6). Commonly used kernels are polynomials
Toward Construction of Wearable Sensing Environments
221
K(x, x ) = (γ · x ∗ x + c)d , the Gaussian Radial Basis Function (RBF), K(x, x ) = exp(−γ ||x − x ||2 ), and the sigmoid, K(x, x ) = tanh(γ · x ∗ x + c). We examined the kernels while changing their parameters, i.e., penalty C: 5000, 50000, and 500000, γ in RBF and the sigmoid: 0.0001, 0.005, 0.001, 0.01, 0.1, and 1, and constant c in RBF and the sigmoid: 0, 0.1, and 1. No kernels demonstrated better performance than linear classification, and the C of 50000 exhibited the best performance. The extension of a 2-class SVM to an N-class can be achieved, e.g., by training N SVMs, i.e., one class will be separated from the others.
5 Evaluation This section discusses our evaluation of the system on the basis of accuracy and power consumption.
5.1 Evaluation Environment The training data and test data to evaluate our system were captured from five different test subjects, who wore five sensors on both wrists, both ankles, and their hip. They acted according to the scenario listed in Table 2. All instructions were very simple and the subjects had a great deal of freedom in their activities, such as stopping halfway through the scenario or walking when talking to other people. This scenario involved nine basic activities: walking, running, ascending steps, descending steps, bicycling, lying, kneeling, sitting, and standing [6]. The first five were dynamic and the last four were static. The sensors they wore were three-axis accelerometers [9]. The sampling frequency was 20 Hz. We used a SONY VAIO VGN-UX90PS (Intel Core Solo Processor 1.2 GHz) as a wearable computer. Even though data were analyzed off-line in the evaluation, an on-line system was also constructed, as shown in Figure 6. This system played a 3D object animation of the current context, displayed contexts into which the current context could change, and drew graphs of the acceleration data. Moreover, the required accuracy for recognition with or without data complements and context transition could also be set. The system found a combination with the least sensors and highest accuracy needed to meet the required accuracy, then controlled the power supply. The algorithm for context awareness was the SVM described in Section 4.4. Figure 7 shows the raw data and the manually labeled contexts of two test subjects in the scenario. As can be seen from the figure, even though subjects’ general actions are similar, their detailed actions are different. The subject in the upper part of the figure occasionally stopped while walking. However, there is little change in the contexts for the subject at the bottom. In addition, before riding a bicycle, the top subject one stands and the bottom one walks. Because the data used in the evaluation contained various characteristics, they were suited for the evaluation. Raw data using a context-aware algorithm would generally not be used but preprocessed by extracting the feature values to enable the meaning of sensed data to be
222
K. Murao, T. Terada, and S. Nishio Table 2 Scenario used for evaluation
Instruction: Go to the co-op by bicycle to buy juice Outdoor phase Laboratory → downstairs → to bicycle shed through corridor → to co-op by bicycle → buy juice from vending machine → return to the lab. Instruction: Read a journal, rest, then go upstairs to complete a job Indoor phase Select a journal from bookshelves → read it while seated → rest on a sofa → recall the job and run upstairs → return to the lab.
Fig. 6 On-line application
understood. Let us assume time t = T , where the constructed context-aware system uses mean μi (T ) and variance σi (T ) for 20 samples of 15-dimensional sensed data (cognitive vector) ci (T ) (i = 1, · · ·, 15) retraced from time t = T . T 1 ci (t) ∑ 20 t=T −19 2 ci (t) − μi(t)
μi (T ) = σi (T ) =
T 1 ∑ 20 t=T −19
(7) (8)
Characteristic vector Z(T ) is normalized using the following equation for 30dimensional vector, X(T ) = [μ1 (T ), · · ·μ15 (T ), σ1 (T ) · · · σ15(T )], where M and S are the mean and the standard deviation of X. Z(T ) =
X(T ) − M S
The mean of Z(T ) becomes 0 and its variance becomes 1 after this conversion.
(9)
Toward Construction of Wearable Sensing Environments
Fig. 7 Raw data and hand-labeled context for the test subjects
The logged data in the scenario were manually labeled, 20% of which became training data and the data for complementing missing data and the remaining 80% were used for testing. Since the amount of data used for complementing missing data was much less than that in testing, our proposal makes a significant contribution in that no all possible data sets are required from the remaining sensors. In addition, the pair database is easy to construct because its data do not need to be labeled.
5.2 Results We first measured the power consumed by our hardware consisting of CLAD and the sensors. The results are listed in Table 3. Each inactive sensor consumes about 11.4 mW as a standby power requirement. “CLAD only” means the power consumed by the CLAD itself without any sensors. From this table, the power consumed in full operation is 297 mW (five active sensors and CLAD). The evaluation in this chapter is based on CLAD. This power is not only used for sensing and communication, but also for detecting errors and controlling power. Even if extremely low-power sensors are developed, CLAD is also expected to consume less power; therefore, CLAD will be beneficial in the future. In addition, this architecture should be more effective in an environment where more sensors are used to recognize more precise contexts.
224
K. Murao, T. Terada, and S. Nishio Table 3 Power consumption @ 5.18 V Hardware Power consumption [mW ] CLAD only 92.204 11.396 Inactive sensor 40.922 Active sensor
5.2.1
Evaluation of Context Group
The first set of results was obtained for the accuracy of the context-groups described in Section 4.2. The results are plotted in the group×group confusion matrices in Figure 8. These results were obtained with five active sensors and without any complementing for missing data. The cells indicate the number of positives per activity (with the true positives diagonally), and the accuracy indicates the rate of true positives over each activity. For example, the walking data in Situation 3 are correctly recognized 17513 times, while only misrecognized 75 times as running. The matrix makes clarifies each context, i.e., what contexts are easy to recognize. As can be seen, the accuracies vastly differ by context: bicycling and lying are high, but descending and kneeling are low. Since the accuracies of contexts differ, it is not fair to calculate accuracy over all data in an environment where the amounts of data are not equal. One clear finding from these results is that more abstract groups achieve better classification rates. The second set of results was obtained for the accuracy in changing complementing methods for each context-group and is plotted in Figure 9. The horizontal axis shows 31 combinations of active and inactive sensors (an open circle means active and a blank means inactive), and the vertical axis shows the accuracy of context recognition. The partitions in the figure mean the borders for the number of active sensors. As previously mentioned, the more abstract a situation is, the more accuracy increases. As sensors are turned off, the accuracies decrease, but these declines are small due to the complementary mechanism described in Section 3. For comparison, we have presented accuracy without complementing for missing data. Here, inactive sensor data were replaced with the average data from the other active sensors. If data are not complemented well, the decreases are very serious [10]. In Situation 1 with more than one sensor, the accuracies are the same as those during full operation. Situation 2 has the same tendency as Situation 1. Even though Situation 3 also has the same tendency, the accuracies on the whole are worse than those for the other situations because of its cognitive complexity. In brief, the power consumption can be reduced by turning off sensors while maintaining accuracy, and the accuracy increases in appropriate situations. There are examples of optimal sensor combinations in each situation are listed in Table 4. The tolerances for accuracy were assumed to be 94, 90, and 85%. We assumed that the tolerances were the accuracies determined by the user or application; under severe conditions, the tolerance would be high, or tolerance would be low to accomplish a long battery life times in daily life. In each situation with each tolerance, our system could select the least number of sensors to satisfy the tolerance for accuracy. In the present
Toward Construction of Wearable Sensing Environments
225 Accuracy
Accuracy
Static
22346
1675
879
89.75%
Static
22561
2339
90.61%
Limited action
1571
23694
673
91.35%
Dynamic
2564
53835
95.28%
Intense action
474
821
32707
96.19%
(a) Situation 1: 92.94% on average (not over all data but contexts)
(b) Situation 2: 92.43% on average (not over all data but contexts) Accuracy
Walking
17513
75
102
155
308
12
68
147
1578
87.75%
Running
132
2001
118
98
0
0
0
0
13
84.72%
Descending
580
13
2340
20
0
0
0
0
46
78.02%
Ascending
489
14
15
2457
5
0
0
0
0
82.45%
Bicycling
304
4
6
23
31141
0
0
35
127
98.42%
Lying
4
0
0
0
0
5684
0
44
28
98.68%
Kneeling
130
0
0
22
138
0
1841
21
168
79.35%
Sitting
147
0
0
3
35
46
20
7183
147
94.75%
Standing
835
51
23
41
160
0
223
51
7856
85.02%
(c) Situation 3: 87.69% on average (not over all data but contexts) Fig. 8 Confusion matrices for each situation
circumstances, we needed to determine an optimal combination of sensor for each situation through actual measurements the same as shown in Figure 9. Power consumption was calculated from Table 3. (e.g., with four active sensors, the power consumption became 92.2 + 40.9 × 4 + 11.4 267 mW .) The rate of reduction was how much power consumption was reduced against that in full operation. Note that if no combination achieved tolerance, the system worked with five active sensors. Even with 94% tolerance, the power consumption was reduced by 20% in Situation 1. Moreover, with 87% tolerance, all five sensors were not required in all situations. 5.2.2
Evaluation of Context Transition
Table 5 lists the accuracy of context recognition taking into consideration the human-context transitions in Table 1. These results were measured with five active sensors and accuracy was improved for all contexts. For example, when a user changed his/her action from bicycling to walking, the system made 75 mistakes for running and 147 mistakes for sitting without taking transition into consideration. However, considering the context transition, running and sitting were
Fig. 9 Accuracy vs. combinations of sensors in each situation Table 4 Optimal sensor combinations and their power consumption Tolerance Situation 94%
90%
87%
1 2 3 1 2 3 1 2 3
No. of sensors 3 5 5 1 2 5 1 2 3
Combination of Power Reduction Accuracy [%] active sensors consumption [mW ] rate [%] L-wrist, Hip, R-leg 94.30 238 19.9 ALL SENSORS 92.72 297 0 ALL SENSORS 87.38 297 0 R-leg 92.03 179 39.8 R-leg, R-wrist 91.50 208 29.8 ALL SENSORS 87.38 297 0 R-leg 92.03 179 39.8 R-wrist, R-leg 91.50 208 29.8 L-leg, Hip, R-leg 87.08 238 19.9
exempted from the candidates of contexts. By removing unimaginable contexts from bicycling, walking could be recognized in 91.74% of cases (3.99% improvement). Figure 10 shows accuracies when context-transition was applied to all sensor combinations. The environment is for Situation 3 (nine contexts). Both axes are the same as those for Figure 9. The accuracies for all combinations were improved (2.55% on average). Though combinations of fewer sensors had not been able to be selected because of poor accuracy, they were able to be selected after context-transition was applied, e.g., the accuracy in full operation before context-transition was applied (87.38%) was improved by using three sensors after context-transition was applied (88.91%). Finally, let us consider a combination of all mechanisms. Context-granularitybased and context-transition-based methods can co-exist. It is difficult to consider context transition using the examples of context groups in this chapter. Static and dynamic groups can transit from one group to the other in Situation 1. Static, limited action, and intense action groups can also interchange. However, where there are more contexts to recognize, there will be many context groups. In such cases,
Toward Construction of Wearable Sensing Environments
227
Table 5 Changes in accuracies by context transition Previous context Next context
Accuracy [%] Before After 87.75 91.74 98.42 98.58 85.02 88.89 87.75 93.19 98.68 99.24 79.35 91.14 94.75 97.53 85.02 87.52
100 95 Accuracy [%]
90 85 80 75 70
Before being applied After being applied
65 60 L-leg L-wrist Hip R-wrist R-leg
Fig. 10 Accuracy vs. sensor combination before and after context-transition was applied
restricting transitions between context-groups results in better performance while using our proposals. Further evaluations and measurements of power are part of our future work.
5.3 Considerations 5.3.1
Processing Time
The processing time for pseudo-data generation for two sensors took 19.6 msec (N=1226), which is the longest of all combinations. The SVM recognized one context in less than 0.1 msec, which is negligible therefore compared to the processing time of pseudo-data generation, the processing time of recognition is negligible. Even though this proposal is supposed to be used in real-time situations, the processing time is no longer than the interval of data acquisition of 50 msec. Junker
228
K. Murao, T. Terada, and S. Nishio
et al. [24] evaluated accuracies by changing the sampling frequency from 10 to 100 Hz shifted in 10 Hz intervals and the resolution from 1 to 16 bits shifted in 1 bit intervals, and reported that only the combinations of either 10 Hz or 1 bit performed worse. Since this evaluation was conducted under 20 Hz and 8 bits, these conditions did not significantly affect the results. 5.3.2
Application to Wireless Connections
We investigated the possibility of applying our system to wireless connections. The power supply itself is shut off over wired connections, while the power can be controlled by sending a sleep command over wireless connections. As a result, lowpower context-aware architectures works properly for wireless wearable systems constructed with Bluetooth- or ZigBee-based sensors. In addition, wireless sensors contain batteries. Different to wearable computers that are in the shade such as in bags or on hips, sensors are wearable since they are put on wrists or ankles. Therefore, batteries should be small and consequently control the power supplied to sensors by using the resources of wearable computers would be useful.
6 Conclusion This chapter described CLAD: a sensor management device for wearable computing, and mechanisms to reduce the power consumption of a context-aware system. CLAD achieves power saving by managing sensor power on the basis of the circumstances and highly reliable data by managing the sensed data. The context-aware system reduced in energy consumption while maintaining accuracy. By assuming the granularity of cognitive contexts would differ according to situations, we defined “context groups” including some similar contexts. In addition, we focused on transition in human actions to improve accuracy by narrowing down the candidates of possible contexts. The proposed system changed the granularity of cognitive contexts for user situations and managed the power supply on the basis of optimal combinations of sensors. From the evaluation, we clarified that not all sensors were needed to recognize required contexts according to the situations. As a result, our system reduced energy consumption. The advantage of our system is that even if the number of sensors changes, since the data for shut-off sensors are complemented by our proposed algorithm, the system does not require any extra classifiers or training data. Highly dependable, highly data-reliable, and low-power consuming systems can be constructed just by connecting sensors when CLAD becomes more popular. In addition, when the use of context-aware systems becomes more widespread, computers can be aware of user conditions and high-quality interactions can be achieved in various fields such as medicine, sports, and daily life. We plan to propose a mechanism for automatically changing the current situation in future work. We had to manually change the situation in our current system or with other devices. The new system may be able to determine the current situation by using cooccurrence information among contexts. In addition, current context
Toward Construction of Wearable Sensing Environments
229
transition is applied with binary decision. There are methods that limit contexts with conditional probability such as Bayesian networks. Such probabilistic approaches are flexible, but have weaknesses with unforeseen events such as changes of context, which seldom occur on the whole. We are continuing to solve this problem in our ongoing study. Furthermore, we think that our approach can be applied to wireless sensors with sleep commands. We also plan to measure power consumption with wireless sensors.
References 1. Miyamae, M., Terada, T., Tsukamoto, M., Nishio, S.: Design and Implementation of an Extensible Rule Processing System for Wearable Computing. In: Proc. of the 1st IEEE Intl. Conference on Mobile and Ubiquitous Systems: Networking and Services (MobiQuitous 2004), August 2004, pp. 392–400 (2004) 2. Kanbara, M., Tenmoku, R., Ogawa, T., Machida, T., Koeda, M., Matsumoto, Y., Kiyokawa, K.: Nara Palace Site Navigator: A Wearable Tour Guide System Based on Augmented Reality. In: Proc. of the 3rd CREST/ISWC Workshop on Advanced Computing and Communicating Techniques for Wearable Information Playing, October 2004, pp. 7–14 (2004) 3. Ouchi, K., Suzuki, T., Doi, M.: LifeMinder: A Wearable Healthcare Support System with Timely Instruction Based on the User’s Context. IEICE transaction on information and systems E87-D(6), 1361–1369 (2004) 4. Toda, M., Akita, J., Sakurazawa, S., Yanagihara, K., Kunita, M., Iwata, K.: Wearable Biomedical Monitoring System Using TextileNet. In: Proc. of the 10th IEEE Intl. Symposium on Wearable Computers (ISWC 2006),October 2006, pp.119–120 (2006) 5. Shen, C.L., Kao, T., Huang, C.T., Lee, J.H.: Wearable Band Using a Fabric-Based Sensor for Exercise ECG Monitoring. In: Proc. of the 10th IEEE Intl. Symposium on Wearable Computers (ISWC 2006), October 2006, pp. 143–144 (2006) 6. Laerhoven, K.V., Gellersen, H.W.: Spine versus Porcupine: a Study in Distributed Wearable Activity Recognition. In: Proc. of the 8th IEEE Intl. Symposium on Wearable Computers (ISWC 2004), October 2004, pp. 142–149 (2004) 7. Naya, F., Ohmura, R., Takayanagi, F., Noma, H., Kogure, K.: Workers’ Routine Activity Recognition using Body Movement and Location Information. In: Proc. of the 10th IEEE Intl. Symposium on Wearable Computers (ISWC 2006), October 2006, pp. 105–108 (2006) 8. Stiefmeier, T., Ogris, G., Junker, H., Lukowics, P., Tr¨oster, G.: Combining Motion Sensors and Ultrasonic Hands Tracking for Continuous Activity Recognition in a Maintenance Scenario. In: Proc. of the 10th IEEE Intl. Symposium on Wearable Computers (ISWC 2006), October 2006, pp. 97–104 (2006) 9. Wirelss Technologies, Inc.: http://www.wireless-t.jp/ 10. Murao, K., Takegawa, Y., Terada, T., Nishio, S.: CLAD: a Sensor Management Device for Wearable Computing. In: Proc. of the 7th Intl. Workshop on Smart Appliances and Wearable Computing (IWSAWC 2007) (June 2007) 11. Murao, K., Tsutomu, T., Takegawa, Y., Nishio, S.: A Context-Aware System that Changes Sensor Combinations Considering Energy Consumption. In: Indulska, J., Patterson, D.J., Rodden, T., Ott, M. (eds.) PERVASIVE 2008. LNCS, vol. 5013, pp. 197– 212. Springer, Heidelberg (2008)
230
K. Murao, T. Terada, and S. Nishio
12. Laerhoven, K.V., Schmidt, A., Gellersen, H.W.: Multi-Sensor Context Aware Clothing. In: Proc. of the 6th Intl. Symposium on Wearable Computers (ISWC 2002), October 2002, pp. 49–57 (2002) 13. Ho, J.: Using Context-Aware Computing to Reduce the Perceived Burden of Interruptions from Mobile Devices. In: Proc. of Conference on Human Factors in Computing System (CHI 2005), April 2005, pp. 909–918 (2005) 14. aicube Inc.: http://www.aicube.com/ 15. Silicon Sensing Systems Japan, Pointman DRM.: http://www.sssj.co.jp/ 16. Michalakos, S., Christou, I.T.: G2G: Location-Aware Mobile Social Networking with Applications in Recommender Systems and Gaming. In: Proc. of the 6th ACM Intl. Conference on Advances in Mobile Computing and Multimedia (MoMM 2008), November 2008, pp. 163–169 (2008) 17. H¨ahnel, D., Burgard, W., Fox, D., Fishkin, K., Philipose, M.: Mapping and Localization with RFID Technology. In: Proc. of IEEE Intl. Conference on Robotics and Automation (ICRA 2004), April 2004, pp. 1015–1020 (2004) 18. Laerhoven, K.V., Borazio, M., Kilian, D., Schiele, B.: Sustained Logging and Discrimination of Sleep Postures with Low-Level, Wrist-Worn Sensors. In: Proc. of the 12th IEEE Intl. Symposium on Wearable Computers (ISWC 2008), September/October 2008, pp. 69–77 (2008) 19. Husemann, D., Narayanaswami, C., Nidd. M.: Personal Mobile Hub. In: Proc. of the 8th IEEE Intl. Symposium on Wearable Computers (ISWC 2004), October 2004, pp. 85–91 (2004) 20. Phidgets Inc.: http://www.phidgets.com/ 21. Wearable Toolkit: http://wearable-toolkit.com/ 22. Vapnik, V.: The Nature of Statistical Learning Theory. Springer, Heidelberg (1995) 23. Kohonen, T.: Self-Organizing Maps. Springer, Heidelberg (1996) 24. Junker, H., Lukowicz, P., Tr¨oster, G.: Sampling Frequency, Signal Resolution and the Accuracy of Wearable Context Recognition Systems. In: Proc. of the 8th Intl. Symposium on Wearable Computers (ISWC 2004), October 2004, pp. 176–177 (2004)
Applying Overlay Networks to Ubiquitous Sensor Management Satoshi Matsuura, Kasutoshi Fujikawa, and Hideki Sunahara
Abstract. Due to the global spread of the Internet and the rapidly diminishing price of wireless sensors, a large number of heterogeneous sensor networks have been developed and interconnected around the globe for the sharing of sensing data on a global scale. Such data greatly affect our daily lives, such as environmental issues and business development. A scalable decentralized system that provides multiple-attribute search is a key mechanism for sharing sensing data on a global scale. In this chapter, current sensor networks and distributed data management approaches are introduced. In particular, we discuss an implementation of distributed mechanisms and propose a geographical-based overlay network for managing ubiquitous sensors. Keywords: large scale sensor networks, distributed data management.
1 Introduction Today’s sensing devices, such as weather sensors and web cameras, have become powerful. In addition, such devices have Internet connectivity and GPS capability. On the other hand, network connectivities are spread globally. We Satoshi Matsuura · Kasutoshi Fujikawa Nara Institute of Science and Technology, 8916-5 Takayama-cho, Ikoma-shi, Nara, 630-0192, Japan e-mail: [email protected],[email protected] Hideki Sunahara Keio University, 4-1-1 Hiyoshi, Kohoku-ku, Yokohama-city, Kanagawa, 223-8526, Japan e-mail: [email protected] Satoshi Matsuura National Institute of Information and Communications Technology, 4-2-1 Nukui-Kitamachi, Koganei, Tokyo 184-8795 Japan
T. Hara et al. (Eds.): WSN Technologies for the Information Explosion Era, SCI 278, pp. 231–247. c Springer-Verlag Berlin Heidelberg 2010 springerlink.com
232
S. Matsuura, K. Fujikawa, and H. Sunahara
can easily assume that the number of sensing devices will increase as their cost decreases. If these sensing devices can immediately collect and provide a large amount of environmental information anywhere and anytime, we can obtain detailed and up-to-date information related to our circumstances. Moreover, a large amount of such information can be applied to many applications, such as for environmental problems, educational materials, and businesses. The next step in the evolution of sensor networks is to interconnect the sensing devices over the Internet, and to create a platform for sharing sensing data on a global scale. Therefore, a decentralized system combined with scalability and multiple-attribute search is a key mechanism. In this chapter, we show the problems and requirements of a ubiquitous sensing environment and propose a geographical-based overlay network called ‘Mill’ for managing ubiquitous sensors. Mill expresses a square region as consecutive one-dimensional IDs and supports geographical region search through O(log N ) messages. In addition the proposed overlay network supports multi-attribute search. The organization of this chapter is as follows. Section 2 describes issues on implementing ubiquitous sensor networks. Section 3 shows current distributed data managements. Section 4 shows our implementation methodology of overlay networks. In section 5, we summarize this chapter.
2 Connecting Sensor Networks over the Internet In this section, we describe approaches to interconnect sensor networks over the Internet, and to manage sensing data on a global scale. In general, most sensor network systems, such as GSI [6], GSN [1] and Live E! [14] consist of three layers: sensing devices, middleware, and applications (see Figure 1). The middleware layer of sensor network systems provides a common interface among different sensing devices so that users can easily share and handle the data acquired by various sensors as well as control the sensing devices. In addition, this layer stores the sensing data and provides the access method to the target sensing data using a query language. As the number of sensors/users increases, the importance of the middleware layer grows. In the following, we describe the requirements for middlewares managing ubiquitous sensor data on the Internet. Scalability: Sensing data are increasingly generated from various kinds of sensing devices including weather sensors, web cameras, PDAs, and mobile phones, as most sensing devices continuously measure environmental conditions. Therefore, the middleware layer of sensor network systems have to continuously handle a large amount of sensing data. Also, we can easily imagine that search queries will be issued by many users. Multi-attribute search: Sensor nodes measure a wide variety of data, Also, sensing data have various attributes. For example, temperature data have its own value (e.g., 23.1 Celsius), as well as place where the data is generated, creation time, type of sensor, owner of the sensor, manufacturer
Applying Overlay Networks to Ubiquitous Sensor Management
233
Fig. 1 General architecture on ubiquitous sensor networks
of the sensor, and accuracy of the data. Users want to search the target data by several kinds of attributes. For example, an office worker wants to know the current weather conditions around his/her office and home. In this case, the middleware layer is required to support search mechanisms by time, geographical location, and sensing data types. On sensor network systems, there are various sensors and many search requirements from users. Thus, the middleware layer should provide the flexibility of a search mechanism. Geographical range queries: In a ubiquitous sensing environment, sensing devices can be distributed anywhere, and the system managing this large amount of data should provide a retrieval mechanism by which users can search a global area for particular data. Consequently, a distributed mechanism should support geographical range queries. Real-time search: Sensing data are constantly generated. In most cases, these data will gradually lose their values because the current status is constantly changing and the acquired data is becoming older. For example, when a user leaves his/her office in the evening, he/she would not want to receive yesterday’s weather conditions. On sensor network systems, the middleware layer is required not only to manage a large amount of sensing data but also to provide a fast search mechanism. Flexibility: On the Internet, we can use a wide variety of communication facilities, such as wired and wireless network devices and communication protocols. If a sensor network system depends on specific hardware or software, the system will not be widely used over the Internet. Therefore,
234
S. Matsuura, K. Fujikawa, and H. Sunahara
the middleware layer must support a great diversity of devices. Furthermore, the system must consider future changes of sensor networks and user requirements.
3 Distributed Approaches for Ubiquitous Sensors In this section we introduce existing overlay approaches that promise to deal with the requirements described. We focus on peer-to-peer (P2P) technologies.
3.1 DHT: Distributed Hash Table For decentralizing information and queries, P2P networks have been widely studied, especially, distributed hash tables (DHTs). Examples of DHTs are Chord [22], CAN [20], Pastry [21], Tapestry [23], and Kademlia [16]. DHTs construct structured overlay networks by hashed IDs, which are scalable for the number of nodes and support fast search. Consequently, DHTs provide distributed environment for several Internet-scale applications. DHTs manage a hash table by distributed nodes, as depicted in Figure 2. Each DHT node manages a part of the hash table, and this part of the
Hash Table Hash Table
DBMS
DHT
Fig. 2 Management of Hash Table
Applying Overlay Networks to Ubiquitous Sensor Management
18
data
Hashed-ID
Node
18
70
27
30
send hashed-ID
HASH
235
ID: 0 ID: 70
DHT network ( ID: 0-99 )
27 HASH ID: 30
join
random 65
ID: 65
data
Fig. 3 Sharing hashed IDs on DHT
hash table is determined by a node ID. During the process of joining in DHT networks, nodes calculate a random ID between zero and the maximum value of the ID space. This ID space is equal to the ID space of a hash table. After joining, nodes calculate hashed IDs of their data, and send these hashed IDs to particular nodes. Figure 3 shows a typical process of sharing hashed IDs. In this figure, the ID-space is one-dimensional circle and the ID-space of a DHT network consists of 100 IDs (0–99). The node with ID-0 manages part of the ID-space from ID-0 to ID-29 because the ID of the next node is 30. Each node calculates hashed IDs from its own data and sends hash-IDs between 0 and 29 to the node with ID-0. After sharing hash-IDs, users can access specific data by calculating hash-IDs of target data and communicating with nodes managing these hashed IDs. Each DHT manages own topology of ID-space (e.g., circle, tree, N-dimensional torus). Each DHT has its own routing mechanism based on IDs to improve search performance. For example, Kademlia has a 160-bit ID-space by SHA-1 [5], and each node knows the distance between it and other nodes. This distance is calculated by digits of its ID XOR other node’s ID and managed as a tree topology called k-buckets. Users can search for target data via O(log N ) messages.
3.2 DHTs Supporting Range Queries DHTs are efficient for exact match queries but are not well suited for range queries because the hash mechanism destroys the ordering of data. A number of approaches have been proposed to realize range queries in DHTs.
236
S. Matsuura, K. Fujikawa, and H. Sunahara
Level 2
12
21
79
KEY
100
101
ID
001
Level 1
Level 0
40
53
68
000
011
110
21
68
79
100
110
101
12
40
53
001
000
011
12
21
40
53
68
79
001
100
000
011
110
101
Fig. 4 Routing mechanism of Skip Graph
Skip Graph [2] is a structured overlay network supporting range queries. Skip Graph uses Skip Lists [19] as a routing algorithm, and nodes have two types of numbers, ‘key’ and ‘ID’. Figure 4 shows the routing mechanism of Skip Graph. The Skip Graph nodes line up in the order of their ‘key’ and create links to other nodes with their ‘ID’. First, nodes create links to the next nodes at level-0. Second, nodes create links to other nodes, which manage the ID, by matching the prefix of length 1 at level-1. Similarly, nodes create links by matching the prefix of length 2 at level-2. After creating links, nodes have a wide range of accessibility to other nodes because ‘IDs’ are random numbers and evenly distributed. Figure 4 also shows the routing process from a node (key:68) to another node (key:12). A node (key:68) can send messages to the target node (key:12) via 2 hops. If users want to obtain data corresponding to the numbers from 15 to 25, users can obtain these data from two nodes (key:12, key:21). Skip Graph supports fast search and range queries by managing these two types of numbers. SkipNet [7] uses the same basic data structure as Skip Graph. SkipNet focuses primarily on the content and path locality properties. On SkipNet networks, nodes store data in neighbor nodes near subnetted networks to the locality of storing and obtaining data. Mercury [4] is a more general overlay network than SkipNet, because Mercury supports range-based lookups on multiple-attributes. Mercury independently manages several ID-spaces, and each attribute is related to one of the attributes. For example, a car has certain attributes (color=#FFFFFF, price=$30,000, weight=1,500 kg). To share car information, Mercury creates three ID-spaces related to color, price, and weight. Chord or Symphony mechanisms manage one ID-space, and Skip Graph and SkipNet can only manage one attribute. On the other hand, users can search cars by color, price or weight in a Mercury network. By managing several ID-spaces, Mercury manages multiple-attributes queries.
Applying Overlay Networks to Ubiquitous Sensor Management [45-51)
Level 0
B
[12-17)
[5-8)
237
[72-75)
[23-29)
[54-61)
Level 1
[81-85)
Level 2
A [0-5)
[8-12)
[17-23)
[34-39)
[29-34)
[39-45)
[51-54)
[61-68)
[75-81)
[68-72)
[89-93)
[85-89)
[93-100)
Level 3
Level 4
Fig. 5 Routing mechanism on BATON
In the database field, B-Trees [3] is the main method of creating indexes of data. BATON [9] supports range queries by managing a balanced tree structure. In this tree structure, each node (e.g., rack-mounted machine, desktop PC) corresponds to a node of the tree. BATON assigns a range of values to each node. The range of values directly managed by a node is required to be to the right of the range managed by its left subtree and to be less than the range managed by its right subtree. This index structure is based on a mainmemory index called T-Tree [11]. Figure 5 shows the routing mechanism of BATON. Nodes manage several links to other nodes at the same level besides managing links to parent and child nodes. Nodes can access other nodes via O(log N ) messages by these links at the same level. If node-A searches for data related to ‘74’, the query can reach node-B via four messages, as depicted in Figure 5. Compared with Skip Graph and SkipNet, BATON improves load balancing by balanced tree structures and supports range queries. However, another cost arises from maintaining the tree topology. BATON* [10] is an extension of BATON for supporting multiple-attribute queries. BATON* prioritizes attributes and reduces three attributes of low priority to one-dimensional indexes using the Hilbert Space Filling Curve [8]. Compared with Mercury, BATON* reduces the cost of managing ID-spaces by reducing the number of attributes.
3.3 The Live E! Project We have operated Live E! servers, as shown in Figure 6. Live E! uses a tree topology. Owners of sensors add Live E! servers to manage sensor data. Owners also can control data accesses by the servers. Our tree topology is suited for implementing an authentication mechanism in distributed environments such as DNS. However, on sensor networks, it is necessary to gather sensor data from several sites and reply to user queries. It is difficult to provide multiattribute search in this environment on a large scale and with many different types of sensor data managed among dozens of sites distributed around the globe. Live E! servers separately manage profiles of sensor devices and sensor
238
S. Matsuura, K. Fujikawa, and H. Sunahara Disaster Management
Science
MultiAttribute search Sensor & Overlay
Japan: 5 nodes
NKXGGTQQV
NKXGGYTCRRGT YGCVJGTVYPKE
YTCRRGT
VY
Live E! on PIAX
Facility Management
Education / Agriculture
NKXGGEQGRJWMGV
RUW
NKXGGJQPIQ
LR NKXGGNTWCEVJ
NTW
NKXGGPKWGFWVY *CF[CN
PKWVY
+28%04*&;RUW
Taiwan: 2 nodes
Thai: 3 nodes
NKXGGMOF
MOFLR
In-Network Data Processing Multi-Domain Sensor Networking
NKXGGPCKUV
PCTCLR
Delay Tolerant Network
Embedded gateway
Fig. 6 Live E! Architecture
data and periodically receive these profiles from children sites. Each Live E! server creates an index table from these profiles. Using this index table as a routing table, Live E! servers support multi-attribute search as well as an authentication mechanism in distributed environments. However, the cost of supporting multi-attribute search is high and response-performance is not so good because such type of queries are passed and calculated through several servers in different countries. Live E! servers could provide basic mechanisms for managing large scale sensor networks. However, these servers should have more scalability and high performance for multi-attribute search.
4 The Mill Approach for Geographical Overlays In this section, we show our geographical location-based overlay network called “Mill”[15]. Mill increases the performance of the Live E! project. Therefore, Mill handles queries from sensors/users and manages fresh data. After merging these data, existing Live E! servers continue data processing. We plan to install nodes of Mill on Live E! network step by step. On Live E! project, we try to construct an information infrastructure in which everyone can access sensor data freely, collaborating with many universities, local governments, companies and organizations.
Applying Overlay Networks to Ubiquitous Sensor Management
239
real space
logical space
2-D: surface
1-D: line
→ B: ID 0→3 C: ID 0→15
A: ID 0 0
11
5
7
13 15
10
4
6
12 14
01
1
3
9
0
15
1
14
2 3
13 12
00
(0001)
0 (0000)
00
4
11
(0011)
2
8
10
(0010)
01
5
11 6
10 10
11
9
8
7
Fig. 7 2D-1D mapping method
4.1 ID Space Management Mill divides two-dimensional space into a grid cell by latitude and longitude. A grid cell is a small square region. If each grid cell is represented by a 64 bit-ID, representing the earth’s surface, the size of a grid cell is in millimeters. This size is enough to represent from small regions to large regions in ubiquitous sensing environments. To explain our ID management method briefly, here we assume that each cell is assigned a 4-bit identifier as Figure 7 shows. Mill manages these IDs as one-dimensional circular IDs, and each Mill node is responsible for some of the circular IDs. The ID of each cell is generated by alternating x- and y-bits. For example, if an x-bit is ‘00’ and a y-bit is ‘11’, a cell ID is ‘0101’. This method is called “Z-ordering” [17]. The ID-space is very small, only 4-bits; however, in real use, Mill’s ID space (the surface of the earth) is represented by 64-bits. Mill expresses a square region as consecutive cell IDs and can search a square region for target data by one query. Mill searches location-related information by a few queries against any region size. Because of this feature, Mill can reduce queries related to geographical locations. For example, we compare search costs among several overlay networks for searching a target region. DHTs only support exact match queries. In a DHT network, a node searches all points in a target region. Therefore, the search cost is 2i × log N , where i is the number of bits, and N is the number of nodes. Overlay networks supporting multi-attribute search manage each attribute independently. In other words, a latitude attribute is not related to a longitude attribute. This
240
S. Matsuura, K. Fujikawa, and H. Sunahara
is why a node searches a large strip of region of latitude and longitude related to a target region. Therefore, the search cost is 2 log N + 2m× 232/2i/2 , where m is the number of nodes in a target region. In a Mill network, a node can search sequential IDs simultaneously. In the target region, a search query is sequentially sent from node to node. Then, the search cost is logN + m. Let ‘i’, ‘m’, and ‘N’ be 54, 20, and 20480 respectively, each search cost is as follows. • DHT s : 254 × log 20480 = 2.58 × 1017 • M ulti-Attributes : 2 × log 20480 + 2 × 20 × 232 /227 = 1309 • M ill : 1/2 log 20480 + 20 = 27 On distributed environments, it is fundamentally difficult to support multiattribute search, because a system has to gather/check the whole data related with queries in advance. Using the data, product sets, which are answers of queries, are calculated. Mill reduces search costs as opposed to other overlay network because it manages square regions as sequential IDs. To support multi-attribute search, Mill replies to queries in two steps. First, Mill searches for a node which manages a region related to the query. After that, the node queries for the other attributes and sends the query result to the users. In other words, Mill lets users search by geographical attributes and supports multi-attribute searches in distributed environments.
4.2 Implementation Methodology of Overlay Network In order to improve the performance for recurring queries, we use a skiplist with iterative routing as a cache for routing information. Figure 8 shows that
Mill node
Keep
A cluster in a particular local area
HTTP
checking current status
HTTP
Mill Network Mill Network
HTTP HTTP
HTTP HTTP
Keep users
uploading
HTTP
sensing data
HTTP
HTTP HTTP
Uploading Client
Receive a notification of event Sensing devices
Fig. 8 Mill network based on HTTP
Applying Overlay Networks to Ubiquitous Sensor Management
241
users and sensing devices communicate with Mill nodes. In a Mill network, nodes manage sensor data based on the geographical location. Consequently, users constantly communicate with the same node if they keep checking the current status of a particular region. Caches of routing information created by skiplist with iterative routing enable queries and sensor data to be dynamically clustered in a particular local area, and nodes and sensing devices can directly connect to a target node. In addition, this cache is effective for searching the adjacent area of the target node. Sensing devices also directly upload data to the target node in the same way. A caching mechanism by iterative routing optimizes the query paths from users and sensing devices. For this reason, the number of relayed queries can be significantly reduced. Users or sensing devices communicate with nodes on a Mill network through HTTP, which is suitable for iterative routing because it basically consists of one request and one response. The request and response correspond to the query and reply of an iterative routing. Users easily obtain sensor data on a Mill network using web browsers or other HTTP clients, without installing any special software. In addition, iterative routing is efficient for avoiding performance reduction caused by a time-out. Recursive routing is fast, but querying clients cannot detect relaying nodes. For this reason, if a time-out occurs, a querying node cannot do anything. On the contrary, by using iterative routing, clients can communicate with all relaying nodes. A querying node, therefore, can find nodes causing the time-out and change the routing path by picking up another node in its own routing table.
4.3 Evaluations We evaluated the performance of Mill. We have implemented nodes of Mill using Perl. Table 1 lists the specifications of the implementation and the experimental environment. We measured the performance of downloading data, uploading data, and notifying of events. One node on the Mill network, consisting of a total of 10 nodes, processes many queries from sensing devices or users. Through this experiment, we clarified the performance limitations. Table 1 Specification of Mill node CPU AMD Opteron252 x 2 Memory DDR400 4GB Hard Disk Ultra320 SCSI 10000rpm 150GB NIC Gigabit Ethernet (PCI-X) OS Linux 2.6.17-10 (ubuntu 6.10) Language Perl 5.8.8 Database SQLite 3.3.5
242
S. Matsuura, K. Fujikawa, and H. Sunahara
1
) c e s ( e im t e s n o p s e r
data-1 data-10 data-100 data-1000
0.1
0.01
320
0.001
10
100
3000
1000 load (data/sec)
20000
10000
42000
100000
Fig. 9 Downloading sensor data
As shown in the above section, Mill determines localities of processing sensor data as well as the accessibility to the whole geographical region. On a Mill network, some clusters related to particular local areas are dynamically created. One cluster consists of one Mill node and a large number of sensing devices and users, and almost all communications are processed within each cluster. If we evaluate the performance of one Mill node, the performance of the whole Mill network is almost clarified. We evaluated the performance of downloading data. It is known that queries generally have localities, i.e., users tend to constantly receive sensor data of the same place. In most cases, users directly communicate with target nodes. For this reason, we evaluated the performance limits of downloading data per node. The limitation of downloading performance at one node clarifies the performance limits of the whole Mill network. Figure 9 shows the results of the downloading performance. The “response time” means the time from when a user sends a query to a target node to when the user receives the answer of the query. One datum consists of the ID of a sensor, sensor type, the time when the datum is generated, Mill-ID where the datum is generated (made from latitude and longitude), and the value of a sensor. The example format (JSON) of one datum is as follows. Users can receive this datum if they access a URL (http://192.168.0.1/get/data?last=1&sensor type=Pressure) of a target node assigned “192.168.0.1” as IP-address. On this URL, users can additionally describe regions (latitudes/longitudes), time and other attributes. In this experiment, users constantly get current sensor data related with a particular region (latitudes/longitudes) and a sensor type. These multi-attribute queries are solved by two phases. First, attributes of latitudes/longitudes are
Applying Overlay Networks to Ubiquitous Sensor Management
243
solved among Mill nodes. Second, other attributes are solved by a particular node. Users can directly access to a target node and get sensor data once geographical attributes are solved. The size of one datum is approximately 150 bytes. { "id":1, "mill_id":"0xed01944ec1361d9e", "sensor_type":"Pressure", "time":"2008-09-24T11:06:44+09:00", "value":"995.2" } In Figure 9, “data-1” means that users receive one sensor datum per query. One Mill node can respond to approximately 320 queries per second. On the other hand, “data-10” means that users receive 10 sensor data packets per query. Therefore, one Mill node can respond to approximately 300 queries per second. This result shows that one Mill node can process almost 9.3 times the amount of data traffic by merging sensor data compared with “data-1”. As users attempt to receive larger amounts of data, downloading performance improves. On the contrary, response time and efficacy worsen. As Figure 9 shows, “data-100” provides better downloading performance than other cases, considering both the downloaded data and the response time. In the Live E![14] network, weather sensors upload current data once a minute. Therefore, if all users want to keep receiving updates of 10 kinds of data packets, one Mill node can handle 18,000 (= 3000(data/sec)÷10(data)× 60(sec)) users within its performance limits. Compared with other overlay networks, Mill reduces almost all relaying queries in an ubiquitous sensing environment, because users can directly access to a target node once the geographical location of its node is solved among Mill nodes. As Figure 8 shows, a Mill node, sensors and users construct a cluster as like a Client/Server model in a particular local area. If users receive data on a 1000-node network, the cost of the search is approximately 10 hops on current overlay networks. The relaying cost is almost equal to that of receiving data because the size of sensor data is very small. Mill reduces the cost of search by approximately 90% on a 1000-node network. This reduction is also efficient if sensing devices upload data. Next, we evaluated the performance of uploading data. Sensor devices directly communicate with a target node because almost all sensors are deployed at a particular location. For this reason, we evaluated the performance limits of uploading data per node. Figure 10 shows the results of the uploading performance. One datum consists of the ID, Mill-ID, sensor type, time, and value. Uploading clients use the latitude and longitude instead of the Mill-ID. As sensing devices (uploading clients) upload larger data, uploading performance improves. In addition,
sensing devices can save their energy by uploading a large amount of data at once. However, “data-1000” is too large to improve performance. To upload sensor data, uploading clients have to register before uploading data. When a Mill node receives sensor data from uploading clients, the node checks that the whether sensor data are sent by a registered sensor. This process of checking data worsens performance.
Applying Overlay Networks to Ubiquitous Sensor Management
245
We then optimized the format for uploading sensor data and compared the performances of normal and optimized uploading. Each of sensor data has MillID and this ID is verified when sensor data are stored on a node. Optimized format consists of profile part and data part. The profile part represents status of a sensor. The data part represents sensor data. Splitting status and data part, nodes verify large sensor data at one time. Ordinary weather sensors have several functions. For example, these sensors measure temperature, humidity, and pressure. Aggregating sensor data sent from the same sensor device, Mill nodes reduce the number of verifications on sensor data. Figure 11 shows the comparison results. “opti” means that Mill nodes can check data blocks simultaneously using the optimized format. As the data becomes larger, the efficacy of optimization becomes better. If a sensing device measures 10 kinds of data and sends the data to a node once per minute, one Mill node can handle 13,000 (≈ 220(data/sec) × 60(sec) = 13200) sensors within its performance limits. If an uploading client aggregates 10 sensors, that means that by sending 100 “data packets” to a node per minute, one Mill node can handle 52,000 (≈ 870(data/sec) × 60(sec) = 52200) sensors per node. The bottleneck of downloading/uploading performance arises from database transactions. The uploading performance is worse than the downloading one, because writing processes need more complex transactions than getting processes. Using faster disks/DBMSs, these performances become better. However, it is more important that costs of downloading/uploading queries are distributed among Mill nodes. If it is necessary to manage larger sensors/users, we just need to install additional nodes without exchanging disks or reconfiguring DBMSs. From these evaluations, we found that hundreds of nodes are needed to manage millions of sensors/users. This amount of nodes is practical if a largescale sensor network is constructed. One of the advantages is that sensors and users dynamically create clusters on a Mill network. If many queries concentrate on a node, the overload does not affect other nodes on other clusters. This is because generation/consumption of sensor data are processed in each cluster. To solve and distribute this overload of query concentrations, it is only necessary to add several nodes around this overloaded node.
5 Summary Development of sensor technologies and the global spread of the Internet tell us that many heterogeneous sensor networks are developing all over the globe and interconnecting for share ubiquitous sensing data. In addition, recent sensing devices have become more intelligent and can easily connect to the Internet. Moreover, the prices of these devices have rapidly diminished. A decentralized system combined with scalability and multiple-attribute search is a key mechanism for sharing sensing data on a global scale.
246
S. Matsuura, K. Fujikawa, and H. Sunahara
In this chapter, current sensor networks and distributed data management technologies were introduced. We implemented a geographical location-based overlay network called “Mill”. This overlay network provides a distributed environment based on a geographical location and supports multi-scale region search using O(log N ) messages and also supports multi-attribute search. The results of our evaluations show that one Mill node can handle more than 10,000 users and 10,000 sensing devices. By installing hundreds/thousands of Mill nodes at different places on the Internet, we will be able to manage a large amount of data in a ubiquitous sensing environment.
References 1. Aberer, K., Hauswirth, M., Salehi, A.: Infrastructure for data processing in large-scale interconnected sensor networks. In: Proceedings of The 8th International Conference on Mobile Data Management, MDM 2007 (May 2007) 2. Aspnes, J., Shah, G.: Skip Graphs. In: Proceedings of the fourteenth annual ACM-SIAM symposium on Discrete algorithms (SODA 2003), January 2003, pp. 384–393 (2003) 3. Bayer, R., McCreight, E.: Organization and maintenance of large ordered indexes. Acta Informatica 1(3), 173–189 (1972) 4. Bharambe, A.R., Agrawal, M., Seshan, S.: Mercury: supporting scalable multiattribute range queries. ACM SIGCOMM Computer Communication Review, 353–366 (2004) 5. Eastlake, D., Jones, P.: US Secure Hash Algorithm 1 (SHA1). RFC 3174 6. Fok, C.L., Romain, G.C., Lu, C.: Towards a flexible global sensing infrastructure. ACM SGBED Review 4(3), 1–6 (2007) 7. Harvey, N.J.A., Jones, M.B., Saroiu, S., Theimer, M., Wolman, A.: SkipNet: A Scalable Overlay Network with Practical Locality Properties. In: Proceedings of Fourth USENIX Symposium on Internet Technologies and Systems, USITS 2003 (March 2003) ¨ 8. Hilbert, D.: Uber die Stetige Abbildung Einer Linie auf Ein Fl¨ achenst¨ uck. Mathematische Annalen 38, 459–460 (1891) 9. Jagadish, H.V., Ooi, B.C., Vu, Q.H.: BATON: A Balanced Tree Structure for Peer-to-Peer Networks. In: Proceedings of the 31st international conference on Very large data bases (VLDB 2005), Seprember 2005, pp. 661–672 (2005) 10. Jagadish, H.V., Ooi, B.C., Tan, K., Vu, Q.H., Zhang, R.: Speeding up Search in Peer-to-Peer Networks with A Multi-way Tree Structure. In: Proceedings of the 2006 ACM SIGMOD international conference on Management of data, June 2006, pp. 1–12 (2006) 11. Lehman, T.J., Carey, M.J.: A Study of Index Structures for Main Memory Database Management Systems. In: Proceedings of the 12th international conference on Very large data bases (VLDB 1986), August 1986, pp. 294–303 (1986) 12. Japan Meteorological Agency, http://www.jma.go.jp/ 13. Japan Ministry of Internal Affairs and Communications Statistics Bureau, http://www.stat.go.jp/ 14. Live E! project, http://www.live-e.org
Applying Overlay Networks to Ubiquitous Sensor Management
247
15. Matsuura, S., Fujikawa, K., Sunahara, H.: Mill: A geographical location oriented overlay network managing data of ubiquitous sensors. IEICE Transactions on Communications E90-B, 2720–2728 (2007) 16. Maymounkov, P., Mazieres, D.: Kademlia: a peer-to-peer information system based on the XOR metric. In: Druschel, P., Kaashoek, M.F., Rowstron, A. (eds.) IPTPS 2002. LNCS, vol. 2429, p. 53. Springer, Heidelberg (2002) 17. Morton, G.M.: A computer Oriented Geodetic Data Base; and a New Technique in File Sequencing. Technical Report, IBM Ltd. Ottawa, Canada (1966) 18. Orlanski, I.: A rational subdivision of scales for atmospheric processes. Bulletin of the American Meteorological Society 56(5), 527–530 (1975) 19. Pugh, W.: Skip Lists: A Probabilistic Alternative to Balanced Trees. Communications of the ACM 33, 668–676 (1990) 20. Ratnasamy, S., Francis, P., Handley, M., Karp, R., Schenker, S.: A Scalable Content-Addressable Network. In: ACM SIGCOMM, pp. 161–172 (2001) 21. Rowstron, A., Druschel, P.: Pastry: Scalable, decentralized object location and routing for large-scale peer-to-peer systems. In: Guerraoui, R. (ed.) Middleware 2001. LNCS, vol. 2218, pp. 329–350. Springer, Heidelberg (2001) 22. Stoica, I., Morris, R., Karger, D., Kaashoek, M.F., Balakrishnan, H.: Chord: A scalable peer-to-peer lookup service for internet applications. In: Proceedings of the 2001 conference on Applications, technologies, architectures, and protocols for computer communications (SIGCOMM 2001), September 2001, pp. 149–160 (2001) 23. Zhou, B., Kubiatowicz, J., Joseph, D.A.: Tapestry: An Infrastructure for Faulttolerant Wide-area Location and Routing. Technical Report UCB/CSD-011141, Computer Science Division, U. C. Berkeley (April 2001)
Abstract. This chapter describes a wireless sensor network testbed, named XSensor, that integrates multiple sensor networks. X-Sensor manages multiple wireless sensor networks constructed at different sites, which can be accessed by remote users. This makes easy for users to conduct experiments in different environments. Moreover, X-Sensor provides real sensor data acquired by sensor nodes in each sensor network, which are useful for researchers working on research fields related to sensor data such as sensor data mining and analysis.
1 Introduction In the research area of wireless sensor networks, a large number of algorithms and protocols have been proposed. To evaluate the performances of these methodologies, a large-scale sensor network testbed using real sensor nodes is required. This is because it is common that wireless sensor networks are constructed by sensor nodes that have very poor resources (e.g., computational power and electrical power supply), and that wireless communication shows very complicated characteristics due to several effects such as shadowing and fading. However, most of conventional studies evaluate performances of their methodologies by simulation experiments because of the difficulty in constructing a testbed. On the other hand, various kinds of wireless sensor networks are being developed in recent years. In the future, it is expected that the numbers of sensor networks Akimitsu Kanzaki · Takahiro Hara · Yoshimasa Ishi · Tomoki Yoshihisa · Yuuichi Teranishi Osaka University, 1-5 Yamadaoka, Suita, Osaka, Japan e-mail: [email protected], [email protected], [email protected], [email protected], [email protected] Shinji Shimojo NICT, 4-2-1 Nukui-Kitamachi, Koganei, Tokyo, Japan e-mail: [email protected] T. Hara et al. (Eds.): WSN Technologies for the Information Explosion Era, SCI 278, pp. 249–271. c Springer-Verlag Berlin Heidelberg 2010 springerlink.com
250
A. Kanzaki et al.
and sensor nodes in each network will drastically increase. In such a situation, a huge amount of sensor data will be generated. Considering such an ‘explosion’ of sensor data, many researchers are trying to develop new sensor data management methodologies. Here, to evaluate the effectiveness of these methodologies, a large amount of real sensor data are required. However, it is difficult for researchers to obtain such a large amount of real sensor data. To solve these problems, several wireless sensor network testbeds have been developed in recent years. These testbeds construct a wireless sensor network and provide an interface for remote users to access the network. By using the interface, users can conduct experiments in a real sensor network. However, since these testbeds independently construct and manage a single sensor network environment, users must learn how to use different testbeds when they want to evaluate their methodologies in different environments. This is very time-consuming. Moreover, in order to evaluate performances of a various kinds of methodologies, a wide variety of testbeds are needed in terms of sensor and communication devices, ambient environment, and so on. To satisfy such requirement with the conventional testbeds, users may have to consume much time to find testbeds appropriate for their methodologies. In this chapter, we present a wireless sensor network testbed we have developed, named X-Sensor, that integrates multiple wireless sensor networks constructed by MICAz Motes [22]. X-Sensor manages multiple sensor networks deployed at different sites, and provides users with a web-based interface to access all sensor networks. This makes easy for users to evaluate their methodologies in different environments. Moreover, since each sensor network in X-Sensor accumulates data acquired by actual sensor nodes deployed at various kinds of environments, users can obtain wide variety of sensor data in terms of types of data and situations of sites. Such a real sensor data archive is very useful for many researchers working on several fields related to sensor data, e.g., sensor data analysis and mining. The reminder of this chapter is organized as follows. In Section 2, we present brief introduction of conventional wireless sensor network testbeds and other related technologies. In Section 3, we present our developed testbed in detail. In Section 4, we outline the future directions of our testbed. Finally, we summarize the chapter in Section 5.
2 Related Work 2.1 Wireless Sensor Network Testbeds In order for researchers to conduct experiments in wireless sensor networks, it is necessary that each node can operate according to the methodologies developed by researchers. To satisfy this requirement, several programmable sensor nodes have been developed in recent years. Using these programmable nodes, a large number of wireless sensor network testbeds have been developed so far. In what follows, we introduce some well-known testbeds.
Fig. 1 Node deployment and network topology of MoteLab (available at: http:// motelab.eecs.harvard.edu/motes-info.php)
MoteLab: MoteLab[23, 32] is one of the most popular sensor network testbeds. This testbed currently deploys 184 Tmote Sky[31] nodes in a building at Harvard University (see Fig. 1), and provides a web-based interface that enables remote users to access the sensor network. In the web-based interface, users can conduct an experiment by uploading executables, associating uploaded executables with nodes, and registering the time period of the experiment. Each node in MoteLab is equipped with not only the radio device but Ethernet interface in order to establish the wireline connection with the central server. Such a wireline connection is used for uploading executables to nodes and obtaining the internal statuses of nodes. Kansei: Kansei[3, 17] constructs an extremely-large scale wireless sensor network testbed that consists of 210 nodes in a warehouse. This testbed also provides a webbased interface for remote users to conduct experiments. The node in Kansei (named XSM) is based on MICA2[21] which is equipped with a radio device on 433MHz band. In addition, each XSM is installed on a Stargate[29] which is equipped with wireless 802.11 and wired Ethernet interfaces. Unlike MoteLab, users can use all communication interfaces (433MHz, 802.11, and wireline connection) for experiments. In addition, users can upload executables not only to XSMs (MICA2) but to Stargates. Thus, users can conduct experiments in a heterogeneous environment in terms of both nodes and communications. Tutornet: Similar to MoteLab, Tutornet[10] deploys nodes in a building at University of Southern California (see Fig. 2). This testbed currently deploys 107 nodes which construct 13 clusters. Each cluster consists of a Stargate and several Motes (MICAz and Tmote Sky) connected with the Stargate via USB cables. In addition, this testbed also provides a CUI-based interface for remote users to conduct experiments. Stargates in Tutornet are equipped with 802.11 interface. Using this interface, Stargates and the central server construct a multihop wireless network. When conducting an experiment, the central server firstly sends executables to Stargates
252
`[
A. Kanzaki et al.
`Z
`Y `X
X] _` __
`W
_^
_]
XX]
_\
X\ _[ XX\
X
z
Y
z
Z
_Z
\_ \^
[Y
[X [W
`
XW`
t
[
_Y
ZY \`
Z_
Z`
]
Z^
Z\
Z]
Z[
XW]
t {pw_XW
_W
^` ^_
^^
X[ ^] ^\ ^[ ^Z ^Y XX[
X ]
YX YW
]]
]\
][
XY XXY
]X ]Z
]Y
Y_
]W
Y^
\]
Y] \\
Y\
[
Y[
YZ
XW[
YY [Z
X` X_
Z X^ XWZ
Z
\
Y`
_X ^W ]^ ^X XZ ]` ]_ XXZ
[
ZX
ZZ \ XW\ ZW
XWX
Y X
^ X]
X\
X[
XZ
XY
Y XX XW
`
_
XWY
[_ \[
_ \Z XW_
3rd fl.
\Y
[[
^
[` [^
\X
\W
XW^
[\
[]
4th fl.
Fig. 2 Node deployment of Tutornet (available at: http://www.citysense.net/maps/ gmap/bigmap.html)
through this multihop network. Each stargate uploads the received executables to connecting Motes via USB. DES-Testbed: DES-Testbed[6, 14] is also an ‘indoor’ wireless sensor network testbed. Unlike other testbeds described above, this testbed deploys nodes in multiple buildings located on the campus of the Free University of Berlin. In other words, this testbed constructs a wireless sensor network spanning several buildings. The node in DES-Testbed is an embedded PC which is equipped with 802.11 interface, wired Ethernet ports, and several sensor devices (basically, temperature and humidity sensors). In addition, this node supports the remote boot over the Ethernet. Also, DES-Testbed provides a web-based interface for remote users to access the sensor network. In this interface, users can not only conduct experiments but obtain several kinds of information. For example, in a visualization service named DES-Vis[7], users can obtain the dynamic change of the network topology during an experiments. CitySense: Unlike the other testbeds described above, CitySense[4, 20] aims to construct an urban-scale sensor network testbed and deploys nodes throughout a city. The node in CitySense consists of an embedded PC, 802.11 interfaces, and several sensor devices. Currently, 37 nodes are deployed (mainly at buildings in Harvard University and BBN Technologies, see Fig. 3). Users can obtain sensor data acquired by nodes through a web-based interface. CitySense plans to deploy 100 nodes that construct a wireless sensor network, and to provide an testbed environment that remote users can conduct experiments using the network. Similar to our X-Sensor, these testbeds aim to provide an environment for conducting experiments and adopt programmable nodes such as Tmote Sky and MICAz. However, as described in Section 1, these testbeds independently construct and manage a single sensor network. Thus, when a user wants to evaluate his/her proposed methodology in different environments (e.g., different network topologies), the user must learn how to use each of different testbeds. On the other hand, since X-Sensor manages multiple sensor networks and provides an interface to access all networks, users can easily conduct experiments in different environments.
Fig. 3 Node deployment of CitySense (available at: http://www.citysense.net/maps/ gmap/bigmap.html)
2.2 Sensor Data Services and Application Platforms Among sensor network testbeds described in the previous subsection, several networks provide not only an experimental testbed but sensor data acquired by sensor nodes. For example, CitySense stores sensor data acquired by nodes in a MySQL database and provides users with a web-based interface to access the database. Similar to CitySense, several sensor data services have been developed in recent years. Also, most of these services provide an application platform that utilizes sensor data. In what follows, we briefly introduce some well-known sensor data services and application platforms. Live E!: Live E! project[18, 19] aims to establish an Internet-based sensor network platform that shares sensor data for any purpose for anyone. Currently, this project deploys over 100 weather sensor nodes called digital instrument shelter (see Fig. 4) all over the world. Each node connects to the Internet and monitors temperature, humidity, air pressure, rainfall, wind speed, and wind direction. These data are managed by 10 servers in a distributed manner. Anyone can access these sensor data via a web-based interface once he/she registers himself/herself to the project. Until now, several applications that utilize data obtained by whether sensor nodes have been developed. For example, Kurashiki-city in Okayama, Japan makes use of the information on rainfall obtained by nodes deployed at some elementary schools to protect against a flooding caused by a heavy rain. On the other hand, some universities in Hiroshima, Japan make use of these actual whether data for educational programs on geophysics. IrisNet: IrisNet (Internet-scale Resource-Intensive Sensor Network Services)[12, 16] is a very popular project. Similar to Live E!, this project aims to establish a
254
A. Kanzaki et al.
Fig. 4 Digital instrument shelter (available at: http://www.live-e.org/en/ instrument/)
worldwide sensor network platform (called worldwide sensor web), in which thousands or millions of Internet-connected sensor nodes are distributed all over the world. For this purpose, IrisNet designs and provides a software architecture which consists of two kinds of agents, sensing agents (SAs) and organizing agents (OAs). An OA is an agent that stores and manages sensor data, and answers queries from users. An OA is managed by a sensor data service in IrisNet. Each sensor data service stores sensor data as an XML document. Here, in order to reduce the loads for query and update processing, an XML document is managed by multiple OAs in a distributed manner. When a user wants to obtain sensor data stored in a service, the user issues an XPath query. On the other hand, a SA is an agent that collects data from sensor devices. In IrisNet, users can control the behavior of SAs by uploading executable codes called senselets. This enables users to design applications more flexibly. This project also has developed several prototype applications based on this architecture such as a parking-space finder and a network monitoring tool. GSN: GSN (Global Sensor Networks)[1, 2, 13] is a software middleware to facilitate the deployment and programming of sensor networks. GSN is a set of Java programs and runs on computers connecting sink nodes. The key feature of GSN is virtual sensor. Virtual sensors can be any kind of data producers, for example, real sensors, cell phones, or combination of virtual sensors. Virtual sensors have several numbers of input data streams and produce exactly one output data stream. The output data stream is generated by processing input streams. The meta-information and definition of virtual sensors are written by XML. By generating useful data streams using virtual sensors, GSN systems can facilitate the development of sensor network applications. These sensor data services and platforms are valuable for both academic and industrial users working on the areas related to sensor data. However, there are only few networks that provide sensor data generated in wireless multihop sensor networks. Thus, our X-Sensor has potential to be another kind of sensor data services and platform.
3 X-Sensor In this section, we present the detail of the current version of our developed XSensor testbed, named X-Sensor1.0.
3.1 Overview As shown in Fig. 5, X-Sensor1.0 is composed of a single testbed server and multiple sensor sites each of which consists of a gateway and a sensor network. Sensor networks: Each sensor network consists of multiple sensor nodes and a single sink node. As described in Section 1, X-Sensor1.0 currently supports only MICAz Mote as the sensor node. The sink node directly connects with the gateway. Gateways: A gateway exists in each sensor site and directly connects with the sink node and manages the sensor network. In addition, the gateway gathers and stores data acquired by all nodes in a PostgreSQL[28] database. The database can be accessed by the gateway itself and the testbed server. Testbed server: The testbed server provides the web-based interface that enables users to access sensor sites. In addition, the testbed server manages information on all gateways.
3.2 Functionalities X-Sensor1.0 provides three functionalities: (i) sensor site search, (ii) sensor data archive, and (iii) experimental testbed. These functionalities are available for users through the web-based interface on the testbed server. In what follows, we present the details of the functionalities.
Users
X-Sensor 1.0
Gateway DB
Testbed server
Gateway
DB
Gateway
DB
Sensor network
Sensor network
Sensor network
Site 1
Site 2
Site 3
Fig. 5 Overview of X-Sensor1.0
256
3.2.1
A. Kanzaki et al.
Sensor Site Search
As described in Section 1, a large number of methodologies in wireless sensor networks have been proposed. Here, each methodology may assume different environment in terms of several aspects such as sensor and communication devices. For example, a researcher working on sleep scheduling mechanism may want to conduct an experiment in an environment where nodes are densely deployed, whereas another researcher working on routing protocol may need a sparse environment. Thus, it is necessary for a user to find sensor sites that are suitable for getting sensor data or conducting an experiment. The ‘sensor site search’ functionality provides an interface for searching sensor sites that a user wants to access. Fig. 6 shows a screenshot of the sensor site search interface. On this interface, users can use the following criteria for searching sensor sites: • Keywords: The name of a sensor site and keywords that indicate the characteristics of a sensor network. Currently, the administrator of each sensor site can specify the name and arbitrary number of keywords. • Sensor Board: Types of sensor devices installed on sensor nodes. Currently, users can select a sensor board among those made by Crossbow Technology[5]. • Status: The status of a sensor site whether sensor data accumulated at the gateway are available for users or not. When an administrator constructs a sensor site only for experiments, he/she can set this status as ‘Not available.’ • Reservations: Whether the user or other users have reservations for experiments in the future.
List of sensor sites (Search results)
Search form
Fig. 6 A screenshot of the sensor site search interface
• Schedule: Whether the user can conduct an experiment in the specified period or not. When another user has already reserved an experiment at a site in the specified period, the site will not be shown in the search results. For example, when a user wants to conduct an experiment that uses a certain sensor board in a certain period, the user can find the appropriate sensor sites by specifying ‘Sensor board’ and ‘Schedule.’ On the other hand, by setting ‘Status’ as ‘Available,’ a user can find the sensor sites that provide sensor data. 3.2.2
Sensor Data Archive
As described in Section 1, researchers working on the areas related to sensor data need real sensor data acquired by nodes in order to evaluate the effectiveness of their methodologies. The ‘sensor data archive’ functionality provides such users with an interface for obtaining real sensor data accumulated at each sensor site. When no experiment is conducted at a sensor site, all sensor nodes in the sensor network periodically send data acquired by all their equipped sensor devices to the gateway. Users can obtain these sensor data accumulated at the gateway of each sensor site. In addition, users can obtain information on the network topology of the sensor network. Fig. 7 shows a screenshot of the sensor data archive interface. The right-hand side of this interface shows the network topology of the sensor network. Users can obtain the logical network topology which shows the communication routes from all sensor nodes to the sink node (see Fig. 8(a)) as well as the physical network topology which shows the actual locations of all nodes (see Fig. 8(b)). On the other hand, accumulated sensor data can be obtained by using the form in the left-hand side of the interface. In order to flexibly handle requirements from various kinds of users, this form enables users to specify the following items: • Data Format: Graph and CSV-formatted data are available as well as the raw data. • Sensor Nodes: By selecting nodes, users can obtain only the sensor data acquired by the selected nodes. • Data Type: Similar to ‘Sensor Nodes,’ users can specify the data types they want to obtain. • Amount of data: Users can specify the time range (for graph and CSV-formatted data) or the number of records (only for CSV-formatted data) they want to obtain. For example, when a user wants to obtain temperature data acquired by nodes, the user specifies temperature as ‘Data Format.’ On the other hand, when another user wants to obtain data acquired by some nodes in a certain period, the user specifies ‘Time Range’ and selects the target ‘Sensor Nodes.’ Fig. 9 shows an example of graph-formatted data.
258
A. Kanzaki et al.
Network topology
Specify nodes / data types / data formats
Fig. 7 A screenshot of the sensor data archive interface
3.2.3
Experimental Testbed
As described in Section 2, it is necessary that each node can operate according to the methodologies developed by researchers in order to conduct experiments. Similar to the conventional testbeds, X-Sensor adopts programmable nodes, and provides users with an interface to conduct their experiments, that we call ‘experimental testbed’ functionality. Fig. 10 shows a screenshot of the experimental testbed interface. Through this interface, a user can conduct an experiment on a sensor network according to the following procedure: 1. Upload programs: The user firstly uploads a zip-compressed file which includes nesC[11, 24] source codes and Makefile, which are required for making an executable. When a file is uploaded, the testbed server extracts the file, modifies the source codes, and makes the executable based on the modified source
codes and Makefile. Here, the modification of source codes is necessary for the experiment to be terminated at the end of the reserved period. Specifically, the testbed server adds a timer that terminates the operation on the executable and restarts periodical data acquisition (i.e., the operation for sensor data archive). When the above process has been successfully completed, the executable is registered on the testbed server. 2. Map programs: Next, the user selects the sink or sensor nodes to which the registered executable will be uploaded. Here, the user can register multiple executables for an experiment. In addition, each executable can be uploaded to a part of nodes in the network. This enables the user to conduct an experiment in which different algorithms or protocols simultaneously exist in a network. 3. Register schedule: Finally, the user specifies the time period during when the experiment will be conducted. If there is no reservation in the specified time period, the experiment is successfully reserved.
260
A. Kanzaki et al.
1. Upload programs
2. Map programs
3. Register schedule
Fig. 10 A screenshot of the experimental testbed interface
Fig. 11 shows the detailed procedure of an experiment after the beginning of reserved time period. First, the testbed server sends the executables and mapping information (specified by the user in step 2) to the gateway. The gateway uploads executables to sensor nodes according to the received mapping information. Here, for uploading executables, X-Sensor1.0 uses XOTAP[25], which is an executable uploading protocol for MICA series. After the upload, each node starts to operate according to the uploaded executable. On the other hand, the gateway starts XServe[33], which is a middleware for communicating with the sink node, in order to record the results (reports from the sink node) to a file. Finally, at the ending time of the experiment, each node terminates the operation in the experiment and restarts periodical data acquisition (this can be done due to the timer appended by the testbed server in step 1). On the other hand, the gateway sends the result of the experiment to the testbed server. After the experiment, a link to the result appears on the experimental testbed interface of the corresponding site. Here, only the user who registered the experiment can download the result.
Packets Operate according to the uploaded executables
Fig. 11 Detailed procedure of an experiment
Additional functions for experiments: In X-Sensor1.0, each site constructs a wireless multihop network. In other words, the sink node cannot directly communicate with all nodes in the network. Thus, it is difficult to obtain the detailed statuses of nodes and communications outside of the communication range of the sink node. However, it is important for researchers working on wireless sensor networks to obtain these statuses during an experiment in order to deeply analyze the characteristics of their methodologies. For example, when a researcher conducts an experiment of a routing protocol, he/she may want to obtain information on dynamic change in the internal status of each node, e.g., received control packets and routing information. Also, when another researcher conducts an experiment of a data aggregation method, he/she may want to obtain information on exchanged packets that the sink node cannot directly receive, e.g., aggregated data transmitted by each node. Moreover, there are several methodologies that have some constraints to work. For example, some data gathering methods assume that all nodes in the network simultaneously perform sensing operations. Also, some sleep scheduling mechanisms assume that time is synchronized among all nodes in the network. In such a case, it is necessary to synchronize the time among all nodes. Although there are a large number of time synchronization protocols in multihop wireless networks, it is very time-consuming for researchers to implement these protocols. To solve these typical problems for conducting experiments in a wireless multihop environment, several existing testbeds described in Section 2 establish the wireline connection between all nodes and the gateway (central server). However, when the number of nodes in a network increases, the cost for preparing such a wireline connection cannot be ignored. Especially, since X-Sensor aims to construct multiple sensor sites, it is desirable to suppress such a cost for constructing a new site. Therefore, we have developed two additional functions, intermediate result collection and node control during an experiment. In what follows, we present these functions in detail.
262
A. Kanzaki et al.
Coverage area of sniffer A Sniffer A overhears packets from nodes 1 and 2.
2 A 4 1
Wireline connection
Sink node
B
Sniffer C 6 3
5
Sensor node
Fig. 12 Intermediate result collection using sniffers
Intermediate result collection: In order to obtain the detailed statuses of nodes and communications during an experiment, we employ sniffer nodes, each of which always overhears wireless communications in its coverage area (the area from which the sniffer can correctly receive radio signals). Specifically, we deploy multiple sniffers in a network in order for all nodes in the network to be covered by at least one sniffer. In addition, we establish wireline connections between all sniffers and the sink node. For example in Fig. 12, sniffers A, B and C respectively overhear packets transmitted by nodes {1, 2}, {4, 5, 6} and {1, 3}, and send information on the received packets to the sink node via wireline connections. By doing so, the sink node can collect information on all packets transmitted in the network. Also, when a user prepares the source codes in order for all nodes to periodically send their internal statuses during the experiment, the sink node can collect information on the nodes’ statuses. Node control: In the intermediate result collection, sniffers are deployed in order to cover all nodes in the network. Thus, when the communication range of each sniffer is equal to (or larger than) its coverage area, all nodes can receive packets transmitted by at least one sniffer. We exploit this characteristics to control nodes during an experiment. Specifically, we use the sniffers deployed in the network as controllers. For example in Fig. 13, similar to Fig. 12, controllers A, B and C can control nodes in their coverage areas by broadcasting control packets. Using this controllers, we have developed a TDMA based packet transmission scheduling. In this function, the time is divided into time units, called time-slots. Each node is assigned a slot in the cyclic transmission schedule (called frame) and gets opportunities to transmit packets only in its assigned slot. By repeating the frame, each node can periodically transmit packets. In this function, a user inputs the slot assignment information (the duration of a slot, the number of slots in a frame, and slots assigned to each node) to the sink
Coverage area of controller A Controller A controls nodes 1 and 2.
2 A 4 1
Wireline connection
Sink node
B
C
Controller 6 3
5
Sensor node
Fig. 13 Controlling nodes using controllers
node. The sink node broadcasts the slot assignment information to all nodes via controllers. When receiving this information, each node assigns a slot to itself, and starts transmitting packets. Here, since the slot assignment information is broadcasted simultaneously among all controllers, the time can be synchronized among all nodes. A use case of the experimental testbed: As a use case of this functionality, we conducted an experiment on X-Sensor1.0. In the experiment, we measured the characteristics of radio propagations which are used by a route construction method which our research group has proposed in [26]. In what follows, we briefly present this method and show the experiment in detail. Our route construction method: The aim of this method is to construct an effective communication route in terms of both power-efficiency and quality of communication. For this aim, this method individually determines the transmission power of each node and a communication route from the node to the sink node based on the measured characteristics of radio propagation. Specifically, this method firstly measures the packet delivery ratios between all nodes. Here, the packet delivery ratio between a pair of nodes is defined as the ratio of the number of packets correctly received by the receiver to that of all those transmitted from the node. After that, the sink node collects information on the packet delivery ratios for all links (all pairs of nodes), and determines the transmission power of each node and the communication route from the node to itself by the following procedure: 1. The sink node sets the temporal transmission power for each link as the minimum power that achieves the maximum packet delivery ratio.
264
A. Kanzaki et al.
2. It constructs the communication routes from all nodes to itself by applying the Dijkstra algorithm[8] in which the temporal transmission powers are set as the costs in the graph. 3. It adjusts the transmission power of each node. Specifically, the sink node sets the transmission power of a node as the minimum power that achieves the required packet delivery ratio from all nodes, which is specified by the application. As described above, this method uses the characteristics of radio propagation measured between nodes in the network. Thus, the performance of this method needs to be evaluated in a real environment. For more details of our method, please see [26]. Experiment: The experiment to evaluate this method includes the following three steps: 1. Packet delivery ratio measurement: First, we measured the packet delivery ratios between nodes. In this step, each node broadcasts 100 probe packets for each transmission power. Here, a probe packet includes the information on the identifier of the transmitter and the transmission power when the packet is sent. The node that received the probe packets counts the number of correctly received packets, and calculates the packet delivery ratio between the transmitter and itself. After all nodes broadcast probe packets, each node sends information on the calculated packet delivery ratios to the sink node. The sink node sends the received information to XServe running on the gateway. 2. Communication and transmission power decision: Next, we decided the transmission power of each node and the communication route from the node to the sink node based on the information obtained in step 1. Here, we set the required packet delivery ratio from all nodes as 0.7. This means that at least 70 % of packets generated from each node must be correctly transferred to the sink node. 3. Performance evaluation: Finally, we evaluated the performance of this method. Specifically, each node periodically generates a data packet and sends it to the sink node according the communication route determined in step 2. The sink node counts the number of correctly received packets from each node and calculates the packet delivery ratio. The information on the calculated packet delivery ratio is sent to XServe running on the gateway. In these steps, it is necessary to conduct experiments in steps 1 and 3. In what follows, we present the details of what we did in step 1 using X-Sensor1.0. Fig. 14 shows the environment of the site where we conducted the experiment. As shown in this figure, the site deploys nine sensor nodes in a building. First, we prepared Makefile, nesC source codes, and header files used by the source codes. Here, the source codes describe the behaviors of nodes in step 1 (i.e., broadcasting probe packets, calculating packet delivery ratios, and sending results of calculation). After that, we registered the experiment by uploading the zip-compressed file that includes the above files.
After the experiment in step 1, we obtained the file shown in Fig. 15 as the result of the measurement. From this file, we extracted information on packet delivery ratios between nodes as shown in Table 1. By using the extracted information, we can derive the transmission power of each node and the communication route according to our method. Fig. 16 shows the communication routes and transmission powers determined by our method.
266
A. Kanzaki et al.
Table 1 A part of extracted information (packet delivery ratios between a pair of nodes in each transmission power) (a) from node 7 to node 5 Transmission power [dBm] -25 -15 -10 -7 -5 -3 -1 0 Packet delivery ratio 0.95 0.94 0.97 1.00 1.00 1.00 1.00 1.00 (b) from node 7 to node 10 Transmission power [dBm] -25 -15 -10 -7 -5 -3 -1 0 Packet delivery ratio 0.89 0.97 0.88 0.91 1.00 1.00 1.00 1.00
− 7[dBm] 4 6 − 7[dBm]
− 25[dBm] 3
− 25[dBm] 5
− 10[dBm] 8 2
− 25[dBm]
10 − 10[dBm]
Sink
− 7[dBm] 7
Fig. 16 Communication routes and transmission powers
4 Our Future Direction – X-Sensor2.0 In X-Sensor1.0, a single testbed server manages all gateways, and executes almost all processes. Thus, the testbed server may become a bottleneck when constructing large-scale sensor networks. Large-scale sensor networks are used for constructing dense sensor networks to get high accuracy, disposing various sensors to get various data, and sensing a wide area. Examples of the usages for large-scale sensor networks are as follows: • Comparison between sensor sites: When some university laboratories have sensor sites for sensing temperature and they are connected with each other, users submit a query to find the hottest or coolest laboratory. • Inquiry to all sites: To find laboratories in which at least one person exists, users submit a query to find sites of which illumination sensor data exceed a threshold. • Mathematical operation: Getting the average of temperature for all sites. In X-Sensor1.0, clients have to collect all data from all sites to process these queries. Unfortunately, this causes very long turnaround time in large-scale sensor networks.
To solve the problem, we are now developing the next version of our testbed, named X-Sensor2.0. X-Sensor2.0 exploits mobile agents. Mobile agents are computer programs traveling among gateways of different sites. They are generated by clients, travel to particular sensor sites, execute the program on each sensor site, and then return to the client. The returned agent has the result of the executed program. Since clients do not need to collect all data, they can get the result faster than that for the case of collecting all data from all sensor sites to the client.
4.1 Design Figure 17 shows the system image of X-Sensor2.0. Circled areas are sensor sites and there are several sensors and a gateway in each site. The sensor sites are connected with each other via a network. We call the network X-Sensor network. The client generates an agent and submits it to the X-Sensor network. A middleware, called mobile agent middleware, supports mobile agent generation, migration, control and so on. X-Sensor2.0 requires the following functions. For mobile agent middleware: The mobile agent middleware manages mobile agents in the X-Sensor network, which includes generation, migration, and control of mobile agents. Also, to move mobile agents, the mobile agent middleware must be able to find the gateway of each sensor site. For clients: Clients connect to the X-Sensor2.0 network and use the mobile agent middleware. Clients generate mobile agents using the mobile agent middleware and describe necessary processes for the mobile agent. X-Sensor Network
Site Gateway Site
Site Internet
Gateway
Gateway
Sensor
client
Mobile Agent
Fig. 17 System image of X-Sensor2.0.
268
A. Kanzaki et al.
For gateways: Each gateway collects sensor data from sensors in its sensor site and store them. Same as clients, gateways connect to the X-Sensor networks and use the mobile agent middleware. For mobile agents: Mobile agents get sensor data by executing the described processes on each sensor site. Therefore, they must be able to run on gateways. They are managed by the mobile agent middleware.
4.2 Implementation We are now implementing an X-Sensor2.0 system based on the design described above. In our implementing system, we use PIAX (P2P Interactive Agent eXtension)[27] as the mobile agent middleware. PIAX: PIAX is a mobile agent platform based on a P2P (Peer-to-Peer) architecture, in which clients communicate with each other directly without the support of a server. The server’s load under the PIAX is not a problem since it utilizes P2P techniques. In PIAX, PIAX peers run on each computer and mobile agents move among PIAX peers. Mobile agents are programmed by Java language and we can make mobile agents execute various processes. PIAX has functions to generate, move, and control mobile agents. Also, PIAX can find PIAX peers by specifying a geographical position such as a pair of the longitude and the latitude. Since PIAX is suitable for handling data in X-Sensor from above reasons, we exploit it as the mobile agent middleware. In X-Sensor2.0, each gateway runs as a PIAX peer. Mobile agent programming: To make mobile agents execute complex processes, we implement several functions and variables for mobile agents. Some of them are shown in Tables 2 and 3. Since the implemented processes are executed on each PIAX peer, that is the gateway of each sensor site, X-sensor2.0 can reduce the load of the server compared to X-Sensor1.0.
Table 2 Functions for mobile agents Name
Parameter Comment
SetCommand Command stores the executing command Command to the memory. Execute Command executes the stored command. When the parameter is given, the mobile agent executes the Command. SQL Query executes the SQL query Query. Print none displays the command result. Return none The mobile agent returns to the original PIAX peer. Travel PeerID moves the mobile agent to the PIAX peer of which ID is PeerID.
A character array storing the command result The number of lines stored in Result[] The result code of the executed command Peer IDs of PIAX peers that the mobile agent moves to The index of the current PIAX peer in PeerIds[]
5 Summary In this chapter, we present our sensor network testbed X-Sensor, that integrates multiple sensor networks. We also describe the design and implementation of XSensor2.0, which is the next version of our testbed. By using X-Sensor, users can not only conduct experiments that evaluate the effectiveness of their proposed methodologies but refers to a huge amount of real sensor data. Since the demands for real sensor data and experimental testbed from researchers are rapidly increasing, our testbed will become very important to accelerate their researches. Acknowledgements. This research is supported by the Ministry of Internal Affairs and Communications of Japan under the Research and Development Program of ‘Ubiquitous Service Platform’ and Grant-in-Aid for Scientific Research on Priority Areas (18049073) of the Ministry of Education, Culture, Sports, Science and Technology, Japan.
References 1. Aberer, K., Hauswirth, M., Salehi, A.: A middleware for fast and flexible sensor network deployment. In: Proc. Int’l Conf. on Very Large Data Bases (VLDB 2006), pp. 1199– 1202 (2006) 2. Aberer, K., Hauswirth, M., Salehi, A.: Infrastructure for data processing in large-scale interconnected sensor networks. In: Proc. Int’l Conf. on Mobile Data Management (MDM 2007), pp. 198–205 (2007) 3. Arora, A., Ertin, E., Ramnath, R., Nesterenko, M., Leal, W.: Kansei: a high-fidelity sensing testbed. IEEE Internet Computing, Special Issue on Large-Scale Sensor Networks 10(2), 35–47 (2006) 4. CitySense, http://citysense.net/ (Cited June 11, 2009) 5. Crossbow Technology: Wireless Sensor Networks, http://www.xbow.com/ (Cited June 8, 2009) 6. DES-Testbed | Making experimentation with testbeds a charm, http://www.des-testbed.net/ (Cited June 22, 2009) 7. DES-Vis | DES-Testbed, http://www.des-testbed.net/DES-Vis (Cited June 30, 2009) 8. Dijkstra, E.W.: A note on two problems in connection with graphs. Numerical Mathematics 1, 269–271 (1959)
270
A. Kanzaki et al.
9. Dimitriou, T., Kolokouris, J., Zarokostas, N.: SenseNeT: a wireless sensor network testbed. In: Proc. Int’l. Workshop on Modeling Analysis and Simulation of Wireless and Mobile Systems (MSWiM 2007), pp. 143–150 (2007) 10. Embedded Networks Laboratory, http://enl.usc.edu/projects/tutornet/ (Cited June 11, 2009) 11. Gay, D., Levis, P., Behren, R., Welsh, M., Brewer, E., Culler, D.: The nesC lauguage: a holistic approach to networked embedded systems. In: Proc. Conf. on Programming Language Design and Implementation (PLDI 2003), pp. 1–11 (2003) 12. Gibbons, P.B., Karp, B., Ke, Y., Nath, S., Seshan, S.: IrisNet: an architecture for a worldwide sensor web. IEEE Pervasive Computing 2(4), 22–33 (2003) 13. GSN, http://sourceforge.net/apps/trac/gsn/ (Cited June 30, 2009) 14. Gunes, M., Blywis, B., Schiller, J.: A hybrid testbed for long-term wireless sensor network studies. In: Proc. Int’l Workshop on Sensor Network Engineering, IWSNE 2008 (2008) 15. Handziski, V., Polastre, J., Hauer, J.-H., Sharp, C.: Flexible hardware abstraction of the TI MSP430 Microcontroller in TinyOS. In: Proc. Int’l Conf. on Embedded Networked Sensor Systems (SenSys 2004), pp. 277–278 (2004) 16. IrisNet: Internet-Scale Resource-Intensive Sensor Network Service, http://www.intel-iris.net/ (Cited June 30, 2009) 17. Kansei Sensor Testbed, http://ceti.cse.ohio-state.edu/kansei/ (Cited June 11, 2009) 18. Live E! Environmental information for a living earth, http://www.live-e.org/en/ (Cited June 30, 2009) 19. Matsuura, S., Ishizuka, H., Ochiai, H., Doi, S., Ishida, S., Nakayama, M., Esaki, H., Sunahara, H.: Live E! project: establishment of infrastructure sharing environmental information. In: Proc. Int’l Workshop on Practical Applications of Sensor Networking, p. 67 (2007) 20. Murty, R.N., Mainland, G., Rose, I., Chowdhury, A.R., Gosain, A., Bers, J., Welsh, M.: CitySense: an urban-scale wireless sensor network and testbed. In: Proc. Int’l Conf. on Technologies for Homeland Security (HST 2008), pp. 583–588 (2008) 21. MICA2 Datasheet.pdf, http://www.xbow.com/Products/Product_pdf_files/Wireless_ pdf/MICA2_Datasheet.pdf (Cited June 22, 2009) 22. MICAz Datasheet.pdf, http://www.xbow.com/Products/Product_pdf_files/Wireless_ pdf/MICAz_Datasheet.pdf (Cited June 8, 2009) 23. MoteLab - Harvard Network Sensor Testbed, http://motelab.eecs.harvard.edu/ (Cited March 15, 2009) 24. nesC: A Programming Language for Deeply Networked Systems, http://nescc.sourceforge.net/ (Cited June 30, 2009) 25. Network Management - XOTAP, http://www.xbow.com/Technology/NetworkManagement.aspx (Cited June 8, 2009) 26. Nose, Y., Kanzaki, A., Hara, T., Nishio, S.: Route construction based on measured characteristics of radio propagation in wireless sensor networks. In: Proc. Int’l Conf. on Advanced Information Networking and Applications (AINA 2009), pp. 560–567 (2009) 27. PIAX - Top Page, http://www.piax.org/en/ (Cited June 8, 2009) 28. PostgreSQL: The world’s most advanced open source database, http://www.postgresql.org/ (Cited June 20, 2009)
